Have you denied permission for all users ?
Same is working in my end, However here what I have did.
Project : RaviTestDelete
Project level permission :
I have given permission to test users Mahesh ( Note : I have not created groups but it will work same for users as well as groups )
D1 Workbbok Permission :
Note : for All Users make permission denied.
D2 Workbook Permission :
D3 Workbook Permission :
Hope it help..
Let me know if you still face any difficulties...