On my 7.4 system, I see it is based on a pyEnableAgileWorkbench when condition so you can check the when rule for the conditions. I see there are a few privileges checked so you can review those and add/remove them to the desired access roles.
You can set access as desired for these privileges:
The manager role for my test application had pxAgileWorkbenchItemModify and pxAgileWorkbenchItemReadOnly specified for the classes below. I set the access to 0 and the Agile Workbench bar no longer appeared.