The issue is the manager is stored in the operator id record and i'm unable to access the OperatorID record in the access when.
It is given in documentation:
Do not enter case attributes orpolicyclass property values in anAccessWhenrulethat is used for conditional logic because doing so causes invalid results or run-time failures.
So OperatorID record is not accessible in the access when rule. So either for a single user (current user/his manager) or a manager hierarchy (manager's manager) i have no way to decide in a access when rule.