Pega provides few options for authentication lockout mechanism via Security Policies and this is only for PRPC Operators using PRBasic default authentication.
Platform don't have any DSS settings for configuring lockout time interval when max attempts is reached. While using PRCustom authentication like LDAP, you may have to customize your LDAPAuthentications activities for invalid credentials to serve your purpose and I believe the external LDAP Server can also be configured with lock-out mechanism in case of failure attempts.
With custom authentication, the default max no of failure attempts is 3 and later that server will throw a 500 internal error. The DSS Authentication/maxLoginAttempts is useful to increase the count like how many times the user is allowed to retry before the server throws an internal 500 error.