Try this setting in the screenshot I attached.

If you use this option it should respect the max limit you have set. But underneath the hood it is treating the input field more like a text input. 

However, what you will lose is the restriction from entering non digits in the first place. But it should still maintain validations as you lose focus/submit. So it should not be that much of an issue.

Test it out and let me know if this helps.