Pega Sales Automation VSTO addin doesn't allow you to connect if destination system is not configured with valid ssl certificate. You don't have any configuration option to bypass this. Why can't you use ip:port instead of domain name or valid ssl certificate with domain name?
As per your comment, system is accessible only with in client private network. So you should connect to network via VPN or pc should be in same client network. Is it correct understanding? Now the question would be, are you trying to launch the outlook addin in the same network (may be using VPN)?
If you are trying to access the outlook add in with in the same network, then you should not face any issue provided if it is not trusted site.
If I see attached outlook screenshot, Why it is showing outlook "working as offline"? It is not connected to network. When you connect to network, what is the side panel error you are getting?