First, the Mashup Security section on the Integration and Security tab of a application record has nothing to do with Authentication. That section is where you defined the top level application base URL that then allows us to communicate with that application via window.postMessage.
Prior to 7.2 we required a Mashup to have Same Domain Origin policy satisfied for interframe communication for the pega mashup events and standard resizing events. We have changed that to use window.postMessage and you no long have to have Same Domain Origin Policy satisfied and we also don't even ship PRGateway anymore as that was it's primary use. (PRGateway is also no longer being worked on)
Think of the Mashup Security section of the application record as you providing PRPC security access to talk to the application that is embedding us.
As for Authentication no changes in 8.2 for Mashup. The authentication is dictated by what Servlet you're using in the "data-pega-url'. When you generate the mashup code it might be defaulting to use the /IAC Servlet. You can change this. We have customers using SAML authentication for Mashup. The IAC Servlet is pointing to the IACAuthentication AuthService that uses the IACAuthentication authentication activity, we fully expect that you will modify this to meet your needs. This is just standard PRCustom authentication.
As always with Mashup you should be thinking about authentication but also thinking about how the top level application is also doing authentication. If the top level application requires authentication then users should not be able to access the Mashup content with out first being authenticated in the top level application as we are embedded in that application, we are part of it. If that is the case then authentication with PRPC is pretty straight forward.
Many thanks for your answer. I've one more doubt, earlier I think there was option to put user credential in the script itself/ in the prGateWay as well. If prgateWay is no longer needed, then how to pass this credential.
My requirement is user credential is to send from mash up script and to authenticate from pega side.
PRGateway is/was just a proxy server. It didn't handle any authentication with PRPC.
The data you need to send for authentication depends on the PRPC Servlet you use. For example in my Lab environment for testing I have setup my IACAUthentication activity to just log me into the system with the user id specified in the "UserIdentifier" parameter. In my Mashup i provide the value like this: