I am in similar problem with modal window.
I am trying the same in 7.3.1. Once modal dialog is launched,tried Back button with following options. Instead of replacing on same window with previous flow action , it is opening in parent window from where the modal dialog is launched. Modal dialog is still displaying the current flow action.
i am using process flow and i tried the same in screen flow as well.
Option 1: pzActionButton with Parameter Onclick as "pega.u.d.handleActionEvent('GoBack',null,event)"
Option 2: Button with action as script
Option 3: Action Perform flow action with previous flow action name and by creating the connector back to previous assignment in flow.
Is there another way to achieve the navigation within modal dialog window without affecting the parent window?