Posted: 27 Jan 2021 7:16 EST Last activity: 12 Mar 2021 0:22 EST
Limitations of scenario testing
Types of scenario tests:
Scenario test cases can be created either for a case type or a portal. It is not possible to record a test case from a certain point of a case type flow. One need to record a test case from the starting of the case type flow to whatever step/stage required. For other scenarios, portal level test cases can be used.
Persona based testing:
Scenario testing framework runs within the platform and can be run only in requestor context i.e. a login to Pega system is needed. Due to this limitation, a single test case can’t offer recording scenarios involving multiple personas or multiple login/logout actions. This is in the roadmap and will be addressed in future.
Setting up and cleaning up test data:
Currently there is no support to setup/clean-up test data for a test case. However, it is possible to supply dynamic data to test case using the data page. Please refer to the section “providing dynamic data to the test case” in following article for more details on how to use it.
The options to setup/clean-up test data for a test case using various in-built actions will be made available very soon, using which you can create work objects, run activities etc. to setup the required data and refer the data dynamically from the test case.
Data driven (Parametrization) approach:
Data is tightly coupled with test cases. It is not possible to run the same test with different input/output datasets. Each test is associated with a single input/output data set. Multiple test cases need to be created to test a scenario with different input/output combinations. This is in the roadmap and will be addressed in future.
Scheduling of tests:
There is no out of the box support to schedule the scenario test cases or the test suites created for an application stack. You may have to leverage open source tools like Jenkins to schedule tests to run in the pipeline on a daily basis.
Running of tests from CI pipeline
To run scenario tests from the pipeline, customers should have accounts with third-party platforms like Cross Browser testing or an in-house selenium grid which can take care of launching the Pega instance and logging the user into the portal. You may refer to following articles to understand how to integrate the tests with CI tools.
A scenario test cannot be included in another scenario test, which is generally the case with other test frameworks that allow atomic level test cases to be included in the main test case
Execution of scenario tests
Scenario tests are portal dependent. i.e., once recorded on a portal, they cannot be manually run from a different portal. They have to run in the portal on which they are recorded. If you are executing the tests from developer studio or through a pipeline, it is possible to execute the test cases from built-on applications across different portals from a top layer application, provided correct application quality settings are configured to include all the required applications.
Support to various UI elements & scenarios for recording:
Though majority of the out of the box UI controls are supported by this framework, there are few controls and known scenarios which are not supported. Please refer following information to understand what is supported and what is not.
Controls not supported
Repeated Dynamic Layout
Reordering, show details
Touch and swipe
Dynamic Layout Group
Adding new tab, Header text, Touch and swipe
Filtering, sorting, reordering, header text
Filtering, sorting, reordering, resizing, node opening, header text, editing on “click row to edit” mode
Filtering, sorting, reordering, resizing, node opening, header text
Layouts not supported
Some known scenarios that are not supported
Popup windows are NOT supported to be recorded or asserted. Currently the recording scope stays with elements of the main window.
Click is needed from mouse or touch pad to record a DOM element and make assertions. Hovering over will NOT be recorded by scenario testing framework as they is no click involved.
Scenario testing framework does NOT support saving or parsing files (like pdf, xlsx) as part of the test case. Hence, file operations like assertions on processed file content, assertions with file contents cannot be done.
Date values entered in date fields can NOT be referred from system to use the presentdate. It will select the same value used while recording over and over.
Modal dialog boxes which is usually used for confirmation to proceed or cancellation flow are NOT supported to be recorded or asserted.
Whole tab refresh while recording a test case might cause abrupt FAILURE during test case execution.
Dropdown control that gets populated with dynamic values/options can NOT be recorded as value to be selected has to be included in test case at the time of recording
***Edited by Moderator Marissa to add the Developer Knowledge Share tag***
Thanks for this information. Can you please let us know the assertion features in details(comprising the role of drop down values and how will it impact the test step) like how can we leverage this. any realtime/hypothetical example will be suffice. Also consider below points.
a) is it pop window or modal window which is not supported
b) in Pega we will have case id generated for each instance whose format could be prefix-Incremental value (i.e. S-1,S-2.....) in our application the format is S-W-1,S-W-2...or S-N-1,S-N-2... how can we configure this so that while execution it should pick dynamic case id which will always have incremental value.
c) we have an option of selecting multiple test suites in Dev Studio for execution, how will it execute once launched from portal or we have to run one test suite at times
d) There is no search button or filter criteria on portal against recorded test cases, imagine if I want to change certain fields/add step, rerecord in a test case on top of which I have created 1000 test cases, I have to click on load more,more... to reach there. this is cumbersome
e) Can user A and B at same time can run the test suite S-1 from their individual login(Portal is same). what will happen in that case?
Thanks for adding time delay settings on individual step
Posted: 1 month ago
Updated: 1 month ago
Posted: 12 Mar 2021 0:12 EST Updated: 12 Mar 2021 0:22 EST
For dropdowns, Scenario test simply captures the entered value and during run time same value is applied. if you want to verify list of values present in dropdown, better to write pegaunits for source of the dropdown. For suppose, if your dropdown source is Data page, better to write pegaunit for data page and assert the list of values returned in data page.
Coming to the assertions feature, As of now, when you record a step for drop down for entering values. scenario test simply takes the DOM properties of dropdown and keep them in assertions. like for a dropdown it simply captures value, tooltip, required, hasErrors etc.. And these assertions will be executed before execution of the step and see if there are any assertion failures and go for actions to perform on dropdown. As of now, we have only one comparator only i.e "Is Eauals" we are planning to extend these comparators in future. Below is the screen shot to explain assertions captured for dropdown.
a) Its Popup window. its like another browser window comes as pop on click of link.
For example, shown in below image. when you click "Advanced" from attachments in case type. popup window comes
b) It will be addressed in release two of scenario testing. from release two onwards, you can setup actions for scenario test cases. ex: Creating a work object or execution of activity before the scenario test execution and you can refer those dynamic values as part of test case steps by accessing dynamic as step data. the release two of scenario test would be some where around early month of May.
c) I did not get your question, Can you please elaborate your use case. but as per my understanding. you can execute test suite from Dev portal and rest service. But it can not be executed from any other portal. But from Dev portal if you select two test suites, it will try to execute is sequence manner, one suite after the other suite.
d) We shall try to address it this in release 2. Alternatively you can open test case from landing page in Dev portal( Configure-> Application->Quality-> Automation testing-> Scenario testing and filter the test case from landing page and open the test case and edit it.
e) Yes, User A and B can execute same test case from different logins, basically when user A and B logins with different logins they are different requestors the will not have any impact of other user execution. In general, its like two different users login and doing their actions.