Testing SAP S/4HANA Cloud Implementations

To accelerate testing activities, SAP S/4HANA Cloud comes with a test automation tool, detailed test scripts, and prepackaged standard automated tests (called test automates) for most business processes.

The test automation tool and test automates are ready to run as part of SAP S/4HANA Cloud and can be used without prior automation tool experience. The tool offers a simplified way to develop custom test automates to achieve automated testing for specific customizations of business processes. Considering the delivery pace of SAP S/4HANA Cloud, SAP extends support for automated testing by triggering tests in SAP S/4HANA Cloud customer quality tenants with each upgrade using the tool called the post upgrade test. Furthermore, while the test automation tool is native to SAP S/4HANA Cloud, you can also use this tool with SAP Cloud ALM.

In this blog post, you’ll learn how SAP enables automated testing for SAP S/4HANA Cloud implementations. We’ll explain the various testing activities and which tools SAP offers to manage these activities efficiently.

Testing SAP S/4HANA Cloud Implementations

Testing is the cornerstone of software product development at SAP and is performed throughout the development lifecycle of SAP S/4HANA Cloud, covering different levels of testing. Common tests include unit tests, functional integration tests, solution acceptance tests, E2E business process tests, and regression tests. The application also undergoes nonfunctional tests to confirm cloud principles and product standard guidelines. Even though tests are executed at every phase of development, progressive testing of the functionality is critical while the system is being configured to implement SAP S/4HANA Cloud. Moreover, you must validate that business processes in use are not impacted by the software changes delivered regularly by SAP.

The figure below shows phases of the SAP Activate methodology and priority areas for test activities. To accelerate automated testing for your specific implementation, SAP provides strong test automation capabilities built into SAP S/4HANA Cloud.

Let’s look at priority areas for test activities by SAP Activate phases, as follows:

• Explore phase: Test activities usually start with the realize phase, but you should start planning testing activities during the explore phase. During this phase, we recommend you explore the test automation tool and the standard test automates delivered with SAP S/4HANA Cloud.
• Realize phase: During this phase, you’ll need to perform appropriate tests for your specific SAP S/4HANA Cloud implementation. You can use standard test automates or create custom test automates to cover the testing of each business process in scope for the implementation project. We recommend you perform progressive testing of the functionality while the system is configured with the following two tests:
• Implementation test: Ensure business processes run as expected with your specific implementation, configurations, customizations, and business data (master data or organizational data).
• User acceptance tests: Check if the configured business processes are ready for productive usage. This test is needed after the initial implementation, after the configuration change, or after every scope change in an existing implementation.
• Run phase: In this phase, we recommend you perform automated regression testing for all planned test requirements and analyze test results. Also crucial is identifying custom test automates that require maintenance and new test requirements (if any) based on the plan for consuming new innovations or additional scope. All tests should be performed on quality tenants—specifically the regression test—and especially after every system change (upgrades or configuration changes), to ensure your existing business processes are still running.

Test Automation Tool for SAP S/4HANA Cloud

The test automation tool for SAP S/4HANA Cloud is an integral part of the solution. The tool has three SAP Fiori apps, which can be accessed via the SAP Fiori launchpad in the Automated Testing group, as shown in the next figure, if the role SAP_BR_ADMIN_TEST_AUTOMATION has been assigned to the user. In the following sections, after describing the available test automation tool apps, we’ll provide an overview of SAP’s test execution infrastructure for SAP S/4HANA Cloud customers. Furthermore, we’ll introduce the available standard test automates and the SAP offering for post-upgrade tests.

Let’s briefly look at each of these apps:

• Manage Your Test Processes: This app lets you define new custom test automates. Business users can use the recording functionality to create custom test automates without programming knowledge, additional configurations, or the setup of any additional infrastructure. Using this app, you can explore standard SAP-delivered, ready-to-run test automates, as well as create custom test automates by making a copy and modify them as needed for custom implementation.
• Test Data Container: This app is used to provide reusable test data that is managed centrally. SAP delivers test data containers as standard templates based on functional areas. (This includes all relevant fields in the UI applications that may be needed for automated test execution.) In addition, the app allows users to add custom fields that are not part of the templates. Once data is centrally maintained, and one or multiple data variants are created, it can be used in relevant test plans for execution. This app is integrated with all other apps of the test tool.
• Test Your Processes: A test user with an appropriate business role is a prerequisite for automated executions, and this app provides the option of adding one or more test users needed to perform automated testing. You can perform automated test executions (or schedule their execution) using this app. To trigger executions, you’ll need to create a test plan by adding standard or custom test automates and adding test data as a data variant. You can review the results of all automated test executions in detail with screenshots. You can download the execution summary or export execution details. Furthermore, this app is used for checking automated test executions performed as part of the post upgrade test.
• Analyze Automated Test Result: This app gives a summary of all automated test executions done using the tool. It also provides a drill-down capability so you can see details of execution and failures.

Key Capabilities Offered by the Test Automation Tool

Now let’s highlight the key capabilities of the test automation tool for SAP S/4HANA Cloud:

• Automation of multiple business flows: The test automation tool comes with a test automate authoring app with a recording feature for development of custom automates. In addition, the user can develop test automates for testing the public APIs in simple steps. Test automation for a UI app or APIs does not need programming skills, and automated testing can be achieved even by a business user. The tool enables automation of a UI app and public APIs to cover end-to-end business flow, thus helping to further expand the scope and coverage for E2E business process testing. Automates can be executed individually, scheduled, and even configured for execution with the post upgrade test.
• Simplified test data management: Empower users through simplification of managing test data and save preparation effort for automated test executions.
• Preprovisioned cloud infrastructure to host test executions: The test automation tool comes with preprovisioned cloud infrastructure; this enables users to trigger automated test executions without worrying about provisioning of any virtual machines or other test infrastructure. Tests can be triggered from any device accessing the test tool application, and executions are done in a cloud infrastructure hosted by SAP.
• Automatic failure analysis and recommendation for corrective actions: Users can save significant analysis effort with this unique offering, which autocategorizes failures due to missing configurations, data, and so on, with appropriate recommendations for corrective action.
• Maintenance safeguarding of custom automate: Automates standardization has been further enhanced to the most basic level of individual UI controls, which means there are fewer instances where users would need to make corrections in custom automates. Adaption to custom automates is needed in the case of change in a functional flow with application changes and implementation of new customizations. Overall, the user does not have to make frequent changes, thus helping to achieve lower TCO with the continuous automated test.
• Post-upgrade test: Users have an option of configuring the automated test to be executed immediately after the quality system update/upgrade. With appropriate consent maintained, SAP triggers the test and analyzes results for the standard automates executed as part of this test. Users also have an option to trigger this test on their own at any time. Users can also include custom automates to achieve a wider test scope and cover specific SAP S/4HANA Cloud implementations with the post upgrade test.

Cloud-Based Shared Test Execution Infrastructure

The test automation tool uses a shared test execution infrastructure hosted by SAP using SAP BTP. All automated executions triggered from different SAP S/4HANA Cloud quality tenants are executed on this shared test execution server, as shown in the following figure. Automated test executions are conducted in a headless browser on this shared test infrastructure, and test results are saved and published in the SAP S/4HANA Cloud software system where automated test execution was triggered.

Managed by SAP, this test execution server helps ensure the automatic scaling of computing resources needed to perform automated test executions. Using hyperscalers and virtual machines (VMs) in the cloud (conforming to data protection and privacy guidelines of SAP, including EU access-only guidelines), the server abstracts the multiple complexities involved with infrastructure, such as setup, regular maintenance, software updates, and scaling. Thus, you can perform preconfigured automated test executions in the cloud. 

Standard Test Automates

Standard test automates are based on test scripts documented in supported best-practice processes of SAP S/4HANA Cloud. Using the Manage Your Test Processes app, you can search these test automates by scope item ID or by the name of a business process, or you can filter by Type and select Standard to view all standard automates. Note that the test automates are visible in the system only when the corresponding business process (scope item) is active. You can copy these test automates as custom test automates and modify them to meet your business needs. Editing or deleting a standard test automate is not allowed; these objects are maintained by SAP and updated with every release upgrade. You must maintain your own custom test automates as required.

Standard automations are ready to run; you can execute them by adding them in a test plan and by providing data variants (adding master data or organizational data needed to execute) using the Test Your Processes app. SAP publishes comprehensive documentation for each standard test automate. Further details about the test tool are available in SAP Note 2129147.

Post Upgrade Tests

SAP offers post upgrade tests for business with live SAP S/4HANA Cloud implementations; testing is performed by SAP in your quality tenants, including the triggering of automated tests, the analysis of test results, and the initiation of necessary corrective measures if needed. With post upgrade tests, tests are performed based on active scope and use a predefined set of standard test automates. In order for you to perform post upgrade test executions, appropriate data (master data and organizational data) needed for executions are automatically picked from the system where tests are executed, test plans are created, and automated execution is triggered, as shown in this figure. Failures are automatically categorized with reasons for failure and failure help for faster resolution of nonapplication errors.

Performed immediately after the SAP S/4HANA Cloud quality tenants are upgraded with the new release, this test mainly identifies any obvious regressions with SAP S/4HANA Cloud software or business processes in your quality tenants and provides a high-level overview of upgrade impact (if any). Automated test execution results performed as part of post upgrade tests are available for review (or re-execution) in the Test Your Processes app. You also have the option of making a copy of post upgrade test plans for enhancement with additional test requirements.