Continuous Integration with ABAP and SAPUI5?

Introduction to Continuous Integration

Having up-to-date software and technology is critical for businesses that want consistent and continuous growth. The right software can help businesses expedite the efficiency of their processes. In particular, with Continuous Integration (CI), automation is possible and can increase the effectiveness of processes in software development. This includes catching bugs faster before going live. Additionally, Continuous Deployment (CD) can ensure these integrations are consistently deployed over time so that the newest version of a process is always implemented.

In this article, we will discuss how to apply these specific concepts within a SAP environment, as well as how they translate in SAP terminology. CI offers automatic testing with AUnit and automatic checks with ATC/SCI. On the other hand, CD offers automatic transport and automatic integration tests with data cleansing, among other tasks. By integrating these developments with the assistance of a SAP Business Technology Platform, businesses can have improved potential to act on-premise and apply cloud development for more efficient and effective processes.

In this article, we will guide you through the process of building a continuous integration pipeline that runs ABAP Unit tests in a set package, as well as an SCI check variant on these objects. Afterward, the outcome will be displayed in a report available on the CI / CD server.

Tools

In this article, we will focus on tools that can be implemented into a Continuous Integration pipeline in an ABAP and SAPUI5 development environment. The following section will outline some of the tools that we will use later.

A main focus for these tools is using them externally from of a classical SAP system environment and rather through the command-line, which allows us to develop scripts that automate our processes.

TeamCity

TeamCity by JetBrains is a general-purpose CI / CD platform. Similar to other build runners, such as Jenkins, TeamCity allows us to run “Build Configurations”, which include any number of “Build Steps”. This can be either a pre-defined action like executing an npm script or even a custom shell script.

We will use TeamCity as our main entry point and management platform for the automated pipelines that will be outlined in this article.

SAP CLI

SAP CLI is command line interface to SAP products. It is an open-source tool written in Python that interfaces with the HTTP APIs. which are also used by the ABAP Development Tools (ADT), so it can be used with any modern SAP system. With SAP CLI, we can run ABAP Unit Tests, SCI checks and even execute code directly on the SAP system by calling function modules or special classes.

Docker

Docker is a tool that allows applications to be executed in lightweight containers. It’s comparable, but not equal to, classic virtualization. In a CI / CD environment, Docker simplifies the process of getting the required tools up and running.

INFO ℹ
Luckily, the SAP CLI repository already provides a Dockerfile, which we can later use to run SAP CLI without having to deal with its dependencies.

Getting started with Continuous Integration

Before implementing the pipeline, we first need to prepare. As mentioned earlier, we will be using #SAP CLI to interact with the SAP system. We will also use #Docker and run all tasks in a container environment. As the time this article was written, SAP CLI does not provide a publicly available Docker image on DockerHub, so the image will need to be built ourselves.

Assuming that both Git and a Docker environment are installed on our local machine, like Docker Desktop on Windows, we can simply run some commands in a terminal to clone the repository and build the image:

git clone https://github.com/jfilak/sapcli.git
cd sapcli
docker build -t sapcli .

After running these commands, the docker image for SAP CLI has been created and tagged with the sapcli label. This allows us to easily run SAP CLI by referencing this tag in docker run. To verify that everything worked correctly, we can ask SAP CLI for its current version:

docker run --rm sapcli

If everything worked, we should get an output explaining the usage of SAP CLI like this:

usage: sapcli [-h] [-v] [--ashost ASHOST] [--sysnr SYSNR] [--client CLIENT]
              [--no-ssl] [--skip-ssl-validation] [--port PORT] [--user USER]
              [--password PASSWORD] [--mshost MSHOST] [--msserv MSSERV]
              [--sysid SYSID] [--group GROUP] [--snc_qop SNC_QOP]
              [--snc_myname SNC_MYNAME] [--snc_partnername SNC_PARTNERNAME]
              [--snc_lib SNC_LIB]
              {program,include,interface,class,ddl,functiongroup,functionmodule,aunit,atc,datapreview,package,cts,checkout,activation,adt,abapgit,rap,table,gcts,startrfc,strust,user,bsp,flp}
              ...

Now we need to tell SAP CLI how to access our SAP machine. To do this, we have two options:

1. Provide all the required connection parameters like hostname, SID, etc. directly to the command line.
2. Pass all these parameters through system environment variables.

To simplify this process, we will be choosing the latter option here as it saves us from passing the parameters with each command. We will also create a simple text file that contains all the required parameters and then simply load this file in the docker run command.

The next step is to create a text file, for example, a4h.sapcli.env, containing the following text (change the variables according to your SAP system):

SAP_ASHOST=a4h.local
SAP_CLIENT=100
SAP_USER=DEVELOPER
SAP_PASSWORD=Secr3t!
SAP_PORT=8000
SAP_SSL=no

We can now verify that our setup works correctly by running the following command, which will read the contents of the table SFLIGHT and print it to the terminal. This command is the command-line version of using the SQL console in the ABAP Development Tools:

docker run --rm --env-file a4h.sapcli.env sapcli datapreview osql 'SELECT * FROM sflight'

TIP 💡
If you encounter any errors, make sure that the credentials in the a4h.sapcli.env file are correct and the user that you have specified in the SAP_USER variable has the necessary authorizations to use the ABAP Development Tools. You can use the transaction SU53 to check if there are any authorization errors.

For the sake of simplicity, we can introduce an alias for our command which replaces the whole docker run command with a simple sapcli command:

alias sapcli='docker run --rm --env-file a4h.sapcli.env sapcli'

TIP 💡
On Windows, the syntax for creating an alias is slightly different:

function RunSapcli { docker run --rm --env-file a4h.sapcli.env sapcli @args }
Set-Alias -Name sapcli -Value RunSapcli

With our newly created alias, we can simplify the command to read data from the table SFLIGHT like this:

sapcli datapreview osql 'SELECT * FROM sflight'

ATTENTION ⚠
In the rest of this article, we assume that this alias has been maintained properly. If a command is run somewhere where the alias is not maintained, the sapcli command has to be replaced with docker run --env-file [FILE] --rm sapcli

Interacting with SAP through the command-line

In the following chapter, we will learn how we can use SAPCLI to run the relevant commands needed in order to build our pipeline. In the introduction, we identified the following scenarios:

• Run ABAP Unit Tests
• Run SCI Checks

Running ABAP Unit Tests

With SAP CLI, running ABAP Unit Tests is easy. To run all the unit tests in the package $TMP we can simply run:

sapcli aunit run package '$TMP'

This outputs a human-readable representation of all the unit test results in the package $TMP and its sub-packages, similar to selecting the package in the ABAP Development Tools and then pressing Shift+F10.

TIP 💡
As the dollar sign $ is a special character in most terminal environments, it must be handled accordingly. In Windows Powershell and Shell environments, the quote signs ' prevent the expansion of the $TMP variable.

Running SCI Checks

Similar to ABAP Unit Tests, SCI / ATC checks can be executed against a set of objects present in a package using one simple command:

sapcli atc run package '$TMP'

TIP 💡
In most SAPCLI commands, package can be replaced with, for example, class in order to run the checks only against a single class.

This will run the default ATC check variant against all objects in the package $TMP, similar to right-clicking the package in the ABAP Development Tools and selecting Run As > ABAP Test Cockpit. The output will be printed in a human-readable format.

If desired, a check variant can be explicitly specified using the --variant argument (or the short form -r):

sapcli atc run --variant Z_MY_VARIANT package '$TMP'

Configuring TeamCity to run Continuous Integration for SAP

Based what we have learned from the from the previous chapter, we can now set up a new Build Configuration in TeamCity. We will automate all the steps we just performed manually in a set of Build Steps. In the end, this will show us the output of the Unit Tests and the SCI / ATC checks in our TeamCity dashboard. This means that we will create steps for the following tasks:

• Build the SAPCLI docker image
• Run the Unit Tests
• Run the SCI / Checks

We will start by creating a new Build Configuration in TeamCity. The steps to create one might be slightly different based on whether your TeamCity installation uses the new experimental UI or the classic UI.

INFO ℹ
Make sure that you have at least one Build Agent available that supports the “Docker” build runner. For more information, refer to the official documentation.

Building the SAPCLI Docker image

TIP 💡
This step is only necessary right now since, at the time this article was written, no official Docker image for SAPCLI can be found on DockerHub. If an official image is published in the future, the following steps can be skipped and the image name can be replaced with the official image.

In the new Build Configuration, we start by creating a new Build Step that builds the SAPCLI image. This is so that we can use it in the following steps. Choose the “Docker” type for the new step and set it up as follows:

Parameter	Value
Docker command	build
Dockerfile source	URL
URL to file	https://github.com/jfilak/sapcli.git
Image name:tag	images.local/sapcli

Using the source type “URL” for the Dockerfile allows us to configure the build runner to first check out the official Git repository for SAPCLI and then immediately build the Docker image for it.

INFO ℹ
Prefixing the image name with images.local/ is not necessary, however, it may prevent issues with locally built images overriding official images on DockerHub. It also prevents accidental push attempts to the official registry.

Setting up the environment variables in TeamCity

The next step is to set up the configuration parameters for SAPCLI as environment variables. These variables will be automatically visible to the docker run commands that we will use in the following steps so we do not need to create any files. In the Build Configuration, navigate to the Parmaters section and set up the following parameters, similar to what we used earlier (notice that prefixing the parameter name with env. will automatically choose “Environment Variable” as the parameter “Kind”):

Parameter	Value
env.SAP_ASHOST	a4h.local
env.SAP_CLIENT	100
env.SAP_USER	DEVELOPER
env.SAP_PASSWORD	Secr3t!
env.SAP_PORT	8000
env.SAP_SSL	no

TIP 💡
When configuring the env.SAP_PASSWORD parameter, make sure to configure it as a password so that TeamCity will hide its value from users. To do so, click the “Edit…” button to edit the Parameter Spec and choose the Type “Password” at the bottom of the dialog. You will notice that the value of the parameter changes to ****** in the overview table.

Configuring ABAP Unit Tests in TeamCity

Next, create a new Build Step of the type “Command Line”, which we will use to run the ABAP Unit Tests. In the “Custom script” text box, enter the following script:

sapcli aunit run package '$TMP' --output junit4 |\
    sed -E "s/status=\"OK\"/status=\"SUCCESS\"/" |\
    sed -E "s/status=\"ERR\"/status=\"FAILED\"/" |\
    sed -E "s/status=\"SKIP\"/status=\"ABORTED\"/" \
    > aunit.xml

There is a lot going on in this script, so let’s break it down step-by-step:

We start with the already known sapcli aunit run package '$TMP' command, which will run all unit tests in the $TMP package. This is followed by the addition --output junit4, which will tell SAPCLI that the result of the Unit Tests should be printed as a JUnit4-compatible format instead of the human-readable format that is the default. Since TeamCity supports the JUnit report format out-of-the-box, we are saved from writing our own parsing logic.

However, as the JUnit4-format reported by SAPCLI uses some status codes that TeamCity does not understand, we have to use the sed command in order to replace these status codes with ones that are compatible with TeamCity´s. This can be done by simply piping the standard output of SAPCLI to the sed command multiple times and doing a regex replacement. Finally, we simply use the redirection operator > to redirect the output to a file called aunit.xml.

Now, in the field “Run step within Docker container”, input the name and tag of the image we built in the previous step: images.local/sapcli. Here, we make use of the fact that the SAPCLI Dockerfile adds the sapcli command to its PATH environment variable, which allows us to pass the script to the Docker container and run it as if we had sapcli globally installed. (Make sure that “Pull image explicitly” is disabled so that TeamCity will use our locally built image)

Finally, we need to enable the XML Report Processing feature for the Build Configuration. This will ensure that TeamCity picks up the report file that contains the Unit Test results. To do so, navigate to the “Build Features” section of the Build Configuration and click “Add build feature”. In the dialog, choose “XML report processing” and select the Report type “Ant JUnit”. In the “Monitoring rules” field, add the name of the XML file: +:aunit.xml

Setting up ATC / SCI checks in TeamCity

The process to setting up ATC / SCI checks is very similar to the way we set up ABAP Unit tests. Start by creating a new Build Step with the type “Command Line” and enter the following into the “Custom Script” field:

sapcli atc run package '$TMP' --output checkstyle --error-level 0 > sci.xml

As we already know, this will run the default ATC check variant for all objects in the $TMP package. However, this time, we use two additions to the command:

• --output checkstyle will output the report in a Checkstyle-compatible format. Similar to the JUnit format used for the ABAP Unit Tests, TeamCity supports this one out-of-the-box
• --error-level 0 tells SAPCLI to not treat any findings as an error. SAPCLI will still output the severity for each finding. However, it will always exit with the exit code 0. This is important because TeamCity will, in the default configuration, treat any non-zero exit code as a build error and stop the build at the current step. The default value of this parameter is 2, which means that any SCI findings with a priority of 2 or lower will cause SAPCLI to exit with any non-zero exit codes

TIP 💡
At this point, we can also configure a custom check variant, just like we did before by adding the --variant argument.

Again, configure the Build Step to be executed within our SAPCLI image by setting the value of “Run step within Docker container” to images.local/sapcli.

Finally, the output of the command is redirected into a file called sci.xml. In order for TeamCity to analyze this file, we need to set up an XML report processing the Build Feature again. Repeat the steps from earlier and configure the feature for the “Checkstyle” report type and input the monitoring rule +:sci.xml.

Setting up Failure Conditions for the Build Configuration

After we have set up all Build Steps, we finally can tell TeamCity at which conditions the Build should be treated as “Failed”. To do so, we navigate to the Failure Conditions section of our Build Configuration and make sure that the option “At least one test failed” is enabled. This will have our Build fail whenever one of the ABAP Unit Tests fails.

Furthermore, we can set up a Build Failure for the results of the ATC / SCI checks. To do this, click the “Add failure condition” button and select “Fail on metric change”. Here, configure the build to fail if its “number of inspection errors” compared to “constant value” is ”more” than ”1 default metric units”:

TIP 💡
You can also use this feature to configure a more detailed failure condition for the ABAP Unit Test results or even have TeamCity use a specific build as a baseline for the values. Check out the official documentation for more details on what can be configured here.

Testing the Build Configuration

Now, with the Build Configuration ready, we can test it. This can be done by pressing the “Run” button next to the Build Configuration in the overview screen.

It may take several minutes to complete depending on the amount of objects affected by our configuration. You can follow the test´s progress by clicking on the active Build Run and observing the Build Log. When the Build Run has ended, you will be able to see results of both the ABAP Unit Tests and the ATC / SCI checks:

In our example, the Build has failed because it found both failing ABAP Unit Tests as well as errors in the ATC / SCI check. You can access detailed information on the findings by clicking the links found in the output. The reports can be accessed any time by switching to the “Tests” or “Code Inspection” tabs in the Build Result:

Possibilites to extend the Build Configuration in TeamCity

Following this guide, we have set up a simple Build Configuration in TeamCity that will allow us to monitor the quality of objects in a certain package ($TMP in our example case). However, this is only the first step toward a fully-fledged CI / CD pipeline. There are many possibilities to extend upon what we had just set up:

• Setting up an automated schedule for the Build Runs
• Integrating with more processes surrounding modern SAP development like automating upload of SAPUI5 repositories
• Running custom tasks on the SAP system
• Automating web-front-end tests, tightly integrated with the corresponding ABAP back-end logic

Furthermore, the implementation of a CI / CD process outside of the SAP ABAP Application Server allows us to further integrate with other related development processes that are not strictly tied to the application server. This could include a companion application written in another programming language, like Java or Python.

Why we use Continuous Integration and why you should too

Now that we completed the example CI / CD pipeline, non-SAP and open-source tools can be implemented into digital processes. For future processes, pipelines can be extended to perform tasks, such as integration tests on special machines, or even with client copy for resetting specific machines.

While we develop using on-premise SAP machines, gicom always strives to modernize our internal processes and adapt them to current standards. As a pioneer in the digital space, we encourage experimentation with CI / CD as sometimes risks are worth the benefit. gicom continues to invest in these innovations in order to improve our software every day and make our processes more efficient.