Integrating SAP S/4HANA with Kafka via SAP Advanced Event Mesh: Part1 – Outbound connection

Introduction

In today's fast-paced business world, the ability to seamlessly communicate and exchange data between different systems is crucial. SAP Advanced Event Mesh (AEM) offers a robust solution for real-time event-driven communication across various SAP systems and external services. In this blog post, we'll explore how to integrate S/4HANA with Kafka using SAP AEM for data streaming and event-driven architecture.

Step-by-Step Guide

Let's break down the connection process between S/4 HANA and Kafka using SAP AEM into 6 sections, each explaining a key part of the connection setup to help you easily understand and implement the process.

1. Login and Setup SAP AEM Service

◉ First, log in to your BTP subaccount and create a subscription for AEM ensuring your user has the required roles. Once subscribed, log in to your SAP AEM tenant and navigate to Cluster Manager to create an Event Broker service. This service enables applications to publish or consume events. Below is the start page of SAP AEM after logging in.

◉ Create an Event Broker Service by clicking on ‘Create Service’.

◉ Provide a meaningful name for the service e.g. – ‘AEM_SERVICE_DEV’, select the service type, and choose the region. Click on "Create Service".

◉ After the service is activated, you'll see the page.

◉ Navigate to "Manage" and then "Authentication". Enable Client Certificate Authentication.

2. Establishing Trust between S/4 HANA and AEM

◉ To implement client certificate-based authentication, you need to establish trust between S/4 HANA and the AEM service broker. In your S/4 HANA system, execute the STRUST transaction to open the Trust Manager. Export the certificates from SSL client (Standard) and upload them into AEM in the next step.

◉ Go to "Manage" and then "Certificate Authorities". Upload the exported certificates by clicking on "Add Client Certificate Authority".

◉ Once done, all the certificates will be displayed as shown below.

◉ Now, import the certificate chain of the SAP AEM service broker host and BTP-IS Subaccount host in the SSL client (Standard) in the STRUST transaction code.

3. Broker Manager Configuration in AEM

◉ Click on "Open Broker Manager" and log in using the "Management Editor Username" and "Management Editor Password". You can find these access details under the "Status" section of the broker service.

◉ Once logged into Broker Manager, create a Queue which will serve as a storage mechanism for messages received by SAP AEM. When S/4HANA will generate any events or messages, they will be placed in the queue before being processed and forwarded to Kafka.

◉ Provide a meaningful name for the Queue e.g. – ‘AEM_DEV’.

◉ Assign a Subscription to the Queue. By creating a subscription, we ensure that our SAP AEM instance is subscribed to the relevant topics or events generated by S/4 HANA.

◉ Go to "Access Control" and create a Client Username with the hostname from the leaf certificate maintained in SSL Client (Standard) in the STRUST.

4. Configure AEM to Kafka connection through Kafka Sender Bridge

◉ The Kafka Sender Bridge is required to facilitate communication between AEM and the target Kafka cluster by converting AEM messages into Kafka events and propagating them to the remote Kafka cluster.

◉ To establish client certificate authentication between AEM and the Kafka cluster, you'll need .jks files of the Keystore and Truststore from your target Kafka broker.

◉ Open the command prompt and use the command 'keytool' to convert the .jks files into .p12 files. Here's how:

keytool -importkeystore -srckeystore C:\OpenSSL\<keystorefilename>.jks -destkeystore C:\OpenSSL\keystore.p12 -srcstoretype jks -deststoretype pkcs12

keytool -importkeystore -srckeystore C:\OpenSSL\<truststorefilename>.jks -destkeystore C:\OpenSSL\truststore.p12 -srcstoretype jks -deststoretype pkcs12

◉ Once converted, copy these .p12 files to the OpenSSL -> Bin folder.

◉ Now, navigate to the 'OpenSSL' directory and convert these .p12 files to .pem files using the commands below:

openssl pkcs12 -in keystore.p12 -out keystore.pem

openssl pkcs12 -in truststore.p12 -out truststore.pem

◉ You'll need to set a passphrase during this process. Note: Remember this passphrase, as you'll need it for client certificate authentication.

◉ From the ‘truststore.pem’ file, copy the content of root and the leaf certificates and save it as .cer files. Add those in our service broker under "Manage" -> "Certificate Authorities" -> "Domain Certificate Authorities".

◉ Now, navigate inside Broker Manager to "Kafka Bridges" and create a "Kafka Sender".

◉ Add the Kafka Broker Host and Port details in the ‘Bootstrap Address List’ and copy the contents of the 'keystore.pem' file and paste them under Client Certificate Authentication - > Content as shown below. Additionally, include the passphrase that we entered while converting the .p12 file to .pem in the ‘Password’.

◉ Once the Kafka Sender is created, go inside, and click on "Queue Binding".

◉ Select our queue – ‘AEM_DEV’ created in section 3.

◉ Go inside the Queue Binding created in earlier step and add the topic name of the target Kafka cluster in the "Remote Topic".

◉ Confirm whether the Kafka connection is up and running.

5. Configure S/4 HANA to SAP AEM connection

◉ Now to establish a connection from S/4 HANA to AEM go to transaction code SM59, create a type-G RFC destination and enter the host and port of the SAP AEM service broker.

◉ In transaction code /IWXBE/CONFIG, create Channel configuration in the S/4 HANA system by clicking on ‘via Service Key -> Advanced’ and assign the RFC destination created in the earlier step. In the ‘Service Key’ section enter the JSON content of the service key created using ‘aem-validation-service-plan’ instance in BTP cockpit.

◉ Save the above changes and activate the channel.

◉ Create an outbound binding and assign any standard topic. For example, select "Business Partner". So whenever a Business Partner is newly created or modified, a standard event will be raised through this outbound channel.

6. Testing the end-to-end connection

◉ To test the end-to-end connection, go to transaction code BP and create a Business Partner. Click on save.

◉ Once saved, an event should be raised. You can check this by going to transaction code /IWXBE/EEE_SUPPORT and then to /IWXBE/R_EVENT_MONITOR.

◉ Select your AEM channel.

◉ You will find a list of all events that were raised and sent to AEM.

◉ Now, go to AEM. In the Kafka sender, you can see the message count in the sent section has increased. This means that the message was successfully received by AEM and then pushed to the Kafka cluster. Additionally, verify the message at the Kafka end.

◉ You can also navigate to the 'Try-Me' section where you can set up the sender and receiver connection. Subsequently, you can subscribe to our topic at the receiver end and observe the incoming message from S/4 HANA as shown below.