What Is SAP HANA Streaming Analytics?

SAP HANA streaming analytics enables you to analyze high-volume, high-velocity data event streams to extract data. This post will introduce you to SAP HANA streaming analytics fundamentals.

Let’s get started with some basic information. Events are messages that are sent when something happens, for example, a stock price went up or a plane landed (too early, too late, or on time). Internet of Things (IoT) devices constantly send out messages, often in different formats, making processing these messages and generating alerts for exceptions difficult. Especially for the most interesting events (exceptions), the goal is to store this information in the database for further analysis to identify patterns, detect problems, and predict trends.

Technology Background

Complex event processing (CEP) technology emerged in the 1990s and quickly become useful for operational intelligence and business process management (BPM). Financial services became early adopters of the event-driven architecture (EDA) for use in algorithmic trading. Software vendors who pioneered the technology were eventually acquired by larger vendors, like TIBCO, Oracle, IBM, and Sybase/SAP. Through these acquisitions, the Aleri CEP engine, the SQL-like CCL from Coral8, and the Stream Processing Language Shell (SPLASH) all made their way into SAP HANA streaming analytics. The explosion of IoT devices and big data provided new use cases under the more generic banner of stream processing.

Today, SAP offers CEP technologies integrated in the SAP HANA platform as SAP HANA streaming analytics, as a service in SAP Streaming Analytics, and through the standalone SAP Event Stream Processor.

For example, let’s say a Formula One car has 120 sensors recording 1,000 data points per second. Including test rounds, for a team of 2 cars, about 9 GB+ of data is collected in a single race. SAP HANA streaming analytics technology can parse data streams like this in real time and perform analytics for real-time responses (before the race is done). With guaranteed delivery of data and no data loss, all messages are processed. Event stream processing runs on dedicated streaming servers, which can be clustered and support high-availability configurations; a complete picture of the architecture is shown here:

To interact with event streams, you can use the CCL, which is like SQL but with its own statements, clauses, and functions, extended in CCLScript, as shown in the following listing.

You can use the SAP Web IDE for SAP HANA or the SAP HANA streaming analytics studio (shown below) to design the event stream processes visually in a streaming project (including filtering, normalizing, combining streams). Apply CCL functions, either built-in or user-defined, to transform your raw data and store the results in SAP HANA for analysis, for example, as inputs for machine learning algorithms. For larger data volumes, you can use Hadoop files or other databases.

SAP HANA streaming analytics also integrates with the SAP HANA cockpit for real-time project monitoring, for starting and stopping streams, and for configuring alerts. You can connect to streaming projects with web service clients; develop custom applications that publish or subscribe to data streams using C, Java, or .NET SDKs; and develop custom adapters to connect SAP HANA streaming analytics to any external data source.