SAP Hana Technology and Hadoop Ecosystem

SAP Hana cutting-edge Technology and Big Data Hadoop Ecology

Hana Hadoop Integration with Federated Access

Data Lake analytics have become real. But the challenge is to access the data quickly and provide meaningful insights. There are several techniques to access the data faster. In this blog we see how we can integrate Hana with Hadoop to get insights for larger data sets quickly.

If you have both Hana & Hadoop in your eco-system. SAP has provided an option to integrate HANA and Hadoop using Hana Spark controller. Where power of In-Memory processing can be used for real time insights and we can in parallel use Hadoop ability to process huge data sets.

What is VORA and How it helps to Bridge the gap between Enterprise data and Big Data

Before getting into the Topic of VORA first we lets try to understand what is Enterprise data, Big Data, HADOOP, SPARK.

What is Enterprise Data – Data that comes from Day today business transactions eg. Sales order, Purchase Order, etc.

What is Big Data – Data that comes from information-sensing mobile devices, aerial (remote sensing), software logs, cameras, microphones, radio-frequency identification (RFID) readers and wireless sensor networks, Social Media and Archived Data.

Vora 1.2 installation Cheat sheet: Concepts, Requirements and Installation

SAP HANA Vora provides an in-memory processing engine which can scale up to thousands of nodes, both on premise and in cloud. Vora fits into the Hadoop Ecosystem and extends the Spark execution framework.

Concepts and Requirements:

Sap HANA VORA 1.2 consists of the two following main components:

• SAP HANA Vora Engine: 

SAP HANA Vora instances hold data in memory and boost the performance.

• SAP HANA Vora Spark Extension Library:
• Provides access to SAP HANA Vora through Spark.
• Makes available additional functionality, such as a hierarchy implementation.

Introducing SAP HANA Vora1.2

SAP HANA Vora 1.2 was released recently and with this new version we have added several new features to the product. Some of the key ones I want to highlight in this blog are

• Support for MapR Hadoop distro
• Introducing new “OLAP” modeler to build hierarchical data models on Vora data
• Discovery service using open source Consul – to register Vora services automatically
• New Catalog to replace Zookeper as metadatstore
• Native persistency for metadata catalog using Distributed shared log
• Thriftserver for client access thru jdbc-spark connectivity

The new installer for Vora in ver1.2 extends the simplified installer to be able to use Hadoop Management tools like MapR Control System to deploy Vora on all the Hadoop/Spark nodes. This is an addition to what was provided in ver1.0 for Cloudera Manager and Ambari admin tools.

Hadoop and HANA Integration

HANA Hadoop:

Lets start with advantages of using Hadoop:

It can easily handle huge amount of data volumes
It is very good for storing Unstructured data
It is reliable, scalable and fault tolerant
It is Open source so is less costly
It provides Batch Processing

Now Lets look at some of the limitations of Hadoop:

It is not efficient to use for small anmount of data
It is less mature
It is difficult to find qualified Talent
It is not suited for real time scenarios

[SAP HANA Academy] Configure the SAP HANA Spark Controller to Read SAP HANA Vora Tables

In another part of the SAP HANA Academy's SAP HANA Vora series Tahir Hussain Babar (Bob) walks through how to configure the SAP HANA Spark Controller in nine tutorial videos. With the SAP HANA Spark Controller you will be able to read your SAP HANA Vora tables from SAP HANA.

Each and every script that Bob uses through out this nine part series can be found here on GitHub.

How to Install Hive and Load Data