SAP HANA SP100 SDA setup with Apache Hadoop

In my documentation I’ll explain how to setup and configure a SAP Hana SP10 SDA with Apache Hadoop. I will show in detail step and configuration point to achieve this it.

HANA revision 100 reserve a lot new features, refer to the following link for the complete list
SAP Hana SP10 what's new

In order execution

• Apahce Hadooop installation
• Setup Hana to consume Hadoop data
• Connect SAP Hana studio to Hadoop for SDA
• Manage Hadoop Cluster with Ambari

Guide used
Simba ODBC Driver for Apache Hive
HDP installation GuideHANA SDA guide

Note used
2165826 - SAP HANA Platform SPS 10 Release Note
2177918 - SAP HANA Hadoop Ambari Cockpit SP10

Link used
http://hortonworks.com/products/releases/hdp-2-0-ga/#installhttp://www.simba.com/connectors/apache-hadoop-hive-odbchttp://hortonworks.com/products/releases/hdp-2-0-ga/#install
Architecture overview

Installation of Apache Hadoop

Apache Hadoop will be installed on our Windows environment, before installing the package, the following software needs to be installed as requirement :

• Microsoft Visual C++ 2010 Redistributable Package (64 bit)
• Oracle JDK 7 64-bit
• Microsoft.NET framework 4.0
• Python 2.7

Once the required software are installed download the latest version from the website (the current version is 2.3)

Open DOS and run "msiexec /lv d:\hdplog.txt /i "D:\Software\Hadoop\hdp-2.0.6.0.winpkg.msi" to launch the program

Choose Derby as DB flavor

Open the command line shortcut and start the hadoop services

All the services are running

Validate the installation by making a SmokeTest

Check the node status and the cluster status

Setup Hana to consume Hadoop Data

The Hadoop server is now up and running but before creating a connection from Hana, two ODBC drivers need to be download on the Hana server.

UnixODBC driver and SimbaODBC driver 

UnixODBC driver can be download from the following website: http://www.unixodbc.org/

Once both are downloaded, start by decompressing the SimbaODBC driver

Use the command “gunzip” to remove the “gz” and use after the command “tar xvf” to decompress the tar file. The simba folder is the extracte. Do the same thing for unixODBC

Move the two folders at the root level

As <SID>adm user, move into the samba setup folder and copy the samba.hiveodbc.ini in the home directory, then do a VI and change parameters

Now install the last version of unixODBC driver for Simba, from unixODBC folder run the following command:

1) ./configure

2) Make

3) Make install

Configure the classpathby creating customer.sh file using VI (~/.customer.sh) with the fallowing entry

And create an odbc.ini file using VI (~/.odbc.ini) with the following entry:

• DSN name
• Driver location
• Host ip of the Hadoop server
• Port to use for Hiveserver(default)
• Hive server type
• Athentication method
• User for authentication
• Password for user auth.

And link this file to the customer.sh file created before by adding the following line:

export ODBCINI=$HOME/.odbc.ini

Do a test connection from Hana server to Hadoopby running : isql –v HIVE

The license for samba driver needs to be installed once it’s done do the test again

Successfully connected

.

Do a “show tables “ to make sure we are on the right system 

Connect SAP Hana to Hadoop for SDA

In provisioning, choose create remote source and create a new

Fill up all the required information

Refresh the remote source panel

The connection is made and can see the tables available 

Manage Hadoop cluster with Ambari

Stating SP10, HANA provide a new delivery unit which allow to manage your Hadoop cluster over Ambari

"HANAHADOOPAMBR10_0-80001012.zip"

Once upload, the new role needs to be assign

and the application is available in the catalogue

Access it and provide the necessary information

and access the cockpit

The simple Hadoop connection over SDA is done.

Source: scn.sap.com