HANA ML DataFrame : End-to-end methods and it’s usage

A small write-up on HANA ML dataframe , it is really a learning , an exposure and a knowledge sharing process to write something beautiful you learn along with your day to day job so holding the passion for technology in both of my hands here come’s my first post of 2020 and topic is interesting enough , everyone’s favorite. HANA Machine learning & it’s about dataframe this time.

Increase HANA data capacity with – SAP HANA Native Storage Extension(NSE).

Introduction:

SAP HANA Native Storage Extension(NSE) is a general-purpose, HANA built-in warm data storage solution. This solution is available on HANA 2.0 SP04. As we all know that now a day’s value of data is enormous from an organization standpoint of view and Hence data storage solution and Access mechanism of data matter a lot. Data is broadly categorized into 3 different temperature tiers – Hot, Warm and Cold. Frozen is also there, normally it is not considered in HANA data tiering options(DTO).

SAP Hana Technology and Hadoop Ecosystem

SAP Hana cutting-edge Technology and Big Data Hadoop Ecology

Calling HANA Views from Apache Spark

Open Source Apache Spark  is fast becoming the de facto standard for Big Data processing and analytics. It’s an ‘in-memory’ data processing engine, utilising the distributed computing power of 10’s or even 1000’s of logical linked host machines (cluster). It’s able to crunch through vast quantities of both structured and unstructured data. You can easily scale out your cluster as your data appetite grows.

In addition to this it can also be used as a data federation layer spanning both traditional databases as well as other popular big data platforms, such as Hadoop HDFS, Hadoop Hbase, Cassandra, Amazon Redshift and S3, to name a few.

Vora 1.2 installation Cheat sheet: Concepts, Requirements and Installation

SAP HANA Vora provides an in-memory processing engine which can scale up to thousands of nodes, both on premise and in cloud. Vora fits into the Hadoop Ecosystem and extends the Spark execution framework.

Concepts and Requirements:

Sap HANA VORA 1.2 consists of the two following main components:

• SAP HANA Vora Engine: 

SAP HANA Vora instances hold data in memory and boost the performance.

• SAP HANA Vora Spark Extension Library:
• Provides access to SAP HANA Vora through Spark.
• Makes available additional functionality, such as a hierarchy implementation.

Introducing SAP HANA Vora1.2

SAP HANA Vora 1.2 was released recently and with this new version we have added several new features to the product. Some of the key ones I want to highlight in this blog are

• Support for MapR Hadoop distro
• Introducing new “OLAP” modeler to build hierarchical data models on Vora data
• Discovery service using open source Consul – to register Vora services automatically
• New Catalog to replace Zookeper as metadatstore
• Native persistency for metadata catalog using Distributed shared log
• Thriftserver for client access thru jdbc-spark connectivity

The new installer for Vora in ver1.2 extends the simplified installer to be able to use Hadoop Management tools like MapR Control System to deploy Vora on all the Hadoop/Spark nodes. This is an addition to what was provided in ver1.0 for Cloudera Manager and Ambari admin tools.