Monday, 3 July 2023

Database Unavailable – HDB Connectivity issue Part 1

Introduction:


Although the database instance was up and all HANA services were active, a database unavailability alert appeared in the “Focussed Run” or in the CCMS alert.

In this article, identify the root cause of the HDB connectivity issue and also check whether the KEY entry corresponding to the database user was present in the hdbuserstore.

Also, identify which database user was required to add the KEY entry in hdbuserstore.

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

Login to  Hana Database host with <sid>adm as unix account user .

Check whether the database instance is up and all services are running as shown below

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

Run the command to verify the hdb connectivity between SAPHostagent and Hana Database  as shown below

/usr/sap/hostctrl/exe/saphostctrl -function GetDatabaseSystemStatus -dbname SYSTEMDB@<SID>  -dbtype hdb

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

As shown above, highlighted in the yellow color, HDB connectivity was working for  “SAPDBCTRL” DB  user for host xxazwls4hxxxx  (lower case characters) for the SYSTEMDB.

Run the command for <SID> tenant DB as shown below

/usr/sap/hostctrl/exe/saphostctrl -function GetDatabaseSystemStatus -dbname <SID>@<SID>  -dbtype hdb

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

HDB connectivity showed status showed error as shown above .

Error message is “no hdbuserstore entry found”.

See the above screenshot regarding DBCredentails – the format is

DBCredential=<hdbuserstore Key name >, <DB user>, Osuser=<sid>adm, Group=sapsys, Datafile(location of SSFS_HDB.DAT file)=/home/<sid>adm/.hdb/<hostname>/SSFS_HDB.DAT; Hdbuseident=<hostname>

Then execute the command  hdbuserstore list  to check the entry present for tenant database as shown below

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

As shown above,  DB user “SAPDBCTRL” for SYSTEMDB was not present in the hdbuserstore list. Secondly, Path for DATAFile was  shown in upper case <HOSTNAME>

Switch to .hdb folder i.e.

cd /home/<sid>adm/.hdb

enter ls -lrt command as shown below

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

As shown above, there are two folders with the same hostname, but one has the letters “HOSTNAME” in capital letters, XXAZWLS4HXXXX, while the other has them in lower case, xxazwls4hxxxx.

Upper case “HOSTNAME” is the default hostname.

hdbuserstore -H <hostname>  list

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

You can see the key name as shown above for key name  “<SID>SAPDBCTRL” for SYSTEMDB but there was no key entry present for the tenant Database name <SID>.

Procedure:

The key name for tenant DB SID is <SID>SAPDBCTRL<SID>.

Check the password for db user “SAPDBCTRL” in the tenant Database <SID>.

The command is

hdbsql -u SAPDBCTRL -d  <SID>  -i <Hana_instance_number>

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

The command to add the  key for  the tenant database with SAPDBCTRL db user using “hdbuserstore set”  is as follows

Do not forget to include -H option for other than the default hostname.

hdbuserstore -H xxazwls4hxxxx  set  <SID>SAPDBCTRL<SID>   xxazwls4hxxxx:32015  SAPDBCTRL  <password>

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

Check  whether   key entry for the tenant database was present  after executing hdbuserstore set command as shown below

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

As previously mentioned, the key  “<SID>SAPDBCTRL<SID>” entry in the hdbuserstore for the hostname xxazwlss4hxxxx for the tenant Database  <SID> was added.

Use the environment variable “HDB_USE_IDENT” that points to a different hostname as shown below to change the default hostname so that the -H option will not be included for the hdbuserstore command (so you can test whether the key name is working or not using the hdbsql command).

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

As shown above, you can see -H option of “hdbuserstore list”  command was not included.

Verify the key name <SID>SAPDBCTRL<ID> was able to connect to the tenant database <SID> with the right password kept in the hdbuserstore by running the hdbsql command as shown below

hdbsql -U  <SID>SAPDBCL<SID>

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

To restore the previous default hostname, unset the environment variable HDB_USE_IDENT.

Verify whether hdb connectivity was established with hdbuserstore ket <SID>SAPDBCTRL<SID> by executing the command

/usr/sap/hostctrl/exe/saphostctrl -function GetDatabaseSystemStatus -dbname <SID>@<SID>  -dbtype hdb

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

As shown above, hdb connectivity  was established and the status showed Running .

To disable the password lifetime for the SAPDBCTRL database user, log in  SYSTEMDB using hdbsql or hana studio with SYSTEM db user or an equivalent having  “USER ADMIN” privilege

This will allow the database user SAPDBCTRL to stay unlocked indefinitely even if you repeatedly input the wrong passwords.

SAP HANA, SAP HANA Career, SAP HANA Skills, SAP HANA Jobs, SAP HANA Preparation, SAP HANA Tutorial and Materials, SAP HANA Learning, SAP HANA Guides, HANA Services

Repeat the above command for tenant Database <SID> too.

No comments:

Post a Comment