HANA Multitenant System Copy using Cockpit and Netbackup (Backint)

There are times when your system copy requirements are more complex than what the guide describes. There are also times when the official documentation is not only innacurate but also completely misleading. This blog is about such a time.

Goal

When performing system copy you usually have several option how to copy the database from the source to the target system. Here I am focusing on restore using backint. In this case I am using Veritas Netbackup, but this procedure should be mostly valid for any certified HANA backint solution.

The reason for writing this blog is that SAP HANA Admin Guide is too generic and keeps refering to HANA studio. Also Netbackup Admin guide is still referring to HANA Studio too. Unfortunately HANA Studio isn’t really developed anymore and the current version can’t be used to do system copy using restore on a multitenant (i.e. HANA 2.0) database.

Rationale

Why would you use restore from a backup rather than take a snapshot or dump the database to a file? The approach you would take depends on your environment and requirements. The main reason is the requirement to restore multiple databases, to the same point in time. To add to the complexity these can be different types of databases. Recently we had to perform system copy of six systems which are running on HANA, Sybase ASE and Max DB. Also we were not able to have an outage of all systems at the same time to take a consistent snapshot or database dump. The only option left – restore from backups.

Prerequisites

Before you attempt to perform a restore to different system (e.g. Prodution to Pre-Production), you obviously need to have a working backup solution and useable backups to restore from. If anyone is interested I can do another blog explaining how to correctly configure HANA backups with Netbackup. Message me or make a comment below.

Another major prerequisite for HANA 2.0 is HANA Cockpit. Both SAP and Netbackup documentation still mention HANA Studio. Although it looks like it is going to work, the restore from Studio is not able to correctly find the backup catalog, which means it’s not possible to restore anything. So, ignore the official guides and follow this one.

Initial configuration

I will refer to the system that you are restoring from (e.g. production ) as a source system and the system you are refreshing/restoring to/copying to (e.g. pre-production) as a target system.

Redirected system restore (for System Copy) has a few additional requirements in addition to the standard backup configuration:

1. Backup catalog must be backed up using backint.

This can be achieved by setting parameter

catalog_backup_using_backint = true

in [backup] section of global.ini.

2. The names of the utl configuration files have to use $(SAPSYSTEMNAME) rather than have hardcoded system SID.

This typically applies to parameters:

log_backup_parameter_file

catalog_backup_parameter_file

data_backup_parameter_file

For example instead of

catalog_backup_parameter_file = /usr/openv/netbackup/ext/db_ext/sap/scripts/sap_hana/initHDB.utl

you should use

catalog_backup_parameter_file = /usr/openv/netbackup/ext/db_ext/sap/scripts/sap_hana/init$(SAPSYSTEMNAME).utl

It looks scary, but it works. This should be set on all your systems, but at least on the target system.

3. Copy the source .utl files to the target server.

The exact location of the .utl files depends on your configuration and a backup system. You need to copy any init<SOURCE>.utl files (there can be more than one) with their names unchanged to the target HANA server. The target server will then have it’s own init<TARGET>.utl files as before, but will also have the init<SOURCE>.utl files as these are used for the restore.

4. Allow the target server to access the source server backups.

This can differ based on your local configuration and on a backup system, but with Netbackup you need edit file /usr/openv/netbackup/ext/db_ext/sap/node_names.txt on the target server and add the source server name to the list. This will allow the target server to read and restore backups that were written by the source server.

Step-by-step process

Now that everything is ready, we can finally start the restore.

1. Select the target tenant

Login to HANA Cockpit, find the target database and click on “Manage Database” link for the SYSTEMDB of the target system. Once on the Database Management screen, select the target tenant.

2. Copy tenant using backup

Make sure you have selected the correct target tenant! From the “Tenant Actions” menu select “Copy Tenant Using Backup”.

3. Tenant will be stopped

After you confirm this prompt the tenant database will be stopped. Make sure you are ready for it. This means you need to stop your SAP instances before you continue any further. If the restore starts succesfully the database will be ovewritten, so when reaching this point you must have finished all the pre-system copy steps on your target system (e.g. saving RFC destinations or exporting certificates, etc.).

4. Database copy type

This is the first step of the database copy wizard. As the goal, in this case, is to perform point-in-time recovery, you have to use both data and log backups to get to the desired state.

5. Point in time

Step two allows a selection of the time zone and desired time. It is important to understand that the recovery may not reach the exact time as there may not be any commited transactions in the database at the exact time you selected.

Also note that the date is always in US format in the current version of HANA Cockpit.

And the last warning. Be aware of summer/winter time in case  you are doing system copy at around the clock change.

6. Source System Type

This is the option that isn’t available in HANA Studio, but only in Cockpit. All HANA 2.0 systems are multitenant and HANA 1.0 goes out of support in June 2021, so this will soon become the only useable option.

7. Source Database and Tenant

In step 5 of the wizard you have to provide the SID of the source HANA system and the tenant. Again, this is the functionality that’s missing in the Studio.

8. Backup list

If the prerequisites have been fulfilled and the initial configuration above was completed correctly, you will see a list of the relevant source system backups. This is the step that is most likely to fail if the configuration isn’t correct. For troubleshooting check the logs in /usr/openv/netbackup/logs directory.

9. Delta backups

It is always a good practice to use any available delta backups. It will speed up the restore process.

10. Check backups

This option will make sure that the availability of the required backups is checked at the beginning of the process, rather than finding out after two hours of restoring that something is missing. This is also the last step of the wizard.

11. Review

Before executing the restore, here is the last chance to check that everything is as it should be. Pressing “Start Copy” at the bottom will start the restore process (after one more prompt). There is also a button “Display SQL statement” which you can use to see what command is Cockpit issuing to the database.

12.Final Warning

This is the last chance to stop the process.

13. Progress and completion

I haven’t captured a screenshot of the progress screen, but it is fairly self explanatory. The progress bar will start growing until the restore has finished. The duration depends on your hardware, backup system, database size and network throughput.

Once completed you will see the total duration of the restore as well as the point in time the system was recovered to. As I mentioned above, it may not be the exact time you selected.