Backing Up orcharhino

When and how to backup orcharhino is a question highly dependent on your individual usage and setup. We recommend performing backups before and after orcharhino upgrades.

The default use case is for orcharhino installations to run on virtual machines, most commonly within a VMware virtualization environment. We recommend using the snapshot functionality provided by the virtualization solution for backups in such a setup. We recommend making a file based backup with your enterprise backup solution if you run orcharhino on bare metal hardware.

Backing Up orcharhino Server Running on a Virtual Machine

We recommend creating snapshots as provided by your virtualization solution orcharhino Server is running on, for example using VMware’s vSphere client, and to backup those snapshots. Your orcharhino VM is ideally powered down before taking a snapshot. Only if powering down your orcharhino is operationally not possible, you may take a live snapshot that also retains the VM’s memory.

You should check if there are any long running tasks before taking the snapshot or stopping any services:

# foreman-rake katello:upgrade_check

This ideally results in output as follows:

Checking for running tasks...
[SUCCESS] - There are 0 active tasks.
         You may proceed with the upgrade.

Newer snapshots will include lots of references to older snapshots. This can be inefficient and negatively affect system performance if too many snapshots are kept at the same time. It is recommended to limit the number of snapshots to no more than three. Consider deleting older snapshots after creating new ones.

Alternatively, you can also create a backup based on the snapshot functionality of LVM, which would include the following steps:

Your enterprise backup solution triggers all pre backup tasks, which include -depending on your backup strategy- stopping all orcharhino services or powering down the orcharhino instance altogether.
Create an LVM snapshot via your backup application. In case the VM is still running, create a snapshot of its memory, too.
Restart all orcharhino services or restart your orcharhino instance.
Create a block-based backup of the snapshot according to your general backup routine.
Have the backup application delete the LVM snapshot.

Updating Repositories after reverting to a Snapshot

If you abort your orcharhino Server upgrade and revert to an earlier snapshot, you have to update the enabled repositories on your orcharhino Server. Run subscription-manager repos --disable="*" to disable all repositories. Run the command as output by /opt/orcharhino/maintain/repo_map/repolist.py -o My_orcharhino_Version --latest-os --sm-command to enable the proper repositories for your orcharhino Server. You can now use both yum update to update packages and orcharhino-installer, for example to install the Microsoft Azure plugin, on your orcharhino Server.

Backing Up orcharhino Server Running on a Bare Metal

Backing up orcharhino running on bare metal strongly depends to how you backup bare metal machines running CentOS, Oracle Linux, and Red Hat Enterprise Linux in general. This part describes a general backup strategy using a file-based approach.

Depending on the size of the data to be backup up, orcharhino will be offline for a correspondingly long time. The required offline period depends largely on the performance of your hardware and backup infrastructure as well as the number and size of synchronized repositories. Note that there are also other valid ways to backup orcharhino, but remember to always backup your orcharhino instance in a consistent manner, which is usually achieved by stopping all orcharhino services.

Procedure

Stop your orcharhino instance with a pre backup hook by your enterprise backup solution:
```
# orcharhino-maintain service stop
```
Check if all orcharhino services have been stopped successfully:
```
# orcharhino-maintain service status
```
Deactivate all cron jobs or similar which could potentially trigger orcharhino tasks with a pre backup hook by your enterprise backup solution. This is necessary as some tasks may alter databases on your orcharhino even though the orcharhino service itself is not running.
Execute a file based backup with your enterprise backup solution.

You must include all relevant paths of the operating system and any installed applications. These might vary from the standard paths depending on your environment.

You must also include all relevant paths of the orcharhino. This can also vary depending on the setup, installed plugins, and running services.

The following list will assume a basic configuration of your orcharhino.

The .hammer directory has to be backed up as it contains configuration files for the Hammer CLI:
```
/root/.hammer/
```
The following directories including all subdirectories must be backed up in order to maintain consistency in case of expanding the orcharhino feature set in the future:
```
/etc/
/opt/
/var/
```
In case you do not want to backup the above mentioned directory structure, at least the following directories and subdirectories must be backed up for a fundamentally functioning orcharhino instance regardless of the operating system:
```
/opt/orcharhino/
/opt/theforeman/
/opt/puppetlabs/
/usr/bin/
/usr/lib/
/usr/sbin/
/usr/share/
```
We always recommend a complete backup of all file systems of the orcharhino system without exceptions.
Restart all orcharhino services with a post backup hook by your enterprise backup solution.
```
# orcharhino-maintain service start
```
Verify if all orcharhino services have been started successfully:
```
# orcharhino-maintain service status
```

Restoring your orcharhino backup takes place according to your general approach regarding restoring bare metal CentOS, Oracle Linux, and Red Hat Enterprise Linux systems.

Backing Up orcharhino Server Using orcharhino-maintain

Ensure that no other administrator plans to run any long running job like synchronizing a repository or creating a content view before starting the backup process. You can use orcharhino-maintain health check to run various health checks on your orcharhino.

There are currently two ways to perform a backup: online and offline backups. While the system is available during an online backup, all orcharhino services are turned off during the offline backup. We recommend using offline backups for production use since this is the only way to ensure data integrity of the backup.

Prerequisites

To perform a backup, you will need to provide a path where the backup files should be stored. Ensure that there is enough free disk space available on this backup path. This example uses /var/backup/ as backup path.
If you use orcharhino-maintain backup snapshot to create backups, ensure that there is more than 2 GiB free disk space on the LVM volume group available where vg-data resides. This is required to make use of LVM snapshots. The free disk space is used during the creation of the snapshot to store all changes that are made to the database. If more disk space is available or required, you can enhance the size, for example to 5 GiB, using --snapshot-block-size 5G.

A full backup of all orcharhino databases may take a long time. Your orcharhino will be unavailable for the entire duration of the backup as all orcharhino services are stopped for offline backups.

The following command creates an offline backup: it stops all orcharhino services first, creates a full backup of all necessary data including Pulp content and all databases, and then starts the services again.

# orcharhino-maintain backup offline /var/backup/

You can use --assumeyes to bypass explicit confirmation when creating a backup or restoring from a backup. This creates a directory like /var/backup/katello-backup-2018-09-12-08-21-11 in which the backup is stored. Use the --preserve-directory (-p) option in case you do not want to create a backup specific subdirectory.

The backup contains all of the Pulp content like RPM and Debian packages, Puppet modules and more. You can omit this step by using the --skip-pulp-content (-s) option.

Restoring a fully working orcharhino from a backup without Pulp content is not possible.

The second way to create a backup is the online mode. Only use an online backup for debugging purposes as the data integrity between the various databases cannot be fully guaranteed. You must not make any modification regarding host provisioning, repository synchronization, content view changes, etc. during an online backup as it leads to inconsistencies in the online backup.

# orcharhino-maintain backup online /var/backup/

It is also possible to make an incremental backup. You need to specify the path to an existing full backup or to another incremental backup. You may create an incremental backup based on the previously created full backup as follows:

# orcharhino-maintain backup offline --incremental /var/backup/katello-backup-2022-03-12-08-21-11 /var/backup/

This creates an incremental backup based on the backup /var/backup/katello-backup-2022-03-12-08-21-11 and stores it in the /var/backup/ directory like /var/backup/katello-backup-2022-03-13-08-23-38.

An incremental backup can be based either on a full backup or another incremental backup. Keep in mind that you will need this backup as well as the last full backup and all incremental backups in between those two to successfully restore your system.

Restoring orcharhino Server Using orcharhino-maintain

Ensure to use the right orcharhino instance and that the orcharhino software version has not changed before starting the restore procedure. The host name within the backup must match the host name of the system you are restoring. In the following example, we will restore the orcharhino based on the previously taken full backup:

# orcharhino-maintain restore /var/backup/katello-backup-2022-03-12-08-21-11

The system is in the state it was in at the time of the backup and all orcharhino services are available again once the restore procedure is finished.

To restore orcharhino to a specific incremental backup state, you need to restore the full backup and all incremental backups up to the wanted state in sequence:

# orcharhino-maintain restore /var/backup/katello-backup-2022-03-12-08-21-11
# orcharhino-maintain restore -i /var/backup/katello-backup-2022-03-13-08-23-38

After you have restored orcharhino from your backup, re-run the orcharhino-installer:

# orcharhino-installer