How to Upgrade Security Server to RHEL 8 Using a Configuration Backup?

This document describes the steps required for migrating a stand-alone Security Server from an existing RHEL 7 host to a new RHEL 8 host. The migration is done taking a backup of the Security Server configuration on the RHEL 7 host and restoring the backup on the RHEL 8 host. Please read carefully through the whole document before starting the upgrade process.

This document assumes that you are using two different server hosts concurrently during the upgrade process:

old server, which runs RHEL 7 (PostgreSQL version 12)
new server, which runs RHEL 8 (PostgreSQL version 12 or up).

Terms old server and new server will be used to refer to these.

Preparation

Connecting a hardware security module (HSM) to a new server may require additional steps that are not covered by these instructions. Check the HSM documentation and verify that it supports RHEL 8.
Ensure that the X-Road software is at least version 7.4.0.
On the old server, use the admin UI to take a backup of the Security Server configuration and download it to a safe location.
- Note that the backup does not include X-Road admin user account(s) or /etc/xroad.properties (database admin credentials; needed when using a remote database). You need to take care of moving these to the new server manually. More about remote database credentials could be found in Security Server Installation Guide for Red Hat Enterprise Linux: 2.6 Remote Database Setup
In order to route traffic to the new server after the upgrade is complete, prepare to update your network configuration.
- After the upgrade, you may need to change the new server's public IP address(es) to match the old public addresses and/or update DNS, firewall, NAT, or other network configuration so that other Security Servers and your information systems can reach the new server. The exact steps depend on your network setup and are not covered in this guide. Note that if the publicly visible IP address of the upgraded Security Server changes, you may need to contact your X-Road Instance operator and/or other members for firewall rule changes.

Upgrade process

Do a clean install of Security Server software version at least 7.4.0 on RHEL 8 (see the Security Server Installation Guide for Red Hat Enterprise Linux ) to the new server.
- The admin UI and internal TLS certificates created during the installation process will be overwritten by the ones restored from the backup.
- The X-Road admin user is not included in the backup (must be created manually).
Restore the Security Server configuration from the backup. For example uninitialized Security Server can be restored with command:
```
sudo -iu xroad /usr/share/xroad/scripts/restore_xroad_proxy_configuration.sh -F -P -N -f <backup file>.tar
```
Encrypted backup archive can be first unencrypted with command:
```
sudo -iu xroad gpg --homedir /etc/xroad/gpghome --output <backup file>.tar --decrypt <backup file>.gpg
```
where /etc/xroad/gpghome keyring should contains old server encryption keys.
For more information see Security Server User Guide: 13 Back up and restore.
Optionally, copy the messagelog database and archived log records from the old server
- On the old server, stop the xroad-proxy process and dump message log data using pg_dump (note that the size of the messagelog database can be large, so this can take time and a space).
- ```
sudo systemctl stop xroad-proxy
sudo -iu postgres pg_dump -d messagelog -Fc -f <dump_file>
```
  - See https://www.postgresql.org/docs/12/app-pgdump.html for more information about creating a database dump.
- Copy the dump to the new server and restore the database on the new server:
- ```
sudo systemctl stop xroad-proxy
sudo -iu postgres pg_restore -d messagelog -c <dump_file>
```
  - See https://www.postgresql.org/docs/12/app-pgrestore.html for more information about restoring a dump.
- On the new server, start the xroad-proxy process.
- ```
sudo systemctl start xroad-proxy
```
- Optionally, copy the archived message records (in /var/lib/xroad) to the new server.
  - Note that some of the logged messages might not have been archived yet and exist only in the database.

Reducing the size of the message log database dump: By default the message log database keeps 30 days of message records but one can (temporarily) change the retention time in order to reduce the number of records in the database, thus reducing the size of the database dump. Changing the setting has no immediate effect since by default the message log cleanup is run twice a day (noon and midnight). See Security Server User Guide: 11.1 Changing the Configuration of the Message Log for more information.

Optionally, copy the operational monitoring database from the old server (if xroad-opmonitoring is installed).
- On the old server, stop the xroad-opmonitor process and dump the database:
- ```
sudo systemctl stop xroad-opmonitor
sudo -iu postgres pg_dump -d "op-monitor" -F c -f <dump_file>
```
- Copy the dump to the new server and restore the database on the new server:
- ```
sudo systemctl stop xroad-opmonitor
sudo -iu postgres pg_restore -d "op-monitor" -c <dump_file>
sudo systemctl start xroad-opmonitor
```
Switch over to the new server (stop the old server and update your network configuration accordingly).
- For example, change the new server's IP address(es) to match the old addresses and/or update DNS, firewall, NAT, or other network configuration.