How to Store the Security Server Message Log Archives in the Azure Blob Storage?

Security Server’s business log is stored in the message log database which contains all the messages processed by the Security Server. Each message is time-stamped and signed which makes it possible to verify the message content afterwards. By default, time-stamped messages are archived from the database to disk every six hours. Time-stamped and archived messages are kept in the message log database for 30 days until they are removed automatically.

The archived log records are not automatically transferred to an external host or log storage for long-term archiving purposes. It is the administrator’s responsibility to configure the transfer of the logs to a long-term storage.

Long-term Archival

Azure Blob Storage can be used as a long-term storage for the Security Server logs. Azure Blob storage is Microsoft's object storage solution for the cloud. Azure Blob Storage always encrypts data both in transit and at rest. For additional security, customer-managed or customer-provided encryption key can be used to encrypt data at rest.

To store and transfer the archived message log records in a Blob Storage container, follow the steps below.

• Create a storage account.
  • Account kind: StorageV2 (general purpose v2)
  • Select the storage account properties (e.g. location, replication, default storage class etc.) based on your requirements.
• Create a blob storage container under the storage account.
  • It is recommended to create a container per Security Server.
  • N.B.! Only users who are authorized to access message log archives should be granted access to the container. See available default roles.
• Enable a system-assigned managed identity on the Security Server host VM.
• Grant the Storage Blob Data Contributor role to the managed identity - the scope of the role must be the container.
  • The role must be granted for a specific container, not for the whole storage account.
  • The built-in Storage Blob Data Contributor role allows read, write and delete access to the container. It's recommended to create a custom role that allows only write access to the container. However, creating custom roles requires Azure AD Premium P1 or P2.
• Go to the container Settings → Access Policy and create a time-based retention policy for the container.
  • The objects (=Security Server log archives) cannot be deleted or updated during the retention period.
• Go to the storage account configuration → Blob service → Lifecycle Management and create lifecycle rules that automatically update storage class or delete objects that have exceeded certain age, e.g. delete objects older than 90 days. Lifecycle policies can be used to automatically delete logs that have exceeded their retention period.
• Install (if not already installed) AzCopy tool on the Security Server host.
• Create a bash script (e.g. /usr/share/xroad/scripts/archive-logs.sh) on the Security Server host and grant xroad user sufficient rights to execute the script. Below there's an example script that transfers archived message log files from Security Server to blob storage, and deletes all the successfully transferred files from the Security Server.
  

  • N.B.! Replace <STORAGE_ACCOUNT> and <CONTAINER> with the names of your storage account and container (line 11).

#!/bin/bash

# Log in by using the system-assigned identity of a VM
LOGIN=$(azcopy login --identity)
if [[ "$LOGIN" != *"Login with identity succeeded"* ]]; then
  echo $LOGIN
  exit 1
fi

# Copy X-Road message log archives to the specified STORAGE_ACCOUNT and CONTAINER
OUTPUT=$(azcopy copy "/var/lib/xroad/*.zip" 'https://<STORAGE_ACCOUNT>.blob.core.windows.net/<CONTAINER>')

# Get the ID of the operation
ID=$(echo $OUTPUT | grep -Po '(\w+-\w+-\w+-\w+-\w+)' | head -1)

# Get the list of files that were successfully copied
FILES=$(azcopy jobs show $ID --with-status=Success)
for fn in $FILES; do
  if [[ "$fn" =~ ^\/var\/lib\/xroad/mlog- ]]; then
    # Remove file from Security Server
    rm "$fn"
  fi
done

exit 0

• Override the configuration parameter archive-transfer-command (create or edit the file "/etc/xroad/conf.d/local.ini") to set up a transferring script. The script is executed every time when the archival process is run. The archiving schedule can be customized using the archive-interval configuration parameter.
  
  • The script uses Azure's azcopy tool to move the archived log records from the Security Server to the Blob Storage container. Once a file is successfully uploaded to the blob storage, the script removes the file from the Security Server.

[message-log]
archive-transfer-command=/usr/share/xroad/scripts/archive-logs.sh

• Restart the xroad-proxy process after updating the archive-transfer-command configuration parameter value.

Related articles

• Page:
  How to Interpret Security Server Proxy Error Messages?
• Page:
  How to Set Up a Local Test Environment Using Docker Compose?
• Page:
  Security Server Cluster RHEL 8 (or RHEL 7) to RHEL 9 Upgrade
• Page:
  How to Upgrade Security Server to RHEL 9 Using a Configuration Backup?
• Page:
  How to Access the Security Server Postgres Database?