Page Comparison

...

• Create a storage account.
  • Account kind: StorageV2 (general purpose v2)
  • Select the storage account properties (e.g. location, replication, default storage class etc.) based on your requirements.
• Create a blob storage container under the storage account.
  • It is recommended to create a container per Security Server.
  • N.B.! Only users who are authorized to access message log archives should be granted access to the container. See available default roles.
• Enable a system-assigned managed identity on the Security Server host VM.
• Grant the Storage Blob Data Contributor role to the managed identity - the scope of the role must be the container.
  • The role must be granted for a specific container, not for the whole storage account.
  • The built-in Storage Blob Data Contributor role allows read, write and delete access to the container. It's recommended to create a custom role that allows only write access to the container. However, creating custom roles requires Azure AD Premium P1 or P2.
• Go to the container Settings → Access Policy and create a time-based retention policy for the container.
  • The objects (=Security Server log archives) cannot be deleted or updated during the retention period.
• Go to the storage account configuration → Blob service → Lifecycle Management and create lifecycle rules that automatically update storage class or delete objects that have exceeded certain age, e.g. delete objects older than 90 days. Lifecycle policies can be used to automatically delete logs that have exceeded their retention period.
• Install (if not already installed) AzCopy tool on the Security Server host.
• Create a bash script (e.g. archiveLogs/usr/share/xroad/scripts/archive-logs.sh) on the Security Server host and grant xroad user sufficient rights to execute the script. Below there's an example script that transfers archived message log files from Security Server to blob storage, and deletes all the successfully transferred files from the Security Server.
  

  • N.B.! Replace <STORAGE_ACCOUNT> and <CONTAINER> with the names of your storage account and container (line 11).

...

#!/bin/bash

# Log in by using the system-assigned identity of a VM
LOGIN=$(azcopy login --identity)
if [[ "$LOGIN" != *"Login with identity succeeded"* ]]; then
  echo $LOGIN
  exit 1
fi

# Copy X-Road message log archives to the specified STORAGE_ACCOUNT and CONTAINER
OUTPUT=$(azcopy copy "/var/lib/xroad/*.zip" 'https://<STORAGE_ACCOUNT>.blob.core.windows.net/<CONTAINER>')

# Get the ID of the operation
ID=$(echo $OUTPUT | grep -Po '(\w+-\w+-\w+-\w+-\w+)' | head -1)

# Get the list of files that were successfully copied
FILES=$(azcopy jobs show $ID --with-status=Success)
for fn in $FILES; do
  if [[ "$fn" =~ ^\/var\/lib\/xroad/mlog- ]]; then
    # Remove file from Security Server
    rm "$fn"
  fi
done

exit 0

• Add the below line to the xroad user's crontab on the Security Server. The line executes the script which transfers all the archived message log files from the Security Server to Blob Storage container every six hoursOverride the configuration parameter archive-transfer-command (create or edit the file "/etc/xroad/conf.d/local.ini") to set up a transferring script. The script is executed every time when the archival process is run. The archiving schedule can be customized using the archive-interval configuration parameter.
  
  • The script uses Azure's azcopy tool to move the archived log records from the Security Server to the Blob Storage container. Once a file is successfully uploaded to the blob storage, the script removes the file from the Security Server.

[message-log]
archive-transfer-command=/usr/share/xroad/scripts/archive-logs.sh

...