LUKS encrypting ephemeral disks - Azure

Cloud providers offer disk-optimized virtual machine types targeted at high-performance distributed document and columnar stores.  These VM types are targeted at systems like Cassandra, ElasticSearch, MongoDB. Databases can consume a disk directly without standard file systems.  Cloud providers leave it up to you to decide how the drives will be configured.  We need to encrypt these drives to protect data "at rest".

Cloud providers leave it up to you to configure and encrypt the drives.  

Virtual Machines and Local Storage

Virtual Machines have several classes of data storage.  
  1. Network dedicated boot and mounted drives 
  2. Network shared similar to SMB or NFS.
  3. Local ephemeral SSDs primarily aimed at temporary directories or swap space.
  4. Local high-performance NVMe/SSD provided to the machines as raw devices.
It is the latter type of drive that databases and caching servers can self-manage to get high performance. They will typically hold data so they need to be encrypted at rest.  They are self-managed so the VM creator needs to implement the encryption method of their choice.  We will LUKS encrypt the devices before presenting them to the O/S or database.
Locally attached drives survive VM restarts.  They are lost when a machine is terminated.

Video

Encryption walkthrough and introduction to https://github.com/freemansoft/ephemeral-disk-encryption-azure

Automounting Encrypted Volumes

The drives must first be prepared.  Then encryption key acquisition scripts are put in /etc and drive/service configuration files must are put in /etc/systemd.  Each local device requires two files.

  • A service definition that opens the LUKS encrypted device to present it as a decrypted device.
  • A mount definition that mounts the decrypted device.  It has a dependency on the service definition and waits until the description command completes.
See the files in the vm-files in the repo for examples. 

The bootstrap file runs one time.  It partitions and encrypts all of the local devices it finds. It then creates service and mount definitions from the templates and puts them in /etc/systemd.

The service and mount definitions are templates that are converted into actual definitions when put into /etc/systemd. There is one .service and one .mount file for each LUKS encrypted device.

Disks after mounting two LUKS encrypted partitions

The 'lsblk' command shows all the block storage devices. The following output shows two NVMe drives each with a single partition.  Each partition is fully consumed by a LUKS encrypted device. The example code xfs formats them and mounts the crypted device as /data<NVMe drive num>.
$ lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
loop0         7:0    0 99.4M  1 loop  /snap/core/11420
loop1         7:1    0  240K  1 loop  /snap/jq/6
sda           8:0    0   80G  0 disk
└─sda1        8:1    0   80G  0 part  /mnt
sdb           8:16   0   30G  0 disk
├─sdb1        8:17   0 29.9G  0 part  /
├─sdb14       8:30   0    4M  0 part
└─sdb15       8:31   0  106M  0 part  /boot/efi
sr0          11:0    1  628K  0 rom
nvme0n1     259:0    0  1.8T  0 disk
└─nvme0n1p1 259:1    0  1.8T  0 part
  └─data0   253:0    0  1.8T  0 crypt /data0
nvme1n1     259:0    0  1.8T  0 disk
└─nvme1n1p1 259:1    0  1.8T  0 part
  └─data1   253:0    0  1.8T  0 crypt /data1

Related blogs and videos


Created 2021 08 09

Comments

Popular posts from this blog

Accelerate Storage Spaces with SSDs in Windows 10 Storage Pool tiers

Java 8 development on Linux/WSL with Visual Studio Code on Windows 10

Docker on a Chromebook on Crostini - Neverware CloudReady is ready