nix-git/modules/btrbk

btrbk

• https://digint.ch/btrbk/doc/readme.html
• https://digint.ch/btrbk/doc/btrbk.1.html
• https://digint.ch/btrbk/doc/btrbk.conf.5.html

btrbk - backup tool for btrfs subvolumes

• Take snapshots (on same disk)
• Create multiple backups (on different disks) over SSH

Further resources:

• https://nixos.wiki/wiki/Btrbk
• https://wiki.gentoo.org/wiki/Btrbk

Snapshots and Backups

subvolume:

snapshot: .[]

timestamp: Can be configured using the timestamp_format option.

N: If multiple snapshots are created at the same time, the increasing counter N differentiates them.

Actions and Informative Commands

run [filter...]

In multiple steps, perform snapshots and backups.

• Read Data: Perform sanity checks
• Create Snapshots
• Create Backups
• Delete Backups
• Delete Snapshots

clean [filter...]

Delete incomplete backups (network error, kill signal while send/receive was ongoing).

stats [filter...]

Print statistics of snapshot and backup subvolumes.

sudo btrbk -c /etc/btrbk/local-backup-hdd.conf stats

SOURCE_SUBVOLUME                       SNAPSHOT_SUBVOLUME                     TARGET_SUBVOLUME                         SNAPSHOTS  BACKUPS
/mnt/data/jc-data/cloud.fykml.de       /mnt/data/snap2/cloud.fykml.de.*       /mnt/backup/snap2/cloud.fykml.de.*              77       17
/mnt/data/jc-data/cloud.privacy1st.de  /mnt/data/snap2/cloud.privacy1st.de.*  /mnt/backup/snap2/cloud.privacy1st.de.*         77       17

Total:
154  snapshots  
 34  backups    (34 correlated)

diff

List the modified files since generation (transid) of subvolume in subvolume .

Columns:

SIZE   file was modified for a total of SIZE bytes
COUNT  file was modified in COUNT generations
FLAGS  "+"  file accessed at offset 0 (at least once)
"c"    COMPRESS flag is set (at least once)
"i"    INLINE flag is set (at least once)

sudo btrbk -c /etc/btrbk/local-backup-hdd.conf diff cloud.privacy1st.de.20240222T2300 cloud.privacy1st.de.20240223T1400

FLAGS  COUNT  SIZE        FILE
+c.        1    2.36 MiB  data/appdata_oc2uzw0ocw65/appstore/apps.json
+..        1   12.00 KiB  data/appdata_oc2uzw0ocw65/preview/a/3/f/c/b/b/7/5105712/256-256-crop.jpg
+..        1   24.00 KiB  data/appdata_oc2uzw0ocw65/preview/a/3/f/c/b/b/7/5105712/384-512-max.jpg
+..        1    8.00 KiB  data/appdata_oc2uzw0ocw65/preview/d/8/4/f/d/0/7/5105717/256-256-crop.jpg
+..        1   20.00 KiB  data/appdata_oc2uzw0ocw65/preview/d/8/4/f/d/0/7/5105717/384-512-max.jpg
+c.        1  208.00 KiB  data/appdata_oc2uzw0ocw65/suspicious_login/models/527
...       18  324.00 KiB  data/nextcloud.log
...      257   40.21 MiB  db/binlog.000119
...      290   48.11 MiB  db/ib_logfile0
+c.        5    5.59 MiB  db/ibdata1
.c.        4  112.00 KiB  db/mysql/innodb_index_stats.ibd
.c.        1   32.00 KiB  db/mysql/innodb_table_stats.ibd
+..        3  264.00 KiB  db/nextcloud/oc_activity.ibd
.c.        1   32.00 KiB  db/nextcloud/oc_activity_mq.ibd
...        2   16.00 KiB  db/nextcloud/oc_appconfig.ibd
+..        3   88.00 KiB  db/nextcloud/oc_authtoken.ibd
...        4   40.00 KiB  db/nextcloud/oc_calendar_reminders.ibd
...        3   40.00 KiB  db/nextcloud/oc_calendarchanges.ibd
+..        6  184.00 KiB  db/nextcloud/oc_calendarobjects.ibd
+..        4  232.00 KiB  db/nextcloud/oc_calendarobjects_props.ibd
.c.        1    8.00 KiB  db/nextcloud/oc_calendarsubscriptions.ibd
...        8    1.18 MiB  db/nextcloud/oc_filecache.ibd
...        2   48.00 KiB  db/nextcloud/oc_filecache_extended.ibd
...        1   32.00 KiB  db/nextcloud/oc_gpodder_episode_action.ibd
...        2   48.00 KiB  db/nextcloud/oc_jobs.ibd
+c.        1   48.00 KiB  db/nextcloud/oc_login_address.ibd
...        3   64.00 KiB  db/nextcloud/oc_login_ips_aggregated.ibd
...        1   40.00 KiB  db/nextcloud/oc_maps_photos.ibd
...        1   24.00 KiB  db/nextcloud/oc_news_feeds.ibd
+..        9    4.98 MiB  db/nextcloud/oc_news_items.ibd
...        4   64.00 KiB  db/nextcloud/oc_notifications.ibd
.c.        2   24.00 KiB  db/nextcloud/oc_notifications_settings.ibd
...        1   16.00 KiB  db/nextcloud/oc_preferences.ibd
...        1   16.00 KiB  db/nextcloud/oc_preview_generation.ibd
...        1   16.00 KiB  db/nextcloud/oc_suspicious_login_model.ibd

Total size: 104.47 MiB

Cleanup

• Adjustments: It is possible to change 2d to something else, e.g. no to keep only the latest snapshot/backup.

• Warning: If all except the latest snapshots and backups are deleted on yodaNas, then the next backup to yodaHedgehog will be non-incremental! Make sure to keep at least one snapshot/backup per subvolume on yodaNas that has already been copied to yodaHedgehog. Or use a config file that covers local (yodaNas) and remote (yodahedgehog) backups!

• First, perform a dryrun. Double-check the output. Then, replace dryrun with run.

sudo btrbk -c /etc/btrbk/local-backup-ssd.conf \
     --override=snapshot_preserve=2d \
     --override=snapshot_preserve_min=latest \
     --override=snapshot_create=no \
     --override=target_preserve=2d \
     --override=target_preserve_min=latest \
     --print-schedule \
     dryrun

SNAPSHOT SCHEDULE
-----------------
ACTION  SUBVOLUME                                          SCHEME              REASON
delete  /mnt/data/snap2/cloud.privacy1st.de.20230927T1827  2d (sunday, 00:00)  -
...
delete  /mnt/data/snap2/cloud.privacy1st.de.20240220T0000  2d (sunday, 00:00)  -
-       /mnt/data/snap2/cloud.privacy1st.de.20240221T0000  2d (sunday, 00:00)  preserve daily: first of day, 2 days ago, at 00:00
delete  /mnt/data/snap2/cloud.privacy1st.de.20240221T0100  2d (sunday, 00:00)  -
...
delete  /mnt/data/snap2/cloud.privacy1st.de.20240221T2300  2d (sunday, 00:00)  -
-       /mnt/data/snap2/cloud.privacy1st.de.20240222T0000  2d (sunday, 00:00)  preserve daily: first of day, 1 days ago, at 00:00
delete  /mnt/data/snap2/cloud.privacy1st.de.20240222T0100  2d (sunday, 00:00)  -
...
delete  /mnt/data/snap2/cloud.privacy1st.de.20240222T2300  2d (sunday, 00:00)  -
-       /mnt/data/snap2/cloud.privacy1st.de.20240223T0000  2d (sunday, 00:00)  preserve daily: first of day, 0 days ago, at 00:00
delete  /mnt/data/snap2/cloud.privacy1st.de.20240223T0100  2d (sunday, 00:00)  -
...
delete  /mnt/data/snap2/cloud.privacy1st.de.20240223T1500  2d (sunday, 00:00)  -
-       /mnt/data/snap2/cloud.privacy1st.de.20240223T1600  2d (sunday, 00:00)  preserve min: latest

BACKUP SCHEDULE
---------------
ACTION  SUBVOLUME                                            SCHEME              REASON
delete  /mnt/backup/snap2/cloud.privacy1st.de.20230927T1827  2d (sunday, 00:00)  -
...
delete  /mnt/backup/snap2/cloud.privacy1st.de.20240220T0000  2d (sunday, 00:00)  -
-       /mnt/backup/snap2/cloud.privacy1st.de.20240221T0000  2d (sunday, 00:00)  preserve daily: first of day, 2 days ago, at 00:00
-       /mnt/backup/snap2/cloud.privacy1st.de.20240222T0000  2d (sunday, 00:00)  preserve daily: first of day, 1 days ago, at 00:00
-       /mnt/backup/snap2/cloud.privacy1st.de.20240223T0000  2d (sunday, 00:00)  preserve daily: first of day, 0 days ago, at 00:00
-       /mnt/backup/snap2/cloud.privacy1st.de.20240223T1600  2d (sunday, 00:00)  preserve min: latest

Configuration

General

volume <volume-directory|<ulr>: Usually the mount point of a btrfs filesystem mounted with subvolid=5.

subvolume <subvolume-name|<url>: Subvolume to be backed up, relative to <volume-directory>.

snapshot_dir <directory>: Directory in which the btrfs snapshots are created, relative to <volume-directory>. Hast to be created manually!

target <target-directory>|<url>: Target directory where the backup subvolumes are to be created. Multiple target sections are allowed.

url: Accepted formats are:

• ssh://<hostname>[:<port>]/<directory>
• <hostname>:<directory>

hostname: Either a host name, IPv4 or IPv6 address

timestamp_format short|long|long-iso: This becomes relevant for setups with multiple btrbk instances, e.g. many snapshot-only instances (spread around the world), and a fetch-only instance on the backup server. Make sure to run btrbk with the same time zone on every host, e.g. by setting the TZ environment variable (see tzset(3)).

Retention

snapshot_preserve <retention_policy>

snapshot_preserve_min all|latest|<number>{h,d,w,m,y}: Preserve all snapshots for a minimum amount of hours (h), days (d), weeks (w), months (m) or years (y).

target_preserve <retention_policy>

target_preserve_min all|latest|no|<number>{h,d,w,m,y}: Preserve all snapshots for a minimum amount of hours (h), days (d), weeks (w), months (m) or years (y). If set to "no", only the backups following the target_preserve policy are created.

<retention_policy>: [<hourly>h] [<daily>d] [<weekly>w] [<monthly>m] [<yearly>y]. Defines how many backups of each period should be preserved.

SSH and Data Stream Options

ssh_compression yes|no

stream_compress <compress_command>: Compress the btrfs send stream before transferring it from/to remote locations.

<compress_command>: gzip, pigz, bzip2, pbzip2, bzip3, xz, lzo, lz4, zstd.

stream_buffer <size>|no: This can give a speed improvement (measured up to 20%) on both local or remote operations, but also increases system load. Make sure that the mbuffer command is available. Leave this option disabled if your main concern is a stable backup process: while recent versions of mbuffer have proven reliable, it is often desirable to keep things simple rather than adding an additional, multi-threaded process to the command pipe.

rate_limit <rate>|no: Limit the read rate of the btrfs send stream to <rate> bytes per second. Implies the stream_buffer option.

System Options

transaction_syslog <facility>|no: Log all transactions to syslog.

lockfile <file>|no

Example: Local Snapshots and Backup

timestamp_format      long

snapshot_preserve_min 2d
snapshot_preserve     24h 7d 4w 6m

target_preserve_min   no
target_preserve       7d 4w 6m

volume /mnt/12tb1-top-level
  snapshot_dir @snap
  target /mnt/12tb2-top-level/@snap
  subvolume    @home
  subvolume    @

Mount top-level subvolume:

mount -o subvolid=5 /dev/mapper/some-disk /mnt/12tb1-top-level

Manually create snapshot directory:

btrfs subvolume create /mnt/12tb1-top-level/@snap
btrfs subvolume create /mnt/12tb2-top-level/@snap

Snapshots only:

btrbk snapshot

Backup only:

btrbk resume

Create an additional snapshot of @home:

btrbk snapshot @home

Testing on Local Host

sudo mkdir -p /mnt/top-level
ls /mnt/top-level/@ || sudo mount -o subvolid=5 /dev/mapper/luks-ea7099e3-320d-4eb3-a4c3-9910a9af817b /mnt/top-level
ls /mnt/top-level/@snap || sudo btrfs subvolume create /mnt/top-level/@snap
ls /mnt/top-level/@foo || {
  sudo btrfs subvolume create /mnt/top-level/@foo
  sudo touch /mnt/top-level/@foo/bar
}

ls ~/btrbk-usb.img || {
  fallocate -l 1G ~/btrbk-usb.img
  sudo mkfs.btrfs ~/btrbk-usb.img
  # Mount and create subvolume @
}

sudo mkdir -p /mnt/usb-top-level
ls /mnt/usb-top-level/@ || sudo mount -o subvolid=5 ~/btrbk-usb.img /mnt/usb-top-level
ls /mnt/usb-top-level/@snap || sudo btrfs subvolume create /mnt/usb-top-level/@snap
}

cat > ~/btrbk.cfg <<'EOF'
timestamp_format      long

snapshot_preserve_min 2d
snapshot_preserve     24h 7d 4w 6m

target_preserve_min  no
target_preserve      7d 4w 6m

volume /mnt/top-level
  snapshot_dir @snap
  target /mnt/usb-top-level/@snap
  subvolume    @foo
EOF

# Check configuration.
sudo btrbk -c ~/btrbk.cfg config print
sudo btrbk -c ~/btrbk.cfg config list


# Dry run.
sudo btrbk -c ~/btrbk.cfg run -n
# Create first snapshot.
sudo btrbk -c ~/btrbk.cfg run
# Print schedule.
sudo btrbk -c ~/btrbk.cfg run -n -S

sleep 5m
sudo btrbk -c ~/btrbk.cfg run

# List snapshots
sudo btrbk -c ~/btrbk.cfg list

NixOS Config

{ config, pkgs, ... }:
{
  services.btrbk = {
    # Lowest scheduling priority.
    niceness = 19;
    # Set of btrbk instances. The instance named btrbk is the default one.
    instances = {
      # This is the configuration equivalent to
      # README.md section "Testing on Local Host".
      "testing-on-local-host" = {
        # The timer is disabled. The service has to be invoked manually.
        onCalendar = null;
        #onCalendar = "hourly";
        #onCalendar = "*:0/15"; # Every 15min
        # Configuration options for btrbk. Nested attrsets translate to subsections.
        settings = {
          timestamp_format = "long";
          stream_compress = "lz4";

          snapshot_preserve_min = "2d";
          snapshot_preserve = "24h 7d 4w 6m";

          target_preserve_min = "no";
          target_preserve = "7d 4w 6m";

          volume."/mnt/top-level" = {
            snapshot_dir = "@snap";
            target = "/mnt/usb-top-level/@snap";
            subvolume = "@foo";
          };
        };
      };
    };
  };
}

pi3bplus Host Config

A local btrbk instance manages snapshots and local backups.

Another remote backup server periodically pulls snapshots to create remote backups. Here is its config:

cat > ~/btrbk.cfg <<'EOF'
timestamp_format    long
stream_compress     lz4
ssh_identity        /mnt/backup/rootNas_ed25519

# Create backups.
target_preserve_min no
# Daily backups -> @daily cron entry
target_preserve          7d 4w 6m
# Hourly backups -> @hourly cron entry
#target_preserve     24h 7d 4w 6m

# Don't create or delete snapshots on remote server "rootNas".
snapshot_preserve_min  all
snapshot_create        no

volume ssh://rootNas/jc-data
  snapshot_dir /snap
  target       /mnt/backup/snap
  subvolume    arch.p1st.de
  subvolume    blogger.privacy1st.de
  subvolume    changedetection.p1st.de
  subvolume    cloud.privacy1st.de
  subvolume    git.privacy1st.de
  subvolume    mastodon-toot-follower.privacy1st.de
  subvolume    money.p1st.de
  subvolume    music.privacy1st.de
  subvolume    paste.p1st.de
  subvolume    proxy
  subvolume    recipe.privacy1st.de
  subvolume    traggo.privacy1st.de
volume ssh://rootNas/mnt/data/jc-data
  snapshot_dir /mnt/data/snap2
  target       /mnt/backup/snap2
  subvolume    cloud.privacy1st.de
  subvolume    cloud.fykml.de
EOF