Oh my Backup - How to setup a "diskless" backup server using "Alpine Linux" and "MinIO"

May 16, 2020·
Dennis
Dennis
· 20 min read
Image credit: DALL-E

This blog post dives into setting up the Open Source “S3” compatible object storage server “MinIO” as backend for “restic” backups. The main focus of this setup is to have a stable storage backend with nearly zero maintenance – ok more or less zero maintenance.

A bit of my journey

I’m used to have a backup server based on rsyncd for years now. Once in a while I run rsync with numerous of options on my local machine to backup my files. What bothers me a lot with this kind of backup : Each time I run the backup the old version is overwritten.

Since I refactored my infrastructure including virtualization and networking over the last few months, I decided to do a make over of my backup setup as well.

“Vision” for setup

The software projects used for my backup shall be …

  • … open source
  • … run containerized on Linux
  • … stable, but created with modern programming languages and techniques
  • … in a good shape, means well maintained
  • … released regularly - at least once a year

“Approaching” the solution

After evaluating different backup solutions, I decided to go with “restic” albeit it has some limitations (issue #2547) and the speed of development has declined over the last few months. There are some fine people working on the technical limitations at least (PR #2340). Besides “restic” I considered the following tools:

As hardware platform I first chose an existing Raspberry PI 3B+ and hard disk (HDD) with USB-interface - not a wise choice. In the next iteration I changed this to an also existing Intel Atom-based Mini-PC with two SATA-HDDs. Maybe I upgrade this again in the far future as the new “HPE Gen10 Plus” barebone looks promising.

“Alpine Linux” was my first choice as operating system. It has an installation mode called “diskless”: You can have an in-RAM-operating system using a CDROM or a USB-stick which reduces unnecessary disk spin-ups and the wear out of the USB-stick. Most of its configuration is saved in an overlay file. This file is all you need to make a backup of your diskless server and can also be used to setup another server with the same configuration.

“restic” supports different storage backends. An interesting one is the S3 backend. There are cloud providers like “Wasabi” and Open Source software projects like “MinIO” which implement an S3 compatible API. The cloud provider(s) can be used for off site backups and “MinIO” for local high volume backups.

Note: Using “restic” with Wasabi will NOT be part of this article.

For the local backup I wanted to have duplication of my data in the backup server. I don’t like to use a software-RAID for terabytes of data with an Atom-CPU. “MinIO” features a cluster mode which syncs your data over multiple servers and/or disks.

I tried

  • K3OS
  • Kubernetes
  • Some other file sharing platforms

Requisites for readers

This article is written for people with a basic understanding in “Linux” operating systems. Writing this article I assume the reader does not have “Alpine Linux” running on her/his workstation. But I assume that you already built your “ODROID H2+” according to the official YouTube Video. I added tags to clarify which commands have to be executed on which of your systems:

  • Workstation: Your local desktop computer or laptop which you use to setup the server
  • Server: The server you’re going to install. It is controlled directly via monitor and keyboard
  • Server (SSH): The server you’re going to install. It is controlled over ssh from your workstation

After this short introduction and explanation, let us switch to the fun part.

Server setup

Prepare installation of the operating system

  • WorkstationDownload the extended ISO image and burn it onto a CD

    Download the extended ISO image from the “Alpine Linux” download site and burn it onto a CD. I use Brasero to burn images onto CDs. At the time of writing “3.12” is the current version of “Alpine Linux”.

  • ServerConnect the “ODROID H2+” to a monitor and keyboard

  • ServerConnect the “ODROID H2+” to your local network by cable

  • ServerBoot the OS from CD

    Make sure your “ODROID H2+” server starts up in UEFI mode. If you’re not sure, please check the documentation about how to configure the hardware start up routine correctly. When the installation OS is booted, please login with root. There’s no password required for the login.

  • ServerRun the “Alpine Linux” installer

    Upon having set up a working network connection you run the installer for the first time. This will generate some files including the repositories file for apk and start some very basic services like an SSH server.

    setup-alpine

    Enter the following into the prompt of the installer.

    Keyboard layout: none (or whatever suits your preference)
Hostname: localhost
Network interface: eth0
IP address: dhcp
Manual network configuration: no
Password: test123
Again password: test123
Time zone: UTC
HTTP proxy: none
NTP client: chrony
Mirror: 1 (or whatever suits your preference)
SSH server: openssh
Setup disk: none
Store configs: none
Apk cache directory: none

    You can find more details in the “Alpine Linux” wiki: Creating a bootable usb stick, Installation of “Alpine Linux”, Saving changes in an diskless setup.

  • ServerFix configuration for OpenSSH server

    IMPORTANT: Make sure you install your server in a secured network.

    The following configuration can be considered as insecure, but is sufficient for a setup of the server in a secured environment. This will activate the login for root via SSH. As we configured a rather weak password, make sure to not use such a configuration for a server in a public network.

    vi /etc/ssh/sshd_config

    # /etc/ssh/sshd_config
PermitRootLogin yes

  • ServerRestart the SSH daemon

    To activate the new configuration, please restart the SSH daemon.

    rc-service sshd restart

  • ServerShow the ip address of your server

    To make operations like copy and paste a bit easier, I usally try to use SSH to connect remotely to a server. Before you can proceed, you need to get the IP address of your server. Nowadays I use ip for this, but ifconfig will work just as fine as well. Make sure you run this command on your server, NOT on your local computer.

    ip address
# => [... ]
# => 2: eth0
# => inet 192.168.x.x
# => [... ]

  • WorkstationConnect to your server via SSH from your local computer

    Please start a new shell on your local system and run the following command.

    ssh root@<IP address of your server>

  • Server (SSH)Update repository meta data

    apk requires information about packages it can install. To fetch this kind of information, run the following command.

    apk update
# => fetch http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/v3.12/main/x86_64/APKINDEX.tar.gz
# => fetch https://alpine-repository.fedux.org/testing/x86_64/APKINDEX.tar.gz
# => fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/main/x86_64/APKINDEX.tar.gz
# => 3.12.0 [/media/cdrom/apks]
# => v3.12.0-245-gcd32bb7c9a [http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/v3.12/main]
# => v3.12.0-245-gcd32bb7c9a [http://dl-cdn.alpinelinux.org/alpine/v3.12/main]
# => OK: 4849 distinct packages available

    There’s a shortcut for this as well: Run apk add -u <package> to update the package information and install the package in one go.

  • Server (SSH)Install lsblk to gather information about your storage setup

    apk add lsblk
# or
apk add -u lsblk

Modified setup storage for operating system

In this guide we are going to use a USB-stick to store the operating system.

  • Server (SSH)Find hard disk (HDD) to install “Alpine Linux”

    We need to find out the name of the hard disk where we can install “Alpine Linux”. In this case it’s an old SSD, but as mentioned before any HDD, SSD or USB-stick is fine as long it’s big enough.

    lsblk
# => NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
# => [...]
# => sdb      8:16   1 57.3G  0 disk
# => [...]

  • Server (SSH)Add gdisk to setup “GPT” disks

    At this point you need to install gdisk to setup “GPT” disks easily.

    apk add gptfdisk
# => (1/4) Installing libgcc (9.3.0-r2)
# => (2/4) Installing libstdc++ (9.3.0-r2)
# => (3/4) Installing libuuid (2.35.2-r0)
# => (4/4) Installing gptfdisk (1.0.5-r0)

  • Server (SSH)Create partitions on disks

    Run gdisk to create partitions. I don’t want to go into too much detail, now. There are plenty of guides for this out there on the internet. IMPORTANT: Make sure to choose “EFI” for the partition type.

    gdisk /dev/sdb

    # => gdisk -l /dev/sdb
# => [...]
# => Number  Start (sector)    End (sector)  Size       Code  Name
# => 1            2048       120176606   57.3 GiB    EF00  EFI system partition

  • Server (SSH)Load kernel module for “VFAT” filesystem

    You need to load the vfat kernel module to be able to format the partition.

    modprobe vfat

  • Server (SSH)Format partition for operating system

    mkfs.vfat /dev/sdb1

Install operating system

  • Server (SSH)Install “Alpine Linux” to a disk

    This command will copy all necessary files for a full “Alpine Linux” diskless installation. You might see a different directory for your source media and destination device.

    IMPORTANT: /dev/sdb1 must not be mounted before you run this command.

    setup-bootable -v /media/cdrom/ /dev/sdb1
# => Using /dev/sdb1 as target (mounted on /media/sdb1)
# => Installing /dev/sdb1 to alpine-extended-3.12.0 200529
# => Copying /media/cdrom/boot to /media/sdb1/.new/
# => Copying /media/cdrom/efi to /media/sdb1/.new/
# => Copying /media/cdrom/apks to /media/sdb1/.new/
# => Copying /media/cdrom/.alpine-release to /media/sdb1/.new/
# => Found boot/syslinux/syslinux.cfg
# => Flushing cache...
# => Replacing existing files...
# => Making /dev/sdb1 bootable...

  • Server (SSH)Restart the server to boot into the installed OS

    Restart the server with “Alpine Linux” installed. From now on you’re modifying your setup.

    reboot

  • ServerDisconnect the device with installation media

    Make sure you disconnect your CD drive and/or change the hardware start up routine to boot from /dev/sdb.

Setup operating system

  • ServerLogin with root and no password to your server

    At this stage of work the installation on your disk is the same as on the installation CD, but you’re going to change this in the following steps.

  • ServerRun the “Alpine Linux” installer

    First we need to run the installer again.

    setup-alpine

    This time you enter all the information you want the installer to persist. You might need to enter something different from the list given below depending on your requirements and local infrastructure.

    keyboard layout: us (or whatever suits your preference)
variant: us-altgr-intl
hostname: <Your Hostname>
network interface: eth0
IP address: dhcp
Manual network configuration: no
password: <Your root Password>
again password: <Your root Password>
time zone: Europe/Berlin (or whatever suits your preference)
HTTP proxy: none
NTP client: chrony
Mirror: 1 (or whatever suits your preference)
SSH server: openssh
Setup disk: none
Store configs: usb
Apk cache directory: /media/usb/cache

  • ServerFix configuration for OpenSSH server temporary

    We need to change the setup of the OpenSSH server again. It’s the same procedure like before: So, make sure, that you install your server in a secured network.

    vi /etc/ssh/sshd_config

    # /etc/ssh/sshd_config
PermitRootLogin yes

  • ServerRestart sshd

    After you have changed the configuration, please restart “sshd”.

    rc-service sshd restart

  • ServerShow the ip address of your server

    Please gather the ip address of your server again. This might be a different one than before.

    ip address
# => [... ]
# => 2: eth0
# => inet 192.168.x.x
# => [... ]

  • Workstation(optional) Copy the SSH public key from your local system to your new server

    If you prefer to use public/private keys with SSH, please open a new terminal on your local system and copy your local public SSH key to the server.

    ssh-copy-id root@<IP address>

  • Server(optional) Commit changes for passwordless access for root permanently

    This steps is only required, if you ran ssh-copy-id earlier. Files in /root are not part of the regular “diskless mode” save list. These commands will save root’s authorized_keys file to the overlay file.

    lbu add /root/.ssh/authorized_keys

  • WorkstationConnect to your server via SSH from your local system

    Please connect to the server via ssh from your local system. Depending on your SSH setup, you may need to unlock your SSH key first or login with username and password.

    ssh root@<IP address>

  • Server (SSH)Fix configuration for OpenSSH server permanently

    vi /etc/ssh/sshd_config

    Remove the following line.

    # /etc/ssh/sshd_config
PermitRootLogin yes

    Add one of the following lines instead. Option 1) will allow SSH access for root via SSH public/private keys, but not with passwords and option 2) will prevent login with the root user at all.

    # /etc/ssh/sshd_config

# 1)
# public / private key only
PermitRootLogin prohibit-password

# or

# 2)
# no root login at all
PermitRootLogin no

  • Server (SSH)Restart sshd

    Again, restart sshd after you saved the configuration file. Do not reboot at this point!

    rc-service sshd restart

  • Server (SSH)(optional) Commit changes for remote access permanently

    lbu commit -d

    You shall see an overlay file on your root disk which contains all changed files.

    ls /media/usb/*.tar.gz
# => /media/usb/<Your Hostname>.apkovl.tar.gz

  • WorkstationConnect to your server via SSH from your local system via a different session

    In order to verify that the sshd is working correctly, connect to the server using a second connection. Please create a new shell on your workstation and run this command. You can close this connection after it was built up successfully.

    ssh root@<IP address>

  • Server (SSH)Make sure all edge repositories are disabled

    The edge repositories contain bleeding edge versions of known packages. Using these might break your setup. This is why you should disable them for your own sake. Please activate the community repository.

    vi /etc/apk/repositories

    /media/usb/apks
http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/v3.12/main
http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/v3.12/community
#http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/edge/main
#http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/edge/community
@edge-testing http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/edge/testing

  • Server (SSH)Add repository for “Realtek” driver

    I setup a repository for “Alpine Linux” where I distribute a compiled version of the “Realtek” driver. You need to add this repository to make the builtin network interfaces work.

    vi /etc/apk/repositories

    # /etc/apk/repositories
https://alpine-repository.fedux.org/testing

  • Server (SSH)Add public part of the signing key for the repository

    apk verifies the signature of each package. To make the verification succeed, please add the following file with the given name.

    vi /etc/apk/keys/build.rsa.pub

    # /etc/apk/keys/build.rsa.pub
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvXotU5i9WV4WaDkj0gt4
+rzGi8Hv3BeYApgYuJc1/Vwj8uDxKXMyq0uVCqx5xHsvQGNzLDLFvn2uB6LO3PmL
hWrXpf0DO+rYbBet6+wGa3v+hdfSfUsHIb6GE8flRILKxxshDODgskm90H2sRu6a
T6oJowF8mb9euqSQ7e1wSgJi78/gDUve27koD2ia/oLwkDrcdY2ZGK6uWRERkGsz
XHfoK59kpGPCeswt5TESIkRJ/7z04aUlyt71E6Ej3UeQJtqfEHn5wAxIPxbN/NqQ
JjtejsxOjLj1DxsF+8aqYMSrkb/s6ccVQbtPoBvBW5XBGIhgv1jc36BXHR/qoxS4
3QIDAQAB
-----END PUBLIC KEY-----

  • Server (SSH)Update repository meta data

    apk update
# => fetch http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/v3.12/main/x86_64/APKINDEX.tar.gz
# => fetch http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/v3.12/community/x86_64/APKINDEX.tar.gz
# => fetch http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/edge/testing/x86_64/APKINDEX.tar.gz
# => fetch https://alpine-repository.fedux.org/testing/x86_64/APKINDEX.tar.gz
# => 3.12.0 [/media/usb/apks]
# => v3.12.1-24-g9d31f224d7 [http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/v3.12/main]
# => v3.12.1-31-ga4a2788986 [http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/v3.12/community]
# => v20200917-3737-g03feac5d7e [http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/edge/testing]
# => testing  [https://alpine-repository.fedux.org/testing]
# => OK: 17098 distinct packages available

  • Server (SSH)Commit configuration made so far

    lbu commit -d

Setup storage for root disk

  • Server (SSH)Add mkinitfs to update the kernel and rebuild the “modloop”-image

    apk add mkinitfs
# => (1/12) Installing lddtree (1.26-r2)
# => (2/12) Installing xz-libs (5.2.5-r0)
# => (3/12) Installing kmod (27-r0)
# => (4/12) Installing kmod-openrc (27-r0)
# => (5/12) Installing libblkid (2.35.2-r0)
# => (6/12) Installing argon2-libs (20190702-r1)
# => (7/12) Installing device-mapper-libs (2.02.186-r1)
# => (8/12) Installing json-c (0.14-r1)
# => (9/12) Installing libuuid (2.35.2-r0)
# => (10/12) Installing cryptsetup-libs (2.3.2-r0)
# => (11/12) Installing kmod-libs (27-r0)
# => (12/12) Installing mkinitfs (3.4.5-r3)
# => Executing mkinitfs-3.4.5-r3.post-install
# => Executing busybox-1.31.1-r16.trigger
# => OK: 46 MiB in 47 packages

  • Server (SSH)Add blkid to add entries to the /etc/fstab-file

    apk add blkid
# => (1/1) Installing blkid (2.35.2-r0)
# => Executing busybox-1.31.1-r16.trigger
# => OK: 46 MiB in 48 packages

  • Server (SSH)Commit configuration

    The next steps might break your setup. That is why this is a good time to commit your configuration now.

    lbu commit -d

  • Server (SSH)Make sure your root disk is writable

    mount -o remount,rw /media/usb/

  • Server (SSH)Update kernel and install required packages

    IMPORTANT: Make sure you’ve got plenty of RAM for this step – at least 8 GiB. If there’s not enough RAM you end up with a broken “modloop”-image.

    As we would like to use “LVM” and “XFS” we also need to install the required packages to mount the filesystems during boot. The package linux-realtek-r8125-lts contains the driver for the “Realtek” ethernet chip used by the “ODROID H2+”. It is served from my own “Alpine Linux” repository mentioned earlier. As of the time writing the kernel module for the “Realtek” ethernet chip was compiled against “Linux 5.4.43-1-lts”.

    IMPORTANT: Add all required features as value for the -F-flag. The default configuration of mkinitfs is only used if the -F-flag is omitted. The default value for this flag is ata base cdrom ext4 keymap kms mmc raid scsi usb virtio and can be found in /etc/mkinitfs/mkinitfs.conf. If you leave out some of the basic features you might end up with a broken “modloop” image.

    update-kernel -F "ata base cdrom lvm xfs keymap kms scsi usb" -p xfsprogs -p lvm2 -p linux-realtek-r8125-lts /media/usb/boot/
# => Warning: extra firmware "" not found!
# => Parallel mksquashfs: Using 4 processors
# => Creating 4.0 filesystem on /tmp/update-kernel.BGDffp/boot/modloop-lts, block size 131072.
# => [==========================================================================================================================================================\] 6459/6459 100%
# =>
# => Exportable Squashfs 4.0 filesystem, xz compressed, data block size 131072
# =>         compressed data, compressed metadata, compressed fragments,
# =>         compressed xattrs, compressed ids
# =>         duplicates are removed
# => Filesystem size 91967.62 Kbytes (89.81 Mbytes)
# =>         23.93% of uncompressed filesystem size (384375.24 Kbytes)
# => Inode table size 44566 bytes (43.52 Kbytes)
# =>         24.37% of uncompressed inode table size (182890 bytes)
# => Directory table size 50190 bytes (49.01 Kbytes)
# =>         41.93% of uncompressed directory table size (119703 bytes)
# => Number of duplicate files found 258
# => Number of inodes 5401
# => Number of files 4530
# => Number of fragments 1004
# => Number of symbolic links  0
# => Number of device nodes 0
# => Number of fifo nodes 0
# => Number of socket nodes 0
# => Number of directories 871
# => Number of ids (unique uids + gids) 3
# => Number of uids 2
# =>         root (0)
# =>         unknown (2291)
# => Number of gids 2
# =>         root (0)
# =>         unknown (1022)

  • Server (SSH)(optional) Repair broken “modloop”-image

    If anything went wrong in the previous step, boot “Alpine Linux” from CD and run the following commands. You might need to change the device names depending on your setup.

    mount -o remount,rw /media/usb/
update-kernel /media/usb/boot/

  • Server (SSH)Add packages to manage “LVM” and “XFS” on your booted server

    This step adds tools to your booted server, so that you can manage your storage devices etc. after the server was booted.

    apk add lvm2 xfsprogs
# => (1/6) Installing libaio (0.3.112-r1)
# => (2/6) Installing device-mapper-event-libs (2.02.186-r1)
# => (3/6) Installing lvm2-libs (2.02.186-r1)
# => (4/6) Installing lvm2 (2.02.186-r1)
# => (5/6) Installing lvm2-openrc (2.02.186-r1)
# => (6/6) Installing xfsprogs (5.6.0-r1)
# => Executing busybox-1.31.1-r16.trigger
# => OK: 52 MiB in 54 packages

    For “LVM” to work correctly, you need to enable the “LVM” daemon.

    rc-update add lvm
# => * service lvm added to runlevel default

  • Server (SSH)Install lsblk to gather information about your storage devices

    You need to install lsblk again. This time this is the persistent installation.

    apk add lsblk
# => (1/3) Installing libmount (2.35.2-r0)
# => (2/3) Installing libsmartcols (2.35.2-r0)
# => (3/3) Installing lsblk (2.35.2-r0)
# => Executing busybox-1.31.1-r16.trigger
# => OK: 53 MiB in 57 packages

  • Server (SSH)Add gdisk to setup “GPT” disks

    You also need to install gdisk again to setup “GPT” disks easily.

    apk add gptfdisk
# => (1/4) Installing libgcc (9.3.0-r2)
# => (2/4) Installing libstdc++ (9.3.0-r2)
# => (3/4) Installing libuuid (2.35.2-r0)
# => (4/4) Installing gptfdisk (1.0.5-r0)

  • Server (SSH)Find disk to setup LVM

    You use lsblk to find out the device name of your storage device.

    lsblk
# => NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
# => [...]
# => sda      8:0    0  3.7T  0 disk
# => [...]

  • Server (SSH)Prepare data disks

    Run gdisk to create partitions. As before, I won’t go into too much detail. Best follow other guides for this on the internet. IMPORTANT: Make sure to choose “Linux LVM” for the partition type.

    gdisk /dev/sda

    gdisk -l /dev/sda
# => [...]
# => Number  Start (sector)    End (sector)  Size       Code  Name
# =>  1            2048      7814037134   3.6 TiB     8E00  Linux LVM

  • Server (SSH)Setup LVM

    Step one: make /dev/sda1 a physical volume.

    pvcreate /dev/sda1
# => Physical volume "/dev/sda1" successfully created.

    Step two: create a volume group using that physical volume.

    vgcreate vg_data /dev/sda1
# => Volume group "vg_data" successfully created

    Step three: the values for the storage size depend on your setup. I bought a 2 TB SSD and decided to create a disk with 32 GiB where my container images shall be stored.

    lvcreate -L +32G vg_data -n lv_images
# => Logical volume "lv_images" created.

    Step four: the rest of the disk is used by the data volume.

    lvcreate -l 100%FREE vg_data -n lv_data
# => Logical volume "lv_data" created.

    Last step: activate the created volumes.

    vgchange -ay
# => 2 logical volume(s) in volume group "vg_data" now active

    This might look similar to your setup so far.

    lsblk
# => NAME                  MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
# => loop0                   7:0    0   91M  1 loop /.modloop
# => sda                     8:0    0  3.7T  0 disk
# => └─sda1                  8:1    0  3.7T  0 part
# =>   ├─vg_data-lv_images 253:0    0   32G  0 lvm
# =>   └─vg_data-lv_data   253:1    0  3.6T  0 lvm
# => sdb                     8:16   1 57.3G  0 disk
# => └─sdb1                  8:17   1 57.3G  0 part /media/usb

  • Server (SSH)Create filesystem on the logical volumes

    At the beginning of this chapter you installed programs in order to install and manage “XFS” filesystems. Now you use one of these programs to setup “XFS” disks.

    First we create the filesystem for the container images.

    mkfs.xfs /dev/mapper/vg_data-lv_images
# => meta-data=/dev/mapper/vg_data-lv_images isize=512    agcount=4, agsize=2097152 blks
# =>          =                       sectsz=4096  attr=2, projid32bit=1
# =>          =                       crc=1        finobt=1, sparse=1, rmapbt=0
# =>          =                       reflink=1
# => data     =                       bsize=4096   blocks=8388608, imaxpct=25
# =>          =                       sunit=0      swidth=0 blks
# => naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
# => log      =internal log           bsize=4096   blocks=4096, version=2
# =>          =                       sectsz=4096  sunit=1 blks, lazy-count=1
# => realtime =none                   extsz=4096   blocks=0, rtextents=0

    Next we create the filesystem for the data.

    mkfs.xfs /dev/mapper/vg_data-lv_data
# => meta-data=/dev/mapper/vg_data-lv_data isize=512    agcount=4, agsize=242091264 blks
# =>          =                       sectsz=4096  attr=2, projid32bit=1
# =>          =                       crc=1        finobt=1, sparse=1, rmapbt=0
# =>          =                       reflink=1
# => data     =                       bsize=4096   blocks=968365056, imaxpct=5
# =>          =                       sunit=0      swidth=0 blks
# => naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
# => log      =internal log           bsize=4096   blocks=472834, version=2
# =>          =                       sectsz=4096  sunit=1 blks, lazy-count=1
# => realtime =none                   extsz=4096   blocks=0, rtextents=0

  • Server (SSH)Add new disks to /etc/fstab

vi /etc/fstab


**Hint:** Using UUIDs does not work and fails during boot.

~~~ini
# /etc/fstab
/dev/mapper/vg_data-lv_images /storage/images xfs defaults 0 2
/dev/mapper/vg_data-lv_data /storage/data xfs defaults 0 2

  • Server (SSH)Create mount points

    mkdir -p /storage/images
mkdir -p /storage/data

  • Server (SSH)Make mount points persistent across reboots

    touch /storage/.keep /storage/images/.keep /storage/data/.keep

  • Server (SSH)Add directory to save list

    To make sure your mount points are part of the overlay image, please run the following commands.

    lbu add /storage/

  • Server (SSH)Commit changes to disk

    After that, please commit all changes to disk.

    lbu commit -d

  • Server (SSH)Mount disks to verify setup

    Now you can use the mount command to verify all disks can be mounted.

    mount -a

    Your setup should look similar.

    mount | grep storage
# => /dev/mapper/vg_data-lv_images on /storage/images type xfs (rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
# => /dev/mapper/vg_data-lv_data on /storage/data type xfs (rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)

  • Server (SSH)Reboot

    At the very end, please reboot your computer to see if your setup works correctly. You should see no errors on mounting the disks in the start up logs. After reboot you should see a login prompt.

    reboot

  • Server (SSH)Install S.M.A.R.T. monitoring for your hard disks

    apk add smartmontools
# => (1/2) Installing smartmontools (7.1-r2)
# => (2/2) Installing smartmontools-openrc (7.1-r2)
# => Executing busybox-1.31.1-r16.trigger
# => OK: 28 MiB in 59 packages

    rc-update add smartd
# =>  * service smartd added to runlevel default

    rc-service smartd start
# =>  * Caching service dependencies ... [ ok ]
# =>  * Starting smartd ...

  • Server (SSH)Install logrotate

    apk add logrotate

    vi /etc/logrotate.conf

    # /etc/logrotate.conf

# [...]
daily
# [...]
rotate 7
# [...]

  • Server (SSH)Install minio

    Please change <myUser> and <MySecretKey>.

    apk add minio@edge-testing minio-client@edge-testing

    rc-update add minio

    vi /etc/minio.conf

    mkdir -p /etc/minio/certs.d

    # defaults to storing data in /srv/minio
# but you can run multiple instances, e.g minio.home in /srv/minio.home
MINIO_VOLUMES="https://host{1...2}.in.example.org/dev/mapper/vg_data-lv_data/{1...4}"

export MINIO_ACCESS_KEY=ssss
export MINIO_SECRET_KEY=xxx

# recommended for production setups
MINIO_OPTS='--quiet --anonymous --certs-dir /etc/minio/certs.d'

# optional additional settings
address="`hostname`.in.example.org:9000"
#export MINIO_DOMAIN=backup.in.example.org

    rc-service minio start

  • S3 policy

/etc/machines/backup-service/ /etc/machines/backup-service/docker-compose.yml /etc/machines/backup-service/ssl /etc/machines/backup-service/ssl/traefik /etc/machines/backup-service/ssl/traefik/server.key /etc/machines/backup-service/ssl/traefik/server.crt /etc/machines/backup-service/config /etc/machines/backup-service/config/minio /etc/machines/backup-service/config/minio/all-buckets.json /etc/machines/backup-service/config/traefik /etc/machines/backup-service/config/traefik/traefik.toml /etc/machines/backup-service/config/traefik/conf.d /etc/machines/backup-service/config/traefik/conf.d/certs.toml /etc/machines/backup-service/config/traefik/conf.d/.keep /etc/machines/backup-service/config/traefik/ssl /etc/machines/backup-service/env /etc/machines/backup-service/env/minio

/usr/local/bin/*

mc admin policy add default all-backup-buckets /etc/machines/backup-service/config/minio/all-buckets.json
# => Added policy `all-backup-buckets` successfully.

mc admin user add default  backup
Enter Secret Key:
Added user `backup` successfully

mc admin policy set default all-backup-buckets user=backup
Policy all-backup-buckets is set on user `backup`

mc admin trace default
# => 19:14:13.925 [404 Not Found] s3.GetBucketLocation backup.example.com/backup-com-example-host1/?location=  10.16.0.66        21.288ms     ↑ 190 B ↓ 594 B

https://docs.min.io/docs/minio-multi-user-quickstart-guide.html https://docs.traefik.io/https/tls/ https://docs.traefik.io/operations/api/

Performance analysis

apk add iptraf-ng sysstat

xlpbkp001:~# cat /etc/local.d/01-sync-apk-cache.stop
#!/bin/sh

# verify the local cache on shutdown
/sbin/apk cache -v sync

# We should always return 0
return 0

/etc/periodic/daily/sync-apk-cache.sh

#!/bin/sh

/sbin/apk cache -v sync
exit 0

/etc/periodic/daily/heal-minio.sh

#/bin/sh

/usr/local/bin/mc admin heal --recursive default
exit 0
S3 Minio Diskless Alpine-Linux Linux Xfs Backup Blob-Storage Object-Storage Restic
Dennis
Authors
Dennis
Technology Enthusiast
My interests include IT automation, continous delivery, data visualization, software development and photography

Related