Help me understand this log output

Support
#1

When backing up my Debian /home directory, the log shows two types of responses that I’m struggling to understand.

Example 1

2022-11-22 02:02:05.080 WARN LIST_SKIP Skipped non-regular file docker_data/overlay2/005e6de6a3d7767ae6e2b4d72ae3ca457dfcd4872ee23935ee083f3caa6e802a/diff/etc/nginx/conf.d/dev.conf

What does WARN LIST_SKIP Skipped non-regular file docker_ tell me. And what is a “non-regular file”

Example 2

2022-11-22 02:02:20.913 WARN SKIP_FILE File docker_data/overlay2/3244e51287046856abdc73c0ea3ba8c42c852240d9565d5a918146d7ba5737c8/diff/usr/bin/msgfilter cannot be opened

A similar message but with cannot be opened. What does that tell me?

This backup runs under root’s crontab.

To clarify the origins of this folder… it is generated by Docker. I relocated it to my /home partition, as it would fill the / partition (15-20 GB, typically)

Thanks for your assistance.

#2

You would not be able to backup files opened in exclusive mode, memory mapped with specific attributes, or otherwise locked. Ideally, you would want to take a snapshot of your filesystem and then mount and backup that snapshot.

More importantly, docker service data must be excluded from backup. Containers are disposable, and should not be retained. Data containers operate on is residing on the host and is mounted into containers; backup those folders instead. This may still often require taking a snapshot of the hosts filesystem depending on the nature of the data, or stopping the container for the duration of the backup if the filesystem does not support snapshots, or if the data is inconsistent in open state (e.g. databases)

#3

You are right. I thought about cycling docker in the backup script…

sudo systemctl stop docker
Duplicacy backup
sudo systemctl start docker

…but I didn’t want to disrupt container activity. It seems I’ll need to figure out Duplicacy’s “exclude” feature to skip this folder, or otherwise, locate it under an alternate /home user (I guess “docker” might be that user).

Thank you!

#4

I agree, restarting containers for backup is a bit brutal.

What is the filesystem on that host? is that Synology with btrfs?

Regardless, I wouldl move docker stuff out of /home, and instead backup the data that maps into container from the host. If that does not work – e.g. if a container uses locked database – then you would need to figure out how to deal with that: even filesystem snapshot may not be enough – you would not want to backup databased in inconsistent state; in this case maybe exporting data from the database and backing up the export is often performed.

An alternative to learning the Duplicacy filtering configuration is to create a separate folder that you will backup with duplicacy and place symlinks to other folders and files you actually want to backup. Duplicacy follows first level symlinks.

#5

Debian 11, ext4 formatted.

I’m stuck using the home partition, as it has the only space for Docker’s overlay2 folder. So I may add user docker and use /home/docker/data for the overlay2 folder, instead of my /home path.

EDIT1: The backup exclude option seemed to work. Only 1 file could not be opened (before it was 531 files).

#6

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.