Errors with full backup on M1 MacBook Air

[Ricky_LaFleur]

Hi,

I’ve just installed Duplicacy Web Edition 1.5.0 on a MacBook Air M1 running macOS 11.2. I’m unable to back up folders outside my own account’s home folder. The log for the failed backup job has multiple entries that look like this:

2021-02-07 15:10:30.405 WARN LIST_FAILURE Failed to list subdirectory

I’ve enabled full disk access for Duplicacy Web Edition in System Preferences, but this hasn’t resolved the problem. Searching the forum, I’ve found mixed reports about whether the current version of Duplicacy works via Rosetta 2 on an M1 Mac.

The first link below suggests that this isn’t possible without enabling full disk access for /usr/sbin/cron which doesn’t seem like a great idea (although I’m far from a *nix expert).

The second link says that although an M1 native version of Duplicacy isn’t yet available, the Intel binaries should work normally on an M1 Mac.

https://forum.duplicacy.com/t/full-disk-access-on-macos/4123/3
https://forum.duplicacy.com/t/duplicacy-support-for-m1-macs/4627

Can anyone help?

Thanks.

[gchen]

In the first link /usr/sbin/cron was given full disk access to run the CLI. You don’t need to do this for the web GUI once it is in the full disk access list in System Preferences. Maybe you just need to restart the web GUI.

[Ricky_LaFleur]

Thanks for your reply. I’ve tried restarting the web GUI, restarting the computer completely, and removing then re-adding Duplicacy Web Edition from /Applications in Full Disk Access, but I’m still seeing the errors. I’m logged in with a standard (non-admin) user account; is this related?

I’ve just come back to macOS after a hiatus of several years, so I’m a little unfamiliar with the details.

[saspus]

Let’s backtrack a bit. Full disk access prevents an app from being denied access when it attempts to read sensitive data, like documents and photos, and does not know how to request those permissions from the user (@gchen, web ui needs to be able to ask user for access explicitly, without having to grant full disk access. )

The traditional user permissions still apply however.

Hence, the question: does the user under which duplicacy is running has permissions to read other users data?

[brian1]

Hence, the question: does the user under which duplicacy is running has permissions to read other users data?

By default, the answer to this question is going to be no. MacOS accounts (even administrator accounts) don’t have permissions for another users files.

This is a question for @gchen or @saspus, is there a way to force the GUI to ask for sudo access? CrashPlan and BackBlaze accomplish this by running their background processes as root.

I haven’t found a way to make Duplicacy GUI run it’s background tasks as root yet.

There may also be an issue with how the M1 handles backing up System files. SIP protects far more on an M1 than any other Mac – it’s to the point where I don’t believe there are any tools that will make a bootable backup of an M1 Mac right now.

[gchen]

I haven’t tested it myself, but you may be able to set the web GUI to start as root on bootup by moving the file ~/Library/LaunchAgents/duplicacy-web.plist to /Library/LaunchAgents/.

[saspus]

@gchen

LaunchAgents

I think it should go to LaunchDaemons — otherwise it will be launched per user. We want one per system.

[brian1]

Yup, for global start at boot, you’re looking for /Library/LaunchDaemons.

Using Authorization Services where required would be ideal, but certainly more development effort than simply placing a plist to autostart Duplicacy in LaunchDaemons.

[Ricky_LaFleur]

Thanks for your input, everyone. I was going to ask if it would be possible to run this as a daemon, but you’re way ahead of me. I’ll give this a shot now.

[Ricky_LaFleur]

OK, some progress: Duplicacy web GUI is now launching on startup as root. I’ve run into a couple of errors, though.

When trying to set the encryption password:

Failed to save the new password: open /.duplicacy-web/duplicacy.json.f76300f0: no such file or directory

Where I should see the current version of the CLI in Settings:

Failed to save the downloaded CLI executable: open /.duplicacy-web/bin/duplicacy_osx_x64_2.7.2: no such file or directory

I tried copying the contents of ~/.duplicacy-web to /var/root/duplicacy-web, but this hasn’t fixed the problem.

[Ricky_LaFleur]

To be clear, I’m not trying to make a bootable backup here. I’m just backing up the contents of /Users and a few folders from /Library/Application support. These are where I’m seeing the failures.

[gchen]

It looks to me the issue is that the directory /.duplicacy-web doesn’t exist. You can try to manually create this directory in a terminal.

[saspus]

You can create a directory for duplicacy (e.g. /Library/Duplicacy) and define environment variable HOME in the plied file pointing to that directory.

[Christoph]

@Ricky_LaFleur, if you managed to get duplicacy running on your M1 without those issues, it would be great if you could post a quick update, perhaps summarizing what Mac users need to watch out for.

[brady]

I went through this recently. A few tips. I set this up using Launch Control application. Just as easily just edit the plist file.

• Make sure you set your Working Directory in the plist to somewhere you have write access. Big Sur has some read only volumes so something like / won’t work. I personally use /Users/Brady/ so it is easier to access the filters with BBEdit (I don’t edit filters using web UI). I still have access to entire filesystem, this is only where the config/cache is saved for convenience.
• There are two REQUIRED environment variables on the daemon
  • DWE_PASSWORD needs to be your password you set in the config. Running as root does not have access to the system keychain. Without this your backups won’t run because DWE can’t read the config file.
  • HOME This one is what caught me up for a month. In the launch daemon you set a working Directory, but DWE does not use that working directory. I also pass that same directory to DWE.

Here is my working plist. I assume when the app is native for m1, we might have to change the program key. I know that is dependent on upstream support from Go (which I believe was just recently released)

com.duplicacy.plist in /Library/LaunchDaemons:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>EnvironmentVariables</key>
	<dict>
		<key>DWE_PASSWORD</key>
		<string>YOURPASSWORDHERE/string>
		<key>HOME</key>
		<string>/Users/brady</string>
	</dict>
	<key>Label</key>
	<string>com.duplicacy</string>
	<key>Program</key>
	<string>/Applications/Duplicacy Web Edition.app/Contents/MacOS/duplicacy_web_osx_x64</string>
	<key>RunAtLoad</key>
	<true/>
	<key>WorkingDirectory</key>
	<string>/Users/brady/</string>
</dict>
</plist>

After you save the plist file to the correct place, make sure you load it using launchctl. Simply dropping the file will not activate it. Once it is active, it should launch on boot.

launchctl load /Library/LaunchDaemons/com.duplicacy.plist

I tried 4 different times to get this working over the past 6 months. I figured this out last week Hope the plist helps.

[saspus]

@brady

access. Big Sur has some read only volumes so something like / won’t work.

/System/Volumes/Data is where writeable root data volume is mounted

But for all-users backup software I would init it in /Library/Duplicacy, not one of the users homes.

Also it is worth noting that app needs to be granted Full Disk Access permission under Privacy.

[Christoph]

@brady

After you save the plist file to the correct place

So I’m creating that file from scratch? Because I don’t see any com.duplicacy.plist in /Library/LaunchDaemons despite having installed the dmg and launched it …

[brady]

Correct, you will create this from scratch.

Also be aware that if you installed from DMG and already launched it… if you set up backups/schedules/etc already then that config I want to say defaults to something like ~/.duplicacy

If you set working directory to something like /Library/Duplicacy (which is good advice from Saspus you should follow) then it will look like a fresh install again. You could copy your existing config over easily.

I’d just disable the “Start on login” from the menu bar then quit the app. When you load the LaunchDaemon you will see it launch and reappear in the menu bar. The start on login option doesn’t matter because you are doing this yourself with the LaunchDaemon, with elevated permissions.