Support for dataless files

Today most of the Drive services adopted the modern way to handle cloud storage on macOS – via the FileProvider API.

For example, Box.com’s native client moved from using custom filesystem driver to FileProvider few years ago, and stability soared. GoogleDrive departed from osxfuse, took the detour of using local SMB server, but as of today, they too adopted file provider. Dropbox did the same. iCloud itself obviously complies to the same protocol.

Now, today for most people data lives in the cloud. Yours truly has a computer with a 512GB local storage and about 4TB worth of data in iCloud and Google Drive combined. Up until recently the only way to backup that data was via mounting the storage locally using various FUSE based solutions, or syncing the whole thing. Fuse is dead for all intents and purposes on macOS – I"m not going to disable all security just to have a privilege of destabilizing my OS with some buggy filesystem implementation. Kernel extensions for mundane things like file sync are a thing of the past.

With the adoption of File providerAPI this is no longer required either. Instead, Duplicacy could materialize the file, if the metadata changed, back it up, and then optionally, ephemerize it back, or just leave it to the OS to manage.

If this justification for the feature is not convincing enough – here is another one: competition already does it:

  • Added a backup plan option so you can specify whether to report, ignore, or materialize dataless (“cloud-only”) files such as Dropbox and Google Drive files.
4 Likes

Bumping this to point out that another competitor is likely to provide this feature quite soon: I’ve been using a dev version of restic that offers support for dataless cloud files, and it works brilliantly.

This enables me to back up to cloud storage on the laptop I share with my wife (which requires us to use the “optimize” feature in iCloud Drive). Looking at the code change, it didn’t seem that complicated to implement in restic.

As @saspus points out here, the move to FileProvider API, along with the ubiquity of synced cloud storage, means that this will be an increasingly helpful and user-requested feature. If the next stable version of restic does indeed merge this branch, it will make it hard for me not to choose restic over duplicacy.

1 Like

I’m in the same boat, 3TB of data in iCloud, and 512GB shared MacBook Air. I keep using Arq. Waiting for duplicacy to catch up. This is a basic requirement by now.

(The Photos Library still requires separate handling, but I’m just exposing last year worth of media every year onto a NAS.)