Duplicacy aborts attempting to backup streamed files from Google Drive

Support
#1

I’m fairly confident I know the answer here, but wanted to get confirmation.

I’ve recently shifted most of my work folders over to Google Drive and switched on streaming. Duplicacy now aborts with a “Failed to read 0 bytes” error once it attempts to read a file on the virtual Google drive.

I’m assuming this is caused by the fact that the files are being streamed from Google’s cloud storage and Duplicacy is unable read the actual file contents. Is that correct? And is there any way to address the issue apart from turning off streaming and mirroring all the files to my local hard drive?

As an aside, I’ve started playing around with Rclone and have set up a sync job to copy my Google Drive folders to a local Linux server. The only wrinkle (and apparently this is a common workaround) is that Rclone converts all of the native Google Docs and Sheets files to Microsoft Word and Excel in order to store the files locally. Still scratching my head over why Google stores these objects in a way that doesn’t allow for them to be copied natively off their cloud service. Well… cynically, I suspect I know the answer to that. :face_with_raised_eyebrow:

Thanks in advance for any feedback / guidance.

#2

On what OS? Recently google drive broke streaming completely on macOS — they switched from osxfuse to smb and stability plummeted.

I’ve been using rclone mount for some time for streaming from google and it worked correctly with duplicacy and other tools. You would want to add arguments for stability, and I wasn’t using google docs in any way so that wasn’t a problem for me.

This approach also supports sparse files — I.e. if you only read 5 bytes from the middle of 1TB file — only few blocks from the middle of the file will be actually downloaded.

Also this shows all files in the virtual filesystem, so if duplicacy decides that the file didn’t change and won’t read the content — the file will be eventually evicted from the cache and only live in the cloud but still present in each backup set, unlike with the selective sync

This is, however, while functional, a deadend/stopgap solution, as both macOS and windows move away from /discourage use of kernel extensions and instead provide FileProvider API for this specific usecase — support of cloud file storage . Why did google not move to that — I too cynically suspect the reason.

IIRC only Box.com client and StrongSync from ExpanDrive take advantage on FileProvider API today. This api however as of today does not support sparse files — so this won’t work well for movie streaming; but for everything else it’s a superior solution — files are downloaded locally and the whole thing is natively supported by the OS services.

1 Like
#3

Bingo - it’s Duplicacy on top of MacOS Monterey.

And thanks for the background / additional info on Rclone mounts. I’ll spend some time playing with that.

#4

I also use rclone mount pointing to google drive and Duplicacy works perfectly.