Restore from Shared Google Drive Folder fails because server closes the connection

Support
#1

I am using Duplicacy with Shared Google Drive Folder.
I tried several times to restore a 300 GB folder but it never went well: it seems that the server closes the connection after downloading a couple of chunks.

The restore command encountered an error:
Failed to download the chunk eb38c770ebfeafec160248a05449bd38c3842153d8520500c58e472a9f07aab0: http2: server sent GOAWAY and closed the connection; LastStreamID=2051, ErrCode=NO_ERROR, debug=“max_age”

Exit code: 100

What can I do to solve the problem?
Thank you in advance.

#2

Maybe you are using too many threads? Reduce to 4 max, better yet - 2.

Another possibility — google drive outage. Try again later.

Another possibility is perhaps auth token becomes stale and needs refresh? Is duplicacy.com accessible?

But 300GB is not an amount of data that should take that much time to restore, unless you have dial-up.

Let’s wait for @gchen to comment on whether duplicacy handles token refreshes mid-restore, but in the meantime, as a workaround, you can use rclone to provide access to google drive storage to duplicacy.

Option one:

  • mount the google drive locally (with or without VFS) cache to e.g. /tmp/duplicacy
  • init a new local storage in duplicacy in that folder
  • restore.

Option two:

  • rclone serve Google drive storage via SFTP
  • init a new sftp storage is duplicacy
  • restore.
#3

Before I wrote here I had already tried with 2 threads, so the problem cannot be threads-related.

Regarding the connection, I have FTTH 1000 Mb/s download and 300 Mb/s upload; during this restore procedure Duplicacy downloads files at 30 - 40 MB/s.
Bonus question: Duplicacy doesn’t use all the bandwith because of the limited threads or because the backup is encrypted?

It’s very unlikely it’s a Google Drive outage: I have tried to do the restore several times on different days. Moreover, Duplicacy always manages to download some of the first chunks but then it suddenly stops due to this problem (always after the same time).

Thank you for the workarounds but I didn’t try them since I am not in a hurry to restore: it was just a try to be sure that it could have been possible to restore everything without any problems.

#4

Try with 1.

This seems to the bandwidth Google Drive allots to you, and it’s quite generous.

See, using Google Drive as a bulk storage is abuse, it wasn’t designed for that. It was designed to store and share documents. Same goes about OneDrive, Dropbox, etc. If you want fast object storage with massive performance to saturate your downstream – google offers a separate product (that duplicacy also supports): Google Cloud Storage. (I’ve been saying for a long time that duplicacy shall drop support for *drive services and only focus on object storage, but it will probably never happen, because supporting those are good on paper. And yet, if you read the forum, the vast majority (if not all) of issues reported were with this kind of services (failed prunes, failed checks, partial uploads, etc).

How long specifically does it take to fail since start?

Then do test the workaround. Another workaround woudl be to install Google Drive and restroe from the mounted share, but depending on your os this may or may not be more reliable. On macOS native google drive client switched from osxfuse to smb and reliability plummeted.

#5

Google Drive works perfectly fine with Duplicacy, although I agree the other ‘drive’ products are much less suitable. There are well-defined API limits - even with the Enterprise grade service - but GCD has proved to be extremely robust at handling TBs even PBs of storage for many users for years.