Rclone serve for Duplicacy like restic API

Feature
#1

Hi all,

I am delighted to open this new topic (I am doing it in both rclone and Duplicacy forum) to request a future collaboration between two of the best tools all over the internet for backing up data which is not yet in place.

Basically this Feature request is to be able to create a serve a remote over a given protocol with Rclone to serve files for Duplicacy so we can perform backups directly similar as the restic integration and we can use duplicacy in those storage provides not supported yet.

Now I am backing up it with “rclone” mount option, but that makes all the files available in the server as a mount point (that can lead to mistakes due to incorrect deletions).

I have tested the restic rclone integration and it works like a charm, hence I am looking forward to see such a good piece of work for the two best backup tools (GitHub - gilbertchen/benchmarking: A performance comparison of Duplicacy, restic, Attic, and duplicity) I have the pleasure to use. Rclone and Duplicacy.

What do you think?

1 Like
#2

After all the tests with the webdav implementation that gave so poor performance, is there any plan to have this option moved forward?

I have been last two year doing backups using rclone mount as remote storage access, so basically the remote storage is visible locally through a local folder, however, each an every backup takes a lot due to the retries after any failure, as after below error it has to start reviewing all the chunks… here is an example of a new backup from scratch in onedrive business (not supported by duplicacy yet)…

Uploaded chunk 1007 size 16601154, 8.29MB/s 1 day 13:05:07 0.9%
Uploaded chunk 1010 size 5747692, 8.29MB/s 1 day 13:05:42 0.9%
Uploaded chunk 1009 size 16777216, 8.29MB/s 1 day 13:05:48 0.9%
Uploaded chunk 1011 size 13877749, 8.29MB/s 1 day 13:06:29 0.9%
Uploaded chunk 1012 size 16777216, 8.29MB/s 1 day 13:06:35 0.9%
Failed to upload the chunk 6b6a31158b7b60803d5fefce29d62d4e7e0dbb198cf87adc4fffdca3d997c860: rename /mnt/OneDrive/Duplicacy/dataDroid/chunks/6b/6a31158b7b60803d5fefce29d62d4e7e0dbb198cf87adc4fffdca3d997c860.jnokewin.tmp /mnt/OneDrive/Duplicacy/dataDroid/chunks/6b/6a31158b7b60803d5fefce29d62d4e7e0dbb198cf87adc4fffdca3d997c860: no such file or directory
Incomplete snapshot saved to /data/.duplicacy/incomplete

I have to say the file does exist…

# ls -lsh /mnt/OneDrive/Duplicacy/dataDroid/chunks/6b/6a31158b7b60803d5fefce29d62d4e7e0dbb198cf87adc4fffdca3d997c860*
128K -rw-r--r-- 1 root root 128K mar  2 01:51 /mnt/OneDrive/Duplicacy/dataDroid/chunks/6b/6a31158b7b60803d5fefce29d62d4e7e0dbb198cf87adc4fffdca3d997c860.jnokewin.tmp

I guess this collaboration would improve a lot the behaviour of the remote backups and would give duplicacy the opportunity of accessing more remote storage providers already investigated by rclone and not yet supported by duplicacy.

Don’t you think is a good idea?