I noticed the Onedrive backend is using large amounts of downlink data during (initial) backup - almost half of the uplink data usage. Is the Onedrive API’s really so bad or why this is happening?
I used nethogs on Linux to monitor the data traffic:
NetHogs version 0.8.5-2
PID USER PROGRAM DEV SENT RECEIVED
30504 user duplicacy eth0 1368.410 622.809 KB/sec
On Windows 10 the downlink traffic is 1/10th of the uplink.
I apologize for not responding earlier. Is this still an issue for you?
The only explanation I can think of is that Duplicacy was listing the chunks file on the storage to construct the list of existing chunks, but this usually shouldn’t take more than a minute unless you’ve already uploaded a large number of chunks from other repositories. After the initial backup from a repository is done, Duplicacy won’t do this chunk listing again for subsequent backups.
This was not a short term observation, but something that continued hours and days. I was trying to backup 700GB data to Onedrive, but since the bandwidth was so bad it never succeeded. It seemed to me there was something wrong with the Onedrive protocol.
I wouldn’t be surprised. I haven’t used onedrive for backups but to sync files between computers and found it very unreliable. I moved all my stuff back to Dropbox and avoid onedrive as much as possible.
The same problem persists:
NetHogs version 0.8.5-2
PID USER PROGRAM DEV SENT RECEIVED
1688 user duplicacy eth0 420.806 154.640 KB/sec
This is not limited to duplicacy by the way. One Drive and Google Drive are quite unreliable it seems for anything other than official clients. Dropbox and Amazon Drive on the other hand worked great. It’s a pity duplicacy’s support of amazon drive is broken.
On a separate note, I would stick to storage providers that support S3. That is de facto standard and most tested lean set of api designed to be used via api and receive a lot of exposure and testing. Various *boxes and *drives are not. Even Backblazes B2 still has bugs and quirks that are not obvious and require workarounds or fixes on the backend. Same reason - there and a half people use it
I use OneDrive as a secondary backup destination in addition to B2 since I have one free Office365 account left in my Family Subscription.
Somehow after weeks of trying I managed to backup 700GB photos to OneDrive. Incremental backups seem to work better.
See also
This is completely unrelated to this high uplink usage. This relates to the Onedrive On-Demand-Files feature introduced last year. This “update” broke many other things too…