OneDrive (odb) issue with jobs dying

I’ve created a forum post to bring this issue https://github.com/gilbertchen/duplicacy/issues/611 to your attention. As it stands OneDrive for Business is not usable especially after a second snapshot has left some tmp files on OneDrive with names matching to a list of chunks that are missing from the repository.

Let me know if you would like any additional debug logs etc.

How often does this happen to you? Can you post all error-related logs?

I ran a test to upload a 100G file and had one 400 error.

Thanks for the response. The problem has been happening all week but typically I was able to create a new backup with an additional 20Gb last night. It succeeded without duplicacy exiting. Maybe it was an issue with OneDrive? I’ll keep an eye on it.

After the successful backup I ran a check which is reporting missing chunks for the latest backup. If I look for the missing chunks they are indeed missing from the backup, however, there are ~tmp files with names that match the missing chunk names. I have set the maximum chunk size to 1MB (I’m backing up large files) and the ~tmp file sizes range from 700k-998k so it looks like the upload has stopped before duplicacy can rename the ~tmp file?

It’s a bit worrying that I get a backup successful message when in reality there are issues. I’ll run my next backup with the debug switch and capture the output to a log file to see if it gives any clues. Anything else I can check?

Thanks

If 1MB is the fixed chunk size it is normal to have 700k-998k chunks due to compression. Those ~tmp files were created by OneDrive – for OneDrive Duplicacy doesn’t upload to temporary files first and then rename.

How many ~tmp files are there? Can you manually rename them and then run duplicacy check -chunks to see if these files are actually complete?

Thanks for the explanation, that makes sense. There’s around 50 ~tmp files and I’ve renamed the chunks that are marked as missing for the latest revision. I’m running a duplicacy check -chunks but some of the chunks are already failing verification so it looks like something has gone wrong. It’s got 1 day to run so I won’t know the full extent of the problem for a while.

Do you have any further suggestions or is it going to be a case of starting again?

You can delete all ~tmp files and then change the repository id and run a backup again to see if these file can be regenerated.

I’ll run some tests to reproduce the issue. In the past OneDrive had a bug that caused incomplete files to be saved: OneDrive Business: ERROR DOWNLOAD_CHUNK. But that has been fixed and the behavior here is different. I wonder if in this case their servers somehow returned responses indicating completion without having saved the complete files.

Thanks again. I’ve deleted all the temp files, deleted the corrupt chunks and ran another backup after increasing the repository id by one. I’ve backed up another 20Gb and it bombed with one 400 error but ran to completion after manually resuming. The amount of 400 errors seems to be related to the amount I’m backing up, whether this is just chance or something specific to OneDrive I’m not sure.

After more backups I’ve done another chunk check and they’ve all verified successfully so looking good for the moment. It makes me a bit nervous about my backups so I’ll leave it a few days and then run another check to see how things are going.