I notice that there is a ~2 GB file at .duplicacy/cache/<storage>/incomplete_files with a very recent last modified time. My last backup was successful so I didn’t expect anything to be left over. What is this file, and should I be worried?
This file is created when an initial backup was aborted. It should be automatically deleted once the initial backup is done. I don’t know why it is still there but it can be deleted without causing any issues.
I don’t think that explains the presence of this file in my case. I have two storage backends, dropbox and b2, and both of them have this file with a last modified time within the last few days, but the repository is many years old.
Additionally, dropbox never had an initial backup run as it was created by using duplicacy copy from b2 which already had many revisions at the time.
As a test, I ran an incremental backup (duplicacy backup -stats -threads 4, to dropbox) and noticed that the incomplete_files kept growing during the scanning phase (before 
outputted Uploaded chunk 1 size 1080927, 528KB/s 17:26:55 0.0%), eventually reaching 2.4GB. Once started uploading chunks, the file stopped changing. After successful completion, the file was left in place without being deleted.
Then I ran the backup again with no changes, and observed the same behavior: incomplete_files started empty and grew until it reached 2.4GB. finished with 0 new, 0 bytes, 0 bytes uploaded and left the incomplete_files there.
Not sure if relevant, but I do have sockets and inaccessible (due to fs permissions) files that skips. But the socket files have no data and files that can’t access don’t have any readable data either, so these can’t total 2.4GB.
Is there anything I can provide to help debug this? Seems a little concerning…
Sorry, I forgot that starting from 3.0.0 the incomplete_files file is the on-disk entry list. It is generated when there are more than a million of files in the source directory, even if it is not an initial backup or if no files have been changed. So it is just a matter of this file not getting properly deleted at the end of the backup job. I’ll fix it for the next CLI release.
Ah! Thank you very much for looking into it!
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.