Duplicacy always tries to re-upload all files

Support
#1

Guys,

On my Windows 11 desktop I have about 2 TB of data that I backup regularly. Usually duplicacy, as expected, scans the files, uploads the changes and the backup is finished in a very short time.

A couple of days ago, I had the idea of rearranging the data in a more logical way, so I moved around a few directory, maybe 7-800 Gb of data. Since then, duplicay is detecting all the files as changed and every backup tries to upload ALL files, so it takes forever even if there are basically no changes and, in the end, duplicacy only saves a few MB of data.

I am using the Duplicacy GUI and, in the end, I decided to stop the “Duplicacy Web Edition” service, deleted everything in the .duplicacy-web directory (storage, backup configs, cache, logs, literally everything) and restarted from a blank slate. Same story.

This is an example from a test backup that I created:

2026-03-26 07:37:00.997 INFO BACKUP_STATS Files: 5075 total, 3,874M bytes; 5069 new, 3,874M bytes
2026-03-26 07:37:00.997 INFO BACKUP_STATS File chunks: 3 total, 3,905M bytes; 0 new, 0 bytes, 0 bytes uploaded
2026-03-26 07:37:00.997 INFO BACKUP_STATS Metadata chunks: 3 total, 808K bytes; 0 new, 0 bytes, 0 bytes uploaded
2026-03-26 07:37:00.997 INFO BACKUP_STATS All chunks: 6 total, 3,906M bytes; 0 new, 0 bytes, 0 bytes uploaded
2026-03-26 07:37:00.997 INFO BACKUP_STATS Total running time: 00:00:14

This is only a few thousand files, so the whole process takes only 14 seconds, however you can clearly see that duplicacy thinks that 5069 out of 5075 files are new, although I didn’t even look at them.

Any idea? In this case, I didn’t specify any flags in the backup.

Thanks.

ETA: I searched the forum and I found others that had similar issues, but no solution.

#2

Duplicacy uses path, size, and modification time during scan. After a large move, one backup can report many files as new because their paths changed, even though dedup finds the content already in storage. “many new files, zero uploaded" during next backup is therefore normal.

Repeated re-detection on later runs is not normal. It’s either modification times are still changing, or the job is not continuing from the snapshot history it should be using.

I would check each possibility.

  1. Check whether file modification times are still changing.Pick a few files that Duplicacy keeps reporting as new. Record their Windows LastWriteTime, run one backup, then check the same files again before the next backup.
    Get-Item "D:\path\to\file" | Select-Object FullName, Length, LastWriteTime
    If LastWriteTime changes between runs, that is the problem. Something on the Windows side is touching metadata.
  2. Check whether the backup is continuing from the expected snapshot history – compare last snapshot version on the storage with what duplicity thinks is the last one from the backup log

I would also run chckdsk d: /f because I woudl not trust NTFS to not get corrupted in a weird and wonderful way. And if on the target there is also NTFS – chckdsk that too :slight_smile:

#3

Thanks saspus. I fully expected the first backup after the move to take longer, but not the subsequent ones.

The file modification times are not changing. That was my first thought.

What I found out yesterday is that the files that I moved are apparently NOT in the backup. Although the backup says that it’s uploading them, they aren’t there, or at least they are not visible. I said apparently because I enabled the debug option during the backup and I saw that duplicacy finds the chunck in which the files are supposed to be, but nevertheless I can’t see them in the list of files and I cannot restore them. Maybe something in the metadata of those files is confusing duplicacy, although I checked everything: access permissions, attributes, timestamps… I see no difference between problematic files and files that are backed up regularly.

Now I am trying a hail mary solution: copy these files to external drives, delete the copy on the Windows machine and copy them back. Let’s see if this helps. Files that I created after the ill-fated move (even in the same directories) and files that I did not move do not seem to be affected by this problem, so maybe re-creating them will help.

Incidentally, chkdsk found nothing. All apparently clear.

#4

The attempt worked. Copying the files to another hard disk, deleting the originals, and copying them back fixed the problem. I have no idea why, but at least my backup system works again.

#5

This is very bizarre. I don’t have an explanation either. Other than some shenanigans with weird NTFS symlinks/reparse points.

#6

No symlinks or reparse points in these directories. I can only think that moving the directories around changed something in the metadata, and that confused duplicacy, but it’s a very long shot. I don’t like the fact that I have no clue why this happened, but I guess that I have to be happy that now all works again.