How to find out if Check command is stuck?

Support
#1

I’ve got about 27TB of data, 4m+ chunks. Trying to run a check command (-threads 16) on the repository that is on a local NAS, connecting via SMB. it has now been stuck on ‘listing chunks’ for 6 hours. CPU 0-2% on the whole system, 3GB ram usage on the whole system (win 10 VM). So basically the machine is idle. Is it really still listing or is it stuck ? How do I troubleshoot this ? How do I unstuck it ?

while this check command has been ‘running’ everything else seems to be functioning fine, running other parallel backups works, but very slowly, clearly though, it’s not a complete crash.

#2

You can add global -d flag to see what’s it doing. I would also reduce number of threads to 1: hard drives are terrible at random IO.

Also, depending on your nases SMB implementations flakiness (looking at you, Synology) it may indeed got concussed.

With local nas I personally found SFTP is the most robust solution (including with respect to ensuring atomicity of file transactions)

1 Like
#3

Another Duplicacy user who recently ran into a similar issue finally figured out what went wrong: the clock on the VM may be off due to an ESXi bug which somehow mysteriously slows down the SMB speed. The issue went away after a clock sync.

If you’re running ESXi too it may be worth a try.

#4

That user was also me :smiley: The backups do run like they should now. The check used to work before though. What I wonder is if the messing with the clock could have messed with duplicacy’s ability to read the backups? because some of them will have timestemps going back and forward in time ?

#5

It’s a 12 bay Qnap nas in raid6 I would think it can handle 16 threads. Non of that matters though as there is barely any traffic going there anyway…

Qnap does not support SFTP…

#6

It could hypothetically be messing up SMB, not duplicacy specifically.

#7

Check if there are stale smbd instances on qnap, and kill them. This used to happen a lot on my old Synology – and both use extremely flaky Samba fork with customizations to support their integration.

Is your host macOS by a chance? if so – AFP will definitely be more stable, inspite of being deprecated, if your NAS supports avahi. In any case, I would not rely on SMB working on these devices, unless you replace the SMBD entirely with modern stable version that QNAP folks did not touch. You will lose some integration but likely gain stability.

WOW. A NAS without SSH?! I… could not have even imagined that could be the case. You should be able to install OpenSSH server though, can’t you?

#8

Update. I bloody love Duplicacy, every time I think I broke it, not it’s something else that’s broken. As always, tuns out that duplicacy itself runs rock solid!

It looks like not 6 but 7h is what it took to complete the check. Left it overnight and it did complete at the end.

Don’t know if that is expected with a repository of that size or not… Still though, would be cool if Duplicacy could utilise 100% CPU instead of idling for 6h… at 2%…

2 Likes
#9

I believe QNAP supports SSH/SFTP out of the box. You can only log in as admin though.