'no space left on device' cant check, prune, WEBUI has nothing useful

Support
#1

Unraid Running Duplicacy container (hotio) which runs WebUI to local disk. Things have been running fine for a couple of months. But my fault I never set a prune job so I think it was just continually filling up. No I get ‘no space left on device’ and I cant seem to get the space to actually delete it. I’ve tried:

  • doing ANYTHING useful through the WebUI. There’s no way to delete snapshots (but then again maybe there’s no way to manually delete snapshots from the CLI) I’m regreting buying WebUI and thinking I should have gone with CLI
  • going inside the container I see nothing but duplicacy_web command which hides prune and all other commands from the user so you cant use it directly
  • I set a pruning job (also with -exhaustive but I get an error that that’s not an option) and also ‘no space left’
  • deleted snapshots manually (just from the snapshots folder) and tried to run prune to clean up those missing snapshots. Still get no space left’
  • set a check job and get 'some chunks reference by some snapshots do not exist in the storage and it fails

I’m so fed up with this, I’m about to delete all snapshots and start over, but for future reference is there anything I can/should do?

Is there really no way to go to my backup drive and manually delete ANY snapshots? All the chunks are combined and I can’t selectively delete anything?

#2

Delete old snapshots from snapshots sub folder. Then run prune -exhaustive to delete chunks that are not used by any of the remaining snapshots. You can add -exclusive flag to do it quicker but ensure there is no other duplicacy process interacting with the datastore at the same time.

Running out of space on a volume is an event that shall be avoided. Most filesystems require at least 10-15% free space to work properly. Once they are full — efficiency drops and does not improve even after you free some space. I would recommend formatting your volume. Setup notifications to know when you fill up the volume to 60% and in response either delete more data or add more free space.

#3

Thanks for your help. I still get ‘no space left on device’

...
2022-08-05 01:44:27.618 INFO FOSSIL_RESURRECT The chunk chunks/e1/d9a523a5d13576b75917f459d38ceac982cd944395efe01d9fd7a7424fa714.fsl has been resurrected
2022-08-05 01:44:27.618 INFO FOSSIL_RESURRECT The chunk chunks/e1/dacd8b13bb73e5e2809a98fd945eed5e6ca6b4c869e3e1b790e3ba9a6c17f1.fsl has been resurrected
2022-08-05 01:44:27.618 ERROR CHUNK_DELETE Failed to fossilize the chunk e1dad34a3f052ce42633f97028d9938aaced14fc1ae8d2e14ec1cd019f5e166f: rename /mnt/disks/backup_local/duplicacy/chunks/e1/dad34a3f052ce42633f97028d9938aaced14fc1ae8d2e14ec1cd019f5e166f /mnt/disks/backup_local/duplicacy/chunks/e1/dad34a3f052ce42633f97028d9938aaced14fc1ae8d2e14ec1cd019f5e166f.fsl: no space left on device
Failed to fossilize the chunk e1dad34a3f052ce42633f97028d9938aaced14fc1ae8d2e14ec1cd019f5e166f: rename /mnt/disks/backup_local/duplicacy/chunks/e1/dad34a3f052ce42633f97028d9938aaced14fc1ae8d2e14ec1cd019f5e166f /mnt/disks/backup_local/duplicacy/chunks/e1/dad34a3f052ce42633f97028d9938aaced14fc1ae8d2e14ec1cd019f5e166f.fsl: no space left on device
#4

You need to delete or move some of the (other) files for the filesystem can work. Right now it’s in a catastrophic state where even rename does not work.

Try -exclusive mode to avoid fossilization and delete immediately.

#5

Having run into this situation, my suggestion to you would be to manually delete the most recent revision of each snapshot ID, as they’re probably toast anyway. Then I would run prune -exclusive -exhaustive together. This should free up enough space to run a proper prune with the -keep flag.

Just make sure not to combine -keep with -exclusive, as it may delete the most recent backups even further (normally, the most recent backup is protected, but -exclusive changes that behaviour). You can run -exclusive after a prior run with -keep - just don’t run them together in the same command.

#6
  • To be clear I should only be deleting from ‘snapshots’ not ‘chunks’ correct’
  • I did delete some snapshots (18 to be exact) but they were not the most recent they were the oldest
  • I’ll try deleting more recent ones and -exclustive and -exhaustive together.
  • If that doesn’t work I’ll just delete all snapshots and all chunks and start over because i’m sick of this.
#7

I can report it now did free up space. It was like @Droolio said: either the most recent deletions or the -exhaustive or combination with -exclusive. Thanks everyone.

#8

Make sure to do a check to make sure everything is alright after all that, preferably with the -chunks flag too.

1 Like