I have a backup repository beginning on 2021-03-21, where backups are run daily. Here’s a snippet of the beginning of that list.
I’m running the docker container GUI and set a prune (dry run) to keep one snapshot every 30 days for snapshots older than 360 days.
But the dry run says there are no snapshots to delete.
Yet I have over 200 snapshots prior to 2023-06-01. Why are there no deletions found?
Probably a simple misunderstanding of mine.
None of your screen shots are visible. Please show duplicacy list output and the full prune command you’re running.
Thanks. Fixed. Oddly they appear in the preview window with markdown encoding.
Run prune with the -d global parameter (before the prune keyword) and post results here.
Running prune command from /cache/localhost/all
Options: [-log -d prune -storage b2_test -keep 30:360 --dry-run]
2024-07-04 14:28:53.559 INFO STORAGE_SET Storage set to b2://ch
2024-07-04 14:28:53.560 DEBUG PASSWORD_ENV_VAR Reading the environment variable DUPLICACY_B2_ID
2024-07-04 14:28:53.560 DEBUG PASSWORD_ENV_VAR Reading the environment variable DUPLICACY_B2_KEY
2024-07-04 14:28:53.995 INFO BACKBLAZE_URL Download URL is: https://f002.backblazeb2.com
2024-07-04 14:28:54.282 DEBUG PASSWORD_ENV_VAR Reading the environment variable DUPLICACY_B2_PASSWORD
2024-07-04 14:28:55.221 TRACE CONFIG_ITERATIONS Using 16384 iterations for key derivation
2024-07-04 14:28:55.335 TRACE BACKBLAZE_CALL [0] URL request 'HEAD https://f002.backblazeb2.com/file/ch/nesting' returned status code 404
2024-07-04 14:28:55.335 DEBUG BACKBLAZE_LIST https://f002.backblazeb2.com/file/ch/nesting did not return headers
2024-07-04 14:28:55.336 DEBUG STORAGE_NESTING Chunk read levels: [1], write level: 1
2024-07-04 14:28:55.351 INFO CONFIG_INFO Compression level: 100
2024-07-04 14:28:55.351 INFO CONFIG_INFO Average chunk size: 4194304
2024-07-04 14:28:55.351 INFO CONFIG_INFO Maximum chunk size: 16777216
2024-07-04 14:28:55.351 INFO CONFIG_INFO Minimum chunk size: 1048576
2024-07-04 14:28:55.351 INFO CONFIG_INFO Chunk seed: 4ec439da5a5ac3fae30eb188bc8841060dd708a9
2024-07-04 14:28:55.351 TRACE CONFIG_INFO Hash key: da741851c9b7739ba089c5a7cdb63a06f37eab59cf5ae
2024-07-04 14:28:55.351 TRACE CONFIG_INFO ID key: 01a8a0682671fbc696d10e2eeb670bf26b
2024-07-04 14:28:55.351 TRACE CONFIG_INFO File chunks are encrypted
2024-07-04 14:28:55.351 TRACE CONFIG_INFO Metadata chunks are encrypted
2024-07-04 14:28:55.352 DEBUG DELETE_PARAMETERS id: duplicacyweb, revisions: [], tags: [], retentions: [30:360], exhaustive: false, exclusive: false, dryrun: true, deleteOnly: false, collectOnly: false
2024-07-04 14:28:55.352 INFO RETENTION_POLICY Keep 1 snapshot every 30 day(s) if older than 360 day(s)
2024-07-04 14:28:55.352 TRACE SNAPSHOT_LIST_IDS Listing all snapshot ids
2024-07-04 14:28:55.837 TRACE SNAPSHOT_LIST_REVISIONS Listing revisions for snapshot xxx_b2
2024-07-04 14:28:56.101 DEBUG DOWNLOAD_FILE_CACHE Loaded file snapshots/xxx_b2/1 from the snapshot cache
2024-07-04 14:28:56.183 DEBUG DOWNLOAD_FILE_CACHE Loaded file snapshots/xxx_b2/2 from the snapshot cache
2024-07-04 14:28:56.265 DEBUG DOWNLOAD_FILE_CACHE Loaded file snapshots/xxx_b2/4 from the snapshot cache
2024-07-04 14:28:56.350 DEBUG DOWNLOAD_FILE_CACHE Loaded file snapshots/xxx_b2/5 from the snapshot cache
2024-07-04 14:28:56.435 DEBUG DOWNLOAD_FILE_CACHE Loaded file snapshots/xxx_b2/14 from the snapshot cache
2024-07-04 14:28:56.515 DEBUG DOWNLOAD_FILE_CACHE Loaded file snapshots/xxx_b2/15 from the snapshot cache
2024-07-04 14:28:56.601 DEBUG DOWNLOAD_FILE_CACHE Loaded file snapshots/xxx_b2/17 from the snapshot cache
<snip>
2024-07-04 14:30:06.266 DEBUG DOWNLOAD_FILE_CACHE Loaded file snapshots/xxx_b2/622 from the snapshot cache
2024-07-04 14:30:06.403 DEBUG DOWNLOAD_FILE_CACHE Loaded file snapshots/xxx_b2/623 from the snapshot cache
2024-07-04 14:30:06.543 DEBUG DOWNLOAD_FILE_CACHE Loaded file snapshots/xxx_b2/624 from the snapshot cache
2024-07-04 14:30:06.685 DEBUG DOWNLOAD_FILE_CACHE Loaded file snapshots/xxx_b2/625 from the snapshot cache
2024-07-04 14:30:06.685 INFO SNAPSHOT_NONE No snapshot to delete
Out of curiosity, I stopped the docker container and re-ran the prune command from the CLI, with remarkably different results… results which were in line with my expectations.
duplicacy -log prune -storage b2_test -keep 30:360 --dry-run
Here’s a snippet of the log:
2024-07-05 16:11:05.511 INFO STORAGE_SET Storage set to b2://ch
2024-07-05 16:11:05.830 INFO BACKBLAZE_URL download URL is: https://f002.backblazeb2.com
2024-07-05 16:11:06.845 INFO RETENTION_POLICY Keep 1 snapshot every 30 day(s) if older than 360 day(s)
2024-07-05 16:12:00.886 INFO SNAPSHOT_DELETE Deleting snapshot xxx_b2 at revision 2
2024-07-05 16:12:00.905 INFO SNAPSHOT_DELETE Deleting snapshot xxx_b2 at revision 14
2024-07-05 16:12:00.905 INFO SNAPSHOT_DELETE Deleting snapshot xxx_b2 at revision 15
2024-07-05 16:12:00.905 INFO SNAPSHOT_DELETE Deleting snapshot xxx_b2 at revision 17
2024-07-05 16:12:00.924 INFO SNAPSHOT_DELETE Deleting snapshot xxx_b2 at revision 20
2024-07-05 16:12:00.945 INFO SNAPSHOT_DELETE Deleting snapshot xxx_b2 at revision 23
2024-07-05 16:12:00.957 INFO SNAPSHOT_DELETE Deleting snapshot xxx_b2 at revision 25
2024-07-05 16:12:00.980 INFO SNAPSHOT_DELETE Deleting snapshot xxx_b2 at revision 27
2024-07-05 16:12:00.980 INFO SNAPSHOT_DELETE Deleting snapshot xxx_b2 at revision 29
<snip>
In addition, over 500 unreferenced chunks were identified.
So … there seems to be some “bug” in the docker image that affects the prune operation.
As a result I re-ran prune on all the storage locations via CLI, and all consistently were affected.
Anyone?
Perhaps the stale local cache is a problem? Try deleting the contents of /cache volume
Downed the container, renamed cache directory and created a new one. Now running prune across all storages. The first two completed, and their logs look to be the same as the CLI log. Maybe that was it. IDK.
Why persistent map a cache volume? What advantage does that offer?
Those contain temporary repositories that webui creates for duplicacy CLI.
If you want to do stuff that is not supported in web ui, such as adding pre-/post- scripts you would want to have access to that folder to place the files in the right places.
You may also have separate volume for transient stuff to avoid extra IOps on your main array.