Hi
I searched the forum and found some similar problems, but no solution that could help me.
I’m using SFTP for two different backups jobs to store the data on a remote server.
According to check (log attached below) and duplicacy portal the backup occupies about 2.5TB.
However, my storage provider says that I occupy 4.5TB and according to size check via SFTP I came to the same value.
I have tried to delete as many snapshots as possible, but this has brought little change, although that should be far more than 1TB. Am I doing something wrong that the chunks are not deleted correctly after the prune job?
I am using two prune jobs with the following options:
-keep 0:180 -keep 7:30 -keep 1:7 -id truenas_personal -threads 4
-keep 0:30 -keep 7:7 -keep 1:1 -id pve_snapshots -threads 4
(I also tried the “-collect-only” and then “-delete-only” option, because I thought there might have any fossils)
Running check command from /cache/localhost/all
Options: [-log check -storage duplicacy -a -tabular]
2022-07-29 22:25:39.901 INFO STORAGE_SET Storage set to sftp://xxxxxx@xxxxxx.xxxxxx.xx:23/duplicacy
2022-07-29 22:25:40.584 INFO SNAPSHOT_CHECK Listing all chunks
2022-07-29 22:31:07.881 INFO SNAPSHOT_CHECK 2 snapshots and 40 revisions
2022-07-29 22:31:07.888 INFO SNAPSHOT_CHECK Total chunk size is 2332G in 498469 chunks
2022-07-29 22:31:08.869 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 1 exist
2022-07-29 22:31:09.826 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 7 exist
2022-07-29 22:31:10.718 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 11 exist
2022-07-29 22:31:11.587 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 12 exist
2022-07-29 22:31:12.453 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 13 exist
2022-07-29 22:31:13.348 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 14 exist
2022-07-29 22:31:14.214 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 15 exist
2022-07-29 22:31:15.077 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 16 exist
2022-07-29 22:31:16.037 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 17 exist
2022-07-29 22:31:17.004 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 18 exist
2022-07-29 22:31:17.940 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 19 exist
2022-07-29 22:31:18.867 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 20 exist
2022-07-29 22:31:19.820 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 21 exist
2022-07-29 22:31:20.769 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 22 exist
2022-07-29 22:31:21.711 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 23 exist
2022-07-29 22:31:22.661 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 24 exist
2022-07-29 22:31:23.647 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 25 exist
2022-07-29 22:31:24.597 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 26 exist
2022-07-29 22:31:25.536 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 27 exist
2022-07-29 22:31:26.493 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 28 exist
2022-07-29 22:31:27.440 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 29 exist
2022-07-29 22:31:28.375 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 30 exist
2022-07-29 22:31:29.317 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 31 exist
2022-07-29 22:31:30.270 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 32 exist
2022-07-29 22:31:31.217 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 33 exist
2022-07-29 22:31:32.174 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 34 exist
2022-07-29 22:31:33.129 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 35 exist
2022-07-29 22:31:34.081 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 36 exist
2022-07-29 22:31:35.017 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 37 exist
2022-07-29 22:31:35.958 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 38 exist
2022-07-29 22:31:36.976 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 39 exist
2022-07-29 22:31:38.013 INFO SNAPSHOT_CHECK All chunks referenced by snapshot truenas_personal at revision 40 exist
2022-07-29 22:31:38.382 INFO SNAPSHOT_CHECK All chunks referenced by snapshot pve_snapshots at revision 28 exist
2022-07-29 22:31:38.881 INFO SNAPSHOT_CHECK All chunks referenced by snapshot pve_snapshots at revision 30 exist
2022-07-29 22:31:39.292 INFO SNAPSHOT_CHECK All chunks referenced by snapshot pve_snapshots at revision 31 exist
2022-07-29 22:31:39.710 INFO SNAPSHOT_CHECK All chunks referenced by snapshot pve_snapshots at revision 32 exist
2022-07-29 22:31:40.097 INFO SNAPSHOT_CHECK All chunks referenced by snapshot pve_snapshots at revision 33 exist
2022-07-29 22:31:40.485 INFO SNAPSHOT_CHECK All chunks referenced by snapshot pve_snapshots at revision 34 exist
2022-07-29 22:31:40.870 INFO SNAPSHOT_CHECK All chunks referenced by snapshot pve_snapshots at revision 35 exist
2022-07-29 22:31:41.252 INFO SNAPSHOT_CHECK All chunks referenced by snapshot pve_snapshots at revision 36 exist
2022-07-29 22:32:41.905 INFO SNAPSHOT_CHECK
snap | rev | | files | bytes | chunks | bytes | uniq | bytes | new | bytes |
pve_snapshots | 28 | @ 2022-07-21 01:03 | 126 | 579,982M | 117565 | 581,452M | 14419 | 71,848M | 117565 | 581,452M |
pve_snapshots | 30 | @ 2022-07-23 01:04 | 126 | 582,132M | 117971 | 583,606M | 4 | 6,993K | 14825 | 74,002M |
pve_snapshots | 31 | @ 2022-07-24 01:03 | 126 | 588,810M | 119163 | 590,304M | 5 | 4,373K | 15021 | 74,456M |
pve_snapshots | 32 | @ 2022-07-25 01:03 | 126 | 591,289M | 119913 | 592,794M | 4 | 3,382K | 15344 | 74,856M |
pve_snapshots | 33 | @ 2022-07-26 01:03 | 126 | 593,219M | 120135 | 594,728M | 0 | 0 | 15220 | 75,362M |
pve_snapshots | 34 | @ 2022-07-27 01:03 | 126 | 593,219M | 120135 | 594,728M | 0 | 0 | 0 | 0 |
pve_snapshots | 35 | @ 2022-07-28 09:23 | 126 | 595,253M | 120682 | 596,766M | 0 | 0 | 15368 | 76,031M |
pve_snapshots | 36 | @ 2022-07-29 01:03 | 126 | 595,253M | 120682 | 596,766M | 0 | 0 | 0 | 0 |
pve_snapshots | all | | | | 193343 | 956,162M | 193342 | 956,162M | | |
snap | rev | | files | bytes | chunks | bytes | uniq | bytes | new | bytes |
truenas_personal | 1 | @ 2022-06-19 09:28 -hash | 164543 | 1334G | 261052 | 1227G | 49 | 101,228K | 261052 | 1227G |
truenas_personal | 7 | @ 2022-06-26 01:00 | 167966 | 1339G | 261879 | 1231G | 45 | 138,301K | 876 | 3,659M |
truenas_personal | 11 | @ 2022-06-30 01:00 | 168006 | 1339G | 261886 | 1231G | 38 | 113,036K | 52 | 154,301K |
truenas_personal | 12 | @ 2022-07-01 01:00 | 168026 | 1339G | 261890 | 1231G | 33 | 101,661K | 43 | 123,819K |
truenas_personal | 13 | @ 2022-07-02 01:00 | 168034 | 1339G | 261890 | 1231G | 26 | 94,196K | 33 | 111,515K |
truenas_personal | 14 | @ 2022-07-03 11:28 | 168086 | 1339G | 261896 | 1231G | 24 | 76,261K | 37 | 112,165K |
truenas_personal | 15 | @ 2022-07-04 01:00 | 168096 | 1339G | 261905 | 1231G | 31 | 85,623K | 33 | 93,888K |
truenas_personal | 16 | @ 2022-07-05 01:00 | 168144 | 1339G | 261893 | 1231G | 24 | 88,880K | 28 | 109,583K |
truenas_personal | 17 | @ 2022-07-06 01:00 | 168681 | 1456G | 285567 | 1340G | 38 | 87,217K | 23726 | 112,378M |
truenas_personal | 18 | @ 2022-07-07 01:00 | 168749 | 1456G | 285568 | 1340G | 30 | 104,252K | 45 | 135,794K |
truenas_personal | 19 | @ 2022-07-08 01:00 | 174896 | 1475G | 286353 | 1344G | 24 | 96,958K | 816 | 4,272M |
truenas_personal | 20 | @ 2022-07-09 01:00 | 175222 | 1475G | 286373 | 1344G | 27 | 99,738K | 51 | 166,678K |
truenas_personal | 21 | @ 2022-07-10 01:00 | 175235 | 1475G | 286383 | 1344G | 33 | 93,063K | 41 | 127,544K |
truenas_personal | 22 | @ 2022-07-11 01:00 | 175280 | 1475G | 286386 | 1345G | 30 | 92,681K | 46 | 157,592K |
truenas_personal | 23 | @ 2022-07-12 01:00 | 175386 | 1475G | 286394 | 1345G | 34 | 91,068K | 43 | 110,372K |
truenas_personal | 24 | @ 2022-07-13 01:00 | 175403 | 1475G | 286387 | 1345G | 22 | 81,332K | 27 | 103,070K |
truenas_personal | 25 | @ 2022-07-14 01:00 | 175407 | 1475G | 286395 | 1345G | 31 | 105,878K | 33 | 115,580K |
truenas_personal | 26 | @ 2022-07-15 01:00 | 175497 | 1475G | 286384 | 1345G | 22 | 85,751K | 36 | 133,675K |
truenas_personal | 27 | @ 2022-07-16 01:00 | 175586 | 1475G | 286401 | 1345G | 29 | 78,708K | 39 | 119,305K |
truenas_personal | 28 | @ 2022-07-17 01:00 | 175591 | 1475G | 286402 | 1345G | 27 | 94,070K | 30 | 99,873K |
truenas_personal | 29 | @ 2022-07-18 01:00 | 175593 | 1475G | 286410 | 1345G | 28 | 96,061K | 38 | 111,277K |
truenas_personal | 30 | @ 2022-07-19 01:00 | 175753 | 1475G | 286409 | 1345G | 22 | 80,279K | 31 | 106,569K |
truenas_personal | 31 | @ 2022-07-20 01:00 | 175775 | 1475G | 286426 | 1345G | 37 | 101,359K | 44 | 116,304K |
truenas_personal | 32 | @ 2022-07-21 01:00 | 175877 | 1475G | 286419 | 1345G | 28 | 94,179K | 32 | 108,194K |
truenas_personal | 33 | @ 2022-07-22 01:00 | 175928 | 1475G | 286425 | 1345G | 34 | 97,640K | 41 | 113,727K |
truenas_personal | 34 | @ 2022-07-23 01:00 | 176454 | 1475G | 286443 | 1345G | 21 | 83,608K | 63 | 238,342K |
truenas_personal | 35 | @ 2022-07-24 01:00 | 176460 | 1475G | 286451 | 1345G | 27 | 98,346K | 30 | 101,876K |
truenas_personal | 36 | @ 2022-07-25 01:00 | 176473 | 1475G | 286451 | 1345G | 27 | 89,773K | 32 | 107,944K |
truenas_personal | 37 | @ 2022-07-26 01:00 | 176481 | 1475G | 286447 | 1345G | 22 | 96,849K | 25 | 101,903K |
truenas_personal | 38 | @ 2022-07-27 01:00 | 176490 | 1475G | 286449 | 1345G | 22 | 88,458K | 24 | 101,399K |
truenas_personal | 39 | @ 2022-07-28 01:00 | 176552 | 1571G | 304037 | 1395G | 24 | 85,208K | 17636 | 51,331M |
truenas_personal | 40 | @ 2022-07-29 01:00 | 176625 | 1571G | 304046 | 1395G | 33 | 112,455K | 33 | 112,455K |
truenas_personal | all | | | | 305116 | 1398G | 305115 | 1398G | | |
Thanks for your help
-Patrick