About check with -threads and -chunks

[iocularis]

Hi,

using Duplicacy Web i always used the following parameter alone for the check commmand:

-threads 10

My reports anyway include the following:

Options: [-log check -storage xxx -threads 10 -a -tabular]

These are automatically added by Duplicacy Web i assume, apart from -threads 10 that’s added by me.

-tabular should imply -stats that shows

deduplication statistics

Where are these statistics in the log? Just want to figure out the parts.

Reading some posts i dediced now to add the -fossils option. So:

-fossils -threads 10

Reading the docs i noted this:

With -threads you can use multiple threads to verify chunks when -chunks is present.

So it’s useless without using -chunks?

How -chunks works anyway?

Thanks!

[gchen]

Some backends, such as Google Drive, can use multiple threads to speed up the listing operation, so this option can be useful for these backends even when -chunks is not present.

[iocularis]

Thanks!

But what -chunks does?

[saspus]

@iocularis

But what -chunks does?

Check command details

SYNOPSIS: duplicacy check - Check the integrity of snapshots USAGE: duplicacy check [command options] OPTIONS: -all, -a check snapshots with any id -id <snapshot id> check snapshots with the specified id rather than the default one -r <revision> [+] the revision number of the snapshot -t <tag> check snapshots with the specified tag -fossils search fossils if a chunk can't be found …

[iocularis]

My bad, i completely missed that.

So my last question is, i use the following options: -fossils -threads 10

In my report i actually see:

Running check command from /cache/localhost/all
Options: [-log check -storage DK_Duplicacy_Backups -fossils -threads 10 -a -tabular]

Now as i read -tabular should:

show tabular usage and deduplication statistics (imply -stats, -all, and all revisions)

This is all true apart that i do not see -stats in my reports’ options listed above. Is this intended?

In the reports the deduplication statistics are these?:

2022-01-10 01:01:18.763 INFO SNAPSHOT_CHECK All chunks referenced by snapshot AppData at revision 273 exist
2022-01-10 01:01:18.838 INFO SNAPSHOT_CHECK All chunks referenced by snapshot AppData at revision 274 exist
2022-01-10 01:01:18.874 INFO SNAPSHOT_CHECK All chunks referenced by snapshot AppData at revision 275 exist
2022-01-10 01:01:18.920 INFO SNAPSHOT_CHECK All chunks referenced by snapshot AppData at revision 276 exist

And i suppose that the tabular usage is obviously the one at the bottom:

snap | rev |                          | files |  bytes | chunks |    bytes | uniq |    bytes | new |    bytes |
 AppData | 273 | @ 2022-01-03 01:00       | 34301 | 1,544M |    455 | 781,272K |   13 |   9,386K | 455 | 781,272K |
 AppData | 274 | @ 2022-01-04 01:00       | 34307 | 1,535M |    457 | 785,506K |   12 |   6,515K |  15 |  13,620K |
 AppData | 275 | @ 2022-01-05 01:00       | 34311 | 1,533M |    459 | 790,464K |   19 |  10,905K |  25 |  21,689K |
 AppData | 276 | @ 2022-01-06 01:00       | 34316 | 1,533M |    455 | 795,751K |   15 |  11,554K |  20 |  19,525K |

Correct?

[gchen]

-stats will show simple information for each revision like these:

Snapshot test1 at revision 68: 35866 files (10,924M bytes), 3,282M total chunk bytes, 253,728K unique chunk bytes
Snapshot test1 at revision 69: 44369 files (13,414M bytes), 4,173M total chunk bytes, 187,921K unique chunk bytes
Snapshot test1 at revision 70: 46994 files (17,070M bytes), 5,414M total chunk bytes, 249,887K unique chunk bytes
Snapshot test1 at revision 83: 786 files (232,385K bytes), 236,988K total chunk bytes, 102,499K unique chunk bytes

But since they already included in the stats table, these log messages are not shown when -tabular is specified.