Curious about this: i checked my machine as well: windows 10 machine with i5-6400 cpu some 6yo ssd more than half filled and 168GB of data out of which only 15GB actually get to be backed up (my userprofile + other folders on a HDD), as the rest are ignored by the filters file ( i use a symlink repo ).
2018-10-16 01:00:21.579 =================================================================
2018-10-16 01:00:21.587 ==== Starting Duplicacy backup process ==========================
2018-10-16 01:00:21.596 ======
2018-10-16 01:00:21.623 ====== Start time is: 2018-10-16 01:00:21
2018-10-16 01:00:21.633 ====== logFile is: C:\duplicacy repositories\tbp-pc\.duplicacy\tbp-logs\2018-10-16 Tuesday\backup-log 2018-10-16 01-00-21_636752484214107081.log
2018-10-16 01:00:21.642 =================================================================
2018-10-16 01:00:21.961 Zipping (and then deleting) the folder: C:\duplicacy repositories\tbp-pc\.duplicacy\tbp-logs\2018-10-15 Monday to the zipFile: C:\duplicacy repositories\tbp-pc\.duplicacy\tbp-logs\2018-10-15 Monday.zip
2018-10-16 01:00:40.656 ===
2018-10-16 01:00:40.664 === Now executting .\z.exe -log -d backup -stats -threads 18
2018-10-16 01:00:40.674 ===
2018-10-16 01:00:43.146 INFO STORAGE_SET Storage set to G:/My Drive/backups/duplicacy
2018-10-16 01:00:43.155 DEBUG PASSWORD_ENV_VAR Reading the environment variable DUPLICACY_PASSWORD
2018-10-16 01:00:43.170 DEBUG PASSWORD_KEYCHAIN Reading password from keychain/keyring
2018-10-16 01:00:43.185 TRACE CONFIG_ITERATIONS Using 16384 iterations for key derivation
2018-10-16 01:00:43.204 DEBUG STORAGE_NESTING Chunk read levels: [1], write level: 1
2018-10-16 01:00:43.205 INFO CONFIG_INFO Compression level: 100
2018-10-16 01:00:43.205 INFO CONFIG_INFO Average chunk size: 4194304
2018-10-16 01:00:43.205 INFO CONFIG_INFO Maximum chunk size: 16777216
2018-10-16 01:00:43.205 INFO CONFIG_INFO Minimum chunk size: 1048576
2018-10-16 01:00:43.205 INFO CONFIG_INFO Chunk seed: 047e223d47da826b44a58bf72c9f701ded4e04e3afb10e39c3a5eb30c93b1e07
2018-10-16 01:00:43.205 DEBUG PASSWORD_ENV_VAR Reading the environment variable DUPLICACY_PASSWORD
2018-10-16 01:00:43.219 DEBUG BACKUP_PARAMETERS top: C:\duplicacy repositories\tbp-pc, quick: true, tag:
2018-10-16 01:00:43.219 TRACE SNAPSHOT_DOWNLOAD_LATEST Downloading latest revision for snapshot tbp-pc
2018-10-16 01:00:43.219 TRACE SNAPSHOT_LIST_REVISIONS Listing revisions for snapshot tbp-pc
2018-10-16 01:00:43.288 DEBUG DOWNLOAD_FILE Downloaded file snapshots/tbp-pc/3123
2018-10-16 01:00:43.319 DEBUG CHUNK_DOWNLOAD Chunk e1c2e8de2264eb5807a8ca4e841f7b734c9abd829b1b2e557cad7c2ba8e50f6a has been downloaded
2018-10-16 01:00:43.520 DEBUG DOWNLOAD_FETCH Fetching chunk 427602ae10b276099c86ab60973236250bfc37c960a57822812c996e2a5e794f
[..]
2018-10-16 01:00:45.687 DEBUG DOWNLOAD_FETCH Fetching chunk ffc387899375df5cd538fa89846d3fc6edca9ecb7c6081463cec78cfd0f26084
2018-10-16 01:00:45.692 DEBUG CHUNK_DOWNLOAD Chunk ffc387899375df5cd538fa89846d3fc6edca9ecb7c6081463cec78cfd0f26084 has been downloaded
2018-10-16 01:00:45.760 INFO BACKUP_START Last backup at revision 3123 found
2018-10-16 01:00:45.760 INFO BACKUP_INDEXING Indexing C:\duplicacy repositories\tbp-pc
2018-10-16 01:00:45.763 DEBUG REGEX_STORED Saved compiled regex for pattern "/iTunes/Album Artwork/Cache/", regex=®exp.Regexp{regexpRO:regexp.regexpRO{expr:"/iTunes/Album Artwork/Cache/", prog:(*syntax.Prog)(0xc044048b10), onepass:(*regexp.onePassProg)(nil), prefix:"/iTunes/Album Artwork/Cache/", prefixBytes:[]uint8{0x2f, 0x69, 0x54, 0x75, 0x6e, 0x65, 0x73, 0x2f, 0x41, 0x6c, 0x62, 0x75, 0x6d, 0x20, 0x41, 0x72, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x2f, 0x43, 0x61, 0x63, 0x68, 0x65, 0x2f}, prefixComplete:true, prefixRune:47, prefixEnd:0x0, cond:0x0, numSubexp:0, subexpNames:[]string{""}, longest:false}, mu:sync.Mutex{state:0, sema:0x0}, machine:[]*regexp.machine(nil)}
[...]
2018-10-16 01:00:45.764 DEBUG REGEX_STORED Saved compiled regex for pattern ".Library/Saved Application State/", regex=®exp.Regexp{regexpRO:regexp.regexpRO{expr:".Library/Saved Application State/", prog:(*syntax.Prog)(0xc044048bd0), onepass:(*regexp.onePassProg)(nil), prefix:"", prefixBytes:[]uint8(nil), prefixComplete:false, prefixRune:0, prefixEnd:0x0, cond:0x0, numSubexp:0, subexpNames:[]string{""}, longest:false}, mu:sync.Mutex{state:0, sema:0x0}, machine:[]*regexp.machine(nil)}
2018-10-16 01:00:45.764 DEBUG REGEX_DEBUG There are 155 compiled regular expressions stored
2018-10-16 01:00:45.764 INFO SNAPSHOT_FILTER Loaded 155 include/exclude pattern(s)
2018-10-16 01:00:45.764 TRACE SNAPSHOT_PATTERN Pattern: e:(?i)/\.git/
[...]
2018-10-16 01:00:45.784 TRACE SNAPSHOT_PATTERN Pattern: e:.Library/Saved Application State/
2018-10-16 01:00:45.784 DEBUG LIST_ENTRIES Listing
2018-10-16 01:00:45.785 DEBUG PATTERN_INCLUDE C__Users_link is included
2018-10-16 01:00:45.786 DEBUG PATTERN_INCLUDE C__all_link is included
2018-10-16 01:01:34.034 TRACE PACK_START Packing [bla]
[...]
2018-10-16 01:01:34.097 INFO PACK_END Packed [bla]
2018-10-16 01:01:36.248 INFO BACKUP_END Backup for C:\duplicacy repositories\tbp-pc at revision 3124 completed
2018-10-16 01:01:36.248 INFO BACKUP_STATS Files: 118642 total, 14,914M bytes; 174 new, 79,947K bytes
2018-10-16 01:01:36.249 INFO BACKUP_STATS File chunks: 11043 total, 18,647M bytes; 8 new, 65,611K bytes, 19,240K bytes uploaded
2018-10-16 01:01:36.249 INFO BACKUP_STATS Metadata chunks: 12 total, 52,673K bytes; 6 new, 22,476K bytes, 5,721K bytes uploaded
2018-10-16 01:01:36.249 INFO BACKUP_STATS All chunks: 11055 total, 18,699M bytes; 14 new, 88,088K bytes, 24,961K bytes uploaded
2018-10-16 01:01:36.249 INFO BACKUP_STATS Total running time: 00:00:53
2018-10-16 01:01:36.288 ===
On my setup listing starts @
2018-10-16 01:00:45.784 DEBUG LIST_ENTRIES Listing
and the packing starts (so listing ends) @
2018-10-16 01:01:34.034 TRACE PACK_START Packing [bla]
so that would be about 45 seconds of work.
I see here that for 160GB of many files it takes less that 30% of your time for the listing part. I also don’t remember ever noticing that the backup runs. There’s no slowdown, no high fan, nothing.
And this is no ultra-powerful computer and i’m using the intel cpu fan (so the cooling is decent at best).
I’m curious what’s going on with the listing on your machine, or how could it be so slow and especially resource intensive. Do you have a HUGE repository?