Indexing infinite , error backup

Support
#1

I have an error when creating a backup from my duplicacy web gui, it keeps Indexing indefinitely for a long time and the logs are stagnant:

Running backup command from /root/.duplicacy-web/repositories/localhost/0 to back up /var/lib/pterodactyl/volumes
Options: [-log -v -d backup -storage eagle_new -threads 8 -limit-rate 25000 -stats]
2024-12-30 17:16:43.003 INFO REPOSITORY_SET Repository set to /var/lib/pterodactyl/volumes
2024-12-30 17:16:43.003 INFO STORAGE_SET Storage set to s3://mi@n0q5.mi.idrivee2-39.com/realisticmiamipremium
2024-12-30 17:16:43.003 DEBUG PASSWORD_ENV_VAR Reading the environment variable DUPLICACY_EAGLE_NEW_S3_ID
2024-12-30 17:16:43.003 DEBUG PASSWORD_ENV_VAR Reading the environment variable DUPLICACY_EAGLE_NEW_S3_SECRET
2024-12-30 17:16:43.147 DEBUG STORAGE_NESTING Chunk read levels: [1], write level: 1
2024-12-30 17:16:43.150 INFO CONFIG_INFO Compression level: 100
2024-12-30 17:16:43.150 INFO CONFIG_INFO Average chunk size: 4194304
2024-12-30 17:16:43.150 INFO CONFIG_INFO Maximum chunk size: 16777216
2024-12-30 17:16:43.150 INFO CONFIG_INFO Minimum chunk size: 1048576
2024-12-30 17:16:43.150 INFO CONFIG_INFO Chunk seed: 6475706c6963616379
2024-12-30 17:16:43.150 TRACE CONFIG_INFO Hash key: 6475706c6963616379
2024-12-30 17:16:43.150 TRACE CONFIG_INFO ID key: 6475706c6963616379
2024-12-30 17:16:43.150 DEBUG BACKUP_PARAMETERS top: /var/lib/pterodactyl/volumes, quick: true, tag: 
2024-12-30 17:16:43.150 TRACE SNAPSHOT_DOWNLOAD_LATEST Downloading latest revision for snapshot eagle_new
2024-12-30 17:16:43.150 TRACE SNAPSHOT_LIST_REVISIONS Listing revisions for snapshot eagle_new
2024-12-30 17:16:43.151 INFO BACKUP_START No previous backup found
2024-12-30 17:16:43.151 INFO BACKUP_LIST Listing all chunks
2024-12-30 17:16:43.151 TRACE LIST_FILES Listing chunks/
2024-12-30 17:16:43.153 INFO BACKUP_INDEXING Indexing /var/lib/pterodactyl/volumes
2024-12-30 17:16:43.153 INFO SNAPSHOT_FILTER Parsing filter file /root/.duplicacy-web/repositories/localhost/0/.duplicacy/filters
2024-12-30 17:16:43.153 DEBUG REGEX_DEBUG There are 0 compiled regular expressions stored
2024-12-30 17:16:43.153 INFO SNAPSHOT_FILTER Loaded 0 include/exclude pattern(s)
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing .sftp/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/.cache/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/.cache/JNA/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/.cache/JNA/temp/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/.trash/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/01b2d1e3-2045-4e11-820e-7d4b82c51682/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.cache/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.cache/JNA/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.cache/JNA/temp/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/com/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/com/google/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/com/google/code/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/com/google/code/gson/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/com/google/code/gson/gson/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/com/google/code/gson/gson/2.8.6/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/me/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/me/lucko/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/me/lucko/jar-relocator/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/me/lucko/jar-relocator/1.4/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/net/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/net/bytebuddy/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/net/bytebuddy/byte-buddy-agent/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/net/bytebuddy/byte-buddy-agent/1.11.0/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/org/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/org/jetbrains/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/org/jetbrains/kotlinx/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/org/jetbrains/kotlinx/kotlinx-coroutines-core/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/org/jetbrains/kotlinx/kotlinx-coroutines-core/1.5.1/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/org/jetbrains/kotlinx/kotlinx-coroutines-core/1.5.1/relocated/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/org/jetbrains/kotlinx/kotlinx-coroutines-core/1.5.1/relocated/MagicItems/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/org/ow2/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/org/ow2/asm/
2024-12-30 17:16:43.153 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/org/ow2/asm/asm/
2024-12-30 17:16:43.154 DEBUG LIST_ENTRIES Listing 01b2d1e3-2045-4e11-820e-7d4b82c51682/?/.slimjar/org/ow2/asm/asm/9.1/
#2

Switch to a better storage provider.

What are you trying to backup? Is the data readable? Can you enumerate its medatada with find . -exec stat {} \;?

I’ve edited your post to fix formatting because logs are hard to read when set in the proportional font. See other supported formatting features here: Formatting posts using markdown, BBCode, and HTML - Using Discourse - Discourse Meta

#3

I have already solved the problem. I have been using idrive for many months without problems.

There was a directory containing 5 000 000 files and for some reason they were not readable, I excluded this directory and now it works.

#4

Many months of uploading data is hardly an endorsement. I recommend searching this and other forums for iDrive, before you spent too much efforts and time and/or lose data.

There seem to be other directories visible in your log snipped that you shall exclude as well — such as cache and trash. The better approach is to only include what you actually need to backup as opposed to covering blanket data.

Excluding data just because it takes longer to backup is not a right approach.