Check and prune to continue on failure

Bug
, ,
#1

Check and prune when encountering failure like this one:

2021-10-01 19:03:52.102 WARN SNAPSHOT_CHECK Some chunks referenced by snapshot obsidian-users at revision 1136 are missing
2021-10-01 19:03:53.983 WARN DOWNLOAD_RESURRECT Fossil 3c28e1bc95819de02607e1980fc3d2efdb6f27206bcf3ec6d8981cf4cdd9609b has been resurrected
2021-10-01 19:03:54.550 ERROR DOWNLOAD_CHUNK Chunk 3c28e1bc95819de02607e1980fc3d2efdb6f27206bcf3ec6d8981cf4cdd9609b can't be found
Chunk 3c28e1bc95819de02607e1980fc3d2efdb6f27206bcf3ec6d8981cf4cdd9609b can't be found

stops.

There should be a way to have it continue for the purposes of identifying all bad snapshots in one go.

3 Likes
Deleting snapshot from google drive fails
#2

Actually, perhaps there’re is another issue lurking here — note how it is the same chunk?

It appears that the chunk was not found, was resurrected, and immediately not found again?

Google drive is eventually consistent — so when writing a file and immediately checking for it it may appear as if it is not there. Is that what happens here?

As duplicacy was running check there was another backup in progress from the same machine to the same account, and separate upload with google drive app to the same account so perhaps my google account was a bit busier than usual and perhaps was throttling api rate which perhaps made the issue worse. It was however pretty consistently repeatable.

Regardless, if duplicacy relies on uploaded files to be there immediately — it’s a bug that needs to be fixed, and it is not specific for google drive.

#3

I guess this chunk is a metadata chunk and its content is needed by another revision. That is why Duplicacy attempted to download it after it had been resurrected.

Why was this chunk marked as fossil in the first place? Are you running prune with -exhaustive?

#4

No, I’m running weekly prune -id obsidian-users -keep 0:1800 -keep 7:30 -keep 1:7 via web UI

I did run exhaustive very long ago (few months back) and after that the check has passed, so I don’t think it’s relevant. is it?

#5

It is possible that a metadata chunk to be deleted is still needed by an ongoing backup that hasn’t posted the revision file. So this could happen even if you don’t use -exclusive.

If you run check or prune again does the error still happen?

#6

Yes, the check still complains about bad snapshots.

The ultimate problem here is however that stale snapshots remain, because prune silently fails to delete snapshots. I’ve just ran it again:

2021-10-03 22:35:50.786 INFO SNAPSHOT_DELETE Deleting snapshot obsidian-users at revision 1134
2021-10-03 22:35:51.693 INFO SNAPSHOT_DELETE Deleting snapshot obsidian-users at revision 1135
2021-10-03 22:35:52.618 INFO SNAPSHOT_DELETE Deleting snapshot obsidian-users at revision 1137
2021-10-03 22:35:53.539 INFO SNAPSHOT_DELETE Deleting snapshot obsidian-users at revision 1138
2021-10-03 22:35:54.463 INFO SNAPSHOT_DELETE Deleting snapshot obsidian-users at revision 1139
2021-10-03 22:35:55.442 INFO SNAPSHOT_DELETE Deleting snapshot obsidian-users at revision 1140
2021-10-03 22:35:56.407 INFO SNAPSHOT_DELETE Deleting snapshot obsidian-users at revision 1142
2021-10-03 22:35:57.385 INFO SNAPSHOT_DELETE Deleting snapshot obsidian-users at revision 1144
2021-10-03 22:35:58.311 INFO SNAPSHOT_DELETE Deleting snapshot obsidian-users at revision 1145
2021-10-03 22:35:59.261 INFO SNAPSHOT_DELETE Deleting snapshot obsidian-users at revision 1146
2021-10-03 22:36:00.209 INFO SNAPSHOT_DELETE Deleting snapshot obsidian-users at revision 1147
2021-10-03 22:36:01.198 INFO SNAPSHOT_DELETE Deleting snapshot obsidian-users at revision 1149
2021-10-03 22:36:02.139 INFO SNAPSHOT_DELETE Deleting snapshot obsidian-users at revision 1152

and yet,

image
image738×1342 81.2 KB

check-20211003-212521.log.zip (591.0 KB)
prune-20211003-223313.log.zip (1.3 KB)

#7

Ultimately, the root cause for why did it get into bad state is this: Deleting snapshot from google drive fails snapshot files don’t get deleted, while their chunks are. Lets continue discussion of that issue there.

However my point was that duplicacy check should move on to the next revision and continue checking, not give up because it stumbled on bad revision.

1 Like