Missing chunks shortly after initial backup to GCS

Well, the good news is that I’m not a total idiot.

The bad news is there does seem to be something weird with backing up to GCS. I took my cue for this from @facboy who said he’d been using GCS with auto class for over a year. Didn’t occur to me that using google infrastructure was cutting edge or risky, lol.

I’ll keep an eye on it, try to get log data, and report back. (While considering switching to Storj.)

don’t know really. i run a check every 24 hours.

Update:

  1. Again I “fixed” the problem by changing the snapshot ID, and then deleting all the revisions under the previous snapshot. (I had some wonkiness trying to delete those revisions, as I got a “no revisions to delete message” and then ran the same prune -id command and it then found revisions to delete; it was odd, but in the end I got all the old revisions for all old snapshots cleaned up.)

  2. I then left town for 4 days, but I had this backup to GCS running twice a day in launchd (backup only, no prune or check).

  3. When I got home I ran check and it reports no missing chunks.

I’ll keep watching carefully.

I’m just following up on my own thread to close it out.

I’ve now had this backup to GCS running automatically for almost two weeks: I have one launch agent run a backup script, twice a day. I have a different agent run a check once a day. No issues whatsoever.

My best guess, and it’s really a guess, is that I must have loaded my launch agent while I was still doing my first big backup, and it ran both a backup and a prune. This corrupted my first backup. And then when I was recovering from that, I somehow failed to fix it all, so it failed again.

Though I’ve read every word of this thread, carefully, multiple times, I still don’t pretend to understand it completely – but perhaps something like this went on in my case.

In any case, GCS seems to be working fine now as a storage, and I’ll be interested to see how autoclass works out when I get to the point where it automatically changes storage class (and makes this super cheap).

1 Like