Activities no longer logged

Bug
#1

Please describe what you are doing to trigger the bug:
backup and check failed due to low disk space. I couldn’t see the logs from the UI showing any details because of the issue below.

Please describe what you expect to happen (but doesn’t):
I expect to see the recent activities in the activities table

Please describe what actually happens (the wrong behaviour):
The activities table no longer contains the list of activities and only shows a 10 minute range.

image
image873×309 8.78 KB

It is still like this after freeing up disk space and manually running the tasks successfully.

#2

Looking into the duplicacy_web.log it seems that something has got corrupted:

2021/04/27 22:03:15 Failed to parse the stats file /dir/.duplicacy-web/stats/schedules/Backup.stats: unexpected end of JSON input
2021/04/27 22:03:15 Failed to parse the stats file /dir/.duplicacy-web/stats/schedules/Check.stats: unexpected end of JSON input
2021/04/27 22:03:15 Failed to parse the stats file /dir/.duplicacy-web/stats/schedules/Prune.stats: unexpected end of JSON input

Checking the content of these files, i see that they’re empty. Any suggestions why duplicacy would allow them to be written out like this?

#3

Similar but related topic, it seems whatever caused the bugs above, also screwed up my configuration too:
Getting this error, seemed to make it look like my disk got unmounted:

image
image950×110 10.5 KB

In fact what actually happened is that the backup record for /mnt/disk1 just went missing entirely! This is something duplicacy did itself since I have not touched it for weeks/months.

Another problem is that this never runs the configured post backup hooks so i never get a healthcheck notification that it failed. Only knew because my other backups completed with “out of disk space” errors.

inside.duplicacy-web/duplicacy.json the “repositories” array is now missing the disk1 respository (previously index 1), and leaving index 0 and index 2 in place. Something is a bit messed up here…

#4

Repository indices in ~/.duplicacy-web/duplicacy.json do not need to be consecutive, so it is fine to have 0 and 2 but not 1. I guess the Invalid source directory error was caused by the corresponding repository directory under ~/.duplicacy-web/repositories/localhost/ not existing. You can try to create this missing directory manually.

#5

@gchen I think you got the wrong idea.
I’ve had three backups running hourly for 2 years: /mnt/disk0 /mnt/disk1 and a third one for documents.

On disk, there are three directories corresponding to these repository indices (0, 1, and 2), and these were in the JSON file under repositories as 0, 1 and 2.

What happened is that my backup healthcheck failed, and when i came to view the error in the UI, I found the error above. Somehow, during the failure*, duplicacy decided to delete my configured repository from the json config file, and then all the jobs referencing that repository failed.

edit:
*Sorry, forgot to mention something important, the job failure happened due to lack of disk space on the drive that duplicacy was running from. That’s not a bug with duplicacy for obvious reasons, but when viewing the failure, two repo backups failed for lack of disk space, the other failed due to “invalid source directory”.

This may be what caused duplicacy to write out empty files (duplicacy bug), and could be removing backups a side-effect.

I have a feeling something like this happened before after upgrading duplicacy. Not lack of disk space, but the config was wiped/corrupted and I had to manually reconfigure it again.

#6

In early versions, duplicacy.json was updated in-place which can lead to corruption but recent versions always write to a temporary file first. I don’t know how lack of disk space can cause a repository index to disappear.

Are you still seeing only 2 backups on the Backup page?

#7

I was, but after manually adding the repository back to the array in duplicacy.json, everything started working again and the third one re-appeared in the backup page in the UI.