Does duplicacy pass any information about the backup to the post-script

dtownsend · 19 February 2019 05:41

When duplicacy runs the post-backup script does it pass it any arguments? It would be super useful to say pass some information about whether the backup failed and counts of failed files and such. Worst case the path to a log file would at least make scraping that data possible.

TheBestPessimist · 19 February 2019 06:05

AFAIK duplicacy only logs pruning, so there’s nothing for the post-script to analize.
Second: I would very much like it that duplicacy does not call the post script (or passes a failure status) in case a failure occured.

dtownsend · 19 February 2019 15:53

Ah, that’s disappointing. Makes it very difficult to do anything useful in response to the backup state unless you’re manually running it.

TheBestPessimist · 19 February 2019 16:30

I guess this is one of the reasons there are scripts for

dtownsend · 19 February 2019 16:42

Sure but is there any way to get the duplicacy web scheduler to run those scripts in preference to directly calling the duplicacy cli?

TheBestPessimist · 19 February 2019 17:40

i’m not sure how to get that to happen, though lets wait for multiple opinions

saspus · 19 February 2019 23:17

I think we had exactly same discussion some time back

Controlling backup behaviour when errors occur in pre/post scripts

TheBestPessimist · 20 February 2019 05:34

I knew we had one (was on phone and i was too lazy to search).

The problem is @dtownsend wants to use the web gui with his scripts, that’s why i’m not sure what advice to give.

gchen · 21 February 2019 03:37

I agree we should pass something to the post-script, like whether the backup has succeeded or failed. Passing the path to the log file may not be a good idea, because at this time the log file is still being written to and the post-script may not be able to read the full content due to buffering.

TheBestPessimist · 22 February 2019 07:02

Which log file? Beside prune no other command creates logs right?

gchen · 22 February 2019 16:01

My bad. There isn’t a log file generally available so there isn’t a way to parse the log information to post-scripts.

magbeat · 4 June 2019 21:48

Hi gchen,

really new to Duplicacy and spent quite some time over the last three days to implement my new family backup on the basis of Duplicacy. I have opted for the GUI version (easy setup on all the family members machines). As central monitoring dashboard I am planning to use https://healthchecks.io/ .

Is there any progress on this feature?

I have used another backup solution (relica based on restic) for some time. The status of the last backup (and the error message) are written to a environment variables which can be accessed by the post-event scripts. Is there something similar for duplicacy-GUI?

Thanks for the great backup solution and all of your work!

Cheers,
Markus

TheBestPessimist · 5 June 2019 05:14

I don’t know if there is such a feature planned currently, but i think my duplicacy-utils scrips could be adapted for this (they are created for fully autonomous CLI backups right now).

This is the important part:

github.com

TheBestPessimist/duplicacy-utils/blob/a4d9d58ee2af148d3fb4fd9ba19da10c85119a82/remote_notifications_util.ps1#L39


# This functionality is useful for monitoring via platforms like `https://healthchecks.io`
#
# Note that doRemotePing does not check the status of $globalSuccessStatus.
#   It is caller's job to check that.
function doRemotePing
{
log "=== doRemotePing"
doHealthchecksIOPing
}


function doHealthchecksIOPing
{
if ($healthchecksIOPingURL)
{
    log "=== doHealthchecksIOPing"
    doPostRequest $healthchecksIOPingURL ""
}
}


# doRemoteNotifications is called at script start/end and before/after a duplicacy command is run.
# The notification contains a $message in the body, it is not empty.

In which you have the variable $healthchecksIOPingURL which should have the url from healthchecks.

magbeat · 5 June 2019 07:11

Thanks for your answer!

This seems to work in the way that you have a wrapper around the duplicacy commands and if all the commands run fine you set the $globalSuccessStatus to true which later determines if the remote notification script is called or not.

In the case of the GUI it is more like: run the duplicacy command and then run the post backup scripts, I guess. And as I understand they are run in any case (failure or not). So there is no way to determine if the backup was successful or not…

github.com

gilbertchen/duplicacy/blob/master/duplicacy/duplicacy_main.go

// Copyright (c) Acrosync LLC. All rights reserved.
// Free for personal use and commercial trial
// Commercial use requires per-user licenses available from https://duplicacy.com

package main

import (
	"encoding/json"
	"fmt"
	"net/http"
	"os"
	"os/exec"
	"os/signal"
	"path"
	"path/filepath"
	"regexp"
	"runtime"
	"strconv"
	"strings"

This file has been truncated. show original

backupManager.Backup(repository, quickMode, threads, context.String("t"), showStatistics, enableVSS, vssTimeout, enumOnly)
runScript(context, preference.Name, "post")

But I am not really sure about that. As I have not found a clear answer or documentation on this behaviour.

I guess it would be enough to set some env variables in at the end of the backup function, but I don’t know go really :

github.com

gilbertchen/duplicacy/blob/master/src/duplicacy_backupmanager.go

// Copyright (c) Acrosync LLC. All rights reserved.
// Free for personal use and commercial trial
// Commercial use requires per-user licenses available from https://duplicacy.com

package duplicacy

import (
	"bytes"
	"encoding/hex"
	"encoding/json"
	"fmt"
	"io"
	"os"
	"path"
	"path/filepath"
	"runtime"
	"sort"
	"strconv"
	"strings"
	"sync"

This file has been truncated. show original

TheBestPessimist · 5 June 2019 07:55

You can check the Exit codes details in the post-backup scripts to see if anything good or bad happened.

I think (maybe someone else can find the topic where this is mentioned) that the post-backup scripts are passed as parameter the exit code, so it should be easy to decide.

magbeat · 5 June 2019 08:17

As far as I understand it happens here:

github.com

gilbertchen/duplicacy/blob/master/duplicacy/duplicacy_main.go

// Copyright (c) Acrosync LLC. All rights reserved.
// Free for personal use and commercial trial
// Commercial use requires per-user licenses available from https://duplicacy.com

package main

import (
	"encoding/json"
	"fmt"
	"net/http"
	"os"
	"os/exec"
	"os/signal"
	"path"
	"path/filepath"
	"regexp"
	"runtime"
	"strconv"
	"strings"

This file has been truncated. show original

in the func runScript(context *cli.Context, storageName string, phase string) on line 165

output, err := exec.Command(script, os.Args...).CombinedOutput()

I have tried it with my post-backup scripts and the arguments that are passed down is the backup command including its arguments, but no error code as far as I can see:

home/<username>/.duplicacy-web/bin/duplicacy_linux_x64_2.2.0 -log backup -storage <storage-name> -threads 10 -stats

@TheBestPessimist do you mean this post?: Will post-script run no matter what?

Just ran a test without Inet on the Laptop. If the backup fails, the post-backup script seems not be run… this would mean if there are no pings to the healthchecks.io the backup has failed (or the client was not used for some time).

magbeat · 5 June 2019 13:32

@gchen created a proof of concept pull request. I have successfully used it with my post-backup script.

The problem is, that if there is an LOG_ERROR the logger terminates to program and no post script is called. If I change the LOG_ERROR to LOG_WARNit works like expected…