"repository" init and add options

[ben-ptl]

I don’t think so. My understanding is that the ¨-repositoryoption just lets you create new repositories without having to navigate to the respective folder. It's a kind of extended arm. Nothing more. After using it, you can't tell that the repository was created using it. The-pref-dir` option initiates the repository in a different way, i.e. with the config stuff in a different location. That is a difference that will persist.

Hmm, it does seem like there’s a larger difference to me. It is true that you don’t have to navigate to the repository directory before creating it, but also that it creates the .duplicacy preferences directory in the current working directory instead of inside the repository directory. In addition, there is no .duplicacy file inside the repository directory that points back to the preferences folder. Inside the preferences file you’ll find that the storage object has a “repository” key that points to the full path of the repository root. Without -repository that key is just an empty string. For example:

{
“name”: “my-custom-store”,
“id”: “second-backup”,
“repository”: “/var/some/deep/path-in-the-file-system”,
“storage”: “/var/my-custom-store”,
“encrypted”: false,
“no_backup”: false,
“no_restore”: false,
“no_save_password”: false,
“nobackup_file”: “”,
“keys”: null
}

[towerbr]

I like the idea of being able to back up your settings like that. I think you’d have the same advantage using just the -repository option.

With respect to backup settings, yes, it will work for both options.

Unless I’m mistaken, with your method you have to remember where the actual repository root is and execute duplicacy from that directory.

I agree. But in my case (small number of repositories) this is not a problem. On the other hand you just gave me the idea of adding the line"repository": in my preferences files, even though I do not use it immediately.
EDIT: Nope, I can’t do this, or I will have problems with .duplicacy files.

Using the -repository option you could use the same preferences folder scheme you mentioned, but execute duplicacy from each of the preferences subfolders.

As I use scripts, it’s going to be the same thing. Today I make a cd command to the repository. I would still have the cd command, but it would be to the preferences folder.

I guess one minor downside […]

Well, IMHO it’s not a minor downside. Or, speaking by the other side, it’s a big advantage the flexibility that the .duplicay file gives to you. You can do a lot of things just copying / creating a tiny txt file…

[Nelvin]

If I don’t miss anything here, completely removing any .duplicacy file or folder from repositories would enable us to have “multiple repositories” sharing the same root folder?
This would be great as it’d already solve f.i. my request to have an option for multiple filters for a repository to use in different situations/frequencies. I’d simply create multiple snapshots pointing to the same repository path but each using it’s own filter.

[Christoph]

it creates the .duplicacy preferences directory in the current working directory instead of inside the repository directory.

Thanks for pointing that out. I should probably try things out instead of guessing how they work. Woukd you mind adding a short explanation to the init-wiki?

If I don’t miss anything here, completely removing any .duplicacy file or folder from repositories would enable us to have “multiple repositories” sharing the same root folder?

If that is how it works, that would indeed be ingenious. Have you tried it?

Though see also this:

Different Jobs from the same repository but with different filters

I think this should work: Create 2 empty directories Under each directory create a symlink to your photo library. This will bring in your photo library as a subdirectory Then initialize each with a different filter set

adding the line "repository": in my preferences files, even though I do not use it immediately.
EDIT: Nope, I can’t do this, or I will have problems with .duplicacy files.

What kind of problems?

[towerbr]

adding the line "repository": in my preferences files, even though I do not use it immediately.
EDIT: Nope, I can’t do this, or I will have problems with .duplicacy files.

What kind of problems?

Having the .duplicacy file in the “official” repository and the same repository indicated in the preferences file, I’m not sure what t will happen if I create a copy of the .duplicacy file in another folder, to restore some files, eg.

Will it “obey” the location of the copy of the .duplicacy file and restore the files to the temporary folder or will it restore to the original repository (indicated in the preferences file) and overwrite the files there? I have to do some tests …

[ben-ptl]

I guess one minor downside […]

Well, IMHO it’s not a minor downside. Or, speaking by the other side, it’s a big advantage the flexibility that the .duplicay file gives to you. You can do a lot of things just copying / creating a tiny txt file…

I can appreciate the simplicity of that method. On the other hand, here is one way you could make restoring to a temporary folder from any of your repositories using this alternative scheme relatively simple:

Edit the preferences file for the repository you want to restore . Copy any of the existing storage objects (likely “default” if there is only one) to a new entry in the array. Change the storage name to something unique like “temporary-restore”. Change the repository to point to the temporary folder you want to restore to, let’s say “/var/temporary-restore-dir”. Then issue the restore command, specifying the name of this new storage. (e.g., duplicacy restore -r 1 -storage temporary-restore). Done.

You could precreate this “temporary-store” storage object in your preferences file for each of your repositories as standard practice and then simply change the “repository” path to point to whatever folder you want when it comes time for a restore.

The preferences file might look something like this:

{
“name”: “default”,
“id”: “my-obscure-deeply-nested-folder-backup”,
“repository”: “/var/some/deep/path-in-the-file-system”,
“storage”: “/mnt/big-external-hdd”,
“encrypted”: false,
“no_backup”: false,
“no_restore”: false,
“no_save_password”: false,
“nobackup_file”: “”,
“keys”: null
},
{
“name”: “temporary-restore”,
“id”: “my-obscure-deeply-nested-folder-backup”,
“repository”: “/var/temporary-restore-dir”,
“storage”: “/mnt/big-external-hdd”,
“encrypted”: false,
“no_backup”: false,
“no_restore”: false,
“no_save_password”: false,
“nobackup_file”: “”,
“keys”: null
}

[ben-ptl]

it creates the .duplicacy preferences directory in the current working directory instead of inside the repository directory.

Thanks for pointing that out. I should probably try things out instead of guessing how they work. Woukd you mind adding a short explanation to the init-wiki?

Yeah, it was an interesting find to see this mostly undocumented feature when playing around with the cli. I’ve been using this application for less than a week, so I may need a minute to muster up the nerve to edit the wiki without feeling like I’m not qualified to make that kind of contribution…

[Christoph]

I may need a minute to muster up the nerve to edit the wiki without feeling like I’m not qualified to make that kind of contribution…

That’s fine. But you’ll see that the wiki has absolutely no infirmation yet, about how the option works (except for the basic -help description), so it’s easy to improve it
And if you’re unsure, you can always flag the post for staff attention after editing.

[TheBestPessimist]

does my understanding of this option seem right?

I have no idea, never knew about it. Although it seems the discussion here is pretty long and insightful.

@TheBestPessimist shouldn’t the wiki texts on Github be replaced with a short notice and a link to the respective forum wiki?

First we have to receive @gchen’s blessing for this and afterwards i’ll do it with pleasure!

[Christoph]

First we have to receive @gchen’s blessing for this and afterwards i’ll do it with pleasure!

He seems to be quite busy with other things these days. If the github wikis have an edit history that makes it easy to undo edits, I see no problem with going ahead and making the change. Gilbert can revert if he disagrees. Still having the github wiki just creates additional work for us if people link to it and we want those links to point to the forum Wiki…

[TheBestPessimist]

I see no problem with going ahead and making the change. Gilbert can revert if he disagrees.

But if he (yet) doesn’t want to -> we would’ve both made useless work.
Let’s just wait for a little more time to speak his mind now that he was announced about our intention .

[towerbr]

here is one way you could make restoring to a temporary folder from any of your repositories using this alternative scheme relatively simple

Liked it! Is really a simple scheme and makes the settings even more centralized, without the use of .duplicacy files. I think I’ll implement it …

[towerbr]

Well, or I’m missing something very obvious or is not so simple …

• I edited the preferences file ("repository": "",) and added the path to repository;
• Removed the .duplicacy file from repository;
• Changed the cd command in my script, replacing the repository with the folder where the preferences file is.

and when I execute a backup…

ERROR REPOSITORY_PATH Repository has not been initialized

If I return the .duplicacy file to repository and also return the cd command (pointing to repository), even without removing the repository path from the preferences file, backup performs perfectly.

Any ideas?

[ben-ptl]

• Changed the cd command in my script, replacing the repository with the folder where the preferences file is.

That’s odd. Did you cd to the .duplicacy folder? Because the preferences file should be inside the .duplicacy folder, and you should actually be changing to the parent directory (one level above) the .duplicacy folder that contains the preferences file.

I would also suggest double checking the repository path is valid, properly quoted, etc. Maybe test this out by using the init command to create a new temporary backup that uses the -repository option to refer to the actual root folder you’re backing up, but have the storage be a local temporary directory. Then inspect the formatting of the resulting preferences file to verify your manual edits look the same. Of course then at least start a backup to the temporary storage to make sure it’s working

I don’t see why this wouldn’t work the same for you as it is for me. I’ve tested on Linux and Windows so far, and it all looks good.

[Nelvin]

Nelvin:

If I don’t miss anything here, completely removing any .duplicacy file or folder from repositories would enable us to have “multiple repositories” sharing the same root folder?

If that is how it works, that would indeed be ingenious. Have you tried it?

I just gave this a try and it seems to work as expected

All you need to do is create a dedicated folder for your preferences, then create a subfolder for each of your, let’s call it “virtual repository”, change the current workdir into one of these and then just init a repository as usual just with an added -repository pointing to the rootfolder of wherever you want to base your virtual repository at.

This way, no .duplicacy file will be created in the actual folder of your repository, deduplication works great as usual and you can have a different set of filters for each of these virtual repositories.

Obviously each virtual repository needs it’s own snapshot-id.

I wonder how reliable this is as it’s a new option, but in my quick local test, everything went fine.

[Christoph]

Excellent! Feel free to enrich the wiki with this information. In order to keep the wiki as concise as possible, I’d suggest to just write a few sentences into the wiki and then add more details to a reply to the wiki and refer to it from the wiki.

Hm, as I write this, I’m thinking: maybe this is a bit confusing? Maybe it’s easier to just refer to the post above?

[ben-ptl]

I may need a minute to muster up the nerve to edit the wiki without feeling like I’m not qualified to make that kind of contribution…

That’s fine. But you’ll see that the wiki has absolutely no infirmation yet, about how the option works (except for the basic -help description), so it’s easy to improve it
And if you’re unsure, you can always flag the post for staff attention after editing.

I gave it a shot:

The -repository option specifies how the repository root directory is defined in the preferences file. This may be specified as either an absolute or relative path. Relative paths are relative to the current working directory of Duplicacy at the time it is executed (when the preferences file is being parsed). This option allows for the possibility of the repository configuration files and the repository itself being maintained in separate file system locations. When not specified, an empty repository path is written to the preferences file, causing Duplicacy to treat its current working directory as the repository root.

[Christoph]

Looks good to me. But @gchen might want to double check.

Duplicacy at the time it is executed

There is a possibility for misunderstanding here, isn’t there? Does this refer to the execution of the init command or any command?

[Nelvin]

@ben-ptl Thanks for adding this to the wiki.

With regards to the current working dir - I guess this should always be the folder where one wants the .duplicacy folder to be created. If you init the repository from a different folder (using the -prefs-dir option), you’ll get the .duplicacy file with the path to the prefs in your repository?

Running the backup is not possible from a different CDW anyway, as there’s no option to define the path to the prefs, if I didn’t miss anything?

[towerbr]

I’m back (short business trip). Direct to the point: problem solved.

• Changed the cd command in my script, replacing the repository with the folder where the preferences file is.

That’s odd. Did you cd to the .duplicacy folder?

I said I wasn’t realizing something obvious: then the folder where the preferences file is (let’s call it “settings folder”) has to be initialized … Well, that’s weird … but obviously It worked.

change the current workdir into one of these and then just init a repository

I’ve adopted another solution: instead of initialize, create the .duplicacy folder. etc, I simply moved the .duplicacy file from the repository to the settings folder.

Now I make a cd command to the settings folder, which finds the .duplicacy file, which points to … the setting folder …

Would not it be much simpler if Duplicacy identified the presence of the preferences file and adopted it?

Probably doesn’t make sense to use the two options (-pref-dir and -repository) as I’m doing. It may be better to re-adopt the .duplicacy folder + the -repository option, and abandon the use of the .duplicacy file. It no longer makes sense with the configuration completely centralized as it is now.

Anyway, it’s working … Thanks for the hints!