-repository flag makes it possible to init the same repo twice –– best practice?

Support
#1

EDIT2 I now see/understand that a line for path to the filters file now exists in the preferences file (in previous forum discussions of the -repository command, this was discussed but not implemented). So I now see that I could add the second storage from within the same working directory, but then specify two different filter files…no reason to work from two different directories as I’m doing below (and as was discussed in earlier forum posts). But it’s still interesting that you can init the same repo twice, and I wonder if this has any implications…

original post
I understand that the primary design of the init command means that if you want to add a second storage to the same repo, you need to use the add command. If you try to init the same repo twice, it fails, as intended behavior. But if you initially init the repo using the -repository flag then duplicacy won’t stop you front initing it a second time from a different location.

To clarify, here’s what I have done:

  1. Created this directory structure for my duplicacy preference files and scripts, etc.
    /duplicacy_preferences/local_storage
    /duplicacy_preferences/remote_storage

  2. cd'd to the “local_storage” directory and used the init command, specifying ~/ as the repository, and an external HD as the storage

  3. cd'd to the “remote_storage” directory and used the init command, again specifying ~/ as the repository, and a temp folder on my internal HD as the storage

Therefore I have init'd the same repo twice, which would be impossible with that the -repository flag.

My questions:
a) Will this cause problems for me?
b) What are the pros (and cons) of using the add command instead? (which I now see can only be done if working from one directory – add fails from a second directory)
c) Is this a bug? (Should duplicacy see the same path (same repo) and fail?)

#2

The -repository flag was a relatively recent addition - to replace and deprecate the -pref-dir option, as it would disassociate the .duplicacy metadata directory from the repository, so was more flexible.

In turn, this allows init (with -repository) to achieve a similar result.

Remember, init has two purposes - 1) create the storage (config) if it doesn’t already exist, 2) add the association of a snapshot ID and storage to the preferences file. The add command only does the latter, but you can also create a copy-compatible storage with -copy, so it has dual purpose. (It also forces you to declare a storage name.)

So, there’s a little bit of overlap, and in theory both commands could probably be merged into one now, but that’s the history. Note the GUI takes advantage of multiple (generated) preference entries - for different repositories in the same preference file (and always uses -repository). Each repo combo is then addressed by the -storage name with the backup command. You could also edit the preference file directly, as long as the storage config was initialised (step 1).

1 Like
#3

Wait, I don’t think this is true, or if it is, I’m really confused. The first thing it says under the add command page here on the forum is this:

Like the init command, if the storage has not been initialized before, a storage configuration file derived from the command line options will be created.

So if you add a second storage – a new storage – to an already used repository, it will do both 1 and 2.

  • The add command assumes there is already a preference file and tries to write to it.
  • The init command assumes there is not a preference file and tries to create it (and write to it).

But if you are using the -repository flag, and working in a new dir, there will not be a preferences file in that dir, so you can only use the init command (the add command will fail) – despite the fact that the actual repository has already been initialized previously.

#4

Ah yes you’re 100% right, thanks for the correction. Got confused a bit there coz add basically does the opposite to what I said - adds a new storage to an existing preferences. Typically, its main purpose is to add a copy-compatible storage, so it has to create a config etc…

The overlap then, is even greater…

#5

I’m just relieved to know that others find it possible to get turned around on this relation :smile: because I’ve gotten completely lost a few times.

My sense of things having spent a lot of time the last few works working with duplicacy and reading all the docs and threads is that there’s a little bit of a tension between: a) the brilliant design choice that lets you hop into a new directory anywhere, connect to a storage, and then either backup that directory or restore it, and b) the fact that for regularizing certain operations, or setting up more complicated workflow practices, it helps to have some central order (location for prefs files, etc.).

#6

Indeed.

Some of the initial setup does seem to be a bit messy and confusing now, especially to a new user. If I was the dev, I’d probably consider making some breaking changes for a v4:

  1. Fully deprecate -pref-dir, and make -repository (or ‘source’) mandatory
  2. Have a single centralised config a la Rclone, so you don’t have to cd to where .duplicacy is.
  3. Either merge init and add OR separate them logically - init to initialise the storage only, add to link an ID with a storage URL.
  4. Make -zstd the default.

I’d probably also rename some terminology:

  • repository → source
  • snapshot ID → source ID or just ID
  • storage → storage label
#7

  1. Yes…having -pref-dir still around only helps to confuse newbies like me.

  2. Important comparison: for someone like me, the rclone documentation is so good, and it’s so easy to do the configuration through the config command

  3. Exactly – either make them one thing, or make them more distinct.

  4. LOL…This is the first I’ve heard of -zstd, and now that I’ve read about it, I wish I’d used it.

#4 points to a bigger issue: this forum is the best place to find info, and it’s just impossible not to continuously click on the links above that describe all the commands. But those are all woefully out of date, so even though I know that the github wiki (or the command line help) has newer info, I often find myself reading this old info.

It would be great if the wiki could be linked to directly here, and I think it just makes sense to remove the “getting started” forum links, as they are now way too old.

EDIT: Just to clarify that right now the only way to know about -zstd compression is to search for it in the forum, and find the link to its announcement. But to do that you have to know what to search for. The command-line help will list it as an option, but not say anything about it, and neither the init forum post here (under getting started) nor the GitHub wiki page say anything about it.