Configure Storj storage in web-UI?

Support
#1

Is there a way to configure a storage using a Storj bucket in the web-UI? I don’t see a dedicated “Storj” provider under S3, and fiddling with the endpoint hasn’t worked for me…

Thanks!

#2

P.S. Duplicacy Web 1.6.3 using Duplicacy CLI 3.0.1

#3

If I read Duplicacy does not fetch updated CLI engine correctly, this will come in a new Duplicacy Web release.

#4

I use Storj as a backend in the following way:

  • Configure a storage bucket

  • Connect Rclone to connect to the bucket using an access grant

  • Mount the Rclone remote

  • Run the Duplicacy image in Docker, pointing a storage path in the image to the Rclone mount point.

  • In the Duplicacy UI, all I do is add a disk location, pointing to the container path that connects to the Rclone mount point.

Everything works ok except Prune. All I did to make Prune work? Add an -exclusive flag and ensure that no other backup process is running concurrently. It’s because a particular command isn’t implemented well in Storj yet.

FYI I tried to make the Storj gateway work out (it’s a Storj uplink and a Minio instance bundled together). It has way too much network overhead. I think it’s intended for a fatter outbound pipe than I’ve got currently.

I also tried to make S3 work; it just isn’t compatible in the way Storj has implemented it. Plus it costs more.

#5

Thanks for tips! Hopefully Storj will be supported natively soon.

#6

It’s already supported:

#7

Using the CLI it is, but not yet through the web-UI.

#8

I would strongly suggest using Rclone serve instead of the mount: This removes a whole host of potential issues related to FUSE and local filesystem, and improves determinism.

That said - maybe adding “rclone remote” would not be a bad idea? This will immediately make all those storages accessible in one go.

Is this planned?

#9

Talking of this… Long ago I noticed Rclone has a serve restic. I haven’t looked too much into Restic, but gather Duplicacy was loosely inspired by some of its features(?). I know Kopia can use Rclone directly as well.

I have to wonder, if leveraging Rclone and/or a REST API is the way forward in supporting new storage backends - perhaps even replacing some tricky ones (OneDrive) - since Rclone’s codebase would be more robust and more thoroughly tested.

2 Likes
#10

Hoping this is added to the web-UI. That said, is there a way to leverage your container and manually add storj via the CLI but have it reflect in the web-UI after being added?

I tried something to the effect of /config/bin/duplicacy_linux_x64_3.1.0 add -e -copy Local -bit-identical -key /config/duplicacy-priv.pem Storj Media storj://xxxxxxxxxxxxxxxxxxxxxxxxxx@us1.storj.io:7777/Duplicacy but I don’t think I’m doing it properly. It said Repository has not been initialized

#11

I don’t think so. The configuration files for CLI are [over]written by the GUI.

You can start backing up via S3 gateway, and then when duplicacy GUI catches up—switch to native.

#12

Got it, I’ll check that out.