Hello, currently I’m testing Backblaze B2 as my second storage option. B2 mentions, that the downloads can be free, if using their CDN. Someone knows how to do that easily? I would like to test it, so I know it’s working, in case I need to restore.
I’m using the webgui version of Duplicacy.
Thanks!
They don’t have a CDN. But transfers between members of a bandwidth alliance is free B2/Cloudflare Partnership Offers the Lowest Cost Media Serving Platform. So you can egress to cloudflare for free. How much does it cost you to egress from cloudflare — depends on your relationship with cloudflare.
Have a look at these resources:
Please review Service-Specific Terms - Cloudflare, as downloading backups like this will likely be violating terms of service of a CDN.
Backblaze B2 egress cost is rather low, especially multiplied by the probability of needing full restore; so I would just pay it as needed.
Thanks for the explanation. I will need to go through the articles if I have some minutes. But sounds it will not work out of the box, like I thought.
Anyway, you’re right, the price is not really high. I will download it once to test if a restore would work. After this, I basically will trust the service.
You don’t really need to download the whole thing — just a few files selectively, and ideally on a new [virtual] machine, mostly to ensure that you have all the necessary keys, passwords, accounts, available somewhere other than on the machine you are trying to restore data for.
Help! I’m really close to being able to connect Duplicacy w/ Backblaze via the Cloudflare CDN. I hope someone can point out where I might be going wrong.
First, what I’ve done so far:
ListAllBucketNames, as well as with [and without] write permission; none of these change the behaviourcurl
duplicacy.json to use b2-custom
Failed to list revisions for backup '1' in the storage b2-restore: The storage has not been initialized
b2-custom, I confirmed I can see all of the revisions when starting a restore [ie. I wanted to verify I was using the correct storage password]How do I get past this? I’m sure there’s something silly I’ve done [or not done].
I’ll include a bit more of my in depth configuration below. Hopefully I haven’t omitted something important to diagnose.
Cloudflare worker configuration:
B2_APPLICATION_KEY – encrypted app key from b2
B2_APPLICATION_KEY_ID – app key id from b2
B2_ENDPOINT – s3.us-west-004.backblazeb2.com
BUCKET_NAME – $path
Verifying the worker:
# curl -I https://b2.foo.bar/bucket-name/config
HTTP/2 200
date: Sat, 11 Jan 2025 20:23:10 GMT
content-type: application/octet-stream
content-length: 842
cf-ray: 90079f7e38c92ee4-LAX
cf-cache-status: DYNAMIC
accept-ranges: bytes
cache-control: public
etag: "e1a0dc3f069b722a153322c01c4e04e1"
last-modified: Fri, 20 Jan 2023 15:54:16 GMT
strict-transport-security: max-age=63072000
x-amz-id-2: aNFo1VjHzMzU232X+YRFktjXqYi4wXjIe
x-amz-request-id: 9414948c77dcc029
x-amz-version-id: 4_z24257173d6eecafd855b0012_f116afed4558c2285_d20230120_m155416_c004_v0402000_t0036_u01674230056345
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=aOhgBySmFFTtZwVlvGuqKZgJtacNboW7Kcfnm0DH0Xgq%2FI%2B6jPp1ZZ6SVjjMKVkrJbsiOEf157VuV3QDXs%2FM5RLtjgv7GuSbVDPSrz1yYHPhaIBOzBQU8Jdlls4%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
alt-svc: h3=":443"; ma=86400
server-timing: cfL4;desc="?proto=TCP&rtt=18141&min_rtt=18024&rtt_var=4014&sent=7&recv=9&lost=0&retrans=0&sent_bytes=3373&recv_bytes=801&delivery_rate=231150&cwnd=221&unsent_bytes=0&cid=18cec05abbfe385e&ts=260&x=0"
And, if I look at the downloaded version of config, it’s starts with duplicacy and the rest is random bytes [because I use an encryption password on the storage]
One thing I noticed [not sure if this is expected or not], but I can’t seem to list directories. Perhaps this is the crux of the issue? But, then, how to fix?:
# curl https://b2.foo.bar/bucket-name/chunks/
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Error>
<Code>NoSuchKey</Code>
<Message>Key not found</Message>
</Error>
duplicacy.json storage configuration
{
"name": "b2-restore",
"url": "b2-custom://b2.foo.bar/bucket-name",
"encrypted": true,
"ras_encrypted": false,
"erasure_coding": "",
"credentials": {
"b2_id": "<<BUCKET_APPLICATION_KEY_ID>>",
"b2_key": "<<BUCKET_APPLICATION_KEY>>",
"password": "<<STORAGE_PASSWORD>>"
}
}
Just to be clear, before you sink any more time debugging this, unless you have enterprise CloudFlare account, and the goal of this exercise is to save costs — it may not save you anything.
Of course, if you are using free personal CloudFlare account — you cannot do that at all. See CloudFlare terms of service: Service-Specific Terms - Cloudflare
Thank you @saspus. Yes, I understand the limitations of this approach. It’s still worthwhile for me to try to get this working.
Just wanted to post some news that I completely missed back in 2023!
Backblaze no longer charges for downloads as long as you stay under three times your average monthly storage.
Check your usage. In my case, I’m way under that limit.
I just ran a quick test by restoring around 9GB of data.
I got an alert message that was concerning at first, but the billing estimate confirms the free download.
It is listed as:
Free data egressed via a peer would be listed as “favored”.
Oh. Wow. That’s super convenient; thanks for finding that @arno .
As for my problem, I’m thinking it’s an issue with either the Cloudflare worker or the BackBlaze S3 APIs. Other users reported the same issue. From what I can tell, S3 can have directories without keys to represent them; resulting in a “key not found” error.