ONEDRIVE RETRY Response code: 503

jannes.leroux · 5 May 2026 08:25

Since 30 April 2026 I am getting this response from OneDrive on Purge or Check jobs.

After some research it seems like Microsoft made a change to their Graph API.

Anyone else experiencing this, or have a solution for it?

This is from my logs:
Running check command from C:\ProgramData/.duplicacy-web/repositories/localhost/all
Options: [-log check -storage OneDrive -a -stats -a -tabular]
2026-05-05 04:00:36.924 INFO STORAGE_SET Storage set to one://backup
2026-05-05 04:01:38.283 INFO ONEDRIVE_RETRY Response code: 503; retry after 524 milliseconds
2026-05-05 04:02:42.283 INFO ONEDRIVE_RETRY Response code: 503; retry after 1667 milliseconds
2026-05-05 04:03:46.286 INFO ONEDRIVE_RETRY Response code: 503; retry after 753 milliseconds

pace.gui · 5 May 2026 12:20

Same here. I created an issue in the github repo outlining the problem and potential solution.

However, I suspect this software might have been abandoned. No releases since May of 2025.

github.com/gilbertchen/duplicacy

OneDrive prune fails when server-honored Retry-After consumes the 12-retry budget

opened 12:16PM - 05 May 26 UTC

guipace

## Summary A scheduled `prune` against a OneDrive backend reliably fails when O…neDrive returns a burst of `503`s with long `Retry-After` headers. Duplicacy honors the `Retry-After` values correctly, but each honored sleep counts against the hardcoded 12-attempt budget in `OneDriveClient.call`, so the operation aborts with `Maximum number of retries reached` even though OneDrive is explicitly telling the client when to come back. The chunks for the snapshot revision had already been deleted by the time this happens; only the final snapshot file delete fails. The repository is left with an orphaned snapshot until the next successful prune. ## Environment - Duplicacy CLI bundled with `saspus/duplicacy-web` (Docker) - Storage backend: OneDrive (personal) - Operation: scheduled `prune` (no `-exclusive`) ## Log excerpt ``` ... thousands of successful CHUNK_DELETE lines ... 05:05:36 INFO ONEDRIVE_RETRY Response code: 503; retry after 611 ms 05:05:37 INFO ONEDRIVE_RETRY Response code: 503; retry after 1532 ms 05:05:39 INFO ONEDRIVE_RETRY Response code: 503; retry after 2070 ms 05:05:41 INFO ONEDRIVE_RETRY Response code: 503; retry after 4299 ms 05:05:45 INFO ONEDRIVE_RETRY Response code: 503; retry after 13317 ms 05:05:59 INFO ONEDRIVE_RETRY Response code: 503; retry after 23299 ms 05:06:22 INFO ONEDRIVE_RETRY Response code: 503; retry after 59494 ms 05:07:22 INFO ONEDRIVE_RETRY Response code: 503; retry after 111185 ms 05:09:13 INFO ONEDRIVE_RETRY Response code: 503; retry after 244996 ms 05:13:18 INFO ONEDRIVE_RETRY Response code: 503; retry after 254199 ms 05:17:32 INFO ONEDRIVE_RETRY Response code: 503; retry after 142863 ms 05:19:56 INFO ONEDRIVE_RETRY Response code: 503; retry after 178443 ms 05:22:54 ERROR SNAPSHOT_DELETE Failed to delete the snapshot AppData at revision 87: Maximum number of retries reached ``` Twelve retries, all with `Retry-After` from the server, then abort. ## Root cause In `src/duplicacy_oneclient.go` (the `call` method), the retry loop is: ```go for i := 0; i < 12; i++ { ... } else if response.StatusCode > 401 && response.StatusCode != 404 { delay := int((rand.Float32()*0.5 + 0.5) * 1000.0 * float32(backoff)) if backoffList, found := response.Header["Retry-After"]; found && len(backoffList) > 0 { retryAfter, _ := strconv.Atoi(backoffList[0]) if retryAfter*1000 > delay { delay = retryAfter * 1000 } } LOG_INFO("ONEDRIVE_RETRY", "Response code: %d; retry after %d ms", ...) time.Sleep(time.Duration(delay) * time.Millisecond) backoff *= 2 ... continue } } return nil, 0, fmt.Errorf("Maximum number of retries reached") ``` The 12-attempt cap was reasonable for transient errors, but it conflates two very different cases: 1. The server returned an error and we're guessing when to retry → counting against a budget makes sense. 2. The server explicitly said "come back in N seconds" via `Retry-After` → this is a cooperative throttle, not a failure. Counting it against the budget effectively punishes the client for doing what Microsoft's [throttling guidance](https://learn.microsoft.com/en-us/graph/throttling) tells it to do. During a heavy prune, OneDrive can issue a sustained throttle window long enough to exhaust 12 retries even when each retry behaves correctly. ## Proposed fixes (any of these would help, in order of smallest scope) 1. **Don't count server-honored `Retry-After` waits against the retry budget.** When the response carries a `Retry-After` header, sleep but do not increment `i` (or use a separate, larger counter for this branch). ~10-line change, fully contained in `oneclient.go`. 2. **Make max retries configurable** — e.g. a `-onedrive-max-retries` flag or `DUPLICACY_ONEDRIVE_MAX_RETRIES` env var, defaulting to 12. Lets users on flaky/throttled OneDrive accounts opt into longer patience without patching the binary. 3. **Make `prune` resilient to a single-object delete failure.** Today, one stuck `SNAPSHOT_DELETE` aborts the whole prune. Logging the failure, continuing with the rest of the deletion plan, and surfacing a summary of skipped objects at the end would let prune make forward progress; the orphaned snapshot would simply be picked up by the next run. (1) is the cheapest and addresses the reported failure directly. (2) is a nice escape hatch. (3) is the most useful long-term and is also relevant to other backends, not just OneDrive.

phillipmcmahon · 14 May 2026 18:35

This sadly looks to be the case. With no sign of active development any change to other storage APIs is going to be a bad day for lots of us.

What a shame.

I’m not looking for change for change sake. But the absolute void in development against an ever growing issues list isn’t a good sign for a tool we’re relying on to save the day when the proverbial hits the fan.

pace.gui · 14 May 2026 19:16

I’ve abandoned the use of this product and am now using duplicati.

saspus · 14 May 2026 21:36

Big mistake. Please do your research. The fact that it appears first in search result is not a vote of quality.

They are so many much better alternatives.

saspus · 14 May 2026 21:38

The practical solution is to to let duplicacy do backup, a task that is polished and stable and just works.

And separate connectivity with some goofy remotes to another piece of software, like rclone.

It baffles me why does duplicacy even try to keep up with 20 different api providers. It’s a fools errand.

Don’t misuse *drive remotes for backup and you won’t have any issues. The problem here is not duplicacy. It’s “storage providers” who keep breaking their API.

AWS s3 works and does not required babysitting, does not it?

Gchen need to make one change — drop Google Drive, Dropbox, and other BS as endpoints. And then freeze the code.

But this is not blocking anyone. Anyone can switch to storage does does not suck and stop chasing ghosts.

phillipmcmahon · 14 May 2026 21:58

I have standardised on S3-compatible storage, I use Storj and Backblaze for remote storage, and I run Garage and a USB target locally in addition to snapshots on my NAS. I think I am covered, and those parts of duplicacy seem stable. However, still concerned about any lack of development or engagement from Gchen, which seems to have gone on for quite some time now.

@saspus, what would you replace duplicacy with if you had to?

pace.gui · 14 May 2026 23:11

@saspus lol. A lot of underlying assumptions in your reply. I’m not emotionally tied to any tool and don’t feel like I have to justify my decision to switch and the tool I chose.

I don’t think asking for more than zero releases with updates a year is too much.

saspus · 15 May 2026 00:05

Strongly depends on the host OS. On a macOS – Arq7. No question about it. The only downside – an abhorrent size of local caches it creates. Which could have been an issue of my own making – I adopted 3TB backup on a 512GB SSD Mac…

Any other OS if duplicacy vanishes into the void – restic. Also pretty much no competition. But duplciacy exist – so I would not have switched to anythign else, have I been still needing backup tool

Since last few (4? 5? years) I use neither. My backup is zfs snapshot replication between servers across few states. Simple like an iceberg, and solid like … also an iceberg. Mac gets backed up to the same NAS with Time Machine, and samba extension creates a snapshot upon successful backup completion, which gets promptly replicated to remote server. I use zrepl. I guess this is my backup: GFS style filesystem snapshots + replication.

I don’t use neither windows, nor linux – so no recommendation there. Servers run freeBSD, zfs is first class citizen there.

Nor should you be. I’m not expecting you to justify. I’m just seeing a fellow forum user heading straight into the open manhole and I feel obligated to attempt to warn. But it’s also ok to learn on your own mistakes, I don’t have any vested interest in which backup solution you end up with.

It depends for what reason you are asking.

To keep chasing ever-changing landscape of storage providers that break their api – yes, it’s too much to ask. The correct solution is to drop support of such providers. Rclone does superb job chasing, and there is no need to replicate effort.
to fix longstanding bugs – such as interrupted prune leaving the storage in inconsistent state – there I’m with you, it’s long overdue and does not receive attention it deserves.

But updates for updates sake – no.

saspus · 15 May 2026 16:34

2 posts were split to a new topic: Snapshot replication as a backup

Droolio · 15 May 2026 12:56

Ever since I learned of Rclone, I concluded the same.

Although, I must stress that it was good Duplicacy natively supported the big providers/protocols out of the box. But I’m also not at all surprised Microsoft breaks stuff so easily. If OneDrive can be fixed, great. If not, it might be a good idea to drop broken providers - but only when they do break and can’t easily be fixed.

(I wonder why anyone hasn’t actually pinged @gchen yet?)

The fact that restic has its own REST API integration through Rclone serve, shows the correct path Duplicacy should take. Perhaps it can adopt the same API and integrate better with Rclone? @gchen