There was no code change regarding the file backend (duplicacy_filestorage.c), but version 2.3.0 was built with go 1.12. I’m not sure if that is the reason.
I’m confirming that I experience the same issue with an SMB mount to a NAS from my laptops. In the destination, I see the .tmp file, as well as a file without the .{ramdom}.tmp extension. The two files differ, so it doesn’t look like I should mess with either of them.
From the client experiencing the issue, I’m on OS X Mojave, and this was working a couple weeks ago with what I presume was 2.2.3. The current version in ~/.duplicacy-web/bin is 2.3.0.
Are you seeing the same error message? That is, the permission issue is with the sync operation.
If you are seeing two files with the same hex name but one with the .tmp suffix, it could be the move operation that was failing. You can delete the .tmp one - both files should contain the same chunk, but they were encrypted with different random keys.
Yes, I see the same “Operation not permitted” message. Though I have full permission to the destination folder, so I don’t know that it’s actually a permission issue rather than an unsupported call.
The error I receive:
2019-12-10 19:27:16.406 ERROR UPLOAD_CHUNK Failed to upload the chunk 8c394bf54eb6771037c0de9553c0a024f182df82464c15361252f955631d1a98: sync /Volumes/storage/Backups/duplicacy/chunks/8c/394bf54eb6771037c0de9553c0a024f182df82464c15361252f955631d1a98.xhbuzpnv.tmp: operation not supported
Failed to upload the chunk 8c394bf54eb6771037c0de9553c0a024f182df82464c15361252f955631d1a98: sync /Volumes/storage/Backups/duplicacy/chunks/8c/394bf54eb6771037c0de9553c0a024f182df82464c15361252f955631d1a98.xhbuzpnv.tmp: operation not supported
I’ll try cleaning up all of the .tmp files in the destination directory, and see if that changes anything. The big concern for me is whether or not the .tmp files contain any data that could be missing in the other files, and therefore could be useful for restore at some point.
After renaming the existing .{random}.tmp file, a new .{randon}.tmp file appeared, and I received the same error message. I have confirmed that from this client, though, the destination file does not exist yet. So the .{random}.tmp file has no file with the hash component without the various tmp extension formats.
I can create a file, and rename a file from the command line, on the client I’m backing up from, in the destination folder used by Duplicacy.
$ echo “foobarbaz” > /Volumes/storage/Backups/duplicacy/chunks/8c/DELETEME.tmp
$ mv /Volumes/storage/Backups/duplicacy/chunks/8c/DELETEME.tmp /Volumes/storage/Backups/duplicacy/chunks/8c/DELETEME
$ ls -ahl /Volumes/storage/Backups/duplicacy/chunks/8c/DEL*
-rwx------ 1 tom 10 Dec 10 19:40 /Volumes/storage/Backups/duplicacy/chunks/8c/DELETEME
I’m not too sure what’s happening here at this point. I’ve tried verbose and debug mode, and the only error message received is
2019-12-10 19:39:17.284 ERROR UPLOAD_CHUNK Failed to upload the chunk 8c394bf54eb6771037c0de9553c0a024f182df82464c15361252f955631d1a98: sync /Volumes/storage/Backups/duplicacy/chunks/8c/394bf54eb6771037c0de9553c0a024f182df82464c15361252f955631d1a98.uukuqxas.tmp: operation not supported
From what I can find, this seems to be the reason.
It looks like this might be an an incompatibility between macOS and most SMB servers (e.g., Samba <4.8) introduced by changes in go 1.12.
go 1.12 release notes: Go 1.12 Release Notes - The Go Programming Language
File.Syncnow usesF_FULLFSYNCon macOS to correctly flush the file contents to permanent storage. This may cause the method to run more slowly than in previous releases.
@372800b839af73b7fc0b what is the NAS in this instance? Seems like Samba didn’t add support for fullsync until Samba 4.8; so if you can upgrade Samba, that might be a potential fix.
Additional reading:
That seems like a likely candidate. But I’m running Samba 4.10, and I have fruit:advertise_fullsync = true enabled in my Samba config with no change.
Either way, since the comment above says no change has happened to file code and that the only appreciable change is Go 1.12, and we see references from other utilities experiencing the same issue, this seems like a decent place to start looking for a fix.
@gchen It looks like anywhere a .Sync() is called on a file descriptor, the error check should contain code that attempts to see if Sync() is supported by the platform. If you take contributions from the community, I could probably spend some time on Thursday evening grepping through the CLI source and adding something simple.
Alternatively, we could do something I’ve had to do previously, and break low-level file handling code out into platform specific modules. I’ve got one package where I’ve some something like filestuff.go and filestuff_darwin.go because I use O_DIRECT flags on everything except OS X.
Indeed the server I first tested on was Ubuntu 18.04 which has Samba 4.7.6+dfsg~ubuntu-0ubuntu2.1.
However, I just tested on a Debian 10 (Buster) which has Samba 4.9.5+dfsg-5+deb10u1, and I have the same issue.
1 Like
Are you sure that’s the correct setting? It looks like some earlier patches for Samba may have used that, but based on the documentation I can find on samba.org the setting to advertise fullsync is fruit:time machine = yes (reference).
What about after setting fruit:time machine = yes on the share?
Still not working.
cat /etc/samba/smb.conf
...
[Docs]
path = /home/share
public = no
valid users= vlad @sambashare
force group = sambashare
browseable = yes
writable = yes
create mode = 0664
directory mode = 0775
vfs objects = catia fruit streams_xattr
fruit:time machine = yes
...
$ duplicacy2.3.0 init VMac /Volumes/Docs
Failed to configure the storage: sync /Volumes/Docs/config.stvxkmap.tmp: operation not supported
$ duplicacy2.2.3 init VMac /Volumes/Docs
/Users/vlad/Dropbox will be backed up to /Volumes/Docs with id VMacInteresting. I set fruit:time machine = yes on the share, and it seems to work. @vlad did you restart or stop and start the smbd service? Using the stop/start trick worked for me, but restart didn’t seem to.
1 Like
That is very strange. Stop and start seems to work just fine.
I can now confirm that with this setting duplicacy 2.3.0 works on a samba storage.
1 Like
Sorry to pull up an old thread but I’m having this issue as well. Unfortunately I’m running a Windows server so I’m not sure it’s even possible to add time machine support.
Are my only options to run duplicacy 2.2.3 or run samba through WSL?
This has been fixed by Fix "Failed to upload the chunk ... sync ...: operation not supported" by fbarthez · Pull Request #590 · gilbertchen/duplicacy · GitHub, which has been merged and will be included in a new CLI release available later tonight.
Thanks! Somehow missed that on my first search.
I’m still running the tests for the new CLI version. Will try to release it tomorrow.
It works, thank you!