Webdav error: unexpected EOF reading trailer

I’m continuing the webdav discussion from Github here.

I am seeing the following issue:

PS C:\> duplicacy check -tabular
Storage set to webdav://user@name@webdav.pcloud.com/Backup/Duplicacy
Listing all chunks
Failed to list the directory chunks/da/: http: unexpected EOF reading trailer
PS C:\> duplicacy check -tabular
Storage set to webdav://user@name@webdav.pcloud.com/Backup/Duplicacy
Listing all chunks
Failed to list the directory chunks/90/: http: unexpected EOF reading trailer
PS C:\> duplicacy check -tabular
Storage set to webdav://user@name@webdav.pcloud.com/Backup/Duplicacy
Listing all chunks
Failed to list the directory chunks/bd/: http: unexpected EOF reading trailer
PS C:\> duplicacy check -tabular
Storage set to webdav://user@name@webdav.pcloud.com/Backup/Duplicacy
Listing all chunks
Failed to list the directory chunks/: unexpected EOF
PS C:\>

I would not be surprised if this is yet another failure of pcloud’s webdav interface, but perhaps there is something that can be improved in duplicacy’s side?

Today it seems to work, so it was probably pcloud’s problem (not the first time their webdav interface is throwing errors)

The problem seems to be back:

$duplicacy check -tabular
Storage set to webdav://<myusername>@webdav.pcloud.com/Backup/Duplicacy
Listing all chunks
Failed to list the directory chunks/: http: unexpected EOF reading trailer
$

This is not limited to the check command, btw. Also my backups are currently failing.

@gchen, could you explain what the “unexpected EOF” message means? Does it mean that the server starts sending the file and then suddenly stops? Or Is it possible that the server doesn’t even start sending the file?

I’m asking because I need to be very exact with the pCloud support, otherwise they just keep coming up with all kinds of excuses. Is there a simple way I can provoke the above error without using duplicacy (i.e. using more generic webdav command?)

“unexpected EOF” means the server unexpectedly closed the connection. “unexpected EOF reading trailer” means the server unexpectedly closed the connection when the client is expecting the http trailers. So maybe their servers don’t send the trailers in a proper way?

Last night the scheduled backup worked (without me chaning anything). So I won’t bother discussing this with their support right now.

But since I think I mentioned elsewhere that pCloud is not officially supporting WebDAV (because that is what theuir support told me), I should mention that someone else from pCloud claims that they officially do support it here.

So I finally had to take this up with pCloud support and they are asking me to try via another browser (duh) or if I’m not accessing via browser “tell us how you use the pCloud’s WebDAV exactly?”

I want to give them the answer that is most likely going to lead to a solution and that means, in particular, that I give them as few alternative sources for the error. (Unfortunately, this was previously one of their routine answers: e.g. “it’s the fault of the software you’re using”, or: “we don’t support that software” , which is of course nonsense when all I’m asking them is to support webdav.

Anyway, I’m thinking the best might be to describe how duplicacy is accessing the webdav storage at a low level. In this case the problem is occurring when trying to do a check, so I’ll stick to that.

@gchen could you tell me how exactly duplicacy is interacting with webdav?

Edit: To provide some context, here are the most recent failure logs:

Running check command from /srv/NAS/duplicacy/duplicacy-web/repositories/localhost/all
Options: [-log check -storage pCloud -a -tabular]
2020-02-02 02:00:01.957 INFO STORAGE_SET Storage set to webdav://myusername@webdav.pcloud.com/Backup/Duplicacy
2020-02-02 02:00:05.669 INFO SNAPSHOT_CHECK Listing all chunks
2020-02-02 04:01:46.900 ERROR LIST_FILES Failed to list the directory chunks/29/: http: unexpected EOF reading trailer
Failed to list the directory chunks/29/: http: unexpected EOF reading trailer

Running check command from /srv/NAS/duplicacy/duplicacy-web/repositories/localhost/all
Options: [-log check -storage pCloud -a -tabular]
2020-02-03 02:00:01.225 INFO STORAGE_SET Storage set to webdav://myusername@webdav.pcloud.com/Backup/Duplicacy
2020-02-03 02:00:04.249 INFO SNAPSHOT_CHECK Listing all chunks
2020-02-03 10:02:48.625 ERROR LIST_FILES Failed to list the directory chunks/6d/: http: unexpected EOF reading trailer
Failed to list the directory chunks/6d/: http: unexpected EOF reading trailer

Running check command from /srv/NAS/duplicacy/duplicacy-web/repositories/localhost/all
Options: [-log check -storage pCloud -a -tabular]
2020-02-04 18:58:31.933 INFO STORAGE_SET Storage set to webdav://myusername@webdav.pcloud.com/Backup/Duplicacy
2020-02-04 18:58:34.307 INFO SNAPSHOT_CHECK Listing all chunks
2020-02-04 19:02:10.573 ERROR LIST_FILES Failed to list the directory chunks/2c/: http: unexpected EOF reading trailer
Failed to list the directory chunks/2c/: http: unexpected EOF reading trailer

Running check command from /srv/NAS/duplicacy/duplicacy-web/repositories/localhost/all
Options: [-log check -storage pCloud -a -tabular]
2020-02-05 20:23:37.839 INFO STORAGE_SET Storage set to webdav://myusername@webdav.pcloud.com/Backup/Duplicacy
2020-02-05 20:23:40.315 INFO SNAPSHOT_CHECK Listing all chunks
2020-02-05 20:27:08.215 ERROR LIST_FILES Failed to list the directory chunks/2f/: http: unexpected EOF reading trailer
Failed to list the directory chunks/2f/: http: unexpected EOF reading trailer

As expected, pCloud now claim that they can’t see any fault on their side and they are asking for a video that shows how the error occurs. Or some other evidence that they can work with to reproduce the error.

@gchen, how exactly is duplicacy interacting with webdav storage when the check command is listing chunks in a directory?

I am suspecting that the problem has something to one or more of the following

  • the large number of files being listed
  • resource problems on the client (the error only seem to occur on my Linux server, not on my desktop PC)
  • a bug in either duplicacy or Golang in how these requests are handled (When searching for “http: unexpected EOF reading trailer”, Google spits suspiciously many hits related to Golang…)
  • a weakness in pClouds webdav implementation (which probably only becomes a problem when combined with another weakness/bug in duplicacy/Golang)

@gchen: if you look at the following post on the duplicati forum, does it give you any ideas?

Or this discussion on Google Groups

Or this?

Currently the WebDAV backend has a simple retry mechanism, but unfortunately this error doesn’t trigger it. So an easy solution is to retry on this error, so we don’t need to figure out whether it is a bug in pCloud’s servers or in Golang.

I’ll make this fix in a couple of days…

1 Like

Excellent! I’m curious whether this simple fix will solve the problem.