Error: too many open files

The job that keeps failing is a check job. backup jobs tend to work fine.

I have not changed anything with regards to nesting so if the default is 1, it is still 1. Looking at my storage, I see that there is only one more level of directories under ../chunks (folders 0a to ff). Each of these folders only contains chunks, no more folders.

@gchen What more can I do to track this down?

1 Like

Can you send me a full log?

Sent you a PM. Hope you find something interesting.

I believe this commit should fix the problem: Improvements for the WebDAV backend · gilbertchen/duplicacy@e3e3e97 · GitHub

From the log you sent me it is obvious that somehow pcloud returns EOFs when Duplicacy tries to create snapshot subdirectories, and Duplicacy keeps trying causing too many connections (I don’t think connections are leaking; it is just that too many connections are created in a short period of time).

This latest commit fixes two things. First, those snapshot subdirectories actually already exist in the storage – we can find them when listing the snapshot directory and store them in the cache. This should avoid unnecessary calls to CreateDirectory(). Second, when pcloud returns EOF on CreateDirectory() we’ll assume the directory already exists.

I’ll make a new CLI release over the weekend which will include this commit.

2 Likes

Excellent! Sounds like we are getting somewhere!

It wouldn’t have occurred to me that the check command creates directories (or anything, for that matter). Why does it do that?

So the “too many open files” are actually too many connections? Because in Linux everything is a file?

I saw release 2.7.0 on Github but when I start a job on the web-ui it still uses 2.6.2. Isn’t it supposed to automatically pull the latest version?

Edit: okay, it’s a bug. I just saw this:

Wow, you nailed it, congratulations! :tada: The problem is solved. First try with version 2.7.0. went through without error. Yeay.

Would it be adequate to retrospectively file this as a #bug?

The error message is a bit confusing, though:

2020-09-30 01:26:58.483 TRACE WEBDAV_MKDIR Can't create directory chunks/49: Moved permanently; error ignored
2020-09-30 01:27:00.189 TRACE WEBDAV_MKDIR Can't create directory chunks/47: Moved permanently; error ignored
2020-09-30 01:27:01.838 TRACE WEBDAV_MKDIR Can't create directory chunks/38: Moved permanently; error ignored
2020-09-30 01:27:02.963 TRACE WEBDAV_MKDIR Can't create directory chunks/21: Moved permanently; error ignored
2020-09-30 01:27:03.949 TRACE WEBDAV_MKDIR Can't create directory chunks/75: Moved permanently; error ignored
2020-09-30 01:27:04.974 TRACE WEBDAV_MKDIR Can't create directory chunks/0c: Moved permanently; error ignored
2020-09-30 01:27:05.720 TRACE WEBDAV_MKDIR Can't create directory chunks/1f: Moved permanently; error ignored
2020-09-30 01:27:06.533 TRACE WEBDAV_MKDIR Can't create directory chunks/a4: Moved permanently; error ignored
2020-09-30 01:27:07.103 TRACE WEBDAV_MKDIR Can't create directory chunks/91: Moved permanently; error ignored
2020-09-30 01:27:07.940 TRACE WEBDAV_MKDIR Can't create directory chunks/20: Moved permanently; error ignored
2020-09-30 01:27:09.420 TRACE WEBDAV_MKDIR Can't create directory chunks/a5: Moved permanently; error ignored
2020-09-30 01:27:10.758 TRACE WEBDAV_MKDIR Can't create directory chunks/4a: Moved permanently; error ignored

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.