Well, that’s pretty self-explanatory, is it not? is using close to 1.8GB RAM in your case, how much actual RAM+swap does your NAS actually have? If you’re running applications on your device, I’d always recommend to upgrade memory to the max, QNAP has upgradeable memory slots (at least QNAPs I have seen).
Apart from that, you can try the latest git build, not sure if it has any memory optimizations already committed. As a last resort (before splitting up your repos/storages) you can increase swap space, this may help in the short term, but this is not an amazing option for variety of reasons.
I am pretty sure it worked on x86-based QNAPs, not sure about arm ones. But this shouldn’t be a problem, you have a ginormous swap as it is (23GB?!), not sure why. At least at the moment you were running free most of the swap was underutilized.
Are you sure you’re not running something else in parallel to that consumes a ton of memory? That would explain behavior you’re seeing assuming reports its memory usage correctly. It shouldn’t be running out of alloc on 1.7GB used with more than 24GB total memory - unless something else consumed 20GB+ of it at that time.
If my math is correct, the backup run that failed above was handling around 97GB worth of chunks.
So around 38MB is being used for file buffering plus almost 940MB for the OS and apps, leaving only about 93MB free.
Even if it were possible to increase the swap space, it unfortunately wouldn’t help.
Imagine that you’re making pizza in your kitchen. The number of pizzas you can bake at the same time is determined by the size and number of ovens there are. You can bake a few dozen mini bagel pizzas at a time, a dozen bagel pizzas, a half dozen 6-inch personal pizzas, a couple of 12-inch pizzas, or some combination thereof. With enough counter space, you can swap pizzas in and out of the oven as needed, but what you probably cannot do is bake a 3-foot diameter pizza all at once. It’s analogous to how the combination of memory modules (physical memory) + swap space (virtual memory) works in operating systems.
The amount of physical memory that’s available determines how big a program (code + data) can be run at any given time. Some parts of a program can be swapped out along with any data that’s not required for the particular set of instructions currently being executed, but the instructions plus data cannot exceed the physical memory that’s free.
The QNAP TS-431+ has a 32-bit ARM CPU, so the version of Duplicacy for it is about 27.7MB. With such a relatively compact program, even if 90% of the code could be swapped out, it’d only free up around 25MB (enough to hold roughly 6 chunks of data on average).
A few suggestions…
Multi-threaded programs don’t consume much memory with each additional thread, but the chunks that Duplicacy uploads in parallel does impact memory usage. Decreasing the thread count further down from 2 to 1 might be worthwhile since with 5 threads you were averaging only about 8.2MB/s (66 Mbps), so using 2 threads might not be much faster than with 1 thread.
Next, if you have access to cross compiler tools (or an inexpensive Raspberry Pi), run “strip” on the Duplicacy binary to remove the symbols and other data used when running in a debugger (e.g., for the 64-bit Linux version it shrunk by 25%). While it’s not going to free up hundreds of megabytes, when you’re limited by 1GB of soldered RAM, every byte counts.
Splitting up the repository into smaller repositories is an option, but a simpler technique is to use the -filters option for the set command. Whatever your storage target is, add it multiple times with different storage names and filters. You’re still backing up to the same storage destination, but with different filters to limit what’s being backed up during each run. If for example your repository has three subdirectories (Documents, Music, Videos), your .duplicacy\preferences file could contain the following lines:
Cute analogy with pizza, but no, that’s not how virtual memory works. VM operates on a page level (typically 4K bytes), it is individual pages that get mapped out either to physical RAM frames or to the swap space. So you can absolutely run programs that require more memory than you physically have RAM, as long as you have enough virtual memory and not failing very large allocs due to memory fragmentation. None of these cases are applicable here. I mean, reports using 1.7GB when there is only 1GB RAM.
The problem with actually using more memory than there is RAM is that VM will constantly swap in and out pages from much slower swap, thus tremendously slowing down execution time. But in many use cases despite allocating a lot of memory, only small part of it is used at any particular time, and swapping out the other parts back to disk has minimal impact on actual execution, that’s why even with large swap utilization it is not always noticeable slower.
Now, if this NAS is indeed 32bit, then a single process cannot allocate more than 2-4GB of virtual memory directly (without memory mapping) no matter how much total memory is available. The example above is close to this limit, but total alloc is not even exceeding 2GB, so I doubt this is a problem at this time.
I haven’t done a deep dive into how ARM’s MMU (Memory Management Unit) works, but on x86 there’s also memory paging at the hardware level. Add in PAE (Physical Address Extension) and things get even more complicated.
While it’s possible to load programs that are larger than the amount of physical memory, any portion of the code and/or data that spills over into swap cannot be executed by the CPU. The code and/or data needs to be swapped back into physical memory in order for the CPU to access it because the CPU doesn’t know anything about file systems (Windows 10/11 use a pair of hidden swap files, C:\pagefile.sys for regular apps and C:\swapfile.sys for Metro/Modern/Universal apps) and swap partitions (as you already know, Un*x systems can mix and match multiple swap files/partitions).
exists at the application layer so it only sees what looks like contiguous memory space even though it’s in reality a combination of physical memory and swap space. The operating system determines which parts of are not required yet and can be moved to swap. Whenever any parts of its code and/or data is required, the OS retrieves it from the swap files/partitions and loads it into main memory where the CPU can reach it and optionally move it into the CPU cache for even faster repeated access.
Using the pizza analogy again, the oven can’t bake an entire 3-foot diameter pie in one pass, but it can easily handle one 1.5-foot long wedge that fits on a typical 17-inch baking sheet. The operator of the oven is responsible for cutting the pie and moving the baking sheet in/out of the oven for cooking each slice because the oven isn’t capable of doing it by itself. The faster the operator works, and the faster the oven can bake, the more there is to .
Very true, for many applications large swap usage might not be noticeable, especially with modern CPUs having triple-level caches (L1, L2, L3) that are also getting larger with each generation – e.g., the 3D chiplet version of the AMD Ryzen 5900X has 192MB of L3 cache.
At least one use case where swapping to virtual memory is very noticeable is when running multiple virtual machines on a host, especially for high density setups when memory ballooning is used by the guests for overprovisioning.
Swapping to a HDD is pretty slow (about 100,000 times slower compared to accessing RAM), but faster and faster SSD/NVMe are helping to narrow the gap. Some day we might be able to replace DRAM with a future super-high-speed flash memory (I wouldn’t mind having a server with 10TB of RAM that also serves as primary storage ).
According to QNAP, the TS-431+ has a dual-core variant of the ARM Cortex-A15 based on the ARMv7-A core, which in turn is a ARM32 (32-bit) architecture. Interestingly, the ARMv7-A core borrows a memory addressing technique from Intel’s x86 to bump the physical address space to 40-bits (good enough for 1TB of memory).
When using up all available ram on a storage device (let alone spilling into a swap) is touched in the discussion — it’s long as derailed into irrelevant specifics and missed a bug picture: you have to have free unused ram on your nas to get any half-decent performance and responsiveness from your storage device. Using up all that ram evicts disk and/or filesystem cache and performance plummets as disks are very bad at servicing random IO.
But what if your NAS only has 1GB of non-upgradable ram? The solution is painfully obvious: don’t run any user software, let alone memory intensive duplicacy, on it. 1GB is barely enough to provide file services. Barely.
It’s crazy that OEMs heavily market multi-terabyte consumer and SMB grade NAS appliances with budget-friendly hardware specs, then pile on a laundry list of features that include backup services, cloud storage, media streaming, gaming – and not to be outdone by the competition – Docker and/or some kind of hypervisor for virtual machines, often all on a dual-core SoC with 1-2 GB of RAM. While I’m a big fan of squeezing as much as possible out of available hardware, it’s like buying a semitruck powered by a 2-cycle lawnmower engine.
One niche scenario where a NAS with 1GB of RAM would be fine is for archival storage via lightweight network services such as NFS, CIFS, FTP and SFTP (CPUs made within the past 10 years or so have hardware accelerated encryption and compression support). Files are uploaded and just pass thru RAM on its way onto the storage medium so a few simultaneous clients at a time is totally doable.
Ha, I remember running CrashPlan (way worse memory hog than ) on a QNAP with 1GB RAM, for quite a while. The real fun is when you run not only out of physical, but also virtual memory, and system starts to kill random processes and/or kernel panic. Debugging this can be fun for the whole family /s
10/10 if you enjoy the pain, otherwise don’t do it
I blame qnap and Synology marketing that market those toys as ”application servers” and replacement for Dropbox, office365, vm server, backup source and target, torrent client, vpn server, Time Machine, media player, media server, and otherwise replacement of the whole industries. 1GB ram? Never mind that, hope users won’t norice and once they bought — what are they are going to do?
My strong offtopic advice would be to cancel the order or return it, and buy or build the TrueNAS Core server instead. I cannot stress this enough, I’ve wasted enough time (years!) with Synology, their bugs, their superficial support (along with migration to the online ticket system with impossible to get through first line of support drones; actual engineers are great but they are a victim of a direction the company was taken), their gimmicky approach to features, and their false promises, that everytime I see another fellow happily marching into the same bucket of worms face first I die inside a little.
If you really have to have a Synology (due to some weird bet — can’t think of another reason) — get higher end model. Telltale sign with Synology — internal PSU. It will be slightly more expensive (albeit much better value) but will protect you from quite a chunk of headaches caused by hardware issues. You will still have to deal with crappy Synology software but at least hardware won’t be in the way.
This was determined to be a marketing limitation, not a technical one; so it indeed will work, albeit with a limited types of ram (but that’s due to decisions Synology made. For example, single-rank ram likely won’t work)
At least in the US (and I will be surprised if EU laws are more lax) warranty and support cannot be denied on that basis. Anecdotally, when I had 32GB in my DS918+ and asked for support due to NVmE hangs rebooting the system where Synology engineering from Taiwan spent a month remotely debugging the issue on my unit, the only conversation regarding ram was “- Did you run the memory test? - Yes. - OK.” That celeron had the same “total maximum system memory 8GB” nonsense in the datasheet.
How much ram is on your system, how many files in the backup set, which version of duplicacy are you using (does it have this fix Memory usage optimization ?) and does DUPLICACY_ATTRIBUTE_THRESHOLD help?
Disregard this, in the older versions you cloud have set this environment variable to a low value, e.g. 1 and this would reduce the size of attribute data kept in ram. In the current version this is no longer applicable.
It appears it had allocated slightly over 3GB and ran out of memory.
3GB is very little. Check what is consuming ram on your device.
Even without duplicacy, you want to have a lot of unused ram, to facilitate filesystem caching and maintain server responsivenes.
Previously, errors occurred at 10% and 65% progress, but this time it happened at 95%. It looks like a memory leak issue happening during the upload process.
I checked the dashboard and noticed that the used memory has stayed around 6GB.
When I run duplicacy backup -stats, it went up to about 6.5GB and is stable.
(I was not allowed to post multiple images in one reply, sorry.)
is using close to 1.8GB RAM in your case, how much actual RAM+swap does your NAS actually have? If you’re running applications on your device, I’d always recommend to upgrade memory to the max, QNAP has upgradeable memory slots (at least QNAPs I have seen).
there is to 
.
).
