Out of memory

Magnus Box Backup needs RAM to run. The main cause for this is to hold deduplication indexes; therefore the amount of RAM used is proportional to the size of the Storage Vault.

You might see these error messages:

• runtime: VirtualAlloc of 1048576 bytes failed with errno=1455 on Windows
• 0x5AF ERROR_COMMITMENT_LIMIT: The paging file is too small for this operation to complete. on Windows
• fatal error: out of memory on all platforms

On Linux, when the system is out of memory (OOM), the kernel "OOM Killer" subsystem will immediately terminate a process of its choosing, to free up memory. If you see an error message like signal: killed in Magnus Box on Linux, this means the process was terminated by a user or a subsystem, that might possibly be the OOM Killer. You can check for this in dmesg or kern.log.

You can reduce Magnus Box Backup's RAM usage by trying to limit how much data is in each Storage Vault. For instance, instead of having multiple devices backing up into a single Storage Vault, create multiple Storage Vaults for each device. This will reduce the deduplication efficiency, but it will also reduce the necessary memory usage.

Trade-offs 

Some trade-offs are possible, that can reduce Magnus Box's memory usage at the expense of other system resource types:

Rescan unchanged files 

This option causes Magnus Box to read more data from the source disk, reading less data from the Storage Vault into in-memory indexes. This can have a varied impact on RAM usage, and may be positive or negative depending on the shape of your directories.

Prefer temporary files instead of RAM 

The "Prefer temporary files instead of RAM" option on a backup job schedule will cause Magnus Box to keep indexes in an on-disk database file, instead of a pure in-memory index. The on-disk database file is mapped into pageable memory, that can more easily be reclaimed by the OS when the system is under memory pressure.

Depending on how you measure Magnus Box's memory usage, this option may not immediately appear to have lower memory usage if your measurement includes mmap disk sections. However, the resident working set is reduced.

There is a major performance penalty for using this option (approximately 5x or worse) and it is not generally recommended.