Error "too many open files"

What it means

The too many open files error means the operating system refused to give the Magnus Box Application another file handle. A file handle covers open files, temporary files and network connections, and every OS limits how many a process (and the whole system) can have open at once.

It usually happens for one of three reasons:

Another program on the device is using a lot of file handles at the same time as the backup.
The device has a lower limit than the OS default, often set by an admin or a hosting provider.
The backup job itself is using more handles than usual, mostly from temporary cache files during the Building cache phase.

For reference, a normal backup job uses about 10–20 handles for files it's reading, about 10–20 for network connections, plus a variable number of temporary cache files.

Quick fixes (any OS)

Don't overlap jobs. Make sure two backup jobs for the same device aren't scheduled at the same time. See Backup schedules explained.
Move other heavy jobs. If another file-heavy task (antivirus scan, indexing, another backup tool) runs at the same time, move one of them.
Turn on Rescan unchanged files (File and Folder Protected Items only). This cuts down the temporary cache files the app uses. The trade-off: backups can take longer, because unchanged files are read again.
Raise the file handle limit using the steps for your OS below.

If the device has a lower limit than the OS default, find out where it was changed before you raise it. Someone may have lowered it on purpose.

macOS

macOS has a system-wide limit and a per-process limit, and the defaults are fairly low.

Open Terminal and check the system-wide limit: sysctl kern.maxfiles
Check the per-process limit: sysctl kern.maxfilesperproc
To raise them so the change survives a restart, create a LaunchDaemon .plist in /Library/LaunchDaemons/ that runs launchctl limit maxfiles with your new values at boot, then restart the Mac.

See Apple's Terminal User Guide if you're new to editing system files on macOS.

Linux

Linux has a system-wide limit plus per-process soft and hard limits. Defaults are usually high, but they're often lowered on shared hosts, web servers, container hosts and OpenVZ servers. Security modules such as SELinux or AppArmor can also affect them. Child processes inherit the limits of the process that started them.

Check the current limits

Limit for new processes started by your current account: ulimit -n
Find the backup processes: pidof backup-tool
Check the soft and hard limits for one of them: grep files /proc/<PID>/limits
System-wide maximum: cat /proc/sys/fs/file-max
Handles open right now: cat /proc/sys/fs/file-nr (or lsof | wc -l; the numbers may differ slightly)

Raise the per-process limit

The backup service runs under systemd, so the reliable fix is to set the limit in its unit:

Run sudo systemctl edit <backup service name> (find the name with systemctl list-units | grep -i backup).
Add:
```
[Service]
LimitNOFILE=1048576
```
(infinity also works).
Run sudo systemctl daemon-reload, then restart the service.

ulimit -n only changes the limit for processes you start from that shell afterward, and it doesn't survive a reboot, so use it only for a quick test. Only root can raise its own limit.

Raise the system-wide limit

Check whether a lower limit is set in /etc/security/limits.conf, a file in /etc/security/limits.d/, or /etc/sysctl.conf. To raise it:

echo "fs.file-max = 10485760" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

On Debian, sysctl is in the procps package.

Windows

Windows rarely hits this error. The per-process handle limit on 64-bit Windows is about 16 million, but it can be lowered by Group Policy. There's also a per-session limit on files opened over the network (default 16,384); run net config server to see it.

To see how many handles a process has open, open Task Manager > Details, right-click a column header, choose Select columns and turn on Handles.

Still stuck?

Email support@magnusbox.com with the username, device name, OS, the job's date and time, and the output of the limit checks above.