Why are some backups failing with a 404 error during cleanup?
Update, November 16: The fix is now deployed to all Magnus Box Cloud Storage gateways, and we're monitoring backup traffic.
What happened
Since November 12, some backups failed with HTTP 500 errors when uploading to Magnus Box Cloud Storage. The cause was two upload bugs in our storage gateways when working with our S3-compatible storage:
- Uploads larger than 16 MiB could fail with a "Content-Md5 did not match" error.
- Uploads smaller than 16 MiB could fail with a "ContentLength with Body length 0" error after a temporary storage error.
While we worked on it, we temporarily rolled back the gateways. That rollback is why some devices then showed a 404 error during the retention (cleanup) step, which stopped the backup job before a new backup ran.
Where things stand
Our vendor released a patch that fixes both upload bugs, and it's now deployed to all of our storage gateways. No backup data has been lost, and existing restore points are safe and restorable. No manual vault repair is needed, and there's nothing you need to change.
If a device is still failing
Let the device run its next scheduled backup. If it still fails, reply to your ticket or email support@magnusbox.com.
Live updates are at status.magnusbox.com.