Disk Image - Technical Write Up

Overview

Using this Protected Item type will incur an add-on charge.

This feature requires Windows 7, or Windows Server 2008 R2, or later.

Magnus Box supports taking disk image backups.

This backup type is only applicable when running on Windows. Disk Image backup on other operating systems is not currently supported by this Protected Item type.

When using the "Disk Image" Protected Item type, on the Items tab, you can select any currently-attached drives for backup or individual partitions from any drive. It is possible to select "all drives" and exclude individual disks or partitions.

Any change to the partition structure of a drive will cause that drive to be recognized differently in Magnus Box. If you had selected such a drive, Magnus Box will warn you that the drive can no longer be found. You would need to reselect the drive and/or partitions in the Magnus Box app interface.

Magnus Box feeds raw data from each disk partition directly into its chunking deduplication engine. The disk image is deduplicated, compressed, and encrypted as it is being saved to the Storage Vault. No extra temporary spool data is generated and no additional disk space is required.

The backed-up disk image data will deduplicate with other data inside the Storage Vault. A 'Files and Folders' type backup of the same data volumes should achieve a high degree of space savings. The effectiveness of any such deduplication may be negatively affected by: (A) filesystem fragmentation on the physical volume; and/or (B) small file sizes.

<code>Magnus Box does not currently allow additional file exclusions within a partition. A future version of Magnus Box may allow selecting files to exclude from supported filesystems (NTFS and FAT32). The files will appear to exist on the resulting disk image but contain only compressed zero ranges, saving disk space.

Unused disk sectors

On supported filesystems, Magnus Box will exclude unused space from the disk in the backup image. Unused space is represented as zero ranges, that are compressed during the backup phase. When restoring the disk image, the file will include uncompressed zero ranges. Please see the "Supported volume types" section for more information about what filesystems are compatible with this feature. You may disable skipping free space by enabling the "Include unused disk sectors for forensic data recovery" option.

The disk must be set as Online in Windows for Magnus Box to exclude unused space. If the disk is set as Offline in Windows, Magnus Box is unable to exclude free space, even from a supported filesystem. You can change a disk's Online/Offline state from Disk Management ( diskmgmt.msc) or from diskpart.

If a disk extent does not contain a filesystem (e.g. if it is a raw byte range), then Magnus Box is unable to determine which disk sectors are needed. If you select a "Raw byte range" extent, it is backed up in its entirety, even if the "Include unused disk sectors for forensic data recovery" option is selected. If the raw data contains mostly zero bytes, it will be highly compressed during the backup phase and when stored as chunks in the Storage Vault; however, if the raw data contains mostly random data, it will not compress well.

Magnus Box always skips backing up the pagefile of the booted Windows installation ( pagefile.sys / swapfile.sys), even if the "Include unused disk sectors for forensic data recovery" option is enabled.

Supported volume types

Please refer to the following table of filesystem support notes:

Filesystem / Skip unused space / Consistency

NTFS (Microsoft) / Yes / Snapshot
ReFS (Microsoft) / Yes / Snapshot
FAT32 (Microsoft) / Yes / If the volume is not in use
exFat (Microsoft) / Yes / If the volume is not in use
UDF (Microsoft) / No / If the volume is not in use

Third-party filesystem drivers (e.g. WinBtrfs, Ext2Ifs, Paragon Linuxfs, ZFSin) have not been officially tested against Magnus Box.

Please refer to the following table of special volume type notes:

Volume type: Basic disks

Supported: Yes

Notes: Fully supported

Volume type: Dynamic disks

Supported: Yes

Notes: The underlying volume will appear as "Raw byte range". For a span or striped volume, you should make sure to only select the dynamic volume for backup, not the underlying raw disk. Please also note that Dynamic disks are deprecated in Windows 8 and above.

Volume type: Storage Spaces

Supported: Yes

Notes: The underlying volume will appear as "Orphaned volume". You should make sure to select only the Storage Space for backup.

Volume type: Bitlocker

Supported: Yes, while unlocked

Notes: The backup can succeed if the Bitlocker volume is unlocked. If the Bitlocker volume is locked, it should be unlocked before running the backup job, otherwise, you may experience an error This drive is locked by BitLocker Drive Encryption. You must unlock this drive from Control Panel.. The resulting partition backup is not protected by Bitlocker and you may extract single files from it without the Bitlocker encryption key, as described below. If you restore to a physical partition, you may wish to re-enable Bitlocker after restoring, via the Windows Control Panel.

Volume type: Cluster Shared Volume

Supported: not tested

Notes: not tested

Volume type: Truecrypt / Veracrypt

Supported: not tested

Notes: not tested

Please refer to the following table of physical media notes:

Physical media: Hard drive (512n)

Supported: Yes

Notes: Fully supported

Physical media: Hard drive (AF)

Supported: Yes

Notes: 512e and 4Kn (Advanced Format) harddrives are supported.

Physical media: Mounted VHD / VHDX

Supported: Yes

Notes: Fully supported

Physical media: Removable USB drive

Supported: Yes

Notes: Some removable drives cannot be completely offlined by the operating system; a restore operation back to the physical removable USB drive may be interrupted by other programs on the PC.

Physical media: Remote iSCSI LUN

Supported: not tested

Notes: not tested

Physical media: Mounted ISO

Supported: No

Notes: Only harddrive (HDD / SSD) disks are supported.

Physical media: Optical drive

Supported: No

Notes: Only harddrive (HDD / SSD) disks are supported.

Physical media: Floppy drive

Supported: No

Notes: Only harddrive (HDD / SSD) disks are supported.

Please refer to the following table of partition table notes:

Partition table: MBR

Supported: Yes

Notes: Fully supported, including Extended partitions (EBR)

Partition table: GPT

Supported: Yes

Notes: Fully supported

Consistency

Magnus Box tries to take a VSS snapshot of the selected partition (without invoking any specific writers for quiescence). If this succeeds, the partition backup is crash-consistent.

Magnus Box tries to lock the volume handle. If this succeeds, the partition backup is crash-consistent.

Otherwise, Magnus Box will print a warning to the job log, and back up the partition in a rolling way. The backup may be inconsistent if other processes are writing to the partition at the same time.

Restoring

Magnus Box stores the disk image files in VMDK format. You can restore these files normally using Magnus Box.

There is one plain-text VMDK descriptor file representing metadata about the whole drive, plus separate raw image files for each partition's extent on the disk.

Partitions of the disk that were not selected for backup are represented as zero extents in the VMDK descriptor file. This means the restored disk image appears to have the full disk size, even if only a small amount of partitions inside it were selected. The zero extents will be compressed inside the Storage Vault.

On Windows, the Magnus Box Backup desktop app offers the option to restore the disk images either back to physical partitions, or as files.

Recovery of single files, with spooling

You can restore the VMDK disk images and then extract single files from them.

At the time of writing, we recommend the following software:

7-Zip

Free and Open Source, Windows (GUI) and macOS / Linux (command-line)
Can open VMDK disk descriptor and also the individual extent files
Supports many filesystems, including NTFS, FAT32, EXT 2/3/4, UDF, HFS, SquashFS
Known issues:
- When loading the VMDK disk descriptor directly instead of the extent files, if no partition table is present (i.e. "Raw byte range" containing the MBR/GPT area at the start of the disk was not selected for backup) then the descriptor will only show an interior 'disk.img' file instead of partition contents.
- You can work around this issue by opening the individual partition extent files.
- Early versions of 7-Zip had only limited support for disk image features. Please manually ensure your 7-Zip installation is up-to-date, as 7-Zip does not have a built-in software update feature.

DiskInternals Linux Reader

Freeware, Windows-only
Despite the product name, also supports Windows filesystems (NTFS, FAT)
Can mount VMDK files as a drive letter
- from the menu > Drives > Mount Image > "VMware virtual disks (*.vmdk)"
Known issues:
- Fails to open the VMDK disk descriptor if there is junk data in "Raw byte range" areas.
- You can workaround this issue by editing the descriptor file to replace these with zero extents.
- e.g. edit disk.vmdk change RW 16065 FLAT "disk-f0000.vmdk" 0 to RW 16065 ZERO

Passmark OSFMount

Freeware, Windows-only
Can mount VMDK extent files as a drive letter
Known issues:
- When loading the VMDK disk descriptor directly instead of the extent files, the disk partitions can be discovered, but mounting fails - both of the individual partitions and also when attempting to mount the VMDK as a raw disk ("Physical Disk Emulation" mode)
- You can workaround this issue by selecting the individual extent files to mount (works using "Logical Drive Emulation" mode)

ImDisk Virtual Disk Driver

Freeware, Windows-only
Can mount individual RAW extents
Can parse the VMDK disk descriptor, scanning for disk volumes, and allows mounting them individually
Not able to mount the VMDK as a whole physical disk, only able to mount its discovered volumes

VMware Workstation

Commercial software with free trial available (Windows / macOS / Linux)
Has a feature to mount VMDK files as a local drive letter. From the "File" menu, choose "Map Virtual Disks"
See more information in the VMware Documentation (docs.vmware.com).

Guestfs

Free and Open Source (Linux-only, command-line)
Install the guestfs-tools package (Debian/Ubuntu: libguestfs-tools, SuSE: guestfs-tools, RHEL/CentOS/Fedora: libguestfs-tools-c)
Supports mounting the VMDK disk descriptor file using an unprivileged FUSE backend
- Usage: guestmount -a disk.vmdk

Loop device

Free and Open Source (part of the Linux kernel, command-line)
Use the losetup tool (from util-linux)
Supports mounting individual partition extents, but not the VMDK disk descriptor file

Recovery of single files, without spooling

<code>Depends on the 'live mount restore' planned feature

Booting into a recovered Windows OS installation

When migrating a Windows OS installation to different hardware, any products which use hardware identifiers as a software licensing component may lose their activation status. This includes, but is not limited to

Windows OS activation, and
Magnus Box device detection

The "C:" does not contain everything needed to boot an operating system. For best results when creating a bootable image, you may wish to ensure that your backup includes

the disk's non-partition space (that includes the GPT/MBR partition table)
the "System Reserved Partition", if present (that contains the volume boot record)
the EFI ESP partition, if present (on GPT disks and/or UEFI-booting machines)

Windows 8.1, Windows 10, and later

Current versions of Windows do generally handle being booted on dissimilar hardware without any issues.

Earlier versions of Windows

When you boot a Windows OS installation, it may automatically become specialized for the running hardware (physical or virtual). This improves performance but can prevent the same OS installation from booting on different hardware if the hardware is sufficiently different. The tolerable differences depend on the hardware in question.

If you experience errors booting a backed-up Windows OS disk image on different hardware (physical or virtual), it may be necessary to prepare the Windows installation for hardware-independence. You can do this by running sysprep inside the installation before taking the disk image; or, you can do this by booting a Windows recovery environment, mounting the image, and running sysprep against the attached disk.

The sysprep tool is installed in the C:\Windows\system32\Sysprep\ directory and is available on all Windows SKUs. From Windows 8.1 onward, its GUI is deprecated in favor of command-line use.

Filesystem smaller than the target volume

When restoring a smaller partition into a larger one, Magnus Box will automatically extend the restored filesystem to fill the target partition. This feature is available on Windows if the filesystem driver supports it (the NTFS and ReFS file systems).

In other cases, the result will be a large partition containing a small filesystem. It appears to have the large size in Disk Management (that looks at the partitions) but the small size in This PC (that looks at the filesystem). The extra space from the new larger partition cannot be practically used until the filesystem is extended, to fill the partition around it.

On Windows, you can independently repeat Magnus Box's attempt at manually extending the filesystem to fill its containing partition by

opening Command Prompt as administrator
run diskpart.exe
type list volume
Identify the target volume from the list, and then type select volume TARGET_NUMBER
type extend filesystem

On Linux, you can resolve this issue by using the ntfsresize command.

Filesystem larger than the target volume

Magnus Box does not support restoring a large backed-up partition into a smaller physical partition. If you are trying to do this, please shrink the partition using the OS's partition manager prior to performing the backup.

Recovery to physical hardware

In order to restore to physical hardware, the target disk or partition should be unmounted. Magnus Box Backup may be able to do this automatically from your current booted OS if no programs are using the target drive (e.g. for a non-boot drive); but in order to restore to your boot drive, you should first reboot the PC into a recovery environment.

The Magnus Box desktop app supports creating a USB Recovery Media from the wizard on the Account screen.

The following options are available:

WinRE

WinRE is the default option for creating USB Recovery Media within the Magnus Box application.

Selecting this option allows you to create a minimal USB Recovery Media based on the Windows Recovery Environment. It requires a removable USB drive of at least 2GB in size and no more than 32GB in size (a future version of Magnus Box will remove the size ceiling). The size requirements may be larger if additional drivers get installed into the image.

This option requires that Windows Recovery Environment is installed and available on your PC. If it is not installed, you may be able to install it via the reagentc /info command.

If you choose to create a WinRE drive from inside the Magnus Box desktop application, the resulting USB drive is created as follows:

Choose an available removable USB drive
Select options
- You can choose whether the current OS drivers are embedded in the image
- This feature extracts in-use third-party drivers from the current OS using the dism /export-driver technology. The exact selected drivers may depend on your running OS. In our experience, it mostly includes OEM drivers. The included drivers could be of any type (chipset/network/graphics/audio/usb/pcie/storage/...). There are no guarantees about what drivers will be added, but it should generally be helpful in making sure you can use the device.
The drive is created
- No additional download is required to create the drive
- The drive uses a hybrid MBR/EFI boot and should boot correctly on both MBR and UEFI PCs
- The drive uses the Microsoft ntldr bootloader and should boot correctly on UEFI PCs requiring Secure Boot. If you experience issues booting the USB Recovery Media drive, you could try to temporarily disable Secure Boot from your UEFI firmware menu
- The drive preserves the custom branding of the installed Magnus Box Backup application
- The drive impersonates your own Device ID and will appear to Magnus Box as the same device (when booted on the same physical hardware)
- The drive will be either x86_32 only, or x86_64 only, depending on your installed Windows OS version
- When booting the drive, the Magnus Box desktop app will appear directly. You can use Magnus Box to restore data. When exiting Magnus Box, the Windows Recovery Environment will appear, allowing you to perform any other pre-boot tasks (e.g. boot repair or access Command Prompt) before rebooting the PC normally.

Limitations

Some features are unavailable from inside the created WinRE USB drive:

Wifi support
- Workaround: Connect to the network via wired Ethernet instead
VSS for backup operations
- Workaround: It should not be necessary to use VSS for backup operations from inside the WinRE boot environment
The Windows Disk Management GUI
- Workaround: Use diskpart commands (see below)

When using a USB-boot environment, the available hard-drives are enumerated as-found. If a local-path Storage Vault has been used for backups, sometimes the listing of drives will correspond to drive-letter-mappings in the existing Windows system, but when it does not, the drive letters will need to be explicitly set.

Setting the newly-enumerated drive-letters:

Boot using the USB
Drop into the command-line via 'Tools' -> 'Command Prompt'
type DISKPART to start the program
type LIST VOLUME to show all available volumes and their current drive letters
identify which volume is the one which needs to be reassigned the label 'F' (e.g. volume 5)
type SELECT VOLUME 5
type ASSIGN LETTER=F
type LIST VOLUME to verify

Usage of UNC paths for Storage Vault locations, rather than drive-letters, normally avoids this issue

The resulting WinRE USB drive is based on your PC's version of WinRE. WinRE is provided and updated by Microsoft and contains a version of the Windows kernel that is specific to the latest feature upgrade (e.g. 1903 / 1909 / 2004). For best results when using the "fix Windows boot problems" feature after a full disk restore, you should avoid using an old USB Recovery Media drive for a newer version of Windows (e.g. using a 2004-based WinRE should be able to boot-repair a 1903-based Windows installation, but perhaps not vice-versa).

Windows To Go

Windows To Go is an alternative option for creating USB Recovery Media within the Magnus Box application.

Selecting this option allows you to create a full Windows boot environment. It requires an external hard drive of at least 32GB in size.

This option requires the Portable Workspace Creator ( pwcreator.exe) to be installed and available on your PC. This tool is included in Windows Server 2012, Windows 8 Pro, Windows 8.1 Pro, Windows 10 Pro but was removed in Windows 10 update 2004 owing to the difficulty of deploying critical software updates to this platform.

No customizations are applied to the generated Windows To Go boot drive. You should boot into the drive, then install Magnus Box normally and use it to perform recovery operations such as restoring data.

Other boot environment

You may also create a recovery environment in any other way. Windows or Linux can be used as a suitable recovery environment. Some possible methods include

creating a Linux bootable USB drive, or
using a third-party tool like Rufus to create a Windows To Go drive, or
using recovery media from your PC OEM vendor (e.g. Lenovo / Dell / HP)

In these cases, you will need to manually launch the Magnus Box app once booted into the recovery environment.

Restore from Windows boot environment

From the Windows boot environment, run Magnus Box, and open the Restore wizard. The Restore wizard inside Magnus Box allows restoring the backed-up disks and partitions directly to your physical disks and partitions, without requiring any temporary spool space.

You can use the "edit" button to repartition the local drives using Windows Disk Management. After doing so, use the "refresh" button to refresh the local disks and partitions for restore.

To do so:

Select a backed-up disk or partition to restore, from the left-hand pane
Select a target disk or partition to write to, from the right-hand pane
Click the "Add to restore queue" button
Repeat steps 1-3 as necessary
Click the "Restore" button to begin the restore job.

Restore from Linux boot environment

To restore an entire disk, with spooling:

Restore all the *.vmdk disk image files to a spool drive.

Convert the main vmdk descriptor file to a physical drive, using the following command: qemu-img convert disk.vmdk -O raw /dev/sdx

Alternatively, you can mount the main vmdk descriptor file as an NBD volume if your kernel has NBD support (you may need to modprobe nbd first):

<code>qemu-nbd --connect /dev/nbd0 disk.vmdk dd if=/dev/nbd0 of=/dev/sdx bs=8M status=progress

To restore an entire disk, without spooling:

Restore just the disk.vmdk file (without the data extents), and open it in a text editor in order to read the partition sizes.
Recreate partitions to the exact target size.
Then you can restore single partitions without any local spool disk, using the "Program Output" restore option, and selecting only a single partition file for restore: dd of=/dev/sdx1 bs=8M

To restore a single partition, with spooling:

Recreate a partition to the exact target size.

Restore the target extent file (e.g. disk-f0000.vmdk)

Use dd to clone the selected extent file (e.g. disk-f0000.vmdk) to a physical partition (e.g. /dev/sdx1) as follows: dd if=disk-f0000.vmdk of=/dev/sdx1 bs=8M status=progress

To restore a single partition, without spooling:

Recreate a partition to the exact target size.

Select the file for backup, and use the "Program Output" restore option to stream the file into a command like dd of=/dev/sdx1 bs=8M, choosing a single partition only

<code>A future version of Magnus Box will add built-in support for physical disk restores from a Linux boot environment.

Recovery to local VM

You can attach the *.vmdk disk image files to a new- or existing Virtual Machine. If the disk image contains a Windows OS installation, it may be bootable.

Virtualization platform / Supports *.vmdk file format

VMware / Yes

QEMU / Yes

Virtualbox / Yes

Hyper-V / No - must convert to VHD or VHDX format

If your PC boots using EFI - for instance, if the source disk contains an EFI System Partition (ESP) - then you should configure the VM to boot in EFI mode ("Generation 2" in Hyper-V). Otherwise, you should configure the VM to boot in "Legacy" / MBR mode ("Generation 1" in Hyper-V).

Recovery to cloud server

You can upload the *.vmdk disk image files to a cloud provider. Depending on the cloud provider's capabilities, it may be possible to boot a new VM from them, or to attach them as extra disks to an existing VM.

If the disk image contains a Windows OS installation, it may be bootable. Not all cloud providers support booting Windows OS installations.

Provider: Amazon EC2

Supports *.vmdk file format: Yes

Information:
https://aws.amazon.com/ec2/vm-import/

https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-image-import.html

Provider: Azure

Supports *.vmdk file format: No - must convert to VHD format

Information: https://docs.microsoft.com/en-us/azure/virtual-machines/windows/upload-generalized-managed

Provider: DigitalOcean

Supports *.vmdk file format: Yes

Information: https://blog.digitalocean.com/custom-images/

Provider: UpCloud

Supports *.vmdk file format: Yes

Information: https://upcloud.com/community/tutorials/import-vmware-images/