VMware vSphere

This Protected Item backs up VMware vSphere virtual machines by connecting directly to either a vCenter server or an ESXi host. Guest virtual machines do not have to be offline for the backup to happen.

For each virtual machine selected for backup, Magnus Box takes a snapshot of the VM, then streams all necessary files (including the vmx configuration, vmdk disks, and other files) into the Chunking engine, optionally using Changed Block Tracking (CBT) for improved performance. Magnus Box then frees the VM snapshot on the host. The machine running Magnus Box does not require any temporary disk space.

Performance​

Large disk image files are streamed from the datastore to Magnus Box. A large amount of traffic takes place over the internal network. For the best performance, Magnus Box should be installed as a VM on the same ESXi cluster and/or specific host that it is backing up.

You can choose how many VMs Magnus Box backs up at a time. Backing up multiple VMs at a time may increase performance.

Changed Block Tracking can be used to read only blocks that have changed since the last successful backup. Additionally, unallocated disk areas are skipped (Standard and CBT) to improve the backup performance further.

If Changed Block Tracking is used, Magnus Box accesses the VMware datastore using the NBDSSL protocol with fastlz compression mode for improved performance.

Requirements​

• Desktop Client
  • Only available for Windows x86_64 clients
  • Running on the same network with the target ESXi/vCenter
• Server
  • vSphere 6.7 (ESXi or vCenter) or later.

Role​

• Global
  • Enable methods
  • Disable methods
• Datastore
  • Allocate space
  • Browse datastore
  • Low-level file operations (ESXi: FileManagement)
• Virtual machine
  • Provisioning
    • Allow read-only disk-access
    • Allow virtual machine download
  • Snapshot management (ESXi: State)
    • Create snapshot
    • Remove snapshot

Protected Item configuration (desktop client)​

1. Select VMware vSphere on the protected item list.

   

2. Select vSphere API  under Connection and click Configure...  to enter user credentials.

   

3. Populate vSphere credentials. Make sure that the credentials used have the required roles set. See vSphere Connection for details.

   

4. Run Test Connection to check the connectivity of the client to the host.

   

5. Click on the plus sign on the right to start selecting virtual machines. All virtual machines can be backed up by selecting the top-level VMware vSphere. Individual virtual machine selection is also possible as well.

   

6. Select the backup type to use for all selected virtual machines. See options for more details on the possible modes.

   

Details​

vSphere Connection​

Magnus Box connects to the vSphere server using the vSphere's SOAP API (VADP). Magnus Box transfers files from the datastore NBDSSL protocol.

Options​

Latest VM state (Changed Block Tracking)​

• Recommended for best performance.
• Back up only changed disk sectors since the last successful backup (this option can be selected even for the first backup job).

Latest VM state (Standard)​

• Back up the current state of all disks and VM configuration. Past snapshots will be flattened, resulting in a single disk with the most recent state.

Virtual Machine List (web)​

When configuring VMware vSphere Protected Item using the web UI, there are some manual options that can be configured for a more customized selection.

• Include All Virtual Machines: Backup all virtual machines found on the host. Selecting the top-level item in the picker will select all and apply this filter. This is useful for dynamically including new virtual machines in the backup without having to reconfigure the Protected Item.
• Include specified Virtual Machine: Only virtual machines with matching names will be backed up.
• Exclude specified Virtual Machine: Virtual machines with matching names will be skipped.
• If manually specifying a virtual machine, a couple of formats are accepted
  • [datacenter-name] vm-name  to back up a specific VM on a datacenter
  • vm-name  to target all matching virtual machines across all data centers

Notes​

For Changed Block Tracking and Standard mode

• Performance is optimized by seeking over unallocated disk sectors.
• Data transfer occurs over the NBDSSL protocol.
• Separate snapshot files are not included in the backup. The current VM state is flattened in a single disk image.
• Disks with independent disk mode are not supported and will be skipped during the backup process.

Restore​

Magnus Box can restore the vmx, vmdk, and other files to the local disk. You must then copy them to the ESXi datastore and import the VM from the ESXi/vCenter interface.

UPCOMING: Future versions of Magnus Box may support restoring directly to the datastore.

If the backup is made while a VM is running off a snapshot, the resulting disk will be a flattened version of the snapshot branches. Do the following to reimport this virtual machine.

• Update VMX config with scsi0:N.fileName = <file>
• (Optional): Reclaim disk space (vmkfstools -K <vmdk> ) to reset unallocated spaces for improved backup performance.

If CBT was enabled. It is recommended to disable/restart CBT before the first boot.

Granular restore of single files and folders is available for NTFS partitions.

Troubleshooting​

Diagnostic logs​

In case the current logs are not sufficient in diagnosing a backup failure, debug logging can be enabled to add more detailed logs on the backup job. 

• Manually start the failed backup job.
• On the storage vault selection, tick Show advanced options.
• Tick Log Level and change it from Default to Debug.

• Click Backup.

  

The job log should have a more verbose logging being displayed on this backup job.

Enabling CBT inside VMware vSphere​

CBT is an optional feature for virtual machines and disks inside VMware vSphere. To use CBT mode in Magnus Box, you should first enable CBT for VMs and their disks inside VMware vSphere.

1. Stop the running VM.
2. In the VMware vSphere web interface, click the "Edit" button.
3. In the "Edit Settings" dialog, select the "VM Options" tab.
4. Find the "General" section -> find the "Configuration Parameters" section -> click "Edit configuration."
5. Set ctkEnabled to TRUE  to enable CBT or FALSE  to disable CBT.
6. Set scsix: x.ctkEnabled  to TRUE  or FALSE  for each attached disk.
7. If you are disabling CBT, use the datastore browser to delete any *-CTK.VMDK  files inside the VM's directory.
8. Start the VM again.

Common errors and warnings​

No parent snapshot found. Fallback to full backup

• A valid reference backup job is not found for CBT. This usually happens during the first backup in a storage vault or if a previous backup fails. This will result in the current job running a full backup instead of CBT. Then next CBT backup after this should be successful without this message.

VM <name> contains <N> snapshots(s). Snapshots will be flattened during backup

• If a virtual machine is running off a snapshot, the final virtual disk image will be a consolidation of the current snapshot up to the root. The snapshot tree is not preserved, but the current state of the target virtual machine will still be backed up.

Unable to run CBT: disk size changed <old> -> <new>

• The virtual disk size has changed since the last backup and is unable to do CBT backup. The current backup job will fall back to a full backup.

Unable to run CBT: inconsistent change id <old> -> <new>

• Change ID information has changed since the last successful backup and is unusable for CBT. The current backup job will fall back to a full backup.

Unsupported disk mode <mode>

• Independent disk modes (persistent and non-persistent) are not supported and will be skipped. This disk mode does not support snapshots, which is a required part of the backup process.

Unsupported disk backing

• The disk was created using an unsupported format. This disk will be skipped.

Error downloading file: download(<file URL>): 401 Unauthorized

• The credentials used do not have the Datastore role(s) set. Click here to review the required roles and permissions on the target host and virtual machine.

CBT is not enabled on <vm name>

• CBT is configured on the Protected Item but is disabled on the target virtual machine. Click here to enable CBT on target virtual machine.

Disk chain before snapshot
<list-of-vmdk>

New snapshot file
<list-of-vmdk>

• Lists virtual disks associated with the virtual machine before and after the snapshot is taken. This also shows the chain that is about to be backed up in case there are multiple snapshot trees for the virtual machine.