Remote Support Start download

Proxmox Storage Migration Live — Without Downtime

ProxmoxStorageMigrationVirtualisierung
Proxmox Storage Migration Live — Without Downtime

A call on Tuesday morning: the SSD pool of a production Proxmox host is throwing SMART warnings. Two drives are in pre-fail state, and the ERP database and terminal server run on top. A classic maintenance window is ten days out at the earliest — and even that would require six hours of downtime. Proxmox VE has exactly the right tool for this scenario: qm move-disk relocates VM disks between any two storages without the virtual machine going offline for a single second.

In this article we walk through the process using a migration from an aging SSD ZFS pool to a fresh NVMe pool, and add the special case of Ceph to ZFS. We look at the underlying mechanics, discuss throughput control and describe what to do in case of failure.

How live migration works in Proxmox

qm move-disk internally uses the QEMU drive-mirror mechanism. As soon as the command starts, QEMU attaches a second backing store to the running VM. Any new write is mirrored to both storages, while in the background a full copy of the existing content is streamed from source to target. Once the copy is complete and both sides are in sync, QEMU switches the active backing store to the target and detaches the source volume.

To the VM the process is transparent: no pause, no RAM migration, no network disconnect. The most you might see inside the guest is a slight increase in I/O latency during the mirror phase, and only if source or target storage is already heavily loaded.

The distinction between online and offline matters:

  • Online mode (VM running): drive-mirror, transparent, zero downtime.
  • Offline mode (VM shut down): simple block copy, considerably faster because there is no sync overhead — but with an outage.

For production systems only online mode is realistic. Offline mode makes sense for templates, cold snapshots or test systems.

Starting point: SSD pool with SMART warning

In our example a Proxmox VE 8.4 host is running with the following storage configuration:

Storage IDBackendDevicesState
ssd-pool-01ZFS RAID104x 3.84TB SATA SSD2 devices in pre-fail
nvme-pool-01ZFS RAID104x 3.84TB Enterprise NVMenew, empty
backup-nfsNFSTrueNAS sharefor PBS snapshots

ssd-pool-01 hosts eight VMs, including the ERP server (VMID 101, 500 GB system disk + 2 TB data disk), two Windows terminal servers and a Linux file server. The goal is a full migration to nvme-pool-01 during business hours, without so much as a heads-up to end users.

A quick look at the pool state first:

zpool status ssd-pool-01
# pool: ssd-pool-01
# state: DEGRADED
# status: One or more devices has experienced an error resulting in data corruption.
# action: Restore the file in question if possible. Otherwise restore the entire pool from backup.

smartctl -a /dev/sdb | grep -E "Reallocated|Wear_Leveling|Media_Wearout"

As long as the pool is not yet FAULTED, live migration works reliably — reads are served by the healthy mirror partners.

The migration command in detail

The core command is straightforward:

qm move-disk 101 scsi0 nvme-pool-01 --delete 1

What this does:

  • 101 is the VMID
  • scsi0 is the disk slot to migrate
  • nvme-pool-01 is the target storage
  • --delete 1 removes the source volume after a successful sync

For the second disk of the VM analogously:

qm move-disk 101 scsi1 nvme-pool-01 --delete 1

The web UI offers the same function under VM > Hardware > Hard Disk > Disk Action > Move Storage. For the standard case we prefer the CLI because it can be scripted and produces clean log output.

Progress can be observed live — the command blocks until the migration is done and prints a percentage regularly. For detached operation on large volumes we recommend tmux or an at job.

Controlling throughput and protecting production

The Achilles heel of any live migration is the additional load on source and target storage. If the source pool is already degraded, an unlimited copy can saturate the remaining devices and produce latency spikes for end users.

Proxmox allows the migration rate to be capped via --bwlimit (in KiB/s):

qm move-disk 101 scsi1 nvme-pool-01 --delete 1 --bwlimit 200000

200000 KiB/s translates to roughly 200 MB/s — a typical value that is easy for an NVMe target to absorb while not choking the source SSDs. In our customer projects we typically see the best results when we throttle the migration to 100-200 MB/s during core business hours and only ramp up to full speed after 6 PM.

Keeping an eye on iostat, zpool iostat and the host’s Grafana dashboards during the migration is mandatory. Starting the process blind and then walking away is exactly how you cause the outage you were trying to avoid.

Special case: Ceph to ZFS

A somewhat more complex case is migrating away from a Ceph cluster to local ZFS storage — for example when consolidating from three to two hosts and Ceph loses its quorum prerequisite. The command itself remains identical:

qm move-disk 205 scsi0 local-zfs --delete 1

Two points to keep in mind: first, Ceph as cluster storage is usually significantly slower on sequential reads than local NVMe ZFS, so the migration is source-bound. Second, you must make sure the VM is no longer HA-managed after the migration if the target storage is local (not shared) — otherwise the HA manager will fail on the next node outage. Check with:

ha-manager config
ha-manager remove vm:205

For the reverse direction (ZFS to Ceph) the same procedure applies — here the Ceph cluster is typically the faster target, provided it has enough OSDs.

Rollback, failure handling and log verification

What happens if the target storage fills up mid-migration or an NVMe fails? qm move-disk will abort, discard the target volume and the VM continues to run unchanged on the source storage. A real rollback is not required, because the source is never modified until the successful switch.

If you realize after the fact that a migration was not optimal (e.g. target pool too small), simply run another qm move-disk in the reverse direction — also live.

For verification, always take a look at the task log in the web UI (Datacenter > Node > Task History) and at /var/log/pve/tasks/. A successful move task ends with:

transferred 2.00 TiB of 2.00 TiB (100.00%)
TASK OK

In addition, in every customer project we verify after each migration:

  • qm config <vmid> — shows the new storage backend
  • zfs list or rbd ls — volume no longer present on source
  • Read/write test inside the VM (fio or application test)
  • Baseline the SMART values of the target pool

Conclusion

Live storage migration in Proxmox VE is a production-ready tool that avoids unplanned maintenance windows in many cases entirely. Prerequisites are sufficient capacity on the target storage, a well-monitored I/O path and the discipline to set --bwlimit sensibly. Once you have internalized the mechanics, SMART warnings and storage swaps stop feeling like crisis projects and turn into regular operations work.

DATAZONE supports you with planning, execution and monitoring of such migrations — from sizing the new NVMe pool through migration strategy to ongoing monitoring integration. We combine our experience from Proxmox consulting and TrueNAS storage so your virtualization platform stays stable after a hardware swap too. Get in touch — an initial call and inventory review are free of charge: Contact.

More on these topics:

Need IT consulting?

Contact us for a no-obligation consultation on Proxmox, OPNsense, TrueNAS and more.

Get in touch