Hi, the below has been written by Opus on request from me.
After dealing with issues when trying to live migrate a VM that has a
windows failover cluster (wsfc) with active roles on it, i encountered
what seems to be a bug. I couldn't decipher it myself, so I had to make
use of Opus to help me figure it out, and to give me a work-around
Opus wrote me a pre-hook script that works as a workaround for this
bug, but it is by no means perfect, and we're missing several
milliseconds, and get a bunch of reservation conflicts before the
script can repair after a migration has happened.
The below is a suggestion by opus as a fix to this issue.
I myself have little to no experience in C, and i hope the below
can help highlight the issue i'm seeing, to confirm wether or not
it is a fixable bug or not.
I am sorry if I'm doing this mailing list wrong.
Opus:
Live migration of SCSI persistent reservations loses registrations on
the destination when the LUN is a dm-multipath device and the guest
holds a reservation. The reservation itself transfers correctly, but
every local initiator port except one ends up unregistered, and the
guest's writes dispatched down the paths of those initiators return
RESERVATION CONFLICT for as long as it runs on that host.
The fix looks like removing a guard that is already in the tree. Details
below.
The mechanism
-------------
scsi_generic_pr_state_preempt() in hw/scsi/scsi-generic.c registers the
guest's key and then self-preempts, with that same key in both the
RESERVATION KEY and SERVICE ACTION RESERVATION KEY fields.
That pair is safe for a single I_T nexus, but libmpathpersist treats the
two service actions asymmetrically. REGISTER goes through
mpath_prout_reg(), which sends it on every local initiator port.
PREEMPT goes through mpath_prout_common(), which sends it down a single
path. SPC then removes every registration holding the preempted key
except that of the issuing I_T nexus, so the PREEMPT deletes the
registrations the REGISTER created on the other initiator ports
microseconds earlier.
The destination is left registered on one initiator while device-mapper
keeps every path in the active path group, so roughly half the guest's
writes conflict.
Evidence
--------
HPE 3PAR over FC, two initiator ports per host (SCSI hosts 9 and 10),
dm-multipath with "reservation_key file" and "all_tg_pt yes",
qemu-pr-helper linked against libmpathpersist. Guest is a two-node
Windows Server failover cluster holding a shared disk with reservation
type 0x05 (Write Exclusive, Registrants Only). LUNs attached as:
-object pr-manager-helper,id=prhelper0,path=/run/pr-helper0.sock
-blockdev node-name=cau0,driver=raw,cache.direct=on,\
file.driver=host_device,file.filename=/dev/mapper/<map>,\
file.pr-manager=prhelper0
ftrace on the destination during one live migration, filtered to PR
IN/OUT on the shared LUN. Timestamps in seconds, last column is the
first three CDB bytes:
361165.665620 host9 lun6 5f 00 00 PR OUT, REGISTER
361165.665666 host10 lun6 5f 00 00 PR OUT, REGISTER
361165.693479 host9 lun6 5f 04 05 PR OUT, PREEMPT, type 0x05
The two REGISTERs are libmpathpersist fanning out across the local
initiators. The PREEMPT goes out on host9 only, and host10's
registration goes away with it:
before PR generation=0xc27, 2 registered reservation keys follow:
0x65e98960001734d
0x65e98960001734d
after PR generation=0xc2c, 1 registered reservation key follows:
0x65e98960001734d
The guest resumes and dmesg on the destination fills with "reservation
conflict", on the paths belonging to host10 and host9 in a roughly even
split, matching the path selector alternating between the two
initiators. Reissuing REGISTER AND IGNORE EXISTING KEY with the same key
through mpathpersist restores the second registration and the conflicts
stop immediately, with no other change.
Affected versions
-----------------
v11.0.0 no re-registration after PREEMPT at all
v11.1.0 re-registers, but only when resv_type == 0
master (9dbf31c5be13) same guard, now with REGISTER AND IGNORE
Every version that implements PR migration is affected whenever the
guest actually holds a reservation.
Why the existing re-registration does not cover this
----------------------------------------------------
b9a3b329d09d ("scsi: register again after PREEMPT without reservation")
added the re-registration because the Linux LIO target removes the
calling nexus' own registration when preempting without a reservation.
That is specific to resv_type == 0, so the guard made sense for it.
The multipath case is not a target quirk and does not depend on
resv_type. It is plain SPC behaviour applied to a REGISTER that
libmpathpersist multiplied across initiator ports and a PREEMPT that it
did not, so it happens on any spec-compliant target for any reservation
type. A guest that holds a reservation, which is the case PR migration
exists to serve, always takes the branch that skips the repair.
It is also independent of all_tg_pt. With "all_tg_pt yes"
libmpathpersist registers once per initiator port, so one registration
is lost on a two-HBA host; with it off it registers on every path, so
the PREEMPT destroys correspondingly more. The ATC_C masking in
scsi_handle_persistent_reserve_in_reply() does not help either, since it
governs what the guest may ask for rather than what libmpathpersist does
on the host below QEMU.
The same pattern came up on this list in February 2026 during the
<linux/pr.h> discussion, from the multipath side: "Preempt the
reservation to move it to an active path, which wipes the registrations
off all the other paths. Then reregister all the active paths again."
2d38eb7c741e ("scsi: clean up persistent reservation state after
migration") handles the mirror image of this on the source host, where
multipathd keeps re-registering a key the destination has preempted
away. The destination-side loss described here looks like the remaining
gap.
Suggested fix
-------------
Make the re-registration unconditional. REGISTER AND IGNORE EXISTING KEY
removes nothing and is idempotent, so running it when the registration
survived costs one PR OUT.
--- a/hw/scsi/scsi-generic.c
+++ b/hw/scsi/scsi-generic.c
@@
if (!scsi_generic_pr_preempt(s, key, resv_type, errp)) {
return false;
}
/*
- * Some SCSI targets, like the Linux LIO target, remove our
- * registration when preempting without a reservation (resv_type is 0).
- * Try to register again but ignore the error since a RESERVATION
- * CONFLICT is expected if our registration remained in place.
+ * Register again after PREEMPT. Two situations require it:
+ *
+ * 1. Some SCSI targets, like the Linux LIO target, remove our
+ * registration when preempting without a reservation (resv_type
+ * is 0).
+ *
+ * 2. On a multipath device libmpathpersist sends REGISTER on every
+ * local initiator port but sends PREEMPT down a single path.
+ * PREEMPT removes every registration holding the preempted key
+ * except that of the issuing I_T nexus, so the registrations
+ * just made on the other initiator ports are removed again. This
+ * happens for every reservation type, not only when resv_type is
+ * 0.
+ *
+ * REGISTER AND IGNORE EXISTING KEY removes nothing, so this is safe
+ * when the registration survived. The error is ignored for that
+ * case.
*/
- if (resv_type == 0) {
- scsi_generic_pr_register_ignore(s, key, NULL);
- }
+ scsi_generic_pr_register_ignore(s, key, NULL);
return true;
}
Fixing it inside post_load matters beyond tidiness: post_load runs
before the guest is resumed on the destination, so an unconditional
re-registration closes the hole with no exposure at all. I am currently
working around it with a hookscript that watches for the signature and
reissues REGISTER AND IGNORE EXISTING KEY, which lands 30-130 ms after
the PREEMPT across repeated migrations. That keeps the failover cluster
healthy, but the guest is running throughout that window and its writes
on the affected paths do conflict. Only QEMU can do this at a point
where no guest I/O is in flight.
Thanks,
Stian Langvann
© 2016 - 2026 Red Hat, Inc.