drivers/block/sunvdc.c | 26 ++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 1 deletion(-)
Since commit a11f6ca9aef9 ("sunvdc: Do not spin in an infinite loop
when vio_ldc_send() returns EAGAIN"), Linux guests in LDOMs see
occasional spurious write failures that manifest as filesystem
corruption. The failure signature is:
sunvdc: vdc_tx_trigger() failure, err=-11
I/O error, dev vdiskc, sector ... op 0x1:(WRITE) ...
vdiskc: writeback error on inode ...
The buffered writeback data is silently dropped. A revert was
rejected [1]; Jens proposed requeuing instead [2], but that patch,
tested in October 2025, appeared to leak memory and the effort
stalled [3].
The leak is real and diagnosed: __send_request() maps the request's
pages into the LDC channel's fixed-size map table (ldc_map_sg())
before ringing the doorbell, and the trigger-failure path never
unmaps them. A descriptor whose doorbell was never sent never
completes, and since dr->prod is not advanced, vdc_requeue_inflight()
never visits it either. With the requeue approach every retry remaps
the same descriptor, orphaning the previous mapping; the 8192-entry
per-channel table drains (contiguous allocation makes large requests
fail first as it fragments) and the disk eventually dies with
-ENOMEM. Mainline leaks the same way, once per spurious EIO.
Patch 1 fixes the leak (also a standalone mainline fix). Patch 2 is
Jens' requeue patch, unchanged, on top.
Why the bounded busy-loop cannot be "tuned" instead: the LDC tx
queue holds 128 doorbell messages while the dring holds 512
in-flight descriptors. Any writeback burst of >128 requests
(ordinary for XFS multi-stream writeback) overflows the message
queue whenever the vds service thread in the primary domain is busy
for a few milliseconds - e.g. one txg sync on the backing pool.
This is normal flow control, not an error condition; OpenBoot's own
disk driver retries it indefinitely.
Testing on an UltraSPARC T4 LDOM (Debian sid guest, 8 vcpu, disk on
a zvol served by a Solaris 11.4 primary), workload = 8-way parallel
kernel-tree copies plus periodic 512-file sync bursts on XFS:
- Unpatched (7.2.2 + nothing): three failure events in 3.6 hours,
each losing writeback data, on an otherwise near-idle host.
- Patched (7.2.2 + this series), identical workload: the same
trigger fired (vdc_tx_trigger message logged) and was absorbed by
the requeue - no I/O error, no data loss, no checksum mismatch in
the verified copies.
Independently, the tester who found the October 2025 memory leak has
re-run his original harness (parallel tree copies, SPARC T4-2, 6.18
plus this series): 20 hours, ~130M IOPS across six cores, 861
requests requeued after EAGAIN timeouts, no memory leak observed [3].
In the original run of that harness, those events surfaced as EIO
and the leaked mappings eventually took the LDom down.
A note on semantics: with this series a permanently unresponsive
backend parks I/O indefinitely (as pre-a11f6ca9aef9 kernels and
Solaris guests do) instead of failing after ~10ms. The per-vdisk
"timeout" property (ldm set-vdisk timeout=N) remains the bounding
knob for deployments that prefer failure over waiting.
[1] https://lore.kernel.org/all/20251006100226.4246-2-glaubitz@physik.fu-berlin.de/
[2] https://lore.kernel.org/all/418310b3-2b77-4534-b2fd-27dcc11e333c@kernel.dk/
[3] https://github.com/sparclinux/issues/issues/2
Stian Halseth (1):
sunvdc: unmap LDC cookies when the descriptor send fails
Jens Axboe (1):
sunvdc: fix -EIO issue due to lack of retries
drivers/block/sunvdc.c | 26 ++++++++++++++++++++++++--
1 file changed, 25 insertions(+), 1 deletion(-)
--
2.53.0
On Tue, 01 Sep 2026 19:39:44 +0200, Stian Halseth wrote:
> Since commit a11f6ca9aef9 ("sunvdc: Do not spin in an infinite loop
> when vio_ldc_send() returns EAGAIN"), Linux guests in LDOMs see
> occasional spurious write failures that manifest as filesystem
> corruption. The failure signature is:
>
> sunvdc: vdc_tx_trigger() failure, err=-11
> I/O error, dev vdiskc, sector ... op 0x1:(WRITE) ...
> vdiskc: writeback error on inode ...
>
> [...]
Applied, thanks!
[1/2] sunvdc: unmap LDC cookies when the descriptor send fails
commit: 0c6da21fa35e03fc74f09895433ccd6d4a9c3530
[2/2] sunvdc: fix -EIO issue due to lack of retries
commit: 5067d4ba713961d8ccea1e06cd4c453793f3121e
Best regards,
--
Jens Axboe
Hi Jens, any chance you could take a look at this series? Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer `. `' Physicist `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
© 2016 - 2026 Red Hat, Inc.