target/riscv/helper.h | 1 + target/riscv/tcg/insn_trans/trans_privileged.c.inc | 6 +++++- target/riscv/tcg/op_helper.c | 22 ++++++++++++++++------ 3 files changed, 22 insertions(+), 7 deletions(-)
Introduce a new function, check_sfence_vma(), to handle
privilege/hypervisor checks currently performed in helper_tlb_flush(),
and make use of it in that function. Then, add helper_tlb_flush_page()
for flushing only a single TLB page, using check_sfence_vma() inside it
similarly to helper_tlb_flush(). Finally, rework the trans_sfence_vma()
function to use helper_tlb_flush_page() when rs1 != 0.
The second patch also includes the fix for a regression in the
'test_tuxrun.TuxRunRiscV64Test.test_riscv64_maxcpu' test encountered
with the draft version of this series, which hanged indefinitely until
killed when the patches were applied. This was fixed by making sure to
use get_address() to canonicalize the address being passed to
helper_tlb_flush_page().
make-check and make-functional were both run with the patches applied on
master, and no regressions for riscv64 were observed (beyond the one
mentioned as fixed above).
Daniel Barboza helped me with initial triage, direction, and cleaning up
this series for submission - thanks again!
Some background context: as part of the RISE Project[1] mission to
improve the RISC-V software ecosystem, the Yocto Project[2] is receiving
assistance with triaging bugs and enhancing RISC-V support in the core
layer. The 'qemuriscv32' and 'qemuriscv64' targets were enabled as part
of the Yocto's CI system, known as the Autobuilder[3] in 2025, and since
that time various intermittent failures have been observed on the
'ptest' ('package test', where a package's test suites are bundled into
an image and then booted to do basic runtime compatibility testing)
case, 'qemuriscv64-ptest'. These almost always manifest as timeouts in
the guest, usually around heavier and/or more complex test cases, and
they additionally seem to occur only under heavy host system (x86_64)
load.
Comparing to other major architectures like x86_64 and arm64, the
riscv64 guests are always emulated in TCG mode, since the Yocto
Autobuilder cluster does not (yet) have native riscv64 hosts to dedicate
for runtime testing with kvm support. While this is likely a major
factor in why qemuriscv64 runs fail at higher rates than other
architectures, it was noted after compiling the triage data[4] that
local comparisons of the same ptest images running on TCG-mode qemuarm64
and qemuriscv64 showed the latter taking significantly longer to
complete (using some of the test suites which have been observed failing
as examples), even with no other meaningful system load on the host. In
particular, the coreutils case showed over 70% longer runtime for
qemuriscv64 compared to qemuarm64 (~1000s vs ~580s).
Fully investigating the source of this discrepancy was a long process.
For Yocto builds, native QEMU binaries are built from source as part of
the overall image build, so the first place to look was in QEMU's
configuration and boot options. Other suggestions included trying to
disable TCG plugins or alter the QEMU and guest package configs (Yocto's
QEMU is RVA23S64 profile by default, while the runtime packages are
built for riscv64gc). However, these changes either produced no
meaningful effect or actually increased runtime. The next step was to
investigate the QEMU source, which is what ultimately led to this patch
series. It turns out that these changes significantly improve the
reference runtime test case for coreutils in Yocto, although it is still
somewhat slower than in qemuarm64. It remains to be seen whether these
changes to the QEMU source eliminate the intermittent issues in Yocto's
testing, but it is a meaningful improvement which opens the door to
further optimizations for RISC-V.
As an example, compare the before and after results on a local run of
core-image-ptest-coreutils for qemuriscv64:
|============================================================================
|Testsuite summary for GNU coreutils 9.11
|============================================================================
|# TOTAL: 733
|# PASS: 561
|# SKIP: 172
|# XFAIL: 0
|# FAIL: 0
|# XPASS: 0
|# ERROR: 0
|============================================================================
|make[1]: Leaving directory '/usr/lib/coreutils/ptest'
|DURATION: 1049
|END: /usr/lib/coreutils/ptest
|2026-08-31T17:23
|STOP: ptest-runner
|TOTAL: 1 FAIL: 0
versus:
|============================================================================
|Testsuite summary for GNU coreutils 9.11
|============================================================================
|# TOTAL: 733
|# PASS: 561
|# SKIP: 172
|# XFAIL: 0
|# FAIL: 0
|# XPASS: 0
|# ERROR: 0
|============================================================================
|make[1]: Leaving directory '/usr/lib/coreutils/ptest'
|DURATION: 646
|END: /usr/lib/coreutils/ptest
|2026-08-31T16:10
|STOP: ptest-runner
|TOTAL: 1 FAIL: 0
|root@qemuriscv64:~#
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
[1]: https://riseproject.dev/
[2]: https://www.yoctoproject.org/
[3]: https://autobuilder.yoctoproject.org/valkyrie/
[4]: https://docs.google.com/document/d/1HUk-RvMDdW2EKV-qE1VNfZBycPscQUsayEZ5AgJ85C8
---
Changes in v2:
- Renamed sfence_vma_allowed() to check_sfence_vma()
- Squashed patches 2 and 3 together
- Reworked check_sfence_vma() and helpers using it to be consistent with
rest of codebase (e.g. no unnecessary returns)
- Adjusted cover letter to match check_sfence_vma() naming
- Link to v1: https://lore.kernel.org/qemu-devel/20260831-riscv_tlb_page_flush-v1-0-4fc6af95673a@baylibre.com
To: qemu-devel@nongnu.org
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Alistair Francis <alistair.francis@wdc.com>
Cc: Weiwei Li <liwei1518@gmail.com>
Cc: Daniel Henrique Barboza <daniel.barboza@oss.qualcomm.com>
Cc: Liu Zhiwei <zhiwei_liu@linux.alibaba.com>
Cc: Chao Liu <chao.liu@processmission.com>
Cc: qemu-riscv@nongnu.org
---
Trevor Gamblin (2):
target/riscv: extract check_sfence_vma() from helper_tlb_flush()
target/riscv: add helper_tlb_flush_page(), use for sfence.vma with vaddr operand
target/riscv/helper.h | 1 +
target/riscv/tcg/insn_trans/trans_privileged.c.inc | 6 +++++-
target/riscv/tcg/op_helper.c | 22 ++++++++++++++++------
3 files changed, 22 insertions(+), 7 deletions(-)
---
base-commit: c513597761d6fc06690b5b7cf10dfb7ab05815d0
change-id: 20260831-riscv_tlb_page_flush-11a83fe9e7b1
Best regards,
--
Trevor Gamblin <tgamblin@baylibre.com>
Richard,
This tlb_flush change is giving a +20% performance boost in riscv64-tuxrun. From
this:
23/23 func-thorough+func-riscv64-thorough+thorough - qemu:func-riscv64-tuxrun OK 92.71s 4 subtests passed
To this:
23/23 func-thorough+func-riscv64-thorough+thorough - qemu:func-riscv64-tuxrun OK 73.45s 4 subtests passed
Other tests weren't that affected - I take it has to do with the amount of
sfence.vma being issued. They look fine in my own testing with an emulated
Ubuntu host as well.
This is all fantastic but I wonder if we're missing something. If you could
please take a look at these 2 patches and see if we're handling the tlb flushes
according that would be terrific.
Thanks,
Daniel
On 9/4/2026 3:43 PM, Trevor Gamblin wrote:
> Introduce a new function, check_sfence_vma(), to handle
> privilege/hypervisor checks currently performed in helper_tlb_flush(),
> and make use of it in that function. Then, add helper_tlb_flush_page()
> for flushing only a single TLB page, using check_sfence_vma() inside it
> similarly to helper_tlb_flush(). Finally, rework the trans_sfence_vma()
> function to use helper_tlb_flush_page() when rs1 != 0.
>
> The second patch also includes the fix for a regression in the
> 'test_tuxrun.TuxRunRiscV64Test.test_riscv64_maxcpu' test encountered
> with the draft version of this series, which hanged indefinitely until
> killed when the patches were applied. This was fixed by making sure to
> use get_address() to canonicalize the address being passed to
> helper_tlb_flush_page().
>
> make-check and make-functional were both run with the patches applied on
> master, and no regressions for riscv64 were observed (beyond the one
> mentioned as fixed above).
>
> Daniel Barboza helped me with initial triage, direction, and cleaning up
> this series for submission - thanks again!
>
> Some background context: as part of the RISE Project[1] mission to
> improve the RISC-V software ecosystem, the Yocto Project[2] is receiving
> assistance with triaging bugs and enhancing RISC-V support in the core
> layer. The 'qemuriscv32' and 'qemuriscv64' targets were enabled as part
> of the Yocto's CI system, known as the Autobuilder[3] in 2025, and since
> that time various intermittent failures have been observed on the
> 'ptest' ('package test', where a package's test suites are bundled into
> an image and then booted to do basic runtime compatibility testing)
> case, 'qemuriscv64-ptest'. These almost always manifest as timeouts in
> the guest, usually around heavier and/or more complex test cases, and
> they additionally seem to occur only under heavy host system (x86_64)
> load.
>
> Comparing to other major architectures like x86_64 and arm64, the
> riscv64 guests are always emulated in TCG mode, since the Yocto
> Autobuilder cluster does not (yet) have native riscv64 hosts to dedicate
> for runtime testing with kvm support. While this is likely a major
> factor in why qemuriscv64 runs fail at higher rates than other
> architectures, it was noted after compiling the triage data[4] that
> local comparisons of the same ptest images running on TCG-mode qemuarm64
> and qemuriscv64 showed the latter taking significantly longer to
> complete (using some of the test suites which have been observed failing
> as examples), even with no other meaningful system load on the host. In
> particular, the coreutils case showed over 70% longer runtime for
> qemuriscv64 compared to qemuarm64 (~1000s vs ~580s).
>
> Fully investigating the source of this discrepancy was a long process.
> For Yocto builds, native QEMU binaries are built from source as part of
> the overall image build, so the first place to look was in QEMU's
> configuration and boot options. Other suggestions included trying to
> disable TCG plugins or alter the QEMU and guest package configs (Yocto's
> QEMU is RVA23S64 profile by default, while the runtime packages are
> built for riscv64gc). However, these changes either produced no
> meaningful effect or actually increased runtime. The next step was to
> investigate the QEMU source, which is what ultimately led to this patch
> series. It turns out that these changes significantly improve the
> reference runtime test case for coreutils in Yocto, although it is still
> somewhat slower than in qemuarm64. It remains to be seen whether these
> changes to the QEMU source eliminate the intermittent issues in Yocto's
> testing, but it is a meaningful improvement which opens the door to
> further optimizations for RISC-V.
>
> As an example, compare the before and after results on a local run of
> core-image-ptest-coreutils for qemuriscv64:
>
> |============================================================================
> |Testsuite summary for GNU coreutils 9.11
> |============================================================================
> |# TOTAL: 733
> |# PASS: 561
> |# SKIP: 172
> |# XFAIL: 0
> |# FAIL: 0
> |# XPASS: 0
> |# ERROR: 0
> |============================================================================
> |make[1]: Leaving directory '/usr/lib/coreutils/ptest'
> |DURATION: 1049
> |END: /usr/lib/coreutils/ptest
> |2026-08-31T17:23
> |STOP: ptest-runner
> |TOTAL: 1 FAIL: 0
>
> versus:
>
> |============================================================================
> |Testsuite summary for GNU coreutils 9.11
> |============================================================================
> |# TOTAL: 733
> |# PASS: 561
> |# SKIP: 172
> |# XFAIL: 0
> |# FAIL: 0
> |# XPASS: 0
> |# ERROR: 0
> |============================================================================
> |make[1]: Leaving directory '/usr/lib/coreutils/ptest'
> |DURATION: 646
> |END: /usr/lib/coreutils/ptest
> |2026-08-31T16:10
> |STOP: ptest-runner
> |TOTAL: 1 FAIL: 0
> |root@qemuriscv64:~#
>
> Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
>
> [1]: https://riseproject.dev/
> [2]: https://www.yoctoproject.org/
> [3]: https://autobuilder.yoctoproject.org/valkyrie/
> [4]: https://docs.google.com/document/d/1HUk-RvMDdW2EKV-qE1VNfZBycPscQUsayEZ5AgJ85C8
>
> ---
> Changes in v2:
> - Renamed sfence_vma_allowed() to check_sfence_vma()
> - Squashed patches 2 and 3 together
> - Reworked check_sfence_vma() and helpers using it to be consistent with
> rest of codebase (e.g. no unnecessary returns)
> - Adjusted cover letter to match check_sfence_vma() naming
> - Link to v1: https://lore.kernel.org/qemu-devel/20260831-riscv_tlb_page_flush-v1-0-4fc6af95673a@baylibre.com
>
> To: qemu-devel@nongnu.org
> Cc: Palmer Dabbelt <palmer@dabbelt.com>
> Cc: Alistair Francis <alistair.francis@wdc.com>
> Cc: Weiwei Li <liwei1518@gmail.com>
> Cc: Daniel Henrique Barboza <daniel.barboza@oss.qualcomm.com>
> Cc: Liu Zhiwei <zhiwei_liu@linux.alibaba.com>
> Cc: Chao Liu <chao.liu@processmission.com>
> Cc: qemu-riscv@nongnu.org
>
> ---
> Trevor Gamblin (2):
> target/riscv: extract check_sfence_vma() from helper_tlb_flush()
> target/riscv: add helper_tlb_flush_page(), use for sfence.vma with vaddr operand
>
> target/riscv/helper.h | 1 +
> target/riscv/tcg/insn_trans/trans_privileged.c.inc | 6 +++++-
> target/riscv/tcg/op_helper.c | 22 ++++++++++++++++------
> 3 files changed, 22 insertions(+), 7 deletions(-)
> ---
> base-commit: c513597761d6fc06690b5b7cf10dfb7ab05815d0
> change-id: 20260831-riscv_tlb_page_flush-11a83fe9e7b1
>
> Best regards,
> --
> Trevor Gamblin <tgamblin@baylibre.com>
>
On 9/7/26 03:14, Daniel Henrique Barboza wrote: > Richard, > > > This tlb_flush change is giving a +20% performance boost in riscv64- > tuxrun. From > this: > > 23/23 func-thorough+func-riscv64-thorough+thorough - qemu:func-riscv64- > tuxrun OK 92.71s 4 subtests passed > > To this: > > 23/23 func-thorough+func-riscv64-thorough+thorough - qemu:func-riscv64- > tuxrun OK 73.45s 4 subtests passed > > Other tests weren't that affected - I take it has to do with the amount of > sfence.vma being issued. They look fine in my own testing with an emulated > Ubuntu host as well. > > This is all fantastic but I wonder if we're missing something. If you > could > please take a look at these 2 patches and see if we're handling the tlb > flushes > according that would be terrific. It looks good. The only possible improvement would be to avoid flushing the PRV_M 1-1 tlb, using tlb_flush_by_mmuidx / tlb_flush_page_by_mmuidx. You could also rename MMUIdx_M to MMUIdx_PHYS and use it in modes other than PRV_M when the mmu is disabled, simplifying the test at the start of get_physical_address. r~
On 9/13/2026 10:14 PM, Richard Henderson wrote: > On 9/7/26 03:14, Daniel Henrique Barboza wrote: >> Richard, >> >> >> This tlb_flush change is giving a +20% performance boost in riscv64- tuxrun. From >> this: >> >> 23/23 func-thorough+func-riscv64-thorough+thorough - qemu:func-riscv64- tuxrun OK 92.71s 4 subtests passed >> >> To this: >> >> 23/23 func-thorough+func-riscv64-thorough+thorough - qemu:func-riscv64- tuxrun OK 73.45s 4 subtests passed >> >> Other tests weren't that affected - I take it has to do with the amount of >> sfence.vma being issued. They look fine in my own testing with an emulated >> Ubuntu host as well. >> >> This is all fantastic but I wonder if we're missing something. If you could >> please take a look at these 2 patches and see if we're handling the tlb flushes >> according that would be terrific. > It looks good. > > The only possible improvement would be to avoid flushing the PRV_M 1-1 tlb, using tlb_flush_by_mmuidx / tlb_flush_page_by_mmuidx. > > You could also rename MMUIdx_M to MMUIdx_PHYS and use it in modes other than PRV_M when the mmu is disabled, simplifying the test at the start of get_physical_address. > Thanks for having a look! I believe we have a potential cleanup to do based on what you said in patch 1. Probably follow-up material since it'll be a bit too much in the context handled here. Cheers, Daniel > > r~
© 2016 - 2026 Red Hat, Inc.