Seen testing of CXL emulation on arm64 (currently out of tree).

CXL interleave occurs at subpage granularity so is emulated using an IO
Memory Region. The memory is general purpose and as such may contain page
tables. FEAT_HADFS using atomic accesses from the page table walkers to
update accessed and dirty bits.

Note that disabling kernel support this ARM 8.1 feature avoids this issue
as the PTW no longer does an atomic update of the page table entries, but
that is a nasty workaround beyond its use in root causing this issue.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---

v3: Thanks Richard and Peter for reviewing.
Much simpler error handle + use of BQL_LOCK_GUARD() (Richard)
Dropped RFC and updated description as seems this is converging!
---
 target/arm/ptw.c | 31 +++++++++++++++++++++++++++++--
 1 file changed, 29 insertions(+), 2 deletions(-)

diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 5eb3577bcd..140afed451 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -711,8 +711,35 @@ static uint64_t arm_casq_ptw(CPUARMState *env, uint64_t old_val,
     void *host = ptw->out_host;
 
     if (unlikely(!host)) {
-        fi->type = ARMFault_UnsuppAtomicUpdate;
-        return 0;
+        /* Page table in MMIO Memory Region */
+        CPUState *cs = env_cpu(env);
+        MemTxAttrs attrs = {
+            .space = ptw->out_space,
+            .secure = arm_space_is_secure(ptw->out_space),
+        };
+        AddressSpace *as = arm_addressspace(cs, attrs);
+        MemTxResult result = MEMTX_OK;
+        BQL_LOCK_GUARD();
+
+        cur_val = (ptw->out_be
+                   ? address_space_ldq_be(as, ptw->out_phys, attrs, &result)
+                   : address_space_ldq_le(as, ptw->out_phys, attrs, &result));
+        if (result == MEMTX_OK && cur_val == old_val) {
+            if (ptw->out_be) {
+                address_space_stq_be(as, ptw->out_phys, new_val, attrs,
+                                     &result);
+            } else {
+                address_space_stq_le(as, ptw->out_phys, new_val, attrs,
+                                     &result);
+            }
+        }
+        if (unlikely(result != MEMTX_OK)) {
+            fi->type = ARMFault_SyncExternalOnWalk;
+            fi->ea = arm_extabort_type(result);
+            return old_val;
+        }
+
+        return cur_val;
     }
 
     /*
-- 
2.39.2

On Fri, 23 Feb 2024 at 18:01, Jonathan Cameron
<Jonathan.Cameron@huawei.com> wrote:
>
> Seen testing of CXL emulation on arm64 (currently out of tree).
>
> CXL interleave occurs at subpage granularity so is emulated using an IO
> Memory Region. The memory is general purpose and as such may contain page
> tables. FEAT_HADFS using atomic accesses from the page table walkers to
> update accessed and dirty bits.
>
> Note that disabling kernel support this ARM 8.1 feature avoids this issue
> as the PTW no longer does an atomic update of the page table entries, but
> that is a nasty workaround beyond its use in root causing this issue.
>
> Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> ---



Applied to target-arm.next, thanks.

-- PMM

On 2/23/24 08:01, Jonathan Cameron wrote:
> Seen testing of CXL emulation on arm64 (currently out of tree).
> 
> CXL interleave occurs at subpage granularity so is emulated using an IO
> Memory Region. The memory is general purpose and as such may contain page
> tables. FEAT_HADFS using atomic accesses from the page table walkers to
> update accessed and dirty bits.
> 
> Note that disabling kernel support this ARM 8.1 feature avoids this issue
> as the PTW no longer does an atomic update of the page table entries, but
> that is a nasty workaround beyond its use in root causing this issue.
> 
> Signed-off-by: Jonathan Cameron<Jonathan.Cameron@huawei.com>
> ---

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~