From nobody Fri Nov 14 17:01:22 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1760623333; cv=none; d=zohomail.com; s=zohoarc; b=M6NIizmNJLYeZyBFCwZy0bEehNFnwdqx49QEbnW+VAqlDNmY/bIIAwlOeqHKDWXwVMnbRgpWMKQCwQXgkQ/hnK0JdLn7RhUQD0GC3dgTqi20RBp3jf7Fqa8gNVhO74Cm+feznwadVNnvvEHuT2YFzY8KtBmPNCjEAusfyG8AHl4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1760623333; h=Content-Transfer-Encoding:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=LR2VehnlxZjq/mQqfxh7BXj+Th1K8OEM2+NrUHwJ/po=; b=ACZBNr+iWI/W+ENSczSPCNKYJQs0Lczc2MyUMwty5cC3i/IgvU5XWJqffiuk3JlyA6mGNnLzepQ4D0vCvhTLFa2md9gysiqe+WGQecUKSz37NlENwD6a5BcQRisTd2zgNwLMI7w8sYlW8NtBgt6L/BkiTBthKDjgsKdBYQxvqcI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1760623333252842.3931077685543; Thu, 16 Oct 2025 07:02:13 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1v9OXd-0002aH-KX; Thu, 16 Oct 2025 10:01:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OXW-0002SZ-Lx for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OXO-0006rx-RS for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:10 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-440-vHdNjlpXMmihTdJTOFsiMA-1; Thu, 16 Oct 2025 10:00:54 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 681A019560AD; Thu, 16 Oct 2025 14:00:52 +0000 (UTC) Received: from laptop.redhat.com (unknown [10.44.32.238]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 53969300019F; Thu, 16 Oct 2025 14:00:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1760623258; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LR2VehnlxZjq/mQqfxh7BXj+Th1K8OEM2+NrUHwJ/po=; b=TBKQ+lySZT0z6DxeTVaEnEtqfl8lmi6ucl1+yYdCX8aKVDigyElari4XgqrwnFpwdR9cJf XzuHcPRoSRTHEpGAUWkui/Ug6bDZN3vMUEefc8gr0kdCYCGkGlsbw66Tu4viY/9clObQMO Par4ot8VxSmLAF5OIOHinIrNGUAhLD0= X-MC-Unique: vHdNjlpXMmihTdJTOFsiMA-1 X-Mimecast-MFC-AGG-ID: vHdNjlpXMmihTdJTOFsiMA_1760623252 From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org, cohuck@redhat.com, maz@kernel.org, oliver.upton@linux.dev, sebott@redhat.com, gshan@redhat.com, ddutile@redhat.com, peterx@redhat.com, philmd@linaro.org, pbonzini@redhat.com Subject: [RESEND PATCH 1/7] target/arm/machine: Improve traces on register mismatch during migration Date: Thu, 16 Oct 2025 15:59:46 +0200 Message-ID: <20251016140039.250111-2-eric.auger@redhat.com> In-Reply-To: <20251016140039.250111-1-eric.auger@redhat.com> References: <20251016140039.250111-1-eric.auger@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eric.auger@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1760623335977154100 Content-Type: text/plain; charset="utf-8" Currently whenthe number of KVM registers exposed by the source is larger than the one exposed on the destination, the migration fails with: "failed to load cpu:cpreg_vmstate_array_len" This gives no information about which registers are causing the trouble. This patches rework the target/arm/machine code so that it becomes able to handle an input stream with a larger set of registers than the destination and print useful information about which registers are causing the trouble. The migration outcome is unchanged: - unexpected registers still will fail the migration - missing ones are print but will not fail the migration, as done today. The input stream can contain MAX_CPREG_VMSTATE_ANOMALIES(10) extra registers compared to what exists on the target. If there are more registers we will still hit the previous "load cpu:cpreg_vmstate_array_len" error. At most, MAX_CPREG_VMSTATE_ANOMALIES missing registers and MAX_CPREG_VMSTATE_ANOMALIES unexpected registers are print. Example: qemu-system-aarch64: kvm_arm_cpu_post_load Missing register in input stream= : 0 0x6030000000160003 fw feat reg 3 qemu-system-aarch64: kvm_arm_cpu_post_load Unexpected register in input str= eam: 0 0x603000000013c103 op0:3 op1:0 crn:2 crm:0 op2:3 qemu-system-aarch64: kvm_arm_cpu_post_load Unexpected register in input str= eam: 1 0x603000000013c512 op0:3 op1:0 crn:10 crm:2 op2:2 qemu-system-aarch64: kvm_arm_cpu_post_load Unexpected register in input str= eam: 2 0x603000000013c513 op0:3 op1:0 crn:10 crm:2 op2:3 qemu-system-aarch64: error while loading state for instance 0x0 of device '= cpu' qemu-system-aarch64: load of migration failed: Operation not permitted Signed-off-by: Eric Auger --- target/arm/cpu.h | 6 +++++ target/arm/kvm.c | 23 ++++++++++++++++ target/arm/machine.c | 58 ++++++++++++++++++++++++++++++++++++----- target/arm/trace-events | 7 +++++ 4 files changed, 88 insertions(+), 6 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index bf221e6f97..a7ed3f34f8 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -936,6 +936,12 @@ struct ArchCPU { uint64_t *cpreg_vmstate_values; int32_t cpreg_vmstate_array_len; =20 + #define MAX_CPREG_VMSTATE_ANOMALIES 10 + uint64_t cpreg_vmstate_missing_indexes[MAX_CPREG_VMSTATE_ANOMALIES]; + int32_t cpreg_vmstate_missing_indexes_array_len; + uint64_t cpreg_vmstate_unexpected_indexes[MAX_CPREG_VMSTATE_ANOMALIES]; + int32_t cpreg_vmstate_unexpected_indexes_array_len; + DynamicGDBFeatureInfo dyn_sysreg_feature; DynamicGDBFeatureInfo dyn_svereg_feature; DynamicGDBFeatureInfo dyn_smereg_feature; diff --git a/target/arm/kvm.c b/target/arm/kvm.c index 0d57081e69..58c6075a9e 100644 --- a/target/arm/kvm.c +++ b/target/arm/kvm.c @@ -1023,6 +1023,29 @@ void kvm_arm_cpu_pre_save(ARMCPU *cpu) =20 bool kvm_arm_cpu_post_load(ARMCPU *cpu) { + int i; + + for (i =3D 0; i < cpu->cpreg_vmstate_missing_indexes_array_len; i++) { + gchar *name; + + name =3D kvm_print_register_name(cpu->cpreg_vmstate_missing_indexe= s[i]); + trace_kvm_arm_cpu_post_load_missing_reg(name); + g_free(name); + } + + for (i =3D 0; i < cpu->cpreg_vmstate_unexpected_indexes_array_len; i++= ) { + gchar *name; + + name =3D kvm_print_register_name(cpu->cpreg_vmstate_unexpected_ind= exes[i]); + error_report("%s Unexpected register in input stream: %i 0x%"PRIx6= 4" %s", + __func__, i, cpu->cpreg_vmstate_unexpected_indexes[i]= , name); + g_free(name); + } + /* Fail the migration if we detect unexpected registers */ + if (cpu->cpreg_vmstate_unexpected_indexes_array_len) { + return false; + } + if (!write_list_to_kvmstate(cpu, KVM_PUT_FULL_STATE)) { return false; } diff --git a/target/arm/machine.c b/target/arm/machine.c index 44a0cf844b..b7a655b066 100644 --- a/target/arm/machine.c +++ b/target/arm/machine.c @@ -9,6 +9,7 @@ #include "migration/qemu-file-types.h" #include "migration/vmstate.h" #include "target/arm/gtimer.h" +#include "trace.h" =20 static bool vfp_needed(void *opaque) { @@ -989,7 +990,13 @@ static int cpu_pre_load(void *opaque) { ARMCPU *cpu =3D opaque; CPUARMState *env =3D &cpu->env; + int arraylen =3D cpu->cpreg_vmstate_array_len + MAX_CPREG_VMSTATE_ANOM= ALIES; =20 + cpu->cpreg_vmstate_indexes =3D g_renew(uint64_t, cpu->cpreg_vmstate_in= dexes, + arraylen); + cpu->cpreg_vmstate_values =3D g_renew(uint64_t, cpu->cpreg_vmstate_val= ues, + arraylen); + cpu->cpreg_vmstate_array_len =3D arraylen; /* * In an inbound migration where on the source FPSCR/FPSR/FPCR are 0, * there will be no fpcr_fpsr subsection so we won't call vfp_set_fpcr= () @@ -1022,7 +1029,7 @@ static int cpu_post_load(void *opaque, int version_id) { ARMCPU *cpu =3D opaque; CPUARMState *env =3D &cpu->env; - int i, v; + int i =3D 0, j =3D 0, k =3D 0, v =3D 0; =20 /* * Handle migration compatibility from old QEMU which didn't @@ -1050,27 +1057,66 @@ static int cpu_post_load(void *opaque, int version_= id) * entries with the right slots in our own values array. */ =20 - for (i =3D 0, v =3D 0; i < cpu->cpreg_array_len - && v < cpu->cpreg_vmstate_array_len; i++) { + trace_cpu_post_load_len(cpu->cpreg_array_len, cpu->cpreg_vmstate_array= _len); + for (; i < cpu->cpreg_array_len && v < cpu->cpreg_vmstate_array_len;) { + trace_cpu_post_load(i, v , cpu->cpreg_indexes[i]); if (cpu->cpreg_vmstate_indexes[v] > cpu->cpreg_indexes[i]) { /* register in our list but not incoming : skip it */ + trace_cpu_post_load_missing(i, cpu->cpreg_indexes[i], v); + if (j < MAX_CPREG_VMSTATE_ANOMALIES) { + cpu->cpreg_vmstate_missing_indexes[j++] =3D cpu->cpreg_ind= exes[i]; + } + i++; continue; } if (cpu->cpreg_vmstate_indexes[v] < cpu->cpreg_indexes[i]) { - /* register in their list but not ours: fail migration */ - return -1; + /* register in their list but not ours: those will fail migrat= ion */ + trace_cpu_post_load_unexpected(v, cpu->cpreg_vmstate_indexes[v= ], i); + if (k < MAX_CPREG_VMSTATE_ANOMALIES) { + cpu->cpreg_vmstate_unexpected_indexes[k++] =3D + cpu->cpreg_vmstate_indexes[v]; + } + v++; + continue; } /* matching register, copy the value over */ cpu->cpreg_values[i] =3D cpu->cpreg_vmstate_values[v]; v++; + i++; } + /* + * if we have reached the end of the incoming array but there are + * still regs in cpreg, continue parsing the regs which are missing + * in the input stream + */ + for ( ; i < cpu->cpreg_array_len; i++) { + if (j < MAX_CPREG_VMSTATE_ANOMALIES) { + trace_cpu_post_load_missing(i, cpu->cpreg_indexes[i], v); + cpu->cpreg_vmstate_missing_indexes[j++] =3D cpu->cpreg_indexes= [i]; + } + } + /* + * if we have reached the end of the cpreg array but there are + * still regs in the input stream, continue parsing the vmstate array + */ + for ( ; v < cpu->cpreg_vmstate_array_len; v++) { + if (k < MAX_CPREG_VMSTATE_ANOMALIES) { + trace_cpu_post_load_unexpected(v, cpu->cpreg_vmstate_indexes[v= ], i); + cpu->cpreg_vmstate_unexpected_indexes[k++] =3D + cpu->cpreg_vmstate_indexes[v]; + } + } + + cpu->cpreg_vmstate_missing_indexes_array_len =3D j; + cpu->cpreg_vmstate_unexpected_indexes_array_len =3D k; =20 if (kvm_enabled()) { if (!kvm_arm_cpu_post_load(cpu)) { return -1; } } else { - if (!write_list_to_cpustate(cpu)) { + if (cpu->cpreg_vmstate_unexpected_indexes_array_len || + !write_list_to_cpustate(cpu)) { return -1; } } diff --git a/target/arm/trace-events b/target/arm/trace-events index 72a2c7d096..c497df8fe7 100644 --- a/target/arm/trace-events +++ b/target/arm/trace-events @@ -13,6 +13,7 @@ arm_gt_update_irq(int timer, int irqstate) "gt_update_irq= : timer %d irqstate %d" =20 # kvm.c kvm_arm_fixup_msi_route(uint64_t iova, uint64_t gpa) "MSI iova =3D 0x%"PRI= x64" is translated into 0x%"PRIx64 +kvm_arm_cpu_post_load_missing_reg(char *name) "Missing register in input s= tream: %s" =20 # cpu.c arm_cpu_reset(uint64_t mp_aff) "cpu %" PRIu64 @@ -23,3 +24,9 @@ arm_powerctl_set_cpu_on(uint64_t mp_aff, unsigned target_= el, const char *mode, u arm_powerctl_set_cpu_on_and_reset(uint64_t mp_aff) "cpu %" PRIu64 arm_powerctl_set_cpu_off(uint64_t mp_aff) "cpu %" PRIu64 arm_powerctl_reset_cpu(uint64_t mp_aff) "cpu %" PRIu64 + +# machine.c +cpu_post_load_len(int cpreg_array_len, int cpreg_vmstate_array_len) "cpreg= _array_len=3D%d cpreg_vmstate_array_len=3D%d" +cpu_post_load(int i, int v, uint64_t regidx) "i=3D%d v=3D%d regidx=3D0x%"P= RIx64 +cpu_post_load_missing(int i, uint64_t regidx, int v) "missing register in = input stream: i=3D%d index=3D0x%"PRIx64" (v=3D%d)" +cpu_post_load_unexpected(int v, uint64_t regidx, int i) "unexpected regist= er in input stream: v=3D%d index=3D0x%"PRIx64" (i=3D%d)" --=20 2.49.0 From nobody Fri Nov 14 17:01:22 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1760623411; cv=none; d=zohomail.com; s=zohoarc; b=HrtR/nFGtc2JHSk24hTjrGtuE1aXs+VZMVBepljSLp2TX+wIafjWi0Jsed+LgVaMgqyvxFN5jwtz3Vcj2QBvTd15bg1/GgwigNtXnaNsV1u1twtiUKkURuYYtuw4KodNo66KRtPZk9NFQSe+sFsa7k5bmy3D+tY6bbsHCNkM7qg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1760623411; h=Content-Type:Content-Transfer-Encoding:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=Qg/bYKOUnPg3adpJtACOEERgoJTmDD5Mw35BYXgsPzs=; b=nCv0RasQdu0tyLNHLDAaYc+T1qQnqzqDX/mXpsvDBLyibUWLPZ7iMYM6hw0Q6Ywmdp/BpHkfOyTuSYMg6LHzL+XARnfLGND1KKBm8xUPRDOUZVtnFkWns8t6AHSxe21BNBOTG6+nMADaN5yCSQq9vi9m/1sHAG+6yGrjN5BJ3uc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1760623411797475.7375942174964; Thu, 16 Oct 2025 07:03:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1v9OXk-0002e8-8l; Thu, 16 Oct 2025 10:01:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OXf-0002cR-HE for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OXZ-0006t5-1w for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:18 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-212-ZZ3CANBKPGOljR_zqKUwIQ-1; Thu, 16 Oct 2025 10:00:58 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2009F1801370; Thu, 16 Oct 2025 14:00:57 +0000 (UTC) Received: from laptop.redhat.com (unknown [10.44.32.238]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id DCC94300019F; Thu, 16 Oct 2025 14:00:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1760623267; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qg/bYKOUnPg3adpJtACOEERgoJTmDD5Mw35BYXgsPzs=; b=QRCaeJwqJnX3AnnbfexiqlgFD5+FRHEAlbRqGQHKGLU+YKX8TkrdSIr3lT27rlUUi7CTzI 44Wv7a/Lw99zBedSUlfACbtWzbXL3BuBruGftJ7qApHyHsnpGEEx2v0jLGdcbd7kIj8Wia f+oddKy8j1+NpTGd5TmstHIpsu1LHmU= X-MC-Unique: ZZ3CANBKPGOljR_zqKUwIQ-1 X-Mimecast-MFC-AGG-ID: ZZ3CANBKPGOljR_zqKUwIQ_1760623257 From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org, cohuck@redhat.com, maz@kernel.org, oliver.upton@linux.dev, sebott@redhat.com, gshan@redhat.com, ddutile@redhat.com, peterx@redhat.com, philmd@linaro.org, pbonzini@redhat.com Subject: [RESEND PATCH 2/7] target/arm/kvm: Introduce the concept of hidden KVM regs Date: Thu, 16 Oct 2025 15:59:47 +0200 Message-ID: <20251016140039.250111-3-eric.auger@redhat.com> In-Reply-To: <20251016140039.250111-1-eric.auger@redhat.com> References: <20251016140039.250111-1-eric.auger@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eric.auger@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, T_SPF_TEMPERROR=0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1760623412704158500 More recent kernels sometimes expose new registers in an unconditionnal manner. This situation breaks backward migration as qemu notices there are more registers in the input stream than supported on the destination host. This leads to a "failed to load cpu:cpreg_vmstate_array_len" error. A good example is the introduction of KVM_REG_ARM_VENDOR_HYP_BMAP_2 pseudo FW register in v6.16 by commit C0000e58c74e (=E2=80=9CKVM: arm64: Introduce KVM_REG_ARM_VENDOR_HYP_BMAP_2=E2=80=9D). Trying to do backward migration from a host kernel that features the commit to a destination host that doesn't, fail with above error. Currently QEMU is not using that feature so ignoring this latter is not a problem. An easy way to fix the migration issue is to teach qemu we don't care about that register and we can simply ignore it when syncing its state during migration. This patch introduces an array of such hidden registers. Soon it will be settable through an array property. If hidden, the register is moved out of the array of cpreg which is built in kvm_arm_init_cpreg_list(). That way their state won't be synced. Signed-off-by: Eric Auger --- v1 -> v2: - Move the property in a separate patch - improve the commit msg - change the trace point to just print info in kvm_arm_init_cpreg_list() - improve comment in cpu.h (Connie) --- target/arm/cpu.h | 11 +++++++++++ target/arm/kvm.c | 31 ++++++++++++++++++++++++++++++- target/arm/trace-events | 2 ++ 3 files changed, 43 insertions(+), 1 deletion(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index a7ed3f34f8..75cf7ee506 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -1041,6 +1041,17 @@ struct ArchCPU { /* KVM steal time */ OnOffAuto kvm_steal_time; =20 + /* + * KVM registers that must be ignored/hidden. While they may be + * exposed by KVM to userspace, they are not intended to be used + * by qemu and more importantly we don't want them to be migrated + * to another host which wouldn't expose them. This would break + * the migration. This may be useful to allow backward migration to + * older kernels with less features. + */ + uint64_t *kvm_hidden_regs; + uint32_t nr_kvm_hidden_regs; + /* Uniprocessor system with MP extensions */ bool mp_is_up; =20 diff --git a/target/arm/kvm.c b/target/arm/kvm.c index 58c6075a9e..60fb743423 100644 --- a/target/arm/kvm.c +++ b/target/arm/kvm.c @@ -754,6 +754,25 @@ static bool kvm_arm_reg_syncs_via_cpreg_list(uint64_t = regidx) } } =20 +/** + * kvm_vcpu_compat_hidden_reg: + * @cpu: ARMCPU + * @regidx: index of the register to check + * + * Depending on the CPU compat returns true if @regidx must be + * ignored during sync & migration + */ +static inline bool +kvm_vcpu_compat_hidden_reg(ARMCPU *cpu, uint64_t regidx) +{ + for (int i =3D 0; i < cpu->nr_kvm_hidden_regs; i++) { + if (cpu->kvm_hidden_regs[i] =3D=3D regidx) { + return true; + } + } + return false; +} + /** * kvm_arm_init_cpreg_list: * @cpu: ARMCPU @@ -788,7 +807,10 @@ static int kvm_arm_init_cpreg_list(ARMCPU *cpu) qsort(&rlp->reg, rlp->n, sizeof(rlp->reg[0]), compare_u64); =20 for (i =3D 0, arraylen =3D 0; i < rlp->n; i++) { - if (!kvm_arm_reg_syncs_via_cpreg_list(rlp->reg[i])) { + uint64_t regidx =3D rlp->reg[i]; + + if (!kvm_arm_reg_syncs_via_cpreg_list(regidx) || + kvm_vcpu_compat_hidden_reg(cpu, regidx)) { continue; } switch (rlp->reg[i] & KVM_REG_SIZE_MASK) { @@ -804,6 +826,8 @@ static int kvm_arm_init_cpreg_list(ARMCPU *cpu) arraylen++; } =20 + trace_kvm_arm_init_cpreg_list_arraylen(arraylen); + cpu->cpreg_indexes =3D g_renew(uint64_t, cpu->cpreg_indexes, arraylen); cpu->cpreg_values =3D g_renew(uint64_t, cpu->cpreg_values, arraylen); cpu->cpreg_vmstate_indexes =3D g_renew(uint64_t, cpu->cpreg_vmstate_in= dexes, @@ -815,9 +839,14 @@ static int kvm_arm_init_cpreg_list(ARMCPU *cpu) =20 for (i =3D 0, arraylen =3D 0; i < rlp->n; i++) { uint64_t regidx =3D rlp->reg[i]; + if (!kvm_arm_reg_syncs_via_cpreg_list(regidx)) { continue; } + if (kvm_vcpu_compat_hidden_reg(cpu, regidx)) { + trace_kvm_arm_init_cpreg_list_skip_hidden_reg(rlp->reg[i]); + continue; + } cpu->cpreg_indexes[arraylen] =3D regidx; arraylen++; } diff --git a/target/arm/trace-events b/target/arm/trace-events index c497df8fe7..31386cc1f2 100644 --- a/target/arm/trace-events +++ b/target/arm/trace-events @@ -14,6 +14,8 @@ arm_gt_update_irq(int timer, int irqstate) "gt_update_irq= : timer %d irqstate %d" # kvm.c kvm_arm_fixup_msi_route(uint64_t iova, uint64_t gpa) "MSI iova =3D 0x%"PRI= x64" is translated into 0x%"PRIx64 kvm_arm_cpu_post_load_missing_reg(char *name) "Missing register in input s= tream: %s" +kvm_arm_init_cpreg_list_arraylen(uint32_t arraylen) "arraylen=3D%d" +kvm_arm_init_cpreg_list_skip_hidden_reg(uint64_t regidx) "hidden 0x%"PRIx6= 4" is skipped" =20 # cpu.c arm_cpu_reset(uint64_t mp_aff) "cpu %" PRIu64 --=20 2.49.0 From nobody Fri Nov 14 17:01:22 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1760623394; cv=none; d=zohomail.com; s=zohoarc; b=FQt5SiGiIY+vwExlV0KJh4ZcA6nMdbTNbd1umWTHozqIgRgQxdbtVfYTuYjPjNsPKSSnJc+uXD5tPvRQrkkd/wiumhIrHRNgnP6Jyn+BwFcWiDn6duifpXq1MZfVUeU5bgQVGPSD4kzoFLjikPFc6SwL0ot7GAy0H91RCS5Djr4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1760623394; h=Content-Type:Content-Transfer-Encoding:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=6pM9KJI9trUfzRX2fsineaVXGBm7YlWm4sFywV3AVfw=; b=TYOMFkRkAsQC0Hi3UR3gh4WTQOMWPVNkxJiWiV5lXA/1wTssBqQlKnDJ8fkitboaSITa6Do615M/h0O4468OaTNlqBMzHnPyCpg3DMR9WeQQPAY0cLdATy0tOiL7CPY4lBIjdzs9Md61Cn999+Kh//j5VVt/GZ10Ptm32jqOaqU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1760623394849604.2279502663293; Thu, 16 Oct 2025 07:03:14 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1v9OXf-0002bq-FP; Thu, 16 Oct 2025 10:01:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OXZ-0002Tq-7E for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OXT-0006sZ-3A for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:12 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-542-3nL0v2VHNaS5eT_8h9qSog-1; Thu, 16 Oct 2025 10:01:02 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 88A0F1954B24; Thu, 16 Oct 2025 14:01:01 +0000 (UTC) Received: from laptop.redhat.com (unknown [10.44.32.238]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8568E300019F; Thu, 16 Oct 2025 14:00:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1760623264; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6pM9KJI9trUfzRX2fsineaVXGBm7YlWm4sFywV3AVfw=; b=Y0lu+lkOhH+OiPWU0UYSgRBKyQNf8TWMvZV7a9MELieHdyRdIZ6HPyQP5tdIwXZyxadc9W U53VmdY5sz9/WZpwvv9U3B8Wqa1R+/7uALiG2MPowo1M1+gjWR/FszBHwZX0JrtIU+BeK0 2+86rKFkPbBRR3C3SY2ufi7WNN0PUnM= X-MC-Unique: 3nL0v2VHNaS5eT_8h9qSog-1 X-Mimecast-MFC-AGG-ID: 3nL0v2VHNaS5eT_8h9qSog_1760623261 From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org, cohuck@redhat.com, maz@kernel.org, oliver.upton@linux.dev, sebott@redhat.com, gshan@redhat.com, ddutile@redhat.com, peterx@redhat.com, philmd@linaro.org, pbonzini@redhat.com Subject: [RESEND PATCH 3/7] target/arm/kvm: Introduce the concept of enforced/fake registers Date: Thu, 16 Oct 2025 15:59:48 +0200 Message-ID: <20251016140039.250111-4-eric.auger@redhat.com> In-Reply-To: <20251016140039.250111-1-eric.auger@redhat.com> References: <20251016140039.250111-1-eric.auger@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eric.auger@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1760623396602158500 Newer kernels may revoke exposure of KVM regs to userspace. This can happen when one notices that some registers were unconditionnally exposed whether they shall be conditionnally exposed for example. An example of such situation is: TCR2_EL1, PIRE0_EL1, PIR_EL1. Associated kernel commits were: 0fcb4eea5345=C2=A0 KVM: arm64: Hide TCR2_EL1 from userspace when disabled f= or guests a68cddbe47ef=C2=A0 KVM: arm64: Hide S1PIE registers from userspace when dis= abled for guests Those commits were actual fixes but the cons is that is breaks forward migration on some HW. Indeed when migrating from an old kernel that does not feature those commits to a more recent one, destination qemu detects there are more KVM regs in the input migration stream than exposed by the destination host and the migration fails with: "failed to load cpu:cpreg_vmstate_array_len" This patchs adds the capability to defined an array of enforced register indexes that teaches QEMU that some registers must exist. If they are not exposed by KVM qemu fakes their presence in the preload phase by adjusting the size of the cpreg_vmstate arrays. On postload we make sure to ignore them when analyzing potential mismatch between registers. The actual cpreg array is never altered meaning those registers are never accessed nor saved. The array will be populated with a deficated array property. Signed-off-by: Eric Auger --- v1 -> v2: - improve comment in target/arm/cpu.h (Connie) --- target/arm/cpu.h | 23 +++++++++++++++++++++++ target/arm/kvm.c | 24 ++++++++++++++++++++++++ target/arm/machine.c | 31 ++++++++++++++++++++++--------- target/arm/trace-events | 2 ++ 4 files changed, 71 insertions(+), 9 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 75cf7ee506..30d59a51d6 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -1052,6 +1052,19 @@ struct ArchCPU { uint64_t *kvm_hidden_regs; uint32_t nr_kvm_hidden_regs; =20 + /* + * KVM registers whose presence must be enforced + * Either they must be exposed to user space by KVM or + * they must be faked for migration sake because the source does + * feature them. + */ + uint64_t *kvm_enforced_regs; + uint32_t nr_kvm_enforced_regs; + + /* registers that must be be faked */ + uint64_t *kvm_fake_regs; + uint32_t nr_kvm_fake_regs; + /* Uniprocessor system with MP extensions */ bool mp_is_up; =20 @@ -2652,6 +2665,16 @@ static inline uint64_t *aa64_vfp_qreg(CPUARMState *e= nv, unsigned regno) /* Shared between translate-sve.c and sve_helper.c. */ extern const uint64_t pred_esz_masks[5]; =20 +static inline bool is_fake_reg(ARMCPU *cpu, uint64_t regidx) +{ + for (int i =3D 0; i < cpu->nr_kvm_fake_regs; i++) { + if (regidx =3D=3D cpu->kvm_fake_regs[i]) { + return true; + } + } + return false; +} + /* * AArch64 usage of the PAGE_TARGET_* bits for linux-user. * Note that with the Linux kernel, PROT_MTE may not be cleared by mprotect diff --git a/target/arm/kvm.c b/target/arm/kvm.c index 60fb743423..7551c43e79 100644 --- a/target/arm/kvm.c +++ b/target/arm/kvm.c @@ -788,6 +788,7 @@ static int kvm_arm_init_cpreg_list(ARMCPU *cpu) struct kvm_reg_list rl; struct kvm_reg_list *rlp; int i, ret, arraylen; + int nr_fake_regs =3D 0; CPUState *cs =3D CPU(cpu); =20 rl.n =3D 0; @@ -801,6 +802,29 @@ static int kvm_arm_init_cpreg_list(ARMCPU *cpu) if (ret) { goto out; } + + /* + * Identify which registers belonging to the enforced list + * need to be faked because they are not exposed to userspace + * by KVM + */ + cpu->kvm_fake_regs =3D g_new(uint64_t, cpu->nr_kvm_enforced_regs); + for (int j =3D 0; j < cpu->nr_kvm_enforced_regs; j++) { + uint64_t v64; + int r; + + r =3D kvm_get_one_reg(cs, cpu->kvm_enforced_regs[j], &v64); + if (errno !=3D ENOENT) { + /* enforced register exists, no need to fake */ + trace_kvm_arm_init_cpreg_exposed(cpu->kvm_enforced_regs[j], v6= 4, r); + continue; + } + cpu->kvm_fake_regs[nr_fake_regs] =3D cpu->kvm_enforced_regs[j]; + trace_kvm_arm_init_cpreg_fake(cpu->kvm_fake_regs[nr_fake_regs]); + nr_fake_regs++; + } + cpu->nr_kvm_fake_regs =3D nr_fake_regs; + /* Sort the list we get back from the kernel, since cpreg_tuples * must be in strictly ascending order. */ diff --git a/target/arm/machine.c b/target/arm/machine.c index b7a655b066..fcf2dd8315 100644 --- a/target/arm/machine.c +++ b/target/arm/machine.c @@ -990,7 +990,8 @@ static int cpu_pre_load(void *opaque) { ARMCPU *cpu =3D opaque; CPUARMState *env =3D &cpu->env; - int arraylen =3D cpu->cpreg_vmstate_array_len + MAX_CPREG_VMSTATE_ANOM= ALIES; + int arraylen =3D cpu->cpreg_vmstate_array_len + + cpu->nr_kvm_fake_regs + MAX_CPREG_VMSTATE_ANOMALIES; =20 cpu->cpreg_vmstate_indexes =3D g_renew(uint64_t, cpu->cpreg_vmstate_in= dexes, arraylen); @@ -1057,6 +1058,10 @@ static int cpu_post_load(void *opaque, int version_i= d) * entries with the right slots in our own values array. */ =20 + /* + * at this point cpu->cpreg_vmstate_array_len was migrated with the + * actual length saved on source + */ trace_cpu_post_load_len(cpu->cpreg_array_len, cpu->cpreg_vmstate_array= _len); for (; i < cpu->cpreg_array_len && v < cpu->cpreg_vmstate_array_len;) { trace_cpu_post_load(i, v , cpu->cpreg_indexes[i]); @@ -1071,10 +1076,14 @@ static int cpu_post_load(void *opaque, int version_= id) } if (cpu->cpreg_vmstate_indexes[v] < cpu->cpreg_indexes[i]) { /* register in their list but not ours: those will fail migrat= ion */ - trace_cpu_post_load_unexpected(v, cpu->cpreg_vmstate_indexes[v= ], i); - if (k < MAX_CPREG_VMSTATE_ANOMALIES) { - cpu->cpreg_vmstate_unexpected_indexes[k++] =3D - cpu->cpreg_vmstate_indexes[v]; + if (is_fake_reg(cpu, cpu->cpreg_vmstate_indexes[v])) { + /* this absence was expected as the reg was tagged as fake= */ + } else { + trace_cpu_post_load_unexpected(v, cpu->cpreg_vmstate_index= es[v], i); + if (k < MAX_CPREG_VMSTATE_ANOMALIES) { + cpu->cpreg_vmstate_unexpected_indexes[k++] =3D + cpu->cpreg_vmstate_indexes[v]; + } } v++; continue; @@ -1100,10 +1109,14 @@ static int cpu_post_load(void *opaque, int version_= id) * still regs in the input stream, continue parsing the vmstate array */ for ( ; v < cpu->cpreg_vmstate_array_len; v++) { - if (k < MAX_CPREG_VMSTATE_ANOMALIES) { - trace_cpu_post_load_unexpected(v, cpu->cpreg_vmstate_indexes[v= ], i); - cpu->cpreg_vmstate_unexpected_indexes[k++] =3D - cpu->cpreg_vmstate_indexes[v]; + if (is_fake_reg(cpu, cpu->cpreg_vmstate_indexes[v])) { + /* this absence was expected as the reg was tagged as fake */ + } else { + if (k < MAX_CPREG_VMSTATE_ANOMALIES) { + trace_cpu_post_load_unexpected(v, cpu->cpreg_vmstate_index= es[v], i); + cpu->cpreg_vmstate_unexpected_indexes[k++] =3D + cpu->cpreg_vmstate_indexes[v]; + } } } =20 diff --git a/target/arm/trace-events b/target/arm/trace-events index 31386cc1f2..fa9d721339 100644 --- a/target/arm/trace-events +++ b/target/arm/trace-events @@ -16,6 +16,8 @@ kvm_arm_fixup_msi_route(uint64_t iova, uint64_t gpa) "MSI= iova =3D 0x%"PRIx64" is kvm_arm_cpu_post_load_missing_reg(char *name) "Missing register in input s= tream: %s" kvm_arm_init_cpreg_list_arraylen(uint32_t arraylen) "arraylen=3D%d" kvm_arm_init_cpreg_list_skip_hidden_reg(uint64_t regidx) "hidden 0x%"PRIx6= 4" is skipped" +kvm_arm_init_cpreg_exposed(uint64_t regidx, uint64_t val, int ret) "enforc= ed reg 0x%"PRIx64" is already exposed by KVM: value=3D0x%"PRIx64 " ret=3D%d= : nothing to do" +kvm_arm_init_cpreg_fake(uint64_t regidx) "register 0x%"PRIx64 " is not exp= osed and is faked" =20 # cpu.c arm_cpu_reset(uint64_t mp_aff) "cpu %" PRIu64 --=20 2.49.0 From nobody Fri Nov 14 17:01:22 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1760623390; cv=none; d=zohomail.com; s=zohoarc; b=jvtbBLV497jjIBYzZzWDRYeZUv1/q5rz1ee9cAMVRcHUg3QmgliW+GLY5RB4JkvZ7qCrM8bgYv06+2n7Saes9CMm/Oz9FlGTqt1gDt2GW/AwldVpzDFt9W6xoFF37rMVMpuQD6htNdn5FgrIEaLFAw/y4Ss+j2WQlAgPMu/judw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1760623390; h=Content-Transfer-Encoding:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=rFvybGKOgQgMcwFiT581cFeXfHcbVpkvXj0ltTBUf/A=; b=BfWaBVJzTqzT3nCNOCQ6+dCH5IU77tDXkrUQna42NQhe3VU8mIJuQqqfD2WISqGpJ2WccfSnxluvfzkbSIKFFrArxmTkenaQhcajqD72stE/UAkSVB11rK14YHd7mJUdIOkW5qVadvxffZWcC3TvdFjsovHkbluKHF/dZsywicI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1760623390152333.82032332886183; Thu, 16 Oct 2025 07:03:10 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1v9OXk-0002fF-Qe; Thu, 16 Oct 2025 10:01:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OXf-0002d0-SJ for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OXa-0006tf-Ja for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:19 -0400 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-257-NTP6JzgHPEiIynHHpxtS9Q-1; Thu, 16 Oct 2025 10:01:08 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A11041954197; Thu, 16 Oct 2025 14:01:06 +0000 (UTC) Received: from laptop.redhat.com (unknown [10.44.32.238]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0B4BE300019F; Thu, 16 Oct 2025 14:01:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1760623272; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rFvybGKOgQgMcwFiT581cFeXfHcbVpkvXj0ltTBUf/A=; b=XRPuSRv5CzQH+0f2JX5U3TxbnHecHPKD9bvmiHCdVlPdOqfPl459nplACNbRQxQmf8rIDk 7udwTOzS0mHS/evUsxypKxdzI+Z3NKwWHJRfKVm5IWAHHAea/paBIjOYBzce6wgwwIpHj9 euZgEiKhR56E96QTFKtd/8wMY6cogRU= X-MC-Unique: NTP6JzgHPEiIynHHpxtS9Q-1 X-Mimecast-MFC-AGG-ID: NTP6JzgHPEiIynHHpxtS9Q_1760623266 From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org, cohuck@redhat.com, maz@kernel.org, oliver.upton@linux.dev, sebott@redhat.com, gshan@redhat.com, ddutile@redhat.com, peterx@redhat.com, philmd@linaro.org, pbonzini@redhat.com Subject: [RESEND PATCH 4/7] kvm-all: Add the capability to blacklist some KVM regs Date: Thu, 16 Oct 2025 15:59:49 +0200 Message-ID: <20251016140039.250111-5-eric.auger@redhat.com> In-Reply-To: <20251016140039.250111-1-eric.auger@redhat.com> References: <20251016140039.250111-1-eric.auger@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eric.auger@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1760623402584158500 Content-Type: text/plain; charset="utf-8" On ARM we want to be able to blacklist registers that are exposed by KVM. To mitigate some mitigation failures that occur when a new register is exposed and does not exist on the destination, some registers are tagged "hidden" and their state won't be saved. As the state is not saved and they are expected not to be used, we want to enforce they aren't. So let's check this. The new CPUClass hide_reg() callback is optional and will be implemented on ARM in a subsequent patch. Signed-off-by: Eric Auger --- include/hw/core/cpu.h | 2 ++ accel/kvm/kvm-all.c | 12 ++++++++++++ 2 files changed, 14 insertions(+) diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h index e79e8e0a8e..8efd555f36 100644 --- a/include/hw/core/cpu.h +++ b/include/hw/core/cpu.h @@ -142,6 +142,7 @@ struct SysemuCPUOps; * the caller will not g_free() it. * @disas_set_info: Setup architecture specific components of disassembly = info * @adjust_watchpoint_address: Perform a target-specific adjustment to an + * @hide_reg: Check if a register must be hidden (optional) * address before attempting to match it against watchpoints. * @deprecation_note: If this CPUClass is deprecated, this field provides * related information. @@ -167,6 +168,7 @@ struct CPUClass { int (*gdb_read_register)(CPUState *cpu, GByteArray *buf, int reg); int (*gdb_write_register)(CPUState *cpu, uint8_t *buf, int reg); vaddr (*gdb_adjust_breakpoint)(CPUState *cpu, vaddr addr); + bool (*hide_reg)(CPUState *cpu, uint64_t regidex); =20 const char *gdb_core_xml_file; const char * (*gdb_arch_name)(CPUState *cpu); diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 56031925c4..610f05fd0d 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -3783,9 +3783,15 @@ bool kvm_device_supported(int vmfd, uint64_t type) =20 int kvm_set_one_reg(CPUState *cs, uint64_t id, void *source) { + CPUClass *cc =3D CPU_GET_CLASS(cs); struct kvm_one_reg reg; int r; =20 + if (cc->hide_reg && cc->hide_reg(cs, id)) { + error_report("%s reg 0x%"PRIx64" is hidden and shall never been ac= cessed", + __func__, id); + g_assert_not_reached(); + } reg.id =3D id; reg.addr =3D (uintptr_t) source; r =3D kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®); @@ -3797,9 +3803,15 @@ int kvm_set_one_reg(CPUState *cs, uint64_t id, void = *source) =20 int kvm_get_one_reg(CPUState *cs, uint64_t id, void *target) { + CPUClass *cc =3D CPU_GET_CLASS(cs); struct kvm_one_reg reg; int r; =20 + if (cc->hide_reg && cc->hide_reg(cs, id)) { + error_report("%s reg 0x%"PRIx64" is hidden and shall never been ac= cessed", + __func__, id); + g_assert_not_reached(); + } reg.id =3D id; reg.addr =3D (uintptr_t) target; r =3D kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®); --=20 2.49.0 From nobody Fri Nov 14 17:01:22 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1760623414; cv=none; d=zohomail.com; s=zohoarc; b=Fox+phuW4XBNZnKkh9TzEqfZ0MzhWWFMAXEaCGf2RpkbKQ1EYay0elH/sdpS+/ASCGQQ60FFIXVjtk0QUIe1i3Ltj49trIJ0M/1s5/ktBUxJTA40rSmQYjLQlRRPNx1j5yFXnErM+Gjwkmyg5X4G1jSKUE3F0ALapKtUk+qLoVY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1760623414; h=Content-Transfer-Encoding:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=ntVvVf8/M035uETjgq/33f4ABncTA+oJSQYYqx0UURM=; b=LHPtl43vSGnKupgiX99zV64T87lecASTjVZYOrtaZ2YL2EqRFokOV3lGFAaDZR/iUXZ9ZE0DKeQ35lufa+brtuDKweZFIRNMvD6fVSz5aIk0M78VLy9s/Pd4oADBK0/bA8tWENkHX6SwIbZQ2wVdyyBQ5cvUNMsYucsBPo9t914= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1760623414134383.27119643811534; Thu, 16 Oct 2025 07:03:34 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1v9OXt-0002hE-9F; Thu, 16 Oct 2025 10:01:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OXr-0002gm-9x for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OXj-0006uk-Sx for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:30 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-94-xAfkGN8PO1GZv_wp7S5tqg-1; Thu, 16 Oct 2025 10:01:13 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 295B1180A229; Thu, 16 Oct 2025 14:01:11 +0000 (UTC) Received: from laptop.redhat.com (unknown [10.44.32.238]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 233EE30001BC; Thu, 16 Oct 2025 14:01:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1760623276; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ntVvVf8/M035uETjgq/33f4ABncTA+oJSQYYqx0UURM=; b=efxmGSIpuFWn7/VzmOF7WRb7pvyqao6CCQwaNRbmnK6FPjzem3JyICMCRIGPab/f7kFt5+ X6cLcLk/zRJZzhZDXwoULVvkQlr/5yctoJ5jhYuOInD4JNM2WBndZANEBYj4efIPa/ju1W 5AiYkpJcqxqb0fWQSJONlh73lNZwYNA= X-MC-Unique: xAfkGN8PO1GZv_wp7S5tqg-1 X-Mimecast-MFC-AGG-ID: xAfkGN8PO1GZv_wp7S5tqg_1760623271 From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org, cohuck@redhat.com, maz@kernel.org, oliver.upton@linux.dev, sebott@redhat.com, gshan@redhat.com, ddutile@redhat.com, peterx@redhat.com, philmd@linaro.org, pbonzini@redhat.com Subject: [RESEND PATCH 5/7] target/arm/cpu: Implement hide_reg callback() Date: Thu, 16 Oct 2025 15:59:50 +0200 Message-ID: <20251016140039.250111-6-eric.auger@redhat.com> In-Reply-To: <20251016140039.250111-1-eric.auger@redhat.com> References: <20251016140039.250111-1-eric.auger@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eric.auger@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_PASS=-0.001, T_SPF_HELO_TEMPERROR=0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1760623415101154100 Content-Type: text/plain; charset="utf-8" Checks if the register is hidden. Signed-off-by: Eric Auger --- target/arm/cpu.h | 2 ++ target/arm/cpu.c | 12 ++++++++++++ target/arm/kvm.c | 23 ++--------------------- 3 files changed, 16 insertions(+), 21 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 30d59a51d6..3ae4d65422 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -2687,4 +2687,6 @@ static inline bool is_fake_reg(ARMCPU *cpu, uint64_t = regidx) #define LOG2_TAG_GRANULE 4 #define TAG_GRANULE (1 << LOG2_TAG_GRANULE) =20 +bool arm_cpu_hide_reg(CPUState *s, uint64_t regidx); + #endif diff --git a/target/arm/cpu.c b/target/arm/cpu.c index 3b556f1404..60eee82742 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -2366,6 +2366,17 @@ static const TCGCPUOps arm_tcg_ops =3D { }; #endif /* CONFIG_TCG */ =20 +bool arm_cpu_hide_reg(CPUState *s, uint64_t regidx) +{ + ARMCPU *cpu =3D ARM_CPU(s); + for (int i =3D 0; i < cpu->nr_kvm_hidden_regs; i++) { + if (cpu->kvm_hidden_regs[i] =3D=3D regidx) { + return true; + } + } + return false; +} + static void arm_cpu_class_init(ObjectClass *oc, const void *data) { ARMCPUClass *acc =3D ARM_CPU_CLASS(oc); @@ -2394,6 +2405,7 @@ static void arm_cpu_class_init(ObjectClass *oc, const= void *data) cc->gdb_get_core_xml_file =3D arm_gdb_get_core_xml_file; cc->gdb_stop_before_watchpoint =3D true; cc->disas_set_info =3D arm_disas_set_info; + cc->hide_reg =3D arm_cpu_hide_reg; =20 #ifdef CONFIG_TCG cc->tcg_ops =3D &arm_tcg_ops; diff --git a/target/arm/kvm.c b/target/arm/kvm.c index 7551c43e79..1a95e2c667 100644 --- a/target/arm/kvm.c +++ b/target/arm/kvm.c @@ -754,25 +754,6 @@ static bool kvm_arm_reg_syncs_via_cpreg_list(uint64_t = regidx) } } =20 -/** - * kvm_vcpu_compat_hidden_reg: - * @cpu: ARMCPU - * @regidx: index of the register to check - * - * Depending on the CPU compat returns true if @regidx must be - * ignored during sync & migration - */ -static inline bool -kvm_vcpu_compat_hidden_reg(ARMCPU *cpu, uint64_t regidx) -{ - for (int i =3D 0; i < cpu->nr_kvm_hidden_regs; i++) { - if (cpu->kvm_hidden_regs[i] =3D=3D regidx) { - return true; - } - } - return false; -} - /** * kvm_arm_init_cpreg_list: * @cpu: ARMCPU @@ -834,7 +815,7 @@ static int kvm_arm_init_cpreg_list(ARMCPU *cpu) uint64_t regidx =3D rlp->reg[i]; =20 if (!kvm_arm_reg_syncs_via_cpreg_list(regidx) || - kvm_vcpu_compat_hidden_reg(cpu, regidx)) { + arm_cpu_hide_reg(&cpu->parent_obj, regidx)) { continue; } switch (rlp->reg[i] & KVM_REG_SIZE_MASK) { @@ -867,7 +848,7 @@ static int kvm_arm_init_cpreg_list(ARMCPU *cpu) if (!kvm_arm_reg_syncs_via_cpreg_list(regidx)) { continue; } - if (kvm_vcpu_compat_hidden_reg(cpu, regidx)) { + if (arm_cpu_hide_reg(&cpu->parent_obj, regidx)) { trace_kvm_arm_init_cpreg_list_skip_hidden_reg(rlp->reg[i]); continue; } --=20 2.49.0 From nobody Fri Nov 14 17:01:22 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1760623453; cv=none; d=zohomail.com; s=zohoarc; b=WV+3x1pE1caSKVqd2bOd3gtQPECKO3e1Y/7S0n2PQVitEZpRklm429fXmj8Lqoa5cO3dv/+0ELmvDe2Q5ROfOerE5AbvNvrjLWm7mnGE5CdyuALgIMEiLUlwJxnVJzudUoARyCK8qDcaUEabIwcobHz63moCScsMq+srBwOZpkU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1760623453; h=Content-Transfer-Encoding:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=VSSOQjHlELjPhsxKkeZNq3eX3cMQavKRRCjGfxib9Ig=; b=ByQElxvzwtiRsZgEzo+zcWfDPLqS8a2EXZWmpWod0/Y5Ysm1JC//A/HO4lkcdBN5950OQ8FFG4V/16vj5mMmLZx1dv87DuSVmBES/hFKf7HyZE284nbjfpsUL9WbR0Arbq2aRuTu9/Zdtf1OY5Jkq+XpQSuciE4iwzFL29eCZM0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1760623453412686.8812759415031; Thu, 16 Oct 2025 07:04:13 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1v9OY2-0002oD-M4; Thu, 16 Oct 2025 10:01:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OXx-0002kH-8K for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OXm-0006vn-4s for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:36 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-657-UC6L7rT9P4WMnrzNWMbkJA-1; Thu, 16 Oct 2025 10:01:17 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B74A11809A08; Thu, 16 Oct 2025 14:01:15 +0000 (UTC) Received: from laptop.redhat.com (unknown [10.44.32.238]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9D10D300019F; Thu, 16 Oct 2025 14:01:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1760623281; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VSSOQjHlELjPhsxKkeZNq3eX3cMQavKRRCjGfxib9Ig=; b=ex40gFcSuAHLEW17GH3+drj6Csa01HqxrRnHrsYjS6p+gXAGuMd/EjFVcLIDN89vck41tn EM98DsxF3/TSP8Z/P19Zw/ilUs5B8NWFXea1Uwf0RzuO5RM8SSq5o2OdEEFR+TmVL0+pb1 GdT3Oy1NAsYAlusfRLmQJim+bCD9BOw= X-MC-Unique: UC6L7rT9P4WMnrzNWMbkJA-1 X-Mimecast-MFC-AGG-ID: UC6L7rT9P4WMnrzNWMbkJA_1760623275 From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org, cohuck@redhat.com, maz@kernel.org, oliver.upton@linux.dev, sebott@redhat.com, gshan@redhat.com, ddutile@redhat.com, peterx@redhat.com, philmd@linaro.org, pbonzini@redhat.com Subject: [RESEND PATCH 6/7] target/arm/kvm: Expose kvm-hidden-regs and kvm-fake-regs properties Date: Thu, 16 Oct 2025 15:59:51 +0200 Message-ID: <20251016140039.250111-7-eric.auger@redhat.com> In-Reply-To: <20251016140039.250111-1-eric.auger@redhat.com> References: <20251016140039.250111-1-eric.auger@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eric.auger@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, T_SPF_TEMPERROR=0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1760623463328158500 Content-Type: text/plain; charset="utf-8" Allows to set the kvm-hidden-regs and kvm-fake-regs array properties. This will allow to define such compat machine props like: static GlobalProperty arm_virt_kernel_compat_10_1[] =3D { /* KVM_REG_ARM_VENDOR_HYP_BMAP_2 */ { TYPE_ARM_CPU, "kvm-hidden-regs", "0x6030000000160003" }, { TYPE_ARM_CPU, "kvm-enforced-regs", /* TCR_EL1, PIRE0_EL1, PIR_EL1 */ "0x603000000013c103, 0x603000000013c512, 0x603000000013c513" }, } Signed-off-by: Eric Auger --- target/arm/kvm.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/target/arm/kvm.c b/target/arm/kvm.c index 1a95e2c667..d103d4293d 100644 --- a/target/arm/kvm.c +++ b/target/arm/kvm.c @@ -39,6 +39,8 @@ #include "qemu/log.h" #include "hw/acpi/acpi.h" #include "hw/acpi/ghes.h" +#include "hw/qdev-properties.h" +#include "hw/qdev-properties-system.h" #include "target/arm/gtimer.h" #include "migration/blocker.h" =20 @@ -484,6 +486,15 @@ static void kvm_steal_time_set(Object *obj, bool value= , Error **errp) ARM_CPU(obj)->kvm_steal_time =3D value ? ON_OFF_AUTO_ON : ON_OFF_AUTO_= OFF; } =20 +static const Property arm_cpu_kvm_compat_hidden_regs_property =3D + DEFINE_PROP_ARRAY("kvm-hidden-regs", ARMCPU, + nr_kvm_hidden_regs, kvm_hidden_regs, qdev_prop_uint6= 4, uint64_t); + +static const Property arm_cpu_kvm_compat_enforced_regs_property =3D + DEFINE_PROP_ARRAY("kvm-enforced-regs", ARMCPU, + nr_kvm_enforced_regs, kvm_enforced_regs, + qdev_prop_uint64, uint64_t); + /* KVM VCPU properties should be prefixed with "kvm-". */ void kvm_arm_add_vcpu_properties(ARMCPU *cpu) { @@ -505,6 +516,9 @@ void kvm_arm_add_vcpu_properties(ARMCPU *cpu) kvm_steal_time_set); object_property_set_description(obj, "kvm-steal-time", "Set off to disable KVM steal time."); + + qdev_property_add_static(DEVICE(obj), &arm_cpu_kvm_compat_hidden_regs_= property); + qdev_property_add_static(DEVICE(obj), &arm_cpu_kvm_compat_enforced_reg= s_property); } =20 bool kvm_arm_pmu_supported(void) --=20 2.49.0 From nobody Fri Nov 14 17:01:22 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1760623388; cv=none; d=zohomail.com; s=zohoarc; b=dTvk8zR3o+L10YgTku5srWM9YiIQcWQ5q/3EyTMrO5xp6R1IvC3eKdMqeJ07kZq0jQ4bq8x1HCfuvlWjsLVKfMCCCIuu7VJRJ/383dxGLtKgT1GKhH6iDyFk+t6Hoz+xjRoFPRcVzhZlaHJk4LdrltWv9S7sgwdDbeBtkJGUrQQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1760623388; h=Content-Transfer-Encoding:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=GcMLpY5zK1nHpg6MUvzduzIT6CGZBv5Zt25S6TZ6+F4=; b=DTp+np6OHQQlqHjR4A54mHMMT54FavpLt1/PSXWGya5VB3lRRQFpJjTJ+EjDhzo8Nugz6TCwmAzUBtClObIAL/s4e+EJ4nszD16M0XAn9rRKukrlDElpAP7VmKY5f+OjA+0wwcozZntHqi/HjMCaFfduiwtXmOzo5btX/w+VZBo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1760623388485424.9668266835189; Thu, 16 Oct 2025 07:03:08 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1v9OY9-0002sB-Rg; Thu, 16 Oct 2025 10:01:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OY8-0002rM-4c for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1v9OXq-0006wZ-J4 for qemu-devel@nongnu.org; Thu, 16 Oct 2025 10:01:47 -0400 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-520-hOHl3aioOFyTKXggNNGC5Q-1; Thu, 16 Oct 2025 10:01:21 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 51A5919560BA; Thu, 16 Oct 2025 14:01:20 +0000 (UTC) Received: from laptop.redhat.com (unknown [10.44.32.238]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 37C123000223; Thu, 16 Oct 2025 14:01:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1760623285; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GcMLpY5zK1nHpg6MUvzduzIT6CGZBv5Zt25S6TZ6+F4=; b=SahbMmV8jHCqbMMfk5odjaxKJ2z7E2/IKHp7l6FVTOFgYRzDlwg4bwDbfNDzDUahMvHeq4 9qGI+t7VgCfi9FJ/or4dvVZuvLA5E467WkBeIdK/La92XSgGqvlu7VcoIYPqilbmPY4Swl krNZSK3SBiFUUI752AVhioqqzrC2aF0= X-MC-Unique: hOHl3aioOFyTKXggNNGC5Q-1 X-Mimecast-MFC-AGG-ID: hOHl3aioOFyTKXggNNGC5Q_1760623280 From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org, cohuck@redhat.com, maz@kernel.org, oliver.upton@linux.dev, sebott@redhat.com, gshan@redhat.com, ddutile@redhat.com, peterx@redhat.com, philmd@linaro.org, pbonzini@redhat.com Subject: [RESEND PATCH 7/7] hw/arm/virt: [DO NOT UPSTREAM] Enforce compatibility with older kernels Date: Thu, 16 Oct 2025 15:59:52 +0200 Message-ID: <20251016140039.250111-8-eric.auger@redhat.com> In-Reply-To: <20251016140039.250111-1-eric.auger@redhat.com> References: <20251016140039.250111-1-eric.auger@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eric.auger@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, T_SPF_TEMPERROR=0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1760623390746158500 Content-Type: text/plain; charset="utf-8" This is an example on how to use the new CPU options. This catters to distributions who want machines to be migratable (forward and backward) accross different host kernel versions in case KVM registers exposed to qemu vary accross kernels. This patch is not meant to be upstreamed as it is really kernel dependent. The goal is to illustrate how this would be used. In this example, For 10_1 machines types and older we apply the following host kernel related compats: 1) Make sure the KVM_REG_ARM_VENDOR_HYP_BMAP_2 exposed from v6.15 onwards is ignored/hidden. 2) Make sure TCR_EL1, PIRE0_EL1, PIR_EL1 are always seen by qemu although not exposed by KVM. They were unconditionnally exposed before v6.13 while from v6.13 they are only exposed if supported by the guest. This will allow 10_1 machines types and older machines to migrate forward and backward from old downstream kernels that do not feature those changes to newer kernels (>=3D v6.15). Signed-off-by: Eric Auger --- hw/arm/virt.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 175023897a..c4f9b82c38 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -97,6 +97,23 @@ static GlobalProperty arm_virt_compat[] =3D { }; static const size_t arm_virt_compat_len =3D G_N_ELEMENTS(arm_virt_compat); =20 +/* + * if a 10_1 machine type or older is used: + * 1) make sure TCR_EL1, PIRE0_EL1, PIR_EL1 are enforced, even if they are= not + * exposed by the kernel + * 2) hide KVM_REG_ARM_VENDOR_HYP_BMAP_2 + */ +static GlobalProperty arm_virt_kernel_compat_10_1[] =3D { + /* KVM_REG_ARM_VENDOR_HYP_BMAP_2 */ + { TYPE_ARM_CPU, "kvm-hidden-regs", "0x6030000000160003" }, + /* TCR_EL1, PIRE0_EL1, PIR_EL1 */ + { TYPE_ARM_CPU, "kvm-enforced-regs", + "0x603000000013c103, 0x603000000013c512, 0x603000000013c513" }, +}; +static const size_t arm_virt_kernel_compat_10_1_len =3D + G_N_ELEMENTS(arm_virt_kernel_compat_10_1); + + /* * This cannot be called from the virt_machine_class_init() because * TYPE_VIRT_MACHINE is abstract and mc->compat_props g_ptr_array_new() @@ -3539,6 +3556,8 @@ static void virt_machine_10_1_options(MachineClass *m= c) virt_machine_10_2_options(mc); mc->smbios_memory_device_size =3D 2047 * TiB; compat_props_add(mc->compat_props, hw_compat_10_1, hw_compat_10_1_len); + compat_props_add(mc->compat_props, + arm_virt_kernel_compat_10_1, arm_virt_kernel_compat_1= 0_1_len); } DEFINE_VIRT_MACHINE(10, 1) =20 --=20 2.49.0