From nobody Fri May 3 22:39:00 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1594462317; cv=none; d=zohomail.com; s=zohoarc; b=glImnpr1z+3GRoGj73m+ix/2b5tGm4QUM8EUQ8T4zyDUo1EuXQimH9ALl2z/cTPxhfed7kKQwYViBZsSg+SD47DjN54NTaQQV0LEXNi9RiJ1rbR8LcVz+Bqf2iVDd3Z7eVt38o/qqKQla16CaXpZxijy7/223HoZqDm76gROimU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594462317; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=V0XJQDKBVgwTNqB7JV+NoE0714DNBcenljLxGJNaaw4=; b=kfYTGwgxOgxvJyZSygbPIhiGCEC9kZEgavMlNI2Bv8AWRNraBSnSkiSF0OCZrCegSnfwnlMTI3MnTwaFlBGryTjpXZlYKxn67PJkC0i2yGYu2kiXnPcB3wWMMMCkmjz2HxvuotsRtxzNAK1hzhpewySjaA4QtYtYf+vEbbF3BLE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594462317401390.3682337648926; Sat, 11 Jul 2020 03:11:57 -0700 (PDT) Received: from localhost ([::1]:41418 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1juCU5-0002BW-Hg for importer@patchew.org; Sat, 11 Jul 2020 06:11:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43854) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1juCT0-0000YP-32 for qemu-devel@nongnu.org; Sat, 11 Jul 2020 06:10:46 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:45728 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1juCSx-0008Cp-Cw for qemu-devel@nongnu.org; Sat, 11 Jul 2020 06:10:45 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-206-zDSPhT-QMkS70ACslbHPKQ-1; Sat, 11 Jul 2020 06:10:39 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BDC51107ACCA; Sat, 11 Jul 2020 10:10:38 +0000 (UTC) Received: from kamzik.brq.redhat.com (unknown [10.40.192.42]) by smtp.corp.redhat.com (Postfix) with ESMTP id 79B9B610F2; Sat, 11 Jul 2020 10:10:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594462241; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V0XJQDKBVgwTNqB7JV+NoE0714DNBcenljLxGJNaaw4=; b=S/z1RjSo0PT1dRKDwZDskygrbbV2JFGcfsiTE+lyBmyY6acfoHo0oUbkeWJofNvOmgmKS0 gibQ0zYuMuWjGx0/x5McVXOZC+Xvj+F/cHmw4Bb7NtAE1o9KPtS16Oi4CFKSMebri+xLz8 tcHwRm2TQdjVDxoX8PkJGIon+nPxglw= X-MC-Unique: zDSPhT-QMkS70ACslbHPKQ-1 From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Subject: [PATCH 1/3] hw/arm/virt: Move post cpu realize check into its own function Date: Sat, 11 Jul 2020 12:10:31 +0200 Message-Id: <20200711101033.47371-2-drjones@redhat.com> In-Reply-To: <20200711101033.47371-1-drjones@redhat.com> References: <20200711101033.47371-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=drjones@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/11 06:10:41 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, beata.michalska@linaro.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" We'll add more to this new function in coming patches so we also state the gic must be created and call it below create_gic(). No functional change intended. Signed-off-by: Andrew Jones Reviewed-by: Peter Maydell --- hw/arm/virt.c | 43 +++++++++++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 16 deletions(-) diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 9005dae356be..cb2fa99b1ef5 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -1672,6 +1672,31 @@ static void finalize_gic_version(VirtMachineState *v= ms) } } =20 +/* + * virt_cpu_post_init() must be called after the CPUs have + * been realized and the GIC has been created. + */ +static void virt_cpu_post_init(VirtMachineState *vms) +{ + bool aarch64; + + aarch64 =3D object_property_get_bool(OBJECT(first_cpu), "aarch64", NUL= L); + + if (!kvm_enabled()) { + if (aarch64 && vms->highmem) { + int requested_pa_size =3D 64 - clz64(vms->highest_gpa); + int pamax =3D arm_pamax(ARM_CPU(first_cpu)); + + if (pamax < requested_pa_size) { + error_report("VCPU supports less PA bits (%d) than " + "requested by the memory map (%d)", + pamax, requested_pa_size); + exit(1); + } + } + } +} + static void machvirt_init(MachineState *machine) { VirtMachineState *vms =3D VIRT_MACHINE(machine); @@ -1873,22 +1898,6 @@ static void machvirt_init(MachineState *machine) fdt_add_timer_nodes(vms); fdt_add_cpu_nodes(vms); =20 - if (!kvm_enabled()) { - ARMCPU *cpu =3D ARM_CPU(first_cpu); - bool aarch64 =3D object_property_get_bool(OBJECT(cpu), "aarch64", = NULL); - - if (aarch64 && vms->highmem) { - int requested_pa_size, pamax =3D arm_pamax(cpu); - - requested_pa_size =3D 64 - clz64(vms->highest_gpa); - if (pamax < requested_pa_size) { - error_report("VCPU supports less PA bits (%d) than request= ed " - "by the memory map (%d)", pamax, requested_pa_= size); - exit(1); - } - } - } - memory_region_add_subregion(sysmem, vms->memmap[VIRT_MEM].base, machine->ram); if (machine->device_memory) { @@ -1900,6 +1909,8 @@ static void machvirt_init(MachineState *machine) =20 create_gic(vms); =20 + virt_cpu_post_init(vms); + fdt_add_pmu_nodes(vms); =20 create_uart(vms, VIRT_UART, sysmem, serial_hd(0)); --=20 2.25.4 From nobody Fri May 3 22:39:00 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1594462316; cv=none; d=zohomail.com; s=zohoarc; b=AtauLWO6LzG8yLhbO/6H7iMHDvl1YPbzdFBCFDz4DaFqPw9GhlfhwmZBoJkNHj8h8Np4z0zgyarY3RjVDBKFahaAYUmU1Y+5cJVI2l/5ykKvAX3D/3yv0OzPUgQ3o5adQTI8Egh8xikv7wX2ZCvyldZgxcc1pfbAFeG54mW2z/M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594462316; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=G/xh1SpgA7XBEzJH+OIkhc62kkci0Ps/7UMsQ+8Gus8=; b=DIbkznwNM28NOeTYK71Cb51E3Z74YA3Uq1PK5B/Gs7s8awlgZGdz9T37YLD87W0Qu28yJkDVUHTYaPZo8LLLCjig7M7DFEfIytRaReL8+OvQX4k2Km7vfS8a3eSXB968wHZL8cIKp1lXfnt96Ou9JglLXE2OQQj6OB7tJ+f7RVk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594462316697646.4468774147902; Sat, 11 Jul 2020 03:11:56 -0700 (PDT) Received: from localhost ([::1]:41562 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1juCU6-0002F1-4o for importer@patchew.org; Sat, 11 Jul 2020 06:11:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43888) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1juCT1-0000Yc-1G for qemu-devel@nongnu.org; Sat, 11 Jul 2020 06:10:47 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:40634 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1juCSy-0008EF-LP for qemu-devel@nongnu.org; Sat, 11 Jul 2020 06:10:46 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-416-W4q6zXckN26SmLxOlAGMDw-1; Sat, 11 Jul 2020 06:10:41 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 61F1410059A9; Sat, 11 Jul 2020 10:10:40 +0000 (UTC) Received: from kamzik.brq.redhat.com (unknown [10.40.192.42]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1E062610F2; Sat, 11 Jul 2020 10:10:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594462244; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=G/xh1SpgA7XBEzJH+OIkhc62kkci0Ps/7UMsQ+8Gus8=; b=czqNP7vZSYyMGCaWIu8Ckjmdt8obK4p1+ehpHLKVmn0xunr426hlhHkZhfjOtjyCu4wIff Hvo5P+lH7RY6bIbsnZbSRCo0H/76bkVQriqCUgdqhF+cRPuf62XBjAJWw6QgaGZLrMJCL2 PwhS98WCe6nczQK6WLFv5qo1f2tM34s= X-MC-Unique: W4q6zXckN26SmLxOlAGMDw-1 From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Subject: [PATCH 2/3] hw/arm/virt: Move kvm pmu setup to virt_cpu_post_init Date: Sat, 11 Jul 2020 12:10:32 +0200 Message-Id: <20200711101033.47371-3-drjones@redhat.com> In-Reply-To: <20200711101033.47371-1-drjones@redhat.com> References: <20200711101033.47371-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=drjones@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/11 06:10:41 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, beata.michalska@linaro.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Move the KVM PMU setup part of fdt_add_pmu_nodes() to virt_cpu_post_init(), which is a more appropriate location. Now fdt_add_pmu_nodes() is also named more appropriately, because it no longer does anything but fdt node creation. No functional change intended. Signed-off-by: Andrew Jones --- hw/arm/virt.c | 33 +++++++++++++++------------------ 1 file changed, 15 insertions(+), 18 deletions(-) diff --git a/hw/arm/virt.c b/hw/arm/virt.c index cb2fa99b1ef5..63ef530933e5 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -521,30 +521,15 @@ static void fdt_add_gic_node(VirtMachineState *vms) =20 static void fdt_add_pmu_nodes(const VirtMachineState *vms) { - CPUState *cpu; - ARMCPU *armcpu; + ARMCPU *armcpu =3D ARM_CPU(first_cpu); uint32_t irqflags =3D GIC_FDT_IRQ_FLAGS_LEVEL_HI; =20 - CPU_FOREACH(cpu) { - armcpu =3D ARM_CPU(cpu); - if (!arm_feature(&armcpu->env, ARM_FEATURE_PMU)) { - return; - } - if (kvm_enabled()) { - if (kvm_irqchip_in_kernel()) { - kvm_arm_pmu_set_irq(cpu, PPI(VIRTUAL_PMU_IRQ)); - } - kvm_arm_pmu_init(cpu); - } - } - if (vms->gic_version =3D=3D VIRT_GIC_VERSION_2) { irqflags =3D deposit32(irqflags, GIC_FDT_IRQ_PPI_CPU_START, GIC_FDT_IRQ_PPI_CPU_WIDTH, (1 << vms->smp_cpus) - 1); } =20 - armcpu =3D ARM_CPU(qemu_get_cpu(0)); qemu_fdt_add_subnode(vms->fdt, "/pmu"); if (arm_feature(&armcpu->env, ARM_FEATURE_V8)) { const char compat[] =3D "arm,armv8-pmuv3"; @@ -1678,11 +1663,23 @@ static void finalize_gic_version(VirtMachineState *= vms) */ static void virt_cpu_post_init(VirtMachineState *vms) { - bool aarch64; + bool aarch64, pmu; + CPUState *cpu; =20 aarch64 =3D object_property_get_bool(OBJECT(first_cpu), "aarch64", NUL= L); + pmu =3D object_property_get_bool(OBJECT(first_cpu), "pmu", NULL); =20 - if (!kvm_enabled()) { + if (kvm_enabled()) { + CPU_FOREACH(cpu) { + if (pmu) { + assert(arm_feature(&ARM_CPU(cpu)->env, ARM_FEATURE_PMU)); + if (kvm_irqchip_in_kernel()) { + kvm_arm_pmu_set_irq(cpu, PPI(VIRTUAL_PMU_IRQ)); + } + kvm_arm_pmu_init(cpu); + } + } + } else { if (aarch64 && vms->highmem) { int requested_pa_size =3D 64 - clz64(vms->highest_gpa); int pamax =3D arm_pamax(ARM_CPU(first_cpu)); --=20 2.25.4 From nobody Fri May 3 22:39:00 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1594462329; cv=none; d=zohomail.com; s=zohoarc; b=gsxDcMmBV8MSAYgrrQ6Uh5vDnplgH+ILTzururbVXfz1moNvbu/SaSgrA0pCYuU7SFFDbimhFuGBn3DkMecAG0bgQxhTSVFZi+WQ+vhi7+dka8wNTfmsl0zcjcf6hqMEct35lT/BIfUWdEvGNtHVojwjhI24IISrLDXGhwzzZ60= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594462329; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=z1UiAD4gFaOW3zEEUp/Atpv/D7pvK1DiGqXGaa7CvaA=; b=KjV+sPYaFEc5xms3JtY4mqbQdlKTZwcMnCZxnk5QzefArozFG6qcEEeDJjZ8s9NeMIxvAmO2MLRDc8Z4HAk+BEGQNl4Fn3wgdMrn/lSW/TT/BB6/eFEQytcQqkH4C1SBqg8qy1RJ8FJluYpOYiOhGXU04wJux9K0l+s0cXA8aMI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159446232908578.41946338674654; Sat, 11 Jul 2020 03:12:09 -0700 (PDT) Received: from localhost ([::1]:42712 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1juCUJ-0002h7-PC for importer@patchew.org; Sat, 11 Jul 2020 06:12:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43928) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1juCT2-0000ay-Jd for qemu-devel@nongnu.org; Sat, 11 Jul 2020 06:10:48 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:34776 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1juCSz-0008Et-UG for qemu-devel@nongnu.org; Sat, 11 Jul 2020 06:10:48 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-86-jD0tJPjLNh6zXQ71StZqHQ-1; Sat, 11 Jul 2020 06:10:43 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 33C11107ACCA; Sat, 11 Jul 2020 10:10:42 +0000 (UTC) Received: from kamzik.brq.redhat.com (unknown [10.40.192.42]) by smtp.corp.redhat.com (Postfix) with ESMTP id B6D2C610F2; Sat, 11 Jul 2020 10:10:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594462245; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z1UiAD4gFaOW3zEEUp/Atpv/D7pvK1DiGqXGaa7CvaA=; b=D5XVsdU35KVMgDKf1G+OkI5YoToc5C0lnGhzFvVOALVvZ2/LQe5u9yqRkqoD9/C8ZaC/R4 Vj/TgCgSTBCgREq+S/+6nLhVBECbQp35ICvtVDRhDgB2LB3b3F0MPwuuL+R1xA0qfy7h96 Y40LZI5XDc9oBvixwJN0nF44liGFv/o= X-MC-Unique: jD0tJPjLNh6zXQ71StZqHQ-1 From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Subject: [PATCH 3/3] hw/arm/virt: Implement kvm-steal-time Date: Sat, 11 Jul 2020 12:10:33 +0200 Message-Id: <20200711101033.47371-4-drjones@redhat.com> In-Reply-To: <20200711101033.47371-1-drjones@redhat.com> References: <20200711101033.47371-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=drjones@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=drjones@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/11 06:10:41 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, beata.michalska@linaro.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" We add the kvm-steal-time CPU property and implement it for machvirt. A tiny bit of refactoring was also done to allow pmu and pvtime to use the same vcpu device helper functions. Signed-off-by: Andrew Jones --- docs/system/arm/cpu-features.rst | 11 +++++ hw/arm/virt.c | 33 ++++++++++++++- include/hw/arm/virt.h | 2 + target/arm/cpu.c | 10 +++++ target/arm/cpu.h | 4 ++ target/arm/kvm.c | 20 +++++++++ target/arm/kvm32.c | 5 +++ target/arm/kvm64.c | 70 +++++++++++++++++++++++++++++--- target/arm/kvm_arm.h | 24 ++++++++++- target/arm/monitor.c | 2 +- tests/qtest/arm-cpu-features.c | 25 ++++++++++-- 11 files changed, 193 insertions(+), 13 deletions(-) diff --git a/docs/system/arm/cpu-features.rst b/docs/system/arm/cpu-feature= s.rst index 2d5c06cd016b..4e590e6e9f54 100644 --- a/docs/system/arm/cpu-features.rst +++ b/docs/system/arm/cpu-features.rst @@ -200,6 +200,17 @@ the list of KVM VCPU features and their descriptions. adjustment, also restoring the legacy (pre-5.0) behavior. =20 + kvm-steal-time Since v5.1, kvm-steal-time is enabled by + default when KVM is enabled, the feature is + supported, and the guest is 64-bit. + + When kvm-steal-time is enabled a 64-bit guest + can account for time its CPUs were not running + due to the host not scheduling the corresponding + VCPU threads. The accounting statistics may + influence the guest scheduler behavior and/or be + exposed to the guest userspace. + SVE CPU Properties =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 63ef530933e5..21965a1665ca 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -151,6 +151,7 @@ static const MemMapEntry base_memmap[] =3D { [VIRT_PCDIMM_ACPI] =3D { 0x09070000, MEMORY_HOTPLUG_IO_LEN }, [VIRT_ACPI_GED] =3D { 0x09080000, ACPI_GED_EVT_SEL_LEN }, [VIRT_NVDIMM_ACPI] =3D { 0x09090000, NVDIMM_ACPI_IO_LEN}, + [VIRT_PVTIME] =3D { 0x090a0000, 0x00010000 }, [VIRT_MMIO] =3D { 0x0a000000, 0x00000200 }, /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that siz= e */ [VIRT_PLATFORM_BUS] =3D { 0x0c000000, 0x02000000 }, @@ -1663,13 +1664,26 @@ static void finalize_gic_version(VirtMachineState *= vms) */ static void virt_cpu_post_init(VirtMachineState *vms) { - bool aarch64, pmu; + bool aarch64, pmu, steal_time; CPUState *cpu; =20 aarch64 =3D object_property_get_bool(OBJECT(first_cpu), "aarch64", NUL= L); pmu =3D object_property_get_bool(OBJECT(first_cpu), "pmu", NULL); + steal_time =3D object_property_get_bool(OBJECT(first_cpu), + "kvm-steal-time", NULL); =20 if (kvm_enabled()) { + hwaddr pvtime_base =3D vms->memmap[VIRT_PVTIME].base; + hwaddr pvtime_size =3D vms->memmap[VIRT_PVTIME].size; + + if (steal_time) { + MemoryRegion *pvtime =3D g_new(MemoryRegion, 1); + + memory_region_init_ram(pvtime, NULL, "pvtime", pvtime_size, NU= LL); + memory_region_add_subregion(get_system_memory(), pvtime_base, + pvtime); + } + CPU_FOREACH(cpu) { if (pmu) { assert(arm_feature(&ARM_CPU(cpu)->env, ARM_FEATURE_PMU)); @@ -1678,6 +1692,17 @@ static void virt_cpu_post_init(VirtMachineState *vms) } kvm_arm_pmu_init(cpu); } + if (steal_time) { + /* + * We need 64 bytes for each CPU[*]. One 64k region gives + * us up to 1024 CPUs, or some growing room for the pvtime + * structure for less CPUs. + * + * [*] See Linux kernel arch/arm64/include/asm/pvclock-abi= .h + */ + assert(pvtime_size >=3D MACHINE_GET_CLASS(vms)->max_cpus *= 64); + kvm_arm_pvtime_init(cpu, pvtime_base + 64 * cpu->cpu_index= ); + } } } else { if (aarch64 && vms->highmem) { @@ -1842,6 +1867,11 @@ static void machvirt_init(MachineState *machine) object_property_set_bool(cpuobj, "kvm-no-adjvtime", true, NULL= ); } =20 + if (vmc->kvm_no_steal_time && + object_property_find(cpuobj, "kvm-steal-time", NULL)) { + object_property_set_bool(cpuobj, false, "kvm-steal-time", NULL= ); + } + if (vmc->no_pmu && object_property_find(cpuobj, "pmu", NULL)) { object_property_set_bool(cpuobj, "pmu", false, NULL); } @@ -2528,6 +2558,7 @@ static void virt_machine_5_0_options(MachineClass *mc) mc->numa_mem_supported =3D true; vmc->acpi_expose_flash =3D true; mc->auto_enable_numa_with_memdev =3D false; + vmc->kvm_no_steal_time =3D true; } DEFINE_VIRT_MACHINE(5, 0) =20 diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h index 54bcf17afd35..b5153afedcdf 100644 --- a/include/hw/arm/virt.h +++ b/include/hw/arm/virt.h @@ -80,6 +80,7 @@ enum { VIRT_PCDIMM_ACPI, VIRT_ACPI_GED, VIRT_NVDIMM_ACPI, + VIRT_PVTIME, VIRT_LOWMEMMAP_LAST, }; =20 @@ -126,6 +127,7 @@ typedef struct { bool no_ged; /* Machines < 4.2 has no support for ACPI GED device */ bool kvm_no_adjvtime; bool acpi_expose_flash; + bool kvm_no_steal_time; } VirtMachineClass; =20 typedef struct { diff --git a/target/arm/cpu.c b/target/arm/cpu.c index 5050e1843a85..f894ee3fdee8 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -1308,6 +1308,16 @@ void arm_cpu_finalize_features(ARMCPU *cpu, Error **= errp) return; } } + + if (kvm_enabled()) { +#ifdef TARGET_AARCH64 + kvm_arm_steal_time_finalize(cpu, &local_err); + if (local_err !=3D NULL) { + error_propagate(errp, local_err); + return; + } +#endif + } } =20 static void arm_cpu_realizefn(DeviceState *dev, Error **errp) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 9e8ed423ea1d..a4d4cb640c77 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -24,6 +24,7 @@ #include "hw/registerfields.h" #include "cpu-qom.h" #include "exec/cpu-defs.h" +#include "qapi/qapi-types-common.h" =20 /* ARM processors have a weak memory model */ #define TCG_GUEST_DEFAULT_MO (0) @@ -859,6 +860,9 @@ struct ARMCPU { bool kvm_vtime_dirty; uint64_t kvm_vtime; =20 + /* KVM steal time */ + OnOffAuto kvm_steal_time; + /* Uniprocessor system with MP extensions */ bool mp_is_up; =20 diff --git a/target/arm/kvm.c b/target/arm/kvm.c index 8bb7318378b5..093a290453f6 100644 --- a/target/arm/kvm.c +++ b/target/arm/kvm.c @@ -192,6 +192,18 @@ static void kvm_no_adjvtime_set(Object *obj, bool valu= e, Error **errp) ARM_CPU(obj)->kvm_adjvtime =3D !value; } =20 +#ifdef TARGET_AARCH64 +static bool kvm_steal_time_get(Object *obj, Error **errp) +{ + return ARM_CPU(obj)->kvm_steal_time !=3D ON_OFF_AUTO_OFF; +} + +static void kvm_steal_time_set(Object *obj, bool value, Error **errp) +{ + ARM_CPU(obj)->kvm_steal_time =3D value ? ON_OFF_AUTO_ON : ON_OFF_AUTO_= OFF; +} +#endif + /* KVM VCPU properties should be prefixed with "kvm-". */ void kvm_arm_add_vcpu_properties(Object *obj) { @@ -207,6 +219,14 @@ void kvm_arm_add_vcpu_properties(Object *obj) "the virtual counter. VM stopped t= ime " "will be counted."); } + +#ifdef TARGET_AARCH64 + cpu->kvm_steal_time =3D ON_OFF_AUTO_AUTO; + object_property_add_bool(obj, "kvm-steal-time", kvm_steal_time_get, + kvm_steal_time_set); + object_property_set_description(obj, "kvm-steal-time", + "Set off to disable KVM steal time."); +#endif } =20 bool kvm_arm_pmu_supported(void) diff --git a/target/arm/kvm32.c b/target/arm/kvm32.c index 0af46b41c847..d3f3195077fa 100644 --- a/target/arm/kvm32.c +++ b/target/arm/kvm32.c @@ -560,6 +560,11 @@ void kvm_arm_pmu_init(CPUState *cs) qemu_log_mask(LOG_UNIMP, "%s: not implemented\n", __func__); } =20 +void kvm_arm_pvtime_init(CPUState *cs, uint64_t ipa) +{ + qemu_log_mask(LOG_UNIMP, "%s: not implemented\n", __func__); +} + #define ARM_REG_DFSR ARM_CP15_REG32(0, 5, 0, 0) #define ARM_REG_TTBCR ARM_CP15_REG32(0, 2, 0, 2) /* diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index 116923790550..a5f4633f87c3 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -17,6 +17,7 @@ #include =20 #include "qemu-common.h" +#include "qapi/error.h" #include "cpu.h" #include "qemu/timer.h" #include "qemu/error-report.h" @@ -398,19 +399,20 @@ static CPUWatchpoint *find_hw_watchpoint(CPUState *cp= u, target_ulong addr) return NULL; } =20 -static bool kvm_arm_pmu_set_attr(CPUState *cs, struct kvm_device_attr *att= r) +static bool kvm_arm_set_device_attr(CPUState *cs, struct kvm_device_attr *= attr, + const char *name) { int err; =20 err =3D kvm_vcpu_ioctl(cs, KVM_HAS_DEVICE_ATTR, attr); if (err !=3D 0) { - error_report("PMU: KVM_HAS_DEVICE_ATTR: %s", strerror(-err)); + error_report("%s: KVM_HAS_DEVICE_ATTR: %s", name, strerror(-err)); return false; } =20 err =3D kvm_vcpu_ioctl(cs, KVM_SET_DEVICE_ATTR, attr); if (err !=3D 0) { - error_report("PMU: KVM_SET_DEVICE_ATTR: %s", strerror(-err)); + error_report("%s: KVM_SET_DEVICE_ATTR: %s", name, strerror(-err)); return false; } =20 @@ -427,7 +429,7 @@ void kvm_arm_pmu_init(CPUState *cs) if (!ARM_CPU(cs)->has_pmu) { return; } - if (!kvm_arm_pmu_set_attr(cs, &attr)) { + if (!kvm_arm_set_device_attr(cs, &attr, "PMU")) { error_report("failed to init PMU"); abort(); } @@ -444,12 +446,29 @@ void kvm_arm_pmu_set_irq(CPUState *cs, int irq) if (!ARM_CPU(cs)->has_pmu) { return; } - if (!kvm_arm_pmu_set_attr(cs, &attr)) { + if (!kvm_arm_set_device_attr(cs, &attr, "PMU")) { error_report("failed to set irq for PMU"); abort(); } } =20 +void kvm_arm_pvtime_init(CPUState *cs, uint64_t ipa) +{ + struct kvm_device_attr attr =3D { + .group =3D KVM_ARM_VCPU_PVTIME_CTRL, + .attr =3D KVM_ARM_VCPU_PVTIME_IPA, + .addr =3D (uint64_t)&ipa, + }; + + if (!ARM_CPU(cs)->kvm_steal_time) { + return; + } + if (!kvm_arm_set_device_attr(cs, &attr, "PVTIME IPA")) { + error_report("failed to init PVTIME IPA"); + abort(); + } +} + static int read_sys_reg32(int fd, uint32_t *pret, uint64_t id) { uint64_t ret; @@ -652,6 +671,47 @@ bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures = *ahcf) return true; } =20 +void kvm_arm_steal_time_finalize(ARMCPU *cpu, Error **errp) +{ + static bool has_steal_time; + static bool probed; + int fdarray[3]; + + if (!probed) { + probed =3D true; + if (kvm_check_extension(kvm_state, KVM_CAP_VCPU_ATTRIBUTES)) { + if (!kvm_arm_create_scratch_host_vcpu(NULL, fdarray, NULL)) { + error_report("Failed to create scratch VCPU"); + abort(); + } + + has_steal_time =3D kvm_device_check_attr(fdarray[2], + KVM_ARM_VCPU_PVTIME_CTR= L, + KVM_ARM_VCPU_PVTIME_IPA= ); + + kvm_arm_destroy_scratch_host_vcpu(fdarray); + } + } + + if (cpu->kvm_steal_time =3D=3D ON_OFF_AUTO_AUTO) { + if (!has_steal_time || !arm_feature(&cpu->env, ARM_FEATURE_AARCH64= )) { + cpu->kvm_steal_time =3D ON_OFF_AUTO_OFF; + } else { + cpu->kvm_steal_time =3D ON_OFF_AUTO_ON; + } + } else if (cpu->kvm_steal_time =3D=3D ON_OFF_AUTO_ON) { + if (!has_steal_time) { + error_setg(errp, "'kvm-steal-time' cannot be enabled " + "on this host"); + return; + } else if (!arm_feature(&cpu->env, ARM_FEATURE_AARCH64)) { + error_setg(errp, "'kvm-steal-time' cannot be enabled " + "for AArch32 guests"); + return; + } + } +} + bool kvm_arm_aarch32_supported(void) { return kvm_check_extension(kvm_state, KVM_CAP_ARM_EL1_32BIT); diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h index adb38514bf20..21d776912b55 100644 --- a/target/arm/kvm_arm.h +++ b/target/arm/kvm_arm.h @@ -267,6 +267,16 @@ void kvm_arm_set_cpu_features_from_host(ARMCPU *cpu); */ void kvm_arm_add_vcpu_properties(Object *obj); =20 +/** + * kvm_arm_steal_time_finalize: + * @cpu: ARMCPU for which to finalize kvm-steal-time + * @errp: Pointer to Error* for error propagation + * + * Validate the kvm-steal-time property selection and set its default + * based on KVM support and guest configuration. + */ +void kvm_arm_steal_time_finalize(ARMCPU *cpu, Error **errp); + /** * kvm_arm_aarch32_supported: * @@ -340,6 +350,16 @@ int kvm_arm_vgic_probe(void); =20 void kvm_arm_pmu_set_irq(CPUState *cs, int irq); void kvm_arm_pmu_init(CPUState *cs); + +/** + * kvm_arm_pvtime_init: + * @cs: CPUState + * @ipa: Per-vcpu guest physical base address of the pvtime structures + * + * Initializes PVTIME for the VCPU, setting the PVTIME IPA to @ipa. + */ +void kvm_arm_pvtime_init(CPUState *cs, uint64_t ipa); + int kvm_arm_set_irq(int cpu, int irqtype, int irq, int level); =20 #else @@ -355,6 +375,7 @@ static inline void kvm_arm_set_cpu_features_from_host(A= RMCPU *cpu) } =20 static inline void kvm_arm_add_vcpu_properties(Object *obj) {} +static inline void kvm_arm_steal_time_finalize(ARMCPU *cpu, Error **errp) = {} =20 static inline bool kvm_arm_aarch32_supported(void) { @@ -383,9 +404,8 @@ static inline int kvm_arm_vgic_probe(void) =20 static inline void kvm_arm_pmu_set_irq(CPUState *cs, int irq) {} static inline void kvm_arm_pmu_init(CPUState *cs) {} - +static inline void kvm_arm_pvtime_init(CPUState *cs, uint64_t ipa) {} static inline void kvm_arm_sve_get_vls(CPUState *cs, unsigned long *map) {} - static inline void kvm_arm_get_virtual_time(CPUState *cs) {} static inline void kvm_arm_put_virtual_time(CPUState *cs) {} #endif diff --git a/target/arm/monitor.c b/target/arm/monitor.c index ba6e01abd037..bd3590604a71 100644 --- a/target/arm/monitor.c +++ b/target/arm/monitor.c @@ -103,7 +103,7 @@ static const char *cpu_model_advertised_features[] =3D { "sve128", "sve256", "sve384", "sve512", "sve640", "sve768", "sve896", "sve1024", "sve1152", "sve1280", "sve1408", "sve1536", "sve1664", "sve1792", "sve1920", "sve2048", - "kvm-no-adjvtime", + "kvm-no-adjvtime", "kvm-steal-time", NULL }; =20 diff --git a/tests/qtest/arm-cpu-features.c b/tests/qtest/arm-cpu-features.c index f7e062c1891e..91b181f38268 100644 --- a/tests/qtest/arm-cpu-features.c +++ b/tests/qtest/arm-cpu-features.c @@ -452,6 +452,7 @@ static void test_query_cpu_model_expansion(const void *= data) assert_set_feature(qts, "max", "pmu", true); =20 assert_has_not_feature(qts, "max", "kvm-no-adjvtime"); + assert_has_not_feature(qts, "max", "kvm-steal-time"); =20 if (g_str_equal(qtest_get_arch(), "aarch64")) { assert_has_feature_enabled(qts, "max", "aarch64"); @@ -493,6 +494,7 @@ static void test_query_cpu_model_expansion_kvm(const vo= id *data) assert_set_feature(qts, "host", "kvm-no-adjvtime", false); =20 if (g_str_equal(qtest_get_arch(), "aarch64")) { + bool kvm_supports_steal_time; bool kvm_supports_sve; char max_name[8], name[8]; uint32_t max_vq, vq; @@ -500,6 +502,10 @@ static void test_query_cpu_model_expansion_kvm(const v= oid *data) QDict *resp; char *error; =20 + assert_error(qts, "cortex-a15", + "We cannot guarantee the CPU type 'cortex-a15' works " + "with KVM on this host", NULL); + assert_has_feature_enabled(qts, "host", "aarch64"); =20 /* Enabling and disabling pmu should always work. */ @@ -507,16 +513,26 @@ static void test_query_cpu_model_expansion_kvm(const = void *data) assert_set_feature(qts, "host", "pmu", false); assert_set_feature(qts, "host", "pmu", true); =20 - assert_error(qts, "cortex-a15", - "We cannot guarantee the CPU type 'cortex-a15' works " - "with KVM on this host", NULL); - + /* + * Some features would be enabled by default, but they're disabled + * because this instance of KVM doesn't support them. Test that the + * features are present, and, when enabled, issue further tests. + */ + assert_has_feature(qts, "host", "kvm-steal-time"); assert_has_feature(qts, "host", "sve"); + resp =3D do_query_no_props(qts, "host"); + kvm_supports_steal_time =3D resp_get_feature(resp, "kvm-steal-time= "); kvm_supports_sve =3D resp_get_feature(resp, "sve"); vls =3D resp_get_sve_vls(resp); qobject_unref(resp); =20 + if (kvm_supports_steal_time) { + /* If we have steal-time then we should be able to toggle it. = */ + assert_set_feature(qts, "host", "kvm-steal-time", false); + assert_set_feature(qts, "host", "kvm-steal-time", true); + } + if (kvm_supports_sve) { g_assert(vls !=3D 0); max_vq =3D 64 - __builtin_clzll(vls); @@ -577,6 +593,7 @@ static void test_query_cpu_model_expansion_kvm(const vo= id *data) assert_has_not_feature(qts, "host", "aarch64"); assert_has_not_feature(qts, "host", "pmu"); assert_has_not_feature(qts, "host", "sve"); + assert_has_not_feature(qts, "host", "kvm-steal-time"); } =20 qtest_quit(qts); --=20 2.25.4