From nobody Fri Jul 24 23:30:48 2026 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF6902F4A05 for ; Wed, 22 Jul 2026 05:52:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784699530; cv=none; b=N8WQA12C+C0FWNFOZTu1WPbSeUhBUU+bp+SG+uQgwXZ2H6n1g3H+siCtMCcVrbCg29VoZolLPMQ00fRhvu7i57GNN/IuF/4VzQVuxum5Mf1tF3w2bhxSlxfXwlIDiRj0RbMI+USxHuwVQJse2DQq7PLbuDMYEBpIV2ExVI4JWcs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784699530; c=relaxed/simple; bh=VCtdpA7YF5advtu0bupY3qxS+FfCYS20j33S3YKJEL0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=FZ7N1gMRxRkhN9k4bySJV1znln9vM8wWypIbglYTzPI3okV/fPT5d6f3XhSrR0f5SUpG7tZcSnyzg/VbnMEI1BUttdN4cKMb5/0dghpKUzGM/y/yuqYM5u365PkTj8XocOZKpLD22knvlHO7Miq8Oi2RVBsKvxtKVTfOHfPkJ3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=OrzUCzXK; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="OrzUCzXK" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2ce7d2adef4so124754815ad.3 for ; Tue, 21 Jul 2026 22:52:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1784699527; x=1785304327; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=k8TjMER5gISFlQ4SyOMczlZr2aeQOkgr25LHVZl1LkM=; b=OrzUCzXKdkGF/5MZeWvAXPbPQUNMdx0DAeA2hkrliNfEFnQkV8/UFk/ZK6fpLZomoS UMK/D3lbxq36aqqxZrL6oaI2sLXzvCCXBMcAjL/62lbCweIlmzBW2jDl/mGXzhR0bdYb q5/hJg08sBsVMi80g03nfM2jRXGCw4MrI90I3eRE2TTvvsMXVMwU41yzjJeVDvlxuwr3 XzwZgF/iROYoKBrG+D8UatQc+8n1ewviWxsHzcPUzx8LB3XXL3dxeTQFykTbz8yzBtXh ZFvOr/M4nlItOwxlezB9pxvkwvIv3A/XrdWMe6km7LMAor+sjY8CA6ysYAucV0tAd6hJ aetg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784699527; x=1785304327; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=k8TjMER5gISFlQ4SyOMczlZr2aeQOkgr25LHVZl1LkM=; b=P91TmcvzDFTkKRqjJ2Hi0KDtkjqaQ/UdvoIUNXcOqOqrJYbSB1XuS45vLaGZ6J0PDR mwiOcElSJsk7/DuzrSxv656F5isuTND2zttD67k1P48KacuxEjZApFuyQSSeDSfzfuRd Mb6CDbkH8cEaN018QR8O3YLMGpBrYE3u/CDlFaxpWjcf9htK+nl02pqha7d5DCFh/S3K PMk4z4TnW72kRmT6FdoyBdUBVKLrttFVZhzwnLtRrwPXZCQJ/O+nlLhK4DvOym1q8xAd GetP4DN4iN1F3sj+W2+qcRgvzS6a+5ES/6E1w6bEpL5sydJIYH/gMkdY29kFGaaIjTc0 Euvg== X-Forwarded-Encrypted: i=1; AHgh+RoLae+kAwvRwHrLIM64o0YlXBiZtLTZ7s6r9N1bpn7jytwa8BtJjr1PvA+yIrXWKPhqPr1XVUhFninyHwY=@vger.kernel.org X-Gm-Message-State: AOJu0YxOmznI2QL0lVHcBEjhN+q0cDWwrdAwbg3C47j9+3ySE7HAmdqD h8rWDqI6/2mH11CwPRpqmbXNYc8/oSsohFlY+jmDGJVcUnN2yjkRzWclxqZiKCISUvg= X-Gm-Gg: AR+sD13eNubO2PfSN42tLIwFZ6SRaswWwmyBTiC2vVQqHuBF5wigUocIzZX+zb2OgpB umJ3DHSl1rL0beZzSbMFEWJnGCVcK0hVjab0tTKj8VLhoSCh/7YP43m8a000bR7/aY9AWiSJPJ8 XSgHXrBqkf9sJu9Rnb1hvk88JQyHcMmfYdHzaR+77s5gfVmyntm3ZBE0E9KgSQe5ujlRsZZ8hBa 2YynLmpmQ/exnq4O4tCAVvfffIVxsHIHraPcR7smAb5uYu8n5z3yJHs3a05uPfkL+ePaNVqiYu3 8wzfEguZ+XgdgA4kR86PrMLVPELv2kMQcngzvGFkfv8X36esS7sLg8a//HLW8UzEVd5V7x7S98s yfyO4Zf1JIL3bk5sQXc5WGpmO5/SC8ZoLhaJli+OtgPBshQ0eusYQpriR99VXgDcXlOfYp7XcqV R8CjZdTmTjL7Z4Wfshh0x+Uw== X-Received: by 2002:a05:6300:95:b0:3c3:75ff:95b6 with SMTP id adf61e73a8af0-3c3ad7a1c86mr22612818637.31.1784699527388; Tue, 21 Jul 2026 22:52:07 -0700 (PDT) Received: from sw07.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-3147e081985sm5133138eec.23.2026.07.21.22.52.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Jul 2026 22:52:06 -0700 (PDT) From: Yong-Xuan Wang Date: Tue, 21 Jul 2026 22:52:02 -0700 Subject: [PATCH v4 1/2] KVM: RISC-V: Clear former VCPU cache on virtualization disable Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260721-kvm-cpu-pm-v4-1-146bf942547d@sifive.com> References: <20260721-kvm-cpu-pm-v4-0-146bf942547d@sifive.com> In-Reply-To: <20260721-kvm-cpu-pm-v4-0-146bf942547d@sifive.com> To: Anup Patel , Atish Patra , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , =?utf-8?q?Radim_Kr=C4=8Dm=C3=A1=C5=99?= , Andrew Jones , Nutty Liu , Jinyu Tang Cc: greentime.hu@sifive.com, vincent.chen@sifive.com, zong.li@sifive.com, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Yong-Xuan Wang X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784699525; l=2593; i=yongxuan.wang@sifive.com; s=20260424; h=from:subject:message-id; bh=VCtdpA7YF5advtu0bupY3qxS+FfCYS20j33S3YKJEL0=; b=JVqpzramZxn2NBonw83b4SpU8QHsxBwc5n13yl9dz3mE7zfum5e6pHLRs/HTawKeBJnVx9mIq jRg8U/NpPlIAX207kgtiWv3SWtVQJ5M8q4R3IGkTs1fF02pBhi3t6i3 X-Developer-Key: i=yongxuan.wang@sifive.com; a=ed25519; pk=+8NCHB1ZJvZthQAmZspOAaqjo+/snaW8mFSiDx45HxY= When a CPU is taken offline or enters deep idle states, hypervisor CSR state is lost. The kvm_former_vcpu fast-path optimization caches the last VCPU that ran on each CPU to avoid expensive CSR restoration when the same VCPU is re-scheduled on the same CPU. However, if this cache is not cleared when CSR state is lost, the next VCPU entry will incorrectly skip CSR restoration, leading to corrupt hypervisor state. Add kvm_riscv_clear_former_vcpu() to invalidate the per-CPU cache and call it from kvm_arch_disable_virtualization_cpu() to ensure proper CSR restoration after CPU offline or system suspend events. Fixes: 1323a5cfe52c ("KVM: riscv: Skip CSR restore if VCPU is reloaded on t= he same core") Signed-off-by: Yong-Xuan Wang --- arch/riscv/include/asm/kvm_host.h | 2 ++ arch/riscv/kvm/main.c | 2 ++ arch/riscv/kvm/vcpu.c | 11 +++++++++++ 3 files changed, 15 insertions(+) diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm= _host.h index 60017ceec9d2..808b4ee1bd10 100644 --- a/arch/riscv/include/asm/kvm_host.h +++ b/arch/riscv/include/asm/kvm_host.h @@ -319,6 +319,8 @@ bool kvm_riscv_vcpu_stopped(struct kvm_vcpu *vcpu); =20 void kvm_riscv_vcpu_record_steal_time(struct kvm_vcpu *vcpu); =20 +void kvm_riscv_clear_former_vcpu(void); + /* Flags representing implementation specific details */ DECLARE_STATIC_KEY_FALSE(kvm_riscv_vsstage_tlb_no_gpa); =20 diff --git a/arch/riscv/kvm/main.c b/arch/riscv/kvm/main.c index 0924c75100a2..350e4f097d6e 100644 --- a/arch/riscv/kvm/main.c +++ b/arch/riscv/kvm/main.c @@ -69,6 +69,8 @@ void kvm_arch_disable_virtualization_cpu(void) csr_write(CSR_HEDELEG, 0); csr_write(CSR_HIDELEG, 0); =20 + kvm_riscv_clear_former_vcpu(); + kvm_riscv_nacl_disable(); } =20 diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c index cf6e231e76e2..e48338f907f4 100644 --- a/arch/riscv/kvm/vcpu.c +++ b/arch/riscv/kvm/vcpu.c @@ -26,6 +26,17 @@ =20 static DEFINE_PER_CPU(struct kvm_vcpu *, kvm_former_vcpu); =20 +void kvm_riscv_clear_former_vcpu(void) +{ + /* + * Clear the per-CPU former VCPU pointer because hypervisor CSR state + * will be lost. This ensures that the next VCPU entry will properly + * restore all CSRs instead of incorrectly skipping CSR restoration + * via the fast-path optimization. + */ + __this_cpu_write(kvm_former_vcpu, NULL); +} + const struct kvm_stats_desc kvm_vcpu_stats_desc[] =3D { KVM_GENERIC_VCPU_STATS(), STATS_DESC_COUNTER(VCPU, ecall_exit_stat), --=20 2.43.7 From nobody Fri Jul 24 23:30:48 2026 Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E20953321DE for ; Wed, 22 Jul 2026 05:52:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784699531; cv=none; b=P/R/D9CQu00x3OVYur84KxMCLjQ3NM+rYUFYNHQ5bekSw/ylY0g6NWufmI0Sl5EKsl4UyVwlJWBJWKmd6h9w+nqZrdw05MYLRL7nHp8OOp6/KGTq6itn4UtnsJ+c856wtla8NrIIr6fD0GeT8sgqY4VmW3wqUnfwaMYkcUgraO4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784699531; c=relaxed/simple; bh=9/BZKX/NjO1hAGhd35XFfyAO3E7Se5ihP2aZWgnztn8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=pWl8kWwnmmifxqiYUCfyeTsuhdpX+UO1NUnAB/vL05LAWkLZpwKMjYXow8sgewnQ4Xcs+qhV/0mYHU9t967COhye6ywYddssOMw6Ri7EeprI9Eb0oiFosThLatPCOss+hGRp7g1UQyUg4nKFXU0Uw+DvtJmwSaU+/wTrn1Smlbw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=XqEXnkh7; arc=none smtp.client-ip=209.85.215.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="XqEXnkh7" Received: by mail-pg1-f172.google.com with SMTP id 41be03b00d2f7-c999f162c9aso7757427a12.3 for ; Tue, 21 Jul 2026 22:52:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1784699529; x=1785304329; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=s+Vs18zUuW9jRHKRWV57BKDh6kBFOJsRh9ahu/u+GUs=; b=XqEXnkh7xwgdA/jMUrtWc29Qvlx0ZM6T3Osa32YvsUO3SeONCvyClj8KsI7ciIl6w1 4JqFIW/5gKSgrocQslr3ZlKRAmkGt+wXJcsYDu9vUro9xmdK7Ffl9ScED8NNUOZgy3PR tkeh94kBrX03QbfJ/sAr5EX9y6IPfvP1vQzeG5zRtQYlAbj2ru2cw3FSuzAoRBTi632d ozVhFjDXoo6Kd9eX64XyBIq/kjNIb2sw4SLxAsgA2c8wrpjCT938feSqJAe968ybfMmo dLPHqdAmrsb2eYb3JMLSbZAs4BVGSAtEkrpxVfgJxQeWIsAWovxBwxSUQBhauEHdn8We 7Xng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784699529; x=1785304329; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=s+Vs18zUuW9jRHKRWV57BKDh6kBFOJsRh9ahu/u+GUs=; b=OtyJUhxFc3KE/M7vu0qMqKoQ3t2C1iLgihA1Tdlp8GacWIXPa0EbO2Eiyjj9Ssl9D9 3X1h3AHYvu323R1qW419SWPvnB5wZc91CcVrKk94fbNCjS5NkB300t3iaiS8JneZ5TkM PCX2eTwVcIJKLLKozKccGTGxbgwSMyP4Z0eWOt8GaRbwEkRNPudC2GMPkouduqzJREhw 4OQlA5gRlx41kVlMZJ/MLMrxc8W8MtgtIbE8uR2Zq/jQWsAqUJOKgY1vmb3MCJ4b5BZ+ 48aiFi/0r1FZtbHETtGtS72/Z3FkJpTmre/a/ZfWIPqAQ5ILlY6XUGrHVl6QW+9fZi8d v+QA== X-Forwarded-Encrypted: i=1; AHgh+RpEhaykQvrnUp/YVF1DcsRJ37aHTx3y27Y+zgzdEm/+QdkOkEpaAfC5105jcl/U86qbhI6JUvvNhoDwy0E=@vger.kernel.org X-Gm-Message-State: AOJu0YwAQhiRIiRMIMFyAanaakFr8hsMYpHSc1hsJd4SfkifYWZKv+P7 DbTP7cmlCZV2O6NYXnr2NdsjS4NTOpSdq+xJENkSgDluSr3V+93OFNDk/OvMfF2LQI4= X-Gm-Gg: AR+sD13oFVvCU1SeYRYjriWHGJXdm6e/9MqjXikTgfWj58Id83uO3GywmyNKOu3/Xw5 dB4wwWCpazcp7g45YMFHeo4m7zf210jXDWZcTYIC65k495T8HBtjmK6d9DNHjQW9k6Ftx6yejIi k+2oMZXe2vUAg4h9Q7desfCPZ9E18sd+0AyNUYgaXRcZNrfM/Eje22RVL83GD4t+5FYkv3GMVqu fIA7hX3FuAGiuhfbkFuF6yFiJTce+bWJM/YqggbcatgJv1oM9tZSYOqFKwSx7fbrMz8FAZfcRTh WIEjtD14sZYsTKHiC1akQbSN9DCKHyT2e35QJxo/TRnKaJJB7J0YMnXQc5Gm2bOzo3F8TJr8Af9 WoLYoUHNxkgD7IB4+QnnRy/Uq0CoNNs22rAO4mjVYpz1T93hWzH5vyMbCM5AIiSsuaWp1AjvnDp syH7Eftt5xb6LclgaKPRvCiA== X-Received: by 2002:a05:6300:95:b0:3c3:719d:dfe2 with SMTP id adf61e73a8af0-3c3ad95dd8cmr22080498637.36.1784699529275; Tue, 21 Jul 2026 22:52:09 -0700 (PDT) Received: from sw07.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-3147e081985sm5133138eec.23.2026.07.21.22.52.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Jul 2026 22:52:07 -0700 (PDT) From: Yong-Xuan Wang Date: Tue, 21 Jul 2026 22:52:03 -0700 Subject: [PATCH v4 2/2] KVM: RISC-V: Add CPU PM notifier for non-retention idle states Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260721-kvm-cpu-pm-v4-2-146bf942547d@sifive.com> References: <20260721-kvm-cpu-pm-v4-0-146bf942547d@sifive.com> In-Reply-To: <20260721-kvm-cpu-pm-v4-0-146bf942547d@sifive.com> To: Anup Patel , Atish Patra , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , =?utf-8?q?Radim_Kr=C4=8Dm=C3=A1=C5=99?= , Andrew Jones , Nutty Liu , Jinyu Tang Cc: greentime.hu@sifive.com, vincent.chen@sifive.com, zong.li@sifive.com, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Yong-Xuan Wang X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784699525; l=11836; i=yongxuan.wang@sifive.com; s=20260424; h=from:subject:message-id; bh=9/BZKX/NjO1hAGhd35XFfyAO3E7Se5ihP2aZWgnztn8=; b=niIBXZSIQus8Jg/hSaPimYNJO/9uNn2M97DA3DTsHcAo3Rhg/aVvGP8FpaZEwd1DvWUXCQwSj cdpsKyDFxnrCLlky72TMk8WrJal86b9LaongPc3l2+BEOX8ChchtW9G X-Developer-Key: i=yongxuan.wang@sifive.com; a=ed25519; pk=+8NCHB1ZJvZthQAmZspOAaqjo+/snaW8mFSiDx45HxY= Register a CPU_PM notifier to restore hypervisor CSR content during CPU non-retention idle states. When a CPU enters a deep idle state that powers off the CPU domain, hypervisor CSRs and VS CSRs lose their state and must be saved before entry and restored after exit. This completes KVM's power management coverage for RISC-V: - CPU hotplug: handled by kvm_online_cpu/kvm_offline_cpu (cpuhp callbacks) - System suspend: handled by kvm_suspend/kvm_resume (syscore ops) - CPU idle (retention): no action needed, CSRs are retained - CPU idle (non-retention): handled by this CPU_PM notifier Signed-off-by: Yong-Xuan Wang --- arch/riscv/include/asm/kvm_aia.h | 15 +++- arch/riscv/kvm/aia.c | 144 ++++++++++++++++++++++-------------= ---- arch/riscv/kvm/main.c | 87 ++++++++++++++++++----- 3 files changed, 163 insertions(+), 83 deletions(-) diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_= aia.h index c67ec5ac0a14..7c1c3250598f 100644 --- a/arch/riscv/include/asm/kvm_aia.h +++ b/arch/riscv/include/asm/kvm_aia.h @@ -165,8 +165,19 @@ int kvm_riscv_aia_alloc_hgei(int cpu, struct kvm_vcpu = *owner, void __iomem **hgei_va, phys_addr_t *hgei_pa); void kvm_riscv_aia_free_hgei(int cpu, int hgei); =20 -void kvm_riscv_aia_enable(void); -void kvm_riscv_aia_disable(void); +/** + * kvm_riscv_aia_enable() - Enable AIA support on current CPU + * @full_cleanup: true =3D full hardware init (CPU hotplug/module load), + * false =3D lightweight CSR restore (CPU non-retention idl= e resume) + */ +void kvm_riscv_aia_enable(bool full_cleanup); + +/** + * kvm_riscv_aia_disable() - Disable AIA support on current CPU + * @full_cleanup: true =3D full hardware cleanup (CPU hotplug/module exit), + * false =3D lightweight CSR save (CPU non-retention idle e= ntry) + */ +void kvm_riscv_aia_disable(bool full_cleanup); int kvm_riscv_aia_init(void); void kvm_riscv_aia_exit(void); =20 diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c index bafb009c5ce5..4cf351f91c64 100644 --- a/arch/riscv/kvm/aia.c +++ b/arch/riscv/kvm/aia.c @@ -25,6 +25,7 @@ struct aia_hgei_control { unsigned long free_bitmap; struct kvm_vcpu *owners[BITS_PER_LONG]; unsigned int nr_hgei; + unsigned long saved_value; }; static DEFINE_PER_CPU(struct aia_hgei_control, aia_hgei); static int hgei_parent_irq; @@ -532,50 +533,59 @@ static void aia_hgei_exit(void) free_percpu_irq(hgei_parent_irq, &aia_hgei); } =20 -void kvm_riscv_aia_enable(void) +void kvm_riscv_aia_enable(bool full_cleanup) { - const struct imsic_global_config *gc; - const struct imsic_local_config *lc; struct aia_hgei_control *hgctrl; unsigned long flags; - int aia_nr_hgei; =20 if (!kvm_riscv_aia_available()) return; =20 - gc =3D imsic_get_global_config(); - lc =3D (gc) ? this_cpu_ptr(gc->local) : NULL; hgctrl =3D this_cpu_ptr(&aia_hgei); =20 - /* Figure-out number of bits in HGEIE */ - csr_write(CSR_HGEIE, -1UL); - hgctrl->nr_hgei =3D fls_long(csr_read(CSR_HGEIE)); - csr_write(CSR_HGEIE, 0); - if (hgctrl->nr_hgei) - hgctrl->nr_hgei--; + if (full_cleanup) { + const struct imsic_global_config *gc; + const struct imsic_local_config *lc; + int aia_nr_hgei; =20 - /* - * Number of usable per-HART HGEI lines should be minimum of - * per-HART IMSIC guest files and number of bits in HGEIE. - */ - if (lc) - hgctrl->nr_hgei =3D min((ulong)hgctrl->nr_hgei, lc->nr_guest_files); - else - hgctrl->nr_hgei =3D 0; + gc =3D imsic_get_global_config(); + lc =3D (gc) ? this_cpu_ptr(gc->local) : NULL; =20 - /* Update the number of IMSIC guest files across all HARTs */ - aia_nr_hgei =3D atomic_read(&kvm_riscv_aia_nr_hgei); - do { - if (aia_nr_hgei <=3D hgctrl->nr_hgei) - break; - } while (!atomic_try_cmpxchg(&kvm_riscv_aia_nr_hgei, &aia_nr_hgei, hgctrl= ->nr_hgei)); + /* Figure-out number of bits in HGEIE */ + csr_write(CSR_HGEIE, -1UL); + hgctrl->nr_hgei =3D fls_long(csr_read(CSR_HGEIE)); + csr_write(CSR_HGEIE, 0); + if (hgctrl->nr_hgei) + hgctrl->nr_hgei--; =20 - raw_spin_lock_irqsave(&hgctrl->lock, flags); - if (!hgctrl->free_bitmap_initialized) { - hgctrl->free_bitmap =3D (hgctrl->nr_hgei) ? GENMASK_ULL(hgctrl->nr_hgei,= 1) : 0; - hgctrl->free_bitmap_initialized =3D true; + /* + * Number of usable per-HART HGEI lines should be minimum of + * per-HART IMSIC guest files and number of bits in HGEIE. + */ + if (lc) + hgctrl->nr_hgei =3D min((ulong)hgctrl->nr_hgei, lc->nr_guest_files); + else + hgctrl->nr_hgei =3D 0; + + /* Update the number of IMSIC guest files across all HARTs */ + aia_nr_hgei =3D atomic_read(&kvm_riscv_aia_nr_hgei); + do { + if (aia_nr_hgei <=3D hgctrl->nr_hgei) + break; + } while (!atomic_try_cmpxchg(&kvm_riscv_aia_nr_hgei, &aia_nr_hgei, + hgctrl->nr_hgei)); + + raw_spin_lock_irqsave(&hgctrl->lock, flags); + if (!hgctrl->free_bitmap_initialized) { + hgctrl->free_bitmap =3D (hgctrl->nr_hgei) ? + GENMASK_ULL(hgctrl->nr_hgei, 1) : + 0; + hgctrl->free_bitmap_initialized =3D true; + } + raw_spin_unlock_irqrestore(&hgctrl->lock, flags); + } else { + csr_write(CSR_HGEIE, hgctrl->saved_value); } - raw_spin_unlock_irqrestore(&hgctrl->lock, flags); =20 csr_write(CSR_HVICTL, aia_hvictl_value(false)); csr_write(CSR_HVIPRIO1, 0x0); @@ -587,16 +597,17 @@ void kvm_riscv_aia_enable(void) csr_write(CSR_HVIPRIO2H, 0x0); #endif =20 - /* Enable per-CPU SGEI interrupt */ - enable_percpu_irq(hgei_parent_irq, - irq_get_trigger_type(hgei_parent_irq)); + if (full_cleanup) + enable_percpu_irq(hgei_parent_irq, + irq_get_trigger_type(hgei_parent_irq)); csr_set(CSR_HIE, BIT(IRQ_S_GEXT)); + /* Enable IRQ filtering for overflow interrupt only if sscofpmf is presen= t */ if (__riscv_isa_extension_available(NULL, RISCV_ISA_EXT_SSCOFPMF)) csr_set(CSR_HVIEN, BIT(IRQ_PMU_OVF)); } =20 -void kvm_riscv_aia_disable(void) +void kvm_riscv_aia_disable(bool full_cleanup) { int i; unsigned long flags; @@ -609,42 +620,49 @@ void kvm_riscv_aia_disable(void) =20 if (__riscv_isa_extension_available(NULL, RISCV_ISA_EXT_SSCOFPMF)) csr_clear(CSR_HVIEN, BIT(IRQ_PMU_OVF)); - /* Disable per-CPU SGEI interrupt */ - csr_clear(CSR_HIE, BIT(IRQ_S_GEXT)); - disable_percpu_irq(hgei_parent_irq); =20 - csr_write(CSR_HVICTL, aia_hvictl_value(false)); - - raw_spin_lock_irqsave(&hgctrl->lock, flags); - - for (i =3D 0; i <=3D hgctrl->nr_hgei; i++) { - vcpu =3D hgctrl->owners[i]; - if (!vcpu) - continue; + if (full_cleanup) { + /* Disable per-CPU SGEI interrupt */ + csr_clear(CSR_HIE, BIT(IRQ_S_GEXT)); + disable_percpu_irq(hgei_parent_irq); + } =20 - /* - * We release hgctrl->lock before notifying IMSIC - * so that we don't have lock ordering issues. - */ - raw_spin_unlock_irqrestore(&hgctrl->lock, flags); + csr_write(CSR_HVICTL, aia_hvictl_value(false)); =20 - /* Notify IMSIC */ - kvm_riscv_vcpu_aia_imsic_release(vcpu); + if (full_cleanup) { + raw_spin_lock_irqsave(&hgctrl->lock, flags); =20 - /* - * Wakeup VCPU if it was blocked so that it can - * run on other HARTs - */ - if (csr_read(CSR_HGEIE) & BIT(i)) { - csr_clear(CSR_HGEIE, BIT(i)); - kvm_vcpu_kick(vcpu); + for (i =3D 0; i <=3D hgctrl->nr_hgei; i++) { + vcpu =3D hgctrl->owners[i]; + if (!vcpu) + continue; + + /* + * We release hgctrl->lock before notifying IMSIC + * so that we don't have lock ordering issues. + */ + raw_spin_unlock_irqrestore(&hgctrl->lock, flags); + + /* Notify IMSIC */ + kvm_riscv_vcpu_aia_imsic_release(vcpu); + + /* + * Wakeup VCPU if it was blocked so that it can + * run on other HARTs + */ + if (csr_read(CSR_HGEIE) & BIT(i)) { + csr_clear(CSR_HGEIE, BIT(i)); + kvm_vcpu_kick(vcpu); + } + + raw_spin_lock_irqsave(&hgctrl->lock, flags); } =20 - raw_spin_lock_irqsave(&hgctrl->lock, flags); + raw_spin_unlock_irqrestore(&hgctrl->lock, flags); + } else { + hgctrl->saved_value =3D csr_read(CSR_HGEIE); } =20 - raw_spin_unlock_irqrestore(&hgctrl->lock, flags); - put_cpu_ptr(&aia_hgei); } =20 diff --git a/arch/riscv/kvm/main.c b/arch/riscv/kvm/main.c index 350e4f097d6e..b48bbd1855c1 100644 --- a/arch/riscv/kvm/main.c +++ b/arch/riscv/kvm/main.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -33,14 +34,9 @@ long kvm_arch_dev_ioctl(struct file *filp, return -EINVAL; } =20 -int kvm_arch_enable_virtualization_cpu(void) +/* Initialize hypervisor CSRs - called during CPU online and non-retention= idle resume */ +static void kvm_riscv_csr_enable(void) { - int rc; - - rc =3D kvm_riscv_nacl_enable(); - if (rc) - return rc; - csr_write(CSR_HEDELEG, 0); csr_write(CSR_HIDELEG, 0); =20 @@ -48,16 +44,11 @@ int kvm_arch_enable_virtualization_cpu(void) csr_write(CSR_HCOUNTEREN, 0x02); =20 csr_write(CSR_HVIP, 0); - - kvm_riscv_aia_enable(); - - return 0; } =20 -void kvm_arch_disable_virtualization_cpu(void) +/* Clear hypervisor CSRs - called during CPU offline and non-retention idl= e entry */ +static void kvm_riscv_csr_disable(void) { - kvm_riscv_aia_disable(); - /* * After clearing the hideleg CSR, the host kernel will receive * spurious interrupts if hvip CSR has pending interrupts and the @@ -70,10 +61,52 @@ void kvm_arch_disable_virtualization_cpu(void) csr_write(CSR_HIDELEG, 0); =20 kvm_riscv_clear_former_vcpu(); +} + +int kvm_arch_enable_virtualization_cpu(void) +{ + int rc; + + rc =3D kvm_riscv_nacl_enable(); + if (rc) + return rc; + + kvm_riscv_csr_enable(); + kvm_riscv_aia_enable(true); =20 + return 0; +} + +void kvm_arch_disable_virtualization_cpu(void) +{ + kvm_riscv_aia_disable(true); + kvm_riscv_csr_disable(); kvm_riscv_nacl_disable(); } =20 +static int kvm_riscv_cpu_pm_notifier(struct notifier_block *self, unsigned= long cmd, void *v) +{ + switch (cmd) { + case CPU_PM_EXIT: + case CPU_PM_ENTER_FAILED: + kvm_riscv_csr_enable(); + kvm_riscv_aia_enable(false); + return NOTIFY_OK; + case CPU_PM_ENTER: + kvm_riscv_aia_disable(false); + kvm_riscv_csr_disable(); + return NOTIFY_OK; + default: + break; + } + + return NOTIFY_DONE; +} + +static struct notifier_block kvm_riscv_cpu_pm_nb =3D { + .notifier_call =3D kvm_riscv_cpu_pm_notifier, +}; + static void kvm_riscv_teardown(void) { kvm_riscv_aia_exit(); @@ -174,22 +207,40 @@ static int __init riscv_kvm_init(void) =20 kvm_register_perf_callbacks(); =20 - rc =3D kvm_init(sizeof(struct kvm_vcpu), 0, THIS_MODULE); - if (rc) { - kvm_riscv_teardown(); - return rc; + /* Register CPU PM notifier for CPU idle non-retention states */ + if (IS_ENABLED(CONFIG_CPU_PM)) { + rc =3D cpu_pm_register_notifier(&kvm_riscv_cpu_pm_nb); + if (rc) { + kvm_err("Failed to register CPU PM notifier: %d\n", rc); + goto err_teardown; + } } =20 + rc =3D kvm_init(sizeof(struct kvm_vcpu), 0, THIS_MODULE); + if (rc) + goto err_unregister_cpu_pm; + if (kvm_riscv_aia_available()) kvm_info("AIA available with %d guest external interrupts\n", atomic_read(&kvm_riscv_aia_nr_hgei)); =20 return 0; + +err_unregister_cpu_pm: + if (IS_ENABLED(CONFIG_CPU_PM)) + cpu_pm_unregister_notifier(&kvm_riscv_cpu_pm_nb); +err_teardown: + kvm_riscv_teardown(); + return rc; } module_init(riscv_kvm_init); =20 static void __exit riscv_kvm_exit(void) { + /* Unregister CPU PM notifier */ + if (IS_ENABLED(CONFIG_CPU_PM)) + cpu_pm_unregister_notifier(&kvm_riscv_cpu_pm_nb); + kvm_exit(); =20 kvm_riscv_teardown(); --=20 2.43.7