From nobody Fri Oct 31 16:00:46 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1746520513; cv=none; d=zohomail.com; s=zohoarc; b=FsLr4AY6vQ6quXynahliOhOlaq1KYm8oZrrBhfFtN9ThA88MHY6uco/cKyjX9ngZdWmvyRab+GyRIc70/6nM4XxHEydFFgGR+2LjddB9ooBjcyb/gaxVqE91ZCaIr0r4oFpJMFgXg1G4VnseIeW0T2Ibts9PR820BkjZEC5rUsI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1746520513; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=st45ZJ9i+zKbNGPJ/gNiCriyCwXBLOobUVRy9hZSFAA=; b=QmINxmhMln088VQHorRLIP5H/o1IxFzYqM7qh649iWHa/LmipafHdfvub/jhMrZ+gHafStfSjyZjZ6jbHOuURZ6FivhRPVk7yK1KfYrPGd6I9Es2we5sUNkhDXJv1HVeffeQBnbR9Yz8j1zvjPVZtiwm1jK4Uzua/DxR0c4oZdI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1746520513174134.10132957394285; Tue, 6 May 2025 01:35:13 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.976747.1363893 (Exim 4.92) (envelope-from ) id 1uCDlK-0004Fr-5Y; Tue, 06 May 2025 08:34:50 +0000 Received: by outflank-mailman (output) from mailman id 976747.1363893; Tue, 06 May 2025 08:34:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlK-0004Fk-2J; Tue, 06 May 2025 08:34:50 +0000 Received: by outflank-mailman (input) for mailman id 976747; Tue, 06 May 2025 08:34:48 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlI-00041m-5G for xen-devel@lists.xenproject.org; Tue, 06 May 2025 08:34:48 +0000 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [2a00:1450:4864:20::42b]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f7235d67-2a54-11f0-9ffb-bf95429c2676; Tue, 06 May 2025 10:34:46 +0200 (CEST) Received: by mail-wr1-x42b.google.com with SMTP id ffacd0b85a97d-39141ffa9fcso5709969f8f.0 for ; Tue, 06 May 2025 01:34:46 -0700 (PDT) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a099ae7cc6sm13132515f8f.55.2025.05.06.01.34.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 May 2025 01:34:45 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f7235d67-2a54-11f0-9ffb-bf95429c2676 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1746520485; x=1747125285; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=st45ZJ9i+zKbNGPJ/gNiCriyCwXBLOobUVRy9hZSFAA=; b=a8ZLtsKkNfiDnWk9AHFIHBY98W9ixl0Z71swNHBGKXOBG4GnMUYVx+fLGP3tAj86yi gFECgZUXWV7kXkO1EYoqAmUnS2Pw3l3SrO6IC8i19OfpTCnNkHRR6KC61Nux3dIEgFA1 Ee67in+25XPxQxgJ0q72BsyPcRlv/1YUW7hIU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746520485; x=1747125285; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=st45ZJ9i+zKbNGPJ/gNiCriyCwXBLOobUVRy9hZSFAA=; b=dGq9O4wPMzAzOemq3HKZTGbxnON8dWdtVTcNcuBf35KCyHL9s7R+HDkB0rnrfBQ+xx nejsUq5XU028k5o7yOC6UftlsCvjm3BCObobu9fpdeNbNRQ01pco+l0M/0MBj6oZ0a8E 7yF7KSRm5DZcp6QLgB6a0V4/JTiwGIu9s3WqQwOG/OkmnUSj2vh2f9frCD4T/mbudqAu kemmO83jfrhjzF7mriEMOzLgOViOoOBNIlWDPw5XmBTakoU0LJXtu2BfBHZvVW3XKjkS xKVDb0W6ZXsuU3tC1CjvnKIKENG45yTasbmeTA2oQvINoju8CCusfMephHV7q8HHCgPg 0Jpg== X-Gm-Message-State: AOJu0YxeXIN8X/ohwUUdYv4vqJJLKULTN9079qH2/H1YI+7oP88qZO+u xPtl6zWNkOaAFFOH9f5KyNbfyXEUQzzGFcG0U7RN5svUy23PqHhfhvYVnDHYYccuaOQWVuGdkDh R X-Gm-Gg: ASbGnct+zEM8EBXwl6ldZJaGujUoVjrfoK3L13LAwsl5ScUpRnIxri9+0wtNPaqZEh4 37xCXPtfL5r927DO3Z4b/o1GhpMR5NflsZLcQQkm+IpvVMVa8gJPnw0SIafd54b4wWG0Gtxq7HI g3D5Y313vm04PI+tXFMzXwhN4HlQBqewOcQUIr0fHPycnz/m1M8ROftyUmNBYjQIsKd3iHho5Qk 4pUUffYDRmGqz9i5aEUFs6RzDjryR/PK/nKWpQv7uxp9wOs7GsCxDifabRt3UtYGeixYvoLpeA1 KITHmpNvSAfoXZ40bRaGzkVzO23TxXzxFFk11LTX89IZIA== X-Google-Smtp-Source: AGHT+IHVj/uKcBnWhZL2WS4VG4BiWU+QA7k/IU9uSQePRA0qdu0xcCV8sfNEBXnjmr16UVzoeHKt1g== X-Received: by 2002:a05:6000:4383:b0:39c:f0d:9146 with SMTP id ffacd0b85a97d-3a0ac1ff746mr1264138f8f.45.1746520485296; Tue, 06 May 2025 01:34:45 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH 1/9] x86/pv: fix MMUEXT_FLUSH_CACHE to flush all pCPU caches Date: Tue, 6 May 2025 10:31:40 +0200 Message-ID: <20250506083148.34963-2-roger.pau@citrix.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250506083148.34963-1-roger.pau@citrix.com> References: <20250506083148.34963-1-roger.pau@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1746520514828116600 The implementation of MMUEXT_FLUSH_CACHE is bogus, as it doesn't account to flush the cache of any previous pCPU where the current vCPU might have run, and hence is likely to not work as expected. Fix this by resorting to use the same logic as MMUEXT_FLUSH_CACHE_GLOBAL, which will be correct in all cases. Fixes: 534ffcb416bb ("Fix WBINVD by adding a new hypercall.") Signed-off-by: Roger Pau Monn=C3=A9 Acked-by: Christian Lindig Acked-by: Jan Beulich --- Alternatively, the hypercall could be made correct by keeping track of the pCPUs the vCPU has run on since the last cache flush. That's however more work. See later in the series. --- xen/arch/x86/mm.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 66c15a3c864f..38e214352201 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -3805,14 +3805,11 @@ long do_mmuext_op( break; =20 case MMUEXT_FLUSH_CACHE: - if ( unlikely(currd !=3D pg_owner) ) - rc =3D -EPERM; - else if ( unlikely(!cache_flush_permitted(currd)) ) - rc =3D -EACCES; - else - wbinvd(); - break; - + /* + * Dirty pCPU caches where the current vCPU has been scheduled= are + * not tracked, and hence we need to resort to a global cache + * flush for correctness. + */ case MMUEXT_FLUSH_CACHE_GLOBAL: if ( unlikely(currd !=3D pg_owner) ) rc =3D -EPERM; --=20 2.48.1 From nobody Fri Oct 31 16:00:46 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1746520508; cv=none; d=zohomail.com; s=zohoarc; b=lQbpvnI3wXUsITmJ7t3t4ab8/SA54ELAqlgYHRGX2IblOa1zoEwynccnxIRYGWgq571dRx6I3h7Uu8da4MRb7OZs6cHALO6+OLa6po5nAxrpuvuJ6oEJKPyskOQUlv2krcXbDKL/qa3x7M7W0as+aZ0ViLFzqUN8npDkN84HfA8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1746520508; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=uFdCdonHRi/V3I4AjrrMy18B5UWA7t/AwxBRWIfPBMg=; b=W7P3ikGGXeRBsOxBn90DHf/DbmmqbCcO1T3DS8vPyeSi7vAWkknuCaOnU1BFh/F3bUFjDPdkaUWvrq36rJwy+YXM+lBB0WxKQ6DkXvBM3pz6rerpmAcN+YLEoEFlP73t5kfvIkKRA3Z1hMi8cs7sUht9ah8htUWeKsgngztN8Ys= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1746520508117830.6135764373846; Tue, 6 May 2025 01:35:08 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.976748.1363898 (Exim 4.92) (envelope-from ) id 1uCDlK-0004MJ-GL; Tue, 06 May 2025 08:34:50 +0000 Received: by outflank-mailman (output) from mailman id 976748.1363898; Tue, 06 May 2025 08:34:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlK-0004Lh-Cy; Tue, 06 May 2025 08:34:50 +0000 Received: by outflank-mailman (input) for mailman id 976748; Tue, 06 May 2025 08:34:49 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlJ-00041m-5V for xen-devel@lists.xenproject.org; Tue, 06 May 2025 08:34:49 +0000 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [2a00:1450:4864:20::436]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f7bd8322-2a54-11f0-9ffb-bf95429c2676; Tue, 06 May 2025 10:34:47 +0200 (CEST) Received: by mail-wr1-x436.google.com with SMTP id ffacd0b85a97d-39c1ef4ae3aso3159025f8f.1 for ; Tue, 06 May 2025 01:34:47 -0700 (PDT) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a099ae0cd6sm12761237f8f.5.2025.05.06.01.34.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 May 2025 01:34:46 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f7bd8322-2a54-11f0-9ffb-bf95429c2676 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1746520486; x=1747125286; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uFdCdonHRi/V3I4AjrrMy18B5UWA7t/AwxBRWIfPBMg=; b=khONc6ytuiUIT1BJ+pmydxnoDLJL7xYURNcVYV2TPgZ+kQQYrgnV7nPCvNWEEsJDKB FrWEnKjvGNFAapErAI4FbOxLESigcYwLNPQZXOCKrqhxsUaKqyKP17ubHXRR57mcAydZ PqqHmwYdpuPo+z03OYgdLr5cf6myJk5bHkzGI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746520486; x=1747125286; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uFdCdonHRi/V3I4AjrrMy18B5UWA7t/AwxBRWIfPBMg=; b=jE/A902WPABNkVH53gPSk1nXHu1TqYQGw5HRDgVoSIrMc16EVfuGvnHGqP2ufHV2wl JDI1Swi2+IQEea/OI8ReyFOmbBMzfTwn9cnyOr5B0qda71Z4kYNzqs66fCz6p00WJJ8U YyxETfElmBfmrGfww/0bDGhsG7wV90H0YA1O2QB96qUSAimuWotGjH9HR24vKIhCLQId MJ5LSK5bQ1rJLw2zPYaPDJno5R6XYcRcgbPR+xbTEGVDIqtvxgYPkPk1K6sQpQhaZwz5 5rgmfEz8nTmETg0XEsdFmf4tMdAIdPup9GdLiApf5VKj0WNpYsjKUvh4KDMxeZvEL3wk RtjQ== X-Gm-Message-State: AOJu0YxpOPyfkgRkCr+wU8TFIO5WLgPP3cHOv7HRInF9uygESwfnSGBp yGknYyTWbttDoEykmbHEG6PXALNRsLiU25QzEhi1lv4RJMnOl2rkn+aaoY/Sj2Y0z6d+Uvl2lZ7 O X-Gm-Gg: ASbGncvUUdHjYs0P8EJzdmWqPGkw67eir056azg5C/iqtWsljRlUdy3chGwUftpsrDl rBvqhFHzJEbBZEQX2qFc7DPqwyLnAhzRczHYt3/WAL53CDwH5ChZ0Bj7x00umvsOGCOSgfBdGGm HAkOeEzWSBUDNypouZg+Q0XtIsf23UzAST3p0boON8Tmi2U4yYD9BPjBGue2PkdsEwG/530MZk2 eOcxHSM5cMyUanmLxvoWfpniOJ09Y3ytJZd/x5bQjihLRIRrkWTtlf33oYNprY8IOao6kJwlN3V k82ElcvRft0ffSwwm3VlIEVtMNHTtwk1353gBQiSMySXAA== X-Google-Smtp-Source: AGHT+IEEJgMoX/Ygxj+QlDyIS89kjsS1lbFo+fjNJSn53rHstrrpDwm8RRtBETzoxMFmPHbFlEJ1/g== X-Received: by 2002:a05:6000:2902:b0:39f:c05:c220 with SMTP id ffacd0b85a97d-3a0ab5b32bemr1901163f8f.22.1746520486355; Tue, 06 May 2025 01:34:46 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH 2/9] x86/pv: fix emulation of wb{,no}invd to flush all pCPU caches Date: Tue, 6 May 2025 10:31:41 +0200 Message-ID: <20250506083148.34963-3-roger.pau@citrix.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250506083148.34963-1-roger.pau@citrix.com> References: <20250506083148.34963-1-roger.pau@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1746520510261019000 The current emulation of wb{,no}invd is bogus for PV guests: it will only flush the current pCPU cache, without taking into account pCPUs where the vCPU had run previously. Since there's no tracking of dirty cache pCPUs currently, resort to flushing the cache on all host pCPUs. Also as a result of having no mechanism to broadcast a wbnoinvd instruction, resort to emulating it using wbinvd behavior, which is more expensive performance wise, but correct. Fixes: 2f6070f0b988 ("Priv-op emulation in Xen, for RDMSR/WRMSR/WBINVD") Signed-off-by: Roger Pau Monn=C3=A9 Acked-by: Christian Lindig --- Note further patches will limit the broadcast cache flush to only pCPU where the vCPU has ran. --- xen/arch/x86/pv/emul-priv-op.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/xen/arch/x86/pv/emul-priv-op.c b/xen/arch/x86/pv/emul-priv-op.c index 70150c272276..089d4cb4d905 100644 --- a/xen/arch/x86/pv/emul-priv-op.c +++ b/xen/arch/x86/pv/emul-priv-op.c @@ -1193,17 +1193,18 @@ static int cf_check cache_op( { ASSERT(op =3D=3D x86emul_wbinvd || op =3D=3D x86emul_wbnoinvd); =20 - /* Ignore the instruction if unprivileged. */ - if ( !cache_flush_permitted(current->domain) ) + /* + * Ignore the instruction if domain doesn't have cache control. + * Non-physdev domain attempted WBINVD; ignore for now since + * newer linux uses this in some start-of-day timing loops. + */ + if ( cache_flush_permitted(current->domain) ) /* - * Non-physdev domain attempted WBINVD; ignore for now since - * newer linux uses this in some start-of-day timing loops. + * Handle wbnoinvd as wbinvd, at the expense of higher cost. Broa= dcast + * the flush to all pCPUs, Xen doesn't track where the vCPU has ran + * previously. */ - ; - else if ( op =3D=3D x86emul_wbnoinvd /* && cpu_has_wbnoinvd */ ) - wbnoinvd(); - else - wbinvd(); + flush_all(FLUSH_CACHE); =20 return X86EMUL_OKAY; } --=20 2.48.1 From nobody Fri Oct 31 16:00:46 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1746520511; cv=none; d=zohomail.com; s=zohoarc; b=BkcHe7QS+B9q3FlUwEtXUAu4K74Pda0uZJqlraLpP/C8H9y5VO4OCTtGwL6+OzApaGgcM9qwHtSKxwe0fvSm8iSB/fKrqo9t1Xu5nFUB5VN2E3oB8b8D0i/6pWfHkLIkwdF6jxMTdHxlMqxyOa940vLFROIquaEiwyXQirDJZ2g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1746520511; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=19/KyvkkozHoKYd6THg8hoTBnLfqHO1KkIE+C5r6Ttw=; b=m0tTGxvAQ172NSKj1kv4bTEingwVVbMFgf5SLTxjCJC0aOFM230DloN0QqCrSHP7AQBxjsQBVFXAlc8UDmML8DdCx6kCbLT4IMFgz7xA9/yd7tEDdnALQLlOAz6i6BfB5y/wV6+ZmZyGdPg6vCo6jamWWdD2z1qUYitHHnuSDmk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1746520511415211.1156512690958; Tue, 6 May 2025 01:35:11 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.976749.1363905 (Exim 4.92) (envelope-from ) id 1uCDlK-0004QV-Qc; Tue, 06 May 2025 08:34:50 +0000 Received: by outflank-mailman (output) from mailman id 976749.1363905; Tue, 06 May 2025 08:34:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlK-0004PL-LH; Tue, 06 May 2025 08:34:50 +0000 Received: by outflank-mailman (input) for mailman id 976749; Tue, 06 May 2025 08:34:49 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlJ-00048v-Jn for xen-devel@lists.xenproject.org; Tue, 06 May 2025 08:34:49 +0000 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [2a00:1450:4864:20::331]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id f859a0a5-2a54-11f0-9eb4-5ba50f476ded; Tue, 06 May 2025 10:34:48 +0200 (CEST) Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-43cebe06e9eso33352265e9.3 for ; Tue, 06 May 2025 01:34:48 -0700 (PDT) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-441b2ad781csm205613275e9.8.2025.05.06.01.34.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 May 2025 01:34:47 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f859a0a5-2a54-11f0-9eb4-5ba50f476ded DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1746520487; x=1747125287; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=19/KyvkkozHoKYd6THg8hoTBnLfqHO1KkIE+C5r6Ttw=; b=DnEpj6Ggz/7xcVhoCfkCW6eRtzC8+kCfh/4vLG/Z1Tz9VUIw0gB2xMaKEa0Hn9Ztyc tvMrtz0PSUC+++8ohnfblXlo+ts2qv4RUQDNQl4i9KhvCUBtgVN+2vNo7YKZ4AXceCGq fmWircFhNXUT/wFIuMuu5QFmAK2l3zT/DX1sY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746520487; x=1747125287; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=19/KyvkkozHoKYd6THg8hoTBnLfqHO1KkIE+C5r6Ttw=; b=mvuNUhrh613unIxfaaZ2feAHxvDtQzELwyREHVZmyJIoKJbS2sun4IA74dZVJzvP6A EMlpb79jVqhT3jY9gsrs8ok2lrX8uArueqA53btGupyLO77ce02Ke7P69j+2yWEkNTqz jVUxFm+sz5ft1y9KWZXEtKn1UD9D/iZCxFjLgOjvnSFihBiZu6d6617J7nb/hqEVNydf BPHgmeNza92Ihhoz9tJyx5+f49xxQckfoKG3WM/qFXJ11RdtT6gF8YHC+0IVj7uCCie8 H9xLNmFEDg/UmQeKWnZzWYyhue+xQjkBpv0E8VrTEtIkM7rfIkWbB0WrWuyUBGgfbgJC YAMQ== X-Gm-Message-State: AOJu0YzS7NVPDyHLu5L8xohxoVUFfubMCXT7Q9BYBtTTGhqLDYiCbwcB I6bB3intj3EMUXV1K2WePShvLvb6d4jxGFuxRE1syfcX0JES7plcaRfqETf/4vHr98Iqp0n2Y9L + X-Gm-Gg: ASbGnct5SpwrEEMwItHDuJIY1sbZ87BES2vCAYrYClfew83n03ItNVxXXriDcdgUuwK LuHL0NexbXtI4TNRkF0TWoSKUXOvF1curWoQBgfZooKvuPCI4UbKcSv+366O0JXiYF2DiBwvzIN BaTYqHzFDiziWTwZpJaRvvWFN2Aav4xjareAovimpsEJk6I6u21Gt3TR8DyrMGCsSe66MEE8Dnk OArjZCw8/xtKoxtt3aR8KxUbM6FHHDo1kH4iorzE7VXEVpx3nvzy2Kwb/4XXfAhTKAlP1fE9fJ5 NUbFg1oO+S3eH+JPVzf7YZ8yvRp/cbky9kfGYhU9Y7JXCA== X-Google-Smtp-Source: AGHT+IGeiYHYC7We2Fl1uR/572QDRTgfKbWh2I7K7bHpEmD3xsWylfGZiO1SHAizD8Ez5pgi7kFCDA== X-Received: by 2002:a05:600c:c17:b0:43c:eeee:b706 with SMTP id 5b1f17b1804b1-441d0531273mr17628335e9.24.1746520487359; Tue, 06 May 2025 01:34:47 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Andrew Cooper , Anthony PERARD , Michal Orzel , Jan Beulich , Julien Grall , Stefano Stabellini Subject: [PATCH 3/9] xen/gnttab: limit cache flush operation to guests allowed cache control Date: Tue, 6 May 2025 10:31:42 +0200 Message-ID: <20250506083148.34963-4-roger.pau@citrix.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250506083148.34963-1-roger.pau@citrix.com> References: <20250506083148.34963-1-roger.pau@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1746520511936019000 Whether a domain is allowed to issue cache-control operations is reported by the cache_flush_permitted() check. Introduce such check to limit the availability of GNTTABOP_cache_flush to only guests that are granted cache control. Fixes: 18e8d22fe750 ("introduce GNTTABOP_cache_flush") Signed-off-by: Roger Pau Monn=C3=A9 Acked-by: Christian Lindig --- xen/common/grant_table.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c index e75ff98aff1c..d874ac5f1241 100644 --- a/xen/common/grant_table.c +++ b/xen/common/grant_table.c @@ -3780,6 +3780,11 @@ long do_grant_table_op( =20 if ( unlikely(!guest_handle_okay(cflush, count)) ) goto out; + + rc =3D -EPERM; + if ( !cache_flush_permitted(current->domain) ) + goto out; + rc =3D gnttab_cache_flush(cflush, &opaque_in, count); if ( rc >=3D 0 ) { --=20 2.48.1 From nobody Fri Oct 31 16:00:46 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1746522062; cv=none; d=zohomail.com; s=zohoarc; b=QVSobt+ojQLkj2KFmKv7bsVgzjGlqbdCGt09AYWR4TbMTdsioIwWlwfyPJTOYr31eseG3QLuTuwXYbu1yH+noxORRjIoif2pSiS4pRsLv3JsOiTB22UDqdMy3NzseaM+W5nqCZorkoj4fhPg64HjmgY3oxJ+afTuvc+jEw3szOE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1746522062; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=N+W9lEzZ3YPqEiQclAW6B381YvWea5VfolcXUQFEZ2o=; b=GAjLlDjDUmphzor1/Ry6r0qfovUwKp4Cyr7ODvRX78Sgn/z3usj+PRnh+4TJze9RYFfihsc+oFTvo+1UUo7czY1J+GT6inh5V8xg8H2+Br+86Rk+vWuG9sX/Kx2wR2cyy/h+5xqdOpzNZwlCGfCvK/yaYI4McdqpiZP9y+rmT7Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1746522062104264.72475000234886; Tue, 6 May 2025 02:01:02 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.976893.1364003 (Exim 4.92) (envelope-from ) id 1uCEAM-0006bM-GH; Tue, 06 May 2025 09:00:42 +0000 Received: by outflank-mailman (output) from mailman id 976893.1364003; Tue, 06 May 2025 09:00:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCEAM-0006bF-D6; Tue, 06 May 2025 09:00:42 +0000 Received: by outflank-mailman (input) for mailman id 976893; Tue, 06 May 2025 09:00:41 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCEAL-0006b9-2R for xen-devel@lists.xenproject.org; Tue, 06 May 2025 09:00:41 +0000 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [2a00:1450:4864:20::633]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 9561280a-2a58-11f0-9eb4-5ba50f476ded; Tue, 06 May 2025 11:00:40 +0200 (CEST) Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-ace94273f0dso251252266b.3 for ; Tue, 06 May 2025 02:00:40 -0700 (PDT) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a099ae34e2sm12632186f8f.24.2025.05.06.01.34.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 May 2025 01:34:48 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9561280a-2a58-11f0-9eb4-5ba50f476ded DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1746522039; x=1747126839; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=N+W9lEzZ3YPqEiQclAW6B381YvWea5VfolcXUQFEZ2o=; b=WAgbHJofF7HgM3TMPI8TiQstYkTEQQwtthrPWF58srf1m8N5cb6pV0eLFI+tf8Xi8a 4tFBhyZp9yg0272GIGPEcVO6l6HfoUSp8/fkLlsCjKiYBiIbsNmD3ZAZDw7AW7whoEa4 +3mZZToVGaJZ3zzUkQVZGWkF8TazU6zB7DPOY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746522039; x=1747126839; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N+W9lEzZ3YPqEiQclAW6B381YvWea5VfolcXUQFEZ2o=; b=ULECoRQa5H+GO103fTqwpdPlnboMArrWMUl0iZOM8SKwQpuGw1jhed81V5Fxh9NkWG o+AMuiFEbt2x4KerQ8o8ltAMMnnN8V4pfoLSVL5wV5y3Az3412OO+7tmyDlYT/lhlw7g kZrCNdpuyWqVhRp/DniaOmnsYyx0gEmdQS8t8qwVUKN1seyPKd6s5j5PwAlTxGQ7Mn4w u2l3RBD0wib9RRa16fV0evGFYJzzOXXypnNfBJlRWyG6ZUW0jAnHXamVUgvyl14hgny5 U7P88MnY1ETAdX5qV/ytfZ7Xg8dQm2z3wsqS2dI7U7auOIc7/cU74sDYn0KD5RH6le6m R/tw== X-Gm-Message-State: AOJu0YxhAgsGxRhMMo0CY2EXd0WM96LnfmFgqQ+UHQbkXZ8vaEoisJEt X7y9+4lhcWQLcpDHtYuuohQw9/gfQWoKsWMzjW8H37Cs5hNWXgRrM0RQJr9ACCgxocaU/obKkYm g X-Gm-Gg: ASbGncvUe163xnv2zbm6U4d6TWb/BgQrrTp6e69uErJ3QiZjaW4wBHFY41XrV41ht1a gewdQa/1vyq/om+WyjB5Kr9LNAESRiShFbrGobR9UpaTjNkuCwRjtCRfr5WqjmbZkwNf9emTHRP iMHhKCehn4O73maQY6FmYZYz5LbFKXVHRluPmziU4N/gXEzG9tgphhdAgjsdnpmywTWWUjpI4Ya gaYHIXLcNjURMBgwhy36rDl40m5drKaGSIEinqoX3cb6EqMWD9bluA81dRNKIdf+WEptZFUGbB2 KrRQASIOrknxN70Kzd0TzTWfgxCvNtSgVKCm0YxdKlezvA== X-Google-Smtp-Source: AGHT+IGhxmBqCb7xdHw4VhQuhl4tgy/BOqgPbggvS46dewpZnmq+sg0FvdSOBL2lOWhwvg/D+TI0Bw== X-Received: by 2002:a05:6000:1886:b0:39c:1f10:c736 with SMTP id ffacd0b85a97d-3a09fdbc906mr9154530f8f.43.1746520488398; Tue, 06 May 2025 01:34:48 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH 4/9] x86/gnttab: do not implement GNTTABOP_cache_flush Date: Tue, 6 May 2025 10:31:43 +0200 Message-ID: <20250506083148.34963-5-roger.pau@citrix.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250506083148.34963-1-roger.pau@citrix.com> References: <20250506083148.34963-1-roger.pau@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1746522064362019000 The current underlying implementation of GNTTABOP_cache_flush on x86 won't work as expected. The provided {clean,invalidate}_dcache_va_range() helpers only do a local pCPU cache flush, so the cache of previous pCPUs where the vCPU might have run are not flushed. However instead of attempting to fix this, make the GNTTABOP_cache_flush operation only available to ARM. There are no known users on x86, the implementation is broken, and other architectures don't have grant-table support yet. With that operation not implemented on x86, the related {clean,invalidate}_dcache_va_range() helpers can also be removed. Fixes: f62dc81c2df7 ("x86: introduce more cache maintenance operations") Fixes: 18e8d22fe750 ("introduce GNTTABOP_cache_flush") Signed-off-by: Roger Pau Monn=C3=A9 Acked-by: Christian Lindig --- I've attempted to introduce a new arch_do_grant_table_op() in a separate arch-specific file, but it required exposing too much functionality from the common grant_table.c, ifdefying is probably better for the time being. --- xen/arch/x86/include/asm/flushtlb.h | 15 --------------- xen/common/grant_table.c | 6 ++++++ 2 files changed, 6 insertions(+), 15 deletions(-) diff --git a/xen/arch/x86/include/asm/flushtlb.h b/xen/arch/x86/include/asm= /flushtlb.h index bb0ad58db49b..d0c9120b5faf 100644 --- a/xen/arch/x86/include/asm/flushtlb.h +++ b/xen/arch/x86/include/asm/flushtlb.h @@ -182,21 +182,6 @@ void flush_area_mask(const cpumask_t *mask, const void= *va, } =20 static inline void flush_page_to_ram(unsigned long mfn, bool sync_icache) = {} -static inline int invalidate_dcache_va_range(const void *p, - unsigned long size) -{ return -EOPNOTSUPP; } -static inline int clean_and_invalidate_dcache_va_range(const void *p, - unsigned long size) -{ - unsigned int order =3D get_order_from_bytes(size); - /* sub-page granularity support needs to be added if necessary */ - flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order)); - return 0; -} -static inline int clean_dcache_va_range(const void *p, unsigned long size) -{ - return clean_and_invalidate_dcache_va_range(p, size); -} =20 unsigned int guest_flush_tlb_flags(const struct domain *d); void guest_flush_tlb_mask(const struct domain *d, const cpumask_t *mask); diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c index d874ac5f1241..cc7c7d004065 100644 --- a/xen/common/grant_table.c +++ b/xen/common/grant_table.c @@ -940,6 +940,7 @@ static void reduce_status_for_pin(struct domain *rd, gnttab_clear_flags(rd, clear_flags, status); } =20 +#ifdef CONFIG_ARM static struct active_grant_entry *grant_map_exists(const struct domain *ld, struct grant_table *rgt, mfn_t mfn, @@ -975,6 +976,7 @@ static struct active_grant_entry *grant_map_exists(cons= t struct domain *ld, =20 return ERR_PTR(-EINVAL); } +#endif /* CONFIG_ARM */ =20 union maptrack_node { struct { @@ -3520,6 +3522,7 @@ gnttab_swap_grant_ref(XEN_GUEST_HANDLE_PARAM(gnttab_s= wap_grant_ref_t) uop, return 0; } =20 +#ifdef CONFIG_ARM static int _cache_flush(const gnttab_cache_flush_t *cflush, grant_ref_t *c= ur_ref) { struct domain *d, *owner; @@ -3631,6 +3634,7 @@ gnttab_cache_flush(XEN_GUEST_HANDLE_PARAM(gnttab_cach= e_flush_t) uop, =20 return 0; } +#endif /* CONFIG_ARM */ =20 long do_grant_table_op( unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop, unsigned int count) @@ -3773,6 +3777,7 @@ long do_grant_table_op( break; } =20 +#ifdef CONFIG_ARM case GNTTABOP_cache_flush: { XEN_GUEST_HANDLE_PARAM(gnttab_cache_flush_t) cflush =3D @@ -3794,6 +3799,7 @@ long do_grant_table_op( } break; } +#endif =20 default: rc =3D -ENOSYS; --=20 2.48.1 From nobody Fri Oct 31 16:00:46 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1746520507; cv=none; d=zohomail.com; s=zohoarc; b=SH3nH0rmgTJqS8CO9PmhfEoM2fn1rNcgMRbP7VdIEKouBpLhBAt1GeON21NR8+hZhkOmeCvj9CCreDJep181T+ouCMIzOWLsUoyprt5rTEyplR5hQ4yBl6CQMRH79byL0c9i3XblKx6Q5aMPVC6QMUELke/Jc70MYkXHD6lcIu0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1746520507; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=p5UfBLcVStfB7Hed8NA5DfE3BRvsSJIoJFqhzuS6DZw=; b=GoVkgg2fRMoDt4wY4W26dhE4yliKvs22yJtAxPmoV1b7Ud9G0lVOfoVv50vSwWx7GeXke3HDN7hq7mqtrruxIqCCtGDbdCkiZMxe6DrN/G2kerVdI2tgjzAUjL+A6AlZHeZzcLJVS6Ej64C/qK+rooX8nSH1Nbeq6/dK0qUUQAM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1746520507653303.31371716977856; Tue, 6 May 2025 01:35:07 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.976750.1363923 (Exim 4.92) (envelope-from ) id 1uCDlN-0004y3-0S; Tue, 06 May 2025 08:34:53 +0000 Received: by outflank-mailman (output) from mailman id 976750.1363923; Tue, 06 May 2025 08:34:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlM-0004xw-TK; Tue, 06 May 2025 08:34:52 +0000 Received: by outflank-mailman (input) for mailman id 976750; Tue, 06 May 2025 08:34:51 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlL-00041m-Ox for xen-devel@lists.xenproject.org; Tue, 06 May 2025 08:34:51 +0000 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [2a00:1450:4864:20::42d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f9972bec-2a54-11f0-9ffb-bf95429c2676; Tue, 06 May 2025 10:34:50 +0200 (CEST) Received: by mail-wr1-x42d.google.com with SMTP id ffacd0b85a97d-39ee5ac4321so6478568f8f.1 for ; Tue, 06 May 2025 01:34:50 -0700 (PDT) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a099ae32fesm12669233f8f.22.2025.05.06.01.34.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 May 2025 01:34:49 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f9972bec-2a54-11f0-9ffb-bf95429c2676 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1746520489; x=1747125289; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p5UfBLcVStfB7Hed8NA5DfE3BRvsSJIoJFqhzuS6DZw=; b=TikPXyvOnU2anYiYC+OXdBZxH2AQ3ySb/erSnJdwqkyXrzVk2DASMhpfLD9RI78l5j ju/DsiLPqtCd0Aoz7QwpmyedkUsCfiC5qFGxXesV+qXxCSMu1nf8Fxe/B/Rs1emiBV+q gIspOu3t59yY1j3igLFGwSdv6tkzWIL6YpPaE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746520489; x=1747125289; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p5UfBLcVStfB7Hed8NA5DfE3BRvsSJIoJFqhzuS6DZw=; b=xUSCApvz1mT0UygWLwa/kA1V1+ftHrN9M3vtKRAKBxxBfb3JECk97YF6T1oJfDUyyT PkYKjMNS9paNET/x5wTkgcyNucqT3b558VBuZHq0SB4b8SkTA+QyfZhzeklcdrAmQ6wG nsu8P9zM3CY02dG/nZsnQRkN2SciqUaplLL7voGN5VSxmFk3Qf0MCrIX0HgJQTrpyJpS RNTyJDXLDlmtMWtv1iEhaT0am1VjmoHSMZGTDs+SiDb2TDfUgRX/GOdDhYT/1PeOeXwF +jhfLj7KdpLpDlnDb1Rd6hkMYGIZ6W9fDOEivQB1XdSUJp0Zr60jjQkhkAE6jl2pg5GQ UcGw== X-Gm-Message-State: AOJu0YyVP2HFpPBFVIbqprNdg9XMNBBsZ1pX5P2MmghIznujQk8pFFCa 2srgh9FrvpPqWBDf4wjTKlbVqnPzhWqn33/EPfC1O5UbUZbdFCWoHgimT8T0R1z9B4dD2JohxFP K X-Gm-Gg: ASbGnctss8B6cNv1Xjzt9m5cnoD5mIc2fAmpV9yxKq7n8rZwBqP5HCbuF9df9yFf0Pi XQRLCPDbFn2V8aR8qIC8RKGsTX3uzixdwIFczCKcN+NT72Ay6RTeTccbcfPZe/vjeFLDS1u9SjQ WhFrqm9ev1wuYo2sauzaWqAqHfBazDACtxub6tJPrw3xwX6gMHejjlRUgFBhNWZe0rN5PVBmfZ+ tQFHPODxr0moWu8ae65XFDSZY6Rag/yd1LZf5HFgWtF3WC9Id3GAuIPz21d5k6DTAanUtMQWSF1 qJM+evdxP3mnTfsMfXg6j5TJeYqTCOmF0Vb0O7t2STP9sHmegB0kUL3h X-Google-Smtp-Source: AGHT+IG8SbqAp820tgfep77WqAq8u/7HTxpv/gmNciMOaCRPklO3tPbEepKkjQU+MEYnB+mrs5NwOg== X-Received: by 2002:a05:6000:2a3:b0:3a0:85ad:5ed9 with SMTP id ffacd0b85a97d-3a09fd7a183mr7019642f8f.4.1746520489409; Tue, 06 May 2025 01:34:49 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH 5/9] x86/mtrr: use memory_type_changed() in hvm_set_mem_pinned_cacheattr() Date: Tue, 6 May 2025 10:31:44 +0200 Message-ID: <20250506083148.34963-6-roger.pau@citrix.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250506083148.34963-1-roger.pau@citrix.com> References: <20250506083148.34963-1-roger.pau@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1746520510136019000 The current logic partially open-codes memory_type_changed(), but doesn't check whether the type change or the cache flush is actually needed. Instead switch to using memory_type_changed(), at possibly a higher expense cost of not exclusively issuing cache flushes when limiting cacheability. However using memory_type_changed() has the benefit of limiting recalculations and cache flushes to strictly only when it's meaningful due to the guest configuration, like having devices or IO regions assigned. Signed-off-by: Roger Pau Monn=C3=A9 Acked-by: Christian Lindig --- xen/arch/x86/hvm/mtrr.c | 22 +++------------------- 1 file changed, 3 insertions(+), 19 deletions(-) diff --git a/xen/arch/x86/hvm/mtrr.c b/xen/arch/x86/hvm/mtrr.c index 797f2ae7fd3a..b88e81eb44b1 100644 --- a/xen/arch/x86/hvm/mtrr.c +++ b/xen/arch/x86/hvm/mtrr.c @@ -605,22 +605,8 @@ int hvm_set_mem_pinned_cacheattr(struct domain *d, uin= t64_t gfn_start, =20 type =3D range->type; call_rcu(&range->rcu, free_pinned_cacheattr_entry); - p2m_memory_type_changed(d); - switch ( type ) - { - case X86_MT_UCM: - /* - * For EPT we can also avoid the flush in this case; - * see epte_get_entry_emt(). - */ - if ( hap_enabled(d) && cpu_has_vmx ) - case X86_MT_UC: - break; - /* fall through */ - default: - flush_all(FLUSH_CACHE); - break; - } + memory_type_changed(d); + return 0; } domain_unlock(d); @@ -682,9 +668,7 @@ int hvm_set_mem_pinned_cacheattr(struct domain *d, uint= 64_t gfn_start, =20 xfree(newr); =20 - p2m_memory_type_changed(d); - if ( type !=3D X86_MT_WB ) - flush_all(FLUSH_CACHE); + memory_type_changed(d); =20 return rc; } --=20 2.48.1 From nobody Fri Oct 31 16:00:46 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1746520521; cv=none; d=zohomail.com; s=zohoarc; b=FceAzI/LC6FHgQjbf1re8LmKj3JvZWQ8xR5QAgcIx0AmhoVtb45cGlZv0sdphEFzM3hbuht4ZjIhPAyKxXmY1sqS+ADPLjmdtaj5W/505NqVejDFiHA8chwSMCwy1TbMTG44VQp0UUykgxa8KKqOJNFwNpq+SYSsB8VtVpulNo8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1746520521; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fgghoeas14+y+/fNB0eokRVfIB1w/ZzOUGRan/5YZkA=; b=c7diXp+eJm/ryz+02pHykBiodOJqzvqiS5yCqCwH7x/CaRnq7g5FCuIwh5ogiWLWKiD4aHbD1+s6NQGzqbO4fFMXf31dHjZjljOE6M/ItxIZ1NUqftv33hwuGcm4qNObCoYPfTcq6mN+Z/xYi1yotzgYhTl5XDpeqPtZbBBBDew= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1746520521204213.1225651715663; Tue, 6 May 2025 01:35:21 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.976751.1363933 (Exim 4.92) (envelope-from ) id 1uCDlO-0005E5-8C; Tue, 06 May 2025 08:34:54 +0000 Received: by outflank-mailman (output) from mailman id 976751.1363933; Tue, 06 May 2025 08:34:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlO-0005DP-3i; Tue, 06 May 2025 08:34:54 +0000 Received: by outflank-mailman (input) for mailman id 976751; Tue, 06 May 2025 08:34:52 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlM-00048v-7Q for xen-devel@lists.xenproject.org; Tue, 06 May 2025 08:34:52 +0000 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [2a00:1450:4864:20::32f]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id fa5497cd-2a54-11f0-9eb4-5ba50f476ded; Tue, 06 May 2025 10:34:51 +0200 (CEST) Received: by mail-wm1-x32f.google.com with SMTP id 5b1f17b1804b1-43d2d952eb1so29529405e9.1 for ; Tue, 06 May 2025 01:34:51 -0700 (PDT) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-441b2b20b36sm207541225e9.25.2025.05.06.01.34.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 May 2025 01:34:50 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: fa5497cd-2a54-11f0-9eb4-5ba50f476ded DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1746520491; x=1747125291; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fgghoeas14+y+/fNB0eokRVfIB1w/ZzOUGRan/5YZkA=; b=PlC5a2Y5PlpVAQwWiv0r0U/TDy/bZtUJRfwTKGNhnvC1jRMnOoaWZcVGhBVPYmo8sl OuMAsNpnjJg7YHKu08OhT5UaSW1+kycz3gr7H0vPkK2kvHaGC0YLZbsmdJq5Su4RXDQI a8ChVLK63iTullXW5ZwVSv67pLBPyerbGgoKM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746520491; x=1747125291; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fgghoeas14+y+/fNB0eokRVfIB1w/ZzOUGRan/5YZkA=; b=iP6iRIF5xDMFp0exGxq8CyH/ov7Mm9yweGKtb5lNQzQL0m+tJPK6S9VKq+d0RJlgLo Yfj6PwcqHtt7gSqOGeFhCoOtjzLMWdfnC/fYQhpxR9ANHY/c/KnleJRT6TVNfphe6lw9 fJici6B4vvJABnZaMS1MFyGaGg4BlhJcT34l2rrxcGCDihA3M2/IelAdMjGMoNGILjXF 48l2rxBCcjs3k9WH01PfL76fjHGVUOc9s53AfoxFFLM9OII0ta8lJe500zZHK+hROk1v C8n0iIMq5zQuNUK8u14CCw1aC+x2V2P297FN5acK00pjETmiZkvpDl82BI/pwsC+lzfT tefg== X-Gm-Message-State: AOJu0Yzzb1AkR0U4HyfAFebJfmNWOvU6eSVg1f1+ucMNJc8DpSKRPCqE Fetk3PDU8cFlis9sKAsGYD7LLrLgacuZoL9Sk2c1ACAjFS9kyOXp4mljKQaVzcThlAofH3WbpYk L X-Gm-Gg: ASbGncsjUBEcCgHWNykAr46pkVYihymmuZbn0RTRQ183BDwQAW9BTxDHK1q2q3kuey8 5kJPOu87wJgF6zcaZNyC3LMq947nElLVhJaX8roULJ3YftgjnQgMLzyIIm9wVXwnW9RsiAZ3mOh zs4+9EmnlzhWFSVcO5ITEmeSHNMPs8u5y6jV3rBZllqkcw3vSVbzsMpccyvv7uYFNJcNX2OM5KS xzk7PZcafzdtvQvl8GgjSysMOIPMBbdFuKepbaqemTXqjtWDdjzjjdrZZwAgUxrILLIarNSFocX nEakoQLtNwTWpRiK/XhXcc5VUQMfUJVshCVEKFQ5/al+xg== X-Google-Smtp-Source: AGHT+IFy3ozUxsGi2jitSr9ubKsaPbVL8wW6yqXgzNmoNPsoSYOYTdbpTX/VTHQ8XuEd/keRVE0rUQ== X-Received: by 2002:a05:600c:5010:b0:43d:526:e0ce with SMTP id 5b1f17b1804b1-441d100b3c5mr16425255e9.21.1746520490691; Tue, 06 May 2025 01:34:50 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH 6/9] x86/p2m: limit cache flush in memory_type_changed() Date: Tue, 6 May 2025 10:31:45 +0200 Message-ID: <20250506083148.34963-7-roger.pau@citrix.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250506083148.34963-1-roger.pau@citrix.com> References: <20250506083148.34963-1-roger.pau@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1746520523100116600 Only do the cache flush when there's a p2m type change to propagate, otherwise there's no change in the p2m effective caching attributes. If the p2m memory_type_changed hook is not set p2m_memory_type_changed() is a no-op, no recalculation of caching attributes is needed, nor flushing of the previous cache contents. Signed-off-by: Roger Pau Monn=C3=A9 Acked-by: Christian Lindig Reviewed-by: Jan Beulich --- xen/arch/x86/hvm/mtrr.c | 3 +-- xen/arch/x86/include/asm/p2m.h | 2 +- xen/arch/x86/mm/p2m.c | 13 ++++++++++++- 3 files changed, 14 insertions(+), 4 deletions(-) diff --git a/xen/arch/x86/hvm/mtrr.c b/xen/arch/x86/hvm/mtrr.c index b88e81eb44b1..e7acfb6e8dc4 100644 --- a/xen/arch/x86/hvm/mtrr.c +++ b/xen/arch/x86/hvm/mtrr.c @@ -767,9 +767,8 @@ HVM_REGISTER_SAVE_RESTORE(MTRR, hvm_save_mtrr_msr, NULL= , hvm_load_mtrr_msr, 1, void memory_type_changed(struct domain *d) { if ( (is_iommu_enabled(d) || cache_flush_permitted(d)) && - d->vcpu && d->vcpu[0] ) + d->vcpu && d->vcpu[0] && p2m_memory_type_changed(d) ) { - p2m_memory_type_changed(d); flush_all(FLUSH_CACHE); } } diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h index b9ce7d8705ba..4358cc15a2a1 100644 --- a/xen/arch/x86/include/asm/p2m.h +++ b/xen/arch/x86/include/asm/p2m.h @@ -700,7 +700,7 @@ void p2m_pod_dump_data(struct domain *d); #ifdef CONFIG_HVM =20 /* Report a change affecting memory types. */ -void p2m_memory_type_changed(struct domain *d); +bool p2m_memory_type_changed(struct domain *d); =20 /* Called by p2m code when demand-populating a PoD page */ bool diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c index 3a39b5d1246b..b9a7c2dc5302 100644 --- a/xen/arch/x86/mm/p2m.c +++ b/xen/arch/x86/mm/p2m.c @@ -126,12 +126,21 @@ static void _memory_type_changed(struct p2m_domain *p= 2m) { if ( p2m->memory_type_changed ) p2m->memory_type_changed(p2m); + else + ASSERT_UNREACHABLE(); } =20 -void p2m_memory_type_changed(struct domain *d) +bool p2m_memory_type_changed(struct domain *d) { struct p2m_domain *hostp2m =3D p2m_get_hostp2m(d); =20 + /* + * The p2m memory_type_changed hook will be the same for the host p2m = or + * the altp2ms, do the check early and return if not set. + */ + if ( !hostp2m->memory_type_changed ) + return false; + p2m_lock(hostp2m); =20 _memory_type_changed(hostp2m); @@ -154,6 +163,8 @@ void p2m_memory_type_changed(struct domain *d) } =20 p2m_unlock(hostp2m); + + return true; } =20 int p2m_set_ioreq_server(struct domain *d, --=20 2.48.1 From nobody Fri Oct 31 16:00:46 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1746520511; cv=none; d=zohomail.com; s=zohoarc; b=Wy8Yrl61yIBY0ex40dQvvukztdo7lWA3QjosyMgGWr4ztjgsy156kN0yDLmdiS1xfXGuIt7wGnTjNEPhL+AUFlEb3VTNUICPA1sAphcQU0v00iMR/sU3Tb1pCWQDo4Ngpab2ZBdd3hjvaaB8YmkTLBAPRq+oCsSexFxiOJ3nsl8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1746520511; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=pUxyovW21TXrkfyeOo/OFKomE9nboVw2dKjf//O9Bi4=; b=hlHIOLoKGFwDBKiiyea72m6W5FXQPTD2gvNY+IEtPKNkGqiBWF9tY6iZIKe0A4PKTFfq0nBhEF84BABtbzQhYdJ6C1wpO0phTxDSj73LJbtyZHrFzI+c5wO4szBboiaJTRBBXm0LXZ7tbiIguqWEnCopcuXBQYlCz27fkVr3A54= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1746520511426139.94091890594314; Tue, 6 May 2025 01:35:11 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.976752.1363943 (Exim 4.92) (envelope-from ) id 1uCDlP-0005UQ-HJ; Tue, 06 May 2025 08:34:55 +0000 Received: by outflank-mailman (output) from mailman id 976752.1363943; Tue, 06 May 2025 08:34:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlP-0005Tr-DE; Tue, 06 May 2025 08:34:55 +0000 Received: by outflank-mailman (input) for mailman id 976752; Tue, 06 May 2025 08:34:54 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlO-00041m-5V for xen-devel@lists.xenproject.org; Tue, 06 May 2025 08:34:54 +0000 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [2a00:1450:4864:20::332]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id fafbc382-2a54-11f0-9ffb-bf95429c2676; Tue, 06 May 2025 10:34:52 +0200 (CEST) Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-43cebe06e9eso33352915e9.3 for ; Tue, 06 May 2025 01:34:52 -0700 (PDT) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a099b107c4sm12878526f8f.76.2025.05.06.01.34.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 May 2025 01:34:51 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: fafbc382-2a54-11f0-9ffb-bf95429c2676 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1746520492; x=1747125292; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pUxyovW21TXrkfyeOo/OFKomE9nboVw2dKjf//O9Bi4=; b=VKSpbpMHxyF6v5UdV+XV58q70CKt/G/mH0tA2TQMd47OYsX94WpAvNOaOUZHJxiDez pcxBOirR7Jygtc5/k1weWmHoHHO0jcXt7wXJXH5nm4aRiZ3WaIFaah2yUSErFGK4Quro DaMF5VrmeEbb2Qyka8IudGzSXlbg2FA3tD/6A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746520492; x=1747125292; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pUxyovW21TXrkfyeOo/OFKomE9nboVw2dKjf//O9Bi4=; b=QnQcZE4SdpwQsaZQRhiISf+1KUXCawS4JXqw06iTEypx1IQUiXfOCFJef8MacBdS4G hvxyJon1oANVXX3x5Qj8cFl5rsibvUPqsEYauVHZp8F0brMaIWca1UpQsEk83dc8OHYJ Qocw6pea2/B7VE3AT7z2q6CjC1dAkUrv002yap/sTYZHmV6WbDPqCGkWVGVf6wGYSvwT Yh+NseotpB3c7xlKSLrlWJkhaWC0eX1tME22MxYWp0mhVb7k52M5eUXOh8Hz5BlbLMFs Jsn8dw8Jjloz0qfOFubuhizla/C36HmdPAWddFPpzLQPUJtU1iP2Bwc2kUpL/rtJJM7C gvlw== X-Gm-Message-State: AOJu0Ywoawg7APmpnrbpmdlxc84L8g6eTivOVpWdfZZMr+VUVtiIS4VN WpVe9VeLXOSYicnvYyTVUm7jJPiBB2+zHj5dXIp+ym7NMyJlz81M/PTigAJ1Wm39XTtDw+6lwxv o X-Gm-Gg: ASbGnctSYsvhCp3zKXyorh8ny+G6iTINxlcg/jxaztbvU8jANJN8PaVLeGP3ODCmz8L Igh2gc5X0I9oonYH/chFGoY3QmE4dS8XjaTF+hFX783r2DRe6USpWA904iozL8UKmIE1mkstwF1 umzf103IpBLUIxxIYwOhA6+E+tKaWcgXE8A6d8kmvIfMT7iTHm/GCuiNGKZ9z0u/Z5k9sdq03ku ZciKXZfPfKuPEYAGIk2G66OnpHC1/nwsj4hoKyVlAPsKCn7Va4DfHKt4Jhac4sHsDxbGZxjX0W9 Min0RIya9w4ELWZ21HCi1L4GQUmGBV4LQB3d4N2jt6wN6gzFZw4smmWM X-Google-Smtp-Source: AGHT+IFr9NSIj+NlQPKwWheTGOLpyFdRFPuyLgygAmUJmDy/hDkqsc6dOXunC/GQzkZLsGAngTYnFw== X-Received: by 2002:a05:600c:4f01:b0:43b:ca39:6c7d with SMTP id 5b1f17b1804b1-441d04f46b2mr18694515e9.3.1746520491734; Tue, 06 May 2025 01:34:51 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH 7/9] xen/x86: rename cache_flush_permitted() to has_arch_io_resources() Date: Tue, 6 May 2025 10:31:46 +0200 Message-ID: <20250506083148.34963-8-roger.pau@citrix.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250506083148.34963-1-roger.pau@citrix.com> References: <20250506083148.34963-1-roger.pau@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1746520512014019000 To better describe the underlying implementation. Define cache_flush_permitted() as an alias of has_arch_io_resources(), so that current users of cache_flush_permitted() are not effectively modified. With the introduction of the new handler, change some of the call sites of cache_flush_permitted() to instead use has_arch_io_resources() as such callers are not after whether cache flush is enabled, but rather whether the domain has any IO resources assigned. Take the opportunity to adjust l1_disallow_mask() to use the newly introduced has_arch_io_resources() macro. No functional change intended. Signed-off-by: Roger Pau Monn=C3=A9 Acked-by: Christian Lindig --- xen/arch/x86/include/asm/iocap.h | 4 +++- xen/arch/x86/mm.c | 3 +-- xen/arch/x86/mm/p2m-pod.c | 4 ++-- xen/common/memory.c | 2 +- xen/include/asm-generic/iocap.h | 4 +++- 5 files changed, 10 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/include/asm/iocap.h b/xen/arch/x86/include/asm/io= cap.h index 53d87ae8a334..61d026dbf5f6 100644 --- a/xen/arch/x86/include/asm/iocap.h +++ b/xen/arch/x86/include/asm/iocap.h @@ -15,10 +15,12 @@ #define ioports_access_permitted(d, s, e) \ rangeset_contains_range((d)->arch.ioport_caps, s, e) =20 -#define cache_flush_permitted(d) \ +#define has_arch_io_resources(d) \ (!rangeset_is_empty((d)->iomem_caps) || \ !rangeset_is_empty((d)->arch.ioport_caps)) =20 +#define cache_flush_permitted has_arch_io_resources + static inline int ioports_permit_access(struct domain *d, unsigned long s, unsigned long e) { diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 38e214352201..59b60b1e62a7 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -172,8 +172,7 @@ static DEFINE_SPINLOCK(subpage_ro_lock); =20 #define l1_disallow_mask(d) \ (((d) !=3D dom_io) && \ - (rangeset_is_empty((d)->iomem_caps) && \ - rangeset_is_empty((d)->arch.ioport_caps) && \ + (!has_arch_io_resources(d) && \ !has_arch_pdevs(d) && \ is_pv_domain(d)) ? \ L1_DISALLOW_MASK : (L1_DISALLOW_MASK & ~PAGE_CACHE_ATTRS)) diff --git a/xen/arch/x86/mm/p2m-pod.c b/xen/arch/x86/mm/p2m-pod.c index df2a1cc0749b..05633fe2ac88 100644 --- a/xen/arch/x86/mm/p2m-pod.c +++ b/xen/arch/x86/mm/p2m-pod.c @@ -338,7 +338,7 @@ p2m_pod_set_mem_target(struct domain *d, unsigned long = target) =20 ASSERT( pod_target >=3D p2m->pod.count ); =20 - if ( has_arch_pdevs(d) || cache_flush_permitted(d) ) + if ( has_arch_pdevs(d) || has_arch_io_resources(d) ) ret =3D -ENOTEMPTY; else ret =3D p2m_pod_set_cache_target(p2m, pod_target, 1/*preemptible*/= ); @@ -1395,7 +1395,7 @@ guest_physmap_mark_populate_on_demand(struct domain *= d, unsigned long gfn, if ( !paging_mode_translate(d) ) return -EINVAL; =20 - if ( has_arch_pdevs(d) || cache_flush_permitted(d) ) + if ( has_arch_pdevs(d) || has_arch_io_resources(d) ) return -ENOTEMPTY; =20 do { diff --git a/xen/common/memory.c b/xen/common/memory.c index 8ca4e1a8425b..46620ed8253d 100644 --- a/xen/common/memory.c +++ b/xen/common/memory.c @@ -86,7 +86,7 @@ static unsigned int max_order(const struct domain *d) unsigned int order =3D domu_max_order; =20 #ifdef CONFIG_HAS_PASSTHROUGH - if ( cache_flush_permitted(d) && order < ptdom_max_order ) + if ( has_arch_io_resources(d) && order < ptdom_max_order ) order =3D ptdom_max_order; #endif =20 diff --git a/xen/include/asm-generic/iocap.h b/xen/include/asm-generic/ioca= p.h index dd7cb45488f7..664bbc8971fe 100644 --- a/xen/include/asm-generic/iocap.h +++ b/xen/include/asm-generic/iocap.h @@ -2,9 +2,11 @@ #ifndef __ASM_GENERIC_IOCAP_H__ #define __ASM_GENERIC_IOCAP_H__ =20 -#define cache_flush_permitted(d) \ +#define has_arch_io_resources(d) \ (!rangeset_is_empty((d)->iomem_caps)) =20 +#define cache_flush_permitted has_arch_io_resources + #endif /* __ASM_GENERIC_IOCAP_H__ */ =20 /* --=20 2.48.1 From nobody Fri Oct 31 16:00:46 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1746520526; cv=none; d=zohomail.com; s=zohoarc; b=Ja4BG7PBzwy6OVitwB8ouPPIxWRs2Ri/x5OTkfXFlteU6w8qW+WnXK5bDJ5XLWR/i7QFtZQAmgVLzi5dmCrEiG9tOMNIRbCX0fSdLuMs/c4z1aCJPzTg7dd96kDQCG+Smgz7u8guoN78EkRhK069st9v2bakA4Iq2a3vu3lIerY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1746520526; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=jVsCpnQ84evwLMOD0Ne9LGSyDVtOtlci+rB7YWLWgTI=; b=I1vQHj16tkItXKRy5usCkhusQRIjW1L1GwzJSUngX34PqEABHOuUjl2Pm8W2V1KdOlHOaIVTBq7Sq0P2tnc6kCyPLHplZ4X+X/KLR20nMttQJ4nuN25ikpZpnUqVATO4Ugi8suBgrLETiuIOOEZDkGnpag9f3NS/fuvXvZu5HFU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1746520526535449.9822549031753; Tue, 6 May 2025 01:35:26 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.976753.1363947 (Exim 4.92) (envelope-from ) id 1uCDlP-0005YH-TV; Tue, 06 May 2025 08:34:55 +0000 Received: by outflank-mailman (output) from mailman id 976753.1363947; Tue, 06 May 2025 08:34:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlP-0005WU-O6; Tue, 06 May 2025 08:34:55 +0000 Received: by outflank-mailman (input) for mailman id 976753; Tue, 06 May 2025 08:34:54 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlO-00048v-Nn for xen-devel@lists.xenproject.org; Tue, 06 May 2025 08:34:54 +0000 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [2a00:1450:4864:20::32a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id fbf559b1-2a54-11f0-9eb4-5ba50f476ded; Tue, 06 May 2025 10:34:54 +0200 (CEST) Received: by mail-wm1-x32a.google.com with SMTP id 5b1f17b1804b1-43cf06eabdaso43079385e9.2 for ; Tue, 06 May 2025 01:34:54 -0700 (PDT) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-441b2af306bsm208429945e9.21.2025.05.06.01.34.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 May 2025 01:34:52 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: fbf559b1-2a54-11f0-9eb4-5ba50f476ded DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1746520493; x=1747125293; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jVsCpnQ84evwLMOD0Ne9LGSyDVtOtlci+rB7YWLWgTI=; b=lgGi3/G/ubnc6ViHuYdQBzRzbBnJOePxnSXhUQfsr+F/GcHn5cCBXd9In5ejiC6SOw Obu35BuPi8+xiVNkGK0/Y8g8G0W3XK9ssMBFn6PRL0OatHuhvz1qZLb6bTfr7GB6DWXy 9eeJ0V6HsGgyjk/mAHQsay9UPdjMPH2qjXNdE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746520493; x=1747125293; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jVsCpnQ84evwLMOD0Ne9LGSyDVtOtlci+rB7YWLWgTI=; b=Ylq1yXOGYCHIkur9iKrH5X92oBf0JYDyrq74CFUE9p4yefxyNZzv54GHjoPI7hnaTi L5u3lDXQibbVABnS6z0QbwDjj/eTajVawFVY9Hecdvn0Z6/ZP/dNJcK1pIGI2vW+TXMT 12dRbskD9Fk6R9GByhs3npsPOeAcSsLXBG5qmkwT5c0EYV2+5iNaoF+lrpOsNDf9xa30 6bjOZuDtBhZjO8AIhhY2mCGdRK9UKb9j0/H8TmbUM59c2OQIlIL5D/DJIPJNBtJUYxS2 bU3Vo97EX0tubzQMPaJFWYKBTo29+9R3fO9I4sBaKFAHrm4PfY6cSpA8kDZqxPsJ9Bn4 Usrg== X-Gm-Message-State: AOJu0YxkBpD78jDmGiIcPi0TzplSF++wrykFbeXOywggQnh/VR4CukN2 nY0VRQELv1H6s2M4yjHk555YGMCC9MCrwYSGoB343hpGCugwAuwh4scxx/5EUKHRkeWH+LTM4z8 6 X-Gm-Gg: ASbGncsyLt8d2XtBflf3x7tFTaJEC6JOY5Ozlt3KMjpIKIxUhzR0ecGPED1cAmSoSco JysNVV5bzasgw0Rkjn0U6lfQIcbDKt5yfyncbX+uUboAXe9J5RGHHfm5jVLiJ8DOg8P3XpilAYV h7rEWhyIAabaEafsSNfmFswUtxoWTLm5cyJjCOvn8LeZZ+87doNLVXD+6Te8SuMAKn9ZnMFuv5V aQ7DA6wONMbFcG857tO73CUThOMKtqbBP3F6SvPdctcnduiwkljqAmKqR3d5g96Q+hOU6QQWywm 3sEHsB2x1FVqA8/EcRAPPnarCUoIdtYMIaxw3q/hV6vZgA== X-Google-Smtp-Source: AGHT+IF3dBRn3/Jgs4mw0xEm6RRkSp1VNCEoLHtuUvy28npJedhfACuCeVrkkTxKolBgMX9AMQ39qw== X-Received: by 2002:a05:600c:4690:b0:43d:762:e0c4 with SMTP id 5b1f17b1804b1-441d054c744mr18730875e9.27.1746520492790; Tue, 06 May 2025 01:34:52 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Anthony PERARD , Andrew Cooper , Michal Orzel , Jan Beulich , Julien Grall , Stefano Stabellini , Juergen Gross , Christian Lindig , David Scott , Bertrand Marquis , Volodymyr Babchuk Subject: [PATCH 8/9] xen: introduce flag when a domain requires cache control Date: Tue, 6 May 2025 10:31:47 +0200 Message-ID: <20250506083148.34963-9-roger.pau@citrix.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250506083148.34963-1-roger.pau@citrix.com> References: <20250506083148.34963-1-roger.pau@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1746520528179019000 Such flag is added to the domain create hypercall, and a matching option is added to xl and libxl to set the flag: `cache_control`. When the flag is set, the domain is allowed the usage of cache control operations. If the flag is not explicitly set, libxl will set it if the domain has any `iomem` or `ioports` ranges assigned. Modify cache_flush_permitted() helper so that it's return value is no longer based on the IO resources assigned to the domain, but rather whether the domain has been explicitly allowed control over the cache, or if it has IOMMU support and there's a single IOMMU in the system that doesn't allow forcing snooping behind the guest back. As a result of this, the return of cache_flush_permitted() is constant for the lifetime of a domain, based on the domain creation flags and the capabilities of the IOMMU if enabled for the domain. Since cache control is now known at domain creation, and doesn't change for the lifetime of a domain, the cache related checks can be simplified, specially in iomem_{permit,deny}_access() and ioports_{permit,deny}_access(), as the EPT EMT values won't depend on whether the domain has IO resources assigned, but rather chosen at creation time. As cache_flush_permitted() now also takes into account if a domain has an IOMMU assigned, and whether the IOMMU supports forcing snooping, some of the checks can be simplified to drop the iommu_snoop or is_iommu_enabled() checks previously used in conjunction with cache_flush_permitted(). Signed-off-by: Roger Pau Monn=C3=A9 Acked-by: Christian Lindig --- docs/man/xl.cfg.5.pod.in | 10 ++++++++++ tools/include/libxl.h | 7 +++++++ tools/libs/light/libxl_create.c | 6 ++++++ tools/libs/light/libxl_types.idl | 3 +++ tools/ocaml/libs/xc/xenctrl.ml | 1 + tools/ocaml/libs/xc/xenctrl.mli | 1 + tools/xl/xl_parse.c | 2 ++ xen/arch/arm/dom0less-build.c | 12 ++++++++++-- xen/arch/arm/domain.c | 3 ++- xen/arch/arm/domain_build.c | 6 ++++++ xen/arch/x86/hvm/mtrr.c | 2 +- xen/arch/x86/hvm/vmx/vmcs.c | 3 +-- xen/arch/x86/hvm/vmx/vmx.c | 2 +- xen/arch/x86/include/asm/iocap.h | 19 ++----------------- xen/arch/x86/mm/p2m-ept.c | 7 +------ xen/arch/x86/setup.c | 7 +++++++ xen/common/domain.c | 3 ++- xen/include/asm-generic/iocap.h | 2 -- xen/include/public/domctl.h | 4 +++- xen/include/xen/iocap.h | 25 ++----------------------- xen/include/xen/sched.h | 6 ++++++ 21 files changed, 74 insertions(+), 57 deletions(-) diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in index 7339c44efd54..d61685de4db3 100644 --- a/docs/man/xl.cfg.5.pod.in +++ b/docs/man/xl.cfg.5.pod.in @@ -1705,6 +1705,16 @@ This feature is a B. =20 =3Dback =20 +=3Ditem B + +If this option is B, Xen allows the guest to perform cache control +operations. + +If this option is missing the default will be set based on whether there a= re +any iomem or ioports (if applicable) assigned to the guest. Note th= at +enabling passthrough support can also allow guest cache control operati= ons +depending on the IOMMU features. + =3Dback =20 =3Dhead2 Paravirtualised (PV) Guest Specific Options diff --git a/tools/include/libxl.h b/tools/include/libxl.h index b7ad7735ca4c..9a55e98f28bb 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -647,6 +647,13 @@ */ #define LIBXL_HAVE_DT_OVERLAY_DOMAIN 1 =20 +/* + * LIBXL_HAVE_CACHE_CONTROL indicates the presence of cache_control boolean + * field in libxl_domain_build_info. The field signals whether a domain is + * allowed access to cache control operations. + */ +#define LIBXL_HAVE_CACHE_CONTROL 1 + /* * libxl memory management * diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_creat= e.c index e03599ea99d1..469fa306e2f0 100644 --- a/tools/libs/light/libxl_create.c +++ b/tools/libs/light/libxl_create.c @@ -493,6 +493,9 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc, b_info->bootloader_user =3D libxl__strdup(gc, getenv("LIBXL_BOOTLOADER_USER")); =20 + libxl_defbool_setdefault(&b_info->cache_control, + b_info->num_iomem || b_info->num_ioports); + return 0; } =20 @@ -667,6 +670,9 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_conf= ig *d_config, if (libxl_defbool_val(b_info->vpmu)) create.flags |=3D XEN_DOMCTL_CDF_vpmu; =20 + if (libxl_defbool_val(b_info->cache_control)) + create.flags |=3D XEN_DOMCTL_CDF_cache_control; + assert(info->passthrough !=3D LIBXL_PASSTHROUGH_DEFAULT); LOG(DETAIL, "passthrough: %s", libxl_passthrough_to_string(info->passthrough)); diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_type= s.idl index 9bb296993199..59303e8df74a 100644 --- a/tools/libs/light/libxl_types.idl +++ b/tools/libs/light/libxl_types.idl @@ -737,6 +737,9 @@ libxl_domain_build_info =3D Struct("domain_build_info",[ =20 ("vpmu", libxl_defbool), =20 + # Allow guest access to cache control instructions. + ("cache_control", libxl_defbool), + ], dir=3DDIR_IN, copy_deprecated_fn=3D"libxl__domain_build_info_copy_deprecated", ) diff --git a/tools/ocaml/libs/xc/xenctrl.ml b/tools/ocaml/libs/xc/xenctrl.ml index 2690f9a92316..15ea2cbb0490 100644 --- a/tools/ocaml/libs/xc/xenctrl.ml +++ b/tools/ocaml/libs/xc/xenctrl.ml @@ -70,6 +70,7 @@ type domain_create_flag =3D | CDF_IOMMU | CDF_NESTED_VIRT | CDF_VPMU + | CDF_cache_control =20 type domain_create_iommu_opts =3D | IOMMU_NO_SHAREPT diff --git a/tools/ocaml/libs/xc/xenctrl.mli b/tools/ocaml/libs/xc/xenctrl.= mli index febbe1f6ae3f..c9ce99a62c16 100644 --- a/tools/ocaml/libs/xc/xenctrl.mli +++ b/tools/ocaml/libs/xc/xenctrl.mli @@ -63,6 +63,7 @@ type domain_create_flag =3D | CDF_IOMMU | CDF_NESTED_VIRT | CDF_VPMU + | CDF_cache_control =20 type domain_create_iommu_opts =3D | IOMMU_NO_SHAREPT diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c index 089a88935aff..b5f3133581cb 100644 --- a/tools/xl/xl_parse.c +++ b/tools/xl/xl_parse.c @@ -2993,6 +2993,8 @@ skip_usbdev: =20 xlu_cfg_get_defbool(config, "vpmu", &b_info->vpmu, 0); =20 + xlu_cfg_get_defbool(config, "cache_control", &b_info->cache_control, 0= ); + xlu_cfg_destroy(config); } =20 diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c index a356fc94fc4d..e0f0b828d0b6 100644 --- a/xen/arch/arm/dom0less-build.c +++ b/xen/arch/arm/dom0less-build.c @@ -1106,6 +1106,8 @@ void __init create_domUs(void) =20 if ( !strcmp(dom0less_iommu, "enabled") ) iommu =3D true; + else + d_cfg.flags |=3D XEN_DOMCTL_CDF_cache_control; } =20 if ( (flags & CDF_hardware) && !(flags & CDF_directmap) && @@ -1120,8 +1122,14 @@ void __init create_domUs(void) has_dtb =3D true; } =20 - if ( iommu_enabled && (iommu || has_dtb) ) - d_cfg.flags |=3D XEN_DOMCTL_CDF_iommu; + if ( iommu || has_dtb ) + /* + * Domain has devices assigned, either enable IOMMU support (if + * present), or explicitly allow cache control operations for = DMA + * coherency. + */ + d_cfg.flags |=3D iommu_enabled ? XEN_DOMCTL_CDF_iommu + : XEN_DOMCTL_CDF_cache_control; =20 if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) ) { diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c index 45aeb8bddcb0..1a8503f8d471 100644 --- a/xen/arch/arm/domain.c +++ b/xen/arch/arm/domain.c @@ -612,7 +612,8 @@ int arch_sanitise_domain_config(struct xen_domctl_creat= edomain *config) unsigned int max_vcpus; unsigned int flags_required =3D (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_h= ap); unsigned int flags_optional =3D (XEN_DOMCTL_CDF_iommu | XEN_DOMCTL_CDF= _vpmu | - XEN_DOMCTL_CDF_xs_domain ); + XEN_DOMCTL_CDF_xs_domain | + XEN_DOMCTL_CDF_cache_control); unsigned int sve_vl_bits =3D sve_decode_vl(config->arch.sve_vl); =20 if ( (config->flags & ~flags_optional) !=3D flags_required ) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 270a6b97e42c..0e7e6a6ac044 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -2381,6 +2381,12 @@ void __init create_dom0(void) =20 if ( iommu_enabled ) dom0_cfg.flags |=3D XEN_DOMCTL_CDF_iommu; + else + /* + * A hardware domain without IOMMU will need cache control to + * ensure DMA coherency. + */ + dom0_cfg.flags |=3D XEN_DOMCTL_CDF_cache_control; =20 if ( opt_dom0_sve ) { diff --git a/xen/arch/x86/hvm/mtrr.c b/xen/arch/x86/hvm/mtrr.c index e7acfb6e8dc4..887994d2b984 100644 --- a/xen/arch/x86/hvm/mtrr.c +++ b/xen/arch/x86/hvm/mtrr.c @@ -766,7 +766,7 @@ HVM_REGISTER_SAVE_RESTORE(MTRR, hvm_save_mtrr_msr, NULL= , hvm_load_mtrr_msr, 1, =20 void memory_type_changed(struct domain *d) { - if ( (is_iommu_enabled(d) || cache_flush_permitted(d)) && + if ( cache_flush_permitted(d) && d->vcpu && d->vcpu[0] && p2m_memory_type_changed(d) ) { flush_all(FLUSH_CACHE); diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c index a44475ae15bd..fba62b5d47d8 100644 --- a/xen/arch/x86/hvm/vmx/vmcs.c +++ b/xen/arch/x86/hvm/vmx/vmcs.c @@ -1921,8 +1921,7 @@ void cf_check vmx_do_resume(void) * 2: execute wbinvd on all dirty pCPUs when guest wbinvd exits. * If VT-d engine can force snooping, we don't need to do these. */ - if ( has_arch_pdevs(v->domain) && !iommu_snoop - && !cpu_has_wbinvd_exiting ) + if ( cache_flush_permitted(v->domain) && !cpu_has_wbinvd_exiting ) { int cpu =3D v->arch.hvm.vmx.active_cpu; if ( cpu !=3D -1 ) diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index 827db6bdd807..639882ceb216 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -3840,7 +3840,7 @@ static void vmx_do_extint(struct cpu_user_regs *regs) =20 static void cf_check vmx_wbinvd_intercept(void) { - if ( !cache_flush_permitted(current->domain) || iommu_snoop ) + if ( !cache_flush_permitted(current->domain) ) return; =20 if ( cpu_has_wbinvd_exiting ) diff --git a/xen/arch/x86/include/asm/iocap.h b/xen/arch/x86/include/asm/io= cap.h index 61d026dbf5f6..d3bb66f0e97a 100644 --- a/xen/arch/x86/include/asm/iocap.h +++ b/xen/arch/x86/include/asm/iocap.h @@ -19,31 +19,16 @@ (!rangeset_is_empty((d)->iomem_caps) || \ !rangeset_is_empty((d)->arch.ioport_caps)) =20 -#define cache_flush_permitted has_arch_io_resources - static inline int ioports_permit_access(struct domain *d, unsigned long s, unsigned long e) { - bool flush =3D cache_flush_permitted(d); - int ret =3D rangeset_add_range(d->arch.ioport_caps, s, e); - - if ( !ret && !is_iommu_enabled(d) && !flush ) - /* See comment in iomem_permit_access(). */ - memory_type_changed(d); - - return ret; + return rangeset_add_range(d->arch.ioport_caps, s, e); } =20 static inline int ioports_deny_access(struct domain *d, unsigned long s, unsigned long e) { - int ret =3D rangeset_remove_range(d->arch.ioport_caps, s, e); - - if ( !ret && !is_iommu_enabled(d) && !cache_flush_permitted(d) ) - /* See comment in iomem_deny_access(). */ - memory_type_changed(d); - - return ret; + return rangeset_remove_range(d->arch.ioport_caps, s, e); } =20 #endif /* __X86_IOCAP_H__ */ diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c index 0cf6818c13f0..d3ce422d76d7 100644 --- a/xen/arch/x86/mm/p2m-ept.c +++ b/xen/arch/x86/mm/p2m-ept.c @@ -505,12 +505,7 @@ int epte_get_entry_emt(struct domain *d, gfn_t gfn, mf= n_t mfn, return -1; } =20 - /* - * Conditional must be kept in sync with the code in - * {iomem,ioports}_{permit,deny}_access(). - */ - if ( type !=3D p2m_mmio_direct && !is_iommu_enabled(d) && - !cache_flush_permitted(d) ) + if ( type !=3D p2m_mmio_direct && !cache_flush_permitted(d) ) { *ipat =3D true; return X86_MT_WB; diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 25189541244d..ec1b9772bec0 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1029,6 +1029,13 @@ static struct domain *__init create_dom0(struct boot= _info *bi) =20 if ( iommu_enabled ) dom0_cfg.flags |=3D XEN_DOMCTL_CDF_iommu; + else if ( !pv_shim ) + /* + * pvshim doesn't support passthrough, so never allow it cache con= trol. + * Otherwise a hardware domain without IOMMU will need cache contr= ol to + * ensure DMA coherency. + */ + dom0_cfg.flags |=3D XEN_DOMCTL_CDF_cache_control; =20 /* Create initial domain. Not d0 for pvshim. */ bd->domid =3D get_initial_domain_id(); diff --git a/xen/common/domain.c b/xen/common/domain.c index abf1969e60e3..07b81a70c7a6 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -721,7 +721,8 @@ static int sanitise_domain_config(struct xen_domctl_cre= atedomain *config) ~(XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap | XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off | XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu | - XEN_DOMCTL_CDF_nested_virt | XEN_DOMCTL_CDF_vpmu) ) + XEN_DOMCTL_CDF_nested_virt | XEN_DOMCTL_CDF_vpmu | + XEN_DOMCTL_CDF_cache_control) ) { dprintk(XENLOG_INFO, "Unknown CDF flags %#x\n", config->flags); return -EINVAL; diff --git a/xen/include/asm-generic/iocap.h b/xen/include/asm-generic/ioca= p.h index 664bbc8971fe..011c1a475072 100644 --- a/xen/include/asm-generic/iocap.h +++ b/xen/include/asm-generic/iocap.h @@ -5,8 +5,6 @@ #define has_arch_io_resources(d) \ (!rangeset_is_empty((d)->iomem_caps)) =20 -#define cache_flush_permitted has_arch_io_resources - #endif /* __ASM_GENERIC_IOCAP_H__ */ =20 /* diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index 5b2063eed984..9134dde7e105 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -66,9 +66,11 @@ struct xen_domctl_createdomain { #define XEN_DOMCTL_CDF_nested_virt (1U << _XEN_DOMCTL_CDF_nested_virt) /* Should we expose the vPMU to the guest? */ #define XEN_DOMCTL_CDF_vpmu (1U << 7) +/* Guest has control over the cache? */ +#define XEN_DOMCTL_CDF_cache_control (1U << 8) =20 /* Max XEN_DOMCTL_CDF_* constant. Used for ABI checking. */ -#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_vpmu +#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_cache_control =20 uint32_t flags; =20 diff --git a/xen/include/xen/iocap.h b/xen/include/xen/iocap.h index ffbc48b60fd5..a753fea2ec50 100644 --- a/xen/include/xen/iocap.h +++ b/xen/include/xen/iocap.h @@ -15,34 +15,13 @@ static inline int iomem_permit_access(struct domain *d, unsigned long s, unsigned long e) { - bool flush =3D cache_flush_permitted(d); - int ret =3D rangeset_add_range(d->iomem_caps, s, e); - - if ( !ret && !is_iommu_enabled(d) && !flush ) - /* - * Only flush if the range(s) are empty before this addition and - * IOMMU is not enabled for the domain, otherwise it makes no - * difference for effective cache attribute calculation purposes. - */ - memory_type_changed(d); - - return ret; + return rangeset_add_range(d->iomem_caps, s, e); } =20 static inline int iomem_deny_access(struct domain *d, unsigned long s, unsigned long e) { - int ret =3D rangeset_remove_range(d->iomem_caps, s, e); - - if ( !ret && !is_iommu_enabled(d) && !cache_flush_permitted(d) ) - /* - * Only flush if the range(s) are empty after this removal and - * IOMMU is not enabled for the domain, otherwise it makes no - * difference for effective cache attribute calculation purposes. - */ - memory_type_changed(d); - - return ret; + return rangeset_remove_range(d->iomem_caps, s, e); } =20 #define iomem_access_permitted(d, s, e) \ diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 559d201e0c7e..908a3b7292ff 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -1222,6 +1222,12 @@ static always_inline bool is_iommu_enabled(const str= uct domain *d) return evaluate_nospec(d->options & XEN_DOMCTL_CDF_iommu); } =20 +static inline bool cache_flush_permitted(const struct domain *d) +{ + return (d->options & XEN_DOMCTL_CDF_cache_control) || + (is_iommu_enabled(d) && !iommu_snoop); +} + #ifdef CONFIG_MEM_PAGING # define mem_paging_enabled(d) vm_event_check_ring((d)->vm_event_paging) #else --=20 2.48.1 From nobody Fri Oct 31 16:00:46 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1746520512; cv=none; d=zohomail.com; s=zohoarc; b=lwGA7FJV3QpPMwQMJhMtl+5wrgrVPT1Mk4Uj4+XSZ57lgq2b0hn2OX78bSm3yHMKN7e9ou8KvX9aGL3xqWfq5QdirvqNFPsBZXnV4P/GQRI1LnOyUS/L7ld8STKp4ynglUOljbBn6s49i3nUprMPbiXLo7hvaskt0V7Re8VUTMk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1746520512; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Lijm60qofSbPzMh7l1KwxnIJENLBQ/io7X5j2PqNzlk=; b=WbkowmhKCPrXEO7RR9mb7Bi1f38McPLzk2RUBieVJTqKc4oxRNAoDlFuVpcoITt9YRhVhafZ1aJINKL9XS3pOsO0q4LkCD9iOOm1eiv5pZOn+veMVpZ6Fo76oAPSLowtxnW3m5gxSHoaOaTTfL/ELNlSzJu9XvZEpalBGy/DLus= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1746520512078614.7965543497818; Tue, 6 May 2025 01:35:12 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.976754.1363964 (Exim 4.92) (envelope-from ) id 1uCDlS-00065H-Ig; Tue, 06 May 2025 08:34:58 +0000 Received: by outflank-mailman (output) from mailman id 976754.1363964; Tue, 06 May 2025 08:34:58 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlS-00064s-Ap; Tue, 06 May 2025 08:34:58 +0000 Received: by outflank-mailman (input) for mailman id 976754; Tue, 06 May 2025 08:34:57 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uCDlQ-00041m-Uj for xen-devel@lists.xenproject.org; Tue, 06 May 2025 08:34:56 +0000 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [2a00:1450:4864:20::32c]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id fc93c8b4-2a54-11f0-9ffb-bf95429c2676; Tue, 06 May 2025 10:34:55 +0200 (CEST) Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-43cf06eabdaso43079535e9.2 for ; Tue, 06 May 2025 01:34:55 -0700 (PDT) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a0b22bd6a3sm331993f8f.27.2025.05.06.01.34.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 May 2025 01:34:53 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: fc93c8b4-2a54-11f0-9ffb-bf95429c2676 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1746520494; x=1747125294; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Lijm60qofSbPzMh7l1KwxnIJENLBQ/io7X5j2PqNzlk=; b=AtKFX+MQFOcCQWM9fQd7PQXiqiL2U6M/PUjGh06wEsr61dipb9Csc8QY+ZY/eZ44qQ 7T7sKaYBPUe/JuqDVucugtejy0b3Dn1Ndt0LljIEmf6z83cG+xGXeYt9ZiTsKU+sr1Gu omFpRdWVps1rBxo56yS/PbnjpHnykLM0Te1uE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746520494; x=1747125294; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Lijm60qofSbPzMh7l1KwxnIJENLBQ/io7X5j2PqNzlk=; b=oivJv41yuEulwrySmzfjbv9ToxdLoyMfcgH9r89YUsbJBtjAKNW7gpmiZyRHz7r8EJ 45Uu+gwuswN2GCMPfmVAt6JdDA65JKA+YxjPqRUgqpvwnYPJkYATFiRMvgGvJ781wiYe mEslfaadItYc49NUr+lsl8wREV0GotXGaSYXISywDJmsQGhpkMDutBUx6bYTVr+ogNeg OpqANPTTwltvXK4rBtygo9InCMYz0yxxIAaT8dR2TCHQqeiqHPeaEo5KYvXX5kegc3Vn 5mQvebesp/lZLl+3nOZpLDU57mjy6uEOOGrcnrQ11nSzL5dWLzzb3ueqHU432XuFeNXs 6uiA== X-Gm-Message-State: AOJu0YxpYerPeQINb6uG1G/O/q1vG/9J1NuwYSLUZalIgBtcyQDDxCST c0Lsw0zNrDla5IQOPyRmz4b/bQ5PhFAEDvNluKQt1hk8ejL9LeqjBIyfPbci8w8VxAimmd7wZW0 K X-Gm-Gg: ASbGncvj3IL0rIHWFM5GOXRdNKsTg8TvKwYRw5h3uJqN8qQmgRmAvZJuWeR55n6zEuk ZSXHvzNbt4CHRu0ken4BCQ6FvA65v08U0N93R4OX6smD33hOfftdx3tHSSEW2HMFJxIBKEtNFDk +WqNvjyNn3+6XhRSh1zwcU+HtjE12oHZq4Ag937asShRLhn9XcgUszzbKmL8+fKzXRvVSaDeqT0 HyTynnlPvCs+BSXGoB49PBHEfdD+y/s8GGeZ92tqhTob2ABR95lLTbF5sNEiFS0NxpHi7zeMUiI y6Z4pr/1i8MMlJTIR0jUMpLHiZSV1R99c7veeRjnQkwN7g== X-Google-Smtp-Source: AGHT+IGGh8zKa04McfL2H41nAuyZq30mLgV+7e5ifR/aNO8hMNZAOt0rGoSXkMlVgW4esZ4GKIMiaw== X-Received: by 2002:a05:6000:3113:b0:39c:e0e:bb46 with SMTP id ffacd0b85a97d-3a0ac0cb00fmr1421475f8f.4.1746520494283; Tue, 06 May 2025 01:34:54 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH 9/9] xen/x86: track dirty pCPU caches for a given vCPU Date: Tue, 6 May 2025 10:31:48 +0200 Message-ID: <20250506083148.34963-10-roger.pau@citrix.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250506083148.34963-1-roger.pau@citrix.com> References: <20250506083148.34963-1-roger.pau@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1746520513318116600 When a guest is allowed access to cache control operations such tracking prevents having to issue a system-wide cache flush, and rather just flush the pCPUs where the vCPU has been scheduled since the last flush. Note that domain-wide flushes accumulate the dirty caches from all the vCPUs, but clearing the vCPU masks will require pausing all vCPUs, which seems overkill. Instead leave the vCPU dirty masks as-is, worse case it will result in redundant flushes in further calls. Signed-off-by: Roger Pau Monn=C3=A9 Acked-by: Christian Lindig --- xen/arch/x86/domain.c | 43 +++++++++++++++++++++++++++++++ xen/arch/x86/hvm/hvm.c | 2 +- xen/arch/x86/hvm/mtrr.c | 2 +- xen/arch/x86/hvm/svm/svm.c | 6 +++-- xen/arch/x86/hvm/vmx/vmx.c | 6 +++-- xen/arch/x86/include/asm/domain.h | 9 +++++++ xen/arch/x86/mm.c | 25 +++++++----------- xen/arch/x86/pv/emul-priv-op.c | 8 ++---- 8 files changed, 73 insertions(+), 28 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index f197dad4c0cd..3d08b829d2db 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -579,6 +579,13 @@ int arch_vcpu_create(struct vcpu *v) =20 if ( (rc =3D init_vcpu_msr_policy(v)) ) goto fail; + + if ( cache_flush_permitted(d) && + !cond_zalloc_cpumask_var(&v->arch.dirty_cache) ) + { + rc =3D -ENOMEM; + goto fail; + } } else if ( (rc =3D xstate_alloc_save_area(v)) !=3D 0 ) return rc; @@ -614,6 +621,7 @@ int arch_vcpu_create(struct vcpu *v) vcpu_destroy_fpu(v); xfree(v->arch.msrs); v->arch.msrs =3D NULL; + FREE_CPUMASK_VAR(v->arch.dirty_cache); =20 return rc; } @@ -628,6 +636,8 @@ void arch_vcpu_destroy(struct vcpu *v) xfree(v->arch.msrs); v->arch.msrs =3D NULL; =20 + FREE_CPUMASK_VAR(v->arch.dirty_cache); + if ( is_hvm_vcpu(v) ) hvm_vcpu_destroy(v); else @@ -2018,6 +2028,9 @@ static void __context_switch(void) cpumask_set_cpu(cpu, nd->dirty_cpumask); write_atomic(&n->dirty_cpu, cpu); =20 + if ( cache_flush_permitted(nd) ) + __cpumask_set_cpu(cpu, n->arch.dirty_cache); + if ( !is_idle_domain(nd) ) { memcpy(stack_regs, &n->arch.user_regs, CTXT_SWITCH_STACK_BYTES); @@ -2606,6 +2619,36 @@ unsigned int domain_max_paddr_bits(const struct doma= in *d) return bits; } =20 +void vcpu_flush_cache(struct vcpu *curr) +{ + ASSERT(curr =3D=3D current); + ASSERT(cache_flush_permitted(curr->domain)); + + flush_mask(curr->arch.dirty_cache, FLUSH_CACHE); + cpumask_clear(curr->arch.dirty_cache); + __cpumask_set_cpu(smp_processor_id(), curr->arch.dirty_cache); +} + +void domain_flush_cache(const struct domain *d) +{ + const struct vcpu *v; + cpumask_t *mask =3D this_cpu(scratch_cpumask); + + ASSERT(cache_flush_permitted(d)); + + cpumask_clear(mask); + for_each_vcpu( d, v ) + cpumask_or(mask, mask, v->arch.dirty_cache); + + flush_mask(mask, FLUSH_CACHE); + /* + * Clearing the mask of vCPUs in the domain would be racy unless all v= CPUs + * are paused, so just leave them as-is, at the cost of possibly doing + * redundant flushes in later calls. It's still better than doing a + * host-wide cache flush. + */ +} + /* * Local variables: * mode: C diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 4cb2e13046d1..aed582a215a0 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -2277,7 +2277,7 @@ void hvm_shadow_handle_cd(struct vcpu *v, unsigned lo= ng value) domain_pause_nosync(v->domain); =20 /* Flush physical caches. */ - flush_all(FLUSH_CACHE); + domain_flush_cache(v->domain); hvm_set_uc_mode(v, 1); =20 domain_unpause(v->domain); diff --git a/xen/arch/x86/hvm/mtrr.c b/xen/arch/x86/hvm/mtrr.c index 887994d2b984..cfe0d44459c2 100644 --- a/xen/arch/x86/hvm/mtrr.c +++ b/xen/arch/x86/hvm/mtrr.c @@ -769,7 +769,7 @@ void memory_type_changed(struct domain *d) if ( cache_flush_permitted(d) && d->vcpu && d->vcpu[0] && p2m_memory_type_changed(d) ) { - flush_all(FLUSH_CACHE); + domain_flush_cache(d); } } =20 diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c index e33a38c1e446..5d1777ace335 100644 --- a/xen/arch/x86/hvm/svm/svm.c +++ b/xen/arch/x86/hvm/svm/svm.c @@ -2315,8 +2315,10 @@ static void svm_vmexit_mce_intercept( =20 static void cf_check svm_wbinvd_intercept(void) { - if ( cache_flush_permitted(current->domain) ) - flush_all(FLUSH_CACHE); + struct vcpu *curr =3D current; + + if ( cache_flush_permitted(curr->domain) ) + vcpu_flush_cache(curr); } =20 static void svm_vmexit_do_invalidate_cache(struct cpu_user_regs *regs, diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index 639882ceb216..9273607d576c 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -3840,11 +3840,13 @@ static void vmx_do_extint(struct cpu_user_regs *reg= s) =20 static void cf_check vmx_wbinvd_intercept(void) { - if ( !cache_flush_permitted(current->domain) ) + struct vcpu *curr =3D current; + + if ( !cache_flush_permitted(curr->domain) ) return; =20 if ( cpu_has_wbinvd_exiting ) - flush_all(FLUSH_CACHE); + vcpu_flush_cache(curr); else wbinvd(); } diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/d= omain.h index 8c0dea12a526..064b51889dc2 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -668,6 +668,12 @@ struct arch_vcpu =20 struct vcpu_msrs *msrs; =20 + /* + * When vCPU is allowed cache control track the pCPUs the vCPU has run= on + * since the last flush. + */ + cpumask_var_t dirty_cache; + struct { bool next_interrupt_enabled; } monitor; @@ -790,6 +796,9 @@ unsigned int domain_max_paddr_bits(const struct domain = *d); #define arch_init_idle_domain arch_init_idle_domain void arch_init_idle_domain(struct domain *d); =20 +void vcpu_flush_cache(struct vcpu *curr); +void domain_flush_cache(const struct domain *d); + #endif /* __ASM_DOMAIN_H__ */ =20 /* diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 59b60b1e62a7..11b59398a2c4 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -3804,26 +3804,19 @@ long do_mmuext_op( break; =20 case MMUEXT_FLUSH_CACHE: - /* - * Dirty pCPU caches where the current vCPU has been scheduled= are - * not tracked, and hence we need to resort to a global cache - * flush for correctness. - */ + if ( unlikely(currd !=3D pg_owner) ) + rc =3D -EPERM; + else if ( likely(cache_flush_permitted(currd)) ) + vcpu_flush_cache(curr); + else + rc =3D -EINVAL; + break; + case MMUEXT_FLUSH_CACHE_GLOBAL: if ( unlikely(currd !=3D pg_owner) ) rc =3D -EPERM; else if ( likely(cache_flush_permitted(currd)) ) - { - unsigned int cpu; - cpumask_t *mask =3D this_cpu(scratch_cpumask); - - cpumask_clear(mask); - for_each_online_cpu(cpu) - if ( !cpumask_intersects(mask, - per_cpu(cpu_sibling_mask, cpu= )) ) - __cpumask_set_cpu(cpu, mask); - flush_mask(mask, FLUSH_CACHE); - } + domain_flush_cache(currd); else rc =3D -EINVAL; break; diff --git a/xen/arch/x86/pv/emul-priv-op.c b/xen/arch/x86/pv/emul-priv-op.c index 089d4cb4d905..076ce8f00457 100644 --- a/xen/arch/x86/pv/emul-priv-op.c +++ b/xen/arch/x86/pv/emul-priv-op.c @@ -1199,12 +1199,8 @@ static int cf_check cache_op( * newer linux uses this in some start-of-day timing loops. */ if ( cache_flush_permitted(current->domain) ) - /* - * Handle wbnoinvd as wbinvd, at the expense of higher cost. Broa= dcast - * the flush to all pCPUs, Xen doesn't track where the vCPU has ran - * previously. - */ - flush_all(FLUSH_CACHE); + /* Handle wbnoinvd as wbinvd, at the expense of higher cost. */ + vcpu_flush_cache(current); =20 return X86EMUL_OKAY; } --=20 2.48.1