From nobody Wed Feb 5 12:54:04 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1736517747; cv=none; d=zohomail.com; s=zohoarc; b=D208XgFm6bdIIistv4SNi9BCQWBMNUzVY2nIDS12dBsLotIDqvCmXaWNpJaArspRBQLVZ9QIC8iK8zotQn6Dj3F86WE5ewWZiVFj5cBHYB9ZpEwu94+nmx6E7wGFJJH5xiK1NDcLwwmnab/iUGLzDRHFQLwCxO0R9148x2jS4io= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1736517747; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=QOPaP/mU6vt7fi9w+3JVCzGSZ59QyAsOxxCexGJ2z2A=; b=Wp5B8o5reU7iBoAX8hQpdbVsJ/gJH2Wz1HI4MvOIgOSgYh4+7FbvlcsBdNzRyOQxK3/t2qrjAUpsoSA7+3yu2K32qVrZZuwvRHZ7M+gxxu6MG2Gmn4liflX+EFa2p8D3fORiRt2etxWb2WUZPNsFvvSrWW+7K8bfTODDQNxfkhM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1736517747572468.8555625149869; Fri, 10 Jan 2025 06:02:27 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.869652.1281113 (Exim 4.92) (envelope-from ) id 1tWFaX-0000GW-Aw; Fri, 10 Jan 2025 14:02:13 +0000 Received: by outflank-mailman (output) from mailman id 869652.1281113; Fri, 10 Jan 2025 14:02:13 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tWFaX-0000GP-7e; Fri, 10 Jan 2025 14:02:13 +0000 Received: by outflank-mailman (input) for mailman id 869652; Fri, 10 Jan 2025 14:02:12 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tWFaW-00087j-H0 for xen-devel@lists.xenproject.org; Fri, 10 Jan 2025 14:02:12 +0000 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [2a00:1450:4864:20::530]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 7cf7048a-cf5b-11ef-a0df-8be0dac302b0; Fri, 10 Jan 2025 15:02:11 +0100 (CET) Received: by mail-ed1-x530.google.com with SMTP id 4fb4d7f45d1cf-5d3f28a4fccso2788498a12.2 for ; Fri, 10 Jan 2025 06:02:12 -0800 (PST) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ab2c9645408sm167312266b.166.2025.01.10.06.01.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jan 2025 06:01:56 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 7cf7048a-cf5b-11ef-a0df-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1736517731; x=1737122531; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QOPaP/mU6vt7fi9w+3JVCzGSZ59QyAsOxxCexGJ2z2A=; b=lgqmdGsm7aX+MHobDZxNnSbI6MFy2MGYFbrvlAudAwgFTdmpJ2oIKGBAXSOqreqtx5 ob4qyVDIgd7UieAmXi+j9IwJo/4FPSz+bvKgSvS8iz27DfUL9G+33Jt08eaYM5Wf7YZq HtJyNZ8HoqXL8m46GWa6CvagUAJZ3Qg70hM2o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736517731; x=1737122531; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QOPaP/mU6vt7fi9w+3JVCzGSZ59QyAsOxxCexGJ2z2A=; b=TIkgHSTPhuigIbnHB+xwKExVXZYl1mhIiy38ka6pekRts89S9wn6zZMVoOmoXhdyy0 t2JXPpAITR3GobdwbY39yDUTGRijdLcxRGy2lzItw39OHO0rgwOAH9NXetwxqmoJwBFl 7EiGsxF3JIZO3pc8C5nZ4jBVlCRRliEmHQt6VMipmmKW6o8CAfqxatm8P0mc8G/P+HvS XYPtGKb8s6PPAlh6E4ZIcjohQUuncDtUrfk6IKkVp+EOl+/uK3YcrJV4qfF6uE4cHgsR nKlhW8DBekgDQ3h0Ss4/WJEJzrmITguHMX9AFy3/pT68UTkaJT5jCERZRChycMUIZ1K5 OGkQ== X-Forwarded-Encrypted: i=1; AJvYcCXw/ygP3Lh1poDP3r6gXDc4iDZE6iFtPfMELLBmq3rwQyasYGU5z1Frem3D7W8UHYnBZQhQz7bBGqI=@lists.xenproject.org X-Gm-Message-State: AOJu0YxZvf+cnUxrH2vzMkHKimJxX13/Qh007T6wXTQFDXcPs8cy6XGC 8JVB6D/aDUoHZx87KpYmpq9El21Hv+vYFHLMlXI+/2Py17P2X7+Osq+6b1SxJuCKuHZNByyOXuj E X-Gm-Gg: ASbGncuk+EtZbOaoXkYebP8OTX1oV4vL637gJ9sZ1xAeMV2Ps5Q/E6i7U8tWSvm9b4H WWmMhD11gw77kSp5l2exUr95BjInSiYfMIMBqcelVA7K3zwysWy+pDKx+ZRkCvkVHsJUdhE9g6t OIiiIvnsrnLPqIA+jckxx18Ir3zSsu2DLYniugWZ+ZONKa1ryI3I4cF9k5lhI6S+SJuerHUjSvs vUoODb66OSam5U456/8Rc76X0kboXFRFmS7KLPKT/hr0+kAr5Gzm4a1Ar6H21StlXU= X-Google-Smtp-Source: AGHT+IHwiF6pZdggWtSRTr6pLMwM8np6twiqSyLvaaV8RNsdWeIBSWUdDT3itr1SN8CI6AcxoSrZBQ== X-Received: by 2002:a17:907:94c8:b0:aaf:7321:f05a with SMTP id a640c23a62f3a-ab2abc6e773mr980949866b.46.1736517717515; Fri, 10 Jan 2025 06:01:57 -0800 (PST) From: Roger Pau Monne To: linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko Subject: [PATCH 1/3] xen/pci: do not register devices outside of PCI segment scope Date: Fri, 10 Jan 2025 15:01:48 +0100 Message-ID: <20250110140152.27624-2-roger.pau@citrix.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250110140152.27624-1-roger.pau@citrix.com> References: <20250110140152.27624-1-roger.pau@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1736517748028116600 The PCI segment value is limited to 16 bits, however there are buses like V= MD that fake being part of the PCI topology by adding segment with a number outside the scope of the PCI firmware specification range (>=3D 0x10000). T= he MCFG ACPI Table "PCI Segment Group Number" field is defined as having a 16 = bit width. Attempting to register or manage those devices with Xen would result in err= ors at best, or overlaps with existing devices living on the truncated equivale= nt segment values. Skip notifying Xen about those devices. Signed-off-by: Roger Pau Monn=C3=A9 --- drivers/xen/pci.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/drivers/xen/pci.c b/drivers/xen/pci.c index 416f231809cb..08e82fd1263e 100644 --- a/drivers/xen/pci.c +++ b/drivers/xen/pci.c @@ -43,6 +43,13 @@ static int xen_add_device(struct device *dev) pci_mcfg_reserved =3D true; } #endif + + if (pci_domain_nr(pci_dev->bus) >> 16) { + dev_info(dev, + "not registering with Xen: invalid PCI segment\n"); + return 0; + } + if (pci_seg_supported) { DEFINE_RAW_FLEX(struct physdev_pci_device_add, add, optarr, 1); =20 @@ -149,6 +156,12 @@ static int xen_remove_device(struct device *dev) int r; struct pci_dev *pci_dev =3D to_pci_dev(dev); =20 + if (pci_domain_nr(pci_dev->bus) >> 16) { + dev_info(dev, + "not unregistering with Xen: invalid PCI segment\n"); + return 0; + } + if (pci_seg_supported) { struct physdev_pci_device device =3D { .seg =3D pci_domain_nr(pci_dev->bus), @@ -182,6 +195,12 @@ int xen_reset_device(const struct pci_dev *dev) .flags =3D PCI_DEVICE_RESET_FLR, }; =20 + if (pci_domain_nr(dev->bus) >> 16) { + dev_info(&dev->dev, + "unable to notify Xen of device reset: invalid PCI segment\n"); + return 0; + } + return HYPERVISOR_physdev_op(PHYSDEVOP_pci_device_reset, &device); } EXPORT_SYMBOL_GPL(xen_reset_device); --=20 2.46.0 From nobody Wed Feb 5 12:54:04 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1736517748; cv=none; d=zohomail.com; s=zohoarc; b=Ls9YjF8gKbmJ02zsusimlcjfjciSfBCR2H7dO6Et/MU8HWGSuRRCEbGieHRGijFVteTPbj/PPK5DbQxXGIAhnWlWLkxSACn2MehwNMTBovOYoQerRwW1Na7aYi/1j0wUcuyisOA23dTjImw8jB3aBhTXSi+1AxQNL+Uv0mK86io= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1736517748; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=c/I7HoDRHZV0XTx2nTCcpli5SpYI0/lHKSshoL/AVNY=; b=icx3UrrFkqihZIu9JqxBFeCgsFKWj2cOqblQb0KmvSoH4E8q7AqWBXj991ONnsYlCVIBQCC/+92zKpBm4QHBiJ4DCJPqhPXgBzO3G0f16xiurnLkWmI9G5BpN48FQFGR2bDfYzHI9ZUctYyXmvC15VzEx/oxqwt9sRe6GVHSzeM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 173651774807896.32357980700556; Fri, 10 Jan 2025 06:02:28 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.869651.1281103 (Exim 4.92) (envelope-from ) id 1tWFaL-0008Lw-3h; Fri, 10 Jan 2025 14:02:01 +0000 Received: by outflank-mailman (output) from mailman id 869651.1281103; Fri, 10 Jan 2025 14:02:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tWFaL-0008Lp-0m; Fri, 10 Jan 2025 14:02:01 +0000 Received: by outflank-mailman (input) for mailman id 869651; Fri, 10 Jan 2025 14:02:00 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tWFaK-00087j-HY for xen-devel@lists.xenproject.org; Fri, 10 Jan 2025 14:02:00 +0000 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [2a00:1450:4864:20::62a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 75d56e1f-cf5b-11ef-a0df-8be0dac302b0; Fri, 10 Jan 2025 15:01:59 +0100 (CET) Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-aaf900cc7fbso396605266b.3 for ; Fri, 10 Jan 2025 06:02:00 -0800 (PST) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ab2c95af680sm172606166b.141.2025.01.10.06.01.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jan 2025 06:01:58 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 75d56e1f-cf5b-11ef-a0df-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1736517719; x=1737122519; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=c/I7HoDRHZV0XTx2nTCcpli5SpYI0/lHKSshoL/AVNY=; b=QMinFTb7SbAa341SjgNGEyyBkdnnG5yi+3eqU9cWVeAoyldRz6kHfZtrMTHPPNHSVi C1XFLytJS6HM3QudySM16wCcgiHcXJHKSrup3U6j+ehsdooENjKcSEjQ8OrpLsVuhT0d hDIZCuFMOpqOI55mnNWKBBsh5oeoUDDTSvidc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736517719; x=1737122519; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c/I7HoDRHZV0XTx2nTCcpli5SpYI0/lHKSshoL/AVNY=; b=h7iejfp883N1VdvX9UARZ+1C62h/P0dboNePuQXqxRE1MEH9VRZ3yeylNu5M5jRlHF ygJgL6OcwlQLtoGq5y5MP9bF2mHgVXs1Uxg9++Fg7T9tZLmD2V6HXyJL86ROtUygG/qf JR8PwbTHjPVZ+PocyFH7UmV4FqaoirMlRVVKMNhtGapkkSTlVDPziPsQua3eTf8Fr0Tr fCEOgvPy7L9a05ToTKkxg6Rxe/PKa8mvUzlPw5abRUY5mBdUf2MscCMdCSvJR/mHTK1Z 5rNwQAZpZZYskln1rAUWAOCQZEUx/azVgd32vZq5sTsxrNqSkjOCRUoYafQRSXDh5sKP uHsw== X-Forwarded-Encrypted: i=1; AJvYcCUpZoKL7RkCA9YQwGjQOpDgataz5ls89gWPDQqLSqIvpI2fdQ8v1dbIY5IDDFsObqN+egVB02gvPU8=@lists.xenproject.org X-Gm-Message-State: AOJu0YwOmuG3ctGO8MiO8ZuK+aQx9HPFFs1F9fJ3Pgul6bEujSwrUxiP tMT2wNKT+2OJpkbhJ7o57HmcAJXTZzJvGfEU6sI9hwCq7ep3Tj45jdIMyVyFE14= X-Gm-Gg: ASbGncurGt2Wd22JaTu9/D+uIcq7yzu9F9n4zH8W58Is62WFgwkKM9kwBzYX0Upm6ri n6uSkJKxAXpt7UUYFf20ojvjiRMCQcg1xKyudc13Lmmcf2AtFDCadUTmXPpicP/R77ReUT1cf6k kvDWmTjTpgAi23Qa6nr5/etWp//5pLlYkVD2DlDOfIyIWoEoPs4Qb1KVyL4opaSfc9TxmXWonkI cSwBrS0F89GDyuqq345CyZ81zBC8M3aYBfCRz8Z6OPueOqN5QockFwJhC8ItG4l19A= X-Google-Smtp-Source: AGHT+IHQABlx0ljsChy/AaNgLOa5tG+fbxwnozWPPmywCwKrdUKAOZN8SDFE4NcVHC8wzrW5y/A8Dw== X-Received: by 2002:a17:907:c588:b0:ab2:c208:732d with SMTP id a640c23a62f3a-ab2c2087cbbmr743393166b.40.1736517719125; Fri, 10 Jan 2025 06:01:59 -0800 (PST) From: Roger Pau Monne To: linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org, linux-pci@vger.kernel.org Cc: Roger Pau Monne , Nirmal Patel , Jonathan Derrick , Lorenzo Pieralisi , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Manivannan Sadhasivam , Rob Herring , Bjorn Helgaas Subject: [PATCH 2/3] vmd: disable MSI remapping bypass under Xen Date: Fri, 10 Jan 2025 15:01:49 +0100 Message-ID: <20250110140152.27624-3-roger.pau@citrix.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250110140152.27624-1-roger.pau@citrix.com> References: <20250110140152.27624-1-roger.pau@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1736517752519019000 MSI remapping bypass (directly configuring MSI entries for devices on the V= MD bus) won't work under Xen, as Xen is not aware of devices in such bus, and hence cannot configure the entries using the pIRQ interface in the PV case,= and in the PVH case traps won't be setup for MSI entries for such devices. Until Xen is aware of devices in the VMD bus prevent the VMD_FEAT_CAN_BYPASS_MSI_REMAP capability from being used when running as any kind of Xen guest. Signed-off-by: Roger Pau Monn=C3=A9 --- drivers/pci/controller/vmd.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c index 264a180403a0..d9b7510ace29 100644 --- a/drivers/pci/controller/vmd.c +++ b/drivers/pci/controller/vmd.c @@ -965,6 +965,15 @@ static int vmd_probe(struct pci_dev *dev, const struct= pci_device_id *id) struct vmd_dev *vmd; int err; =20 + if (xen_domain()) + /* + * Xen doesn't have knowledge about devices in the VMD bus. + * Bypass of MSI remapping won't work in that case as direct + * write to the MSI entries won't result in functional + * interrupts. + */ + features &=3D ~VMD_FEAT_CAN_BYPASS_MSI_REMAP; + if (resource_size(&dev->resource[VMD_CFGBAR]) < (1 << 20)) return -ENOMEM; =20 --=20 2.46.0 From nobody Wed Feb 5 12:54:04 2025 Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99D872101A2 for ; Fri, 10 Jan 2025 14:02:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736517757; cv=none; b=U8S1Dr7Hro1NGDpyfDRLwfoG3xfraR+1urPdnzLjoYpxWsQ7KZmQtuk6k2R81GunM47SJkT863/3proPH5nSElotrbkTn6mds+Tdm+7TssG+oEibkCOIgRu6joiauWI5SNBzZsMyCJZYhLXaIqoZkfi3wv9tAhIjAOwDrTHVh+4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736517757; c=relaxed/simple; bh=NA2qf4BofQl5Z9fSyXzPqlQB2uDCZs5Xl7Dvz/os7cs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=NBSO9/Jt2GHvSnMxyfuvQtkcv17QkbSwflQxTE/xin4mJS3j4YsNkLtlf4VKC0gLQHOYHk9QoqNBLHbV9wAX0EogLFc8aBBy8xTzJOeTS8JIY1xtw7w11wYgn5WcUYSToL0dser4QiZq4bSDWTlo5UUV7N0BGG2OfSYlUps3Jbg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=citrix.com; spf=pass smtp.mailfrom=cloud.com; dkim=pass (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b=ioInHy/H; arc=none smtp.client-ip=209.85.218.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=citrix.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cloud.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="ioInHy/H" Received: by mail-ej1-f42.google.com with SMTP id a640c23a62f3a-ab2bb0822a4so421684966b.3 for ; Fri, 10 Jan 2025 06:02:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1736517752; x=1737122552; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bXUrrXc7otEMLuPDik4Ot8RLIKG6b+qL1a+y6J/W2hw=; b=ioInHy/Hne0Q8ZAZ8B+MwMM+jT+XTI6RZbPdRWtH6bBtbUVNY9llbrh9U+CZ/u375V vqStTDCgMGErS6ddzWVUlzbyjDxCSQaSMhrvA+//Dhbskb8yGZDtdR7bmCFlwpFPeQjJ PUszz6S/6SrUU8ECw5NoMdd8qImptR+bCM6ak= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736517752; x=1737122552; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bXUrrXc7otEMLuPDik4Ot8RLIKG6b+qL1a+y6J/W2hw=; b=dySQDcn3BpWnmfecmCHrq2Iq/Ls+e9dnVOXGCuEbpEHVKJ3oIDKJ29R0EYS9lzVxw9 pfYjZvUs0NTZkgnASeSgzZIyZFbsuzTtoq8g78CXLnjhHbKN4mZxRvl2+HRR2cKJLkh6 6x6zhpSujCxCFpzN148hh7n90HEiAiTFOHXxcMyn0suJ9Coxol9DBrdiqE/ZrUzzq+uO 2XcgAAMdV2Cqj9NG19B5lfCgLhSm2g/j/+Ece30FCMW5eWbZ/tm5i2vzWjjzPB1N2zRp tTq083/pLfEjQ0Xv2xQSGPgsEI3+azbZqjzqVdIRQb+n7nn1mwNlnP9UhtR8bcdUXYsC HmYA== X-Gm-Message-State: AOJu0YyHhsDpBJx1mlmtt+TYB1o9MsGB510LWsUhF80c6tT5JrLISN9z mCDhJORYnZ+dYrVNBJKRDOL/1hk5IrZvZAJ2cVPhT5hgW9pY7N+l+qtEBLzFCXtotDSWEaF94an e X-Gm-Gg: ASbGncsIfK9+70T7FGRccv+K4TH5QO4uYeRHBZVsGl7KApvlar7mdeHvDcHJpV8rJsj +JMJOeCUw2YvJzMVw0J/nx7p0gdlVyEzSJ4+pgn1qRm3waTs/GS4vuS1xiCVP5062sp8iD0IQOo 4ObEHECHLJkcr/B+kKW40ydJrLhCTW9AXZFtshskcJMViUAska/JJGAfBl8JxUy19WZQ+QeSkgP KyfuAlcWkFr+E7oGXO1IjjMqJI2EcA8llvCOhwOFzoVPijSK4UtqkoFgyHiN4u0myk= X-Google-Smtp-Source: AGHT+IGy+s65n1QI7kpn2XGnPHW44cwj7nn4OvI00oi9CML5cFM7+b0vWH7nmPweruWMP0Y17IUp2g== X-Received: by 2002:a17:906:7311:b0:a9a:bbcc:5092 with SMTP id a640c23a62f3a-ab2ab6bfb7amr910685366b.39.1736517721914; Fri, 10 Jan 2025 06:02:01 -0800 (PST) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ab2c95b09a6sm168510666b.145.2025.01.10.06.02.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jan 2025 06:02:01 -0800 (PST) From: Roger Pau Monne To: linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org, linux-pci@vger.kernel.org Cc: Roger Pau Monne , Juergen Gross , Bjorn Helgaas , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" Subject: [PATCH 3/3] pci/msi: remove pci_msi_ignore_mask Date: Fri, 10 Jan 2025 15:01:50 +0100 Message-ID: <20250110140152.27624-4-roger.pau@citrix.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250110140152.27624-1-roger.pau@citrix.com> References: <20250110140152.27624-1-roger.pau@citrix.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Setting pci_msi_ignore_mask inhibits the toggling of the mask bit for both = MSI and MSI-X entries globally, regardless of the IRQ chip they are using. Only Xen sets the pci_msi_ignore_mask when routing physical interrupts over event channels, to prevent PCI code from attempting to toggle the maskbit, as it's Xen that controls the bit. However, the pci_msi_ignore_mask being global will affect devices that use = MSI interrupts but are not routing those interrupts over event channels (not us= ing the Xen pIRQ chip). One example is devices behind a VMD PCI bridge. In th= at scenario the VMD bridge configures MSI(-X) using the normal IRQ chip (the p= IRQ one in the Xen case), and devices behind the bridge configure the MSI entri= es using indexes into the VMD bridge MSI table. The VMD bridge then demultipl= exes such interrupts and delivers to the destination device(s). Having pci_msi_ignore_mask set in that scenario prevents (un)masking of MSI entries for devices behind the VMD bridge. Move the signaling of no entry masking into the MSI domain flags, as that allows setting it on a per-domain basis. Set it for the Xen MSI domain that uses the pIRQ chip, while leaving it unset for the rest of the cases. Remove pci_msi_ignore_mask at once, since it was only used by Xen code, and with Xen dropping usage the variable is unneeded. This fixes using devices behind a VMD bridge on Xen PV hardware domains. Signed-off-by: Roger Pau Monn=C3=A9 --- arch/x86/pci/xen.c | 8 ++------ drivers/pci/msi/msi.c | 36 ++++++++++++++++++++---------------- include/linux/msi.h | 3 ++- kernel/irq/msi.c | 2 +- 4 files changed, 25 insertions(+), 24 deletions(-) diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c index 0f2fe524f60d..b8755cde2419 100644 --- a/arch/x86/pci/xen.c +++ b/arch/x86/pci/xen.c @@ -436,7 +436,8 @@ static struct msi_domain_ops xen_pci_msi_domain_ops =3D= { }; =20 static struct msi_domain_info xen_pci_msi_domain_info =3D { - .flags =3D MSI_FLAG_PCI_MSIX | MSI_FLAG_FREE_MSI_DESCS | MSI_FLAG_DEV_S= YSFS, + .flags =3D MSI_FLAG_PCI_MSIX | MSI_FLAG_FREE_MSI_DESCS | + MSI_FLAG_DEV_SYSFS | MSI_FLAG_NO_MASK, .ops =3D &xen_pci_msi_domain_ops, }; =20 @@ -484,11 +485,6 @@ static __init void xen_setup_pci_msi(void) * in allocating the native domain and never use it. */ x86_init.irqs.create_pci_msi_domain =3D xen_create_pci_msi_domain; - /* - * With XEN PIRQ/Eventchannels in use PCI/MSI[-X] masking is solely - * controlled by the hypervisor. - */ - pci_msi_ignore_mask =3D 1; } =20 #else /* CONFIG_PCI_MSI */ diff --git a/drivers/pci/msi/msi.c b/drivers/pci/msi/msi.c index 3a45879d85db..cb42298f6a97 100644 --- a/drivers/pci/msi/msi.c +++ b/drivers/pci/msi/msi.c @@ -15,7 +15,6 @@ #include "msi.h" =20 int pci_msi_enable =3D 1; -int pci_msi_ignore_mask; =20 /** * pci_msi_supported - check whether MSI may be enabled on a device @@ -285,6 +284,8 @@ static void pci_msi_set_enable(struct pci_dev *dev, int= enable) static int msi_setup_msi_desc(struct pci_dev *dev, int nvec, struct irq_affinity_desc *masks) { + const struct irq_domain *d =3D dev_get_msi_domain(&dev->dev); + const struct msi_domain_info *info =3D d->host_data; struct msi_desc desc; u16 control; =20 @@ -295,8 +296,7 @@ static int msi_setup_msi_desc(struct pci_dev *dev, int = nvec, /* Lies, damned lies, and MSIs */ if (dev->dev_flags & PCI_DEV_FLAGS_HAS_MSI_MASKING) control |=3D PCI_MSI_FLAGS_MASKBIT; - /* Respect XEN's mask disabling */ - if (pci_msi_ignore_mask) + if (info->flags & MSI_FLAG_NO_MASK) control &=3D ~PCI_MSI_FLAGS_MASKBIT; =20 desc.nvec_used =3D nvec; @@ -600,12 +600,15 @@ static void __iomem *msix_map_region(struct pci_dev *= dev, */ void msix_prepare_msi_desc(struct pci_dev *dev, struct msi_desc *desc) { + const struct irq_domain *d =3D dev_get_msi_domain(&dev->dev); + const struct msi_domain_info *info =3D d->host_data; + desc->nvec_used =3D 1; desc->pci.msi_attrib.is_msix =3D 1; desc->pci.msi_attrib.is_64 =3D 1; desc->pci.msi_attrib.default_irq =3D dev->irq; desc->pci.mask_base =3D dev->msix_base; - desc->pci.msi_attrib.can_mask =3D !pci_msi_ignore_mask && + desc->pci.msi_attrib.can_mask =3D !(info->flags & MSI_FLAG_NO_MASK) && !desc->pci.msi_attrib.is_virtual; =20 if (desc->pci.msi_attrib.can_mask) { @@ -655,9 +658,6 @@ static void msix_mask_all(void __iomem *base, int tsize) u32 ctrl =3D PCI_MSIX_ENTRY_CTRL_MASKBIT; int i; =20 - if (pci_msi_ignore_mask) - return; - for (i =3D 0; i < tsize; i++, base +=3D PCI_MSIX_ENTRY_SIZE) writel(ctrl, base + PCI_MSIX_ENTRY_VECTOR_CTRL); } @@ -710,6 +710,8 @@ static int msix_setup_interrupts(struct pci_dev *dev, s= truct msix_entry *entries static int msix_capability_init(struct pci_dev *dev, struct msix_entry *en= tries, int nvec, struct irq_affinity *affd) { + const struct irq_domain *d =3D dev_get_msi_domain(&dev->dev); + const struct msi_domain_info *info =3D d->host_data; int ret, tsize; u16 control; =20 @@ -740,15 +742,17 @@ static int msix_capability_init(struct pci_dev *dev, = struct msix_entry *entries, /* Disable INTX */ pci_intx_for_msi(dev, 0); =20 - /* - * Ensure that all table entries are masked to prevent - * stale entries from firing in a crash kernel. - * - * Done late to deal with a broken Marvell NVME device - * which takes the MSI-X mask bits into account even - * when MSI-X is disabled, which prevents MSI delivery. - */ - msix_mask_all(dev->msix_base, tsize); + if (!(info->flags & MSI_FLAG_NO_MASK)) { + /* + * Ensure that all table entries are masked to prevent + * stale entries from firing in a crash kernel. + * + * Done late to deal with a broken Marvell NVME device + * which takes the MSI-X mask bits into account even + * when MSI-X is disabled, which prevents MSI delivery. + */ + msix_mask_all(dev->msix_base, tsize); + } pci_msix_clear_and_set_ctrl(dev, PCI_MSIX_FLAGS_MASKALL, 0); =20 pcibios_free_irq(dev); diff --git a/include/linux/msi.h b/include/linux/msi.h index b10093c4d00e..59a421fc42bf 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -73,7 +73,6 @@ struct msi_msg { }; }; =20 -extern int pci_msi_ignore_mask; /* Helper functions */ struct msi_desc; struct pci_dev; @@ -556,6 +555,8 @@ enum { MSI_FLAG_PCI_MSIX_ALLOC_DYN =3D (1 << 20), /* PCI MSIs cannot be steered separately to CPU cores */ MSI_FLAG_NO_AFFINITY =3D (1 << 21), + /* Inhibit usage of entry masking */ + MSI_FLAG_NO_MASK =3D (1 << 22), }; =20 /** diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c index 396a067a8a56..7682c36cbccc 100644 --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -1143,7 +1143,7 @@ static bool msi_check_reservation_mode(struct irq_dom= ain *domain, if (!(info->flags & MSI_FLAG_MUST_REACTIVATE)) return false; =20 - if (IS_ENABLED(CONFIG_PCI_MSI) && pci_msi_ignore_mask) + if (info->flags & MSI_FLAG_NO_MASK) return false; =20 /* --=20 2.46.0