From nobody Sat Nov 23 19:31:16 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1731080636; cv=none; d=zohomail.com; s=zohoarc; b=h+AYNYqlX5ss+q9U5VOHYQ6QgAbAh5hT25k3sb8o4DTLhUmSt5EP037YtJesffU+bzw3LqNEeqJRJ8zeOfpZqDvpZGElmo+0lcm+nvMjDvHiJ3BObkIoJtiNu6rbH8Y+I87CIQRtf9uch+9hUp+B2CnWcVcf4tCdS3zMlcO1/ZE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1731080636; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=cnDnIMcekCsnd3CGKNWtp2v1ooPSGgZR3bcu+6t5wxM=; b=DgudKeBe4ZXFi18GvQDjf27TjlHSZkcEO5dL8OzZsjwM1NDpOoip/Au4RqTZptMKZq7+dZBafIgIVXsUBy49+TaQAaYXUHpa78cbUsyNVYZ//GjsyYH5yKQjZL36Mu/nSexUm4f2qAGFJ4A+UJAXyyRVfLlHYDOoXVzmvMoQk+U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1731080636047401.45233597373874; Fri, 8 Nov 2024 07:43:56 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t9R8K-0002By-O0; Fri, 08 Nov 2024 10:42:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t9R8H-0002BX-HB for qemu-devel@nongnu.org; Fri, 08 Nov 2024 10:42:45 -0500 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t9R8F-0005Fh-Q9 for qemu-devel@nongnu.org; Fri, 08 Nov 2024 10:42:45 -0500 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-71e5130832aso1863125b3a.0 for ; Fri, 08 Nov 2024 07:42:43 -0800 (PST) Received: from wheely.local0.net (124-171-217-17.tpgi.com.au. [124.171.217.17]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7f41f48d32bsm3110444a12.17.2024.11.08.07.42.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 07:42:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731080562; x=1731685362; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cnDnIMcekCsnd3CGKNWtp2v1ooPSGgZR3bcu+6t5wxM=; b=lsznyKcKnw6JBuo8bT9QxRIah3bqg+AsqD7SCNl1aI6E1zps/EG+mD2f9FD1d4G1SJ ojf1w7C1b58cYTq4wNP8r/LGPnEielgwE8T7AqQoS1WrsZJFFng9548vK2dBCOm8PqXl 1J86p03N9b0eXgq3YkAJBlYS9uMUS6Y8+/YlAdlyBplcJC9v5YIcFqxZcS819CHcce4r CbjJ2Y3E5CkHdSJHW1EAG7fejsF1uJwspsduJxixRLJlgn7Kqkv2oXjh5GKMrFspjlDY oc75dzOpSPKsL9U5jRvFcx0Cxw3ycYUN9vVUAFU9ZrV75yG+7JUSyi+gU+ERD8cdFJGu CZnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731080562; x=1731685362; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cnDnIMcekCsnd3CGKNWtp2v1ooPSGgZR3bcu+6t5wxM=; b=D0VGGLupB22w5dmzA+bArIJ+JlEPbwL9xjYcUy0fzsh+gy1PsL8y9deyD440yOEGu4 pe8YU0dmc0FsXMPnITgq5WDeo/khxttdfyYjHQUn+3OAEadrP1v3UKPkvzopRiH9Ifsv f/ZTX7kE407uWe2GRrLUtgzxgsOimTNST1GDm853EulTXGCcTafPjzer9Zq8E8tITEhw X91FxDBTciKfzNtBXp4RswL9YLYkwRWYLR+hNFNSItc3vRJOjMz65QAroQSG3S51q2RY klfS01CjLhEepKCtEJJXs4Oj5SDVgjapVU3JRaiMNhXpBD+BI9wFnWmN+Ksewe2k0pbo sIfA== X-Gm-Message-State: AOJu0YwGugqroixNOTI33a5mP32N4Gjs+m/Njx11NaYxi8VF3hX4A4Ut ZDCR3hDgGzmYq7mSFQPo738y7ymFDZ+KTT1TMuTqymglOYlY93HHfabvGg== X-Google-Smtp-Source: AGHT+IHGjCysYi2G94UlgtzFj/peajwuS9DR2KidZDYU9XgpmSCraqLwfeYr89ojXk7Y5LaSSPWRYg== X-Received: by 2002:a05:6a20:840d:b0:1db:eb56:be7c with SMTP id adf61e73a8af0-1dc22b1b74fmr4834228637.35.1731080561861; Fri, 08 Nov 2024 07:42:41 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S. Tsirkin" , Marcel Apfelbaum , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Gerd Hoffmann Subject: [RFC PATCH 1/5] qtest/pci: Enforce balanced iomap/unmap Date: Sat, 9 Nov 2024 01:42:24 +1000 Message-ID: <20241108154229.263097-2-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108154229.263097-1-npiggin@gmail.com> References: <20241108154229.263097-1-npiggin@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=npiggin@gmail.com; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1731080637355116600 Content-Type: text/plain; charset="utf-8" Add assertions to ensure a BAR is not mapped twice, and only previously mapped BARs are unmapped. This can help catch some bugs. Signed-off-by: Nicholas Piggin Reviewed-by: Fabiano Rosas --- tests/qtest/libqos/ahci.h | 1 + tests/qtest/libqos/pci.h | 2 ++ tests/qtest/libqos/virtio-pci.h | 1 + tests/qtest/ahci-test.c | 2 ++ tests/qtest/libqos/ahci.c | 6 ++++++ tests/qtest/libqos/pci.c | 32 +++++++++++++++++++++++++++++++- tests/qtest/libqos/virtio-pci.c | 6 +++++- 7 files changed, 48 insertions(+), 2 deletions(-) diff --git a/tests/qtest/libqos/ahci.h b/tests/qtest/libqos/ahci.h index a0487a1557..5d7e26aee2 100644 --- a/tests/qtest/libqos/ahci.h +++ b/tests/qtest/libqos/ahci.h @@ -575,6 +575,7 @@ QPCIDevice *get_ahci_device(QTestState *qts, uint32_t *= fingerprint); void free_ahci_device(QPCIDevice *dev); void ahci_pci_enable(AHCIQState *ahci); void start_ahci_device(AHCIQState *ahci); +void stop_ahci_device(AHCIQState *ahci); void ahci_hba_enable(AHCIQState *ahci); =20 /* Port Management */ diff --git a/tests/qtest/libqos/pci.h b/tests/qtest/libqos/pci.h index 8389614523..9dc82ea723 100644 --- a/tests/qtest/libqos/pci.h +++ b/tests/qtest/libqos/pci.h @@ -65,6 +65,8 @@ struct QPCIDevice { QPCIBus *bus; int devfn; + bool bars_mapped[6]; + QPCIBar bars[6]; bool msix_enabled; QPCIBar msix_table_bar, msix_pba_bar; uint64_t msix_table_off, msix_pba_off; diff --git a/tests/qtest/libqos/virtio-pci.h b/tests/qtest/libqos/virtio-pc= i.h index f5115cacba..efdf904b25 100644 --- a/tests/qtest/libqos/virtio-pci.h +++ b/tests/qtest/libqos/virtio-pci.h @@ -26,6 +26,7 @@ typedef struct QVirtioPCIDevice { uint64_t config_msix_addr; uint32_t config_msix_data; =20 + bool enabled; int bar_idx; =20 /* VIRTIO 1.0 */ diff --git a/tests/qtest/ahci-test.c b/tests/qtest/ahci-test.c index 5a1923f721..b3dae7a8ce 100644 --- a/tests/qtest/ahci-test.c +++ b/tests/qtest/ahci-test.c @@ -1483,6 +1483,8 @@ static void test_reset_pending_callback(void) /* Wait for throttled write to finish. */ sleep(1); =20 + stop_ahci_device(ahci); + /* Start again. */ ahci_clean_mem(ahci); ahci_pci_enable(ahci); diff --git a/tests/qtest/libqos/ahci.c b/tests/qtest/libqos/ahci.c index 34a75b7f43..cfc435b666 100644 --- a/tests/qtest/libqos/ahci.c +++ b/tests/qtest/libqos/ahci.c @@ -217,6 +217,12 @@ void start_ahci_device(AHCIQState *ahci) qpci_device_enable(ahci->dev); } =20 +void stop_ahci_device(AHCIQState *ahci) +{ + /* Map AHCI's ABAR (BAR5) */ + qpci_iounmap(ahci->dev, ahci->hba_bar); +} + /** * Test and initialize the AHCI's HBA memory areas. * Initialize and start any ports with devices attached. diff --git a/tests/qtest/libqos/pci.c b/tests/qtest/libqos/pci.c index b23d72346b..a42ca08261 100644 --- a/tests/qtest/libqos/pci.c +++ b/tests/qtest/libqos/pci.c @@ -93,12 +93,17 @@ QPCIDevice *qpci_device_find(QPCIBus *bus, int devfn) void qpci_device_init(QPCIDevice *dev, QPCIBus *bus, QPCIAddress *addr) { uint16_t vendor_id, device_id; + int i; =20 qpci_device_set(dev, bus, addr->devfn); vendor_id =3D qpci_config_readw(dev, PCI_VENDOR_ID); device_id =3D qpci_config_readw(dev, PCI_DEVICE_ID); g_assert(!addr->vendor_id || vendor_id =3D=3D addr->vendor_id); g_assert(!addr->device_id || device_id =3D=3D addr->device_id); + + for (i =3D 0; i < 6; i++) { + g_assert(!dev->bars_mapped[i]); + } } =20 static uint8_t qpci_find_resource_reserve_capability(QPCIDevice *dev) @@ -531,6 +536,8 @@ QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t= *sizeptr) uint64_t loc; =20 g_assert(barno >=3D 0 && barno <=3D 5); + g_assert(!dev->bars_mapped[barno]); + bar_reg =3D bar_reg_map[barno]; =20 qpci_config_writel(dev, bar_reg, 0xFFFFFFFF); @@ -574,12 +581,35 @@ QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64= _t *sizeptr) } =20 bar.addr =3D loc; + + dev->bars_mapped[barno] =3D true; + dev->bars[barno] =3D bar; + return bar; } =20 void qpci_iounmap(QPCIDevice *dev, QPCIBar bar) { - /* FIXME */ + static const int bar_reg_map[] =3D { + PCI_BASE_ADDRESS_0, PCI_BASE_ADDRESS_1, PCI_BASE_ADDRESS_2, + PCI_BASE_ADDRESS_3, PCI_BASE_ADDRESS_4, PCI_BASE_ADDRESS_5, + }; + int bar_reg; + int i; + + for (i =3D 0; i < 6; i++) { + if (!dev->bars_mapped[i]) { + continue; + } + if (dev->bars[i].addr =3D=3D bar.addr) { + dev->bars_mapped[i] =3D false; + bar_reg =3D bar_reg_map[i]; + qpci_config_writel(dev, bar_reg, 0xFFFFFFFF); + /* FIXME: the address space is leaked */ + return; + } + } + g_assert_not_reached(); } =20 QPCIBar qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr) diff --git a/tests/qtest/libqos/virtio-pci.c b/tests/qtest/libqos/virtio-pc= i.c index 485b8f6b7e..2b59fb181c 100644 --- a/tests/qtest/libqos/virtio-pci.c +++ b/tests/qtest/libqos/virtio-pci.c @@ -304,11 +304,15 @@ void qvirtio_pci_device_enable(QVirtioPCIDevice *d) { qpci_device_enable(d->pdev); d->bar =3D qpci_iomap(d->pdev, d->bar_idx, NULL); + d->enabled =3D true; } =20 void qvirtio_pci_device_disable(QVirtioPCIDevice *d) { - qpci_iounmap(d->pdev, d->bar); + if (d->enabled) { + qpci_iounmap(d->pdev, d->bar); + d->enabled =3D false; + } } =20 void qvirtqueue_pci_msix_setup(QVirtioPCIDevice *d, QVirtQueuePCI *vqpci, --=20 2.45.2 From nobody Sat Nov 23 19:31:16 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1731080676; cv=none; d=zohomail.com; s=zohoarc; b=GqLQYyucDUNu8zFJsH7Ci5Vyf99b+SooPABzkkbrqN8PE+6ilr6JGcWRTIDyi7/5/kPKbOXHg7vUc5y61vAdjUKSXwS/Yc16dfAciDMsINKdzDckkJzjH1iMz8nuxQVoVgMAxVOnltfAD9BfAkTZ3a/5RmmKrzVov6289ynD0Vc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1731080676; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=QxmRUlMoi7S4/7eFiKj9UJtfcRvcjfOPXhSE6dH7PHU=; b=MnX0xqIIEcGLAfgSBHSD9G9QDRDKYywOMpZUHOvwNFL08oI131xasPHZc5wrqlCb75s2UHrPMPIvm/kUkJsKVaW425/AjEuWBXiKV867JBAr2ozhKHCuEfsKGqT9Wb45eEWhaoWhZFekGgS34kYFNCBsmeWn+D3uFN/ytG2gjIQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17310806762901007.8398889627928; Fri, 8 Nov 2024 07:44:36 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t9R8M-0002CP-TH; Fri, 08 Nov 2024 10:42:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t9R8L-0002CC-3S for qemu-devel@nongnu.org; Fri, 08 Nov 2024 10:42:49 -0500 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t9R8J-0005H3-Ig for qemu-devel@nongnu.org; Fri, 08 Nov 2024 10:42:48 -0500 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-71e52582cf8so1902751b3a.2 for ; Fri, 08 Nov 2024 07:42:47 -0800 (PST) Received: from wheely.local0.net (124-171-217-17.tpgi.com.au. [124.171.217.17]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7f41f48d32bsm3110444a12.17.2024.11.08.07.42.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 07:42:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731080566; x=1731685366; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QxmRUlMoi7S4/7eFiKj9UJtfcRvcjfOPXhSE6dH7PHU=; b=YOQ4T6FIdorQ+s59FtNJovrYWkno843FoLEtYfkwj/skYq6xj18jICh9GPfJHJjwvc aFCwt7P9LoPktklGbrEWFBOEUQ3LcSP2vH4rd7HrMD2/6QTBodv9yEVazP40TOUcc8Le I9S1t8JgvexahSH5VkYeRNhX+PdCpzqPA9sw079iXwq5DojKpps/12R6aoQXgiMD6t+7 CE/JFLPF0uIy3PgKlGOZvnb1/5LY6CTuRSrb+nJunHvFqJqaJdxsYgHMJ9ItrqylMD9k q6tNXbXRGEGb6gnyxBc0Kf9wPNzx8hjRe6NwcYSlUqCm3aUWlmnK17v15qIXjRXEo7h/ K5UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731080566; x=1731685366; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QxmRUlMoi7S4/7eFiKj9UJtfcRvcjfOPXhSE6dH7PHU=; b=S84HOu7Fl+GT4uyjt5Lkn7Hqe/KcrCiCdZUKiqi+KpD8+NO4t3iNLxjdHogARuOsn8 4BjDGv57osxi4lU6M6M7UN2VEl6rTee+k8IyvZPSBFeB06FEGSiKloB7V2B8Rdp4wKfq F/F1GJ2JFEh1bkiwN5/ADSwqjI84LnR9VIWVgpXGyoIQTcYkNULbxesskOU3e9qoSe1M tpH+zBs4e8Xc8lKYiMMWBKgw5scnzncDBhBDfYdJ/UyKv8nJry7tnZF5H/pDdWmDcy8I b6BACm3w+mNLolZR9bXDiA+pNATNZAA8vdWkRjUWthSZPxmKIdRUtCGSnuE7f8Abu9i1 agaQ== X-Gm-Message-State: AOJu0Yzae8cCbISmIjGMLNttoGTOTOtq2E/E54alnsQuFhg3wodmGOSW TnJDOOb4UGr6aZiSNY0g9bN6uVA3fFZMnvRzJE+tMKWmOQK2gjA2xeHqCA== X-Google-Smtp-Source: AGHT+IGE1qzR0q0Jmc/he8szERQxzQ6kXiDuo7+FtMnEUwk9jLd33jyDjWncmE1uqoEhzwd9ZeXszw== X-Received: by 2002:a05:6a20:a10e:b0:1db:ebf4:2cb8 with SMTP id adf61e73a8af0-1dc22b38e98mr4198376637.38.1731080565838; Fri, 08 Nov 2024 07:42:45 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S. Tsirkin" , Marcel Apfelbaum , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Gerd Hoffmann Subject: [RFC PATCH 2/5] qtest/libqos/pci: Fix msix_enable sharing bar0 Date: Sat, 9 Nov 2024 01:42:25 +1000 Message-ID: <20241108154229.263097-3-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108154229.263097-1-npiggin@gmail.com> References: <20241108154229.263097-1-npiggin@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::436; envelope-from=npiggin@gmail.com; helo=mail-pf1-x436.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1731080680626116600 Content-Type: text/plain; charset="utf-8" Devices where the MSI-X addresses are shared with other MMIO on BAR0 can not use msi_enable because it unmaps and remaps BAR0, which interferes with device MMIO mappings. xhci-nec is one such device we would like to test msix with. Keep track of each the BAR iomaps for each device and add code in msix to use existing iomap if the msix bars are already mapped. Signed-off-by: Nicholas Piggin Reviewed-by: Fabiano Rosas --- tests/qtest/libqos/pci.h | 1 + tests/qtest/libqos/pci.c | 22 ++++++++++++++++------ 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/tests/qtest/libqos/pci.h b/tests/qtest/libqos/pci.h index 9dc82ea723..5a7b2454ad 100644 --- a/tests/qtest/libqos/pci.h +++ b/tests/qtest/libqos/pci.h @@ -68,6 +68,7 @@ struct QPCIDevice bool bars_mapped[6]; QPCIBar bars[6]; bool msix_enabled; + bool msix_table_bar_iomap, msix_pba_bar_iomap; QPCIBar msix_table_bar, msix_pba_bar; uint64_t msix_table_off, msix_pba_off; }; diff --git a/tests/qtest/libqos/pci.c b/tests/qtest/libqos/pci.c index a42ca08261..45199c7dc4 100644 --- a/tests/qtest/libqos/pci.c +++ b/tests/qtest/libqos/pci.c @@ -288,15 +288,21 @@ void qpci_msix_enable(QPCIDevice *dev) =20 table =3D qpci_config_readl(dev, addr + PCI_MSIX_TABLE); bir_table =3D table & PCI_MSIX_FLAGS_BIRMASK; - dev->msix_table_bar =3D qpci_iomap(dev, bir_table, NULL); + if (dev->bars_mapped[bir_table]) { + dev->msix_table_bar =3D dev->bars[bir_table]; + } else { + dev->msix_table_bar_iomap =3D true; + dev->msix_table_bar =3D qpci_iomap(dev, bir_table, NULL); + } dev->msix_table_off =3D table & ~PCI_MSIX_FLAGS_BIRMASK; =20 table =3D qpci_config_readl(dev, addr + PCI_MSIX_PBA); bir_pba =3D table & PCI_MSIX_FLAGS_BIRMASK; - if (bir_pba !=3D bir_table) { - dev->msix_pba_bar =3D qpci_iomap(dev, bir_pba, NULL); + if (dev->bars_mapped[bir_pba]) { + dev->msix_pba_bar =3D dev->bars[bir_pba]; } else { - dev->msix_pba_bar =3D dev->msix_table_bar; + dev->msix_pba_bar_iomap =3D true; + dev->msix_pba_bar =3D qpci_iomap(dev, bir_pba, NULL); } dev->msix_pba_off =3D table & ~PCI_MSIX_FLAGS_BIRMASK; =20 @@ -315,10 +321,14 @@ void qpci_msix_disable(QPCIDevice *dev) qpci_config_writew(dev, addr + PCI_MSIX_FLAGS, val & ~PCI_MSIX_FLAGS_ENAB= LE); =20 - if (dev->msix_pba_bar.addr !=3D dev->msix_table_bar.addr) { + if (dev->msix_pba_bar_iomap) { + dev->msix_pba_bar_iomap =3D false; qpci_iounmap(dev, dev->msix_pba_bar); } - qpci_iounmap(dev, dev->msix_table_bar); + if (dev->msix_table_bar_iomap) { + dev->msix_table_bar_iomap =3D false; + qpci_iounmap(dev, dev->msix_table_bar); + } =20 dev->msix_enabled =3D 0; dev->msix_table_off =3D 0; --=20 2.45.2 From nobody Sat Nov 23 19:31:16 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1731080620; cv=none; d=zohomail.com; s=zohoarc; b=cETiKexaTm6q2+ZFD6WOv6yTIXBvEhdOMasC7d+Q3bn2ae4tQmQWinc07QqoW3vMbTCbFxoeZKp/w0RbMS+EJWz9+ehqGgaeHo7lqeVQCaKRXrxj9aU9MKf1pfiMeueEjScUwdpFrV2hpo8KRUfYExuwXGYSML5xteaLMCxGJ28= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1731080620; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=WjZQkS7NlGSMkoZpKFfXMBWMv1VZGQCZ11MhEVoVebk=; b=V2aFBxBVccZ6sqahZSdKulf52C93Yzodyy9slCsdLm7sIFKRtTZXs7/O56PducGNGPGjk5dtExQXts4Pq9XZnjRUS7/nwoXG9bOyLOFfTs93DuLkimwJwLbVP4/CcMsFLfmHZA+hWV61t/KiYdgEYKX7YDIXAQeYscqYKn/qldY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1731080620618124.46704080375082; Fri, 8 Nov 2024 07:43:40 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t9R8Q-0002Fv-TF; Fri, 08 Nov 2024 10:42:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t9R8P-0002Ck-Ds for qemu-devel@nongnu.org; Fri, 08 Nov 2024 10:42:53 -0500 Received: from mail-oa1-x2a.google.com ([2001:4860:4864:20::2a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t9R8N-0005Hd-Od for qemu-devel@nongnu.org; Fri, 08 Nov 2024 10:42:53 -0500 Received: by mail-oa1-x2a.google.com with SMTP id 586e51a60fabf-288d70788d6so1117975fac.1 for ; Fri, 08 Nov 2024 07:42:51 -0800 (PST) Received: from wheely.local0.net (124-171-217-17.tpgi.com.au. [124.171.217.17]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7f41f48d32bsm3110444a12.17.2024.11.08.07.42.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 07:42:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731080570; x=1731685370; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WjZQkS7NlGSMkoZpKFfXMBWMv1VZGQCZ11MhEVoVebk=; b=eb4UyjGqktgHerF9D+cZPgXHodI13T+HXH0GtrZA8CYbL99eI0IMWwgYhjs+Pvy2Jh dJD8tE9C/zC1m5ZXG+mnwcKDYLnuGpdw4OkznK9461td8MMgtQrIZmKFt6XDgdxXOFmH S1dVszapsqA5xt1v6GpHZu3nqQmHuQt9dmVn21rpWTienNefw0QFk+X5rfH2B6X7i7BY wapqJ0ACzVPM64dL7qXhYe8huXi32tY/UIDk5L6WWauHaSW4+Xfj1X6aCicYjL48ja/I LPcUGcqgYyvOW1dJCAh1Tyn9Epd/O8sTYjEDmt5j5F4qPvmDslp7jrT1gHpfj/uE9U0o dF3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731080570; x=1731685370; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WjZQkS7NlGSMkoZpKFfXMBWMv1VZGQCZ11MhEVoVebk=; b=DD23dJKlRRHJu+mqfBVSV4QxjbdVR0dwmCpJiWwGoNkS4bbECyT6Vg2e9fcAnZa5wI Hb8O1Ox85mn3dSVwnxU1qjumfMMQocZKPSRCAfWvzkosiVCINPAOqTjjcU9kSMkn7J2H 7075tjzmzpi/E2RK2SqawoxBDBkzB3/oq3/sJgYpMZICWBnO1XPiqZqssCjUftDCCab8 Urziv+pG8hmy7uVKLnyv5AeigLnpwiP47lJ2Zm8SxczGbPnF7akQOeMJgq/9KRvZq+9G UOMLwJhz0wWfOCSZiCvl8Di/WgimLgVn6dGKGbBBN58pvdF5DINIT5TnV/PrbNRNE8qU neKg== X-Gm-Message-State: AOJu0Yz959kyIS7RzPINnp/BG2yoB0XwXxdPPgIDEY6f6vz0sEs/d4NT 7YAnN3ej8gH9QbKxe3PWkLqUQp8/IWi/OIE5acWF1Y+ku4tCWzfEzSU8JA== X-Google-Smtp-Source: AGHT+IEdWy59ywOfVbsBRmId44Ktn5Mjpu81dl5SV+t76TNTxXv4r4Rb3l019CY2HXaBlrI52Ejpng== X-Received: by 2002:a05:6870:e0d4:b0:260:f43e:7d89 with SMTP id 586e51a60fabf-2956000774dmr3598139fac.2.1731080569893; Fri, 08 Nov 2024 07:42:49 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S. Tsirkin" , Marcel Apfelbaum , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Gerd Hoffmann Subject: [RFC PATCH 3/5] pci/msix: Implement PBA writes Date: Sat, 9 Nov 2024 01:42:26 +1000 Message-ID: <20241108154229.263097-4-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108154229.263097-1-npiggin@gmail.com> References: <20241108154229.263097-1-npiggin@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:4860:4864:20::2a; envelope-from=npiggin@gmail.com; helo=mail-oa1-x2a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1731080621412116600 Content-Type: text/plain; charset="utf-8" Implement PBA write 1 to trigger and 0 to clear. This is used by qtests which mask the MSI irq and so the bits remain pending and expect to be cleared with stores. Some devices like e1000e seem to have MSIX PBA pending tied to some device state level, as such they call msix_clr_pending() directly, and clearing pending via a store to PBA causes this to go out of synch. So the qpci_msix_pending() function is changed to avoid clearing, and a new test-and-clear function is added for tests that would like to clear. Signed-off-by: Nicholas Piggin --- tests/qtest/libqos/pci.h | 1 + hw/pci/msix.c | 16 ++++++++++++++++ tests/qtest/libqos/pci.c | 21 ++++++++++++++++++--- 3 files changed, 35 insertions(+), 3 deletions(-) diff --git a/tests/qtest/libqos/pci.h b/tests/qtest/libqos/pci.h index 5a7b2454ad..de540f7803 100644 --- a/tests/qtest/libqos/pci.h +++ b/tests/qtest/libqos/pci.h @@ -94,6 +94,7 @@ uint8_t qpci_find_capability(QPCIDevice *dev, uint8_t id,= uint8_t start_addr); void qpci_msix_enable(QPCIDevice *dev); void qpci_msix_disable(QPCIDevice *dev); bool qpci_msix_pending(QPCIDevice *dev, uint16_t entry); +bool qpci_msix_test_clear_pending(QPCIDevice *dev, uint16_t entry); bool qpci_msix_masked(QPCIDevice *dev, uint16_t entry); uint16_t qpci_msix_table_size(QPCIDevice *dev); =20 diff --git a/hw/pci/msix.c b/hw/pci/msix.c index 487e49834e..b16b03b888 100644 --- a/hw/pci/msix.c +++ b/hw/pci/msix.c @@ -260,6 +260,22 @@ static uint64_t msix_pba_mmio_read(void *opaque, hwadd= r addr, static void msix_pba_mmio_write(void *opaque, hwaddr addr, uint64_t val, unsigned size) { + PCIDevice *dev =3D opaque; + unsigned vector_start =3D addr * 8; + unsigned vector_end =3D MIN(addr + size * 8, dev->msix_entries_nr); + unsigned i; + + for (i =3D vector_start; i < vector_end; i++) { + if ((val >> i) & 1) { + if (!msix_is_pending(dev, i)) { + msix_notify(dev, i); + } + } else { + if (msix_is_pending(dev, i)) { + msix_clr_pending(dev, i); + } + } + } } =20 static const MemoryRegionOps msix_pba_mmio_ops =3D { diff --git a/tests/qtest/libqos/pci.c b/tests/qtest/libqos/pci.c index 45199c7dc4..6726c401cc 100644 --- a/tests/qtest/libqos/pci.c +++ b/tests/qtest/libqos/pci.c @@ -343,11 +343,26 @@ bool qpci_msix_pending(QPCIDevice *dev, uint16_t entr= y) =20 g_assert(dev->msix_enabled); pba_entry =3D qpci_io_readl(dev, dev->msix_pba_bar, dev->msix_pba_off = + off); - qpci_io_writel(dev, dev->msix_pba_bar, dev->msix_pba_off + off, - pba_entry & ~(1 << bit_n)); - return (pba_entry & (1 << bit_n)) !=3D 0; + return (pba_entry & (1 << bit_n)); } =20 +bool qpci_msix_test_clear_pending(QPCIDevice *dev, uint16_t entry) +{ + uint32_t pba_entry; + uint8_t bit_n =3D entry % 32; + uint64_t off =3D (entry / 32) * PCI_MSIX_ENTRY_SIZE / 4; + + g_assert(dev->msix_enabled); + pba_entry =3D qpci_io_readl(dev, dev->msix_pba_bar, dev->msix_pba_off = + off); + if (pba_entry & (1 << bit_n)) { + qpci_io_writel(dev, dev->msix_pba_bar, dev->msix_pba_off + off, + pba_entry & ~(1 << bit_n)); + return true; + } + return false; +} + + bool qpci_msix_masked(QPCIDevice *dev, uint16_t entry) { uint8_t addr; --=20 2.45.2 From nobody Sat Nov 23 19:31:16 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1731080779; cv=none; d=zohomail.com; s=zohoarc; b=S4NwRHhVcGLWbSVP+waSK88TbyCRPAngi6VKdpQP5QMMA7TNGkgkQGf2nvCpUMJM/E5rIByESBq6wdkomXK4CrGkXApK1V2CmohtsmfBewnPupm9AI0plgyNi6Xc6uxPrMZGJOBVdz3BdXpTXejEV8CzFnr/BlbDdLQesZ607F8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1731080779; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=SceYCbCcY2IaNyvN5s9PaHH9tnhvxAvk7SxOm3Prwj0=; b=I0X4rUeT2l5tERym9Mz4clX7VlortumGG1dHnINI344asN0H3IsokFrtthGLrcvNcwWIft1VSIkWPxgLhm/v2fxEfhfQ5qVaajiTItal5tDgHBjCojXzRSkeHn9kyOq3xZp99ceqo2P3p0/o69wh1n/1BF4tE2eG60yF6IZ7pfA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1731080779769264.9281760420175; Fri, 8 Nov 2024 07:46:19 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t9R8V-0002JW-LP; Fri, 08 Nov 2024 10:42:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t9R8U-0002JM-VO for qemu-devel@nongnu.org; Fri, 08 Nov 2024 10:42:58 -0500 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t9R8S-0005I7-45 for qemu-devel@nongnu.org; Fri, 08 Nov 2024 10:42:58 -0500 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-720c2db824eso2447710b3a.0 for ; Fri, 08 Nov 2024 07:42:55 -0800 (PST) Received: from wheely.local0.net (124-171-217-17.tpgi.com.au. [124.171.217.17]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7f41f48d32bsm3110444a12.17.2024.11.08.07.42.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 07:42:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731080574; x=1731685374; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SceYCbCcY2IaNyvN5s9PaHH9tnhvxAvk7SxOm3Prwj0=; b=UakeVkK7djN7xOgtEJ65PCZ7XoL7OkgTR7g31qNDjMjb5UiiQytTkFlj0AhkRGvww0 K5Mqh624tS40UwRA+/q5ocaPF46BYeQTHG/hGk12n2/Lt7Z29EqVpU7QTu53+uuCawFd UBhiYiMz2wDKESY+r0nggj0hlCfg4+mmTDpFSw9aUf/vF59/Ky9lUDITcn6xjlqOMMJl XAhgJy1OcRP8cPYQod4CgAB0urH0SbUmUfEjy9tAXHe5SE4NlrFA75vp3VQcr7McHYZS 2m/yU3v5koHVIt4CpUMYf+bi1QuUnoOP3lAeXII2eAcB1zH92W3LJx5wAGj/LYtDt21x zyDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731080574; x=1731685374; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SceYCbCcY2IaNyvN5s9PaHH9tnhvxAvk7SxOm3Prwj0=; b=JsjXVPD0FevTU29o1f6ovCUqQKDwmkKcRWcdZVqJTkB4233Uv6ONxtYSN00T6URVS7 HRYU+v0bgvDIMutEgA6r3tBbLbMUVsykjy5rLzQvIcN7tjuLa1P8LAPob12Yd48TIUq7 yGbD4XlbE6hk4OZmcxX7eA/TgRwjxoC+49tx5W1+4NambXYezEAkYeJ79i3tALr/Ould t679VBMU1l/qWLcKSEuCtELgwIXrNP4m3iTa0ZmTXXPsY0DfQ0i/U8Itj7k1P/9gFJLW Yf08Wo76ys3RT+ifr218ZOrjb7YzPiIMgRF5Y5KsXzcHUYjU6hbP+yMVjf/o4Mq3Sq1a IkPQ== X-Gm-Message-State: AOJu0Yw2v12b3CpNZmvlh0FVVTw5sjDur2uheWVU23E2RjWHjUuPskHW +L9tQBmHJS7/akNqLFYul3nGFfIKOPQaf/qGSpmz+RunSLpGomNoADhFRw== X-Google-Smtp-Source: AGHT+IE+j/xh9XiiMFOjlZOfYaGAfK+yx4v5IGKimnDwPCC900vVgGnhRGyzUuq6Tsu++1TkLBkTbA== X-Received: by 2002:a05:6a00:198a:b0:71e:692e:7afb with SMTP id d2e1a72fcca58-7241325f64amr4126535b3a.5.1731080574044; Fri, 08 Nov 2024 07:42:54 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S. Tsirkin" , Marcel Apfelbaum , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Gerd Hoffmann Subject: [RFC PATCH 4/5] qtest/xhci: Add controller and device setup and ring tests Date: Sat, 9 Nov 2024 01:42:27 +1000 Message-ID: <20241108154229.263097-5-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108154229.263097-1-npiggin@gmail.com> References: <20241108154229.263097-1-npiggin@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::436; envelope-from=npiggin@gmail.com; helo=mail-pf1-x436.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1731080781388116600 Content-Type: text/plain; charset="utf-8" Add tests which init the host controller registers to the point where command and event rings, irqs are operational. Enumerate ports and set up an attached device context that enables device transfer ring to be set up and tested. This test does a bunch of things at once and is yet well librified, but it allows testing basic mechanisms and gives a starting point for further work. Signed-off-by: Nicholas Piggin --- tests/qtest/usb-hcd-xhci-test.h | 232 +++++++++++++++ tests/qtest/usb-hcd-xhci-test.c | 506 +++++++++++++++++++++++++++++++- 2 files changed, 732 insertions(+), 6 deletions(-) create mode 100644 tests/qtest/usb-hcd-xhci-test.h diff --git a/tests/qtest/usb-hcd-xhci-test.h b/tests/qtest/usb-hcd-xhci-tes= t.h new file mode 100644 index 0000000000..88d65b4f7e --- /dev/null +++ b/tests/qtest/usb-hcd-xhci-test.h @@ -0,0 +1,232 @@ +#ifndef TESTS_USB_HCD_XHCI_H +#define TESTS_USB_HCD_XHCI_H + +typedef enum TRBType { + TRB_RESERVED =3D 0, + TR_NORMAL, + TR_SETUP, + TR_DATA, + TR_STATUS, + TR_ISOCH, + TR_LINK, + TR_EVDATA, + TR_NOOP, + CR_ENABLE_SLOT, + CR_DISABLE_SLOT, + CR_ADDRESS_DEVICE, + CR_CONFIGURE_ENDPOINT, + CR_EVALUATE_CONTEXT, + CR_RESET_ENDPOINT, + CR_STOP_ENDPOINT, + CR_SET_TR_DEQUEUE, + CR_RESET_DEVICE, + CR_FORCE_EVENT, + CR_NEGOTIATE_BW, + CR_SET_LATENCY_TOLERANCE, + CR_GET_PORT_BANDWIDTH, + CR_FORCE_HEADER, + CR_NOOP, + ER_TRANSFER =3D 32, + ER_COMMAND_COMPLETE, + ER_PORT_STATUS_CHANGE, + ER_BANDWIDTH_REQUEST, + ER_DOORBELL, + ER_HOST_CONTROLLER, + ER_DEVICE_NOTIFICATION, + ER_MFINDEX_WRAP, + /* vendor specific bits */ + CR_VENDOR_NEC_FIRMWARE_REVISION =3D 49, + CR_VENDOR_NEC_CHALLENGE_RESPONSE =3D 50, +} TRBType; + +typedef enum TRBCCode { + CC_INVALID =3D 0, + CC_SUCCESS, + CC_DATA_BUFFER_ERROR, + CC_BABBLE_DETECTED, + CC_USB_TRANSACTION_ERROR, + CC_TRB_ERROR, + CC_STALL_ERROR, + CC_RESOURCE_ERROR, + CC_BANDWIDTH_ERROR, + CC_NO_SLOTS_ERROR, + CC_INVALID_STREAM_TYPE_ERROR, + CC_SLOT_NOT_ENABLED_ERROR, + CC_EP_NOT_ENABLED_ERROR, + CC_SHORT_PACKET, + CC_RING_UNDERRUN, + CC_RING_OVERRUN, + CC_VF_ER_FULL, + CC_PARAMETER_ERROR, + CC_BANDWIDTH_OVERRUN, + CC_CONTEXT_STATE_ERROR, + CC_NO_PING_RESPONSE_ERROR, + CC_EVENT_RING_FULL_ERROR, + CC_INCOMPATIBLE_DEVICE_ERROR, + CC_MISSED_SERVICE_ERROR, + CC_COMMAND_RING_STOPPED, + CC_COMMAND_ABORTED, + CC_STOPPED, + CC_STOPPED_LENGTH_INVALID, + CC_MAX_EXIT_LATENCY_TOO_LARGE_ERROR =3D 29, + CC_ISOCH_BUFFER_OVERRUN =3D 31, + CC_EVENT_LOST_ERROR, + CC_UNDEFINED_ERROR, + CC_INVALID_STREAM_ID_ERROR, + CC_SECONDARY_BANDWIDTH_ERROR, + CC_SPLIT_TRANSACTION_ERROR +} TRBCCode; + +/* bit definitions */ +#define USBCMD_RS (1<<0) +#define USBCMD_HCRST (1<<1) +#define USBCMD_INTE (1<<2) +#define USBCMD_HSEE (1<<3) +#define USBCMD_LHCRST (1<<7) +#define USBCMD_CSS (1<<8) +#define USBCMD_CRS (1<<9) +#define USBCMD_EWE (1<<10) +#define USBCMD_EU3S (1<<11) + +#define USBSTS_HCH (1<<0) +#define USBSTS_HSE (1<<2) +#define USBSTS_EINT (1<<3) +#define USBSTS_PCD (1<<4) +#define USBSTS_SSS (1<<8) +#define USBSTS_RSS (1<<9) +#define USBSTS_SRE (1<<10) +#define USBSTS_CNR (1<<11) +#define USBSTS_HCE (1<<12) + + +#define PORTSC_CCS (1<<0) +#define PORTSC_PED (1<<1) +#define PORTSC_OCA (1<<3) +#define PORTSC_PR (1<<4) +#define PORTSC_PLS_SHIFT 5 +#define PORTSC_PLS_MASK 0xf +#define PORTSC_PP (1<<9) +#define PORTSC_SPEED_SHIFT 10 +#define PORTSC_SPEED_MASK 0xf +#define PORTSC_SPEED_FULL (1<<10) +#define PORTSC_SPEED_LOW (2<<10) +#define PORTSC_SPEED_HIGH (3<<10) +#define PORTSC_SPEED_SUPER (4<<10) +#define PORTSC_PIC_SHIFT 14 +#define PORTSC_PIC_MASK 0x3 +#define PORTSC_LWS (1<<16) +#define PORTSC_CSC (1<<17) +#define PORTSC_PEC (1<<18) +#define PORTSC_WRC (1<<19) +#define PORTSC_OCC (1<<20) +#define PORTSC_PRC (1<<21) +#define PORTSC_PLC (1<<22) +#define PORTSC_CEC (1<<23) +#define PORTSC_CAS (1<<24) +#define PORTSC_WCE (1<<25) +#define PORTSC_WDE (1<<26) +#define PORTSC_WOE (1<<27) +#define PORTSC_DR (1<<30) +#define PORTSC_WPR (1<<31) + +#define CRCR_RCS (1<<0) +#define CRCR_CS (1<<1) +#define CRCR_CA (1<<2) +#define CRCR_CRR (1<<3) + +#define IMAN_IP (1<<0) +#define IMAN_IE (1<<1) + +#define ERDP_EHB (1<<3) + +#define TRB_SIZE 16 + +enum { + PLS_U0 =3D 0, + PLS_U1 =3D 1, + PLS_U2 =3D 2, + PLS_U3 =3D 3, + PLS_DISABLED =3D 4, + PLS_RX_DETECT =3D 5, + PLS_INACTIVE =3D 6, + PLS_POLLING =3D 7, + PLS_RECOVERY =3D 8, + PLS_HOT_RESET =3D 9, + PLS_COMPILANCE_MODE =3D 10, + PLS_TEST_MODE =3D 11, + PLS_RESUME =3D 15, +}; + +#define CR_LINK TR_LINK + +#define TRB_C (1<<0) +#define TRB_TYPE_SHIFT 10 +#define TRB_TYPE_MASK 0x3f +#define TRB_TYPE(t) (((t).control >> TRB_TYPE_SHIFT) & TRB_TYPE_MA= SK) + +#define TRB_EV_ED (1<<2) + +#define TRB_TR_ENT (1<<1) +#define TRB_TR_ISP (1<<2) +#define TRB_TR_NS (1<<3) +#define TRB_TR_CH (1<<4) +#define TRB_TR_IOC (1<<5) +#define TRB_TR_IDT (1<<6) +#define TRB_TR_TBC_SHIFT 7 +#define TRB_TR_TBC_MASK 0x3 +#define TRB_TR_BEI (1<<9) +#define TRB_TR_TLBPC_SHIFT 16 +#define TRB_TR_TLBPC_MASK 0xf +#define TRB_TR_FRAMEID_SHIFT 20 +#define TRB_TR_FRAMEID_MASK 0x7ff +#define TRB_TR_SIA (1<<31) + +#define TRB_TR_DIR (1<<16) + +#define TRB_CR_SLOTID_SHIFT 24 +#define TRB_CR_SLOTID_MASK 0xff +#define TRB_CR_EPID_SHIFT 16 +#define TRB_CR_EPID_MASK 0x1f + +#define TRB_CR_BSR (1<<9) +#define TRB_CR_DC (1<<9) + +#define TRB_LK_TC (1<<1) + +#define TRB_INTR_SHIFT 22 +#define TRB_INTR_MASK 0x3ff +#define TRB_INTR(t) (((t).status >> TRB_INTR_SHIFT) & TRB_INTR_MAS= K) + +#define EP_TYPE_MASK 0x7 +#define EP_TYPE_SHIFT 3 + +#define EP_STATE_MASK 0x7 +#define EP_DISABLED (0<<0) +#define EP_RUNNING (1<<0) +#define EP_HALTED (2<<0) +#define EP_STOPPED (3<<0) +#define EP_ERROR (4<<0) + +#define SLOT_STATE_MASK 0x1f +#define SLOT_STATE_SHIFT 27 +#define SLOT_STATE(s) (((s)>>SLOT_STATE_SHIFT)&SLOT_STATE_MASK) +#define SLOT_ENABLED 0 +#define SLOT_DEFAULT 1 +#define SLOT_ADDRESSED 2 +#define SLOT_CONFIGURED 3 + +#define SLOT_CONTEXT_ENTRIES_MASK 0x1f +#define SLOT_CONTEXT_ENTRIES_SHIFT 27 + +typedef enum EPType { + ET_INVALID =3D 0, + ET_ISO_OUT, + ET_BULK_OUT, + ET_INTR_OUT, + ET_CONTROL, + ET_ISO_IN, + ET_BULK_IN, + ET_INTR_IN, +} EPType; + +#endif /* TESTS_USB_HCD_XHCI_H */ diff --git a/tests/qtest/usb-hcd-xhci-test.c b/tests/qtest/usb-hcd-xhci-tes= t.c index 0cccfd85a6..d66e76f070 100644 --- a/tests/qtest/usb-hcd-xhci-test.c +++ b/tests/qtest/usb-hcd-xhci-test.c @@ -8,17 +8,188 @@ */ =20 #include "qemu/osdep.h" +#include "qemu/bswap.h" +#include "libqtest.h" +#include "libqos/libqos-pc.h" #include "libqtest-single.h" #include "libqos/usb.h" +#include "hw/pci/pci_ids.h" +#include "hw/pci/pci_regs.h" +#include "usb-hcd-xhci-test.h" + +/*** Test Setup & Teardown ***/ +typedef struct XHCIEvRingSeg { + uint32_t addr_low; + uint32_t addr_high; + uint32_t size; + uint32_t rsvd; +} XHCIEvRingSeg; + +typedef struct XHCITRB { + uint64_t parameter; + uint32_t status; + uint32_t control; +} XHCITRB; + +typedef struct XHCIQSlotState { + /* In-memory arrays */ + uint64_t device_context; + uint64_t transfer_ring; + + uint32_t tr_trb_entries; + uint32_t tr_trb_idx; + uint32_t tr_trb_c; +} XHCIQSlotState; + +typedef struct XHCIQState { + /* QEMU PCI variables */ + QOSState *parent; + QPCIDevice *dev; + QPCIBar bar; + uint64_t barsize; + uint32_t fingerprint; + + /* In-memory arrays */ + uint64_t dc_base_array; + uint64_t command_ring; + uint64_t event_ring_seg; + uint64_t event_ring; + + uint32_t cr_trb_entries; + uint32_t cr_trb_idx; + uint32_t cr_trb_c; + uint32_t er_trb_entries; + uint32_t er_trb_idx; + uint32_t er_trb_c; + + /* Host controller properties */ + uint32_t rtoff, dboff; + uint32_t maxports, maxslots, maxintrs; + + XHCIQSlotState slots[32]; +} XHCIQState; + +#define XHCI_NEC_ID (PCI_DEVICE_ID_NEC_UPD720200 << 16 | \ + PCI_VENDOR_ID_NEC) + +/** + * Locate, verify, and return a handle to the XHCI device. + */ +static QPCIDevice *get_xhci_device(QTestState *qts, uint32_t *fingerprint) +{ + QPCIDevice *xhci; + uint32_t xhci_fingerprint; + QPCIBus *pcibus; + + pcibus =3D qpci_new_pc(qts, NULL); + + /* Find the XHCI PCI device and verify it's the right one. */ + xhci =3D qpci_device_find(pcibus, QPCI_DEVFN(0x1D, 0x0)); + g_assert(xhci !=3D NULL); + + xhci_fingerprint =3D qpci_config_readl(xhci, PCI_VENDOR_ID); + switch (xhci_fingerprint) { + case XHCI_NEC_ID: + break; + default: + /* Unknown device. */ + g_assert_not_reached(); + } + + if (fingerprint) { + *fingerprint =3D xhci_fingerprint; + } + return xhci; +} + +static void free_xhci_device(QPCIDevice *dev) +{ + QPCIBus *pcibus =3D dev ? dev->bus : NULL; + + /* libqos doesn't have a function for this, so free it manually */ + g_free(dev); + qpci_free_pc(pcibus); +} + +/** + * Start a Q35 machine and bookmark a handle to the XHCI device. + */ +G_GNUC_PRINTF(1, 0) +static XHCIQState *xhci_vboot(const char *cli, va_list ap) +{ + XHCIQState *s; + + s =3D g_new0(XHCIQState, 1); + s->parent =3D qtest_pc_vboot(cli, ap); + alloc_set_flags(&s->parent->alloc, ALLOC_LEAK_ASSERT); + + /* Verify that we have an XHCI device present. */ + s->dev =3D get_xhci_device(s->parent->qts, &s->fingerprint); + s->bar =3D qpci_iomap(s->dev, 0, &s->barsize); + /* turns on pci.cmd.iose, pci.cmd.mse and pci.cmd.bme */ + qpci_device_enable(s->dev); + + return s; +} + +/** + * Start a Q35 machine and bookmark a handle to the XHCI device. + */ +G_GNUC_PRINTF(1, 2) +static XHCIQState *xhci_boot(const char *cli, ...) +{ + XHCIQState *s; + va_list ap; + + if (cli) { + va_start(ap, cli); + s =3D xhci_vboot(cli, ap); + va_end(ap); + } else { + s =3D xhci_boot("-M q35 " + "-device nec-usb-xhci,id=3Dxhci,bus=3Dpcie.0,addr=3D= 1d.0 " + "-drive id=3Ddrive0,if=3Dnone,file=3Dnull-co://," + "file.read-zeroes=3Don,format=3Draw"); + } + + return s; +} + +/** + * Clean up the PCI device, then terminate the QEMU instance. + */ +static void xhci_shutdown(XHCIQState *xhci) +{ + QOSState *qs =3D xhci->parent; + +// xhci_clean_mem(xhci); + free_xhci_device(xhci->dev); + g_free(xhci); + qtest_shutdown(qs); +} + +/*** tests ***/ =20 static void test_xhci_hotplug(void) { - usb_test_hotplug(global_qtest, "xhci", "1", NULL); + XHCIQState *s; + QTestState *qts; + + s =3D xhci_boot(NULL); + qts =3D s->parent->qts; + + usb_test_hotplug(qts, "xhci", "1", NULL); + + xhci_shutdown(s); } =20 static void test_usb_uas_hotplug(void) { - QTestState *qts =3D global_qtest; + XHCIQState *s; + QTestState *qts; + + s =3D xhci_boot(NULL); + qts =3D s->parent->qts; =20 qtest_qmp_device_add(qts, "usb-uas", "uas", "{}"); qtest_qmp_device_add(qts, "scsi-hd", "scsihd", "{'drive': 'drive0'}"); @@ -30,25 +201,347 @@ static void test_usb_uas_hotplug(void) =20 qtest_qmp_device_del(qts, "scsihd"); qtest_qmp_device_del(qts, "uas"); + + xhci_shutdown(s); } =20 static void test_usb_ccid_hotplug(void) { - QTestState *qts =3D global_qtest; + XHCIQState *s; + QTestState *qts; + + s =3D xhci_boot(NULL); + qts =3D s->parent->qts; =20 qtest_qmp_device_add(qts, "usb-ccid", "ccid", "{}"); qtest_qmp_device_del(qts, "ccid"); /* check the device can be added again */ qtest_qmp_device_add(qts, "usb-ccid", "ccid", "{}"); qtest_qmp_device_del(qts, "ccid"); + + xhci_shutdown(s); +} + +static uint64_t xhci_guest_zalloc(XHCIQState *s, uint64_t size) +{ + char mem[0x1000]; + uint64_t ret; + + g_assert(size <=3D 0x1000); + + memset(mem, 0, size); + + ret =3D guest_alloc(&s->parent->alloc, size); + qtest_memwrite(s->parent->qts, ret, mem, size); + + return ret; +} + +static uint32_t xhci_cap_readl(XHCIQState *s, uint64_t addr) +{ + return qpci_io_readl(s->dev, s->bar, addr); +} + +static uint32_t xhci_op_readl(XHCIQState *s, uint64_t addr) +{ + return qpci_io_readl(s->dev, s->bar, 0x40 + addr); +} + +static void xhci_op_writel(XHCIQState *s, uint64_t addr, uint32_t value) +{ + qpci_io_writel(s->dev, s->bar, 0x40 + addr, value); +} + +static uint32_t xhci_port_readl(XHCIQState *s, uint32_t port, uint64_t add= r) +{ + return xhci_op_readl(s, 0x400 + port * 0x10 + addr); +} + +static uint32_t xhci_rt_readl(XHCIQState *s, uint64_t addr) +{ + return qpci_io_readl(s->dev, s->bar, s->rtoff + addr); +} + +static void xhci_rt_writel(XHCIQState *s, uint64_t addr, uint32_t value) +{ + qpci_io_writel(s->dev, s->bar, s->rtoff + addr, value); +} + +static void xhci_db_writel(XHCIQState *s, uint32_t db, uint32_t value) +{ + qpci_io_writel(s->dev, s->bar, s->dboff + db * 4, value); +} + +static void wait_event_trb(XHCIQState *s, XHCITRB *trb) +{ + XHCITRB t; + uint64_t er_addr =3D s->event_ring + s->er_trb_idx * sizeof(*trb); + uint32_t value; + guint64 end_time =3D g_get_monotonic_time() + 5 * G_TIME_SPAN_SECOND; + + /* Wait for event interrupt */ + + do { + if (g_get_monotonic_time() >=3D end_time) { + g_error("Timeout expired"); + } + qtest_clock_step(s->parent->qts, 10000); + + value =3D xhci_op_readl(s, 0x4); /* USBSTS */ + } while (!(value & USBSTS_EINT)); + + value =3D xhci_rt_readl(s, 0x20 + 0x0); /* IMAN */ + + /* With MSI-X enabled, IMAN IP is cleared after raising the interrupt = */ + g_assert(!(value & IMAN_IP)); + + /* Ensure MSI-X interrupt is pending */ + assert(qpci_msix_test_clear_pending(s->dev, 0)); + /* Then cleared */ + assert(!qpci_msix_pending(s->dev, 0)); + + xhci_op_writel(s, 0x4, USBSTS_EINT); /* USBSTS clear EINT */ + + qtest_memread(s->parent->qts, er_addr, &t, sizeof(t)); + + trb->parameter =3D le64_to_cpu(t.parameter); + trb->status =3D le32_to_cpu(t.status); + trb->control =3D le32_to_cpu(t.control); + + g_assert((trb->status >> 24) =3D=3D CC_SUCCESS); + g_assert((trb->control & TRB_C) =3D=3D s->er_trb_c); /* C bit has been= set */ + + s->er_trb_idx++; + if (s->er_trb_idx =3D=3D s->er_trb_entries) { + s->er_trb_idx =3D 0; + s->er_trb_c ^=3D 1; + } + /* Update ERDP to processed TRB addr and EHB bit, which clears EHB */ + er_addr =3D s->event_ring + s->er_trb_idx * sizeof(*trb); + xhci_rt_writel(s, 0x38, (er_addr & 0xffffffff) | ERDP_EHB); +} + +static void set_link_trb(XHCIQState *s, uint64_t ring, uint32_t c, uint32_= t entries) +{ + XHCITRB trb; + + g_assert(entries > 1); + + memset(&trb, 0, sizeof(trb)); + trb.parameter =3D cpu_to_le64(ring); + trb.control =3D cpu_to_le32(c | /* C */ + (TR_LINK << TRB_TYPE_SHIFT) | + TRB_LK_TC); + qtest_memwrite(s->parent->qts, ring + sizeof(trb) * (entries - 1), &tr= b, sizeof(trb)); } =20 +static void submit_cr_trb(XHCIQState *s, XHCITRB *trb) +{ + XHCITRB t; + uint64_t cr_addr =3D s->command_ring + s->cr_trb_idx * sizeof(*trb); + + trb->control |=3D s->cr_trb_c; /* C */ + + t.parameter =3D cpu_to_le64(trb->parameter); + t.status =3D cpu_to_le32(trb->status); + t.control =3D cpu_to_le32(trb->control); + + qtest_memwrite(s->parent->qts, cr_addr, &t, sizeof(t)); + s->cr_trb_idx++; + /* Last entry contains the link, so wrap back */ + if (s->cr_trb_idx =3D=3D s->cr_trb_entries - 1) { + set_link_trb(s, s->command_ring, s->cr_trb_c, s->cr_trb_entries); + s->cr_trb_idx =3D 0; + s->cr_trb_c ^=3D 1; + } + xhci_db_writel(s, 0, 0); /* doorbell 0 */ +} + +static void pci_xhci_stress_rings(void) +{ + XHCIQState *s; + uint32_t value; + uint64_t input_context; + XHCIEvRingSeg ev_seg; + XHCITRB trb; + uint32_t hcsparams1; + uint32_t slotid; + void *mem; + int i; + + mem =3D g_malloc(0x1000); + memset(mem, 0, 0x1000); + + s =3D xhci_boot("-M q35 " + "-device nec-usb-xhci,id=3Dxhci,bus=3Dpcie.0,addr=3D1d.0 " + "-device usb-storage,bus=3Dxhci.0,drive=3Ddrive0 " + "-drive id=3Ddrive0,if=3Dnone,file=3Dnull-co://," + "file.read-zeroes=3Don,format=3Draw " + ); +// "-d trace:*xhci*,trace:*usb*,trace:*msi*"); + + hcsparams1 =3D xhci_cap_readl(s, 0x4); /* HCSPARAMS1 */ + s->maxports =3D (hcsparams1 >> 24) & 0xff; + s->maxintrs =3D (hcsparams1 >> 8) & 0x3ff; + s->maxslots =3D hcsparams1 & 0xff; + + s->dboff =3D xhci_cap_readl(s, 0x14); /* DBOFF */ + s->rtoff =3D xhci_cap_readl(s, 0x18); /* RTOFF */ + + s->dc_base_array =3D xhci_guest_zalloc(s, 0x800); + s->command_ring =3D xhci_guest_zalloc(s, 0x1000); + s->event_ring =3D xhci_guest_zalloc(s, 0x1000); + s->event_ring_seg =3D xhci_guest_zalloc(s, 0x100); + + /* Arbitrary small sizes so we can make them wrap */ + s->cr_trb_entries =3D 0x20; + s->cr_trb_c =3D 1; + s->er_trb_entries =3D 0x10; + s->er_trb_c =3D 1; + + ev_seg.addr_low =3D cpu_to_le32(s->event_ring & 0xffffffff); + ev_seg.addr_high =3D cpu_to_le32(s->event_ring >> 32); + ev_seg.size =3D cpu_to_le32(0x10); + ev_seg.rsvd =3D 0; + qtest_memwrite(s->parent->qts, s->event_ring_seg, &ev_seg, sizeof(ev_s= eg)); + + xhci_op_writel(s, 0x0, USBCMD_HCRST); /* USBCMD */ + do { + value =3D xhci_op_readl(s, 0x4); /* USBSTS */ + } while (value & (1 << 11)); /* CNR */ + + xhci_op_writel(s, 0x38, s->maxslots); /* CONFIG */ + + /* DCBAAP */ + xhci_op_writel(s, 0x30, s->dc_base_array & 0xffffffff); + xhci_op_writel(s, 0x34, s->dc_base_array >> 32); + + /* CRCR */ + xhci_op_writel(s, 0x18, (s->command_ring & 0xffffffff) | s->cr_trb_c); + xhci_op_writel(s, 0x1c, s->command_ring >> 32); + + xhci_rt_writel(s, 0x28, 1); /* ERSTSZ */ + + /* ERSTBA */ + xhci_rt_writel(s, 0x30, s->event_ring_seg & 0xffffffff); + xhci_rt_writel(s, 0x34, s->event_ring_seg >> 32); + + /* ERDP */ + xhci_rt_writel(s, 0x38, s->event_ring & 0xffffffff); + xhci_rt_writel(s, 0x3c, s->event_ring >> 32); + + qpci_msix_enable(s->dev); + xhci_op_writel(s, 0x0, USBCMD_RS | USBCMD_INTE); /* RUN + INTE */ + + /* Enable interrupts on ER IMAN */ + xhci_rt_writel(s, 0x20, IMAN_IE); + + assert(!qpci_msix_pending(s->dev, 0)); + + /* Wrap the command and event rings with no-ops a few times */ + for (i =3D 0; i < 100; i++) { + /* Issue a command ring no-op */ + memset(&trb, 0, sizeof(trb)); + trb.control |=3D CR_NOOP << TRB_TYPE_SHIFT; + trb.control |=3D TRB_TR_IOC; + submit_cr_trb(s, &trb); + wait_event_trb(s, &trb); + } + + /* Query ports */ + for (i =3D 0; i < s->maxports; i++) { + value =3D xhci_port_readl(s, i, 0); /* PORTSC */ + + /* Only first port should be attached and enabled */ + if (i =3D=3D 0) { + g_assert(value & PORTSC_CCS); + g_assert(value & PORTSC_PED); + /* Port Speed must be identified */ + g_assert(((value >> PORTSC_SPEED_SHIFT) & PORTSC_SPEED_MASK) !=3D 0); + } else { + g_assert(!(value & PORTSC_CCS)); + g_assert(!(value & PORTSC_PED)); + g_assert(((value >> PORTSC_PLS_SHIFT) & PORTSC_PLS_MASK) =3D= =3D 5); + } + } + + /* Issue a command ring enable slot */ + memset(&trb, 0, sizeof(trb)); + trb.control |=3D CR_ENABLE_SLOT << TRB_TYPE_SHIFT; + trb.control |=3D TRB_TR_IOC; + submit_cr_trb(s, &trb); + wait_event_trb(s, &trb); + slotid =3D (trb.control >> TRB_CR_SLOTID_SHIFT) & 0xff; + + s->slots[slotid].transfer_ring =3D xhci_guest_zalloc(s, 0x1000); + s->slots[slotid].tr_trb_entries =3D 0x10; + s->slots[slotid].tr_trb_c =3D 1; + + /* 32-byte input context size, should check HCCPARAMS1 for 64-byte siz= e */ + input_context =3D xhci_guest_zalloc(s, 0x420); + + /* Set input control context */ + ((uint32_t *)mem)[1] =3D cpu_to_le32(0x3); /* Add device contexts 0 an= d 1 */ + ((uint32_t *)mem)[8] =3D cpu_to_le32(1 << 27); /* 1 context entry */ + ((uint32_t *)mem)[9] =3D cpu_to_le32(1 << 16); /* 1 port number */ + + /* Set endpoint 0 context */ + ((uint32_t *)mem)[16] =3D 0; + ((uint32_t *)mem)[17] =3D cpu_to_le32((ET_CONTROL << EP_TYPE_SHIFT) | + (0x200 << 16)); /* max packet sz X= XX? */ + ((uint32_t *)mem)[18] =3D cpu_to_le32((s->slots[slotid].transfer_ring = & 0xffffffff) | 1); /* DCS=3D1 */ + ((uint32_t *)mem)[19] =3D cpu_to_le32(s->slots[slotid].transfer_ring >= > 32); + ((uint32_t *)mem)[20] =3D cpu_to_le32(0x200); /* Average TRB length */ + qtest_memwrite(s->parent->qts, input_context, mem, 0x420); + + s->slots[slotid].device_context =3D xhci_guest_zalloc(s, 0x400); + + ((uint64_t *)mem)[0] =3D cpu_to_le64(s->slots[slotid].device_context); + qtest_memwrite(s->parent->qts, s->dc_base_array + 8*slotid, mem, 8); + + /* Issue a command ring address device */ + memset(&trb, 0, sizeof(trb)); + trb.parameter =3D input_context; + trb.control |=3D CR_ADDRESS_DEVICE << TRB_TYPE_SHIFT; + trb.control |=3D slotid << TRB_CR_SLOTID_SHIFT; + submit_cr_trb(s, &trb); + wait_event_trb(s, &trb); + + /* XXX: Check EP state is running? */ + + /* Shut it down */ + qpci_msix_disable(s->dev); + + guest_free(&s->parent->alloc, s->slots[slotid].device_context); + guest_free(&s->parent->alloc, s->slots[slotid].transfer_ring); + guest_free(&s->parent->alloc, input_context); + guest_free(&s->parent->alloc, s->event_ring); + guest_free(&s->parent->alloc, s->event_ring_seg); + guest_free(&s->parent->alloc, s->command_ring); + guest_free(&s->parent->alloc, s->dc_base_array); + + xhci_shutdown(s); +} + +/* tests */ int main(int argc, char **argv) { int ret; + const char *arch; =20 g_test_init(&argc, &argv, NULL); =20 + /* Check architecture */ + arch =3D qtest_get_arch(); + if (strcmp(arch, "i386") && strcmp(arch, "x86_64")) { + g_test_message("Skipping test for non-x86"); + return 0; + } + + if (!qtest_has_device("nec-usb-xhci")) { + return 0; + } + qtest_add_func("/xhci/pci/hotplug", test_xhci_hotplug); if (qtest_has_device("usb-uas")) { qtest_add_func("/xhci/pci/hotplug/usb-uas", test_usb_uas_hotplug); @@ -56,11 +549,12 @@ int main(int argc, char **argv) if (qtest_has_device("usb-ccid")) { qtest_add_func("/xhci/pci/hotplug/usb-ccid", test_usb_ccid_hotplug= ); } + if (qtest_has_device("usb-storage")) { + qtest_add_func("/xhci/pci/xhci-stress-rings", pci_xhci_stress_ring= s); + } =20 - qtest_start("-device nec-usb-xhci,id=3Dxhci" - " -drive id=3Ddrive0,if=3Dnone,file=3Dnull-co://," - "file.read-zeroes=3Don,format=3Draw"); ret =3D g_test_run(); + qtest_end(); =20 return ret; --=20 2.45.2 From nobody Sat Nov 23 19:31:16 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1731080721; cv=none; d=zohomail.com; s=zohoarc; b=fvpUS23/xKVaCNQYJyTCkypWq978XdCnZQlfxh+U4YDdfpws0S6nBS+zxXE7qEVEQLt1Z+tYkSe8+9XuYfbdhQSRGRI8S2FkvE5Ky2LPDlD8bd23wfAIDYWK6hUVrRXgq6gAz0A9Gy0CisN8ej/79oYKZqNthNZWt5oIewsSr80= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1731080721; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=8YXGNtau+JOHmNo5R4BLXeKKLnW7DMVKJ7snPHk/S1E=; b=SxPNh8TeHCYW8dh0+T47XhD8tqVLTubqywcbIyD0w568LzE4iy6ks7cQWRDtrf1jPvcBSIMWXa6wL4jLR2hHjyhvTMHTP+r5Fqg3Rva67QKhYKrIu2w9GPSKBADmT/3ZHLUOXJVFjFWKUfvonUGTOR/E2bvNGhy5F/OrKlIohwI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1731080720977883.3554558862152; Fri, 8 Nov 2024 07:45:20 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t9R8Z-0002K2-6G; Fri, 08 Nov 2024 10:43:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t9R8X-0002Jp-Og for qemu-devel@nongnu.org; Fri, 08 Nov 2024 10:43:01 -0500 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t9R8W-0005Ia-4W for qemu-devel@nongnu.org; Fri, 08 Nov 2024 10:43:01 -0500 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-7205b6f51f3so1883179b3a.1 for ; Fri, 08 Nov 2024 07:42:59 -0800 (PST) Received: from wheely.local0.net (124-171-217-17.tpgi.com.au. [124.171.217.17]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7f41f48d32bsm3110444a12.17.2024.11.08.07.42.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 07:42:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731080578; x=1731685378; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8YXGNtau+JOHmNo5R4BLXeKKLnW7DMVKJ7snPHk/S1E=; b=ZMcvx19WArgR95ydIGmIvPHGP+qaU9mBgUPhb7VU6Lrs8YpZpLnoMgLINlFx0uKLdn XVrtuampQXtTApNI5mQmRE+7arcKypCEGgEzVr7BhMFm5tgxR7UDnnkbo+95bOtk9cus SfSyTUXPeTw0Pm+KgbYHYG773Zac2ImZVjnEu4jwMxERFlMFYBW0D8SZvvgTOSLiL15D 3hyp39UwdpAkFN4u9FU/ab1kb+9VqN/41m9UlOIGnvAkx26EGUdjV/QKSUOxa+xFAjp5 fbUj8nLJCuovZZeAyRdeiYsh0vDmCMrzLn6NkcjnfcpNFX6GuHdQKbm2WbtLkaSljE9r xF3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731080578; x=1731685378; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8YXGNtau+JOHmNo5R4BLXeKKLnW7DMVKJ7snPHk/S1E=; b=oBwA/RGemPPYANtSXf2RgZjzSrrUx+b0NllT56EmSBwKwqRUeBboc4qUJwlBHSJAn2 k2yUC6exQvGZJXizNGTl4rUQ6OUkWzERnUgacSRqo67qwFUNRCnhveU7LM6pXIVgA84s FQs3Rl5KXYaS5nuekRrhAWxH3lJFk/BTXtXR1bLWr/ymtKLP1S1k1v+n3OCEcZvB5YkA nNkdJz5+lgqnt+nVHyCt5rNCPN/QydxfQ8Pbfdg4Kj+Fdi2NMZEq3qpht35hDwZYVQDT QSMTC3spVG5Dj1OCE/foOV1bc1H44FbK6iIbWNapxLf3gR8Sp8nab1xw1JURwQE1oPSP gPoA== X-Gm-Message-State: AOJu0YwxYJw0asnx6s+MRgRw7TtXX0VVgdGQVYyAaxQG/iE6DeWgCCEJ yFPbxiEDbK/jh0kwS09zfZT7rLK8syUEr+vOGLiJ7WS1dJZTGaGSq+7KRQ== X-Google-Smtp-Source: AGHT+IEu64Vj33987GfRKRL3Kkgfx4OVYyvViKQytS1L06GskA/2TPkWC8T7Ez0mCHOiwj32wO4cHQ== X-Received: by 2002:a05:6a21:7881:b0:1db:e425:c8a2 with SMTP id adf61e73a8af0-1dc229a40e4mr4002760637.19.1731080578014; Fri, 08 Nov 2024 07:42:58 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S. Tsirkin" , Marcel Apfelbaum , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Gerd Hoffmann Subject: [RFC PATCH 5/5] hw/usb: Support XHCI TR NOOP commands Date: Sat, 9 Nov 2024 01:42:28 +1000 Message-ID: <20241108154229.263097-6-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108154229.263097-1-npiggin@gmail.com> References: <20241108154229.263097-1-npiggin@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=npiggin@gmail.com; helo=mail-pf1-x434.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1731080723020116600 Content-Type: text/plain; charset="utf-8" Implement TR NOOP commands by setting up then immediately completing the packet. Add a TR NOOP test to the xhci qtest. The IBM AIX XHCI driver uses NOOP commands to check driver and hardware health. Signed-off-by: Nicholas Piggin --- hw/usb/hcd-xhci.c | 28 ++++++++++++++++++++++++++- tests/qtest/usb-hcd-xhci-test.c | 34 +++++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+), 1 deletion(-) diff --git a/hw/usb/hcd-xhci.c b/hw/usb/hcd-xhci.c index d85adaca0d..9e223acd83 100644 --- a/hw/usb/hcd-xhci.c +++ b/hw/usb/hcd-xhci.c @@ -1832,6 +1832,20 @@ static int xhci_fire_transfer(XHCIState *xhci, XHCIT= ransfer *xfer, XHCIEPContext return xhci_submit(xhci, xfer, epctx); } =20 +static int xhci_noop_transfer(XHCIState *xhci, XHCITransfer *xfer) +{ + /* + * TR NOOP conceptually probably better not call into USB subsystem + * (usb_packet_setup() via xhci_setup_packet()). In practice it + * works and avoids code duplication. + */ + if (xhci_setup_packet(xfer) < 0) { + return -1; + } + xhci_try_complete_packet(xfer); + return 0; +} + static void xhci_kick_ep(XHCIState *xhci, unsigned int slotid, unsigned int epid, unsigned int streamid) { @@ -1954,6 +1968,8 @@ static void xhci_kick_epctx(XHCIEPContext *epctx, uns= igned int streamid) =20 epctx->kick_active++; while (1) { + bool noop =3D false; + length =3D xhci_ring_chain_length(xhci, ring); if (length <=3D 0) { if (epctx->type =3D=3D ET_ISO_OUT || epctx->type =3D=3D ET_ISO= _IN) { @@ -1982,10 +1998,20 @@ static void xhci_kick_epctx(XHCIEPContext *epctx, u= nsigned int streamid) epctx->kick_active--; return; } + if (type =3D=3D TR_NOOP) { + noop =3D true; + } } xfer->streamid =3D streamid; =20 - if (epctx->epid =3D=3D 1) { + if (noop) { + if (length !=3D 1) { + qemu_log_mask(LOG_GUEST_ERROR, + "%s: NOOP TR TRB within TRB chain!\n", __fun= c__); + /* Undefined behavior, we no-op the entire chain */ + } + xhci_noop_transfer(xhci, xfer); + } else if (epctx->epid =3D=3D 1) { xhci_fire_ctl_transfer(xhci, xfer); } else { xhci_fire_transfer(xhci, xfer, epctx); diff --git a/tests/qtest/usb-hcd-xhci-test.c b/tests/qtest/usb-hcd-xhci-tes= t.c index d66e76f070..8a36f42522 100644 --- a/tests/qtest/usb-hcd-xhci-test.c +++ b/tests/qtest/usb-hcd-xhci-test.c @@ -357,6 +357,30 @@ static void submit_cr_trb(XHCIQState *s, XHCITRB *trb) xhci_db_writel(s, 0, 0); /* doorbell 0 */ } =20 +static void submit_tr_trb(XHCIQState *s, int slot, XHCITRB *trb) +{ + XHCITRB t; + uint64_t tr_addr =3D s->slots[slot].transfer_ring + s->slots[slot].tr_= trb_idx * sizeof(*trb); + + trb->control |=3D s->slots[slot].tr_trb_c; /* C */ + + t.parameter =3D cpu_to_le64(trb->parameter); + t.status =3D cpu_to_le32(trb->status); + t.control =3D cpu_to_le32(trb->control); + + qtest_memwrite(s->parent->qts, tr_addr, &t, sizeof(t)); + s->slots[slot].tr_trb_idx++; + /* Last entry contains the link, so wrap back */ + if (s->slots[slot].tr_trb_idx =3D=3D s->slots[slot].tr_trb_entries - 1= ) { + set_link_trb(s, s->slots[slot].transfer_ring, + s->slots[slot].tr_trb_c, + s->slots[slot].tr_trb_entries); + s->slots[slot].tr_trb_idx =3D 0; + s->slots[slot].tr_trb_c ^=3D 1; + } + xhci_db_writel(s, slot, 1); /* doorbell slot, EP0 target */ +} + static void pci_xhci_stress_rings(void) { XHCIQState *s; @@ -509,6 +533,16 @@ static void pci_xhci_stress_rings(void) =20 /* XXX: Check EP state is running? */ =20 + /* Wrap the transfer ring a few times */ + for (i =3D 0; i < 100; i++) { + /* Issue a transfer ring slot 0 noop */ + memset(&trb, 0, sizeof(trb)); + trb.control |=3D TR_NOOP << TRB_TYPE_SHIFT; + trb.control |=3D TRB_TR_IOC; + submit_tr_trb(s, slotid, &trb); + wait_event_trb(s, &trb); + } + /* Shut it down */ qpci_msix_disable(s->dev); =20 --=20 2.45.2