From nobody Wed Apr 2 13:35:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=reject dis=none) header.from=linux.ibm.com ARC-Seal: i=1; a=rsa-sha256; t=1743188900; cv=none; d=zohomail.com; s=zohoarc; b=YF1XzsPSmyWr82TU+C70dHf/TReWymqF9HO3lX7JBDePvvsaSV84eN2NSzJQ3+4YLpTmY1hltxN4o5Ipa56A1xMzOsUAsVdc+A94rqIrLOQVVnw7LzgIeOlPC82B2VrkccEEj0M4K9vnSp5pNEH7zexrZP1TKz+1/TauFDAmymo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1743188900; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ApBKIxcvkYHwABDFsm6vwkCfcskVCxxcrZbUhNz2sdk=; b=ElqO1PqYxUlYSXkE+YQD22SuGgCYEz//idKl4AfgN3CNTeZ7lDrGovi/u4v0LENw34/xM0m+j01I4Ek25uw7w0JV28WLZlDDQBE8wMrQN0UiY183IZUngl4c/fcb/6tL/p5U77TPkav2ber+WjOb8HNKoAkoKe8y3W9R7fDG4ro= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1743188900932724.5349196729824; Fri, 28 Mar 2025 12:08:20 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tyF2p-0006qB-5O; Fri, 28 Mar 2025 15:07:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tyF2Z-0006l9-2Q; Fri, 28 Mar 2025 15:06:53 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tyF2W-0008OQ-B1; Fri, 28 Mar 2025 15:06:50 -0400 Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 52SIe45V028163; Fri, 28 Mar 2025 19:06:35 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45ndupwu1x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 28 Mar 2025 19:06:34 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 52SHLBs8012239; Fri, 28 Mar 2025 19:06:34 GMT Received: from smtprelay02.wdc07v.mail.ibm.com ([172.16.1.69]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 45j91mksuq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 28 Mar 2025 19:06:34 +0000 Received: from smtpav02.dal12v.mail.ibm.com (smtpav02.dal12v.mail.ibm.com [10.241.53.101]) by smtprelay02.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 52SJ6WF929033106 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 28 Mar 2025 19:06:33 GMT Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A996D58060; Fri, 28 Mar 2025 19:06:32 +0000 (GMT) Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B0A0F5805A; Fri, 28 Mar 2025 19:06:31 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.244.227]) by smtpav02.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 28 Mar 2025 19:06:31 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=ApBKIxcvkYHwABDFs m6vwkCfcskVCxxcrZbUhNz2sdk=; b=UOP7d1a6myyIIdqYE0pTNGVxt3n3wHSro q6fgE0Kat2aRnPdFedCKXSOBR+jJ2J0+ZjsWCJcrV7lbQRIS2PuCDHQwJPAdwx4V RigZj6kx7/y36UIhMhTwhTQTd3iYTOAq/9BLHJ2PQwvQ/j7c8YyOqgXII33vRbX4 b7NtIGG6W5sT4W+gVsxb9ShzKusopVceONNFvAUZnNNQT+nrpHicByYPnV0OVBUC rhPcsjnVAKVRaTGzWCIPvJutJnlRQyD6pBcuyFYhKtU4CuvRtT/dhfr1FNCVi4tf 0lwu2on1n56MPXSGL11CK22YjaH2PXDmUDnIlltyaLzpYJAslPkww== From: Farhan Ali To: qemu-devel@nongnu.org Cc: alifm@linux.ibm.com, mjrosato@linux.ibm.com, schnelle@linux.ibm.com, qemu-block@nongnu.org, qemu-s390x@nongnu.org, stefanha@redhat.com, fam@euphon.net, philmd@linaro.org, kwolf@redhat.com, hreitz@redhat.com, thuth@redhat.com Subject: [PATCH v2 1/3] util: Add functions for s390x mmio read/write Date: Fri, 28 Mar 2025 12:06:25 -0700 Message-ID: <20250328190627.3025-2-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250328190627.3025-1-alifm@linux.ibm.com> References: <20250328190627.3025-1-alifm@linux.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: bwII2NYyqgBiudUD21LzNM1IWc3e2JAf X-Proofpoint-ORIG-GUID: bwII2NYyqgBiudUD21LzNM1IWc3e2JAf X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-03-28_09,2025-03-27_02,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 phishscore=0 bulkscore=0 lowpriorityscore=0 impostorscore=0 adultscore=0 mlxlogscore=662 priorityscore=1501 mlxscore=0 suspectscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2503280129 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.158.5; envelope-from=alifm@linux.ibm.com; helo=mx0b-001b2d01.pphosted.com X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ibm.com) X-ZM-MESSAGEID: 1743188903464019100 Content-Type: text/plain; charset="utf-8" Starting with z15 (or newer) we can execute mmio instructions from userspace. On older platforms where we don't have these instructions available we can fallback to using system calls to access the PCI mapped resources. This patch adds helper functions for mmio reads and writes for s390x. Reviewed-by: Stefan Hajnoczi Signed-off-by: Farhan Ali Reviewed-by: Niklas Schnelle --- include/qemu/s390x_pci_mmio.h | 23 ++++++ util/meson.build | 2 + util/s390x_pci_mmio.c | 148 ++++++++++++++++++++++++++++++++++ 3 files changed, 173 insertions(+) create mode 100644 include/qemu/s390x_pci_mmio.h create mode 100644 util/s390x_pci_mmio.c diff --git a/include/qemu/s390x_pci_mmio.h b/include/qemu/s390x_pci_mmio.h new file mode 100644 index 0000000000..aead791475 --- /dev/null +++ b/include/qemu/s390x_pci_mmio.h @@ -0,0 +1,23 @@ +/* + * s390x PCI MMIO definitions + * + * Copyright 2025 IBM Corp. + * Author(s): Farhan Ali + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ +#ifndef S390X_PCI_MMIO_H +#define S390X_PCI_MMIO_H + +uint8_t s390x_pci_mmio_read_8(const void *ioaddr); +uint16_t s390x_pci_mmio_read_16(const void *ioaddr); +uint32_t s390x_pci_mmio_read_32(const void *ioaddr); +uint64_t s390x_pci_mmio_read_64(const void *ioaddr); + +void s390x_pci_mmio_write_8(void *ioaddr, uint8_t val); +void s390x_pci_mmio_write_16(void *ioaddr, uint16_t val); +void s390x_pci_mmio_write_32(void *ioaddr, uint32_t val); +void s390x_pci_mmio_write_64(void *ioaddr, uint64_t val); + + +#endif diff --git a/util/meson.build b/util/meson.build index 780b5977a8..acb21592f9 100644 --- a/util/meson.build +++ b/util/meson.build @@ -131,4 +131,6 @@ elif cpu in ['ppc', 'ppc64'] util_ss.add(files('cpuinfo-ppc.c')) elif cpu in ['riscv32', 'riscv64'] util_ss.add(files('cpuinfo-riscv.c')) +elif cpu =3D=3D 's390x' + util_ss.add(files('s390x_pci_mmio.c')) endif diff --git a/util/s390x_pci_mmio.c b/util/s390x_pci_mmio.c new file mode 100644 index 0000000000..820458a026 --- /dev/null +++ b/util/s390x_pci_mmio.c @@ -0,0 +1,148 @@ +/* + * s390x PCI MMIO definitions + * + * Copyright 2025 IBM Corp. + * Author(s): Farhan Ali + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include +#include +#include "qemu/s390x_pci_mmio.h" +#include "elf.h" + +union register_pair { + unsigned __int128 pair; + struct { + uint64_t even; + uint64_t odd; + }; +}; + +static bool is_mio_supported; + +static __attribute__((constructor)) void check_is_mio_supported(void) +{ + is_mio_supported =3D !!(qemu_getauxval(AT_HWCAP) & HWCAP_S390_PCI_MIO); +} + +static uint64_t s390x_pcilgi(const void *ioaddr, size_t len) +{ + union register_pair ioaddr_len =3D { .even =3D (uint64_t)ioaddr, + .odd =3D len }; + uint64_t val; + int cc; + + asm volatile( + /* pcilgi */ + ".insn rre,0xb9d60000,%[val],%[ioaddr_len]\n" + "ipm %[cc]\n" + "srl %[cc],28\n" + : [cc] "=3Dd"(cc), [val] "=3Dd"(val), + [ioaddr_len] "+&d"(ioaddr_len.pair) :: "cc"); + + if (cc) { + val =3D -1ULL; + } + + return val; +} + +static void s390x_pcistgi(void *ioaddr, uint64_t val, size_t len) +{ + union register_pair ioaddr_len =3D {.even =3D (uint64_t)ioaddr, .odd = =3D len}; + + asm volatile ( + /* pcistgi */ + ".insn rre,0xb9d40000,%[val],%[ioaddr_len]\n" + : [ioaddr_len] "+&d" (ioaddr_len.pair) + : [val] "d" (val) + : "cc", "memory"); +} + +uint8_t s390x_pci_mmio_read_8(const void *ioaddr) +{ + uint8_t val =3D 0; + + if (is_mio_supported) { + val =3D s390x_pcilgi(ioaddr, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_read, ioaddr, &val, sizeof(val)); + } + return val; +} + +uint16_t s390x_pci_mmio_read_16(const void *ioaddr) +{ + uint16_t val =3D 0; + + if (is_mio_supported) { + val =3D s390x_pcilgi(ioaddr, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_read, ioaddr, &val, sizeof(val)); + } + return val; +} + +uint32_t s390x_pci_mmio_read_32(const void *ioaddr) +{ + uint32_t val =3D 0; + + if (is_mio_supported) { + val =3D s390x_pcilgi(ioaddr, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_read, ioaddr, &val, sizeof(val)); + } + return val; +} + +uint64_t s390x_pci_mmio_read_64(const void *ioaddr) +{ + uint64_t val =3D 0; + + if (is_mio_supported) { + val =3D s390x_pcilgi(ioaddr, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_read, ioaddr, &val, sizeof(val)); + } + return val; +} + +void s390x_pci_mmio_write_8(void *ioaddr, uint8_t val) +{ + if (is_mio_supported) { + s390x_pcistgi(ioaddr, val, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_write, ioaddr, &val, sizeof(val)); + } +} + +void s390x_pci_mmio_write_16(void *ioaddr, uint16_t val) +{ + if (is_mio_supported) { + s390x_pcistgi(ioaddr, val, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_write, ioaddr, &val, sizeof(val)); + } +} + +void s390x_pci_mmio_write_32(void *ioaddr, uint32_t val) +{ + if (is_mio_supported) { + s390x_pcistgi(ioaddr, val, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_write, ioaddr, &val, sizeof(val)); + } +} + +void s390x_pci_mmio_write_64(void *ioaddr, uint64_t val) +{ + if (is_mio_supported) { + s390x_pcistgi(ioaddr, val, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_write, ioaddr, &val, sizeof(val)); + } +} + --=20 2.43.0 From nobody Wed Apr 2 13:35:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=reject dis=none) header.from=linux.ibm.com ARC-Seal: i=1; a=rsa-sha256; t=1743188883; cv=none; d=zohomail.com; s=zohoarc; b=Gn5iR8FCr4099WO4wrmq8gZaQe0XU6IKgIUjyXLX9rB17OL28DbgdUuO0eWDxy1ZMGZk/fNwtxCAsHBD5H8glZxbnKTQRaOXjU+S5FL9/cw3E/T+oAI/chS/IeJI2PBvT2nXhu5NZCJVPIcboxMzzejwuMH1izmXdeg51sikzPQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1743188883; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=J0mwcD75hRL6j0S6B13UW9e3EdY89HRLmo8A7+2dqFo=; b=k9g19NvsecWHntpC72NyI+3CxxUSw4OCe8eMQKcbDX1nove11oFtiZgSBYix7B48eJ2PDPrPcYhyILnpAh/U2geqy7GkAvK+GDl0vhx1RQ4h7VxzSrJUMVi0Dls5ANgTcXjMNL+iAyF6vx4v0hB1435sbShSJOPPZ7iU08EJq4U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 174318888364515.524575028375693; Fri, 28 Mar 2025 12:08:03 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tyF2p-0006qZ-Rw; Fri, 28 Mar 2025 15:07:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tyF2W-0006kw-HF; Fri, 28 Mar 2025 15:06:49 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tyF2S-0008Ku-KN; Fri, 28 Mar 2025 15:06:48 -0400 Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 52SE3Cwe017920; Fri, 28 Mar 2025 19:06:37 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45nw6p9kkf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 28 Mar 2025 19:06:36 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 52SGXprB030418; Fri, 28 Mar 2025 19:06:35 GMT Received: from smtprelay03.wdc07v.mail.ibm.com ([172.16.1.70]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 45j7htv31v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 28 Mar 2025 19:06:35 +0000 Received: from smtpav02.dal12v.mail.ibm.com (smtpav02.dal12v.mail.ibm.com [10.241.53.101]) by smtprelay03.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 52SJ6W9625559576 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 28 Mar 2025 19:06:32 GMT Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DBDF25805C; Fri, 28 Mar 2025 19:06:33 +0000 (GMT) Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CD5965805A; Fri, 28 Mar 2025 19:06:32 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.244.227]) by smtpav02.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 28 Mar 2025 19:06:32 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=J0mwcD75hRL6j0S6B 13UW9e3EdY89HRLmo8A7+2dqFo=; b=PCK9fiAC+J5jca3XKh5PC8bByyN8AYr8M gNkF5KeOGWm4077NbiGNVohjTjQBw3NJIb5OHTwqbr0Ee8lfVSFuUg3isNvzimSo un7b6XHoohotbTu1XM8kUOhLdsVjHF9x2DD3VZBGMxPoRV8Pd2EPnxA66uGlPxz2 8E4NFHlXgBQcj8s2G5WdqLpjdqUON4aeo7mLiBOSB9ziDJZ0I3qpoZ6CmDoyc8TE ImlxP4F5k1t3xIfm5W196jvYeWsYUUpri1GSLN+bmatDnOfbRB94T27s7QrgawiL Laf+h5dzLXiNJu2Zzo3U1D7B/ZOl50lAw2q0dOxsWPYABNV2ey3+Q== From: Farhan Ali To: qemu-devel@nongnu.org Cc: alifm@linux.ibm.com, mjrosato@linux.ibm.com, schnelle@linux.ibm.com, qemu-block@nongnu.org, qemu-s390x@nongnu.org, stefanha@redhat.com, fam@euphon.net, philmd@linaro.org, kwolf@redhat.com, hreitz@redhat.com, thuth@redhat.com Subject: [PATCH v2 2/3] include: Add a header to define PCI MMIO functions Date: Fri, 28 Mar 2025 12:06:26 -0700 Message-ID: <20250328190627.3025-3-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250328190627.3025-1-alifm@linux.ibm.com> References: <20250328190627.3025-1-alifm@linux.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 2mcVHPskHn3GEU9IBFL19nAaVkp1quhX X-Proofpoint-GUID: 2mcVHPskHn3GEU9IBFL19nAaVkp1quhX X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-03-28_09,2025-03-27_02,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 spamscore=0 mlxscore=0 mlxlogscore=715 suspectscore=0 impostorscore=0 clxscore=1015 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2503280129 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.156.1; envelope-from=alifm@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ibm.com) X-ZM-MESSAGEID: 1743188885827019000 Content-Type: text/plain; charset="utf-8" Add a generic QEMU API for PCI MMIO reads/writes. The functions access little endian memory and returns the result in host cpu endianness. Signed-off-by: Farhan Ali Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- include/qemu/pci-mmio.h | 116 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100644 include/qemu/pci-mmio.h diff --git a/include/qemu/pci-mmio.h b/include/qemu/pci-mmio.h new file mode 100644 index 0000000000..2ef92455b1 --- /dev/null +++ b/include/qemu/pci-mmio.h @@ -0,0 +1,116 @@ +/* + * QEMU PCI MMIO API + * + * Copyright 2025 IBM Corp. + * Author(s): Farhan Ali + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef QEMU_PCI_MMIO_H +#define QEMU_PCI_MMIO_H + +#ifdef __s390x__ +#include "s390x_pci_mmio.h" +#endif + +static inline uint8_t qemu_pci_mmio_read_8(const void *ioaddr) +{ + uint8_t ret =3D 0; +#ifdef __s390x__ + ret =3D s390x_pci_mmio_read_8(ioaddr); +#else + /* Prevent the compiler from optimizing away the load */ + ret =3D *((volatile uint8_t *)ioaddr); +#endif + + return ret; +} + +static inline uint16_t qemu_pci_mmio_read_16(const void *ioaddr) +{ + uint16_t ret =3D 0; +#ifdef __s390x__ + ret =3D s390x_pci_mmio_read_16(ioaddr); +#else + /* Prevent the compiler from optimizing away the load */ + ret =3D *((volatile uint16_t *)ioaddr); +#endif + + return le16_to_cpu(ret); +} + +static inline uint32_t qemu_pci_mmio_read_32(const void *ioaddr) +{ + uint32_t ret =3D 0; +#ifdef __s390x__ + ret =3D s390x_pci_mmio_read_32(ioaddr); +#else + /* Prevent the compiler from optimizing away the load */ + ret =3D *((volatile uint32_t *)ioaddr); +#endif + + return le32_to_cpu(ret); +} + +static inline uint64_t qemu_pci_mmio_read_64(const void *ioaddr) +{ + uint64_t ret =3D 0; +#ifdef __s390x__ + ret =3D s390x_pci_mmio_read_64(ioaddr); +#else + /* Prevent the compiler from optimizing away the load */ + ret =3D *((volatile uint64_t *)ioaddr); +#endif + + return le64_to_cpu(ret); +} + +static inline void qemu_pci_mmio_write_8(void *ioaddr, uint8_t val) +{ + +#ifdef __s390x__ + s390x_pci_mmio_write_8(ioaddr, val); +#else + /* Prevent the compiler from optimizing away the store */ + *((volatile uint8_t *)ioaddr) =3D val; +#endif +} + +static inline void qemu_pci_mmio_write_16(void *ioaddr, uint16_t val) +{ + val =3D cpu_to_le16(val); + +#ifdef __s390x__ + s390x_pci_mmio_write_16(ioaddr, val); +#else + /* Prevent the compiler from optimizing away the store */ + *((volatile uint16_t *)ioaddr) =3D val; +#endif +} + +static inline void qemu_pci_mmio_write_32(void *ioaddr, uint32_t val) +{ + val =3D cpu_to_le32(val); + +#ifdef __s390x__ + s390x_pci_mmio_write_32(ioaddr, val); +#else + /* Prevent the compiler from optimizing away the store */ + *((volatile uint32_t *)ioaddr) =3D val; +#endif +} + +static inline void qemu_pci_mmio_write_64(void *ioaddr, uint64_t val) +{ + val =3D cpu_to_le64(val); + +#ifdef __s390x__ + s390x_pci_mmio_write_64(ioaddr, val); +#else + /* Prevent the compiler from optimizing away the store */ + *((volatile uint64_t *)ioaddr) =3D val; +#endif +} + +#endif --=20 2.43.0 From nobody Wed Apr 2 13:35:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=reject dis=none) header.from=linux.ibm.com ARC-Seal: i=1; a=rsa-sha256; t=1743188880; cv=none; d=zohomail.com; s=zohoarc; b=Sr1aX6lhY5S4yYmPXspmDqBQfp04Iljv2aQaV0Zd/eXRlYZ9JXSyye82Ne8hsf2sQ0IVWIV3fJJsXirClfWkWlGzWDR8T81veaHmgnBCP6BviRp5I5yUaHt7pPoS3KEykd4b5PiHXUHDkd2BRoSHjBCOi2jlsg46JUVhjZeYiWg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1743188880; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=+BIV4jx856itY0L7K69Qe7sM+j+o/HzSXu9hCouYCBo=; b=Ldq7cfLKpL/KSOAQf/jOUFtbyHNcRHL62vp+iQlIfYIbkl93BUv743Xqpw25soYIrW9ti+sE73PrzqR9ZfoHMCeOwHCSiumz5Q3dkq3cBfhSDal3YstOy/gSwCLPEyxyJMu4SmNGagV/Yj3E/I0sJgNPIbCSyuvP6IPgQGyDczY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1743188880003732.7389102180684; Fri, 28 Mar 2025 12:08:00 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tyF2n-0006nO-8R; Fri, 28 Mar 2025 15:07:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tyF2X-0006ky-GF; Fri, 28 Mar 2025 15:06:49 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tyF2T-0008M2-7E; Fri, 28 Mar 2025 15:06:48 -0400 Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 52SCS1VD011339; Fri, 28 Mar 2025 19:06:38 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45nut2j40n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 28 Mar 2025 19:06:37 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 52SGK1Za025485; Fri, 28 Mar 2025 19:06:36 GMT Received: from smtprelay05.wdc07v.mail.ibm.com ([172.16.1.72]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 45j7x0m0kt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 28 Mar 2025 19:06:36 +0000 Received: from smtpav02.dal12v.mail.ibm.com (smtpav02.dal12v.mail.ibm.com [10.241.53.101]) by smtprelay05.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 52SJ6Zwt28967470 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 28 Mar 2025 19:06:35 GMT Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 02E3E5805E; Fri, 28 Mar 2025 19:06:35 +0000 (GMT) Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0ACF95805A; Fri, 28 Mar 2025 19:06:34 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.244.227]) by smtpav02.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 28 Mar 2025 19:06:33 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=+BIV4jx856itY0L7K 69Qe7sM+j+o/HzSXu9hCouYCBo=; b=faaOqWu1tmSC+R0BupCQ/WRVvM/Oy7tyQ U00DQivTWawtlbJJkwb9yC1BaTeKnpgP+bzj9nOCn06i1CIHGlYLn8vFc7+BoFn8 VfUZnZa1cbWb2ebnNXvsDABuog+FYFPkFBJBbeYlAf1odYnGJRpnQts1TNI2RaP5 R9aYpWKwd5g821i16K6VHrQRUEMSmyrSo95cy8m75Z4IZYBlzLJLLKZDp1paEjew FwvbBcnMsCedD2qv5f2e+8UO7nQ3vP1XedpXciOJD08XoRutLk+XwWwgXciimMVR 4k+g6jIx/TU9jMHd3QjVMxo6tEptwCr0eVg7d9A+Yxdq+RO/VTZdA== From: Farhan Ali To: qemu-devel@nongnu.org Cc: alifm@linux.ibm.com, mjrosato@linux.ibm.com, schnelle@linux.ibm.com, qemu-block@nongnu.org, qemu-s390x@nongnu.org, stefanha@redhat.com, fam@euphon.net, philmd@linaro.org, kwolf@redhat.com, hreitz@redhat.com, thuth@redhat.com Subject: [PATCH v2 3/3] block/nvme: Use QEMU PCI MMIO API Date: Fri, 28 Mar 2025 12:06:27 -0700 Message-ID: <20250328190627.3025-4-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250328190627.3025-1-alifm@linux.ibm.com> References: <20250328190627.3025-1-alifm@linux.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: mgHXy3PYj21fuBkS4N-dftVNQoXdefRq X-Proofpoint-GUID: mgHXy3PYj21fuBkS4N-dftVNQoXdefRq X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-03-28_09,2025-03-27_02,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 phishscore=0 clxscore=1015 mlxscore=0 suspectscore=0 mlxlogscore=999 impostorscore=0 spamscore=0 bulkscore=0 malwarescore=0 adultscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2503280129 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.156.1; envelope-from=alifm@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ibm.com) X-ZM-MESSAGEID: 1743188881852019000 Content-Type: text/plain; charset="utf-8" Use the QEMU PCI MMIO functions to read/write to NVMe registers, rather than directly accessing them. Signed-off-by: Farhan Ali Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Stefan Hajnoczi --- block/nvme.c | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index bbf7c23dcd..ea932609e6 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -23,6 +23,7 @@ #include "qemu/cutils.h" #include "qemu/option.h" #include "qemu/memalign.h" +#include "qemu/pci-mmio.h" #include "qemu/vfio-helpers.h" #include "block/block-io.h" #include "block/block_int.h" @@ -60,7 +61,7 @@ typedef struct { uint8_t *queue; uint64_t iova; /* Hardware MMIO register */ - volatile uint32_t *doorbell; + uint32_t *doorbell; } NVMeQueue; =20 typedef struct { @@ -100,7 +101,7 @@ struct BDRVNVMeState { QEMUVFIOState *vfio; void *bar0_wo_map; /* Memory mapped registers */ - volatile struct { + struct { uint32_t sq_tail; uint32_t cq_head; } *doorbells; @@ -292,7 +293,7 @@ static void nvme_kick(NVMeQueuePair *q) assert(!(q->sq.tail & 0xFF00)); /* Fence the write to submission queue entry before notifying the devi= ce. */ smp_wmb(); - *q->sq.doorbell =3D cpu_to_le32(q->sq.tail); + qemu_pci_mmio_write_32(q->sq.doorbell, q->sq.tail); q->inflight +=3D q->need_kick; q->need_kick =3D 0; } @@ -441,7 +442,7 @@ static bool nvme_process_completion(NVMeQueuePair *q) if (progress) { /* Notify the device so it can post more completions. */ smp_mb_release(); - *q->cq.doorbell =3D cpu_to_le32(q->cq.head); + qemu_pci_mmio_write_32(q->cq.doorbell, q->cq.head); nvme_wake_free_req_locked(q); } =20 @@ -460,7 +461,7 @@ static void nvme_process_completion_bh(void *opaque) * so notify the device that it has space to fill in more completions = now. */ smp_mb_release(); - *q->cq.doorbell =3D cpu_to_le32(q->cq.head); + qemu_pci_mmio_write_32(q->cq.doorbell, q->cq.head); nvme_wake_free_req_locked(q); =20 nvme_process_completion(q); @@ -749,9 +750,10 @@ static int nvme_init(BlockDriverState *bs, const char = *device, int namespace, int ret; uint64_t cap; uint32_t ver; + uint32_t cc; uint64_t timeout_ms; uint64_t deadline, now; - volatile NvmeBar *regs =3D NULL; + NvmeBar *regs =3D NULL; =20 qemu_co_mutex_init(&s->dma_map_lock); qemu_co_queue_init(&s->dma_flush_queue); @@ -779,7 +781,7 @@ static int nvme_init(BlockDriverState *bs, const char *= device, int namespace, /* Perform initialize sequence as described in NVMe spec "7.6.1 * Initialization". */ =20 - cap =3D le64_to_cpu(regs->cap); + cap =3D qemu_pci_mmio_read_64(®s->cap); trace_nvme_controller_capability_raw(cap); trace_nvme_controller_capability("Maximum Queue Entries Supported", 1 + NVME_CAP_MQES(cap)); @@ -805,16 +807,17 @@ static int nvme_init(BlockDriverState *bs, const char= *device, int namespace, bs->bl.request_alignment =3D s->page_size; timeout_ms =3D MIN(500 * NVME_CAP_TO(cap), 30000); =20 - ver =3D le32_to_cpu(regs->vs); + ver =3D qemu_pci_mmio_read_32(®s->vs); trace_nvme_controller_spec_version(extract32(ver, 16, 16), extract32(ver, 8, 8), extract32(ver, 0, 8)); =20 /* Reset device to get a clean state. */ - regs->cc =3D cpu_to_le32(le32_to_cpu(regs->cc) & 0xFE); + cc =3D qemu_pci_mmio_read_32(®s->cc); + qemu_pci_mmio_write_32(®s->cc, (cc & 0xFE)); /* Wait for CSTS.RDY =3D 0. */ deadline =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + timeout_ms * SCA= LE_MS; - while (NVME_CSTS_RDY(le32_to_cpu(regs->csts))) { + while (NVME_CSTS_RDY(qemu_pci_mmio_read_32(®s->csts))) { if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) { error_setg(errp, "Timeout while waiting for device to reset (%" PRId64 " ms)", @@ -843,19 +846,21 @@ static int nvme_init(BlockDriverState *bs, const char= *device, int namespace, s->queues[INDEX_ADMIN] =3D q; s->queue_count =3D 1; QEMU_BUILD_BUG_ON((NVME_QUEUE_SIZE - 1) & 0xF000); - regs->aqa =3D cpu_to_le32(((NVME_QUEUE_SIZE - 1) << AQA_ACQS_SHIFT) | - ((NVME_QUEUE_SIZE - 1) << AQA_ASQS_SHIFT)); - regs->asq =3D cpu_to_le64(q->sq.iova); - regs->acq =3D cpu_to_le64(q->cq.iova); + qemu_pci_mmio_write_32(®s->aqa, + ((NVME_QUEUE_SIZE - 1) << AQA_ACQS_SHIFT) | + ((NVME_QUEUE_SIZE - 1) << AQA_ASQS_SHIFT)); + qemu_pci_mmio_write_64(®s->asq, q->sq.iova); + qemu_pci_mmio_write_64(®s->acq, q->cq.iova); =20 /* After setting up all control registers we can enable device now. */ - regs->cc =3D cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << CC_IOCQES_SHIF= T) | + qemu_pci_mmio_write_32(®s->cc, + (ctz32(NVME_CQ_ENTRY_BYTES) << CC_IOCQES_SHIFT)= | (ctz32(NVME_SQ_ENTRY_BYTES) << CC_IOSQES_SHIFT)= | CC_EN_MASK); /* Wait for CSTS.RDY =3D 1. */ now =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME); deadline =3D now + timeout_ms * SCALE_MS; - while (!NVME_CSTS_RDY(le32_to_cpu(regs->csts))) { + while (!NVME_CSTS_RDY(qemu_pci_mmio_read_32(®s->csts))) { if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) { error_setg(errp, "Timeout while waiting for device to start (%" PRId64 " ms)", --=20 2.43.0