From nobody Fri Dec 19 04:38:28 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=reject dis=none) header.from=linux.ibm.com ARC-Seal: i=1; a=rsa-sha256; t=1744666640; cv=none; d=zohomail.com; s=zohoarc; b=bnyBkQTtp+5vLUJX9hALZxqFYP0KVhy4r1DbJbp+jRq4XIvKR0mBC6l6UKsMIG4TzMSYSjhehIHgiDNjn1NQC4LR4nyGWAbssAlbWlDX5iChXE5xtaAWYDa3YURi5wb2s24TKKIqv/YSCuMCuEypWU0JaHNgRMGAFif/jCvDWNo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744666640; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=XbxOmOxaCGJ2wrgJRqkKjhBcMBVejLmaBs+JzNXKqsI=; b=LCJoeuQI2LqgcKvVH88mJ5M+oOQ0ay8hxL3/AfSBlmGA0UkmZnl/28n2+5Y+WfUBaB1bJpifWxdyugyDnLC4sCl9gzqBwYdvlp4cltfG0p8Vk62/GJVNzwaeULASIt5paDMsHN+yMXttHc0QoNA+WPfVxHGxBShoP6PyQE02pYY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744666640785387.98607170463356; Mon, 14 Apr 2025 14:37:20 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4RTw-00080o-41; Mon, 14 Apr 2025 17:36:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4RTn-0007vC-SE; Mon, 14 Apr 2025 17:36:36 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4RTk-0003Gi-QJ; Mon, 14 Apr 2025 17:36:34 -0400 Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 53EIZ6rI023528; Mon, 14 Apr 2025 21:36:24 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 460q7x568m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 14 Apr 2025 21:36:24 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 53EHkBv9016746; Mon, 14 Apr 2025 21:36:23 GMT Received: from smtprelay03.wdc07v.mail.ibm.com ([172.16.1.70]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 460571ys32-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 14 Apr 2025 21:36:23 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay03.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 53ELaJsG28574366 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Apr 2025 21:36:19 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DBB1558054; Mon, 14 Apr 2025 21:36:21 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6ADF95803F; Mon, 14 Apr 2025 21:36:20 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.251.75]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Mon, 14 Apr 2025 21:36:20 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=XbxOmOxaCGJ2wrgJR qkKjhBcMBVejLmaBs+JzNXKqsI=; b=Lxee5wajJoNNnU1X/Qw8+u+5kwP2sb0SJ w9X4f+kZc0Ekb+N/vafXAUpLFEz1dD2OeKwQr1YRtgj3F1AryNBuE9eMeN6Gh/gp Sz6soeM76siKuNKB69Kf5yKSnwSo9KFnRqKrOWPei8MyN7dGP1SaIhQhPyJ0409l 1TwNb9e45ZhUZKdWuS+W80DWuuXf58P4TJ9cKyTOCXIG9QUrJPYD1wkdfh0EyhHw hqFletXeEWXhMPBVzd9h/GPLCTkChsEtUBphbOZmy2pVD6cQiBO0ixENyFmlAzxM zqGa1mntBTuwXmSbcLnfw39ud9W14AX7uy0nLa9K7bMPzMtYPuKEw== From: Farhan Ali To: qemu-devel@nongnu.org Cc: qemu-s390x@nongnu.org, alifm@linux.ibm.com, mjrosato@linux.ibm.com, schnelle@linux.ibm.com, stefanha@redhat.com, philmd@linaro.org, kwolf@redhat.com, hreitz@redhat.com, thuth@redhat.com, fam@euphon.net, alex.williamson@redhat.com Subject: [PATCH v4 1/3] util: Add functions for s390x mmio read/write Date: Mon, 14 Apr 2025 14:36:14 -0700 Message-ID: <20250414213616.2675-2-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250414213616.2675-1-alifm@linux.ibm.com> References: <20250414213616.2675-1-alifm@linux.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: uS6Hs53Qwd3AziXq0WnynD4Ge3OdycjY X-Proofpoint-GUID: uS6Hs53Qwd3AziXq0WnynD4Ge3OdycjY X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-04-14_08,2025-04-10_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 adultscore=0 phishscore=0 clxscore=1015 bulkscore=0 mlxscore=0 lowpriorityscore=0 mlxlogscore=625 suspectscore=0 malwarescore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2504140157 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.158.5; envelope-from=alifm@linux.ibm.com; helo=mx0b-001b2d01.pphosted.com X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ibm.com) X-ZM-MESSAGEID: 1744666642958019100 Content-Type: text/plain; charset="utf-8" Starting with z15 (or newer) we can execute mmio instructions from userspace. On older platforms where we don't have these instructions available we can fallback to using system calls to access the PCI mapped resources. This patch adds helper functions for mmio reads and writes for s390x. Reviewed-by: Stefan Hajnoczi Reviewed-by: Niklas Schnelle Signed-off-by: Farhan Ali --- include/qemu/s390x_pci_mmio.h | 24 ++++++ util/meson.build | 2 + util/s390x_pci_mmio.c | 148 ++++++++++++++++++++++++++++++++++ 3 files changed, 174 insertions(+) create mode 100644 include/qemu/s390x_pci_mmio.h create mode 100644 util/s390x_pci_mmio.c diff --git a/include/qemu/s390x_pci_mmio.h b/include/qemu/s390x_pci_mmio.h new file mode 100644 index 0000000000..c5f63ecefa --- /dev/null +++ b/include/qemu/s390x_pci_mmio.h @@ -0,0 +1,24 @@ +/* + * s390x PCI MMIO definitions + * + * Copyright 2025 IBM Corp. + * Author(s): Farhan Ali + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ +#ifndef S390X_PCI_MMIO_H +#define S390X_PCI_MMIO_H + +#ifdef __s390x__ +uint8_t s390x_pci_mmio_read_8(const void *ioaddr); +uint16_t s390x_pci_mmio_read_16(const void *ioaddr); +uint32_t s390x_pci_mmio_read_32(const void *ioaddr); +uint64_t s390x_pci_mmio_read_64(const void *ioaddr); + +void s390x_pci_mmio_write_8(void *ioaddr, uint8_t val); +void s390x_pci_mmio_write_16(void *ioaddr, uint16_t val); +void s390x_pci_mmio_write_32(void *ioaddr, uint32_t val); +void s390x_pci_mmio_write_64(void *ioaddr, uint64_t val); +#endif /* __s390x__ */ + +#endif /* S390X_PCI_MMIO_H */ diff --git a/util/meson.build b/util/meson.build index 780b5977a8..acb21592f9 100644 --- a/util/meson.build +++ b/util/meson.build @@ -131,4 +131,6 @@ elif cpu in ['ppc', 'ppc64'] util_ss.add(files('cpuinfo-ppc.c')) elif cpu in ['riscv32', 'riscv64'] util_ss.add(files('cpuinfo-riscv.c')) +elif cpu =3D=3D 's390x' + util_ss.add(files('s390x_pci_mmio.c')) endif diff --git a/util/s390x_pci_mmio.c b/util/s390x_pci_mmio.c new file mode 100644 index 0000000000..820458a026 --- /dev/null +++ b/util/s390x_pci_mmio.c @@ -0,0 +1,148 @@ +/* + * s390x PCI MMIO definitions + * + * Copyright 2025 IBM Corp. + * Author(s): Farhan Ali + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include +#include +#include "qemu/s390x_pci_mmio.h" +#include "elf.h" + +union register_pair { + unsigned __int128 pair; + struct { + uint64_t even; + uint64_t odd; + }; +}; + +static bool is_mio_supported; + +static __attribute__((constructor)) void check_is_mio_supported(void) +{ + is_mio_supported =3D !!(qemu_getauxval(AT_HWCAP) & HWCAP_S390_PCI_MIO); +} + +static uint64_t s390x_pcilgi(const void *ioaddr, size_t len) +{ + union register_pair ioaddr_len =3D { .even =3D (uint64_t)ioaddr, + .odd =3D len }; + uint64_t val; + int cc; + + asm volatile( + /* pcilgi */ + ".insn rre,0xb9d60000,%[val],%[ioaddr_len]\n" + "ipm %[cc]\n" + "srl %[cc],28\n" + : [cc] "=3Dd"(cc), [val] "=3Dd"(val), + [ioaddr_len] "+&d"(ioaddr_len.pair) :: "cc"); + + if (cc) { + val =3D -1ULL; + } + + return val; +} + +static void s390x_pcistgi(void *ioaddr, uint64_t val, size_t len) +{ + union register_pair ioaddr_len =3D {.even =3D (uint64_t)ioaddr, .odd = =3D len}; + + asm volatile ( + /* pcistgi */ + ".insn rre,0xb9d40000,%[val],%[ioaddr_len]\n" + : [ioaddr_len] "+&d" (ioaddr_len.pair) + : [val] "d" (val) + : "cc", "memory"); +} + +uint8_t s390x_pci_mmio_read_8(const void *ioaddr) +{ + uint8_t val =3D 0; + + if (is_mio_supported) { + val =3D s390x_pcilgi(ioaddr, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_read, ioaddr, &val, sizeof(val)); + } + return val; +} + +uint16_t s390x_pci_mmio_read_16(const void *ioaddr) +{ + uint16_t val =3D 0; + + if (is_mio_supported) { + val =3D s390x_pcilgi(ioaddr, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_read, ioaddr, &val, sizeof(val)); + } + return val; +} + +uint32_t s390x_pci_mmio_read_32(const void *ioaddr) +{ + uint32_t val =3D 0; + + if (is_mio_supported) { + val =3D s390x_pcilgi(ioaddr, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_read, ioaddr, &val, sizeof(val)); + } + return val; +} + +uint64_t s390x_pci_mmio_read_64(const void *ioaddr) +{ + uint64_t val =3D 0; + + if (is_mio_supported) { + val =3D s390x_pcilgi(ioaddr, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_read, ioaddr, &val, sizeof(val)); + } + return val; +} + +void s390x_pci_mmio_write_8(void *ioaddr, uint8_t val) +{ + if (is_mio_supported) { + s390x_pcistgi(ioaddr, val, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_write, ioaddr, &val, sizeof(val)); + } +} + +void s390x_pci_mmio_write_16(void *ioaddr, uint16_t val) +{ + if (is_mio_supported) { + s390x_pcistgi(ioaddr, val, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_write, ioaddr, &val, sizeof(val)); + } +} + +void s390x_pci_mmio_write_32(void *ioaddr, uint32_t val) +{ + if (is_mio_supported) { + s390x_pcistgi(ioaddr, val, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_write, ioaddr, &val, sizeof(val)); + } +} + +void s390x_pci_mmio_write_64(void *ioaddr, uint64_t val) +{ + if (is_mio_supported) { + s390x_pcistgi(ioaddr, val, sizeof(val)); + } else { + syscall(__NR_s390_pci_mmio_write, ioaddr, &val, sizeof(val)); + } +} + --=20 2.43.0 From nobody Fri Dec 19 04:38:28 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=reject dis=none) header.from=linux.ibm.com ARC-Seal: i=1; a=rsa-sha256; t=1744666634; cv=none; d=zohomail.com; s=zohoarc; b=FquURYPT3BzekEWRxzgErf9XatzZ/oSWn/ROhtF1kdLEeSNa5+CNiKwlsWywM76LOL835pbpWyDcrjhVJZAXWUlb5nAwAmDyLPOO7P7cygQFDyfPxCjgxUDDYwroScuXVOUA4Yl1tVVJMK5zgA6Vrg5oifo5fzwuFW16Gchgsf4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744666634; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=TvFELmmpf/sdEtzfjCrR1PgRLUIupH4ECKzApQitp2s=; b=TEA/BBfDkNZiQwlmrbFKr7caa/MijibKe6W0jTQWe61ogd8DP5KdYvVlkIxEUlBFD2EEXf2sn/d47uQAyvJMAlZOJjdWC5ZXVJr2a1w5H73LBGrF8A30Ui+nuRqn8Fj/z6EpZh7bgx4iECrb1Bvd6OyvbCa7R4izVLZeuVrqQUY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744666634317530.4225413786811; Mon, 14 Apr 2025 14:37:14 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4RTs-0007xu-P1; Mon, 14 Apr 2025 17:36:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4RTl-0007un-S4; Mon, 14 Apr 2025 17:36:34 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4RTj-0003Go-Ev; Mon, 14 Apr 2025 17:36:33 -0400 Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 53EIXlsv026521; Mon, 14 Apr 2025 21:36:26 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 46109f36db-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 14 Apr 2025 21:36:26 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 53ELBuic017170; Mon, 14 Apr 2025 21:36:24 GMT Received: from smtprelay05.wdc07v.mail.ibm.com ([172.16.1.72]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 46040kqypd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 14 Apr 2025 21:36:24 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay05.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 53ELaNiD27853392 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Apr 2025 21:36:23 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9F7505804E; Mon, 14 Apr 2025 21:36:23 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 20AE15803F; Mon, 14 Apr 2025 21:36:22 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.251.75]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Mon, 14 Apr 2025 21:36:21 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=TvFELmmpf/sdEtzfj CrR1PgRLUIupH4ECKzApQitp2s=; b=aSwVEMgH+aY4R/t89/EwkV+d+dYijKrZw G01GUVdjWN72k4qJXZ64pWe0dJYDIbZjt/hYoBq3bTW+QsiUhqZZlNCaNQrjiNRi L27Hu/Q6Sx+EvtN2aJA5D85RiLMGwzWpHQuuzy0lJ/Q+5w+RzzZB5BmU1y/OKbWc 8sqF7Oi5Ede3N1E5ssmpHMQunwwh7n6M4uflQEmrHTUGasWOIdP/pWfTMpcCwHbJ OfVB2zzHeDAqODL9guMY7mQkUCXNdU9+9E/DTUyr8rCKodBBOw6a8DWi26by5Ro8 nENbMmTWm5ruuuPZWshwcPJwHD/UyqW9NUeXk0qQmuea6PM+4kdUA== From: Farhan Ali To: qemu-devel@nongnu.org Cc: qemu-s390x@nongnu.org, alifm@linux.ibm.com, mjrosato@linux.ibm.com, schnelle@linux.ibm.com, stefanha@redhat.com, philmd@linaro.org, kwolf@redhat.com, hreitz@redhat.com, thuth@redhat.com, fam@euphon.net, alex.williamson@redhat.com Subject: [PATCH v4 2/3] include: Add a header to define host PCI MMIO functions Date: Mon, 14 Apr 2025 14:36:15 -0700 Message-ID: <20250414213616.2675-3-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250414213616.2675-1-alifm@linux.ibm.com> References: <20250414213616.2675-1-alifm@linux.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: qF3nRASHSHWyMIfZCTXG-YHhPBtzWlqM X-Proofpoint-ORIG-GUID: qF3nRASHSHWyMIfZCTXG-YHhPBtzWlqM X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-04-14_08,2025-04-10_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 suspectscore=0 phishscore=0 bulkscore=0 adultscore=0 priorityscore=1501 mlxlogscore=853 malwarescore=0 clxscore=1015 spamscore=0 mlxscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2504140157 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.156.1; envelope-from=alifm@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ibm.com) X-ZM-MESSAGEID: 1744666637823019000 Content-Type: text/plain; charset="utf-8" Add a generic API for host PCI MMIO reads/writes (e.g. Linux VFIO BAR accesses). The functions access little endian memory and returns the result in host cpu endianness. Signed-off-by: Farhan Ali --- include/qemu/host-pci-mmio.h | 141 +++++++++++++++++++++++++++++++++++ 1 file changed, 141 insertions(+) create mode 100644 include/qemu/host-pci-mmio.h diff --git a/include/qemu/host-pci-mmio.h b/include/qemu/host-pci-mmio.h new file mode 100644 index 0000000000..de17d67e3a --- /dev/null +++ b/include/qemu/host-pci-mmio.h @@ -0,0 +1,141 @@ +/* + * API for host PCI MMIO accesses (e.g. Linux VFIO BARs) + * + * Copyright 2025 IBM Corp. + * Author(s): Farhan Ali + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef HOST_PCI_MMIO_H +#define HOST_PCI_MMIO_H + +#include "qemu/bswap.h" +#include "qemu/s390x_pci_mmio.h" + + +static inline uint8_t host_pci_ldub_p(const void *ioaddr) +{ + uint8_t ret =3D 0; +#ifdef __s390x__ + ret =3D s390x_pci_mmio_read_8(ioaddr); +#else + ret =3D ldub_he_p(ioaddr); +#endif + + return ret; +} + +static inline uint16_t host_pci_lduw_le_p(const void *ioaddr) +{ + uint16_t ret =3D 0; +#ifdef __s390x__ + ret =3D le16_to_cpu(s390x_pci_mmio_read_16(ioaddr)); +#else + ret =3D lduw_le_p(ioaddr); +#endif + + return ret; +} + +static inline uint32_t host_pci_ldl_le_p(const void *ioaddr) +{ + uint32_t ret =3D 0; +#ifdef __s390x__ + ret =3D le32_to_cpu(s390x_pci_mmio_read_32(ioaddr)); +#else + ret =3D (uint32_t)ldl_le_p(ioaddr); +#endif + + return ret; +} + +static inline uint64_t host_pci_ldq_le_p(const void *ioaddr) +{ + uint64_t ret =3D 0; +#ifdef __s390x__ + ret =3D le64_to_cpu(s390x_pci_mmio_read_64(ioaddr)); +#else + ret =3D ldq_le_p(ioaddr); +#endif + + return ret; +} + +static inline void host_pci_stb_le_p(void *ioaddr, uint8_t val) +{ + +#ifdef __s390x__ + s390x_pci_mmio_write_8(ioaddr, val); +#else + stb_p(ioaddr, val); +#endif +} + +static inline void host_pci_stw_le_p(void *ioaddr, uint16_t val) +{ + +#ifdef __s390x__ + s390x_pci_mmio_write_16(ioaddr, cpu_to_le16(val)); +#else + stw_le_p(ioaddr, val); +#endif +} + +static inline void host_pci_stl_le_p(void *ioaddr, uint32_t val) +{ + +#ifdef __s390x__ + s390x_pci_mmio_write_32(ioaddr, cpu_to_le32(val)); +#else + stl_le_p(ioaddr, val); +#endif +} + +static inline void host_pci_stq_le_p(void *ioaddr, uint64_t val) +{ + +#ifdef __s390x__ + s390x_pci_mmio_write_64(ioaddr, cpu_to_le64(val)); +#else + stq_le_p(ioaddr, val); +#endif +} + +static inline uint64_t host_pci_ldn_le_p(const void *ioaddr, int sz) +{ + switch (sz) { + case 1: + return host_pci_ldub_p(ioaddr); + case 2: + return host_pci_lduw_le_p(ioaddr); + case 4: + return host_pci_ldl_le_p(ioaddr); + case 8: + return host_pci_ldq_le_p(ioaddr); + default: + g_assert_not_reached(); + } +} + +static inline void host_pci_stn_le_p(void *ioaddr, int sz, uint64_t v) +{ + switch (sz) { + case 1: + host_pci_stb_le_p(ioaddr, v); + break; + case 2: + host_pci_stw_le_p(ioaddr, v); + break; + case 4: + host_pci_stl_le_p(ioaddr, v); + break; + case 8: + host_pci_stq_le_p(ioaddr, v); + break; + default: + g_assert_not_reached(); + } +} + +#endif --=20 2.43.0 From nobody Fri Dec 19 04:38:28 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=reject dis=none) header.from=linux.ibm.com ARC-Seal: i=1; a=rsa-sha256; t=1744666667; cv=none; d=zohomail.com; s=zohoarc; b=Xb3C/TfLmg765ys5nm0BKrEcAKJDySFe+1atpL6ErWb/ZwZG+fr/Ebf9HU/7h6bfk6hAUMyB0EzggfHJhIkiWsc7j4pnHLS2xuMitVU64Wl/LGDpVysc8s14TCZ+UxSoXlzNkLHKRR1bExynimeL3E2Yq4M4Z97HwgiUXn8l0dY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744666667; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=npw1c10lIHO/vnFKDXGoskioVP246OaHi7szH0FBSyI=; b=Bp4Hs211jPHIlv1J/oz7Hrf6rMUcF71m+KjEJxheXy91hJyDJ6pqnMsnICXpmBOLB4puPw1p8g4lzn1nASLgo5Q2pWK9h8vcStclvasWxAJzLJchmhHZQKK2qwOg/Y0bIJiFr6S63z8l4CZqJc06qKoue5/PHPNHal/ojE0BDS4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 174466666754237.984739935953144; Mon, 14 Apr 2025 14:37:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4RTv-0007zv-CT; Mon, 14 Apr 2025 17:36:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4RTr-0007xm-JK; Mon, 14 Apr 2025 17:36:39 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4RTo-0003I0-Jn; Mon, 14 Apr 2025 17:36:38 -0400 Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 53EGlkXb016142; Mon, 14 Apr 2025 21:36:28 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 460q7x568v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 14 Apr 2025 21:36:27 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 53EHMg7R010467; Mon, 14 Apr 2025 21:36:26 GMT Received: from smtprelay07.wdc07v.mail.ibm.com ([172.16.1.74]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4604qjyu7v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 14 Apr 2025 21:36:26 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay07.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 53ELaP6P21037792 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Apr 2025 21:36:25 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5A9A758054; Mon, 14 Apr 2025 21:36:25 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D22E45803F; Mon, 14 Apr 2025 21:36:23 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.251.75]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Mon, 14 Apr 2025 21:36:23 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=npw1c1 0lIHO/vnFKDXGoskioVP246OaHi7szH0FBSyI=; b=R/Iftk15tDntqQAOsUzr4b hLX8VMklzJMBAk0zjWuNn1DXIIGgt9i2ZLKmJPK4OevT1zwfzjtnB6ad8YPUqP7T 5Z32O29rUAlY/gGiRg0u7+vKuaTyWKQkseQDSEHr/8FXSjS77YJxwStW0jMG2S6/ Z7YyoqNbRG74AGJ+GxKgI9wv5sCHaRuNkE+b+ua46V8sxcJU7/tr8i2PO6OZ6jPE nXUMJOqeA9u/TERt97dlfQhiDniAKOj+ZKClyfHbz0CXjDzQ56g2JS5p96a1iGi6 KYlR4Pn2drfNsEpKGSKcv++fTo+99aVQnVml2AkGU1+1xGyygVpw/YQjQmn9wOtg == From: Farhan Ali To: qemu-devel@nongnu.org Cc: qemu-s390x@nongnu.org, alifm@linux.ibm.com, mjrosato@linux.ibm.com, schnelle@linux.ibm.com, stefanha@redhat.com, philmd@linaro.org, kwolf@redhat.com, hreitz@redhat.com, thuth@redhat.com, fam@euphon.net, alex.williamson@redhat.com Subject: [PATCH v4 3/3] block/nvme: Use host PCI MMIO API Date: Mon, 14 Apr 2025 14:36:16 -0700 Message-ID: <20250414213616.2675-4-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250414213616.2675-1-alifm@linux.ibm.com> References: <20250414213616.2675-1-alifm@linux.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: nVkDQiQ97aSeq9v_NSVkC3aZzF3xqpHI X-Proofpoint-GUID: nVkDQiQ97aSeq9v_NSVkC3aZzF3xqpHI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-04-14_08,2025-04-10_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 adultscore=0 phishscore=0 clxscore=1015 bulkscore=0 mlxscore=0 lowpriorityscore=0 mlxlogscore=999 suspectscore=0 malwarescore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2504140157 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.158.5; envelope-from=alifm@linux.ibm.com; helo=mx0b-001b2d01.pphosted.com X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ibm.com) X-ZM-MESSAGEID: 1744666668636019000 Use the host PCI MMIO functions to read/write to NVMe registers, rather than directly accessing them. Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Stefan Hajnoczi Signed-off-by: Farhan Ali --- block/nvme.c | 41 +++++++++++++++++++++++------------------ 1 file changed, 23 insertions(+), 18 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index bbf7c23dcd..8df53ee4ca 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -18,6 +18,7 @@ #include "qobject/qstring.h" #include "qemu/defer-call.h" #include "qemu/error-report.h" +#include "qemu/host-pci-mmio.h" #include "qemu/main-loop.h" #include "qemu/module.h" #include "qemu/cutils.h" @@ -60,7 +61,7 @@ typedef struct { uint8_t *queue; uint64_t iova; /* Hardware MMIO register */ - volatile uint32_t *doorbell; + uint32_t *doorbell; } NVMeQueue; =20 typedef struct { @@ -100,7 +101,7 @@ struct BDRVNVMeState { QEMUVFIOState *vfio; void *bar0_wo_map; /* Memory mapped registers */ - volatile struct { + struct { uint32_t sq_tail; uint32_t cq_head; } *doorbells; @@ -292,7 +293,7 @@ static void nvme_kick(NVMeQueuePair *q) assert(!(q->sq.tail & 0xFF00)); /* Fence the write to submission queue entry before notifying the devi= ce. */ smp_wmb(); - *q->sq.doorbell =3D cpu_to_le32(q->sq.tail); + host_pci_stl_le_p(q->sq.doorbell, q->sq.tail); q->inflight +=3D q->need_kick; q->need_kick =3D 0; } @@ -441,7 +442,7 @@ static bool nvme_process_completion(NVMeQueuePair *q) if (progress) { /* Notify the device so it can post more completions. */ smp_mb_release(); - *q->cq.doorbell =3D cpu_to_le32(q->cq.head); + host_pci_stl_le_p(q->cq.doorbell, q->cq.head); nvme_wake_free_req_locked(q); } =20 @@ -460,7 +461,7 @@ static void nvme_process_completion_bh(void *opaque) * so notify the device that it has space to fill in more completions = now. */ smp_mb_release(); - *q->cq.doorbell =3D cpu_to_le32(q->cq.head); + host_pci_stl_le_p(q->cq.doorbell, q->cq.head); nvme_wake_free_req_locked(q); =20 nvme_process_completion(q); @@ -749,9 +750,10 @@ static int nvme_init(BlockDriverState *bs, const char = *device, int namespace, int ret; uint64_t cap; uint32_t ver; + uint32_t cc; uint64_t timeout_ms; uint64_t deadline, now; - volatile NvmeBar *regs =3D NULL; + NvmeBar *regs =3D NULL; =20 qemu_co_mutex_init(&s->dma_map_lock); qemu_co_queue_init(&s->dma_flush_queue); @@ -779,7 +781,7 @@ static int nvme_init(BlockDriverState *bs, const char *= device, int namespace, /* Perform initialize sequence as described in NVMe spec "7.6.1 * Initialization". */ =20 - cap =3D le64_to_cpu(regs->cap); + cap =3D host_pci_ldq_le_p(®s->cap); trace_nvme_controller_capability_raw(cap); trace_nvme_controller_capability("Maximum Queue Entries Supported", 1 + NVME_CAP_MQES(cap)); @@ -805,16 +807,17 @@ static int nvme_init(BlockDriverState *bs, const char= *device, int namespace, bs->bl.request_alignment =3D s->page_size; timeout_ms =3D MIN(500 * NVME_CAP_TO(cap), 30000); =20 - ver =3D le32_to_cpu(regs->vs); + ver =3D host_pci_ldl_le_p(®s->vs); trace_nvme_controller_spec_version(extract32(ver, 16, 16), extract32(ver, 8, 8), extract32(ver, 0, 8)); =20 /* Reset device to get a clean state. */ - regs->cc =3D cpu_to_le32(le32_to_cpu(regs->cc) & 0xFE); + cc =3D host_pci_ldl_le_p(®s->cc); + host_pci_stl_le_p(®s->cc, cc & 0xFE); /* Wait for CSTS.RDY =3D 0. */ deadline =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + timeout_ms * SCA= LE_MS; - while (NVME_CSTS_RDY(le32_to_cpu(regs->csts))) { + while (NVME_CSTS_RDY(host_pci_ldl_le_p(®s->csts))) { if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) { error_setg(errp, "Timeout while waiting for device to reset (%" PRId64 " ms)", @@ -843,19 +846,21 @@ static int nvme_init(BlockDriverState *bs, const char= *device, int namespace, s->queues[INDEX_ADMIN] =3D q; s->queue_count =3D 1; QEMU_BUILD_BUG_ON((NVME_QUEUE_SIZE - 1) & 0xF000); - regs->aqa =3D cpu_to_le32(((NVME_QUEUE_SIZE - 1) << AQA_ACQS_SHIFT) | - ((NVME_QUEUE_SIZE - 1) << AQA_ASQS_SHIFT)); - regs->asq =3D cpu_to_le64(q->sq.iova); - regs->acq =3D cpu_to_le64(q->cq.iova); + host_pci_stl_le_p(®s->aqa, + ((NVME_QUEUE_SIZE - 1) << AQA_ACQS_SHIFT) | + ((NVME_QUEUE_SIZE - 1) << AQA_ASQS_SHIFT)); + host_pci_stq_le_p(®s->asq, q->sq.iova); + host_pci_stq_le_p(®s->acq, q->cq.iova); =20 /* After setting up all control registers we can enable device now. */ - regs->cc =3D cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << CC_IOCQES_SHIF= T) | - (ctz32(NVME_SQ_ENTRY_BYTES) << CC_IOSQES_SHIFT)= | - CC_EN_MASK); + host_pci_stl_le_p(®s->cc, + (ctz32(NVME_CQ_ENTRY_BYTES) << CC_IOCQES_SHIFT) | + (ctz32(NVME_SQ_ENTRY_BYTES) << CC_IOSQES_SHIFT) | + CC_EN_MASK); /* Wait for CSTS.RDY =3D 1. */ now =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME); deadline =3D now + timeout_ms * SCALE_MS; - while (!NVME_CSTS_RDY(le32_to_cpu(regs->csts))) { + while (!NVME_CSTS_RDY(host_pci_ldl_le_p(®s->csts))) { if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) { error_setg(errp, "Timeout while waiting for device to start (%" PRId64 " ms)", --=20 2.43.0