From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255111; cv=pass; d=zohomail.com; s=zohoarc; b=YlW05/f3HktKZK5GXVUEzQk/XMQRCYT2A532CqLP3qSesE7n/6/2/fD0noFi5ApHfn+eUnSyRS/yhe8mT6uq5knLUvhhDhf2X7OuOyV03MWNGs1Na+VLfblPEjSAu0lPov2bn6aGmMEo2jnxiaQ7xc5KVGkiWxC69dO77QOBexI= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255111; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=2Kqsm7S7nVPh0w+Hj7C+jzcdhYJsfDn8jkSJyyG3nhE=; b=m7fJR3vO0QADgKkCaR7ZJgI3t3YVdfGLfhOgtw+kjfUNoYlNBCWdIidLP7Ih8KS3ZZMlEf+TG+EzvOWkpxWlgRoU1mUtS5K4xz8CfzFHoQOcSmZakYA4q6Nq3GXubvwsY6tWK2bDxwJoRtBSZ3314r2Cj4nhLINo2AAuXnCXKD4= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255111665830.2953983667227; Fri, 6 Jun 2025 17:11:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9X-0002w2-Ul; Fri, 06 Jun 2025 20:11:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9V-0002v3-Ob for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:13 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9P-0006NF-7H for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:13 -0400 Received: from pps.filterd (m0127840.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556K9XAd027707; Fri, 6 Jun 2025 17:11:03 -0700 Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11on2127.outbound.protection.outlook.com [40.107.236.127]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 471g813xsu-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:03 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:00 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:00 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=2Kqsm7S7nVPh0w+Hj7C+jzcdhYJsfDn8jkSJyyG3n hE=; b=wSnUVDqKhm5QWLDatwrSLhcXKHFLZveq3Z5sk43eYmStb9k9nIouO4ASe AF2aElOWZqHIYdsY8SlNdEPmEaW278xX01YbdvpL4ZeKubC3AvAUZPT2S0XQx+mt JvMAOfoHfCW1tydEvB4Ip7x6DEr13lWuK5llbn6K1G6b2hPIY1w7JoUgJ9VRnn+Z 6rNAVS1IjJ1A3TYHxOmxfSSlz+MfbO1Zl1Q05il4ZYrHxVf2yTirgpTTrGaRKJWn fXCRgMpTBnptZNKbNGKJ5JztUfHdLQKQ7ex24a14QuJ5HEldox5u+Gq/gjnpevlT OHLRaqgDbU5uA2jWNYGrs1ZGZ8mtQ== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=X8P7fLS2VQJojeKKotDehRY9njVs/L3XC43pg47w0eGp70bx0/SY6QYt/6U93S+h5zRcGp+HJB8Qn88OwtxrkvJITd/zYmvwkTW7CoT6ggYhKlTxUK+/wnK3GObdnZyTFXze71npLmaIJh2XmfDfNH/0YL2f1KKewS0zjxx0rJZprq9Ag/gTR+rU5k94XXvLgQ6s/lf02gL5TNKkLPljk9qOjnXDVpTLl9gqtX6nixiRDQbReZjPDriV5HoQtHLDZV7ygO9yl5C0g51fghjMz2qeH9GgIc6KE0/aWj3SQLr+isUne2/GpcjorVuskAUImj8fhirxAorUtF2sp+JDsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2Kqsm7S7nVPh0w+Hj7C+jzcdhYJsfDn8jkSJyyG3nhE=; b=b9ZMwQOw1snkyZplXzX91K5l74KXZFRelf1x1y9/1pWhsTYLyEHrcx/R2vP4237zF9JRGABOSB5uTjEPopDUuB5Md/7+/992+cJU9tSnQLekpXVI/eO6wVhvAkJkRxKhu51BJZSg8jjXUMYJipn4AzhegSsF9y9ODCTxV/7Jh7yRVnCjakluu9kLvVM2Ftf/9m/xqI5nSGq/S/HWMlF+NKsoC6cLabyNfx7XFrZIMW1ZZ9zLKHjJ5DFqQN+3JEPJF/CBTDQc9zBPW00hY4oiLu3ajf8cBaojxz/udRPg7uv4bj0+bCrQsFeGU3aRUNG+dKDOGaYJovvPISotQPBHtQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2Kqsm7S7nVPh0w+Hj7C+jzcdhYJsfDn8jkSJyyG3nhE=; b=Wc/8fRSgYYx1qqHw2cN9U/RgflRRJFRoL6+V+lT4p4i4zXq8WtD852krkw5zAhjE9TK+zbd+yNHz/RTJBm6AULnOdXiXGOvEIPQMHpsi7PRSC2lPJ2OcIH40HIn0JprXnvm3EOdRpd5qHOhcSuklCjTD7r2qfqpvW4ma9dsO1YVy2/8bqXeKDzYP4arTXVWgTPO1j8d8gEcq6eA9bjQfbkIEnOEMocmzehlemBbwY3lVdDZEJXx5ry1jffAVL3aPh6+cqZUEyeSPYrcqtcxsfVhtNPooGcmY2X6aFQkPBFSbXP+jFFggnEdKZlBHb7eGsBXgopAriWxY+HCA86dMKQ== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon Subject: [PATCH v3 01/23] vfio: export PCI helpers needed for vfio-user Date: Fri, 6 Jun 2025 17:10:33 -0700 Message-ID: <20250607001056.335310-2-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: d4106bb2-2fa3-44c4-283e-08dda557c899 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?4WRtPhQKSFKemYSrC9fyndGrriXteuHEICG/pkDj0hpodFAWdNxpBT7Ip7+0?= =?us-ascii?Q?bEVd2nNHe8XykHWAV+gswwgzlCItSjlTCXorH3I3q4bCO9sCZ7QL3EXrv5M/?= =?us-ascii?Q?vGu+swJn9jepvc8sfxcAQDdJohnUx7zpQi352/2E324qxAHGRgLOocnP7Fao?= =?us-ascii?Q?VEof60v7/MeMwEpjgqMbYNzqkI5TIHuRjBSxO7nHIcAaYyNASoqz4Kbw4XAg?= =?us-ascii?Q?PXWgTBeKTUMF+yEM3ke7CpDJR0F151Q5njRFg8u0RjOrFJ8MwURzbAHdF270?= =?us-ascii?Q?E3CLhWbsP1x+zgQoHm4q3qmJXpwOXyXJH0Yr8wHmB9oE/hCGc05rqfywQ7AH?= =?us-ascii?Q?cevaEXmQGrDQVSdTurGD3gBx+IwC3b8pcqyp51H3MTLr1GLt1mz8HYzewaTb?= =?us-ascii?Q?uqp9uI1qEzcNbmmfCad+vd1ivGHv33X7aFsL0YXFoVH+BiE82BCPD0hdh5kB?= =?us-ascii?Q?zdQ8WHFyFQagGVqvP8VoHUZ/WBWoz2nH7NTaY0LlHZ8SHuFkT/cijdRt1sZH?= =?us-ascii?Q?JGRG4WypnOJEoJBdSuz0UtltZFrfEUBZQCVKBmEvy0hcynoTATNPNV2m1X8P?= =?us-ascii?Q?qLrB2p7igseVKqANQbmehDnQsY3O1gNyDT2E7nLZCx5U8Cd7MJiab0mX60cq?= =?us-ascii?Q?1zsRD0O3dDIfOpWgBgkJfc8W6GWsRd/LLtvps4dnmisOKSg8hGFdTvhFdg+Q?= =?us-ascii?Q?21gIz1EdKi+17E58k82skaF4Sx6aEVV2+2c3qIS+8wLFdSUtWqY5Cgr22xkW?= =?us-ascii?Q?o85f0q1FIfC8aZ6l1lz908Dxuk35YguJRc54aGQBAyWrgDNg+VcEIz3IOOuo?= =?us-ascii?Q?nlwgm2PWiFQu9pKivZCGhJZo5/MKMEFLDfiivGjOSKcJB+IYSKLxUSzLZujE?= =?us-ascii?Q?1U4huVGdZT+M/axLZgwGbTl8OHl7KijVPcpKRUq689L6J1D20kbB58UHL6oT?= =?us-ascii?Q?1qRq0r9tXO1354jwbR88g6pWhqqcrWtKDa+BiJy60ORkX71UsZL+y+eQnLK9?= =?us-ascii?Q?nqYrIdZpJHj9FFhB0VDGzVNG8MZG8Bngo0cBIvcprdQCRqDSxgvyfZ+t8C2O?= =?us-ascii?Q?3UAtTTfXLvbXHDfHBBBxb4EQGN+y0F8UyDgKQOjHU3tR7J31hFu3BFLm4HWt?= =?us-ascii?Q?ifUtoW/OsxKbr6AdWRo8HU9jjX7ibd8egZY1c0EFkXNYag+HkixOHB5tqLJP?= =?us-ascii?Q?kA7g607Zun6t5TVdALuxAQbfyrG4i3Y/gLU3n0/ZtLCWrk8+mhGIm/+a+rtv?= =?us-ascii?Q?E/VuEG/Zv7ki0JXAVTqsOWmBEuLDZ3/GrBbFkgRn85KabyFZU4HC8JRkpsnF?= =?us-ascii?Q?dx31d8RIDJOytIXRIHGdlDQsHsd3hwWqq5iSTlAvdHt2E2XJvpL6KwexucOf?= =?us-ascii?Q?jLUAhQW0xYMrJrHy+lAT6bMU3JsM1p5kh8bybdhTqXrjB93F5sCzgwKCOpEM?= =?us-ascii?Q?s1mnolQVZKI=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?8I9uToGgzWioepNcrm620+dgOVA3gNuC1ob82g5KbiMLm1oolhfx2iynBtZU?= =?us-ascii?Q?TnwIayr2XGhJeDx/Nd2CNwTWvrZpuM8T0GIQQPZCywJEtYKCXfcJZFJyujP7?= =?us-ascii?Q?A8anrdfILWd84bypRpsd997lypaU0vYj7Czwew+xCwzV7YwRO0UUInt9p3Rr?= =?us-ascii?Q?wnYWXmEVdN3E8qwcC7WVp9eExIvGPnF0wNhx1AfXT1JP7MltkO0hAkD+juzf?= =?us-ascii?Q?wCBhmE/2xNub06ELOR46yUMMMVlpjADytynwgE44dHqzFfgBlOX1lVIgAfbH?= =?us-ascii?Q?JZ2jSfjtPSnVS8umpyCRfoFSxckd5I49u06NjoObqexzWxBAtm9Wb6fbb/EO?= =?us-ascii?Q?9oAm//eMOnYIAWnCU/DUZNDPq6fNfSKHFgUk1lE2f4/PoRzv39xgNXvVTwfo?= =?us-ascii?Q?53O0Vetn4RyXqISqQWt7LRMCeiGo55rpJSJcCgIRPN2sPcQaHU9Iwrawsj6Y?= =?us-ascii?Q?ADpjf+75cPchthBO3GiOKanL4FRPKc2/M0lEEH0cvPXSTF78zaz3gkQ8J8XS?= =?us-ascii?Q?It6eFoVp6NnaJ6VJ7eO/bInNs3dlChEDYFj9lVvdCxPQfDSAWGfXRb8OoSAd?= =?us-ascii?Q?nQ6uahqOoR4pJJLj+Xk2f+t5xx+DST1RGOSzPGioAM1U5OGXh+r87EPO5cEU?= =?us-ascii?Q?125Rb0vF38ssNAgQF5Msy7UHpjX45yNgbvxnk1Wpu6z/hwNQRyovSe9sUFLB?= =?us-ascii?Q?IuLBbgJoLQ4Ayt8LheHvehdUHEQjSLr4pBytyj4Q0VFQuA7SspTrzbXC+zWx?= =?us-ascii?Q?275r/1Lb8tUKjFNOIdegLOv3ExWGhmWFz6unu7p/NCIWdD7MHjYEzmLXwIwh?= =?us-ascii?Q?GjqNffIPbTyYE991uBZJWpswGUHqoztpA0RBOMYo9BgAlcZCsMrG8JDQKCKB?= =?us-ascii?Q?7Gc2An+c+YVy+bjROl/nJsZarCtRAOoQE4WMuPKIp5jOemZZB/gJmUr7j4li?= =?us-ascii?Q?11TmnQYPP8RyGeyl29PgeKDlXADE+Cr2oRDitJKDkZa9sFD4JIUzKUm4MNrB?= =?us-ascii?Q?S/WuKzy11jcnflkxxV12rLyGEjTYzbnwgPt6LcjKD0lEO5f1XfdaVa+QEFtC?= =?us-ascii?Q?EB3v37FlJvqrXfhiz4H3RbvGro1+4jTIdEAnPRfQ5JqkM5jmzrZADkGv3ptp?= =?us-ascii?Q?XCG9mWN8qVUOtZ+dKAAZ1csnjAKAYqfBgtmmlCfEbBF3XmITn9NdbsU9XOaN?= =?us-ascii?Q?xVGFu54RTJJarwGBWoxSrWsOhn1uT/YeX+DyHKPZ4u7zOjQYanQH8cevlbz4?= =?us-ascii?Q?ALV0PYO2N8C9PT+/wTGpuIJ2gcjx4xtGR4FS+S62pEnG86GZ3aIUDLVxcSfD?= =?us-ascii?Q?qFKG2ytWetyuMNyXrKW92jJciOMnS55ger0dJ9N2D2IoAHkrZ/m9IAN54z7s?= =?us-ascii?Q?naTP99qzcFOtp/WNL/dXGEGQA+LwLBcKgfg7I/KVtMkFaVvQJVLJ40xvtvFt?= =?us-ascii?Q?2U8f8zTMlZtAQq1dodgLEvNuKTgaBO7sfU7MbjiIWziOQhPHrWW5m7J0drLT?= =?us-ascii?Q?e6Z4orXUTzGSSh0F82jn+fDkbjgiw4Vc3pFgSBKRoHr/3kKk9oh8D7fXNzaZ?= =?us-ascii?Q?2kWaC6VPY45xxCw3Ivepot2oRDR+Dy6kTqcP3nAi?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: d4106bb2-2fa3-44c4-283e-08dda557c899 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:00.5491 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UnGwPZaesA/CpVkMRMW4g2zXRdM2nxHH7qVaxW+ap4LOb1wMGbYtizY2gYyaGyoOfFauw3uijsEbw43M9kNsmQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfXyCc5IQVz8kMB OkHrOoKByVvKIOFPiWVgVMXlrak8B1d+iAddKw5tuvvadvJj6KeHkzlfWO8xj6zHzNyWp3fraYz qlcVxDJKBhCW1eFV6q6eG5M/gfuHepBxzTTdQcwB1EbAFJDjTwAo0Zk4CvRNXcRrUaEH2kEdRZZ DfiwJeG7sJpU0OREcU0F0mP8stOALCw2BYcfezDZYqqbxo7ivGXSBCdO2zSFDHuezpIYG/310Qb ejfdJ4YEjel+Rsbtv6c0Yl0hibubnNHS6Abl4TIdFWbkyzgMxgDFTWHnWa2dFK0bTMEvVN99YDo s2G6EkymkWwPxBdMBMCKWiUC0ucKIX/ZlVhSjTiSUauQ2btzdjeTN3Sphn5hjetKpNTtSMlZ4tB gn4NyUcPrEcxCXNAkz+nxgl0rgGwmMy7bX9h5uNsqbwgPj3AFRK6ytN8rUaWAbd64FxMVjVQ X-Authority-Analysis: v=2.4 cv=SY33duRu c=1 sm=1 tr=0 ts=68438397 cx=c_pps a=PuycMPbAwArgayjkkQxHog==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=64Cc0HZtAAAA:8 a=29XCgI2txZ024MPHgJUA:9 X-Proofpoint-ORIG-GUID: OUgP1CwhwLRDhV5XmJXcgW3OD7BJ9Kd_ X-Proofpoint-GUID: OUgP1CwhwLRDhV5XmJXcgW3OD7BJ9Kd_ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255114046116600 Content-Type: text/plain; charset="utf-8" The vfio-user code will need to re-use various parts of the vfio PCI code. Export them in hw/vfio/pci.h, and rename them to the vfio_pci_* namespace. Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Mark Cave-Ayland --- hw/vfio/pci.h | 11 ++++++++++ hw/vfio/pci.c | 48 ++++++++++++++++++++++---------------------- hw/vfio/trace-events | 6 +++--- 3 files changed, 38 insertions(+), 27 deletions(-) diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index 5ce0fb916f..d4c6b2e7b7 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -248,4 +248,15 @@ void vfio_display_finalize(VFIOPCIDevice *vdev); =20 extern const VMStateDescription vfio_display_vmstate; =20 +void vfio_pci_bars_exit(VFIOPCIDevice *vdev); +bool vfio_pci_add_capabilities(VFIOPCIDevice *vdev, Error **errp); +bool vfio_pci_config_setup(VFIOPCIDevice *vdev, Error **errp); +bool vfio_pci_interrupt_setup(VFIOPCIDevice *vdev, Error **errp); +void vfio_pci_intx_eoi(VFIODevice *vbasedev); +void vfio_pci_put_device(VFIOPCIDevice *vdev); +bool vfio_pci_populate_device(VFIOPCIDevice *vdev, Error **errp); +void vfio_pci_register_err_notifier(VFIOPCIDevice *vdev); +void vfio_pci_register_req_notifier(VFIOPCIDevice *vdev); +void vfio_pci_teardown_msi(VFIOPCIDevice *vdev); + #endif /* HW_VFIO_VFIO_PCI_H */ diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index b1250d85bf..a49405660a 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -103,7 +103,7 @@ static void vfio_intx_interrupt(void *opaque) } } =20 -static void vfio_intx_eoi(VFIODevice *vbasedev) +void vfio_pci_intx_eoi(VFIODevice *vbasedev) { VFIOPCIDevice *vdev =3D container_of(vbasedev, VFIOPCIDevice, vbasedev= ); =20 @@ -111,7 +111,7 @@ static void vfio_intx_eoi(VFIODevice *vbasedev) return; } =20 - trace_vfio_intx_eoi(vbasedev->name); + trace_vfio_pci_intx_eoi(vbasedev->name); =20 vdev->intx.pending =3D false; pci_irq_deassert(&vdev->pdev); @@ -236,7 +236,7 @@ static void vfio_intx_update(VFIOPCIDevice *vdev, PCIIN= TxRoute *route) } =20 /* Re-enable the interrupt in cased we missed an EOI */ - vfio_intx_eoi(&vdev->vbasedev); + vfio_pci_intx_eoi(&vdev->vbasedev); } =20 static void vfio_intx_routing_notifier(PCIDevice *pdev) @@ -1743,7 +1743,7 @@ static bool vfio_msix_setup(VFIOPCIDevice *vdev, int = pos, Error **errp) return true; } =20 -static void vfio_teardown_msi(VFIOPCIDevice *vdev) +void vfio_pci_teardown_msi(VFIOPCIDevice *vdev) { msi_uninit(&vdev->pdev); =20 @@ -1839,7 +1839,7 @@ static void vfio_bars_register(VFIOPCIDevice *vdev) } } =20 -static void vfio_bars_exit(VFIOPCIDevice *vdev) +void vfio_pci_bars_exit(VFIOPCIDevice *vdev) { int i; =20 @@ -2430,7 +2430,7 @@ static void vfio_add_ext_cap(VFIOPCIDevice *vdev) g_free(config); } =20 -static bool vfio_add_capabilities(VFIOPCIDevice *vdev, Error **errp) +bool vfio_pci_add_capabilities(VFIOPCIDevice *vdev, Error **errp) { PCIDevice *pdev =3D &vdev->pdev; =20 @@ -2706,7 +2706,7 @@ static int vfio_pci_load_config(VFIODevice *vbasedev,= QEMUFile *f) static VFIODeviceOps vfio_pci_ops =3D { .vfio_compute_needs_reset =3D vfio_pci_compute_needs_reset, .vfio_hot_reset_multi =3D vfio_pci_hot_reset_multi, - .vfio_eoi =3D vfio_intx_eoi, + .vfio_eoi =3D vfio_pci_intx_eoi, .vfio_get_object =3D vfio_pci_get_object, .vfio_save_config =3D vfio_pci_save_config, .vfio_load_config =3D vfio_pci_load_config, @@ -2777,7 +2777,7 @@ bool vfio_populate_vga(VFIOPCIDevice *vdev, Error **e= rrp) return true; } =20 -static bool vfio_populate_device(VFIOPCIDevice *vdev, Error **errp) +bool vfio_pci_populate_device(VFIOPCIDevice *vdev, Error **errp) { VFIODevice *vbasedev =3D &vdev->vbasedev; struct vfio_region_info *reg_info =3D NULL; @@ -2823,7 +2823,7 @@ static bool vfio_populate_device(VFIOPCIDevice *vdev,= Error **errp) return false; } =20 - trace_vfio_populate_device_config(vdev->vbasedev.name, + trace_vfio_pci_populate_device_config(vdev->vbasedev.name, (unsigned long)reg_info->size, (unsigned long)reg_info->offset, (unsigned long)reg_info->flags); @@ -2845,7 +2845,7 @@ static bool vfio_populate_device(VFIOPCIDevice *vdev,= Error **errp) ret =3D vfio_device_get_irq_info(vbasedev, VFIO_PCI_ERR_IRQ_INDEX, &ir= q_info); if (ret) { /* This can fail for an old kernel or legacy PCI dev */ - trace_vfio_populate_device_get_irq_info_failure(strerror(-ret)); + trace_vfio_pci_populate_device_get_irq_info_failure(strerror(-ret)= ); } else if (irq_info.count =3D=3D 1) { vdev->pci_aer =3D true; } else { @@ -2857,7 +2857,7 @@ static bool vfio_populate_device(VFIOPCIDevice *vdev,= Error **errp) return true; } =20 -static void vfio_pci_put_device(VFIOPCIDevice *vdev) +void vfio_pci_put_device(VFIOPCIDevice *vdev) { vfio_display_finalize(vdev); vfio_bars_finalize(vdev); @@ -2905,7 +2905,7 @@ static void vfio_err_notifier_handler(void *opaque) * and continue after disabling error recovery support for the * device. */ -static void vfio_register_err_notifier(VFIOPCIDevice *vdev) +void vfio_pci_register_err_notifier(VFIOPCIDevice *vdev) { Error *err =3D NULL; int32_t fd; @@ -2964,7 +2964,7 @@ static void vfio_req_notifier_handler(void *opaque) } } =20 -static void vfio_register_req_notifier(VFIOPCIDevice *vdev) +void vfio_pci_register_req_notifier(VFIOPCIDevice *vdev) { struct vfio_irq_info irq_info; Error *err =3D NULL; @@ -3018,7 +3018,7 @@ static void vfio_unregister_req_notifier(VFIOPCIDevic= e *vdev) vdev->req_enabled =3D false; } =20 -static bool vfio_pci_config_setup(VFIOPCIDevice *vdev, Error **errp) +bool vfio_pci_config_setup(VFIOPCIDevice *vdev, Error **errp) { PCIDevice *pdev =3D &vdev->pdev; VFIODevice *vbasedev =3D &vdev->vbasedev; @@ -3124,7 +3124,7 @@ static bool vfio_pci_config_setup(VFIOPCIDevice *vdev= , Error **errp) return true; } =20 -static bool vfio_interrupt_setup(VFIOPCIDevice *vdev, Error **errp) +bool vfio_pci_interrupt_setup(VFIOPCIDevice *vdev, Error **errp) { PCIDevice *pdev =3D &vdev->pdev; =20 @@ -3214,7 +3214,7 @@ static void vfio_pci_realize(PCIDevice *pdev, Error *= *errp) goto error; } =20 - if (!vfio_populate_device(vdev, errp)) { + if (!vfio_pci_populate_device(vdev, errp)) { goto error; } =20 @@ -3228,7 +3228,7 @@ static void vfio_pci_realize(PCIDevice *pdev, Error *= *errp) goto out_teardown; } =20 - if (!vfio_add_capabilities(vdev, errp)) { + if (!vfio_pci_add_capabilities(vdev, errp)) { goto out_unset_idev; } =20 @@ -3244,7 +3244,7 @@ static void vfio_pci_realize(PCIDevice *pdev, Error *= *errp) vfio_bar_quirk_setup(vdev, i); } =20 - if (!vfio_interrupt_setup(vdev, errp)) { + if (!vfio_pci_interrupt_setup(vdev, errp)) { goto out_unset_idev; } =20 @@ -3288,8 +3288,8 @@ static void vfio_pci_realize(PCIDevice *pdev, Error *= *errp) } } =20 - vfio_register_err_notifier(vdev); - vfio_register_req_notifier(vdev); + vfio_pci_register_err_notifier(vdev); + vfio_pci_register_req_notifier(vdev); vfio_setup_resetfn_quirk(vdev); =20 return; @@ -3310,8 +3310,8 @@ out_unset_idev: pci_device_unset_iommu_device(pdev); } out_teardown: - vfio_teardown_msi(vdev); - vfio_bars_exit(vdev); + vfio_pci_teardown_msi(vdev); + vfio_pci_bars_exit(vdev); error: error_prepend(errp, VFIO_MSG_PREFIX, vbasedev->name); } @@ -3338,9 +3338,9 @@ static void vfio_exitfn(PCIDevice *pdev) if (vdev->intx.mmap_timer) { timer_free(vdev->intx.mmap_timer); } - vfio_teardown_msi(vdev); + vfio_pci_teardown_msi(vdev); vfio_pci_disable_rp_atomics(vdev); - vfio_bars_exit(vdev); + vfio_pci_bars_exit(vdev); vfio_migration_exit(vbasedev); if (!vbasedev->mdev) { pci_device_unset_iommu_device(pdev); diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index e90ec9bff8..f06236f37b 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -2,7 +2,7 @@ =20 # pci.c vfio_intx_interrupt(const char *name, char line) " (%s) Pin %c" -vfio_intx_eoi(const char *name) " (%s) EOI" +vfio_pci_intx_eoi(const char *name) " (%s) EOI" vfio_intx_enable_kvm(const char *name) " (%s) KVM INTx accel enabled" vfio_intx_disable_kvm(const char *name) " (%s) KVM INTx accel disabled" vfio_intx_update(const char *name, int new_irq, int target_irq) " (%s) IRQ= moved %d -> %d" @@ -35,8 +35,8 @@ vfio_pci_hot_reset(const char *name, const char *type) " = (%s) %s" vfio_pci_hot_reset_has_dep_devices(const char *name) "%s: hot reset depend= ent devices:" vfio_pci_hot_reset_dep_devices(int domain, int bus, int slot, int function= , int group_id) "\t%04x:%02x:%02x.%x group %d" vfio_pci_hot_reset_result(const char *name, const char *result) "%s hot re= set: %s" -vfio_populate_device_config(const char *name, unsigned long size, unsigned= long offset, unsigned long flags) "Device '%s' config: size: 0x%lx, offset= : 0x%lx, flags: 0x%lx" -vfio_populate_device_get_irq_info_failure(const char *errstr) "VFIO_DEVICE= _GET_IRQ_INFO failure: %s" +vfio_pci_populate_device_config(const char *name, unsigned long size, unsi= gned long offset, unsigned long flags) "Device '%s' config: size: 0x%lx, of= fset: 0x%lx, flags: 0x%lx" +vfio_pci_populate_device_get_irq_info_failure(const char *errstr) "VFIO_DE= VICE_GET_IRQ_INFO failure: %s" vfio_mdev(const char *name, bool is_mdev) " (%s) is_mdev %d" vfio_add_ext_cap_dropped(const char *name, uint16_t cap, uint16_t offset) = "%s 0x%x@0x%x" vfio_pci_reset(const char *name) " (%s)" --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255147; cv=pass; d=zohomail.com; s=zohoarc; b=Hg7mTaDkCGGOGN2IE2BQiO3x8g0vxt57rxIVpAo6kgFTxGviORf89YIA5WQ8T1WljCF/atKeR5xaP7qru/HC+4Y1vuSHyAurSWXDmsaDilQmC9vmwrBDR/AyFzVOja8+9VPLd7t+Ta8El3vwmcp9ULX5uZ2dKSuwSP4FoxvTLUA= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255147; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=bIjg7jCKjTH3HvYY7WwsaSn1m5QpED/N/yYQXRQ3sZg=; b=YfpHJsa+3iiR4e4GuWZfojVf6/G7slncIRY/ZB5/zgw+OcVHo6hHBscf3gacCrsmur49D/64kBCf16VHp+6biBCNpugH49HGtFd9jHdU3+roS/fYOsrxO05Jd0qCHkj/5ITlGa9OFybSQrJqUjLSvtyi+HTDHw73iQYytqaFl8E= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255147802307.75636294014816; Fri, 6 Jun 2025 17:12:27 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9V-0002ud-AJ; Fri, 06 Jun 2025 20:11:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9T-0002uF-NN for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:11 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9P-0006NJ-7J for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:11 -0400 Received: from pps.filterd (m0127840.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556K9XAe027707; Fri, 6 Jun 2025 17:11:04 -0700 Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11on2127.outbound.protection.outlook.com [40.107.236.127]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 471g813xsu-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:03 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:02 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=bIjg7jCKjTH3HvYY7WwsaSn1m5QpED/N/yYQXRQ3s Zg=; b=Kvx6NN4O3o5Qwc+SGQ67N2UtIPyEJ22bQIkf3FcOn7d9eARRVaUCanvpa UnwHVKsgUivX3mgeHG7K4kdS2Y3weLpFk9i85wSkxaYsvYMN5pz7+e0Tc7P/xuRt knkgzjfdkUC1hhIz9vvpK3QFezQegJ/5wCeMiV2Xwl2MaJdvIhyhws7Qq0z8M2x4 XCX3g5ZaDnlCCK1iJ9BPdxKlGAAEvto9QmJMnG4Dv/6nH258a7M2EnILhPYFoO7V 8uaptgYh+UXHZ2yV796QRT/J9VlC74U0skxEGOqsJdIWD9X6NLeUCyUsP/Z9jINf nB1bOmXuAiU/sqKvZ9nPu8xB4na5A== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=a50JJDZz+fes5JaK9KGmU+F7muIJAgcjhUHuRxrIr4nq7Ez0Annqx4aJ/f3eALgadEby90jq+Z2Ke75od+9/vmmKn/E8vIIfvGuBVrFO3oVzBxpm66mSxmEkDHBJvBz1gT///KOrWQv6uoeEHP/6TQ43NPpYlSsjmstp9YdhAxJuzDcXizwBiPZlUwKMHz9Th911DUVfOXNbV8ZKk9x0XGEOL+o0I4wBtg/QFBAA/3DOoMInW02SJO3W8LQrVs1eeZ0ZoVYRB9/DX57eu8YnYiA/Uo/bBQzoWdFVQ11DhEUhH+DPPnm7S0yNb5+VHoKP/B+lZ7vRCxX6Ny9nl9EPhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bIjg7jCKjTH3HvYY7WwsaSn1m5QpED/N/yYQXRQ3sZg=; b=NfDWixZwuhWGTEIqbeV/m4L4K03YCjvY8Uve/KRPS8uoqtUH5ry6VtJf4JpKHWZH7ckgFdN2KRgt+xU9XOl4LvOkjkUZ++b9oA6RGhacSEMEI55ZaONxfcG0+CqwQ0fl7zhsTImZTL/B8wR4QRAhZRwMH7+0WeY46gN0VgAS9sVTsGIUMZbTj+HeGfuOHI70Mump09y34bmab0M0TrRloy8wvfwiHA9QxCWiI2SZLSuSEi3xakY8B/brbGFI0Wej1v7ItNwHaKC2GuWNhjb7EIUVt/BGhv3/sKawG5fTokFR+2U0YEIjYqryMZOTea+mXBTgbW9WJqf+S+Pgd9VzYw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bIjg7jCKjTH3HvYY7WwsaSn1m5QpED/N/yYQXRQ3sZg=; b=sLLFtRtgPuOynbYde8ymyFdNIZqS/mVF9uLeQUyoeC+LtH2mjLLiWZVmE2YrFCNzmhwHwLcDrKiR8Hr6GKqQfD4oUocsPpKUVD/aRoiaw3cej/xDBpcHijAIzNc+Zb2fcF0zgDnGFz6AGYWOxCqqbyAkJvVQ2mQ2CHktNt6ft4UE1sx6YRH/3/Eu59a/VOGi/qKTNE3l8HfTaN6drdkS1G0ay4MexBzUYAHFkeiHRgFBTOi8ISwVhHC8mAdNBa3UDA7dJ2/lJdIyjADd6I/ScjtK0GnYoJkwFtYzmltsL3ClvWmlbb99Phtj+JRvSslHmUulK/LMLjHo4XpMakcf+Q== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 02/23] vfio: enable per-IRQ MSI-X masking Date: Fri, 6 Jun 2025 17:10:34 -0700 Message-ID: <20250607001056.335310-3-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: 46ae518e-ea50-4cce-654a-08dda557c97e x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?uH7bqYMyQwha99wtNzH1nJvDxW3YID0PepHoh4c/E5eMLXf0IgcexkAcscoZ?= =?us-ascii?Q?s4AEtc7K0wGdhB6hYO+D1ER5zblZLbpvSbXfASNfQJfjCQ42zLWqU7mTzQIE?= =?us-ascii?Q?8txsvJJMpoceg08g6iugye7//zm+yFV2+OZ6RGQyr1K439eGs1nYLECeyK+9?= =?us-ascii?Q?RU0Rufh8K92cacdEMPQVAkblycN8TEl69uv66ex1FkeHutFuhbAS76lurPfy?= =?us-ascii?Q?S1zBM4enyk7X/qIOO+iiYV9hCNpC8w/nExYkCHfvcm5AMXgyDAYJaWJlWKfZ?= =?us-ascii?Q?zH6va/kSAeYNbiG6ARN7bHpTufi7Kg6MaKwMcc7FHeNaomw5/41KJCjuJMEB?= =?us-ascii?Q?ijLiuNT6aZHUQGq//EMxVPzzpRcxw5SKIi6WCQzg8X+msbR3Zd1qBOyNVzAw?= =?us-ascii?Q?H8H7HMS3G0YdsOWGI2oeCNJvIGKsHmSYiz4SerIhas7TtrAaep2J4Y+RPqYs?= =?us-ascii?Q?cu0T1Q+jiPEHGFIatHWlP3mGUAGjXUXUOYNFYCKogSA+3NCL0k5rqcRJn5cL?= =?us-ascii?Q?TE2ME6t11Fcwm6e0edvMCXTs1u70ePQs5nTr06uWOYJe7nhRMzfRONAuJr+V?= =?us-ascii?Q?zjS2ryiMJOzKhShHXxiS7x3BAHmva87UMH+T894EHuAPW2U67grKOv6ZNXxv?= =?us-ascii?Q?H5w6Qbo9L8BmIYOFNkXl0zi6YAl6SaMvRZNKsS3DdWcOAimrVZgfvKxabqX7?= =?us-ascii?Q?L1crj+6CdNCQWJ2D7bQgMqCc/y01nT8gTKYCanGMsoZn/htLQA/H79fyrY+9?= =?us-ascii?Q?9T/9CVhJfFAHHJR4Gsa+Xz2w8EpzWanN9ZTxgYZ45H/eMqvIWH3f5W2urfVW?= =?us-ascii?Q?4fYnH5MCbi9ScTq3AiVMmaoJ5tKwExBy80AZjjMniV8PKEg3tuN5ceXUIiqZ?= =?us-ascii?Q?9qtuU+Y4T8m8BNe5C9w/GFit6hroTtVpUZCqsE+56BjDOBe18dqDwSObVL+B?= =?us-ascii?Q?yrtsGgECjKdbj3v9QbG/aDE3wg8qna7r7DECR4mS5meLM494lMfgOS3rKA1L?= =?us-ascii?Q?ipyZCUTWOHQ+iAz/8xFxRYtHIuutCb5VF34FybWgEyiSYBiFauk+6ul4YKsR?= =?us-ascii?Q?22GpcFy04FibBr0qn9pTkaAIBNCaxUdZDvAdMEAGCdVjOG6jwLMJRuyBA+o8?= =?us-ascii?Q?uRmBn+qVJMGbC4s9/wsQrW+Yhui/EX0MpQ7M2wdQhqQcxliK9guoiaEGVb7L?= =?us-ascii?Q?AGKutYi9laA777bLN7MPBU2EwxyMOrW9UBueulS881pm4Dl1dj5doNuRdawH?= =?us-ascii?Q?IRw3bZ0IDqp4GI2+ed4WXbR1KPyhGFVEEzgowxJTqjduBiDnLTlmBVdoGQWq?= =?us-ascii?Q?cWkTGppHGqgtlALEMfGkXeZ+PGEOwXhYW0XiTxmUodrgZJwGaoXPOTNyg0LE?= =?us-ascii?Q?KQqptVIbW4vxEYQIro/TTR7pRx6CSbCRNWvCaa4ph4tNSN7xV3q5ujWJIWJt?= =?us-ascii?Q?j/JyPzu37MU=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?F7didSPEFBLWof24Y2h0Rzrmw3CuTAGEhJUvlU42p0KIT8nzljR+twquQrVT?= =?us-ascii?Q?9VDuqV04IzPE4rp3CSpogSLE4vOWpuDB9gjGCW6C0l0XEO8+DSfdp4n0Mm40?= =?us-ascii?Q?MyxBfpRwHhpEklPhbWZYbpTarAd9jjc0ST2xGHoiF/hVCIBXmWWbIV5V7gzE?= =?us-ascii?Q?VC4fUOQn/R4XkdcsclRoyN6RK4L7N5MqOEDkpjVzJfsfAWLwXc91sSdxFBqE?= =?us-ascii?Q?pEbpkG43iM857Lxt/CeuluTbv5N9fzOgQ1MWe0AOAMGXOx17hTKmktyOLIzh?= =?us-ascii?Q?ou3TQeroIKBAwBs7za7IvcL8L+SDwXrICV6/LwP34GjwyCZtKMyWOyonPdji?= =?us-ascii?Q?Yk8sZX15qg6+FQBBxgpZBlJ0dnEC2V2BXnxiZbLSmNtmulRKM4nGe/Pm0N4m?= =?us-ascii?Q?y1wI2ktA9MjhE488ko8OBQTl1kxMKDHA2EkrLZNXjRp/JFKtgJ0ltR+Edydy?= =?us-ascii?Q?+iFwovWY2I/jOXrOMbFS7n4HYJFWyqvFaGoWDz620qodiVxRNLA34dep5hWI?= =?us-ascii?Q?SRlMl9L/9osLlw/4RKku6YOkZp/nZdDy22KPWCnQJigXT620TDLGXtEDqrZD?= =?us-ascii?Q?SAGT5C8Rig5zrU17fbR3Iay7/ZM8k1uHNWMeQa+WC9IjwbGJtHkAxOj4tY8f?= =?us-ascii?Q?31qQDfPE+nYFMX93IVKEElMpCxrFwn2CoblvrREreSR3WBYPyMBzwQ3rGU8U?= =?us-ascii?Q?cN3L88O719zY1S1eOo/3EQG8AXFDeG/CySnMtwViXkmXGkxBJ6SvCNNTb6cw?= =?us-ascii?Q?ogp7aJAMGJOjAkPbPkYs5LGa66QBLuywZ72eyisDJnqjw1iHjiMWx+lrobrq?= =?us-ascii?Q?2ivBuxTJFcRK4tEVUp8t2tuyL65G1OcUtSejN7rjgkBJlNWicE9AIDOsTHJ7?= =?us-ascii?Q?c6CFgYwll+LLYdd4xNS5uWuVRKYnh1eSc6iDZbRhEGbw0ekxs6XWBjQzBoce?= =?us-ascii?Q?4duR/qd8tfa9v0ShcT+7YHR9y1lDegw/HMbVoHe4nPD9rg0lz/un7UkMQyF/?= =?us-ascii?Q?F3GTSzYh3b443Fgo9HTPaQ+PXqLovbhuOMEcikamLjBdAWnUQP6s0ngbLEYK?= =?us-ascii?Q?YABClSwxE68EeB4gseoFbysvi/82QvM1oF+AcKcIsnvrxkSnelROE1gVzfkE?= =?us-ascii?Q?F0h1u+qiJrno052PCnXnzlSPe1LlWM1Y72yttAeQu6ifzvuI2Bc4zNPNM9qO?= =?us-ascii?Q?J7xWhqfDrwCagMuNUaBLZjBXH9cEcxHmDb4IQlOA56ISp6m0yez01G+VQF2i?= =?us-ascii?Q?ad0v9BXNhFwl4sc7/IKm5nz5If1SVgaLdSnnCUjujS3fV6XvXFL8X3K/S5/Z?= =?us-ascii?Q?gC+A3/CFJEc7I0+NV5mRnTvkgDtL8OpfPH2lnIFm9Ss1/SSi/+i7y73+bGhH?= =?us-ascii?Q?TA2MsrNFF1BQums1wKbwgX+YuGrxgTKPaQt4AHWqST7sZ3dFIsJVFujTsZu3?= =?us-ascii?Q?ic+JDx2jMP5udTpFNQH+MT7ck4m4vFYH4QOvWjJzw5EaOPj5SxsUrgdKr1DF?= =?us-ascii?Q?ROmoIMuUXjrPeMtf9nDBGQc/92Tz91EWWBO5k8lm63+tJkfw4ygbdNjc5nXm?= =?us-ascii?Q?/NgpnyMx/ReVxbiOGZy/KxDRMriemAIB+A0Op40z?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 46ae518e-ea50-4cce-654a-08dda557c97e X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:01.9216 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: vH+W/GSB85esu0K2kPwpY216FMloaJVvxd0ck2m7VjcyCZx5WmN+sv8+xCuhAOzNbfouaZGbES1j8txAHYNotg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX5Dem2Uctk+e+ d5PAve2yMgjyr8bhDdNNUoy+LalqWhbtmAb0uvo6gajZ4CNS+FTgWFtPkLUM2QmDVdLAZxBSyR8 ug4sSXlS/RI/wpgV1m5xsF5TZ9gqAeksmh8W/95zHoyNCK1iXy9Y0bRr6PhtfGles0oeXmzK5Nh MQ1F9rabx/82HVXw19VcDXFtELRk8RLgvmJbYTGM+9lM6reB0zRLMMYUr7NK6D+4K02Fj/zwQPr TSCPVmaevvuJHCAUjz4PED2fcATW/NJs4ruGVWXAdSsfqOE2RY/GX0RlkqMlYytpzi4Vf0Y7XOj AGfUZ+bEVmG3kSzbNQ5sO0Bf4pvKzRbgBQsaLfTIbUN4b7WXbDRqupr+v9pVXa3oZcIbopNef0i TdS2ZGziqnNyF42eH72LNY6kMe2TA8em23yWErZr+fK3Qdf8pLVEbFPadA2LRaXCeRWcqRvN X-Authority-Analysis: v=2.4 cv=SY33duRu c=1 sm=1 tr=0 ts=68438397 cx=c_pps a=PuycMPbAwArgayjkkQxHog==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=_brLm_9mC3lTafiPIV8A:9 X-Proofpoint-ORIG-GUID: g8w3Lm7NdVTuso9bIhvH2UlxF-Xm2_Ov X-Proofpoint-GUID: g8w3Lm7NdVTuso9bIhvH2UlxF-Xm2_Ov X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255148368116600 Content-Type: text/plain; charset="utf-8" If VFIO_IRQ_INFO_MASKABLE is set for VFIO_PCI_MSIX_IRQ_INDEX, record this in ->can_mask_msix, and use it to individually mask MSI-X interrupts as needed. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio/pci.h | 1 + include/hw/vfio/vfio-device.h | 2 ++ hw/vfio/device.c | 26 ++++++++++++++++++++ hw/vfio/pci.c | 46 ++++++++++++++++++++++++++++++----- 4 files changed, 69 insertions(+), 6 deletions(-) diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index d4c6b2e7b7..e3a7d7bdca 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -191,6 +191,7 @@ struct VFIOPCIDevice { bool defer_kvm_irq_routing; bool clear_parent_atomics_on_exit; bool skip_vsc_check; + bool can_mask_msix; VFIODisplay *dpy; Notifier irqchip_change_notifier; }; diff --git a/include/hw/vfio/vfio-device.h b/include/hw/vfio/vfio-device.h index 8bcb3c19f6..923f9cd116 100644 --- a/include/hw/vfio/vfio-device.h +++ b/include/hw/vfio/vfio-device.h @@ -133,7 +133,9 @@ struct VFIODeviceOps { (ret < 0 ? strerror(-ret) : "short write") =20 void vfio_device_irq_disable(VFIODevice *vbasedev, int index); +void vfio_device_irq_unmask_single(VFIODevice *vbasedev, int index, int ir= q); void vfio_device_irq_unmask(VFIODevice *vbasedev, int index); +void vfio_device_irq_mask_single(VFIODevice *vbasedev, int index, int irq); void vfio_device_irq_mask(VFIODevice *vbasedev, int index); bool vfio_device_irq_set_signaling(VFIODevice *vbasedev, int index, int su= bindex, int action, int fd, Error **errp); diff --git a/hw/vfio/device.c b/hw/vfio/device.c index 9fba2c7272..d0068086ae 100644 --- a/hw/vfio/device.c +++ b/hw/vfio/device.c @@ -85,6 +85,19 @@ void vfio_device_irq_disable(VFIODevice *vbasedev, int i= ndex) vbasedev->io_ops->set_irqs(vbasedev, &irq_set); } =20 +void vfio_device_irq_unmask_single(VFIODevice *vbasedev, int index, int ir= q) +{ + struct vfio_irq_set irq_set =3D { + .argsz =3D sizeof(irq_set), + .flags =3D VFIO_IRQ_SET_DATA_NONE | VFIO_IRQ_SET_ACTION_UNMASK, + .index =3D index, + .start =3D irq, + .count =3D 1, + }; + + vbasedev->io_ops->set_irqs(vbasedev, &irq_set); +} + void vfio_device_irq_unmask(VFIODevice *vbasedev, int index) { struct vfio_irq_set irq_set =3D { @@ -98,6 +111,19 @@ void vfio_device_irq_unmask(VFIODevice *vbasedev, int i= ndex) vbasedev->io_ops->set_irqs(vbasedev, &irq_set); } =20 +void vfio_device_irq_mask_single(VFIODevice *vbasedev, int index, int irq) +{ + struct vfio_irq_set irq_set =3D { + .argsz =3D sizeof(irq_set), + .flags =3D VFIO_IRQ_SET_DATA_NONE | VFIO_IRQ_SET_ACTION_MASK, + .index =3D index, + .start =3D irq, + .count =3D 1, + }; + + vbasedev->io_ops->set_irqs(vbasedev, &irq_set); +} + void vfio_device_irq_mask(VFIODevice *vbasedev, int index) { struct vfio_irq_set irq_set =3D { diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index a49405660a..714d37e227 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -535,6 +535,7 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, uns= igned int nr, { VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); VFIOMSIVector *vector; + bool new_vec =3D false; int ret; bool resizing =3D !!(vdev->nr_vectors < nr + 1); =20 @@ -575,6 +576,7 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, uns= igned int nr, kvm_irqchip_commit_route_changes(&vfio_route_change); vfio_connect_kvm_msi_virq(vector); } + new_vec =3D true; } } =20 @@ -584,6 +586,9 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, uns= igned int nr, * in use, so we shutdown and incrementally increase them as needed. * nr_vectors represents the total number of vectors allocated. * + * Otherwise, unmask the vector if the vector is already setup (and we= can + * do so) or send the fd if not. + * * When dynamic allocation is supported, let the host only allocate * and enable a vector when it is in use in guest. nr_vectors represen= ts * the upper bound of vectors being enabled (but not all of the ranges @@ -594,13 +599,20 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, u= nsigned int nr, } =20 if (!vdev->defer_kvm_irq_routing) { - if (vdev->msix->noresize && resizing) { - vfio_device_irq_disable(&vdev->vbasedev, VFIO_PCI_MSIX_IRQ_IND= EX); - ret =3D vfio_enable_vectors(vdev, true); - if (ret) { - error_report("vfio: failed to enable vectors, %s", - strerror(-ret)); + if (resizing) { + if (vdev->msix->noresize) { + vfio_device_irq_disable(&vdev->vbasedev, + VFIO_PCI_MSIX_IRQ_INDEX); + ret =3D vfio_enable_vectors(vdev, true); + if (ret) { + error_report("vfio: failed to enable vectors, %d", ret= ); + } + } else { + set_irq_signalling(&vdev->vbasedev, vector, nr); } + } else if (vdev->can_mask_msix && !new_vec) { + vfio_device_irq_unmask_single(&vdev->vbasedev, + VFIO_PCI_MSIX_IRQ_INDEX, nr); } else { set_irq_signalling(&vdev->vbasedev, vector, nr); } @@ -630,6 +642,13 @@ static void vfio_msix_vector_release(PCIDevice *pdev, = unsigned int nr) =20 trace_vfio_msix_vector_release(vdev->vbasedev.name, nr); =20 + /* just mask vector if peer supports it */ + if (vdev->can_mask_msix) { + vfio_device_irq_mask_single(&vdev->vbasedev, VFIO_PCI_MSIX_IRQ_IND= EX, + nr); + return; + } + /* * There are still old guests that mask and unmask vectors on every * interrupt. If we're using QEMU bypass with a KVM irqfd, leave all = of @@ -702,6 +721,13 @@ static void vfio_msix_enable(VFIOPCIDevice *vdev) error_report("vfio: failed to enable vectors, %s", strerror(-ret)); } + } else if (vdev->can_mask_msix) { + /* + * If we can use single irq masking, send an invalid fd on vector 0 + * to enable MSI-X without any vectors enabled. + */ + vfio_device_irq_set_signaling(&vdev->vbasedev, VFIO_PCI_MSIX_IRQ_I= NDEX, + 0, VFIO_IRQ_SET_ACTION_TRIGGER, -1, = NULL); } else { /* * Some communication channels between VF & PF or PF & fw rely on = the @@ -2842,6 +2868,14 @@ bool vfio_pci_populate_device(VFIOPCIDevice *vdev, E= rror **errp) } } =20 + ret =3D vfio_device_get_irq_info(vbasedev, VFIO_PCI_MSIX_IRQ_INDEX, + &irq_info); + if (ret =3D=3D 0 && (irq_info.flags & VFIO_IRQ_INFO_MASKABLE)) { + vdev->can_mask_msix =3D true; + } else { + vdev->can_mask_msix =3D false; + } + ret =3D vfio_device_get_irq_info(vbasedev, VFIO_PCI_ERR_IRQ_INDEX, &ir= q_info); if (ret) { /* This can fail for an old kernel or legacy PCI dev */ --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255219; cv=pass; d=zohomail.com; s=zohoarc; b=Wk1nphGgpvQAwYVuK2Uie1KKk5HZV39iMQNYDxAsCi5I5FJIdRPu1B2ngkezX8jjuGHKXTdGG5kTkpFPz8bIcews1KfoUDmp5AlOy5VrPybtpt99q/w4UoXrKYDvxhxuxM35qL4JLqj8cjnCTEPHOaWSiPZYtUczIfcmfO+vG34= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255219; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=TzOG5lHzWpuuyob4EorkqBxc9Wvjp4DQ/aa8BE5J7Cg=; b=ncUjiQXv6q9vfDIIre9L+Q3IN3bbsR5xK9zjHc6KaEauyzlGiMhh0C4tMWDglHoTE7JkJt+4F2MgP4ufBPp747PkLNwA3tF9Q6SDJKGBERDoHsh3WxCsZ/VxyyvZfjnw994yL4zBN1hrJSMM+46EhUBuHKQhI8rlBxL/UksJHe0= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255219821995.7684181224303; Fri, 6 Jun 2025 17:13:39 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9X-0002vV-4L; Fri, 06 Jun 2025 20:11:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9U-0002uO-0e for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:12 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9P-0006NT-7K for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:11 -0400 Received: from pps.filterd (m0127840.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556K9XAf027707; Fri, 6 Jun 2025 17:11:05 -0700 Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11on2127.outbound.protection.outlook.com [40.107.236.127]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 471g813xsu-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:04 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:03 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=TzOG5lHzWpuuyob4EorkqBxc9Wvjp4DQ/aa8BE5J7 Cg=; b=bgQfeR35fQEk9MKZBlACXtDDzR4UGVupb8F8yYFy8QjaU1P54gIDFPp3a RBXn6KHsnEGhxOvu/ObukSzPEv1hcLnYVlwjL9sia7YiH/tq+hYFHLJl7ZeOSjJD owH0xNx2ZbGZcNmj6fXzQeHxT1YozOSOACD2/ljePF47t/U165kcCxtxh9Z6T1hD TDqw3gOaLBbiH5YieS4xni/qhROxow7AnQJRX6eSROj/OuzoCeSgKW7A2GiiGbbO Z/vnrZ1U12a9YAf7vjEGHMUVNXw8bgBAx5IDzPiB0PUWRf1NlVteEBw1HU9UlYgM RlQiaRnGhQUrgDGq/4q0lu/3QfEJw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=bpnpqmdVQxSI192MMqLKSiegLemJc1YsY+ZSQsqCiHYT5rZwaeSmA1uspwCLh8TXljq/2cHd7UX/d1gi4IVLgsfBw++bhCk+EGWyl31z4mOyjwcvAjT9OBa3us/MPY8Mzy3FQVQ3R7ADJD/eUsUki2UxrcTphbGAsphYIqSaz4P25XXfDsfExvofUT/YSDT6FmDJ1MVpkujFhDErfK3HPw1lfqaJeo4cn829bJKFzgY/xSmgB7v6WF+I1E6ZoNgjwyFlDCboWGuVH8vuv5SbrEvf4kiv6VMpZHbSfuDlLZ5W384NUDAzpmhOmPjmOwBStRD2HF+jVZPPe1aaGD6fLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TzOG5lHzWpuuyob4EorkqBxc9Wvjp4DQ/aa8BE5J7Cg=; b=x93zdT+S7uqhGPpZmG9EcbEzZQCARfj0m+YNq3K0e7UeBoPw7JdE31kQD4ZycY0fy6TRMpQqpowy2bpfOpR6bQA9skee+BZZBd11acr++dB8JxaVJZikgR0bPb/KiIlIY2QF6Z7la3LU7Q3kjg5qVmDqI+No2Gy2gSAFuTKJ9ryQ8dNmv2Pakul5evEzZUf5J1WefLXXRceljUy7YecXX/UL/+cJ+jidi2mHsmp12F1PtGClDKqlKXEL/FPhEnKek18veM8A5J9Cx99Zlzh04oVbNDVNktL8o16ntaN3ocuEc+tBVTwzWgb+474NJZx0/XNyBC+AyjA603PJ55xgxw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TzOG5lHzWpuuyob4EorkqBxc9Wvjp4DQ/aa8BE5J7Cg=; b=DtbLd46tq9y2kmRelBfbmOaUMRzZBQFvoLB2CU5taCcTPiIBvQz3WWFsnyYcdn7lh5cGn9Q7ceAokNizg1vxM1KEooJuYSmd9ZeynUSUWNdb7ydziVdak/XY3Mo1MS/21B1wV+Mzr5TjcC+RrHBzFiugmZKrQ/oW9cBNkNmgvYGZPim+JWG+N1BJ6FHae7l/yaKkcwXrryGDI5m88AKi9rzZ7DV3mlCaNpfGGoysi0JXhWhw/RN2+7s3IfLs2Xo03sUWhXVWHL7m/vM0x4qaC7Al5a2IB4N6bk8nmH1z5PCl4ZWB6cfswNmSpnpYQEfaz+yG8OhopzRT3WNZm9eRZg== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon Subject: [PATCH v3 03/23] vfio: add per-region fd support Date: Fri, 6 Jun 2025 17:10:35 -0700 Message-ID: <20250607001056.335310-4-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: ec4b6eb2-c8da-4c24-4e2d-08dda557ca3d x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?xWx7njxUsfMjZASckcywSZAjTnX+2ZRHKtM+3dU2DoqJ2tClufzUaKFyAIFz?= =?us-ascii?Q?1fvl3mvLOdo0BLTw1To3orvXChG6VfNuVxQVK6wN7B3MQSdeLoFTcpy5eQBZ?= =?us-ascii?Q?U2rAsAO5eB7e1Q0hEkrUg9QOrvqTrQfCRW8jyQ0rnyFKXXTkersJyI5RNHVb?= =?us-ascii?Q?W9tXMYdKyCWuDtvHF8TucTdE1IUGpjwSgqCIs7aG+HE43pQUg+zIu/mbOD1i?= =?us-ascii?Q?QykOARb2tcBJdqL914BUF+Z+En5Qryknr7gRY1DWWmXkUMQh3q8q48JSBoL7?= =?us-ascii?Q?8L3T3G0mdrs7MyWvvQlkMXpY1yoCwg3FEd2ZIs1G3zpSUQ6croRS44B1GLaK?= =?us-ascii?Q?kIlpMG7kg5vTMEZmOsLRl9abllSB0B71lF6QemGAAK8gAM6uJG7XsBy8Ohw7?= =?us-ascii?Q?PtmKXaKR8PWzjns2a4Nen3DBKdJYAKyvivuImO3Pa590N5Y9VPfXBQkWHdWt?= =?us-ascii?Q?HQDWOr95WLJgejyVljyv7H6jQxGkC3p4jJNDHMHvh+47StTZ3xd5cNi33wQH?= =?us-ascii?Q?0BNhLHAN5BkLJQgySL61IFbFiim9RRs3/hT34DFDzUGG9ncnGzjeel2y8s7o?= =?us-ascii?Q?AjMy8iCxDGvYa81nCBJio5FKH3pz4j4w97OzKPP7P7GQGkdtWjnZVKLtxAcC?= =?us-ascii?Q?Sp6D9M1W5bUeZRDBq5Fxp9HCqIlDMht8L1DvYQybkJEW24/lGPbMwmA3LMrC?= =?us-ascii?Q?S9VtMQrDxvCjuW9ydNu74gi+3WlAv0ko8scs4YE/5MnnPtJJv8vqUFa6eCGX?= =?us-ascii?Q?s9AgshXlWgz3yrZrB3mkJcEyJmulWK1RpWg4il3Uz41Mlip6oxwuALGwewsn?= =?us-ascii?Q?YNjas+QWyvyEZpUjbrnuQvPhPArgorgPGxjNrCIwYSFo3M4lmYxIw2tww2E1?= =?us-ascii?Q?8V4HXaHsJkkMouVFEjwsbFEUGZT2d1XjkNxSDh0E3p1VmUqiuwEyjqTqnX/j?= =?us-ascii?Q?OWjrVZv93VZmGir5fXbi0EKq9iQO2cMQFEobqh20IhhhNgx2vio/dtD6QuKn?= =?us-ascii?Q?bjbmAo6JsUhUpDhtoeYYPt5gClqVBwLLzytitYLb0+4lp5uQahkhmX/EhVg4?= =?us-ascii?Q?ZFMc9Uyq/7T2ZysEStMlk2sRwxQaRYrA16xumexUeAYaSg9KvqM57LVsmngp?= =?us-ascii?Q?ml+1STgC67KvVISPwwkfOTdEz7R00+7mPjsO3ySIjyh56jb4Rkd6ipzaus7P?= =?us-ascii?Q?4UlWUdm9YkyPwfTg/nBBJhDntiec1J/xPMSGvPZfwidyTNuKqdExZ9O/AaSE?= =?us-ascii?Q?EzyBqCsx0jaNISYLg+UOVPfaS3ShkqwY+lfsYHtLVdssrXhl0FufCIjk784t?= =?us-ascii?Q?IJAXv8UVwXC5H29kRknKaX/yTzrXKRFmR5wH04FlysoatsTDQPRaUtS/N8//?= =?us-ascii?Q?nnHI2IzZw5Ug0xAZSk5XWQvDYGW92gaC6WYrbFek7Cxr/wKN4yDYZfbdpB/9?= =?us-ascii?Q?imT+0L2m8eg=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?G8k1s1WUCzFHSPxdGjFCBrxpaLNPsKDLbprAxVml0Wy4F4Cye54r3kAPi6Zj?= =?us-ascii?Q?cL9wmq1bz4RklZ/hYoYzUmOP6SoCxaa5EmwTbmqt0YK3osX0okAGWs5GHsZB?= =?us-ascii?Q?WxcLcNDuyJEmHZOwp7zdLmK0EVNaiTPVMqO/avxhtPQv9fmBYYre+F4R/jcC?= =?us-ascii?Q?Zz3rmyIHAZJO526skQ0FwtLv+XhfWwgYBJ+016FUmUfVdJ1MnuYacfM2rekB?= =?us-ascii?Q?ULzAa0pBFGiKdc29RbfaoFI6kkno9VnJwmUEW9Q6wDnypXNZ1nRMrt1w0hwB?= =?us-ascii?Q?vN3JGIH2Oyc8Pprx2kJvOl3BHbSyiRK4gI28XY8TMne9DQ2Ax+yOJCAhMWW3?= =?us-ascii?Q?nQkUfKBa1JNkxwyU3xqjGWevutXHe02w4j1Y4E4Xr6LFoF6q0mXPLCwgSM/Y?= =?us-ascii?Q?JOF1Gh9HxVvwqey5tUt2nG6nm7owc55D6BTEQUV5YDNZN9+lBZLmXOfI567C?= =?us-ascii?Q?ayhO9he72ZcwDSAptgQG5flLZr2cxti+ItKb1HMgDJy/ubASEzq7R1VFb5lq?= =?us-ascii?Q?KNwv4/nITSYnk44Oy+GrBErtWb/efMYl096eW2BGlLcoEH4bPuKqkt5fCwix?= =?us-ascii?Q?k++pI9uB2FRvVm7L1DRq6SR5mQxL1hsZQr/ZUjR8PNnZNxnRG9QCQKLl1msn?= =?us-ascii?Q?LQmjqJeZDNXZTdrUYfvwrfTe9nWQ9bK2P6KdQsXi03hiJvZPtMPNERig4SfO?= =?us-ascii?Q?fVUbLE9pVB7ZxHNlOnXLteR691Jg4Ni1PlSI0hbBMIlfS8WfbcRnLZnFiOZe?= =?us-ascii?Q?LfvHqXbJkriQj+3602MHXx6KxymSzxcqkHOiSUXPPpCn2+3GbLveVQz8z5A0?= =?us-ascii?Q?S/4PNc2C6W/P4lqxyeVE2dSV88BnJN0SERVG5WhtHdMKDmgVUlNwEAiCx8P6?= =?us-ascii?Q?ElA6OQRiy3EY50qNa2sXAvkyReTqMAZbZk41x7Z7o3NScDQO7e/6gXoHolzX?= =?us-ascii?Q?bsO3mM6qB1z7gQQ19rhjwX+Xl/oRWayZVYMGMOUw5UcMmdE0ez73X5XfhwTE?= =?us-ascii?Q?SIi//j2SC7XMSa1C+awA0zNOY6D8CEOkfdtrDUi0QANyUsHT9qaGam8eA/kn?= =?us-ascii?Q?/2jQEeAhlUxZu43mkUELy7AImlos6NtOgeysVAmPyx7E48GwHagBO+CfSQpS?= =?us-ascii?Q?kbLtKBouvxGHFWXZgtF2YS/fP5RoG3qoLk+KKUU6Rg/RaMgcX87BFaZk/fuf?= =?us-ascii?Q?xG45rbo311BXRZp20qUGUqWWqgrarLZCKTWBxX2y59JYiFoXEKyenn4+vfsB?= =?us-ascii?Q?htyrpddUX4DDBX6RtgHEOy0kyvHa6g7W8us//dnBy9AknbscCqFHzz/izdLH?= =?us-ascii?Q?vunAYkgRx09FBhPMTrYw8IQLOwjQi6boTHj+YkoFV88eIhAM8Wo/IoPSsZxX?= =?us-ascii?Q?JbGZzoSTkQF8Ik9ZwH/9iLZHF5ndl1uiF09nJOo5vNQ1EcDtMPWwMjCXWPe2?= =?us-ascii?Q?14fl/rkxXsPVRGkh1HYRgvLTjHUI/q68GTPG5C5QRL5+nGWtD04nkPzknF76?= =?us-ascii?Q?2kwKz73j0NfRZG3gBFCjtxFM/iMW6t+CwLzrvPGThrDJvH6noVh3idZGlA6U?= =?us-ascii?Q?J0Yx9n+pvd5CCygVbpLv1frtncnM/MGZ//Tpm7S+?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: ec4b6eb2-c8da-4c24-4e2d-08dda557ca3d X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:03.1887 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PCiVl9mGEa3X4n0WnP90Hxj/U9i0ebh8yEby6qEZawn3p3tM5FMBPn3uYqvkZKYF8vwwvE/OvO2I6uDUBCmTlQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX9boENOywHE7m cCsXf1Oo7JYAK+pRYCshQ5dwAzZRGZ4lQdaFz6celeLzm6HtUUrQiFq0dp/rBc+QrqpGS1e/8Ad RUjEJcijdd4MF9Hk/ootxIdhhIB+rxrw8SnpAd+cLwneEEf/znhJmsPv/dRFZAoTt0ppVhrjE4L FUJw8/fFWiSYXWxkDDTrnlm32PWnupexm7U19S+MDA3U2sudlwXPw0iKiu9z1PWF5ysRdZOu+e1 fVpAX9ugx07kjtk0BiQVYBzXA4wC8VQrVe/xl8J3+hNSlcw60YBMe6wDud0BYu1dQPuSU26htUz V/hcUgBTPO0AHq1zY5LG5adUhJ9FA/5V4fCGI0lb8HUqylK79mXnJUFguwljmpnxBgkgFcC11tN WmznWzRnIeqTEhmNdimmvWoz+ANUsebXlWe/q+baWyylz8Q6XEriX0UI57nuE4zjbgouoZER X-Authority-Analysis: v=2.4 cv=SY33duRu c=1 sm=1 tr=0 ts=68438398 cx=c_pps a=PuycMPbAwArgayjkkQxHog==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=64Cc0HZtAAAA:8 a=w5k4ZZm-TmW3-Hdf4mgA:9 X-Proofpoint-ORIG-GUID: vIXB9IIhEiRzMOCAa4n3atmF2moY4hnG X-Proofpoint-GUID: vIXB9IIhEiRzMOCAa4n3atmF2moY4hnG X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255221080116600 Content-Type: text/plain; charset="utf-8" For vfio-user, each region has its own fd rather than sharing vbasedev's. Add the necessary plumbing to support this, and use the correct fd in vfio_region_mmap(). Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- include/hw/vfio/vfio-device.h | 7 +++++-- hw/vfio/device.c | 29 +++++++++++++++++++++++++---- hw/vfio/region.c | 9 +++++++-- 3 files changed, 37 insertions(+), 8 deletions(-) diff --git a/include/hw/vfio/vfio-device.h b/include/hw/vfio/vfio-device.h index 923f9cd116..5cb817fd6a 100644 --- a/include/hw/vfio/vfio-device.h +++ b/include/hw/vfio/vfio-device.h @@ -66,6 +66,7 @@ typedef struct VFIODevice { OnOffAuto enable_migration; OnOffAuto migration_multifd_transfer; bool migration_events; + bool use_region_fds; VFIODeviceOps *ops; VFIODeviceIOOps *io_ops; unsigned int num_irqs; @@ -84,6 +85,7 @@ typedef struct VFIODevice { VFIOIOASHwpt *hwpt; QLIST_ENTRY(VFIODevice) hwpt_next; struct vfio_region_info **reginfo; + int *region_fds; } VFIODevice; =20 struct VFIODeviceOps { @@ -172,10 +174,11 @@ struct VFIODeviceIOOps { /** * @get_region_info * - * Fill in @info with information on the region given by @info->index. + * Fill in @info (and optionally @fd) with information on the region g= iven + * by @info->index. */ int (*get_region_info)(VFIODevice *vdev, - struct vfio_region_info *info); + struct vfio_region_info *info, int *fd); =20 /** * @get_irq_info diff --git a/hw/vfio/device.c b/hw/vfio/device.c index d0068086ae..29a8d72deb 100644 --- a/hw/vfio/device.c +++ b/hw/vfio/device.c @@ -226,6 +226,7 @@ int vfio_device_get_region_info(VFIODevice *vbasedev, i= nt index, struct vfio_region_info **info) { size_t argsz =3D sizeof(struct vfio_region_info); + int fd =3D -1; int ret; =20 /* check cache */ @@ -240,7 +241,7 @@ int vfio_device_get_region_info(VFIODevice *vbasedev, i= nt index, retry: (*info)->argsz =3D argsz; =20 - ret =3D vbasedev->io_ops->get_region_info(vbasedev, *info); + ret =3D vbasedev->io_ops->get_region_info(vbasedev, *info, &fd); if (ret !=3D 0) { g_free(*info); *info =3D NULL; @@ -251,11 +252,19 @@ retry: argsz =3D (*info)->argsz; *info =3D g_realloc(*info, argsz); =20 + if (fd !=3D -1) { + close(fd); + fd =3D -1; + } + goto retry; } =20 /* fill cache */ vbasedev->reginfo[index] =3D *info; + if (vbasedev->region_fds !=3D NULL) { + vbasedev->region_fds[index] =3D fd; + } =20 return 0; } @@ -360,6 +369,7 @@ void vfio_device_init(VFIODevice *vbasedev, int type, V= FIODeviceOps *ops, vbasedev->io_ops =3D &vfio_device_io_ops_ioctl; vbasedev->dev =3D dev; vbasedev->fd =3D -1; + vbasedev->use_region_fds =3D false; =20 vbasedev->ram_block_discard_allowed =3D ram_discard; } @@ -470,6 +480,9 @@ void vfio_device_prepare(VFIODevice *vbasedev, VFIOCont= ainerBase *bcontainer, =20 vbasedev->reginfo =3D g_new0(struct vfio_region_info *, vbasedev->num_regions); + if (vbasedev->use_region_fds) { + vbasedev->region_fds =3D g_new0(int, vbasedev->num_regions); + } } =20 void vfio_device_unprepare(VFIODevice *vbasedev) @@ -478,9 +491,14 @@ void vfio_device_unprepare(VFIODevice *vbasedev) =20 for (i =3D 0; i < vbasedev->num_regions; i++) { g_free(vbasedev->reginfo[i]); + if (vbasedev->region_fds !=3D NULL && vbasedev->region_fds[i] !=3D= -1) { + close(vbasedev->region_fds[i]); + } + } - g_free(vbasedev->reginfo); - vbasedev->reginfo =3D NULL; + + g_clear_pointer(&vbasedev->reginfo, g_free); + g_clear_pointer(&vbasedev->region_fds, g_free); =20 QLIST_REMOVE(vbasedev, container_next); QLIST_REMOVE(vbasedev, global_next); @@ -502,10 +520,13 @@ static int vfio_device_io_device_feature(VFIODevice *= vbasedev, } =20 static int vfio_device_io_get_region_info(VFIODevice *vbasedev, - struct vfio_region_info *info) + struct vfio_region_info *info, + int *fd) { int ret; =20 + *fd =3D -1; + ret =3D ioctl(vbasedev->fd, VFIO_DEVICE_GET_REGION_INFO, info); =20 return ret < 0 ? -errno : ret; diff --git a/hw/vfio/region.c b/hw/vfio/region.c index 34752c3f65..cb172f2136 100644 --- a/hw/vfio/region.c +++ b/hw/vfio/region.c @@ -241,6 +241,7 @@ int vfio_region_mmap(VFIORegion *region) { int i, ret, prot =3D 0; char *name; + int fd; =20 if (!region->mem) { return 0; @@ -271,14 +272,18 @@ int vfio_region_mmap(VFIORegion *region) goto no_mmap; } =20 + /* Use the per-region fd if set, or the shared fd. */ + fd =3D region->vbasedev->region_fds ? + region->vbasedev->region_fds[region->nr] : + region->vbasedev->fd, + map_align =3D (void *)ROUND_UP((uintptr_t)map_base, (uintptr_t)ali= gn); munmap(map_base, map_align - map_base); munmap(map_align + region->mmaps[i].size, align - (map_align - map_base)); =20 region->mmaps[i].mmap =3D mmap(map_align, region->mmaps[i].size, p= rot, - MAP_SHARED | MAP_FIXED, - region->vbasedev->fd, + MAP_SHARED | MAP_FIXED, fd, region->fd_offset + region->mmaps[i].offset); if (region->mmaps[i].mmap =3D=3D MAP_FAILED) { --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255268; cv=pass; d=zohomail.com; s=zohoarc; b=bDdxreA6kTRzvZu6gXWoU5Ul5zdIrUVBL66aZYay8c6r8MUOh8gIis4xmzfEbfQzBFORGs7AQR69oDVBZlQ92DeMY+sqbE4vTSS4J7XAPSd3iKAseyw84/+yp/Hxc2oZkZ8O0krbGanLVTqPUilWtvxLHp8GuWwLPk3PFMeXKQI= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255268; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=TuJSdzq/oMTOZ0c8W2bi4glELx/Lv8O4XX2pLqNHz1o=; b=Fxrv5Y52Vz9fs2+hqRAFm7jbg7yisOx++g3aG8TBlGd3uUJIx0IZe8uCFGXb4HX1Z8GWW7xqaQhIpxCNDZcrZgVpLIElqE2IJuOuBRbV+lmitXYXWLugEzFFxUVfpRYV3Ondf7bcyelHLZaldolWPCZyuuwSiQHqWoCi3iJVGmA= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255268196772.253979743011; Fri, 6 Jun 2025 17:14:28 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9Z-0002wo-Rt; Fri, 06 Jun 2025 20:11:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9V-0002um-Bo for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:13 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9Q-0006Nb-BN for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:13 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556Lu6G9022566; Fri, 6 Jun 2025 17:11:06 -0700 Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12on2134.outbound.protection.outlook.com [40.107.243.134]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 472njsxqah-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:06 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:04 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=TuJSdzq/oMTOZ0c8W2bi4glELx/Lv8O4XX2pLqNHz 1o=; b=S1ik48KhYdjUWn+7c8M6JQHmrAU/ldwnNMriIzPmt7Hb3XPHEOddgrpwx X9hz8fqxtItfjSJDfJNekKWIvkY8vw2iXhFt1rUuQo/y48ls+S4WCG9bFNAzkGBQ hwrqIG+wr6WcJhFEIin4tqEMkWfTd72Iu53f0XKyQ3VxCmW+Xu336Ck7niCiZ5Oj VZ4jeVCvFqpk5KrRhnIGrFEPF4AXjzQDkHwF2IMd7S4QETZXJ7khtZMgL0ADhCMS TvWpCLDBR+Xi104Rz7vxnNqFja0fCkGPoQHHPHWqcT5AlHEtXOEcEA9pHCjil0MB kIeQuEu6jI6ZQArkJ2Bc2OP7abf8A== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=FMRc8E3onWZn/+FU9aPOZe2YS1HQPw2WOZWU2hWxZBQQmCfbOGzjSGwt02lV8+/C9l68VjKy0L1LCtZB1NDQ9mFY/u7Pd3UlY/tMdB9RUJHsNcFpsdfgNlYd0AwcsJ7qX51d404zMylGS9KX/DIrD1qVfVQ2R4/E5XLTzHo4d/WtTJXBxdOVJFzTNvbMeD0l4NXNyq1WXuLeXP+SWMzi1bcoOBaIFM1CxlTemrJgF8fiqa25t6wBZ1ceBK7klELl8Mw3Hs1FFb/AUxBLlEOLi0Tpq/V06K/tgubWW42KmSdGQ5F2r5409zuOitLj66p+PSPFA74JXQcbw1ccek+3IA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TuJSdzq/oMTOZ0c8W2bi4glELx/Lv8O4XX2pLqNHz1o=; b=OVrmhDVD/KBJbZ9gTAPNRmpwxfkqHCWc21SZ+DU6CbVId8nwS1le57ZxjoEjDJzAONWoHi3t2YgFGyHuBoIx79XfUho3dQbVarRgyefTzD94/WMfadEyKwvyQ+etkfOsh8DBP6IAiq2uZnFYnKVgqy7gUb9LxtGIuezTk6ELOp9HyEZRdr/kLBjwSvEiO+mIKSnea6dtNauDCc6i2hiJhViLLZBtVfDVIxIzLve5zvMoROqfX+Dz8RQ/YQNViRvZbit5tYLUhnjGgZ8EgUSJMvshl+vaUYjiGxpDLXECbFjp1bn28CYRzKLF6Y0rkmXr+yD4U6e6h0nU8xg2Rhz07g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TuJSdzq/oMTOZ0c8W2bi4glELx/Lv8O4XX2pLqNHz1o=; b=wdn4GdKsqc0olMhEoh7wonNglTSkCWt7wn3FrNwjejVdXpklIicJkwRfiA5DkJLgwF2TR7wuYunM50sLnRn+PTx5NU8qMstnSibIeTGDtIc/6fL3gxwhuTuhyv9yO0qkSH8za3YHAYE3jnPkFSdSCCu4Dto9fuwQs2ioEhZRnI0iFlNNJUFhfBnHCD11Qe6BdOmBGomf+3xYOj1p4q/rTIw2VHS1pw02eOilkUNdL8rgW5HN869M20E8dp+/W+RCMI7cn8kbUIOzzB0H2+NMm5P5vECaM3Dv+CoA4oPm/lw/e5ME2DZQruNjGLpzm3l5xqoEEGzRCuXVNSgLH7uUtQ== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon Subject: [PATCH v3 04/23] vfio: mark posted writes in region write callbacks Date: Fri, 6 Jun 2025 17:10:36 -0700 Message-ID: <20250607001056.335310-5-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: 0c2c3a13-20ca-44b9-a836-08dda557caf2 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?dldNWGpOcWZWQmt4TUNYam0zTm9FVkZBWHRVRGh3c1I1bWtHT1lPckNHM21u?= =?utf-8?B?aFJUWmJWSGhJQlMxbFMrNjdJTTVwcEJUeHZlREZMY2RYTWRVTVh1Q29RNEFn?= =?utf-8?B?dTJvd3VQa1JaUmRsVEJCLzdCVEZIWU0rTmYxakcyaTQ3blBkT3AzTHFNbndn?= =?utf-8?B?TWRyZVVEOG94VHI4cUdnRWdiS04yTk4zSkl3RFYrVXpWSTZJR0QxWDZEaGdE?= =?utf-8?B?SzRVbGtPeC91M3RaN1o1emZzK3U1MGtGT3VhbmV3RlFUcTk2TWxCVmhYdTV6?= =?utf-8?B?VVlHLy9sT29JWlRXcitxZzB2c2RmamZSRnB4NXNOV01oM0paTktvbUlsdFVF?= =?utf-8?B?ekFwdXYyL09nL25HLzBKbTdwUUxpVVNWa1Y2UjMycU5MRkdPZFZwU1NobjZ5?= =?utf-8?B?RmxjSmJkYm5oN0kxWUFWYTFyZnpZbzJzMzVSQWE0YkN4aVB4c3pFMTRlYTlx?= =?utf-8?B?N2U4MDRmY1hpS0tldnR0NWZHbmI0c1UwRVFwQmQwc29JRy83cnZDRXdxemNP?= =?utf-8?B?VktNa0RZRnVsYVFiS1ZUNTVjN1hiZjVPd1F5M25zcTN1cXZ2MTlvZFk3UnpE?= =?utf-8?B?Uy9EN2FKcGtYdTdDYisvZnhrQlZzb3NleXdYZDY0ekRVdm05aUdjS08wVFpz?= =?utf-8?B?dHV1blZ4eDNqTnh3dUpvd2Z1VDdsN3l3NUdTb216UWZNQmFzcDZjWCtNci80?= =?utf-8?B?R0o0eEpHeVN5YlJUVldiM0xNUnEyUzhOamtDMjlPWUpmOERFZEhtUTNJZDha?= =?utf-8?B?djMxUUhEcDRwSHIrRDJOaHZCNW94QW1TN29NWDB6blBqY0tobUJaWTY1d2pr?= =?utf-8?B?cVprMHpnYmNic2UrRDgzQy9Sd3FWejZyQnlaaGMrd3ZDc1g2eFFSb0tBSXlt?= =?utf-8?B?Z082WVYrQ0NuRUV0N1dMV0lmTmllSWF5Z2l0QkZlWm9heWo4UU9YZm1jejdM?= =?utf-8?B?L3pKMkVxRmhlMnN2ZGZCYU91Mnl3VDZqN1BpaWJHd2hEZG9qMEdmUFBINHMz?= =?utf-8?B?WUNHUTYwamEzWEVQcDlNSDU2VTlJWTdoaUY2djV1MEFyZkFpZE5oTUtLM0dS?= =?utf-8?B?TW1aQ3FDczFUbVM4K09ISXBoM3lNMk5FQnFRcGtXYys4MHQyWitSUnJFbG9N?= =?utf-8?B?T0JKVGV2Q3lsSWt5WkwrdW14bTJPblIzNEpuUlNORmEzMFM4cE5ZTnk2dC8w?= =?utf-8?B?UGNKc25INWNRLy9VMnFFOHRiZXVZMjBVVnBLRFo4R2xVNHV0RHIzVGtzdzlN?= =?utf-8?B?Nk5Rd1lFRVM3VWU1dXRsb2puS285WlcvWkY2enV2TWx0dXdKa0tFUFdTL2g3?= =?utf-8?B?T2o2M3Q1TEZQa0E0NGFtK08rbVVWZnA2dlZrUU5lMGtTVDZrRzZxR0JzOHhS?= =?utf-8?B?cnI3dTE5cFI0OGdRL0pqRWFDeEkzSmFMaHZZU0g3aHFmbmJ1YW5HclJaakVr?= =?utf-8?B?NHN3SlJhdE14VXJXbmZqTEc2bXdNS1lvd3drTm1LSzhJRmMwSmpQZGVSL3JK?= =?utf-8?B?SGpBV1FLN2prM2JnWTcrRkpISk42blZ3dVBSOFlGbXFKc0Qza1ZLOTErZFk4?= =?utf-8?B?ZXpNZ3E3NU9Tc0ZVVllRdGFYblBqc0dnZ0NYSyt2L2hLdWJvbjhDUXIyazZT?= =?utf-8?B?SG1OTmlLdnJxbjhvblMxNkkwU1c4WG45OWRzbW1WWStDYUV2U2g0VzM4NmJs?= =?utf-8?B?b1BPRFJZVzlzeFFlcEgzNlI0RG4zU0FoMmdYMEZGajVab1pnZHJNanRPbzRx?= =?utf-8?B?ZWlWcGRNMHkwNi9ISktNRVFYUEF1ZTk4cjk4ei83cHI2bTMyenBMMzRuK04r?= =?utf-8?B?T09SUW9TOUFCLzI4WHhNUWhWQTBJZmhhTXRQc3NLNExUUitSNjUzZE1hbTFI?= =?utf-8?B?ektKcTZ1SUZhNnBQYzRZOVc2a2J2Wm1IZ0R4S2I2U3h0TlQvU2YvUmZ6OGp2?= =?utf-8?Q?ttMlR5pY4UA=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cnlKYmpzeGNiYjhvSkdmbEEyNFh4aFhHZHBwbHJ3blUyVmlTeEg0SFN4T0RR?= =?utf-8?B?NVMxK2JZZWpwSGl3c211ZUxGQUpPK2xabU84bnVtMW1kQjJKdk9tNUdsN0xM?= =?utf-8?B?Mkt4YjlqaVZaQ0dTZjlaK1VvU1U0Y3B2TmtqNExkMHJnaC8xV2dHYW0yMWhK?= =?utf-8?B?MUtTREZwODdFb2tzempoYStjOU5QWVFZb2puYk05YTVac01XU2JESU5jN3ZJ?= =?utf-8?B?ZDBJYnhDcm1ZNjZlRThmcm50MW1hSFN1Wmpmc2RGTU15SDVpY3JGTzlZYWpZ?= =?utf-8?B?R1FlUEZBMDlWejk0ZTl2YTBSN1ZoR0ZSWWFIOU53enpQb1lhS0o2b1kzdWJS?= =?utf-8?B?M2lUNmRaeUd3NWthNkcrWksyVjFLeHhJS0ZXZ2h5WDYrd0xrdlRPcVJ5STQ5?= =?utf-8?B?aDVranJEV2pCM0FTZnJid0lpWE5JR1h0TEw4a2pMc3FTV0ljaTBuTFJaZGt6?= =?utf-8?B?Vk1Da21CeGZNRmIxR1hlRkNISE03THZlNlFiN0VmODdlcWxONTlzMjFFTE5P?= =?utf-8?B?ZW02VDVmc21iZnVSYldRT1ErMDRwNTd6L0N5eGRDZFp6SUNKSXJYT1hEdXh3?= =?utf-8?B?dUEzdzV5WU92dEFuZUxsb0RISWF0K3Q1c0Y2YWZyUjNSZ3lCQjBzWVhnekVi?= =?utf-8?B?SFlidDE0a2ppMWh5dndwdG1KZmFYVXREYzFRMWNKcmdHcnZlLytHV1IxeGMz?= =?utf-8?B?VFF5RThhNUNNWlc2d0pCQllZUkFVVHRKdFdFejc2VlI0UjdraFVEZXlDMUVo?= =?utf-8?B?V0FCdEVpZUxYVWZHK1hnc2hiYk9LVDFzUzZ3eFpEOWNEeE1pbWRHaS8xYmJN?= =?utf-8?B?SjMxbmg2WGxCcTZRTjBlTFpvOEFlTlg1alhzdkkzUEJ1cHhtUTk1UHlwUTVE?= =?utf-8?B?b2pUMTJjdHRyK2hGSkNKdHNLazMvYmlqNjhQV3Jhdk9zUHdvNWlYcWpjb2Fm?= =?utf-8?B?bTZYRFJycnhEUVQ1eHg5bkVRSTBCUzA0NjJPb3ZzZnY0VlRSd3ZuU1Y5OWZo?= =?utf-8?B?OFpSSXVYbVFHR2hWZTIybEFtR3VtN3ZqOTQ4SUFhWkxpTk5sUDVkRTUzeEtu?= =?utf-8?B?TGl1eTF5TDNLWFpoc1o2RzcvdUNJZVRaalMrVEhlcWtzSVdld3FoSjRaNTlr?= =?utf-8?B?Zi9UM1UvZTNud29yUnhjaG5nZlpmdUhMaE5ucjM3VEh2VDYxdDY3MWZUYld6?= =?utf-8?B?NnNETGlhZnhGVmlkakowenRERFJSYUo1NTg1eHFENGRadEphVFhVOXBSR2JC?= =?utf-8?B?ZE8vNG5GalM5R1prUUVoR2dSS29WQlNBTUErMnZUOW5VZGRvYktiVWUyS0x1?= =?utf-8?B?cWxVVjZFU0Z6TlUzSFZKQVZTS1ZwRVJBdVFDYjRsNW5vLy9IY0JhcWp3YW1B?= =?utf-8?B?OU5nWmU0dG93RGtTM2xuOEE3ait5Zk12V09EYjM4WXJkOWpVY3AvRzZFSzRn?= =?utf-8?B?KzBMWTVUUU85eTI4T1RqcVlFbC9mdERNYlVhU0tCWTZyY0o5T3dacFFXeFRK?= =?utf-8?B?U1BrTDBSS2t2SUhjcXJFYkRHZS9nM2tSZ1IxbGVRYjRYbDZqaEpMS0c1NEx2?= =?utf-8?B?eWhOY1l6bHM0WUhyeXBjWWZiVjA2SGFFMDJ2MGJpUjdmSWRGenBMczg1MC9i?= =?utf-8?B?K1BaV2tFY2J0TmhzVXAzb2NWQVI2WVBxaXgxRWdGZzlxN0Z0S3pNcnRXWEFJ?= =?utf-8?B?dWg1SEpua3lzb3BHbStBdEk1ZU92WnJ5WWZYU002QThON3lTakd5dTNuSnZu?= =?utf-8?B?N2t2U2hHci9xd0s4MTRIcThhVE9WMWNTWVR4NHB0bUpkTlJ0R1pBcllLaWpC?= =?utf-8?B?Y1AxWW5jb1BjeHpRbDFtbWVtcXJ5ZnMrTjVmZWN3aFJHdlRwSWEvTkhaM3lw?= =?utf-8?B?L04wVmxKQ0Y3bzNmdXJ5RXFnMVpxMzFEVGczUHBRTEczVzFIR0x4RnF4MDd3?= =?utf-8?B?MWxRR3hBeVJSdHpZT1VTUGF0Q0ZUeStUSVdQM0d4eHdkcnpsbFFjSkgxbXlZ?= =?utf-8?B?ejdnMkZlMTFQRTl5dFM0TVNEdmRWMmtNcUc2NmdHMCt1cnZyQXdQeHAybFZ6?= =?utf-8?B?ZXk3RUNHODMwSUxxQWIvbnFCN1FYeEluN2xORzQ0NUl5YmtaWE9lTEl4QU40?= =?utf-8?Q?Z4xJF37Ox7aD0/DERXozLr/hu?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0c2c3a13-20ca-44b9-a836-08dda557caf2 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:04.3779 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: a9FT6zoVPQnIePAZvb+I58upiXQZJAEBcOsIE+0UArGaszLf38LE8E4Ef/uMaCJwtgx5twPVdeBxgEhwX+pucw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Authority-Analysis: v=2.4 cv=f9JIBPyM c=1 sm=1 tr=0 ts=6843839a cx=c_pps a=Ft05trlkDaS5pCTE1vB2kw==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=64Cc0HZtAAAA:8 a=20KFwNOVAAAA:8 a=2LkpXHDO6wTvJR3M3DsA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-GUID: iaf7qQSB7d9-6AcYK8dQLMG_2qzIxYvV X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfXwh11FnGmkJi+ I5mpBVxNJLHdc/VUExljwBQqvrZNyeomRUb+GG1n2tX5Pf4h/0PWIF5XWA9TdNcieW9BfY8ZW9Q Tt2zBK+9DIK5vPVqXegxHnqlSk7QtGY0RpwPL0X2oz9XOkDeCPdVMKJ39X2VB1Hrce7ZdkutwOq MvDFsgCiVnyEXujoj0c3urt/vCx3pc8PhxaBa31y/eRu/Kpk50Rq2eROczYDQCMVdbOsHGbA5X5 VWDSsD/GBQgbPb+k0usle58c5AiBofxPsVd7pE3gUHDcZjxcEyij4IeQ7TsyWAJtAvcVpScacdi 06De/2ZVgZ2ytSsRppQRCitCIAjkuacDm1D66BBUDUOaV6/BqF9V4tLMi8oAsqpo/ANRJyL7ie/ U/hhlwN4/+2zEnntK9VXC1kV64fZQXOxRb1ImtuB+daRWaLd9QEsnkOhb2JZiiRj3JUmtxAP X-Proofpoint-ORIG-GUID: iaf7qQSB7d9-6AcYK8dQLMG_2qzIxYvV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255271621116600 For vfio-user, the region write implementation needs to know if the write is posted; add the necessary plumbing to support this. Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- include/hw/vfio/vfio-device.h | 4 ++-- include/hw/vfio/vfio-region.h | 1 + hw/vfio/device.c | 3 ++- hw/vfio/pci.c | 5 ++++- hw/vfio/region.c | 3 ++- 5 files changed, 11 insertions(+), 5 deletions(-) diff --git a/include/hw/vfio/vfio-device.h b/include/hw/vfio/vfio-device.h index 5cb817fd6a..a23ef4ea13 100644 --- a/include/hw/vfio/vfio-device.h +++ b/include/hw/vfio/vfio-device.h @@ -207,10 +207,10 @@ struct VFIODeviceIOOps { * @region_write * * Write @size bytes to the region @nr at offset @off from the buffer - * @data. + * @data; if @post, the write is posted. */ int (*region_write)(VFIODevice *vdev, uint8_t nr, off_t off, uint32_t = size, - void *data); + void *data, bool post); }; =20 void vfio_device_prepare(VFIODevice *vbasedev, VFIOContainerBase *bcontain= er, diff --git a/include/hw/vfio/vfio-region.h b/include/hw/vfio/vfio-region.h index cbffb26962..ede6e0c8f9 100644 --- a/include/hw/vfio/vfio-region.h +++ b/include/hw/vfio/vfio-region.h @@ -29,6 +29,7 @@ typedef struct VFIORegion { uint32_t nr_mmaps; VFIOMmap *mmaps; uint8_t nr; /* cache the region number for debug */ + bool post_wr; /* writes can be posted */ } VFIORegion; =20 =20 diff --git a/hw/vfio/device.c b/hw/vfio/device.c index 29a8d72deb..ad0bac9765 100644 --- a/hw/vfio/device.c +++ b/hw/vfio/device.c @@ -569,7 +569,8 @@ static int vfio_device_io_region_read(VFIODevice *vbase= dev, uint8_t index, } =20 static int vfio_device_io_region_write(VFIODevice *vbasedev, uint8_t index, - off_t off, uint32_t size, void *dat= a) + off_t off, uint32_t size, void *dat= a, + bool post) { struct vfio_region_info *info; int ret; diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 714d37e227..d68c1412d2 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -1015,7 +1015,7 @@ static int vfio_pci_config_space_write(VFIOPCIDevice = *vdev, off_t offset, { return vdev->vbasedev.io_ops->region_write(&vdev->vbasedev, VFIO_PCI_CONFIG_REGION_INDE= X, - offset, size, data); + offset, size, data, false); } =20 static uint64_t vfio_rom_read(void *opaque, hwaddr addr, unsigned size) @@ -1819,6 +1819,9 @@ static void vfio_bar_prepare(VFIOPCIDevice *vdev, int= nr) bar->type =3D pci_bar & (bar->ioport ? ~PCI_BASE_ADDRESS_IO_MASK : ~PCI_BASE_ADDRESS_MEM_MASK); bar->size =3D bar->region.size; + + /* IO regions are sync, memory can be async */ + bar->region.post_wr =3D (bar->ioport =3D=3D 0); } =20 static void vfio_bars_prepare(VFIOPCIDevice *vdev) diff --git a/hw/vfio/region.c b/hw/vfio/region.c index cb172f2136..f5b8e3cbf1 100644 --- a/hw/vfio/region.c +++ b/hw/vfio/region.c @@ -66,7 +66,7 @@ void vfio_region_write(void *opaque, hwaddr addr, } =20 ret =3D vbasedev->io_ops->region_write(vbasedev, region->nr, - addr, size, &buf); + addr, size, &buf, region->post_wr= ); if (ret !=3D size) { error_report("%s(%s:region%d+0x%"HWADDR_PRIx", 0x%"PRIx64 ",%d) failed: %s", @@ -200,6 +200,7 @@ int vfio_region_setup(Object *obj, VFIODevice *vbasedev= , VFIORegion *region, region->size =3D info->size; region->fd_offset =3D info->offset; region->nr =3D index; + region->post_wr =3D false; =20 if (region->size) { region->mem =3D g_new0(MemoryRegion, 1); --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255257; cv=pass; d=zohomail.com; s=zohoarc; b=b2I3rTtmFlVdviKW3SdqavLR52juaFv3ZkAM7kvGKBQTVZbRdcAkLNt7Qa1CmkylxbtNHOkQceoXxpSKQkvyjNSEthLuN35Fi9+uVVZjajng9/I3NCRiPycqiQS57irhTvUF1ESRxZPZsi8fq43788Hzu3iIcoS5ODqr8Tcgj8U= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255257; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=twdQUgTlOtslwXBo5zOSpwo81vTneNtUK/aEHfy9A78=; b=KwTiPDt8YxJW8O2JTmI+h01HKnc7ZIP8UuouB/BgqxJLjXDvTRiYpIcDP5f4ASMePolUDUCBMMVLJNiWnrQyDKn9S/vy4vLLowFmhdrsr9XSX+e1t3BUgyU+v98GMdcma7+SNyIbEqr2ftk6JMJuMSfoMeJfz+kpb08jV4lwg1E= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255257256511.37558927826603; Fri, 6 Jun 2025 17:14:17 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9b-0002xb-5j; Fri, 06 Jun 2025 20:11:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9Y-0002wL-KZ for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:16 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9R-0006Nt-SC for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:16 -0400 Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556Kvtir017734; Fri, 6 Jun 2025 17:11:08 -0700 Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12on2138.outbound.protection.outlook.com [40.107.243.138]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 4747u3r8f8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:07 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:05 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=twdQUgTlOtslwXBo5zOSpwo81vTneNtUK/aEHfy9A 78=; b=zOW8wX+eVJiggtijRplZCL9uY+g2kiHQlFxTc6WOCwaIupuu6Yy8JFKWX wOwOybAVovSBu/cGsHY24svh5F147cM2ltQZApL+smxTJCz4/x5gVuKt919CCSIO 16DDHfa43hAICeahYAeN9arM27Hrk29MqP3r9i7hs4V2dFqtcpKYg5xhZf5lDO+x V9LE+yIpvHV0/B4yInjkDIF2UeRAFAxYmnzNUETjet3gbFZOjtt8UCi+zPwsu7g9 gDltygO3I20Zw8zh9O1y+bFvAGKr17TwaYDwd5IPEbJudZFC2U58EcNlfYcpq9JO a5W98kXUr5H59MP6Pu8cym2uA4Mgw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WpOFjzFIO6eYP2ADGUHZwVW1Lu20TJgTzYHsxponj7HPicQ1l7CjKgM+wgfJssAgzWwU7mLPBqphqi3pYLS2VO/i94ttHGs+v/GlVnNpxTso+kEwaNPdTVoVwtPztJ4OHgomIYi/Radteod2H/3rIq6tKUEo1SXX2VGWRSGO/8LWI4iluJBqYnsr6a3vtu5A47lTpfsvl4+KfhZbqW6FIG8kTCN43xdMREo5uHFHGHpwzx57FAyRg2JtGsiXv+sxdPkZsFTtpgoMlBvu4icv3C/pkLEk8hlQi2c/ecDGm3UfDiEnzcRNxb1E265br68xFbOTbP6iEXqZ90cjWrFlJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=twdQUgTlOtslwXBo5zOSpwo81vTneNtUK/aEHfy9A78=; b=rckGgOpRZ1I2UI1p7zvjARjjKwHR5O+Z+DhtkMVe8wnNAUtB4athdoHcpJMunxBH1Izn6PjzkiQFQ7k2nzB0NjoVHL3iAHEFZWxRPC5DM76MOcdUoame0a4rkbcuSOIFCdmylCzG073J8hpInkopYoVllPc4pqEHGg+8rCyJvyF84MV798BbI6bxYHwCTNDCq/tDeu+yvwMIR0LUij5FDAaCrTYHa6MDltEu8T4ANj8G3RdGaWNx3Y0IseTlqmt/8S86bm1OtFbKe0Aw9vuZe5MyFKfiUL6pzzMljBJ6CF/BzlRfu0ky6xeIdQb19HCaQekBmWBzgVTtMP45j2rJtQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=twdQUgTlOtslwXBo5zOSpwo81vTneNtUK/aEHfy9A78=; b=eUazSlHj344Gy3vH1iJ6ZN4Rm/8uJ16vNrEhgVGrDUwCjw7wQtOK8IYXB3YxgA5cgktfCrUEd3bBt7EL/ndC8zgWZ832qZ7we7x5IWEfAKgGMtQnNiHsjrclO5znPtIzGBKE1wKVx9adcBL3bj4BQU8u71m9rdYUHmPYV2flD1tvgeavDhT2heUElY6j26Zdno5kIqlr6CeLoErQy0C8srm3Fsd3JXYwN1Xcj0WBCQuxT4eJxwezxo+iwIHVbLk66+qFZW0j1Me3+BEiyK4UsNwFslRIr4j17aSmigkpkKFqkwZ5Wo1QNAfPez2zPu5UsopFRpZPsuKiVlliozZSFQ== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon Subject: [PATCH v3 05/23] vfio-user: introduce vfio-user protocol specification Date: Fri, 6 Jun 2025 17:10:37 -0700 Message-ID: <20250607001056.335310-6-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: 599ff991-a609-4c2d-5fb2-08dda557cbb7 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?EdP6AFAYRApldBOidrmn/jTs7soxfqxicEwZY0UCfgDS4JKyuXrvUxlUvuHS?= =?us-ascii?Q?9umHci+iQom1S2KwlfE6nwB0BrE7mAHKaS5a3GrsLeZco6zKSVYoEV2r90M3?= =?us-ascii?Q?znEOAQQPdy0B4fOd4nk65CpQ6w1yFPmXZIG6sDXQe0wkhmQ4I6ONUgQkNzck?= =?us-ascii?Q?Qj/Olyx5cryI/g81tfawZb8U6RjRxYjgx2WSYyWqEMI88+2ZpDObBfrPoskb?= =?us-ascii?Q?fwkQSa/56UN/+qR/C4Mo1OOZlAI78rU9iSVxwlVR8oOXCeotDgLu2IoaHXvO?= =?us-ascii?Q?T9z0THWLMtyx0xTum70Y4CtAScyEg3gp3ngYoAmCA09Dgxxi+jUK+CcILgaN?= =?us-ascii?Q?vsUTmYiToQFmBrjuZqPwavTgRxKD8o26YJDh900LHGXjY1y6Z4hbeuSRVsHV?= =?us-ascii?Q?/e+PHV8hKqyKWZz00Jxts4FWyjge/5fdOt65B77UNQbAJFLFqVFM9JkBqvO+?= =?us-ascii?Q?VewSDjKXXvdYm7dYkVXsnUIbqodnhGKx0wvn/8KxKU8t/ZqvurGfpBwq+Rpo?= =?us-ascii?Q?yb8EDwV5TEZ+v9Jz/G2Xa/LBxj+7FyREMyAkhJXivIBpfkLYVn5C3/PSsuWx?= =?us-ascii?Q?wL8mHnz0WIRfuHmmHDH3U+K8GB6c+oUCuDPbyn8v4rlNWazcSzB9tWzStSI7?= =?us-ascii?Q?85qULHZhuGFLyEGg9WVTj8jdiFujpDalWzwRjg+c27iulqzsrCOWokpZp9QL?= =?us-ascii?Q?3XtMcD6yVynTdeuvh3MhdRqtRLzYyXX7QowJW8zqR897ZKQd0CeWZxq5sDJg?= =?us-ascii?Q?jw29Ahwk3ka0F1gfbC2NhgcjUvbSErXu1mA7pFWI4/LMCB0Qs8oa0HUW5djn?= =?us-ascii?Q?7T7T4hUciW1bnKtRk+tR0OngsA5wGBktz/l1oIz5ndqJeGM22JSHrgWC2gx8?= =?us-ascii?Q?BXPIgYn2ZXNkX8V//lK83LrZA3eGEBShO/usDc7kUqWbqkO3kpGhOGrCDf0f?= =?us-ascii?Q?WY5YH4Yyo27bhT7ozrdrFX/5NPbY5UFfHD6/19vXWBXxookWlQRHJ/h3F9Ac?= =?us-ascii?Q?F7hTpFxDiGc/T+uVCAOvnSpiWtyW8AhP0IbwXbp1396EyvzbkGcGHWDRqvOJ?= =?us-ascii?Q?l5HNtVaKoUBSYHyyzhPGPH1u/enKN18cH28E4gj6wMFempTLsUDbsCRVV0oN?= =?us-ascii?Q?VYEB9AUHNu2vaf+1qfwSgtgbdcHX0KEl6p3D1oKiboC9G9nJ/0hkGZZc8A1/?= =?us-ascii?Q?9gdwEw70HVmQ3lOH3eO9ax6uj+X8v/GKXs55etMnj+mbxn9UQ7ju3aW5IrIt?= =?us-ascii?Q?Aaq7SnkItIZ1/t4w96UjXb2eaW/G/BC5drZmtCF56Z4RSyt3QNEn0foMb7hK?= =?us-ascii?Q?JmJJGN/Vl1A26OWdpTZhlFi9OIS+Z9GbKjMC2xhUsunkUWyEIGZufSgO9/nV?= =?us-ascii?Q?U5ygurVLr2xjB0F1BcaOLNZku73PvUL7VM/XyIMAuU4D2g6T37iUbAeRv1oo?= =?us-ascii?Q?gSmenkvNJqY=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?NOJt6PTAS/FoK5rEEqRU2liPkMSzV371aYhS5T88Xr/PsyW3fY1+UU0tAvMa?= =?us-ascii?Q?h2+5hetS3uaPk3eZgH6eSsimATQiE6JFrVFn9SSAdgy+q+PIEb9JU4lsoQOT?= =?us-ascii?Q?eRlbMwk9LM5YHdK5ohtHbfWhhZl4tYhK5vOKNLewZDQyCZgWwLjDfG2F/vyZ?= =?us-ascii?Q?Kz6/yhWxsIabsvY1NvBTNVVzpx0ixmgQsP8Ndc9YPW1L0L4FfNRNU2kKnTpW?= =?us-ascii?Q?1YvXBNwmvIl82/0B6F27gd9ZNUmJCIWq+cS5V3ddEuXXKUNqPDmCjDra51+1?= =?us-ascii?Q?nvP+MQxh88sP28KMLJ+47MaI06Wg7+FZT25vGJslNXaPVXSASKTPidQtGj36?= =?us-ascii?Q?3XNemcENfg8spV7lH1D+pXRxQwHNkzhN+Zgk8T0CJdK24IToeMn96KihQg0d?= =?us-ascii?Q?yBj5zZYhvxeBUSsJtqfT6VcSynw3X11+OTveTBUq5M6SRVhHl23lbyAuH8jW?= =?us-ascii?Q?RU/I7QQdEG+uzRoJz10MsSZP0mHftjQYGbMiJI99xeqRJeN8MVsKVcRJyNgj?= =?us-ascii?Q?eOiJe75opg3/olpfXIZciAxpgK6keFs1zlzv4GZdFA1XTPBgRKyUQ763Rjko?= =?us-ascii?Q?3Hi18LzDXQFIRf8MPNnQHYDh1QVo53ULavi35UWU0ZXjNcxWc/HnS7YLIk2O?= =?us-ascii?Q?W/LEAXvPSc2DnK7XC7pOWZM905YldJwjy3ed51XFtx3XqO0ACLCW6th7256e?= =?us-ascii?Q?l/t6NfVkeiEcJ1J/TgfKQ8AFrc2ylFlus6K9UTR8bM8ndXoJt7iP2eoxjc6U?= =?us-ascii?Q?XoPE1loNi+741+duet4LR4IuME8nUsJPN2mKAcK3QlIuZJX9CentFaD6dTV+?= =?us-ascii?Q?O7eBGuUwFtKnrPHaprxf4qVTEFczVAYdRtgl0y14WxXXAn25d4QXaSNeUr5E?= =?us-ascii?Q?N5rB97kJTdj0ZnArpBYmNRoYvQj4Mm5+fdST8uDEMlPTOIXG0f6G2B6zR+mU?= =?us-ascii?Q?2uUt5tTT9TgrM57nGzmrmzAPGlsGxOEAuf2VTm/AzBaiRUjaRVAYVLd7CgjY?= =?us-ascii?Q?6tTyQ3+kBNYIXWKceEpm8JmbhfsjsHEMi9zVsFrSYQrb9mlgT7ZxJC0U/4cw?= =?us-ascii?Q?LDrh37xebpdN/mu4yGetyF1IXUYZddvk4NNq8AWIPg7WTL5c3zRuxqJQKMIQ?= =?us-ascii?Q?5eCErGow9M3nGPi1TGIe9oQxy9zVkG92+ZCQ31gQOXfveB4uNQfsY4TlPqny?= =?us-ascii?Q?AZ013eU9it3bw+heIzlPcBO7nEzE49JkK7Dm1mN1kMUspGKc6yWv7EtFCXSl?= =?us-ascii?Q?mj5GV+ju2E1yN0zlpa3qLu0OyA4IizJ4aL1QAoHivpHvN1PTb9/0U0s1IDma?= =?us-ascii?Q?+dQEPurmGDPmJMiUKB8K3+qHc/Ukqz4FtMTW12QJRIW6DBzsIObDhk8wGkYw?= =?us-ascii?Q?TITlwGUgWwIFRsPmSglpCKAG2sd2Nfo6MwtGFw5uB9Z+nTs8PJsbF6EX+fRN?= =?us-ascii?Q?Tt1MsOZ3wzGFfCKfMws50stge4oRuNTyt7OZNq6+O6O20aTVwygWGmav3lOl?= =?us-ascii?Q?OgSRWoKVq+mff2TgDOLBdBiLuD430Yl4kYDMRYonO1g/wxpPpXfRkunXhp6o?= =?us-ascii?Q?USkxdI1MaSLYXG8tcqxDXJG6JwJekCa2pAft0lwo?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 599ff991-a609-4c2d-5fb2-08dda557cbb7 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:05.6947 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7vwItsDcve+U0E90jRTtTJkJ2cZDpQIPYPT9RO+ncm8Z5do88CzfVzrlBjXZfgVFIINuiP6h/wVOcpt5f7Tfww== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Authority-Analysis: v=2.4 cv=cpSbk04i c=1 sm=1 tr=0 ts=6843839b cx=c_pps a=j6Q7tV1r71QiW8XCak6Opg==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=VwQbUJbxAAAA:8 a=pGLkceISAAAA:8 a=64Cc0HZtAAAA:8 a=20KFwNOVAAAA:8 a=z4glEzOvAAAA:8 a=h9SPtozqHe0Wh_k42joA:9 a=x5O-bmgOEKQMiMGR:21 a=N0yriQgkpQ4A:10 a=HplO-upLQ7EA:10 a=92dS5hN0c3Q7EetK7xW5:22 X-Proofpoint-ORIG-GUID: ZHVfypGI9q7pgfYFTn9pLvhOmm1HotVa X-Proofpoint-GUID: ZHVfypGI9q7pgfYFTn9pLvhOmm1HotVa X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfXztF3StL8PXIy CfPCNzzKxruDMXKdkdd3kd6+Cs4ih1AiqWkskODOcmdB60MONnCm5NAeZFlDa6IE2BizNP1vN6Q Ky04d378xc/9icJtBl6r1slyZNf4RL/aIki/UwIBiFrPcI8Qe4P2Xf5fPi/qox8WECYJZUl2VcW NwOiiVJJvldXDlaO+F4rjbWXf4NSbSr2WO2y9BFDGf45qck7obeDrnVcxN9WHQne2o8D8jYl78C 4XcVCBoif6xuZkuk6Yl7V7gQ9jLsj1fqaH34WLzv3PZv2l8Fo8u3g2t3MRr5L5CJDIk1HLvKKyr 7Yh8JKoEvWjuVI2rw+7zzu3y9fMo5EZZJOxpmkdIEUPgfuC49YhT8a/rLopTwwLMobi5+4VpjPI +togJMNxVM0bwwW8ACzFlWrc/5DgTslsoAqpw14MSmZ7QDvsGthD1mByLsebRz+1cMYFrnJh X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, WEIRD_QUOTING=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255259761116600 Content-Type: text/plain; charset="utf-8" From: Thanos Makatos This patch introduces the vfio-user protocol specification (formerly known as VFIO-over-socket), which is designed to allow devices to be emulated outside QEMU, in a separate process. vfio-user reuses the existing VFIO defines, structs and concepts. It has been earlier discussed as an RFC in: "RFC: use VFIO over a UNIX domain socket to implement device offloading" Signed-off-by: Thanos Makatos Signed-off-by: John Levon --- MAINTAINERS | 8 +- docs/devel/index-internals.rst | 1 + docs/devel/vfio-user.rst | 1522 ++++++++++++++++++++++++++++++++ 3 files changed, 1530 insertions(+), 1 deletion(-) create mode 100644 docs/devel/vfio-user.rst diff --git a/MAINTAINERS b/MAINTAINERS index aa6763077e..dbc393989a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -4241,13 +4241,19 @@ F: hw/remote/proxy-memory-listener.c F: include/hw/remote/proxy-memory-listener.h F: hw/remote/iohub.c F: include/hw/remote/iohub.h -F: subprojects/libvfio-user F: hw/remote/vfio-user-obj.c F: include/hw/remote/vfio-user-obj.h F: hw/remote/iommu.c F: include/hw/remote/iommu.h F: tests/functional/test_multiprocess.py =20 +VFIO-USER: +M: John Levon +M: Thanos Makatos +S: Supported +F: docs/devel/vfio-user.rst +F: subprojects/libvfio-user + EBPF: M: Jason Wang R: Andrew Melnychenko diff --git a/docs/devel/index-internals.rst b/docs/devel/index-internals.rst index 7a0678cbdd..1b04311f06 100644 --- a/docs/devel/index-internals.rst +++ b/docs/devel/index-internals.rst @@ -22,6 +22,7 @@ Details about QEMU's various subsystems including how to = add features to them. tracing uefi-vars vfio-iommufd + vfio-user writing-monitor-commands virtio-backends crypto diff --git a/docs/devel/vfio-user.rst b/docs/devel/vfio-user.rst new file mode 100644 index 0000000000..0d96477a68 --- /dev/null +++ b/docs/devel/vfio-user.rst @@ -0,0 +1,1522 @@ +.. include:: +******************************** +vfio-user Protocol Specification +******************************** + +-------------- +Version_ 0.9.1 +-------------- + +.. contents:: Table of Contents + +Introduction +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +vfio-user is a protocol that allows a device to be emulated in a separate +process outside of a Virtual Machine Monitor (VMM). vfio-user devices cons= ist +of a generic VFIO device type, living inside the VMM, which we call the cl= ient, +and the core device implementation, living outside the VMM, which we call = the +server. + +The vfio-user specification is partly based on the +`Linux VFIO ioctl interface `_. + +VFIO is a mature and stable API, backed by an extensively used framework. = The +existing VFIO client implementation in QEMU (``qemu/hw/vfio/``) can be lar= gely +re-used, though there is nothing in this specification that requires that +particular implementation. None of the VFIO kernel modules are required for +supporting the protocol, on either the client or server side. Some source +definitions in VFIO are re-used for vfio-user. + +The main idea is to allow a virtual device to function in a separate proce= ss in +the same host over a UNIX domain socket. A UNIX domain socket (``AF_UNIX``= ) is +chosen because file descriptors can be trivially sent over it, which in tu= rn +allows: + +* Sharing of client memory for DMA with the server. +* Sharing of server memory with the client for fast MMIO. +* Efficient sharing of eventfd's for triggering interrupts. + +Other socket types could be used which allow the server to run in a separa= te +guest in the same host (``AF_VSOCK``) or remotely (``AF_INET``). Theoretic= ally +the underlying transport does not necessarily have to be a socket, however= we do +not examine such alternatives. In this protocol version we focus on using = a UNIX +domain socket and introduce basic support for the other two types of socke= ts +without considering performance implications. + +While passing of file descriptors is desirable for performance reasons, su= pport +is not necessary for either the client or the server in order to implement= the +protocol. There is always an in-band, message-passing fall back mechanism. + +Overview +=3D=3D=3D=3D=3D=3D=3D=3D + +VFIO is a framework that allows a physical device to be securely passed th= rough +to a user space process; the device-specific kernel driver does not drive = the +device at all. Typically, the user space process is a VMM and the device = is +passed through to it in order to achieve high performance. VFIO provides a= n API +and the required functionality in the kernel. QEMU has adopted VFIO to all= ow a +guest to directly access physical devices, instead of emulating them in +software. + +vfio-user reuses the core VFIO concepts defined in its API, but implements= them +as messages to be sent over a socket. It does not change the kernel-based = VFIO +in any way, in fact none of the VFIO kernel modules need to be loaded to u= se +vfio-user. It is also possible for the client to concurrently use the curr= ent +kernel-based VFIO for one device, and vfio-user for another device. + +VFIO Device Model +----------------- + +A device under VFIO presents a standard interface to the user process. Man= y of +the VFIO operations in the existing interface use the ``ioctl()`` system c= all, and +references to the existing interface are called the ``ioctl()`` implementa= tion in +this document. + +The following sections describe the set of messages that implement the vfi= o-user +interface over a socket. In many cases, the messages are analogous to data +structures used in the ``ioctl()`` implementation. Messages derived from t= he +``ioctl()`` will have a name derived from the ``ioctl()`` command name. E= .g., the +``VFIO_DEVICE_GET_INFO`` ``ioctl()`` command becomes a +``VFIO_USER_DEVICE_GET_INFO`` message. The purpose of this reuse is to sh= are as +much code as feasible with the ``ioctl()`` implementation``. + +Connection Initiation +^^^^^^^^^^^^^^^^^^^^^ + +After the client connects to the server, the initial client message is +``VFIO_USER_VERSION`` to propose a protocol version and set of capabilitie= s to +apply to the session. The server replies with a compatible version and set= of +capabilities it supports, or closes the connection if it cannot support the +advertised version. + +Device Information +^^^^^^^^^^^^^^^^^^ + +The client uses a ``VFIO_USER_DEVICE_GET_INFO`` message to query the serve= r for +information about the device. This information includes: + +* The device type and whether it supports reset (``VFIO_DEVICE_FLAGS_``), +* the number of device regions, and +* the device presents to the client the number of interrupt types the devi= ce + supports. + +Region Information +^^^^^^^^^^^^^^^^^^ + +The client uses ``VFIO_USER_DEVICE_GET_REGION_INFO`` messages to query the +server for information about the device's regions. This information descri= bes: + +* Read and write permissions, whether it can be memory mapped, and whether= it + supports additional capabilities (``VFIO_REGION_INFO_CAP_``). +* Region index, size, and offset. + +When a device region can be mapped by the client, the server provides a fi= le +descriptor which the client can ``mmap()``. The server is responsible for +polling for client updates to memory mapped regions. + +Region Capabilities +""""""""""""""""""" + +Some regions have additional capabilities that cannot be described adequat= ely +by the region info data structure. These capabilities are returned in the +region info reply in a list similar to PCI capabilities in a PCI device's +configuration space. + +Sparse Regions +"""""""""""""" +A region can be memory-mappable in whole or in part. When only a subset of= a +region can be mapped by the client, a ``VFIO_REGION_INFO_CAP_SPARSE_MMAP`` +capability is included in the region info reply. This capability describes +which portions can be mapped by the client. + +.. Note:: + For example, in a virtual NVMe controller, sparse regions can be used so + that accesses to the NVMe registers (found in the beginning of BAR0) are + trapped (an infrequent event), while allowing direct access to the door= bells + (an extremely frequent event as every I/O submission requires a write to + BAR0), found in the next page after the NVMe registers in BAR0. + +Device-Specific Regions +""""""""""""""""""""""" + +A device can define regions additional to the standard ones (e.g. PCI inde= xes +0-8). This is achieved by including a ``VFIO_REGION_INFO_CAP_TYPE`` capabi= lity +in the region info reply of a device-specific region. Such regions are ref= lected +in ``struct vfio_user_device_info.num_regions``. Thus, for PCI devices this +value can be equal to, or higher than, ``VFIO_PCI_NUM_REGIONS``. + +Region I/O via file descriptors +------------------------------- + +For unmapped regions, region I/O from the client is done via +``VFIO_USER_REGION_READ/WRITE``. As an optimization, ioeventfds or ioregi= onfds +may be configured for sub-regions of some regions. A client may request +information on these sub-regions via ``VFIO_USER_DEVICE_GET_REGION_IO_FDS`= `; by +configuring the returned file descriptors as ioeventfds or ioregionfds, the +server can be directly notified of I/O (for example, by KVM) without takin= g a +trip through the client. + +Interrupts +^^^^^^^^^^ + +The client uses ``VFIO_USER_DEVICE_GET_IRQ_INFO`` messages to query the se= rver +for the device's interrupt types. The interrupt types are specific to the = bus +the device is attached to, and the client is expected to know the capabili= ties +of each interrupt type. The server can signal an interrupt by directly inj= ecting +interrupts into the guest via an event file descriptor. The client configu= res +how the server signals an interrupt with ``VFIO_USER_SET_IRQS`` messages. + +Device Read and Write +^^^^^^^^^^^^^^^^^^^^^ + +When the guest executes load or store operations to an unmapped device reg= ion, +the client forwards these operations to the server with +``VFIO_USER_REGION_READ`` or ``VFIO_USER_REGION_WRITE`` messages. The serv= er +will reply with data from the device on read operations or an acknowledgem= ent on +write operations. See `Read and Write Operations`_. + +Client memory access +-------------------- + +The client uses ``VFIO_USER_DMA_MAP`` and ``VFIO_USER_DMA_UNMAP`` messages= to +inform the server of the valid DMA ranges that the server can access on be= half +of a device (typically, VM guest memory). DMA memory may be accessed by the +server via ``VFIO_USER_DMA_READ`` and ``VFIO_USER_DMA_WRITE`` messages ove= r the +socket. In this case, the "DMA" part of the naming is a misnomer. + +Actual direct memory access of client memory from the server is possible i= f the +client provides file descriptors the server can ``mmap()``. Note that ``mm= ap()`` +privileges cannot be revoked by the client, therefore file descriptors sho= uld +only be exported in environments where the client trusts the server not to +corrupt guest memory. + +See `Read and Write Operations`_. + +Client/server interactions +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D + +Socket +------ + +A server can serve: + +1) one or more clients, and/or +2) one or more virtual devices, belonging to one or more clients. + +The current protocol specification requires a dedicated socket per +client/server connection. It is a server-side implementation detail whethe= r a +single server handles multiple virtual devices from the same or multiple +clients. The location of the socket is implementation-specific. Multiplexi= ng +clients, devices, and servers over the same socket is not supported in this +version of the protocol. + +Authentication +-------------- + +For ``AF_UNIX``, we rely on OS mandatory access controls on the socket fil= es, +therefore it is up to the management layer to set up the socket as require= d. +Socket types that span guests or hosts will require a proper authentication +mechanism. Defining that mechanism is deferred to a future version of the +protocol. + +Command Concurrency +------------------- + +A client may pipeline multiple commands without waiting for previous comma= nd +replies. The server will process commands in the order they are received.= A +consequence of this is if a client issues a command with the *No_reply* bi= t, +then subsequently issues a command without *No_reply*, the older command w= ill +have been processed before the reply to the younger command is sent by the +server. The client must be aware of the device's capability to process +concurrent commands if pipelining is used. For example, pipelining allows +multiple client threads to concurrently access device regions; the client = must +ensure these accesses obey device semantics. + +An example is a frame buffer device, where the device may allow concurrent +access to different areas of video memory, but may have indeterminate beha= vior +if concurrent accesses are performed to command or status registers. + +Note that unrelated messages sent from the server to the client can appear= in +between a client to server request/reply and vice versa. + +Implementers should be prepared for certain commands to exhibit potentially +unbounded latencies. For example, ``VFIO_USER_DEVICE_RESET`` may take an +arbitrarily long time to complete; clients should take care not to block +unnecessarily. + +Socket Disconnection Behavior +----------------------------- +The server and the client can disconnect from each other, either intention= ally +or unexpectedly. Both the client and the server need to know how to handle= such +events. + +Server Disconnection +^^^^^^^^^^^^^^^^^^^^ +A server disconnecting from the client may indicate that: + +1) A virtual device has been restarted, either intentionally (e.g. because= of a + device update) or unintentionally (e.g. because of a crash). +2) A virtual device has been shut down with no intention to be restarted. + +It is impossible for the client to know whether or not a failure is +intermittent or innocuous and should be retried, therefore the client shou= ld +reset the VFIO device when it detects the socket has been disconnected. +Error recovery will be driven by the guest's device error handling +behavior. + +Client Disconnection +^^^^^^^^^^^^^^^^^^^^ +The client disconnecting from the server primarily means that the client +has exited. Currently, this means that the guest is shut down so the devic= e is +no longer needed therefore the server can automatically exit. However, the= re +can be cases where a client disconnection should not result in a server ex= it: + +1) A single server serving multiple clients. +2) A multi-process QEMU upgrading itself step by step, which is not yet + implemented. + +Therefore in order for the protocol to be forward compatible, the server s= hould +respond to a client disconnection as follows: + + - all client memory regions are unmapped and cleaned up (including closin= g any + passed file descriptors) + - all IRQ file descriptors passed from the old client are closed + - the device state should otherwise be retained + +The expectation is that when a client reconnects, it will re-establish IRQ= and +client memory mappings. + +If anything happens to the client (such as qemu really did exit), the cont= rol +stack will know about it and can clean up resources accordingly. + +Security Considerations +----------------------- + +Speaking generally, vfio-user clients should not trust servers, and vice v= ersa. +Standard tools and mechanisms should be used on both sides to validate inp= ut and +prevent against denial of service scenarios, buffer overflow, etc. + +Request Retry and Response Timeout +---------------------------------- +A failed command is a command that has been successfully sent and has been +responded to with an error code. Failure to send the command in the first = place +(e.g. because the socket is disconnected) is a different type of error exa= mined +earlier in the disconnect section. + +.. Note:: + QEMU's VFIO retries certain operations if they fail. While this makes s= ense + for real HW, we don't know for sure whether it makes sense for virtual + devices. + +Defining a retry and timeout scheme is deferred to a future version of the +protocol. + +Message sizes +------------- + +Some requests have an ``argsz`` field. In a request, it defines the maximum +expected reply payload size, which should be at least the size of the fixed +reply payload headers defined here. The *request* payload size is defined = by the +usual ``msg_size`` field in the header, not the ``argsz`` field. + +In a reply, the server sets ``argsz`` field to the size needed for a full +payload size. This may be less than the requested maximum size. This may be +larger than the requested maximum size: in that case, the full payload is = not +included in the reply, but the ``argsz`` field in the reply indicates the = needed +size, allowing a client to allocate a larger buffer for holding the reply = before +trying again. + +In addition, during negotiation (see `Version`_), the client and server m= ay +each specify a ``max_data_xfer_size`` value; this defines the maximum data= that +may be read or written via one of the ``VFIO_USER_DMA/REGION_READ/WRITE`` +messages; see `Read and Write Operations`_. + +Protocol Specification +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +To distinguish from the base VFIO symbols, all vfio-user symbols are prefi= xed +with ``vfio_user`` or ``VFIO_USER``. In this revision, all data is in the +endianness of the host system, although this may be relaxed in future +revisions in cases where the client and server run on different hosts +with different endianness. + +Unless otherwise specified, all sizes should be presumed to be in bytes. + +.. _Commands: + +Commands +-------- +The following table lists the VFIO message command IDs, and whether the +message command is sent from the client or the server. + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Name Command Request Direction +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +``VFIO_USER_VERSION`` 1 client -> server +``VFIO_USER_DMA_MAP`` 2 client -> server +``VFIO_USER_DMA_UNMAP`` 3 client -> server +``VFIO_USER_DEVICE_GET_INFO`` 4 client -> server +``VFIO_USER_DEVICE_GET_REGION_INFO`` 5 client -> server +``VFIO_USER_DEVICE_GET_REGION_IO_FDS`` 6 client -> server +``VFIO_USER_DEVICE_GET_IRQ_INFO`` 7 client -> server +``VFIO_USER_DEVICE_SET_IRQS`` 8 client -> server +``VFIO_USER_REGION_READ`` 9 client -> server +``VFIO_USER_REGION_WRITE`` 10 client -> server +``VFIO_USER_DMA_READ`` 11 server -> client +``VFIO_USER_DMA_WRITE`` 12 server -> client +``VFIO_USER_DEVICE_RESET`` 13 client -> server +``VFIO_USER_REGION_WRITE_MULTI`` 15 client -> server +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Header +------ + +All messages, both command messages and reply messages, are preceded by a +16-byte header that contains basic information about the message. The head= er is +followed by message-specific data described in the sections below. + ++----------------+--------+-------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| Message ID | 0 | 2 | ++----------------+--------+-------------+ +| Command | 2 | 2 | ++----------------+--------+-------------+ +| Message size | 4 | 4 | ++----------------+--------+-------------+ +| Flags | 8 | 4 | ++----------------+--------+-------------+ +| | +-----+------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0-3 | Type | | +| | +-----+------------+ | +| | | 4 | No_reply | | +| | +-----+------------+ | +| | | 5 | Error | | +| | +-----+------------+ | ++----------------+--------+-------------+ +| Error | 12 | 4 | ++----------------+--------+-------------+ +| | 16 | variable | ++----------------+--------+-------------+ + +* *Message ID* identifies the message, and is echoed in the command's reply + message. Message IDs belong entirely to the sender, can be re-used (even + concurrently) and the receiver must not make any assumptions about their + uniqueness. +* *Command* specifies the command to be executed, listed in Commands_. It = is + also set in the reply header. +* *Message size* contains the size of the entire message, including the he= ader. +* *Flags* contains attributes of the message: + + * The *Type* bits indicate the message type. + + * *Command* (value 0x0) indicates a command message. + * *Reply* (value 0x1) indicates a reply message acknowledging a previ= ous + command with the same message ID. + * *No_reply* in a command message indicates that no reply is needed for = this + command. This is commonly used when multiple commands are sent, and o= nly + the last needs acknowledgement. + * *Error* in a reply message indicates the command being acknowledged had + an error. In this case, the *Error* field will be valid. + +* *Error* in a reply message is an optional UNIX errno value. It may be ze= ro + even if the Error bit is set in Flags. It is reserved in a command messa= ge. + +Each command message in Commands_ must be replied to with a reply message, +unless the message sets the *No_Reply* bit. The reply consists of the hea= der +with the *Reply* bit set, plus any additional data. + +If an error occurs, the reply message must only include the reply header. + +As the header is standard in both requests and replies, it is not included= in +the command-specific specifications below; each message definition should = be +appended to the standard header, and the offsets are given from the end of= the +standard header. + +``VFIO_USER_VERSION`` +--------------------- + +.. _Version: + +This is the initial message sent by the client after the socket connection= is +established; the same format is used for the server's reply. + +Upon establishing a connection, the client must send a ``VFIO_USER_VERSION= `` +message proposing a protocol version and a set of capabilities. The server +compares these with the versions and capabilities it supports and sends a +``VFIO_USER_VERSION`` reply according to the following rules. + +* The major version in the reply must be the same as proposed. If the clie= nt + does not support the proposed major, it closes the connection. +* The minor version in the reply must be equal to or less than the minor + version proposed. +* The capability list must be a subset of those proposed. If the server + requires a capability the client did not include, it closes the connecti= on. + +The protocol major version will only change when incompatible protocol cha= nges +are made, such as changing the message format. The minor version may change +when compatible changes are made, such as adding new messages or capabilit= ies, +Both the client and server must support all minor versions less than the +maximum minor version it supports. E.g., an implementation that supports +version 1.3 must also support 1.0 through 1.2. + +When making a change to this specification, the protocol version number mu= st +be included in the form "added in version X.Y" + +Request +^^^^^^^ + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D =3D=3D=3D= =3D +Name Offset Size +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D =3D=3D=3D= =3D +version major 0 2 +version minor 2 2 +version data 4 variable (including terminating NUL). Optional. +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D =3D=3D=3D= =3D + +The version data is an optional UTF-8 encoded JSON byte array with the fol= lowing +format: + ++--------------+--------+-----------------------------------+ +| Name | Type | Description | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| capabilities | object | Contains common capabilities that | +| | | the sender supports. Optional. | ++--------------+--------+-----------------------------------+ + +Capabilities: + ++--------------------+---------+------------------------------------------= ------+ +| Name | Type | Description = | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| max_msg_fds | number | Maximum number of file descriptors that c= an be | +| | | received by the sender in one message. = | +| | | Optional. If not specified then the recei= ver | +| | | must assume a value of ``1``. = | ++--------------------+---------+------------------------------------------= ------+ +| max_data_xfer_size | number | Maximum ``count`` for data transfer messa= ges; | +| | | see `Read and Write Operations`_. Optiona= l, | +| | | with a default value of 1048576 bytes. = | ++--------------------+---------+------------------------------------------= ------+ +| pgsizes | number | Page sizes supported in DMA map operation= s | +| | | or'ed together. Optional, with a default = value | +| | | of supporting only 4k pages. = | ++--------------------+---------+------------------------------------------= ------+ +| max_dma_maps | number | Maximum number DMA map windows that can b= e | +| | | valid simultaneously. Optional, with a = | +| | | value of 65535 (64k-1). = | ++--------------------+---------+------------------------------------------= ------+ +| migration | object | Migration capability parameters. If missi= ng | +| | | then migration is not supported by the se= nder. | ++--------------------+---------+------------------------------------------= ------+ +| write_multiple | boolean | ``VFIO_USER_REGION_WRITE_MULTI`` messages= | +| | | are supported if the value is ``true``. = | ++--------------------+---------+------------------------------------------= ------+ + +The migration capability contains the following name/value pairs: + ++-----------------+--------+----------------------------------------------= ----+ +| Name | Type | Description = | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+ +| pgsize | number | Page size of dirty pages bitmap. The smallest= | +| | | between the client and the server is used. = | ++-----------------+--------+----------------------------------------------= ----+ +| max_bitmap_size | number | Maximum bitmap size in ``VFIO_USER_DIRTY_PAGE= S`` | +| | | and ``VFIO_DMA_UNMAP`` messages. Optional, = | +| | | with a default value of 256MB. = | ++-----------------+--------+----------------------------------------------= ----+ + +Reply +^^^^^ + +The same message format is used in the server's reply with the semantics +described above. + +``VFIO_USER_DMA_MAP`` +--------------------- + +This command message is sent by the client to the server to inform it of t= he +memory regions the server can access. It must be sent before the server can +perform any DMA to the client. It is normally sent directly after the vers= ion +handshake is completed, but may also occur when memory is added to the cli= ent, +or if the client uses a vIOMMU. + +Request +^^^^^^^ + +The request payload for this message is a structure of the following forma= t: + ++-------------+--------+-------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+-------------+ +| flags | 4 | 4 | ++-------------+--------+-------------+ +| | +-----+------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | readable | | +| | +-----+------------+ | +| | | 1 | writeable | | +| | +-----+------------+ | ++-------------+--------+-------------+ +| offset | 8 | 8 | ++-------------+--------+-------------+ +| address | 16 | 8 | ++-------------+--------+-------------+ +| size | 24 | 8 | ++-------------+--------+-------------+ + +* *argsz* is the size of the above structure. Note there is no reply paylo= ad, + so this field differs from other message types. +* *flags* contains the following region attributes: + + * *readable* indicates that the region can be read from. + + * *writeable* indicates that the region can be written to. + +* *offset* is the file offset of the region with respect to the associated= file + descriptor, or zero if the region is not mappable +* *address* is the base DMA address of the region. +* *size* is the size of the region. + +This structure is 32 bytes in size, so the message size is 16 + 32 bytes. + +If the DMA region being added can be directly mapped by the server, a file +descriptor must be sent as part of the message meta-data. The region can be +mapped via the mmap() system call. On ``AF_UNIX`` sockets, the file descri= ptor +must be passed as ``SCM_RIGHTS`` type ancillary data. Otherwise, if the D= MA +region cannot be directly mapped by the server, no file descriptor must be= sent +as part of the message meta-data and the DMA region can be accessed by the +server using ``VFIO_USER_DMA_READ`` and ``VFIO_USER_DMA_WRITE`` messages, +explained in `Read and Write Operations`_. A command to map over an existi= ng +region must be failed by the server with ``EEXIST`` set in error field in = the +reply. + +Reply +^^^^^ + +There is no payload in the reply message. + +``VFIO_USER_DMA_UNMAP`` +----------------------- + +This command message is sent by the client to the server to inform it that= a +DMA region, previously made available via a ``VFIO_USER_DMA_MAP`` command +message, is no longer available for DMA. It typically occurs when memory is +subtracted from the client or if the client uses a vIOMMU. The DMA region = is +described by the following structure: + +Request +^^^^^^^ + +The request payload for this message is a structure of the following forma= t: + ++--------------+--------+------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++--------------+--------+------------------------+ +| flags | 4 | 4 | ++--------------+--------+------------------------+ +| address | 8 | 8 | ++--------------+--------+------------------------+ +| size | 16 | 8 | ++--------------+--------+------------------------+ + +* *argsz* is the maximum size of the reply payload. +* *flags* is unused in this version. +* *address* is the base DMA address of the DMA region. +* *size* is the size of the DMA region. + +The address and size of the DMA region being unmapped must match exactly a +previous mapping. + +Reply +^^^^^ + +Upon receiving a ``VFIO_USER_DMA_UNMAP`` command, if the file descriptor is +mapped then the server must release all references to that DMA region befo= re +replying, which potentially includes in-flight DMA transactions. + +The server responds with the original DMA entry in the request. + + +``VFIO_USER_DEVICE_GET_INFO`` +----------------------------- + +This command message is sent by the client to the server to query for basic +information about the device. + +Request +^^^^^^^ + ++-------------+--------+--------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+--------------------------+ +| flags | 4 | 4 | ++-------------+--------+--------------------------+ +| | +-----+-------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_DEVICE_FLAGS_RESET | | +| | +-----+-------------------------+ | +| | | 1 | VFIO_DEVICE_FLAGS_PCI | | +| | +-----+-------------------------+ | ++-------------+--------+--------------------------+ +| num_regions | 8 | 4 | ++-------------+--------+--------------------------+ +| num_irqs | 12 | 4 | ++-------------+--------+--------------------------+ + +* *argsz* is the maximum size of the reply payload +* all other fields must be zero. + +Reply +^^^^^ + ++-------------+--------+--------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+--------------------------+ +| flags | 4 | 4 | ++-------------+--------+--------------------------+ +| | +-----+-------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_DEVICE_FLAGS_RESET | | +| | +-----+-------------------------+ | +| | | 1 | VFIO_DEVICE_FLAGS_PCI | | +| | +-----+-------------------------+ | ++-------------+--------+--------------------------+ +| num_regions | 8 | 4 | ++-------------+--------+--------------------------+ +| num_irqs | 12 | 4 | ++-------------+--------+--------------------------+ + +* *argsz* is the size required for the full reply payload (16 bytes today) +* *flags* contains the following device attributes. + + * ``VFIO_DEVICE_FLAGS_RESET`` indicates that the device supports the + ``VFIO_USER_DEVICE_RESET`` message. + * ``VFIO_DEVICE_FLAGS_PCI`` indicates that the device is a PCI device. + +* *num_regions* is the number of memory regions that the device exposes. +* *num_irqs* is the number of distinct interrupt types that the device sup= ports. + +This version of the protocol only supports PCI devices. Additional devices= may +be supported in future versions. + +``VFIO_USER_DEVICE_GET_REGION_INFO`` +------------------------------------ + +This command message is sent by the client to the server to query for +information about device regions. The VFIO region info structure is define= d in +```` (``struct vfio_region_info``). + +Request +^^^^^^^ + ++------------+--------+------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+ +| argsz | 0 | 4 | ++------------+--------+------------------------------+ +| flags | 4 | 4 | ++------------+--------+------------------------------+ +| index | 8 | 4 | ++------------+--------+------------------------------+ +| cap_offset | 12 | 4 | ++------------+--------+------------------------------+ +| size | 16 | 8 | ++------------+--------+------------------------------+ +| offset | 24 | 8 | ++------------+--------+------------------------------+ + +* *argsz* the maximum size of the reply payload +* *index* is the index of memory region being queried, it is the only field + that is required to be set in the command message. +* all other fields must be zero. + +Reply +^^^^^ + ++------------+--------+------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+ +| argsz | 0 | 4 | ++------------+--------+------------------------------+ +| flags | 4 | 4 | ++------------+--------+------------------------------+ +| | +-----+-----------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_REGION_INFO_FLAG_READ | | +| | +-----+-----------------------------+ | +| | | 1 | VFIO_REGION_INFO_FLAG_WRITE | | +| | +-----+-----------------------------+ | +| | | 2 | VFIO_REGION_INFO_FLAG_MMAP | | +| | +-----+-----------------------------+ | +| | | 3 | VFIO_REGION_INFO_FLAG_CAPS | | +| | +-----+-----------------------------+ | ++------------+--------+------------------------------+ ++------------+--------+------------------------------+ +| index | 8 | 4 | ++------------+--------+------------------------------+ +| cap_offset | 12 | 4 | ++------------+--------+------------------------------+ +| size | 16 | 8 | ++------------+--------+------------------------------+ +| offset | 24 | 8 | ++------------+--------+------------------------------+ + +* *argsz* is the size required for the full reply payload (region info str= ucture + plus the size of any region capabilities) +* *flags* are attributes of the region: + + * ``VFIO_REGION_INFO_FLAG_READ`` allows client read access to the region. + * ``VFIO_REGION_INFO_FLAG_WRITE`` allows client write access to the regi= on. + * ``VFIO_REGION_INFO_FLAG_MMAP`` specifies the client can mmap() the reg= ion. + When this flag is set, the reply will include a file descriptor in its + meta-data. On ``AF_UNIX`` sockets, the file descriptors will be passed= as + ``SCM_RIGHTS`` type ancillary data. + * ``VFIO_REGION_INFO_FLAG_CAPS`` indicates additional capabilities found= in the + reply. + +* *index* is the index of memory region being queried, it is the only field + that is required to be set in the command message. +* *cap_offset* describes where additional region capabilities can be found. + cap_offset is relative to the beginning of the VFIO region info structur= e. + The data structure it points is a VFIO cap header defined in + ````. +* *size* is the size of the region. +* *offset* is the offset that should be given to the mmap() system call for + regions with the MMAP attribute. It is also used as the base offset when + mapping a VFIO sparse mmap area, described below. + +VFIO region capabilities +"""""""""""""""""""""""" + +The VFIO region information can also include a capabilities list. This lis= t is +similar to a PCI capability list - each entry has a common header that +identifies a capability and where the next capability in the list can be f= ound. +The VFIO capability header format is defined in ```` (``stru= ct +vfio_info_cap_header``). + +VFIO cap header format +"""""""""""""""""""""" + ++---------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D+ +| id | 0 | 2 | ++---------+--------+------+ +| version | 2 | 2 | ++---------+--------+------+ +| next | 4 | 4 | ++---------+--------+------+ + +* *id* is the capability identity. +* *version* is a capability-specific version number. +* *next* specifies the offset of the next capability in the capability lis= t. It + is relative to the beginning of the VFIO region info structure. + +VFIO sparse mmap cap header +""""""""""""""""""""""""""" + ++------------------+----------------------------------+ +| Name | Value | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| id | VFIO_REGION_INFO_CAP_SPARSE_MMAP | ++------------------+----------------------------------+ +| version | 0x1 | ++------------------+----------------------------------+ +| next | | ++------------------+----------------------------------+ +| sparse mmap info | VFIO region info sparse mmap | ++------------------+----------------------------------+ + +This capability is defined when only a subrange of the region supports +direct access by the client via mmap(). The VFIO sparse mmap area is defin= ed in +```` (``struct vfio_region_sparse_mmap_area`` and ``struct +vfio_region_info_cap_sparse_mmap``). + +VFIO region info cap sparse mmap +"""""""""""""""""""""""""""""""" + ++----------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D= =3D+ +| nr_areas | 0 | 4 | ++----------+--------+------+ +| reserved | 4 | 4 | ++----------+--------+------+ +| offset | 8 | 8 | ++----------+--------+------+ +| size | 16 | 8 | ++----------+--------+------+ +| ... | | | ++----------+--------+------+ + +* *nr_areas* is the number of sparse mmap areas in the region. +* *offset* and size describe a single area that can be mapped by the clien= t. + There will be *nr_areas* pairs of offset and size. The offset will be ad= ded to + the base offset given in the ``VFIO_USER_DEVICE_GET_REGION_INFO`` to for= m the + offset argument of the subsequent mmap() call. + +The VFIO sparse mmap area is defined in ```` (``struct +vfio_region_info_cap_sparse_mmap``). + + +``VFIO_USER_DEVICE_GET_REGION_IO_FDS`` +-------------------------------------- + +Clients can access regions via ``VFIO_USER_REGION_READ/WRITE`` or, if prov= ided, by +``mmap()`` of a file descriptor provided by the server. + +``VFIO_USER_DEVICE_GET_REGION_IO_FDS`` provides an alternative access mech= anism via +file descriptors. This is an optional feature intended for performance +improvements where an underlying sub-system (such as KVM) supports communi= cation +across such file descriptors to the vfio-user server, without needing to +round-trip through the client. + +The server returns an array of sub-regions for the requested region. Each +sub-region describes a span (offset and size) of a region, along with the +requested file descriptor notification mechanism to use. Each sub-region = in the +response message may choose to use a different method, as defined below. = The +two mechanisms supported in this specification are ioeventfds and ioregion= fds. + +The server in addition returns a file descriptor in the ancillary data; cl= ients +are expected to configure each sub-region's file descriptor with the reque= sted +notification method. For example, a client could configure KVM with the +requested ioeventfd via a ``KVM_IOEVENTFD`` ``ioctl()``. + +Request +^^^^^^^ + ++-------------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+------+ +| flags | 4 | 4 | ++-------------+--------+------+ +| index | 8 | 4 | ++-------------+--------+------+ +| count | 12 | 4 | ++-------------+--------+------+ + +* *argsz* the maximum size of the reply payload +* *index* is the index of memory region being queried +* all other fields must be zero + +The client must set ``flags`` to zero and specify the region being queried= in +the ``index``. + +Reply +^^^^^ + ++-------------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+------+ +| flags | 4 | 4 | ++-------------+--------+------+ +| index | 8 | 4 | ++-------------+--------+------+ +| count | 12 | 4 | ++-------------+--------+------+ +| sub-regions | 16 | ... | ++-------------+--------+------+ + +* *argsz* is the size of the region IO FD info structure plus the + total size of the sub-region array. Thus, each array entry "i" is at off= set + i * ((argsz - 32) / count). Note that currently this is 40 bytes for bot= h IO + FD types, but this is not to be relied on. As elsewhere, this indicates = the + full reply payload size needed. +* *flags* must be zero +* *index* is the index of memory region being queried +* *count* is the number of sub-regions in the array +* *sub-regions* is the array of Sub-Region IO FD info structures + +The reply message will additionally include at least one file descriptor i= n the +ancillary data. Note that more than one sub-region may share the same file +descriptor. + +Note that it is the client's responsibility to verify the requested values= (for +example, that the requested offset does not exceed the region's bounds). + +Each sub-region given in the response has one of two possible structures, +depending whether *type* is ``VFIO_USER_IO_FD_TYPE_IOEVENTFD`` or +``VFIO_USER_IO_FD_TYPE_IOREGIONFD``: + +Sub-Region IO FD info format (ioeventfd) +"""""""""""""""""""""""""""""""""""""""" + ++-----------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D+ +| offset | 0 | 8 | ++-----------+--------+------+ +| size | 8 | 8 | ++-----------+--------+------+ +| fd_index | 16 | 4 | ++-----------+--------+------+ +| type | 20 | 4 | ++-----------+--------+------+ +| flags | 24 | 4 | ++-----------+--------+------+ +| padding | 28 | 4 | ++-----------+--------+------+ +| datamatch | 32 | 8 | ++-----------+--------+------+ + +* *offset* is the offset of the start of the sub-region within the region + requested ("physical address offset" for the region) +* *size* is the length of the sub-region. This may be zero if the access s= ize is + not relevant, which may allow for optimizations +* *fd_index* is the index in the ancillary data of the FD to use for ioeve= ntfd + notification; it may be shared. +* *type* is ``VFIO_USER_IO_FD_TYPE_IOEVENTFD`` +* *flags* is any of: + + * ``KVM_IOEVENTFD_FLAG_DATAMATCH`` + * ``KVM_IOEVENTFD_FLAG_PIO`` + * ``KVM_IOEVENTFD_FLAG_VIRTIO_CCW_NOTIFY`` (FIXME: makes sense?) + +* *datamatch* is the datamatch value if needed + +See https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt, *4.59 +KVM_IOEVENTFD* for further context on the ioeventfd-specific fields. + +Sub-Region IO FD info format (ioregionfd) +""""""""""""""""""""""""""""""""""""""""" + ++-----------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D+ +| offset | 0 | 8 | ++-----------+--------+------+ +| size | 8 | 8 | ++-----------+--------+------+ +| fd_index | 16 | 4 | ++-----------+--------+------+ +| type | 20 | 4 | ++-----------+--------+------+ +| flags | 24 | 4 | ++-----------+--------+------+ +| padding | 28 | 4 | ++-----------+--------+------+ +| user_data | 32 | 8 | ++-----------+--------+------+ + +* *offset* is the offset of the start of the sub-region within the region + requested ("physical address offset" for the region) +* *size* is the length of the sub-region. This may be zero if the access s= ize is + not relevant, which may allow for optimizations; ``KVM_IOREGION_POSTED_W= RITES`` + must be set in *flags* in this case +* *fd_index* is the index in the ancillary data of the FD to use for ioreg= ionfd + messages; it may be shared +* *type* is ``VFIO_USER_IO_FD_TYPE_IOREGIONFD`` +* *flags* is any of: + + * ``KVM_IOREGION_PIO`` + * ``KVM_IOREGION_POSTED_WRITES`` + +* *user_data* is an opaque value passed back to the server via a message o= n the + file descriptor + +For further information on the ioregionfd-specific fields, see: +https://lore.kernel.org/kvm/cover.1613828726.git.eafanasova@gmail.com/ + +(FIXME: update with final API docs.) + +``VFIO_USER_DEVICE_GET_IRQ_INFO`` +--------------------------------- + +This command message is sent by the client to the server to query for +information about device interrupt types. The VFIO IRQ info structure is +defined in ```` (``struct vfio_irq_info``). + +Request +^^^^^^^ + ++-------+--------+---------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------+--------+---------------------------+ +| flags | 4 | 4 | ++-------+--------+---------------------------+ +| | +-----+--------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_IRQ_INFO_EVENTFD | | +| | +-----+--------------------------+ | +| | | 1 | VFIO_IRQ_INFO_MASKABLE | | +| | +-----+--------------------------+ | +| | | 2 | VFIO_IRQ_INFO_AUTOMASKED | | +| | +-----+--------------------------+ | +| | | 3 | VFIO_IRQ_INFO_NORESIZE | | +| | +-----+--------------------------+ | ++-------+--------+---------------------------+ +| index | 8 | 4 | ++-------+--------+---------------------------+ +| count | 12 | 4 | ++-------+--------+---------------------------+ + +* *argsz* is the maximum size of the reply payload (16 bytes today) +* index is the index of IRQ type being queried (e.g. ``VFIO_PCI_MSIX_IRQ_I= NDEX``) +* all other fields must be zero + +Reply +^^^^^ + ++-------+--------+---------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------+--------+---------------------------+ +| flags | 4 | 4 | ++-------+--------+---------------------------+ +| | +-----+--------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_IRQ_INFO_EVENTFD | | +| | +-----+--------------------------+ | +| | | 1 | VFIO_IRQ_INFO_MASKABLE | | +| | +-----+--------------------------+ | +| | | 2 | VFIO_IRQ_INFO_AUTOMASKED | | +| | +-----+--------------------------+ | +| | | 3 | VFIO_IRQ_INFO_NORESIZE | | +| | +-----+--------------------------+ | ++-------+--------+---------------------------+ +| index | 8 | 4 | ++-------+--------+---------------------------+ +| count | 12 | 4 | ++-------+--------+---------------------------+ + +* *argsz* is the size required for the full reply payload (16 bytes today) +* *flags* defines IRQ attributes: + + * ``VFIO_IRQ_INFO_EVENTFD`` indicates the IRQ type can support server ev= entfd + signalling. + * ``VFIO_IRQ_INFO_MASKABLE`` indicates that the IRQ type supports the ``= MASK`` + and ``UNMASK`` actions in a ``VFIO_USER_DEVICE_SET_IRQS`` message. + * ``VFIO_IRQ_INFO_AUTOMASKED`` indicates the IRQ type masks itself after= being + triggered, and the client must send an ``UNMASK`` action to receive new + interrupts. + * ``VFIO_IRQ_INFO_NORESIZE`` indicates ``VFIO_USER_SET_IRQS`` operations= setup + interrupts as a set, and new sub-indexes cannot be enabled without dis= abling + the entire type. +* index is the index of IRQ type being queried +* count describes the number of interrupts of the queried type. + +``VFIO_USER_DEVICE_SET_IRQS`` +----------------------------- + +This command message is sent by the client to the server to set actions for +device interrupt types. The VFIO IRQ set structure is defined in +```` (``struct vfio_irq_set``). + +Request +^^^^^^^ + ++-------+--------+------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------+--------+------------------------------+ +| flags | 4 | 4 | ++-------+--------+------------------------------+ +| | +-----+-----------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_IRQ_SET_DATA_NONE | | +| | +-----+-----------------------------+ | +| | | 1 | VFIO_IRQ_SET_DATA_BOOL | | +| | +-----+-----------------------------+ | +| | | 2 | VFIO_IRQ_SET_DATA_EVENTFD | | +| | +-----+-----------------------------+ | +| | | 3 | VFIO_IRQ_SET_ACTION_MASK | | +| | +-----+-----------------------------+ | +| | | 4 | VFIO_IRQ_SET_ACTION_UNMASK | | +| | +-----+-----------------------------+ | +| | | 5 | VFIO_IRQ_SET_ACTION_TRIGGER | | +| | +-----+-----------------------------+ | ++-------+--------+------------------------------+ +| index | 8 | 4 | ++-------+--------+------------------------------+ +| start | 12 | 4 | ++-------+--------+------------------------------+ +| count | 16 | 4 | ++-------+--------+------------------------------+ +| data | 20 | variable | ++-------+--------+------------------------------+ + +* *argsz* is the size of the VFIO IRQ set request payload, including any *= data* + field. Note there is no reply payload, so this field differs from other + message types. +* *flags* defines the action performed on the interrupt range. The ``DATA`` + flags describe the data field sent in the message; the ``ACTION`` flags + describe the action to be performed. The flags are mutually exclusive for + both sets. + + * ``VFIO_IRQ_SET_DATA_NONE`` indicates there is no data field in the com= mand. + The action is performed unconditionally. + * ``VFIO_IRQ_SET_DATA_BOOL`` indicates the data field is an array of boo= lean + bytes. The action is performed if the corresponding boolean is true. + * ``VFIO_IRQ_SET_DATA_EVENTFD`` indicates an array of event file descrip= tors + was sent in the message meta-data. These descriptors will be signalled= when + the action defined by the action flags occurs. In ``AF_UNIX`` sockets,= the + descriptors are sent as ``SCM_RIGHTS`` type ancillary data. + If no file descriptors are provided, this de-assigns the specified + previously configured interrupts. + * ``VFIO_IRQ_SET_ACTION_MASK`` indicates a masking event. It can be used= with + ``VFIO_IRQ_SET_DATA_BOOL`` or ``VFIO_IRQ_SET_DATA_NONE`` to mask an in= terrupt, + or with ``VFIO_IRQ_SET_DATA_EVENTFD`` to generate an event when the gu= est masks + the interrupt. + * ``VFIO_IRQ_SET_ACTION_UNMASK`` indicates an unmasking event. It can be= used + with ``VFIO_IRQ_SET_DATA_BOOL`` or ``VFIO_IRQ_SET_DATA_NONE`` to unmas= k an + interrupt, or with ``VFIO_IRQ_SET_DATA_EVENTFD`` to generate an event = when the + guest unmasks the interrupt. + * ``VFIO_IRQ_SET_ACTION_TRIGGER`` indicates a triggering event. It can b= e used + with ``VFIO_IRQ_SET_DATA_BOOL`` or ``VFIO_IRQ_SET_DATA_NONE`` to trigg= er an + interrupt, or with ``VFIO_IRQ_SET_DATA_EVENTFD`` to generate an event = when the + server triggers the interrupt. + +* *index* is the index of IRQ type being setup. +* *start* is the start of the sub-index being set. +* *count* describes the number of sub-indexes being set. As a special case= , a + count (and start) of 0, with data flags of ``VFIO_IRQ_SET_DATA_NONE`` di= sables + all interrupts of the index. +* *data* is an optional field included when the + ``VFIO_IRQ_SET_DATA_BOOL`` flag is present. It contains an array of bool= eans + that specify whether the action is to be performed on the corresponding + index. It's used when the action is only performed on a subset of the ra= nge + specified. + +Not all interrupt types support every combination of data and action flags. +The client must know the capabilities of the device and IRQ index before it +sends a ``VFIO_USER_DEVICE_SET_IRQ`` message. + +In typical operation, a specific IRQ may operate as follows: + +1. The client sends a ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_EVENTFD|VFIO_IRQ_SET_ACTION_TRIGGER)`` alo= ng + with an eventfd. This associates the IRQ with a particular eventfd on t= he + server side. + +#. The client may send a ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_EVENTFD|VFIO_IRQ_SET_ACTION_MASK/UNMASK)``= along + with another eventfd. This associates the given eventfd with the + mask/unmask state on the server side. + +#. The server may trigger the IRQ by writing 1 to the eventfd. + +#. The server may mask/unmask an IRQ which will write 1 to the correspondi= ng + mask/unmask eventfd, if there is one. + +5. A client may trigger a device IRQ itself, by sending a + ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_NONE/BOOL|VFIO_IRQ_SET_ACTION_TRIGGER)``. + +6. A client may mask or unmask the IRQ, by sending a + ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_NONE/BOOL|VFIO_IRQ_SET_ACTION_MASK/UNMASK)= ``. + +Reply +^^^^^ + +There is no payload in the reply. + +.. _Read and Write Operations: + +Note that all of these operations must be supported by the client and/or s= erver, +even if the corresponding memory or device region has been shared as mappa= ble. + +The ``count`` field must not exceed the value of ``max_data_xfer_size`` of= the +peer, for both reads and writes. + +``VFIO_USER_REGION_READ`` +------------------------- + +If a device region is not mappable, it's not directly accessible by the cl= ient +via ``mmap()`` of the underlying file descriptor. In this case, a client c= an +read from a device region with this message. + +Request +^^^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ + +* *offset* into the region being accessed. +* *region* is the index of the region being accessed. +* *count* is the size of the data to be transferred. + +Reply +^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ +| data | 16 | variable | ++--------+--------+----------+ + +* *offset* into the region accessed. +* *region* is the index of the region accessed. +* *count* is the size of the data transferred. +* *data* is the data that was read from the device region. + +``VFIO_USER_REGION_WRITE`` +-------------------------- + +If a device region is not mappable, it's not directly accessible by the cl= ient +via mmap() of the underlying fd. In this case, a client can write to a dev= ice +region with this message. + +Request +^^^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ +| data | 16 | variable | ++--------+--------+----------+ + +* *offset* into the region being accessed. +* *region* is the index of the region being accessed. +* *count* is the size of the data to be transferred. +* *data* is the data to write + +Reply +^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ + +* *offset* into the region accessed. +* *region* is the index of the region accessed. +* *count* is the size of the data transferred. + +``VFIO_USER_DMA_READ`` +----------------------- + +If the client has not shared mappable memory, the server can use this mess= age to +read from guest memory. + +Request +^^^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 8 | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. This address = must have + been previously exported to the server with a ``VFIO_USER_DMA_MAP`` mess= age. +* *count* is the size of the data to be transferred. + +Reply +^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 8 | ++---------+--------+----------+ +| data | 16 | variable | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. +* *count* is the size of the data transferred. +* *data* is the data read. + +``VFIO_USER_DMA_WRITE`` +----------------------- + +If the client has not shared mappable memory, the server can use this mess= age to +write to guest memory. + +Request +^^^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 8 | ++---------+--------+----------+ +| data | 16 | variable | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. This address = must have + been previously exported to the server with a ``VFIO_USER_DMA_MAP`` mess= age. +* *count* is the size of the data to be transferred. +* *data* is the data to write + +Reply +^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 4 | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. +* *count* is the size of the data transferred. + +``VFIO_USER_DEVICE_RESET`` +-------------------------- + +This command message is sent from the client to the server to reset the de= vice. +Neither the request or reply have a payload. + +``VFIO_USER_REGION_WRITE_MULTI`` +-------------------------------- + +This message can be used to coalesce multiple device write operations +into a single messgage. It is only used as an optimization when the +outgoing message queue is relatively full. + +Request +^^^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| wr_cnt | 0 | 8 | ++---------+--------+----------+ +| wrs | 8 | variable | ++---------+--------+----------+ + +* *wr_cnt* is the number of device writes coalesced in the message +* *wrs* is an array of device writes defined below + +Single Device Write Format +"""""""""""""""""""""""""" + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ +| data | 16 | 8 | ++--------+--------+----------+ + +* *offset* into the region being accessed. +* *region* is the index of the region being accessed. +* *count* is the size of the data to be transferred. This format can + only describe writes of 8 bytes or less. +* *data* is the data to write. + +Reply +^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| wr_cnt | 0 | 8 | ++---------+--------+----------+ + +* *wr_cnt* is the number of device writes completed. + + +Appendices +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Unused VFIO ``ioctl()`` commands +-------------------------------- + +The following VFIO commands do not have an equivalent vfio-user command: + +* ``VFIO_GET_API_VERSION`` +* ``VFIO_CHECK_EXTENSION`` +* ``VFIO_SET_IOMMU`` +* ``VFIO_GROUP_GET_STATUS`` +* ``VFIO_GROUP_SET_CONTAINER`` +* ``VFIO_GROUP_UNSET_CONTAINER`` +* ``VFIO_GROUP_GET_DEVICE_FD`` +* ``VFIO_IOMMU_GET_INFO`` + +However, once support for live migration for VFIO devices is finalized some +of the above commands may have to be handled by the client in their +corresponding vfio-user form. This will be addressed in a future protocol +version. + +VFIO groups and containers +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The current VFIO implementation includes group and container idioms that +describe how a device relates to the host IOMMU. In the vfio-user +implementation, the IOMMU is implemented in SW by the client, and is not +visible to the server. The simplest idea would be that the client put each +device into its own group and container. + +Backend Program Conventions +--------------------------- + +vfio-user backend program conventions are based on the vhost-user ones. + +* The backend program must not daemonize itself. +* No assumptions must be made as to what access the backend program has on= the + system. +* File descriptors 0, 1 and 2 must exist, must have regular + stdin/stdout/stderr semantics, and can be redirected. +* The backend program must honor the SIGTERM signal. +* The backend program must accept the following commands line options: + + * ``--socket-path=3DPATH``: path to UNIX domain socket, + * ``--fd=3DFDNUM``: file descriptor for UNIX domain socket, incompatible= with + ``--socket-path`` +* The backend program must be accompanied with a JSON file stored under + ``/usr/share/vfio-user``. + +TODO add schema similar to docs/interop/vhost-user.json. --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255297; cv=pass; d=zohomail.com; s=zohoarc; b=oLkPkva1AF++4wzEPrB+zKzmpX/h5j2HlW0a8V4RlKui1WH+eiVwKG3S18MX9kfSkwC8a4HjejHfer9pXzBv6QWyX9sARahE0OfXMUyRYC9wFQOtyO3TlFU2xgASXgTdudGFT2rbSkKbeDdxMdH3KK7ywEa8dxL7uywzO0II2I8= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255297; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Pxk0pmyQRnF1FFetlAKUuAq+ITTwv2bTb91xXUYCuUI=; b=VKpetcz1IixyRCqWb1Qvrr29agwHQLMF4H1jeVYo25a5RK+zLx+KRy5wjtAkJWj7o8C9DKWu1/KVNIkofJmZ+Uk53KGLRmh77YX5vyGUXU24UbWklrrq5IPs82B714p5M7Keyoi/8IRKSoQ0amXB7o1wOYfIXqqYRf3UMfnGYwA= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255297569648.9452568877882; Fri, 6 Jun 2025 17:14:57 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9Z-0002wZ-AD; Fri, 06 Jun 2025 20:11:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9X-0002vx-GZ for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:15 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9S-0006Nx-DL for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:15 -0400 Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556Kvtis017734; Fri, 6 Jun 2025 17:11:09 -0700 Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12on2138.outbound.protection.outlook.com [40.107.243.138]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 4747u3r8f8-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:08 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:07 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:07 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=Pxk0pmyQRnF1FFetlAKUuAq+ITTwv2bTb91xXUYCu UI=; b=H1VVm4kz1PywNwxjUGKGJSHV6UtHhAlZQIt+NBV0O8mEjimpQbGvoY6X6 1mL635UrR0196GAlNTyOmkkTwRs9hTsl1nFtW8Pwg8/XzPOa3JwstaSufOo7rF6J BuzfKIEn8nFQAvHoZ5OhVtYgTH5mqo+qdOFhdO+9W0CBqlaOEsr4dF0vV2SawtVv 16VmjS8jfa1qkosZ+/yFMtgorAMZbTsWEolE5YrM7/anhzgM4jpCOLHBwfo1H4KX 6mGommuzyxGiev1O4Y0vPiklOaLrMoQIfLpZloJVqNBER+8CvVLMMGVf4lEUC7tY XdJGHkPlNFD36otqa8ibsIssBujlA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=FhJlvaKsuhs0RclTdlW4JNwj7aOpCafsOsrvy7SqyFxCtxdBMU67hxSVWOPSW0YXzI4JlGZVpXmeCOyJw02E8cwjdAjSpNHnIkwObiOvfNSMZNyIcNfdG8f9e+QkPnga7UKAROous9SYZOPC2mBlAYJ8knjkEq8YzyOSTh9aQOIeA5U2hniV9QMACkOeClf/nYMg5MFKFrTTl3ICEqous/Eo3U4CukKbvfKFU8GNBuIs9w9j0Jmtj6e8xk7jutkK4geEY8TE05sac5kSWWZt74lPtlO8RqCCnPge4YbPUKE8HHfpBjnSQF6Qsctw4OmJL5p3jLtE+s15m3XNXdP6bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Pxk0pmyQRnF1FFetlAKUuAq+ITTwv2bTb91xXUYCuUI=; b=xPDISPnZRXddB/ATFl72M1YKWDwOdUL/SfW2pRiIi4yooqLpXqBDPmAC31TiAnPgTmb3BUdyM4q6jNEpksH674rLwkJ+dWwJ1/rV4Uce8MlSPu5Sibq6Mnkbb9u+U0jANBzBCRNLwQIoq8r/sRvUtgPBpadUo7lrZFDMqYodTeuGk/5k6KoChVBzxClDuQeFNZX3+oo+jpj89u1cEBcZKJpHH5uoxJ5nJRxnXqlhfUHjEkKSNJoCBD3spGYSq88nYGQEGLKu+wE3IK+TFqxiDEkV6LGfdZdEutzMQvyTOHB+REFPBSHlxorwz7oH8RgyvWz1PQZfhbTicvKJHBbpLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Pxk0pmyQRnF1FFetlAKUuAq+ITTwv2bTb91xXUYCuUI=; b=WrTsaJvw/zPf38QC+ckt3RoByj3eivstwCeSlsTrqFg7Vpu86spahKihhNDPyDaEYBDkXIRS/e3ksBMQlamONlJSunfflzYoTKsSLFdH1DT3ISW8v0dQK819F5t67Y23TXCzMVVVzfBB0B/tx1CFNLSGP5QCcqdIyVG857BezxMKVhTMHGSig/XM6q1C27n0u/6aZxBPYXVA1A8zaFR5PmIx/GF1QgR9wWqYJ42os6ipwGfahmjyi4qnRlxZaijAe3+bGlwbiC+OZn5vk6eiCshdduoHMWlA+Wjio9qADOblU6xrONkzuobd/IKefU91lQAv+dEp96D+DRI+wLcveQ== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 06/23] vfio-user: add vfio-user class and container Date: Fri, 6 Jun 2025 17:10:38 -0700 Message-ID: <20250607001056.335310-7-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: c41a4a17-2524-43ba-80ed-08dda557cca4 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?QWtCZmdqdG90aW4rcFFlWmhodUlsbXpXRHdPTTNCQkxaZVBMTDJGZThPL0Zi?= =?utf-8?B?QzhSNVZxaDR6MGFGRGlKNFByaVFjRmtHUGtJUUhDajZhM2dudlpNSHBWVWE1?= =?utf-8?B?cEY5ODdVQjdsenlCOWFqN3liOEhrSUFvRFg5eHhzdEVJWktGdm85VkdKdDg3?= =?utf-8?B?U0tDRkJ4ejZScFdzYXhVa3F3TXFUMk13MmJtdTZnQ0Z6OFhhcGVvaXhveWo3?= =?utf-8?B?RzlJU2d4Tndia3FJTjBxdkpWckRzWldhVmxrOENGU2tXQXFNaytIQXBuWmZL?= =?utf-8?B?VHBzR1JZYjZ6V2JaRE5KbU16cmxsT3MzUFNvQ1lPMVV1MHB4SjU4bWErL0Nv?= =?utf-8?B?djBVaExrWmhWNVJySkRlaG1FYnZIZTdnUjFpS3g5WDlQL2RDYy9uWmV0L0Nj?= =?utf-8?B?di9ZV2xlc0Q0dnphTUw0U29TUThWNlZzcVRXQ1U2UWZIUHpJYTJhNFB4SWtG?= =?utf-8?B?T0tSRnNTelNDTmQwbGxDMzJwZ25jcitGNFJZK2lMMVVibzNuRWxFeVVTU2hZ?= =?utf-8?B?MHQ2TVZvTGZzMWtaeUhEUUNwQ1hqMG5uSFZSYTkwallTSzhqN0xmNTFwcTR5?= =?utf-8?B?TkNFeFExTUZqV0dUeWdCT215UzhjVVBUOE1pOTlxT2pnNlRWY3lNVCsxRThM?= =?utf-8?B?dEFLaURpMVZHVXZPWktzMzJwOXR6QmEwWExBeVFlbEtMY1gwTytEMGVFaEJa?= =?utf-8?B?OExzKzdlQmpxY3pHUnZJc2hnN1R6QktNdEEySHhiR2NXOXlVSDRWcGFibmo3?= =?utf-8?B?TFdaYmtpN1U1OHRxS21seGliRHR2SkpIZXdOUDkvNmVWbWpRSHNUYVh2UUUv?= =?utf-8?B?aWtSVGp0U2ZWYnJzeDBxaGxIYUcxZjJLSGxXWTRQa01IRzhLTzlJcGJsa0JB?= =?utf-8?B?YStiSnBkTEY4eWVWNHFnTGpmdlg5clpQb2VJTU1MOEJCak5mbzE0UDAxN2Z5?= =?utf-8?B?NkRxcjc5RUxrNzJqZ1RNY3hGNlU2OUNDb3Y4VkR5TjdpV3ZRU0lLZHJ3Kzd0?= =?utf-8?B?MWJ1and2S3hjTkNCY0dkaTVNTXlzQVVGNE9mV3hzQytBWVByUDBnRkRhaXkr?= =?utf-8?B?dEdjemcxSFNrZis2MWszZGtZbU1KQWlxdHc3U1pleFM1bVQzdTVMT2RsTTc0?= =?utf-8?B?UmpVQVByRnBwSmNvd1dZbmIvYVlKYU1aOXZURG5uem4rODNHYTJQTUw3RG1j?= =?utf-8?B?d2FHekx3RzJONWI4Mi9Ua2JZSnpjTFZqUHlseE11cWNNWEZXS2hXZTBpbnRk?= =?utf-8?B?THo2bE9yanNBbXlhL2F6TnBpSXVmdzhQTlFhT0Uvd1hFWEw1ZGdCdDFrNDVV?= =?utf-8?B?M0orbzlxamRQblRTZTk2dXRRU2RtUWRBTytYUXpsZmtKYm53clFDOFhnMUhI?= =?utf-8?B?UkVobml2RHVtb1FRRUl0V0tRTzdKMEh2YWNPczdhb1NVMzlWejBuY2ptT0xL?= =?utf-8?B?Wk4rK0YzTjE0a1hTUUtybkRkVnBlamxRYSswVklzb2FlcTJWSldmUE51OGxK?= =?utf-8?B?MncwLzlWOTRDOURjR01KWExiZW1pWVk4SSsvY3VFbUhUYXQ5ODZOamUyZkc5?= =?utf-8?B?NmlKODUrbm5mYzlqbjQ1MVZidEdtRWJTbCtXSmM2MlRCbG5tMmdwUC9vajM4?= =?utf-8?B?UGpEV2pRaDdJTFhhUklxc0orcHBhU0k0RDAyZlY2bmJCZFo3V1djRFhTRTRj?= =?utf-8?B?bGs2QTNMVzAwdTBjU2VzOTVkdzU5M2JuSkxSd3NESC93bWdCZ0x5L1FGU3hY?= =?utf-8?B?b1M0bmNvdmx4N0FqamhQSkRjY0xPQzRDcnp1ZU9TanBrWXZzTkwrMEhxRHhU?= =?utf-8?B?NEVhbCs3Tyt3MFZ3cXBlT2RJSURLOExVeTk0WU1rZkxlUlkvZm8zS05xYU1J?= =?utf-8?B?WG9hbVBrNkxaL2QvU0YxcjBMZDRDamU5QlphejlPTFBWblF2TU9ubWV5YmE5?= =?utf-8?Q?R14yCT2r3yk=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ODBPRktXS2FWOUFNbEpsVjN4T1JZL3N5bDR2R3pvQkVYRVJySUdhS09FMmVx?= =?utf-8?B?TDdNRUh6NmtXeEFjOEpZTVMrNVMvS0h1OEc1RDdhNDdTa3pqclFTT3FaOGYy?= =?utf-8?B?WFJYRkNQRzBSNG5UT0dDdU9OcFVoTGpxQTQvYnVnLzJqdmFQem1VUU9PTEhs?= =?utf-8?B?ZTl4NHFuL1A4MmkrSVlvVnV3ZTZ4b00ydE1Ud2xublEwc0xGYXVwOUVPWjFQ?= =?utf-8?B?QnF0ZzNPUkgzT0l4c2dtNVN3U2k5TjF0czduR0hJOU1CL0s4cHpsUnVHVyti?= =?utf-8?B?UHNpQzZONzErTDdxS0diN0dkTXVZNjA2MmJ4cGVpeG92elpRMVNiZzJjQmQz?= =?utf-8?B?U1VKaGg2ZTM4QXBGYkpZZWl5bnRLemFmV1lyZkZ5V3o4WkJ6UENFb1VRWkxU?= =?utf-8?B?NmQzdEhVU0ozejNYMENxUGxNeTlHY2lndXpxUW5mb0NuUDVrdGxmOUlDWm5K?= =?utf-8?B?M25CemVxMzIrci9meTRRRERVbVZRZk1uR21aYmZsNzFGZkhhTUpJMlFFUWVi?= =?utf-8?B?M2FOY0pPRkY5aWdoTEsra3c3MTBUL0c0Zm9CNy93VzdYNFVGUVJ3TEhiWWNX?= =?utf-8?B?M1p1VWh5MlNEcEIyNjVYU1NGbDdyc3d6Z2wyWXRjR2plWjhiN0l4dG03QlRY?= =?utf-8?B?Ui92Ty96NEZ5WkR4Z1ZDZXNxQ1BDOGc3QlpXdEZMVG4wOHpkdWlOcXNYYm56?= =?utf-8?B?TlNQUXlGTm8reGNVZHRwaHY2TUYra0d5L2hIR3hvVkFWaWYzNmZMNThONHdt?= =?utf-8?B?NThQaWI1MFU3eEJZWHp0MEVoTVBuWlNDd0swelVSSktQNDVoM3NuamFUS0h4?= =?utf-8?B?YTV2Qks3aFNNN1NIdDdPdzErOUFWTGVMY0Z6ZnhEOTI5OWNEV1dEeEVFNUlV?= =?utf-8?B?clY4SUZoblRBTklHd3JadzBiOCtJNGJLZDFMTWhZNm5CcjU4ZkluLzNhWEVD?= =?utf-8?B?Vlp5RFIybWxRdGkwaFBPcXZoSkJkZ2s4bXVPdW52NWxlcHN1aE1QZjBycDhi?= =?utf-8?B?ZldGMlp0OXAyMVg3OGkvbTNwV0hjNG5iWlJwRC9YMlFLMm0xZUV1S05ualR3?= =?utf-8?B?eUc1OVRkakVmK1BES1pPNGJ2bGxmZlRlMGRvMGFhTlJva3ZWMEdxcWVpSUlO?= =?utf-8?B?cGZEYXZBOUp4TkFSandNdm00REZ4YXo3TFJZTHlpWUg5RldLYStvTVowQUJi?= =?utf-8?B?SGFhaFltRm9jRkt3eTdsSnJuRHIycUQ4eDhtaUhtYlNnZXpBZjZGUEdoY1lN?= =?utf-8?B?cEV0V01TOG13c2E0blVpYXZET2F6VVdmaklEbVl1dWtid2NFRURHRDI3eVR0?= =?utf-8?B?TWFpS1dpOXUycDhuZGlmODV4UmtkR0MyQnI4Qjg5ZzN0SkVDdWNmQzhQOGR2?= =?utf-8?B?WGRPeWRoaHBRQWx2aG5pOFdGeWRnRFNJWVhQMTM1RmFreE5taFB2ZEpXQVBs?= =?utf-8?B?TEgwZ2ttcEFXclJQKzB4Q2dLa2RkOWt3bHB3dmxTZ3RTeHJFc0RvYzIrWE9N?= =?utf-8?B?TzhYSXloK3Zxbm5KN2xBOVQ5a05TdkJ3RjR6R04zejZyY3BaaEQwZFljcmpL?= =?utf-8?B?SHhJeTd0T1QrbGh3aWM0TW00UmxUeVdNdExGNEhMTkxKZnowS05VU1IwMWpS?= =?utf-8?B?Rm52RnMrU1FqVkZJeGtBSU5waUZ2WFVleWN3di9kQ0xHQ2xpczRFaXFtR0Uw?= =?utf-8?B?Qzg5NGZVMjkvRmNKMC9UckgwbUUxd25zRXdoWHVVcTgyTkxqSXMra21lRDJz?= =?utf-8?B?R2EyQkNmYW1jSERNQ2FNNzBFZXl3R1prTmJtMkpSQ050RU05UjFOLytMNmxs?= =?utf-8?B?N1FiUUZEVk9wdmpVVkVqR0lsRlExUVZwQkZTRjRUS0pWK1RhNkErNnZrYUQw?= =?utf-8?B?aC9HbDRTbmFidko5SkVHY1hhR0wvMkpsUkVOdFdJQk1XZUlxV0dWaUpWWE1L?= =?utf-8?B?c1pCbFZyQUJuaEppcGRRbWQ5N3V0bGZqZGZzY3JmcUlUYzNrR3doYXloSVYr?= =?utf-8?B?Rk53cXkyWVpQODFmSWU3WHJ5NEkzWGNBY3RPSGZHdUlFMWY1TnU3ZEVhN2Zq?= =?utf-8?B?RDBQZno5amZtZmwyTGNlMzNmM0JCWXpnU0N3WmROY1REMG5Gb25DRk82Wldo?= =?utf-8?Q?YY12QJHIan8R0ymufLv/15a+9?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: c41a4a17-2524-43ba-80ed-08dda557cca4 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:07.2795 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wB1qgxE0eLNRArPxzVkMXT7zEofGmQ9k8hvTlwgm3j0KeWDblZaz8OGpJ/ty+XqVi2LcbBR2uKwmW91LsYRvtA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Authority-Analysis: v=2.4 cv=cpSbk04i c=1 sm=1 tr=0 ts=6843839c cx=c_pps a=j6Q7tV1r71QiW8XCak6Opg==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=Zlyvcu6OI-jkaoIL6xgA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: NSskSD858M_sJJe4QbbVIOQOwkvrLs1y X-Proofpoint-GUID: NSskSD858M_sJJe4QbbVIOQOwkvrLs1y X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX1mtmTytb7Po9 6BupjQZtVNVu10X6JDUyru/vhMbhB6Yo5Mg1HX7IooUisjRZRR8MzvyzFrrvYYj030cl0tboFCo yYFRRAExGK/vuSubRUWz+3kqiQVojEV7+7rw4V10hoYFGTKh37yrNzrmKYHnuN1Urs25uCTX12l g9DtOZHwjq79m87vyE4jKR1wJbpx6vhkDdPARNAOh1BPoZ7/gd6ZsDRvMC8JFVBHpPqRbxP4J1e 14euwHg4g0Vqiyi3HywQJUvr6QTGOdGI+WB2mXFoQcVJ1Uok2Mdqjc3cGXX54JH7rV0S5gjrpRp mqvv7IqN806OY7M/5rr8NLehVpyoL4Qe8PnTJ+E+DbnZOTaaQgtON1XRgucpJIiM5cVWE91dM/x eerAyT4fCTMlmOIBrR6AU/48jLkP6L6/ho3hn0AyLn4JZVU8cdvMD4XgaIZ6ml8fEtx3Qvj0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255300120116600 Introduce basic plumbing for vfio-user behind a new --enable-vfio-user-client option. We introduce VFIOUserContainer in hw/vfio-user/container.c, which is a container type for the "IOMMU" type "vfio-iommu-user", and share some common container code from hw/vfio/container.c. Add hw/vfio-user/pci.c for instantiating VFIOUserPCIDevice objects, sharing some common code from hw/vfio/pci.c. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- MAINTAINERS | 2 + hw/vfio-user/container.h | 25 +++ include/hw/vfio/vfio-container-base.h | 1 + hw/vfio-user/container.c | 212 ++++++++++++++++++++++++++ hw/vfio-user/pci.c | 155 +++++++++++++++++++ hw/meson.build | 1 + hw/vfio-user/meson.build | 9 ++ meson_options.txt | 2 + scripts/meson-buildoptions.sh | 4 + 9 files changed, 411 insertions(+) create mode 100644 hw/vfio-user/container.h create mode 100644 hw/vfio-user/container.c create mode 100644 hw/vfio-user/pci.c create mode 100644 hw/vfio-user/meson.build diff --git a/MAINTAINERS b/MAINTAINERS index dbc393989a..328bab8d19 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -4252,6 +4252,8 @@ M: John Levon M: Thanos Makatos S: Supported F: docs/devel/vfio-user.rst +F: hw/vfio-user/* +F: include/hw/vfio-user/* F: subprojects/libvfio-user =20 EBPF: diff --git a/hw/vfio-user/container.h b/hw/vfio-user/container.h new file mode 100644 index 0000000000..3cd3303e68 --- /dev/null +++ b/hw/vfio-user/container.h @@ -0,0 +1,25 @@ +/* + * vfio-user specific definitions. + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef HW_VFIO_USER_CONTAINER_H +#define HW_VFIO_USER_CONTAINER_H + +#include +#include + +#include "hw/vfio/vfio-container-base.h" + +/* MMU container sub-class for vfio-user. */ +typedef struct VFIOUserContainer { + VFIOContainerBase bcontainer; +} VFIOUserContainer; + +OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserContainer, VFIO_IOMMU_USER); + +#endif /* HW_VFIO_USER_CONTAINER_H */ diff --git a/include/hw/vfio/vfio-container-base.h b/include/hw/vfio/vfio-c= ontainer-base.h index 9d37f86115..28899ca0a6 100644 --- a/include/hw/vfio/vfio-container-base.h +++ b/include/hw/vfio/vfio-container-base.h @@ -109,6 +109,7 @@ vfio_container_get_page_size_mask(const VFIOContainerBa= se *bcontainer) #define TYPE_VFIO_IOMMU_LEGACY TYPE_VFIO_IOMMU "-legacy" #define TYPE_VFIO_IOMMU_SPAPR TYPE_VFIO_IOMMU "-spapr" #define TYPE_VFIO_IOMMU_IOMMUFD TYPE_VFIO_IOMMU "-iommufd" +#define TYPE_VFIO_IOMMU_USER TYPE_VFIO_IOMMU "-user" =20 OBJECT_DECLARE_TYPE(VFIOContainerBase, VFIOIOMMUClass, VFIO_IOMMU) =20 diff --git a/hw/vfio-user/container.c b/hw/vfio-user/container.c new file mode 100644 index 0000000000..2892845b4f --- /dev/null +++ b/hw/vfio-user/container.c @@ -0,0 +1,212 @@ +/* + * Container for vfio-user IOMMU type: rather than communicating with the = kernel + * vfio driver, we communicate over a socket to a server using the vfio-us= er + * protocol. + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include +#include +#include "qemu/osdep.h" + +#include "hw/vfio-user/container.h" +#include "hw/vfio/vfio-cpr.h" +#include "hw/vfio/vfio-device.h" +#include "hw/vfio/vfio-listener.h" +#include "qapi/error.h" +#include "trace.h" + +static int vfio_user_dma_unmap(const VFIOContainerBase *bcontainer, + hwaddr iova, ram_addr_t size, + IOMMUTLBEntry *iotlb, bool unmap_all) +{ + return -ENOTSUP; +} + +static int vfio_user_dma_map(const VFIOContainerBase *bcontainer, hwaddr i= ova, + ram_addr_t size, void *vaddr, bool readonly, + MemoryRegion *mrp) +{ + return -ENOTSUP; +} + +static int +vfio_user_set_dirty_page_tracking(const VFIOContainerBase *bcontainer, + bool start, Error **errp) +{ + error_setg_errno(errp, ENOTSUP, "Not supported"); + return -ENOTSUP; +} + +static int vfio_user_query_dirty_bitmap(const VFIOContainerBase *bcontaine= r, + VFIOBitmap *vbmap, hwaddr iova, + hwaddr size, Error **errp) +{ + error_setg_errno(errp, ENOTSUP, "Not supported"); + return -ENOTSUP; +} + +static bool vfio_user_setup(VFIOContainerBase *bcontainer, Error **errp) +{ + error_setg_errno(errp, ENOTSUP, "Not supported"); + return -ENOTSUP; +} + +static VFIOUserContainer *vfio_user_create_container(Error **errp) +{ + VFIOUserContainer *container; + + container =3D VFIO_IOMMU_USER(object_new(TYPE_VFIO_IOMMU_USER)); + return container; +} + +/* + * Try to mirror vfio_container_connect() as much as possible. + */ +static VFIOUserContainer * +vfio_user_container_connect(AddressSpace *as, Error **errp) +{ + VFIOContainerBase *bcontainer; + VFIOUserContainer *container; + VFIOAddressSpace *space; + VFIOIOMMUClass *vioc; + + space =3D vfio_address_space_get(as); + + container =3D vfio_user_create_container(errp); + if (!container) { + goto put_space_exit; + } + + bcontainer =3D &container->bcontainer; + + if (!vfio_cpr_register_container(bcontainer, errp)) { + goto free_container_exit; + } + + vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); + assert(vioc->setup); + + if (!vioc->setup(bcontainer, errp)) { + goto unregister_container_exit; + } + + vfio_address_space_insert(space, bcontainer); + + if (!vfio_listener_register(bcontainer, errp)) { + goto listener_release_exit; + } + + bcontainer->initialized =3D true; + + return container; + +listener_release_exit: + vfio_listener_unregister(bcontainer); + if (vioc->release) { + vioc->release(bcontainer); + } + +unregister_container_exit: + vfio_cpr_unregister_container(bcontainer); + +free_container_exit: + object_unref(container); + +put_space_exit: + vfio_address_space_put(space); + + return NULL; +} + +static void vfio_user_container_disconnect(VFIOUserContainer *container) +{ + VFIOContainerBase *bcontainer =3D &container->bcontainer; + VFIOIOMMUClass *vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); + + vfio_listener_unregister(bcontainer); + if (vioc->release) { + vioc->release(bcontainer); + } + + VFIOAddressSpace *space =3D bcontainer->space; + + vfio_cpr_unregister_container(bcontainer); + object_unref(container); + + vfio_address_space_put(space); +} + +static bool vfio_user_device_get(VFIOUserContainer *container, + VFIODevice *vbasedev, Error **errp) +{ + struct vfio_device_info info =3D { 0 }; + + vbasedev->fd =3D -1; + + vfio_device_prepare(vbasedev, &container->bcontainer, &info); + + return true; +} + +/* + * vfio_user_device_attach: attach a device to a new container. + */ +static bool vfio_user_device_attach(const char *name, VFIODevice *vbasedev, + AddressSpace *as, Error **errp) +{ + VFIOUserContainer *container; + + container =3D vfio_user_container_connect(as, errp); + if (container =3D=3D NULL) { + error_prepend(errp, "failed to connect proxy"); + return false; + } + + return vfio_user_device_get(container, vbasedev, errp); +} + +static void vfio_user_device_detach(VFIODevice *vbasedev) +{ + VFIOUserContainer *container =3D container_of(vbasedev->bcontainer, + VFIOUserContainer, bcontai= ner); + + vfio_device_unprepare(vbasedev); + + vfio_user_container_disconnect(container); +} + +static int vfio_user_pci_hot_reset(VFIODevice *vbasedev, bool single) +{ + /* ->needs_reset is always false for vfio-user. */ + return 0; +} + +static void vfio_iommu_user_class_init(ObjectClass *klass, const void *dat= a) +{ + VFIOIOMMUClass *vioc =3D VFIO_IOMMU_CLASS(klass); + + vioc->setup =3D vfio_user_setup; + vioc->dma_map =3D vfio_user_dma_map; + vioc->dma_unmap =3D vfio_user_dma_unmap; + vioc->attach_device =3D vfio_user_device_attach; + vioc->detach_device =3D vfio_user_device_detach; + vioc->set_dirty_page_tracking =3D vfio_user_set_dirty_page_tracking; + vioc->query_dirty_bitmap =3D vfio_user_query_dirty_bitmap; + vioc->pci_hot_reset =3D vfio_user_pci_hot_reset; +}; + +static const TypeInfo types[] =3D { + { + .name =3D TYPE_VFIO_IOMMU_USER, + .parent =3D TYPE_VFIO_IOMMU, + .instance_size =3D sizeof(VFIOUserContainer), + .class_init =3D vfio_iommu_user_class_init, + }, +}; + +DEFINE_TYPES(types) diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c new file mode 100644 index 0000000000..74b0c61f9b --- /dev/null +++ b/hw/vfio-user/pci.c @@ -0,0 +1,155 @@ +/* + * vfio PCI device over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include +#include "qemu/osdep.h" + +#include "hw/qdev-properties.h" +#include "hw/vfio/pci.h" + +#define TYPE_VFIO_USER_PCI "vfio-user-pci" +OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserPCIDevice, VFIO_USER_PCI) + +struct VFIOUserPCIDevice { + VFIOPCIDevice device; + char *sock_name; +}; + +/* + * Emulated devices don't use host hot reset + */ +static void vfio_user_compute_needs_reset(VFIODevice *vbasedev) +{ + vbasedev->needs_reset =3D false; +} + +static Object *vfio_user_pci_get_object(VFIODevice *vbasedev) +{ + VFIOUserPCIDevice *vdev =3D container_of(vbasedev, VFIOUserPCIDevice, + device.vbasedev); + + return OBJECT(vdev); +} + +static VFIODeviceOps vfio_user_pci_ops =3D { + .vfio_compute_needs_reset =3D vfio_user_compute_needs_reset, + .vfio_eoi =3D vfio_pci_intx_eoi, + .vfio_get_object =3D vfio_user_pci_get_object, + /* No live migration support yet. */ + .vfio_save_config =3D NULL, + .vfio_load_config =3D NULL, +}; + +static void vfio_user_pci_realize(PCIDevice *pdev, Error **errp) +{ + ERRP_GUARD(); + VFIOUserPCIDevice *udev =3D VFIO_USER_PCI(pdev); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); + VFIODevice *vbasedev =3D &vdev->vbasedev; + AddressSpace *as; + + /* + * TODO: make option parser understand SocketAddress + * and use that instead of having scalar options + * for each socket type. + */ + if (!udev->sock_name) { + error_setg(errp, "No socket specified"); + error_append_hint(errp, "Use -device vfio-user-pci,socket=3D= \n"); + return; + } + + vbasedev->name =3D g_strdup_printf("VFIO user <%s>", udev->sock_name); + + /* + * vfio-user devices are effectively mdevs (don't use a host iommu). + */ + vbasedev->mdev =3D true; + + as =3D pci_device_iommu_address_space(pdev); + if (!vfio_device_attach_by_iommu_type(TYPE_VFIO_IOMMU_USER, + vbasedev->name, vbasedev, + as, errp)) { + error_prepend(errp, VFIO_MSG_PREFIX, vbasedev->name); + return; + } +} + +static void vfio_user_instance_init(Object *obj) +{ + PCIDevice *pci_dev =3D PCI_DEVICE(obj); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); + VFIODevice *vbasedev =3D &vdev->vbasedev; + + device_add_bootindex_property(obj, &vdev->bootindex, + "bootindex", NULL, + &pci_dev->qdev); + vdev->host.domain =3D ~0U; + vdev->host.bus =3D ~0U; + vdev->host.slot =3D ~0U; + vdev->host.function =3D ~0U; + + vfio_device_init(vbasedev, VFIO_DEVICE_TYPE_PCI, &vfio_user_pci_ops, + DEVICE(vdev), false); + + vdev->nv_gpudirect_clique =3D 0xFF; + + /* + * QEMU_PCI_CAP_EXPRESS initialization does not depend on QEMU command + * line, therefore, no need to wait to realize like other devices. + */ + pci_dev->cap_present |=3D QEMU_PCI_CAP_EXPRESS; +} + +static void vfio_user_instance_finalize(Object *obj) +{ + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); + + vfio_pci_put_device(vdev); +} + +static const Property vfio_user_pci_dev_properties[] =3D { + DEFINE_PROP_UINT32("x-pci-vendor-id", VFIOPCIDevice, + vendor_id, PCI_ANY_ID), + DEFINE_PROP_UINT32("x-pci-device-id", VFIOPCIDevice, + device_id, PCI_ANY_ID), + DEFINE_PROP_UINT32("x-pci-sub-vendor-id", VFIOPCIDevice, + sub_vendor_id, PCI_ANY_ID), + DEFINE_PROP_UINT32("x-pci-sub-device-id", VFIOPCIDevice, + sub_device_id, PCI_ANY_ID), + DEFINE_PROP_STRING("socket", VFIOUserPCIDevice, sock_name), +}; + +static void vfio_user_pci_dev_class_init(ObjectClass *klass, const void *d= ata) +{ + DeviceClass *dc =3D DEVICE_CLASS(klass); + PCIDeviceClass *pdc =3D PCI_DEVICE_CLASS(klass); + + device_class_set_props(dc, vfio_user_pci_dev_properties); + dc->desc =3D "VFIO over socket PCI device assignment"; + pdc->realize =3D vfio_user_pci_realize; +} + +static const TypeInfo vfio_user_pci_dev_info =3D { + .name =3D TYPE_VFIO_USER_PCI, + .parent =3D TYPE_VFIO_PCI_BASE, + .instance_size =3D sizeof(VFIOUserPCIDevice), + .class_init =3D vfio_user_pci_dev_class_init, + .instance_init =3D vfio_user_instance_init, + .instance_finalize =3D vfio_user_instance_finalize, +}; + +static void register_vfio_user_dev_type(void) +{ + type_register_static(&vfio_user_pci_dev_info); +} + + type_init(register_vfio_user_dev_type) diff --git a/hw/meson.build b/hw/meson.build index b91f761fe0..791ce21ab4 100644 --- a/hw/meson.build +++ b/hw/meson.build @@ -39,6 +39,7 @@ subdir('uefi') subdir('ufs') subdir('usb') subdir('vfio') +subdir('vfio-user') subdir('virtio') subdir('vmapple') subdir('watchdog') diff --git a/hw/vfio-user/meson.build b/hw/vfio-user/meson.build new file mode 100644 index 0000000000..f1fee70c85 --- /dev/null +++ b/hw/vfio-user/meson.build @@ -0,0 +1,9 @@ +vfio_user_ss =3D ss.source_set() +vfio_user_ss.add(files( + 'container.c', + 'pci.c', +)) + +if get_option('vfio_user_client').enabled() + specific_ss.add_all(vfio_user_ss) +endif diff --git a/meson_options.txt b/meson_options.txt index a442be2995..97d3db44cd 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -109,6 +109,8 @@ option('multiprocess', type: 'feature', value: 'auto', description: 'Out of process device emulation support') option('relocatable', type : 'boolean', value : true, description: 'toggle relocatable install') +option('vfio_user_client', type: 'feature', value: 'disabled', + description: 'vfio-user client support') option('vfio_user_server', type: 'feature', value: 'disabled', description: 'vfio-user server support') option('dbus_display', type: 'feature', value: 'auto', diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index f09ef9604f..2c5673769a 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -201,6 +201,8 @@ meson_options_help() { printf "%s\n" ' vdi vdi image format support' printf "%s\n" ' vduse-blk-export' printf "%s\n" ' VDUSE block export support' + printf "%s\n" ' vfio-user-client' + printf "%s\n" ' vfio-user client support' printf "%s\n" ' vfio-user-server' printf "%s\n" ' vfio-user server support' printf "%s\n" ' vhdx vhdx image format support' @@ -529,6 +531,8 @@ _meson_option_parse() { --disable-vdi) printf "%s" -Dvdi=3Ddisabled ;; --enable-vduse-blk-export) printf "%s" -Dvduse_blk_export=3Denabled ;; --disable-vduse-blk-export) printf "%s" -Dvduse_blk_export=3Ddisabled = ;; + --enable-vfio-user-client) printf "%s" -Dvfio_user_client=3Denabled ;; + --disable-vfio-user-client) printf "%s" -Dvfio_user_client=3Ddisabled = ;; --enable-vfio-user-server) printf "%s" -Dvfio_user_server=3Denabled ;; --disable-vfio-user-server) printf "%s" -Dvfio_user_server=3Ddisabled = ;; --enable-vhdx) printf "%s" -Dvhdx=3Denabled ;; --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255319; cv=pass; d=zohomail.com; s=zohoarc; b=fthBO3jlzZD96Y4el8WEAMIUf8/EbWzKbl0hPahprQz/hUtnOJbSGnf1jQm9nVI24mTOiJm4rf5zCrehXzZNpvMqIyEHq8kJ635zqVuikpg1L6wLtnMLNeyomxB4NnARgPq4TyGcG7slFsa9TgmWUaGWlyeOeJurtZ7xMg83Q7I= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255319; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Lfi3fqoZwgLm7PZmiEhG+WX0QYLAq3BqEjm16KRZuqc=; b=dXEzzPqNV5sX19OLQad2zyzBUgsMMqbJ+doE86L66d2x8N5W7T/DpafZt0c1rtxEOuvp5uYgmfdA1tSm3PN8PfwvHEt5t66VLnkIHFmSXi5TW0De6pHNogI7wTS/HsdPiZSXMXubowlpOO9hjiOxSw8qxZOiwg1UudQKtUW1hlM= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255319311209.8414269073669; Fri, 6 Jun 2025 17:15:19 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9a-0002xQ-AW; Fri, 06 Jun 2025 20:11:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9X-0002vw-Ek for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:15 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9U-0006OM-Pn for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:15 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556NqBwU023142; Fri, 6 Jun 2025 17:11:11 -0700 Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11on2122.outbound.protection.outlook.com [40.107.223.122]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 472njsxqan-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:10 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:08 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=Lfi3fqoZwgLm7PZmiEhG+WX0QYLAq3BqEjm16KRZu qc=; b=skYCn+eoOjZUCzaFDv1pVfNUf0yuVZaOfOvoC4NPcdV1k0b/BitqL5kum GzPmV+Ss1NQn+JaK1r+6yxzHPOydN34SSJiU5Bh+zPgQoWNvQQly93vnC2Yq4Ju+ Fl8QOHqf5Rxa9d9IECFXKFkVdWu5jZGzODGLdCtHN5A9ECf/Pc+jpfy7FoJoTsLc 37/1FAsOY1s0LL3sGu/oczUoMGNyJyRlLk1yTC/raHmM7rbFookEbZKaNdEgKm8N fWmltrfqXHcizOSDQmjpkmc36HVlOyU60j92gNSQ4UJ5DNjpyVxgCSWk6G9rK1hN Cm0hOBf4UZr5qA2aDniWsBiXIwVnA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HdsvwanH7XOLGiR4S9szyg70LmwiEjOrKJd2qv1n/uh3JdWoYl9EkLclDuquiUBdOMkfPj0kK3fmM+VIQTvWxsx6SzvsuZsALJfvP9uDQaMhCOUnMFrDNC0K49OeB2tiPiOY3o1Y9Sw2j8nQGmWkoRChAWquwlS91yVcomSSoevEFPgezGtGx87G0hxU0eBDePB07b/DhgeBDU7GLCGGuEhJs4UytLkexBDC78DN5sgUIrLW2X8EWNHg62k+UcKI1nbAOr9XsX9+ymB8uAOXnUOeqf6S9phT577zrHyvzsp3HqtFBROcm0U22u9wdN24giUXzcnilRRQdjxl+gA0MQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Lfi3fqoZwgLm7PZmiEhG+WX0QYLAq3BqEjm16KRZuqc=; b=kQ/YCI7Z48AtIxttoMit8p81nJzU9R5zYx3OeT/ij8ZfcTByi+AYgF/QBWL/VKuG/lMZc6vUwQ+dBbBiwT6+9l/D6Bu9AMXLhOj8ebFiuEqtA+3NY5JHiaqDjI3nKNvF4oaV/PwdOb8PUQEf+jTMu+ss8IqTZG67urCwsSLkU390d86VPWuch5Lg6wMxGXAYR8OnWV6XrB8XaClBsQEBFuCSS9Guu7HSZIX0iS1rHbhSdSfpOq3EkjGDjI/8pF0r/X/p9Kk5ZPysZdfDK849HEDG099lAqbyDLCO5OgJHnOXBotGQK/WrwMzkrz2ETPH4VRqY+vheG5/naBOC+bsEA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Lfi3fqoZwgLm7PZmiEhG+WX0QYLAq3BqEjm16KRZuqc=; b=WQBZDQXICz6OOOkUEZuv5nUo1qdl6niWctyoxpt6mg/C8Yg0A3lkJxMy41rd+N24VKpvSDR44Zk9DMXoqJBUl9O4fOxiPcYIeiQwKKmBmteZtdbCBmPCe42ObuKMMxheyhiIGFPuuRlZtSDPTSBBNqiaQEvMJHRg0Q+oHp1zZkPlqzfhvCkv43KUymoeLKGHAe46vE3Px0QJdA71zkAHN87g9DH9lKJAZsNc2R6OpwnXNR1RIZN0LK0V/I3MMIsOG7DbG7aIXXIpRqTaR9uHZoZvoeA4jSizxnp6A0D+xcrIPwqkEqF6D1PMtG80r8/UjPPiOKDlrMQgdZJhNtdrCg== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 07/23] vfio-user: connect vfio proxy to remote server Date: Fri, 6 Jun 2025 17:10:39 -0700 Message-ID: <20250607001056.335310-8-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: 8adf2060-d392-43de-c991-08dda557cd8e x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?M3JhQzZPN1dpTHJkSll6S0lXUURTbkdRRlhRdnlIU2tuQVBwTnRsdmphVUFR?= =?utf-8?B?b1I1Wm5UQXZjSThUNllNU2E5TllwZ3p3VkRlcElEVXRqMGtFcHR6MmNCQkRT?= =?utf-8?B?ODRyYXVaV0F1c1pqcE50TEppZE9lTS9ZV1VQK25Bd2tFSlAvbHhuR0Q0VDdM?= =?utf-8?B?bTcvdUNDOTFvaXArNURWL3haYjJrdE1FOGs3Rllkc0xDOGJmSzNwYUNhKzdD?= =?utf-8?B?M2JvMVM3T0xGSlJhSmpCWTRyWlNzSDJWUWd1VEtNbTROZGsrc2J6c1VpS2Jo?= =?utf-8?B?ZEwvRXVBMTR1Q0R3cDgxS1RUdmdHVndRMmNZOXNheFRnU2dIM1VQUStwSEtZ?= =?utf-8?B?WGRyNEJLTG1nYzIvSUpKRTVRbllDUDBmaXNVT2RwM3dBTGhjckxlcnBRcjhO?= =?utf-8?B?QXlBV28zV2x2cmpHVHZNWlZaam1LR0l6ZWgvUzZCK1RjWU1KZjMzOFV3WDVD?= =?utf-8?B?eWhUeVBxWllUYUxOVWhVSkxQOHJjTDNrMjVCNlk0MmxJVExsS2FiNDhvcDVz?= =?utf-8?B?NC9MN3Zlanp4UFZNN1NhUk5VLzVsRTgyYUJaejRpYmEwZTVwZnpPbHRKWDh5?= =?utf-8?B?bVk3ZUorNTVudjBqVUlMTE9tbzJSYmNMWXVKQ3NOWWdSL2wvZWExNnNiQWlX?= =?utf-8?B?ZnFKUldCVk1HSEgzcmRKRGlHQnBQTjB1dW4zYTdtZGVZaWJRYnpjTHFoQXpP?= =?utf-8?B?SFpUUk0zWEUzaVFmNXBDeVdKd3doTE5EUms4WjBmNTVPMWIzOEVIZVhaQ3Jw?= =?utf-8?B?eDVmVkk0bEQzOWtKeXp3RzhzRE04TTBkeVVJMThCVGJ2Mi9paGhUdlZJSGZS?= =?utf-8?B?M1owTE9IMkVSTXB3ZThqTmorRmNEYkViUEdQOG9FTXNUTWswREpUQi9QOEVB?= =?utf-8?B?NDVTbzZ0UzRhMGlGQ1RIK2ZiS3grVXd1Z3JyRHdaN3MrUXloY2VGK2VnZVJx?= =?utf-8?B?ZGd4RUNaUVdlSUNkNGpRUk0zR1dnWjNQbTdSS3RQUnJQR2dEU1J0Q1FtaXJr?= =?utf-8?B?RnBMTEtURVdJTjd1eUxvajdtekwrUUhPQnVtRDk4TWRhV3g5dUdudkJNcGUy?= =?utf-8?B?Q0hiVEc0U1ZVUURZREd1RDZxMkhYRWl1MytVQU9ML0Z0N2tVUUY4S01FNVZq?= =?utf-8?B?c2FMNjNUZTEwdWRxVU13V3JiRmxsOGd2MzRqWkdOVzk1NW1sWkJDZDZlbWt2?= =?utf-8?B?aTN4QlNoYUZYRUdlWDBseVdjdkNpeVd0ejRrRkNyTmRhRTlaY1EvL3hySDNa?= =?utf-8?B?bFNYNUxZT2l4aERkNnFkSW42KzRPYjk1QkpLcTBpVm5RLzRraStaWkl5V082?= =?utf-8?B?bmVCVlJUVkovcERicW9tek9NZDAzbE40Y2JkS0lLcUkwMlpzV3h6V0NyRkdT?= =?utf-8?B?WGZLNmNYTWxEV3J1eEhqYTBkbHdoMmJPclY4OWw5VWZzNGF5cXBwa3NGcDBH?= =?utf-8?B?RkZ1cVVaVGNrU1ZGUkhDZEN6amY1VU9POGhoZ3prdGRhVHZrMnFtM2dOM0lZ?= =?utf-8?B?bERuN1FJTG5iZXpJZFBEYXM5cCtVZG01dGxJcDhhTkZLM295b0xidzNNTjRT?= =?utf-8?B?dUcyV25UWXVMRHVwckQxOU1EN2Rvd3paS2lrZE8xay9nelFoVkhvMlRrYkZM?= =?utf-8?B?Uy9WbmRMLzNLaTlTSjZHUHYwMzFhRmNqaFJYU04rcVdidlA1QXptWHBrSkhi?= =?utf-8?B?eHpjZzlTR2ZVaWlXRGJnV3N4MDU3M2hMeEdxQVVJcHZPQXBFVUw4MFlIcVFv?= =?utf-8?B?MzhZQ3ArUWc4TDJkemlibEw5WXVkVExrQkNidUpyQjJZVFR2cnlyNGtSTjM3?= =?utf-8?B?ZGtHZWVOUUZqbzFZa0xGRE1JVFF4VEtjL2JmOHdKTzIxaE9xZyt4OXRoTWto?= =?utf-8?B?dFpldEZQMlVoQUZUb0ZGWithWGR4WkJyVkdhS3ZBYXd6WG9mUzRiN1d4UDI2?= =?utf-8?Q?mSS3YymiLYA=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?THArQXBoZ2QzQVRaMDNnV0RjOWliUE9mWjJicVdCU0NrT0NCb0VHK0pMUmhi?= =?utf-8?B?UUlweTA1WmJOMnZVWGtjT3RleTJhRk1YM2VZbFBVSEFsWHNKTFByeWxuRlpy?= =?utf-8?B?NDI1S0VFMTBLTGk4bkZLQ1FpSFExLzl3Q2o2TWc2dThNSGE4OENrZHZoQjhN?= =?utf-8?B?MUpjbXdCVThRenlZS1kxTHAybkwxY2dNcjN3aE5CVk53bVNweGMxS1d5bEJa?= =?utf-8?B?THhIc0pxbUtuV1NlbHMrS1YvT3BlTlp1WUZKRDRDUmNhczU1aVE1S2VGVnNw?= =?utf-8?B?WnU5aUo0bUFlaWZXMTJuQUozTE9oKzJtS1RlOEltS29vVmtyK0ZUOWpsUlJ5?= =?utf-8?B?bmJuaEtaUUpLWjJjekVPUGFDTFhiWklqN1NGN3ZsMG16OXdNTEQwM0VNMHJ4?= =?utf-8?B?cGo5aTMvWGM2M1pHaWlReFR0M3JRL0NLT1FqNzQ1QjZ6T3pyaW54bXNpWHE3?= =?utf-8?B?UjBiUkUrMEJtUVdsYk1yS3hvMXZkOXl5YVdOY2gyL3lianJuY2xFNzZpLytG?= =?utf-8?B?VWtQdSt0WEpMa082SzdFaEsxTHBZV1Rpc3pZQUYyNTFWLy9yaWNGZk9yNHlF?= =?utf-8?B?Rno3MlZ6MnBqMXVzODVXM3FWak9nWkk4SXVETEtYQ09QYmVLNjNtS0cwaEYy?= =?utf-8?B?MGNnNGIwdWlBamlKQjBBNDhZUk5GT1A2RGtXOEdCb0F4WEJzZEJkQjl3SnFR?= =?utf-8?B?NEJnejVrN2p5NUw5WERFbjZ4emZYTnV0N0JCbWVhOUFnSm5XWHMzL0paL0Nh?= =?utf-8?B?T1lkeWVCNkJGYzFVY0dIRTNiRHNFdVZFMXlNZ2NXYVI4Z0RKMzhQN3Vxa3Ft?= =?utf-8?B?Q2JzTjBGa1ZUM25nLzFQeFhnazVYblhvbDM5SWxvWGdjTjR2bFR4dHpiQWxJ?= =?utf-8?B?R0wwcm5MejlmaXpjQmtuZFdwbUxtSjdzN3Rxc1lXU1VhZGNSWkU2eEphaXQ2?= =?utf-8?B?QVZQaHY1UE1RNk4wYnNBR3RHbzJWTmNsQTA0bWQyUGJUdlhyaWl5OFNhVnFL?= =?utf-8?B?TjBGZ005Z1ZUc29mUnpIWEFBaStxcXJGc0lVRjRFQUp0Z3NhQUVkcXVQRnp5?= =?utf-8?B?d0RuaWNwaFpxT3NMQ2NkSlMzVTloVmJDcThxYnd0UFNjTTNaUitwWG1STlAx?= =?utf-8?B?bDErNHB0aGNKQ0FHMHIxaGRKYWNJektaaWZDT1owRW12b3U3RGhXamF1MzVQ?= =?utf-8?B?Z1RocElOVTN4QTYrVllrTzFuTkl0OGVQRHRTQVZqL2pKcTJIdlRUR1dXeGc0?= =?utf-8?B?cUljalQxanNHdDlsR2N0Y1hnVW9ZcnBJSlptZGowbk8zaXYwVG9KLzlKVVFj?= =?utf-8?B?MDJPR2Njd3VaVzd6ZExmS2MxRkhyVVhIaG9ORVNzdGJmU2xuOUswM3U0SFk5?= =?utf-8?B?bVQ1SjYxWDdSMTU3WVVEalpOb3kvVXJlV3dsNXczekllWVpVczZKckpUa2Vo?= =?utf-8?B?SWh0LzVYMmR6Q2VGQ29wT3Ztd3lwV0ZzdU9mMDJ4R0hNNGxUZVpEN1Q2b0JG?= =?utf-8?B?RUFpWGJxendLOTN4S29CZklsSE9OTzl2cFpSZWZGWW1EK2k1K1FJNisrZ3Vy?= =?utf-8?B?a0w1c0hSWjhSR09kQ1RnMnB5YmRuUUdkY1Rkc29MSU1TM3gvT3l4blQ5K0ov?= =?utf-8?B?L2dxUmI3dlVNMlk0Qm5wSFFHL3FmZVBMN3FQeEpMT1R3VklXdUlHR1ZSc3dC?= =?utf-8?B?QmxUSGhSclhLWFo3RmRZUDRMcGtNVi9aZE5TcHJHb0QrUVJRdG1FOUNpSHNB?= =?utf-8?B?b211emowWWtyUFczWlVoWTc5NnphQmIvOTByc3I3ZHB1emtkZ3BmVDI3V0k1?= =?utf-8?B?aXNUelZQNFZwcVR5OUZxTmFjUGZpcXc4VGVRTVJPYkluTWoza0JKUkhRamRn?= =?utf-8?B?Zzl4SkZuYThiN28rbkNoYnR5c2YxeTJJaE1iWWlsb1pVcGRMbjFRUWxQbks0?= =?utf-8?B?aEt1a2JwQVBuM1I2TmVsb2hWbWhaT0NDM0s5QVYway9BKzUwL1VrS0RzMm1a?= =?utf-8?B?aFdHMW9LMFhLdFZPYzNTYTBYK1JGbTdia1VmcXh2dy96d1h1WHRvRHBrcWhq?= =?utf-8?B?UEkyVHdQd0NzZ2o5bEpYaFlHVTFNcDAxUjd2aWdSNktkVVFQRkRTdkxFTWZk?= =?utf-8?Q?RFz46UGlst4x1BXVK4w4LRQ24?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8adf2060-d392-43de-c991-08dda557cd8e X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:08.7640 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nNKQU9lLHhczih8+APq1XGkl0sSQG2qHSsilExV0WbHt5nh+jrkVibVfdjhbDXqEoFE9QRjGWIhQanf84HOIvA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Authority-Analysis: v=2.4 cv=f9JIBPyM c=1 sm=1 tr=0 ts=6843839e cx=c_pps a=E00YimykegQh2ruOzwX4zg==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=vhHEnE-nI2wpHcbFAcgA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-GUID: dcrF1k0oejixIN6JznsyFcSZlsLdW-bX X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX5zxuADkksZ2Q f5XSz0x5cnlei9mo8SBGH5hnlW53FoWehYiSXacgU9xdarSOmZdLxMDiYnOwJ4ydfAhcGtvD3EH YNg+xmAYRbMrsqc11TLSGWkoKp/IG+H9DaT76Cz7aq2r77z5FnAGypJVBguObzyeqKhCEypDZ2O F5/un80cdelewoxrVuRMyiwp1CkKBnQl2SKl9z+4uZosN5K4d/220TAustTDluv41oiD7Vhhgxu vjnbbJApH6GvoDIpVta1JFgPcx025InG+VMy/mkq9vLIhLaRVDbqfVtoDHk14tkJdjytZA5VEVU uXqi7F9KgT08OKNhso3jMtZDL3EIZO/uMg3Al9oyuYzb6cHIrmr9oBLEGFDicPNsnDqKrM0Y0Ra PkyAGav/AnMEGQXgU/PHWZvFAhA2sDmgnWhH8u0e+j03o3Iw4EOoFrk4f+ywzb036i4bmGqR X-Proofpoint-ORIG-GUID: dcrF1k0oejixIN6JznsyFcSZlsLdW-bX X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255320419116600 Introduce the vfio-user "proxy": this is the client code responsible for sending and receiving vfio-user messages across the control socket. The new files hw/vfio-user/proxy.[ch] contain some basic plumbing for managing the proxy; initialize the proxy during realization of the VFIOUserPCIDevice instance. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/proxy.h | 82 +++++++++++++++++ include/hw/vfio/vfio-device.h | 2 + hw/vfio-user/pci.c | 17 ++++ hw/vfio-user/proxy.c | 165 ++++++++++++++++++++++++++++++++++ hw/vfio-user/meson.build | 1 + 5 files changed, 267 insertions(+) create mode 100644 hw/vfio-user/proxy.h create mode 100644 hw/vfio-user/proxy.c diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h new file mode 100644 index 0000000000..60fe3e0b6d --- /dev/null +++ b/hw/vfio-user/proxy.h @@ -0,0 +1,82 @@ +#ifndef VFIO_USER_PROXY_H +#define VFIO_USER_PROXY_H + +/* + * vfio protocol over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "io/channel.h" +#include "io/channel-socket.h" + +typedef struct { + int send_fds; + int recv_fds; + int *fds; +} VFIOUserFDs; + +enum msg_type { + VFIO_MSG_NONE, + VFIO_MSG_ASYNC, + VFIO_MSG_WAIT, + VFIO_MSG_NOWAIT, + VFIO_MSG_REQ, +}; + +typedef struct VFIOUserMsg { + QTAILQ_ENTRY(VFIOUserMsg) next; + VFIOUserFDs *fds; + uint32_t rsize; + uint32_t id; + QemuCond cv; + bool complete; + enum msg_type type; +} VFIOUserMsg; + + +enum proxy_state { + VFIO_PROXY_CONNECTED =3D 1, + VFIO_PROXY_ERROR =3D 2, + VFIO_PROXY_CLOSING =3D 3, + VFIO_PROXY_CLOSED =3D 4, +}; + +typedef QTAILQ_HEAD(VFIOUserMsgQ, VFIOUserMsg) VFIOUserMsgQ; + +typedef struct VFIOUserProxy { + QLIST_ENTRY(VFIOUserProxy) next; + char *sockname; + struct QIOChannel *ioc; + void (*request)(void *opaque, VFIOUserMsg *msg); + void *req_arg; + int flags; + QemuCond close_cv; + AioContext *ctx; + QEMUBH *req_bh; + + /* + * above only changed when BQL is held + * below are protected by per-proxy lock + */ + QemuMutex lock; + VFIOUserMsgQ free; + VFIOUserMsgQ pending; + VFIOUserMsgQ incoming; + VFIOUserMsgQ outgoing; + VFIOUserMsg *last_nowait; + enum proxy_state state; +} VFIOUserProxy; + +/* VFIOProxy flags */ +#define VFIO_PROXY_CLIENT 0x1 + +VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp); +void vfio_user_disconnect(VFIOUserProxy *proxy); + +#endif /* VFIO_USER_PROXY_H */ diff --git a/include/hw/vfio/vfio-device.h b/include/hw/vfio/vfio-device.h index a23ef4ea13..09f1a21bf8 100644 --- a/include/hw/vfio/vfio-device.h +++ b/include/hw/vfio/vfio-device.h @@ -46,6 +46,7 @@ typedef struct VFIOMigration VFIOMigration; =20 typedef struct IOMMUFDBackend IOMMUFDBackend; typedef struct VFIOIOASHwpt VFIOIOASHwpt; +typedef struct VFIOUserProxy VFIOUserProxy; =20 typedef struct VFIODevice { QLIST_ENTRY(VFIODevice) next; @@ -86,6 +87,7 @@ typedef struct VFIODevice { QLIST_ENTRY(VFIODevice) hwpt_next; struct vfio_region_info **reginfo; int *region_fds; + VFIOUserProxy *proxy; } VFIODevice; =20 struct VFIODeviceOps { diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 74b0c61f9b..f1a2d4374d 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -14,6 +14,7 @@ =20 #include "hw/qdev-properties.h" #include "hw/vfio/pci.h" +#include "hw/vfio-user/proxy.h" =20 #define TYPE_VFIO_USER_PCI "vfio-user-pci" OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserPCIDevice, VFIO_USER_PCI) @@ -55,6 +56,8 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error = **errp) VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); VFIODevice *vbasedev =3D &vdev->vbasedev; AddressSpace *as; + SocketAddress addr; + VFIOUserProxy *proxy; =20 /* * TODO: make option parser understand SocketAddress @@ -67,6 +70,15 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error= **errp) return; } =20 + memset(&addr, 0, sizeof(addr)); + addr.type =3D SOCKET_ADDRESS_TYPE_UNIX; + addr.u.q_unix.path =3D udev->sock_name; + proxy =3D vfio_user_connect_dev(&addr, errp); + if (!proxy) { + return; + } + vbasedev->proxy =3D proxy; + vbasedev->name =3D g_strdup_printf("VFIO user <%s>", udev->sock_name); =20 /* @@ -112,8 +124,13 @@ static void vfio_user_instance_init(Object *obj) static void vfio_user_instance_finalize(Object *obj) { VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); + VFIODevice *vbasedev =3D &vdev->vbasedev; =20 vfio_pci_put_device(vdev); + + if (vbasedev->proxy !=3D NULL) { + vfio_user_disconnect(vbasedev->proxy); + } } =20 static const Property vfio_user_pci_dev_properties[] =3D { diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c new file mode 100644 index 0000000000..ac481553ba --- /dev/null +++ b/hw/vfio-user/proxy.c @@ -0,0 +1,165 @@ +/* + * vfio protocol over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include + +#include "hw/vfio/vfio-device.h" +#include "hw/vfio-user/proxy.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "qemu/lockable.h" +#include "system/iothread.h" + +static IOThread *vfio_user_iothread; + +static void vfio_user_shutdown(VFIOUserProxy *proxy); + + +/* + * Functions called by main, CPU, or iothread threads + */ + +static void vfio_user_shutdown(VFIOUserProxy *proxy) +{ + qio_channel_shutdown(proxy->ioc, QIO_CHANNEL_SHUTDOWN_READ, NULL); + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, NULL, + proxy->ctx, NULL, NULL); +} + +/* + * Functions only called by iothread + */ + +static void vfio_user_cb(void *opaque) +{ + VFIOUserProxy *proxy =3D opaque; + + QEMU_LOCK_GUARD(&proxy->lock); + + proxy->state =3D VFIO_PROXY_CLOSED; + qemu_cond_signal(&proxy->close_cv); +} + + +/* + * Functions called by main or CPU threads + */ + +static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D + QLIST_HEAD_INITIALIZER(vfio_user_sockets); + +VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp) +{ + VFIOUserProxy *proxy; + QIOChannelSocket *sioc; + QIOChannel *ioc; + char *sockname; + + if (addr->type !=3D SOCKET_ADDRESS_TYPE_UNIX) { + error_setg(errp, "vfio_user_connect - bad address family"); + return NULL; + } + sockname =3D addr->u.q_unix.path; + + sioc =3D qio_channel_socket_new(); + ioc =3D QIO_CHANNEL(sioc); + if (qio_channel_socket_connect_sync(sioc, addr, errp)) { + object_unref(OBJECT(ioc)); + return NULL; + } + qio_channel_set_blocking(ioc, false, NULL); + + proxy =3D g_malloc0(sizeof(VFIOUserProxy)); + proxy->sockname =3D g_strdup_printf("unix:%s", sockname); + proxy->ioc =3D ioc; + proxy->flags =3D VFIO_PROXY_CLIENT; + proxy->state =3D VFIO_PROXY_CONNECTED; + + qemu_mutex_init(&proxy->lock); + qemu_cond_init(&proxy->close_cv); + + if (vfio_user_iothread =3D=3D NULL) { + vfio_user_iothread =3D iothread_create("VFIO user", errp); + } + + proxy->ctx =3D iothread_get_aio_context(vfio_user_iothread); + + QTAILQ_INIT(&proxy->outgoing); + QTAILQ_INIT(&proxy->incoming); + QTAILQ_INIT(&proxy->free); + QTAILQ_INIT(&proxy->pending); + QLIST_INSERT_HEAD(&vfio_user_sockets, proxy, next); + + return proxy; +} + +void vfio_user_disconnect(VFIOUserProxy *proxy) +{ + VFIOUserMsg *r1, *r2; + + qemu_mutex_lock(&proxy->lock); + + /* our side is quitting */ + if (proxy->state =3D=3D VFIO_PROXY_CONNECTED) { + vfio_user_shutdown(proxy); + if (!QTAILQ_EMPTY(&proxy->pending)) { + error_printf("vfio_user_disconnect: outstanding requests\n"); + } + } + object_unref(OBJECT(proxy->ioc)); + proxy->ioc =3D NULL; + + proxy->state =3D VFIO_PROXY_CLOSING; + QTAILQ_FOREACH_SAFE(r1, &proxy->outgoing, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->outgoing, r1, next); + g_free(r1); + } + QTAILQ_FOREACH_SAFE(r1, &proxy->incoming, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->incoming, r1, next); + g_free(r1); + } + QTAILQ_FOREACH_SAFE(r1, &proxy->pending, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->pending, r1, next); + g_free(r1); + } + QTAILQ_FOREACH_SAFE(r1, &proxy->free, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->free, r1, next); + g_free(r1); + } + + /* + * Make sure the iothread isn't blocking anywhere + * with a ref to this proxy by waiting for a BH + * handler to run after the proxy fd handlers were + * deleted above. + */ + aio_bh_schedule_oneshot(proxy->ctx, vfio_user_cb, proxy); + qemu_cond_wait(&proxy->close_cv, &proxy->lock); + + /* we now hold the only ref to proxy */ + qemu_mutex_unlock(&proxy->lock); + qemu_cond_destroy(&proxy->close_cv); + qemu_mutex_destroy(&proxy->lock); + + QLIST_REMOVE(proxy, next); + if (QLIST_EMPTY(&vfio_user_sockets)) { + iothread_destroy(vfio_user_iothread); + vfio_user_iothread =3D NULL; + } + + g_free(proxy->sockname); + g_free(proxy); +} diff --git a/hw/vfio-user/meson.build b/hw/vfio-user/meson.build index f1fee70c85..695b341547 100644 --- a/hw/vfio-user/meson.build +++ b/hw/vfio-user/meson.build @@ -2,6 +2,7 @@ vfio_user_ss =3D ss.source_set() vfio_user_ss.add(files( 'container.c', 'pci.c', + 'proxy.c', )) =20 if get_option('vfio_user_client').enabled() --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255277; cv=pass; d=zohomail.com; s=zohoarc; b=AN5SA7hM56LxvBjFG0oym3kgwr93L//5S5gDB8YvnnWtI+rrAQetW5+vfb6zxGMGuFFSOl0QPyp5pwzB19eXYemsibA9t8vHNDI/XofnqCS85nt8GYqijl497OCRmWVdoOvHnDdlDykDSpQVSx0ka4qFDJcV+tHD6tn48h+aBJc= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255277; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=tIKtr/+mn1tAXBrtC42SicLWuIuviQ4ynt6E9Khvg9E=; b=mf3PKw1e+u9hYsHN6DyrCUqsTCPzarKfKBtJHnHKde05gjT4M8eDVCzdVqV1zr2h5siKlZgNYtRFxt5VHUmcjqrB6oZTXMT86VETZvACB87tk0nPv8nUaF0/y+MGLLYe5Ty1S4JfZETOZOOu3vvipRYaf2SXgWFpvZCNyCwMDiE= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255277776763.1002443733428; Fri, 6 Jun 2025 17:14:37 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9c-0002z6-58; Fri, 06 Jun 2025 20:11:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9Z-0002x4-Pf for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:17 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9W-0006Op-EV for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:17 -0400 Received: from pps.filterd (m0127842.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556J65f9008460; Fri, 6 Jun 2025 17:11:13 -0700 Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11on2108.outbound.protection.outlook.com [40.107.223.108]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 471g843x7d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:13 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:10 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=tIKtr/+mn1tAXBrtC42SicLWuIuviQ4ynt6E9Khvg 9E=; b=0KolF5O2IEW0fZhH4VRdkeuphXK4jN4XvdspLhlJvX61SzaueLxlClZR+ +goiJjJFsAZs/V2+0roeupCByM/h73bbfRm+s3PFO7Fhu1HLTTNbo0Q2ROtwzwCm FVaVTQt70huU/bF7AzfO9TvD4tmCE198NnusMFgEj/+Zq2Cfomd2+iBfto0sEmjm vIuoq7tYrk9nFtU59cnfuj7Xw0euPHO1Q3mP7NXVKKoRx9gOde+X2l7qw11uB35F 9jT8DnEsKQbaN7f4nP6M2/DaqTw83aKP25iS4+uUBm4U+mkb4RScm75HO+i9btcH 5K5ykoddG8iZRAZJphaaq7b27rhLg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Iy+CAOCcxl9Nh4NVNyCWmbKV/Tqek1Jh2eYwdt+KJrXnuRT1/TqF8D3aZ6XJRTkYp3gxZawGYkNw493hPZQaiN9klvuHCLbTakYqPlT8XPkH2I7Nbycoz3i0+3D/zz9cfhf+2UXjCV2cXrT+YMJqxT5KMOca+cnM0WV+kOrpELmx2Aut3YJjWnac4sJ9JJrvimoYzBMkJtanFf7cJdBkIvgI5xUXAMtZEVYMCEXMsz8YJM5D3T82LPdz9UxTYaFrqz2wijHkSTA++xEGtg7dYoK6TMVFRM3ZaeqifJS0LMP9FagRzN6Jh3Nh5AjxTbdIc6jDtpyy8pjrxiXq3/iTDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tIKtr/+mn1tAXBrtC42SicLWuIuviQ4ynt6E9Khvg9E=; b=dsdiMdD2G8UJTXRydVUdvv16UCW9c90wrZD1xFJe9k+QkLrsZdU3wVjfcNxkve9UDiFXLdKnYqL3WSTHxQViXJVprQPX2tL+eywIF/9cIo91p4dhifpF95VeLr/7kUEMjbZ1UaKpjuIkArYUMtS3X18lLXaQVyNTOXJ8/yIQ89TMJB+8wRJD4TGlwBFnCsRAe45hL6RUsIh0OEqvSFADV+pG4+vzeyqKmpMpLaSgRrjd7gC4l1ygDkF99bS0ZZzfFTtEZFp0VBlSjLg79+l4xGke1i480RgM584xfbpqIKTlulcBzUe4zBjzPYSJ47Mof04bokfhS7x1wXYDPD08Kg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tIKtr/+mn1tAXBrtC42SicLWuIuviQ4ynt6E9Khvg9E=; b=DK3Mjw+taSCaY6lFmVBPRQ1cZDYlrW/7TmMCktJQFk/UD/PbxUbo0oTgzXBuavM9AJhuKUYj2U0ugt77tawFDitzqsumFIuJFqIV6DaEVwVMBQ/RjDRr3srQz6Mexljavmm/iruqPuP9pUi4e24qLTmvaYompbnnEY2A0NFsMpdkx1wwCXT2LGyw8GS7HDhrcfo5ttz3KA+RuP56oOqHMJMTDnz4qH+qZOshWvMsL1AAA66+pBXmcyNWSRwcPl7BOdyMB9EJiWSjuAlRKacSyBq1Z3B8jibe1juYTV+oEfi4SNcqbLAiZ3B+rDzg2V+ZRKIGelTn0qCCxmIQBBaAaQ== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 08/23] vfio-user: implement message receive infrastructure Date: Fri, 6 Jun 2025 17:10:40 -0700 Message-ID: <20250607001056.335310-9-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: 530f2422-5645-4209-7af7-08dda557ce65 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?dWZ1TGNvV2dZSmF0SmhrVGNORllNdzZDS2xqZERTbVczY0ltZlZrTENwZGVX?= =?utf-8?B?Unp6ZE9XRmN5WHJrQ1hDMXFHZTFMaGRleW9GMi9BNCszMkI3MG5xbURyK2pT?= =?utf-8?B?dHJkRDBLcXNkSU5yZG5yZkE1dGUzM1ZvUjhiM0pKd3lFODlUNk1aV0RZQW16?= =?utf-8?B?blI3dnpMUVZEUUFxS1JtWEJseVF3MlJUbEZjUEVycGhGQWptYlhzRXF0eXlX?= =?utf-8?B?WlN2ZkkxWHU3NG9xRVlyTjgwR1M4S0lEQ1NLN0VNUndINXRHSUpqZzFDQnZr?= =?utf-8?B?RTliQm50Z21JMklCUHVCR0cyY2wwZVloa1BGbXNKM1R6Y3pESm03cVM5WGRD?= =?utf-8?B?VzhKRTV5VnBsajEzam5WQk16VTdpU3JsMFhLbUNRSXVXZHM0WFNDK0VMbG9K?= =?utf-8?B?KzVvYW56bDFONzROcTZDWmI4YzFZcDRnamlUK2JMM09lblBoYXpmMDdCYlFl?= =?utf-8?B?UHJuTVlsZWhkRjRyeUtIdVdMWUoydmk3NGJ6QldQdTNTcjJrWkZYZTdzVzkv?= =?utf-8?B?STVFTWlVT0F3RDU5Tm9lc3IxYys1VmJuYjBWczJkK1poNWV0eXFIaW1EQS9G?= =?utf-8?B?MHloTlZMNmFSbHlhQm5EbFNYbjJXM2hMQnF0TnhpbDlMd0owSUhxQnNiVmpw?= =?utf-8?B?RTNoMGJmaU1IYjVjUzlRKzhxa0RvNERHWDJFN2J0Z1RIWjd3ZjE4UmZJN2Q2?= =?utf-8?B?akRqb3ZFUkJDT0xBdEhDQXlvWGl5ZWxoVFJDVVkxM3J6TlVNMDBzSHdxcDdn?= =?utf-8?B?L1JUendndkxqSDhOeEEvOGxTcHRZeXM3c0ZLTGFiZFp2L0YyYzJhaG82ZHZC?= =?utf-8?B?OC9IMEx5ak9iaTNXNFlyMXZacnQweTNrWlJ1NXYzeVZkUTlpVWh0OGFWamVZ?= =?utf-8?B?OEFGbVVHSlZ0b1ZJSWpuaVpxVHRycXJtREJPSjVrdGQ5Z3Vsb0ZYZ2RyOW1G?= =?utf-8?B?NXpnNDdXVHV2Qzl6bDg4UnpvWjYwb0s0TFJBZ2tKZ3VzanExNmNiRVFNeXJQ?= =?utf-8?B?WWZNUVBWMEpidklCL3JoK05tdm45TTRaQmpDM3hzRXR4WlJEVURSMy9MR0Rs?= =?utf-8?B?WmxxNWRoT01YZVEzMXlrdVNRZjV5SXM3bjROb2xDWHlUUHoxTHNmMFlZSk0w?= =?utf-8?B?K3Mza3lPUWIzNURlZHVpVkNEZkdmZHZGMHh5Wi9KQklvZXZlNTVHR1lNdklZ?= =?utf-8?B?ejQrYXBpeHhrNmpWdlJDK212b1RKWWladlg1cmV5UTlsVjB6TEZZNE5FVUJK?= =?utf-8?B?OTFoRXZVbndndkduQ3dnZEZIZC9uWWkwRHdUbnUwTTdvYXZaNDVTZmJQRy9j?= =?utf-8?B?cWlWUkxmcmg3SUhqbCtRTnYvMXVNaG9yRkVnMHJLUEM4dlRqdlp1K0Vwait3?= =?utf-8?B?bytPbUFlNTF1Q1ErTXZScFpyV3lyaUIzcXQvY3d3R21vRndOMi9ycGcrU2x0?= =?utf-8?B?cDNrWHorSUQ3WjBDK3FJS2wyN2hnUHo3bjZoZ2oxZzNPeURTRStYeVJscU9q?= =?utf-8?B?RHg2bEpHYy85VmpYTEpNR2FIaTV0WkZ1QkJwcVppNm5xa1JqcU9rYVlGTHNH?= =?utf-8?B?TlhpUUVrbENhVGhFUnM0bW1Jd2toOWNMai9OSWVUbUpJL1E1SGlSa1JKY1Iw?= =?utf-8?B?cEpaVjZmUTVkV2hMUlpnKzloaW16TWdCTHBBWUk0cVY4WlNLN2RGQkk1UUdV?= =?utf-8?B?QURjSDB6TGwyY2NCdERZem1JTVhFTUFpdnhwbFA1OXRIeEVsTm56citGSllV?= =?utf-8?B?YW1oMnFZeXY1QVdPU2JUZS83eWU2R01XOE9ZN1dFS3pJYmNMVDNTbnZRK0I4?= =?utf-8?B?dDkyWkQ0WFBQaExDU1ZtUHVnU3Y4QzJOeHMxNkd1aXhqZ2s5UzVlbmo1Wnpq?= =?utf-8?B?SU1MOTQ2YU5Rc28vSHlpQkpBaDZ1YktmWFhua2gxYis2YkR2cXhqQktTZDBD?= =?utf-8?Q?zlpwg6XgNyM=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dGhGSlFlU1hYSU13WGRiV2tNajEwWkRXZkhackhIQXZrcndDTXc5TXV5OE0z?= =?utf-8?B?dlRXSEZvMlNzTFFDK0VPbGRBTlZFVW5QMTBIT3h1Q0Zqa3MxR2RIbitlNkRu?= =?utf-8?B?bmRWaUNCMU84ZURtMGVhZTNYT0ZqZmFRSlA3d1c1UjEwQS9Ec1p4dUVFZDI2?= =?utf-8?B?Zk02UWUxRVo5a2pKcmtEL1BZekFrUHVwYTVVY3hoc1VEbmRkNFR0QWFMcmhZ?= =?utf-8?B?SGdybzRDWEtEdlU3SGlaQ3JLTzJXR2VZYkZOdDBibzE2SE40VnBUd1hYQ0dJ?= =?utf-8?B?dUdKTitFSW11M2JSdWY4STBNajA1T2JQelQvLzBiSGo3Vk9DUEE0Vnc3UHJt?= =?utf-8?B?VlpoYlR2OSsrdUVCb3NCd2N2QWxYaENFb1JBd3REa0l1SktUajkyTFlNbG00?= =?utf-8?B?eXVVai9BWXFrSkcvYmZLZHVwQko1R1h0TXBlNllYVHovWURvRngzQm9FRHpP?= =?utf-8?B?czNJcWhDUVc0eVpwTy9sR05MdGZPTGZxU3hzMjRVdnZpSDFyOTR5TVNQQlg5?= =?utf-8?B?UW1GZkllNlRURHdPdU5lc1VLUXNRR2xGNUlKMCsxMGJYenFiWlFYbk9ab2JI?= =?utf-8?B?NFUwYmNvdVVMT3VuOXlDeEUvdkVXQnlsd0U5S2hoYVd3dWw5L212OUJjTURC?= =?utf-8?B?bDgzTFRWL2ZGV1ZMWU1hWXd3RjNYV2JwdUVONkpleHZ3YnFOcHo2UEVZTHdk?= =?utf-8?B?ZUlWQlZpaklQdTZmRjlIRUJvMXBMNWRSS3lOaXZwbXhDcnFyWWU1czVYRXdr?= =?utf-8?B?Sk0vVC9nRFoveXlyaWFzVXhDQTBkRWRDZm5tYzA1ckV1WU1xZ1g1MU5vcDhD?= =?utf-8?B?Tk01RlpXRnU1U2EyNmlmR3ZCb2lLdkJaUUliWEZuZ0hRNXdvMVhPSnNTRkhG?= =?utf-8?B?UDd5eG5jRk0xUUdPOGtzUlh0eGhNeUh6T280ZzlneDBTMEZqWWJid0wvR0E5?= =?utf-8?B?OW9UK2N5aEJ6N0crZ0tIQVBZaUZBbThwdEpXbWZhVDNZNXAyQzkzRVFoQVJU?= =?utf-8?B?VTVrdWJWVmVWME9HZVhQYi9UZExza0dHZW4zYnlvNDY4YTJBRzdpWkJzc3hV?= =?utf-8?B?QU4wRjlNbGFRUHF5a2JwS0NhbTIvWXhKZjdoL3BodTJ4Z1ZUUWNweUsrZFRX?= =?utf-8?B?QTFKckJ4VHFMT0Y3QnY0VVprNlR6c0lMMWhlZUVOQzZ2a0g4Y2Y4RWpCY0VR?= =?utf-8?B?NktsTmJIK1JFZGZVU3dlZjBNeWQ1blRmbWRXaS9Ob0JJbzk5R1pESGNBNWM5?= =?utf-8?B?TkY1L0hWRWhrU3B5UG43ZUQ3eUNPLzFMOEUvTjRneUFKYU1BcVRzZlVVSzNK?= =?utf-8?B?R1pVaEpZVVdTTXB6UnNXVGI4bDN0Q3J2S3RvMzRtRlRZRUxYZThWMFhYYkMr?= =?utf-8?B?SmxncS8yKzR5M2hLY2xSN1NsWk1oZEIvQ0UraXNnbkhoY0x1bUl0OUdveHVX?= =?utf-8?B?eHI3d1dpVGo0aldoZ1BJNnJHNkp3NVZEdjE3eTBsOTBYUFRQR0c3a3cxOFFt?= =?utf-8?B?QWNHUWVDQXJJQ3JBVEtBOWxOQXZKK01ORlNwcHVGTVMxUmRFb3V1M21DSzhC?= =?utf-8?B?Mkxkb014YXhmbEJ2dFN0UHl3RDk1Y294bXhzRXRFTDRJdnV2cnRqTDRCWURM?= =?utf-8?B?WlpUNGVqTWd0QnpkT2svS2hORzhnazN2ZnNydC9YdGtoT2lMeThIMjlxcHEx?= =?utf-8?B?SCs4dGh1TnVVU1c2bnFCUTNVdWxVeDdaVVFyWGdOdEFJTk5Sb3cxbUIrMGJr?= =?utf-8?B?MlA0MVA3V2RMM3FkTDJETFNQUTJPM1NjSlREM2Y4NXQ2dUZUQ1N0Z0U3bVBS?= =?utf-8?B?azcvSnNmalhVa1doMThiWHp2S1FJSnJzWFNkTzVsUU5FOU1uRkRNdWswMzBr?= =?utf-8?B?Q0JWdkZkWU16YVZzSzVsTVNSM1NCRjR1QzBkRGxRbmlFbUFtZlhnR2lldHlM?= =?utf-8?B?MXFRaVFER1hUWEk3ckJrbjdFVzQ4eFlWaVJFYkQ3UDBIREJJL1dzWXNadWpm?= =?utf-8?B?cStKZVhTbCtmczkydjRJWVRjY3VHY3h3aDdtRzYxMWJCMEFweDM3NmV4ZXJC?= =?utf-8?B?RS84OEdMaTd1KzNDYnFMc2d2WUJVTHB2VVlWdHJaUG5IKzc3aGRoYWVmSXRl?= =?utf-8?Q?OTQGKrf+odvEDq8yjKThoO+o8?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 530f2422-5645-4209-7af7-08dda557ce65 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:10.1562 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ypjxwYWQvFtEeExG3wPZD0VKNSn5b1P8n7rRPLmfvJ2S/mw4+DZdDrWQGhU1GUqwKABte8A75KkoMIseUapH4g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX10DK+soCtGx/ ClRf2Luem/ckI05dtFP9gPVpXXk4sDlL7VkeaYRT5ZSplqu2+Gn2NVdXQl/wonUugpvebxg3PV/ t+t9g/D+ikwPb3ciQZOA/uu6rrMGdm+86LLiSFbTLU4mwkHmMdxs8JGPKS+fR5Mp54HGsugRFsj LyGEiN06cV35Sfjz2Kd4Y/NXYYQu3NgCOtoRE0ZePAPULShI7RYW2hUEoYi/UvXzPBQKuE/yFew ItCM825A5DsFz4zDYbuuMhClUGzlh3S9/L4lOfbLMbS4obBJoRS/ND3s5l9kwdfwFLtm1m4iD0Q J0ZJbSpUk0QlGLH0spm+pEINWd7k9ltDpBprTb6CGL0iH4CxMcHGDa4H9z/tCIBloOvYmI8Ame8 E1h+0JbzEilNrzswgGChb/Leak6+blYLBrgo0hN79IUooysrVCjzWq7sGCJtKUX/y0sUe6GJ X-Proofpoint-GUID: cBWnFcLjvHADbPed9oRull-RBXRBHqMY X-Authority-Analysis: v=2.4 cv=TvXmhCXh c=1 sm=1 tr=0 ts=684383a1 cx=c_pps a=A3LX+wu56ikakVX7ADtWTg==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=vXCtUE0a-urccVGwgxAA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: cBWnFcLjvHADbPed9oRull-RBXRBHqMY X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255279841116600 Add the basic implementation for receiving vfio-user messages from the control socket. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- meson.build | 1 + hw/vfio-user/protocol.h | 56 ++++++ hw/vfio-user/proxy.h | 11 + hw/vfio-user/trace.h | 1 + hw/vfio-user/pci.c | 11 + hw/vfio-user/proxy.c | 409 ++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 6 + 7 files changed, 495 insertions(+) create mode 100644 hw/vfio-user/protocol.h create mode 100644 hw/vfio-user/trace.h create mode 100644 hw/vfio-user/trace-events diff --git a/meson.build b/meson.build index 967a10e80b..45aec2a4ad 100644 --- a/meson.build +++ b/meson.build @@ -3679,6 +3679,7 @@ if have_system 'hw/ufs', 'hw/usb', 'hw/vfio', + 'hw/vfio-user', 'hw/virtio', 'hw/vmapple', 'hw/watchdog', diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h new file mode 100644 index 0000000000..6300c681e7 --- /dev/null +++ b/hw/vfio-user/protocol.h @@ -0,0 +1,56 @@ +#ifndef VFIO_USER_PROTOCOL_H +#define VFIO_USER_PROTOCOL_H + +/* + * vfio protocol over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + * Each message has a standard header that describes the command + * being sent, which is almost always a VFIO ioctl(). + * + * The header may be followed by command-specific data, such as the + * region and offset info for read and write commands. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +typedef struct { + uint16_t id; + uint16_t command; + uint32_t size; + uint32_t flags; + uint32_t error_reply; +} VFIOUserHdr; + +/* VFIOUserHdr commands */ +enum vfio_user_command { + VFIO_USER_VERSION =3D 1, + VFIO_USER_DMA_MAP =3D 2, + VFIO_USER_DMA_UNMAP =3D 3, + VFIO_USER_DEVICE_GET_INFO =3D 4, + VFIO_USER_DEVICE_GET_REGION_INFO =3D 5, + VFIO_USER_DEVICE_GET_REGION_IO_FDS =3D 6, + VFIO_USER_DEVICE_GET_IRQ_INFO =3D 7, + VFIO_USER_DEVICE_SET_IRQS =3D 8, + VFIO_USER_REGION_READ =3D 9, + VFIO_USER_REGION_WRITE =3D 10, + VFIO_USER_DMA_READ =3D 11, + VFIO_USER_DMA_WRITE =3D 12, + VFIO_USER_DEVICE_RESET =3D 13, + VFIO_USER_DIRTY_PAGES =3D 14, + VFIO_USER_MAX, +}; + +/* VFIOUserHdr flags */ +#define VFIO_USER_REQUEST 0x0 +#define VFIO_USER_REPLY 0x1 +#define VFIO_USER_TYPE 0xF + +#define VFIO_USER_NO_REPLY 0x10 +#define VFIO_USER_ERROR 0x20 + +#endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 60fe3e0b6d..0105479d02 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -15,6 +15,9 @@ #include "io/channel.h" #include "io/channel-socket.h" =20 +#include "qemu/sockets.h" +#include "hw/vfio-user/protocol.h" + typedef struct { int send_fds; int recv_fds; @@ -31,6 +34,7 @@ enum msg_type { =20 typedef struct VFIOUserMsg { QTAILQ_ENTRY(VFIOUserMsg) next; + VFIOUserHdr *hdr; VFIOUserFDs *fds; uint32_t rsize; uint32_t id; @@ -70,13 +74,20 @@ typedef struct VFIOUserProxy { VFIOUserMsgQ incoming; VFIOUserMsgQ outgoing; VFIOUserMsg *last_nowait; + VFIOUserMsg *part_recv; + size_t recv_left; enum proxy_state state; } VFIOUserProxy; =20 /* VFIOProxy flags */ #define VFIO_PROXY_CLIENT 0x1 =20 +typedef struct VFIODevice VFIODevice; + VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp); void vfio_user_disconnect(VFIOUserProxy *proxy); +void vfio_user_set_handler(VFIODevice *vbasedev, + void (*handler)(void *opaque, VFIOUserMsg *msg), + void *reqarg); =20 #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/trace.h b/hw/vfio-user/trace.h new file mode 100644 index 0000000000..574b59aa89 --- /dev/null +++ b/hw/vfio-user/trace.h @@ -0,0 +1 @@ +#include "trace/trace-hw_vfio_user.h" diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index f1a2d4374d..93bd0e1361 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -24,6 +24,16 @@ struct VFIOUserPCIDevice { char *sock_name; }; =20 +/* + * Incoming request message callback. + * + * Runs off main loop, so BQL held. + */ +static void vfio_user_pci_process_req(void *opaque, VFIOUserMsg *msg) +{ + +} + /* * Emulated devices don't use host hot reset */ @@ -78,6 +88,7 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error = **errp) return; } vbasedev->proxy =3D proxy; + vfio_user_set_handler(vbasedev, vfio_user_pci_process_req, vdev); =20 vbasedev->name =3D g_strdup_printf("VFIO user <%s>", udev->sock_name); =20 diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index ac481553ba..4a654d7597 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -14,15 +14,32 @@ =20 #include "hw/vfio/vfio-device.h" #include "hw/vfio-user/proxy.h" +#include "hw/vfio-user/trace.h" #include "qapi/error.h" #include "qemu/error-report.h" #include "qemu/lockable.h" +#include "qemu/main-loop.h" #include "system/iothread.h" =20 static IOThread *vfio_user_iothread; =20 static void vfio_user_shutdown(VFIOUserProxy *proxy); +static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, + VFIOUserFDs *fds); +static VFIOUserFDs *vfio_user_getfds(int numfds); +static void vfio_user_recycle(VFIOUserProxy *proxy, VFIOUserMsg *msg); =20 +static void vfio_user_recv(void *opaque); +static int vfio_user_recv_one(VFIOUserProxy *proxy); +static void vfio_user_cb(void *opaque); + +static void vfio_user_request(void *opaque); + +static inline void vfio_user_set_error(VFIOUserHdr *hdr, uint32_t err) +{ + hdr->flags |=3D VFIO_USER_ERROR; + hdr->error_reply =3D err; +} =20 /* * Functions called by main, CPU, or iothread threads @@ -35,10 +52,340 @@ static void vfio_user_shutdown(VFIOUserProxy *proxy) proxy->ctx, NULL, NULL); } =20 +static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, + VFIOUserFDs *fds) +{ + VFIOUserMsg *msg; + + msg =3D QTAILQ_FIRST(&proxy->free); + if (msg !=3D NULL) { + QTAILQ_REMOVE(&proxy->free, msg, next); + } else { + msg =3D g_malloc0(sizeof(*msg)); + qemu_cond_init(&msg->cv); + } + + msg->hdr =3D hdr; + msg->fds =3D fds; + return msg; +} + +/* + * Recycle a message list entry to the free list. + */ +static void vfio_user_recycle(VFIOUserProxy *proxy, VFIOUserMsg *msg) +{ + if (msg->type =3D=3D VFIO_MSG_NONE) { + error_printf("vfio_user_recycle - freeing free msg\n"); + return; + } + + /* free msg buffer if no one is waiting to consume the reply */ + if (msg->type =3D=3D VFIO_MSG_NOWAIT || msg->type =3D=3D VFIO_MSG_ASYN= C) { + g_free(msg->hdr); + if (msg->fds !=3D NULL) { + g_free(msg->fds); + } + } + + msg->type =3D VFIO_MSG_NONE; + msg->hdr =3D NULL; + msg->fds =3D NULL; + msg->complete =3D false; + QTAILQ_INSERT_HEAD(&proxy->free, msg, next); +} + +static VFIOUserFDs *vfio_user_getfds(int numfds) +{ + VFIOUserFDs *fds =3D g_malloc0(sizeof(*fds) + (numfds * sizeof(int))); + + fds->fds =3D (int *)((char *)fds + sizeof(*fds)); + + return fds; +} + /* * Functions only called by iothread */ =20 +/* + * Process a received message. + */ +static void vfio_user_process(VFIOUserProxy *proxy, VFIOUserMsg *msg, + bool isreply) +{ + + /* + * Replies signal a waiter, if none just check for errors + * and free the message buffer. + * + * Requests get queued for the BH. + */ + if (isreply) { + msg->complete =3D true; + if (msg->type =3D=3D VFIO_MSG_WAIT) { + qemu_cond_signal(&msg->cv); + } else { + if (msg->hdr->flags & VFIO_USER_ERROR) { + error_printf("vfio_user_process: error reply on async "); + error_printf("request command %x error %s\n", + msg->hdr->command, + strerror(msg->hdr->error_reply)); + } + /* youngest nowait msg has been ack'd */ + if (proxy->last_nowait =3D=3D msg) { + proxy->last_nowait =3D NULL; + } + vfio_user_recycle(proxy, msg); + } + } else { + QTAILQ_INSERT_TAIL(&proxy->incoming, msg, next); + qemu_bh_schedule(proxy->req_bh); + } +} + +/* + * Complete a partial message read + */ +static int vfio_user_complete(VFIOUserProxy *proxy, Error **errp) +{ + VFIOUserMsg *msg =3D proxy->part_recv; + size_t msgleft =3D proxy->recv_left; + bool isreply; + char *data; + int ret; + + data =3D (char *)msg->hdr + (msg->hdr->size - msgleft); + while (msgleft > 0) { + ret =3D qio_channel_read(proxy->ioc, data, msgleft, errp); + + /* error or would block */ + if (ret <=3D 0) { + /* try for rest on next iternation */ + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + proxy->recv_left =3D msgleft; + } + return ret; + } + trace_vfio_user_recv_read(msg->hdr->id, ret); + + msgleft -=3D ret; + data +=3D ret; + } + + /* + * Read complete message, process it. + */ + proxy->part_recv =3D NULL; + proxy->recv_left =3D 0; + isreply =3D (msg->hdr->flags & VFIO_USER_TYPE) =3D=3D VFIO_USER_REPLY; + vfio_user_process(proxy, msg, isreply); + + /* return positive value */ + return 1; +} + +static void vfio_user_recv(void *opaque) +{ + VFIOUserProxy *proxy =3D opaque; + + QEMU_LOCK_GUARD(&proxy->lock); + + if (proxy->state =3D=3D VFIO_PROXY_CONNECTED) { + while (vfio_user_recv_one(proxy) =3D=3D 0) { + ; + } + } +} + +/* + * Receive and process one incoming message. + * + * For replies, find matching outgoing request and wake any waiters. + * For requests, queue in incoming list and run request BH. + */ +static int vfio_user_recv_one(VFIOUserProxy *proxy) +{ + VFIOUserMsg *msg =3D NULL; + g_autofree int *fdp =3D NULL; + VFIOUserFDs *reqfds; + VFIOUserHdr hdr; + struct iovec iov =3D { + .iov_base =3D &hdr, + .iov_len =3D sizeof(hdr), + }; + bool isreply =3D false; + int i, ret; + size_t msgleft, numfds =3D 0; + char *data =3D NULL; + char *buf =3D NULL; + Error *local_err =3D NULL; + + /* + * Complete any partial reads + */ + if (proxy->part_recv !=3D NULL) { + ret =3D vfio_user_complete(proxy, &local_err); + + /* still not complete, try later */ + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + return ret; + } + + if (ret <=3D 0) { + goto fatal; + } + /* else fall into reading another msg */ + } + + /* + * Read header + */ + ret =3D qio_channel_readv_full(proxy->ioc, &iov, 1, &fdp, &numfds, 0, + &local_err); + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + return ret; + } + + /* read error or other side closed connection */ + if (ret <=3D 0) { + goto fatal; + } + + if (ret < sizeof(hdr)) { + error_setg(&local_err, "short read of header"); + goto fatal; + } + + /* + * Validate header + */ + if (hdr.size < sizeof(VFIOUserHdr)) { + error_setg(&local_err, "bad header size"); + goto fatal; + } + switch (hdr.flags & VFIO_USER_TYPE) { + case VFIO_USER_REQUEST: + isreply =3D false; + break; + case VFIO_USER_REPLY: + isreply =3D true; + break; + default: + error_setg(&local_err, "unknown message type"); + goto fatal; + } + trace_vfio_user_recv_hdr(proxy->sockname, hdr.id, hdr.command, hdr.siz= e, + hdr.flags); + + /* + * For replies, find the matching pending request. + * For requests, reap incoming FDs. + */ + if (isreply) { + QTAILQ_FOREACH(msg, &proxy->pending, next) { + if (hdr.id =3D=3D msg->id) { + break; + } + } + if (msg =3D=3D NULL) { + error_setg(&local_err, "unexpected reply"); + goto err; + } + QTAILQ_REMOVE(&proxy->pending, msg, next); + + /* + * Process any received FDs + */ + if (numfds !=3D 0) { + if (msg->fds =3D=3D NULL || msg->fds->recv_fds < numfds) { + error_setg(&local_err, "unexpected FDs"); + goto err; + } + msg->fds->recv_fds =3D numfds; + memcpy(msg->fds->fds, fdp, numfds * sizeof(int)); + } + } else { + if (numfds !=3D 0) { + reqfds =3D vfio_user_getfds(numfds); + memcpy(reqfds->fds, fdp, numfds * sizeof(int)); + } else { + reqfds =3D NULL; + } + } + + /* + * Put the whole message into a single buffer. + */ + if (isreply) { + if (hdr.size > msg->rsize) { + error_setg(&local_err, "reply larger than recv buffer"); + goto err; + } + *msg->hdr =3D hdr; + data =3D (char *)msg->hdr + sizeof(hdr); + } else { + buf =3D g_malloc0(hdr.size); + memcpy(buf, &hdr, sizeof(hdr)); + data =3D buf + sizeof(hdr); + msg =3D vfio_user_getmsg(proxy, (VFIOUserHdr *)buf, reqfds); + msg->type =3D VFIO_MSG_REQ; + } + + /* + * Read rest of message. + */ + msgleft =3D hdr.size - sizeof(hdr); + while (msgleft > 0) { + ret =3D qio_channel_read(proxy->ioc, data, msgleft, &local_err); + + /* prepare to complete read on next iternation */ + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + proxy->part_recv =3D msg; + proxy->recv_left =3D msgleft; + return ret; + } + + if (ret <=3D 0) { + goto fatal; + } + trace_vfio_user_recv_read(hdr.id, ret); + + msgleft -=3D ret; + data +=3D ret; + } + + vfio_user_process(proxy, msg, isreply); + return 0; + + /* + * fatal means the other side closed or we don't trust the stream + * err means this message is corrupt + */ +fatal: + vfio_user_shutdown(proxy); + proxy->state =3D VFIO_PROXY_ERROR; + + /* set error if server side closed */ + if (ret =3D=3D 0) { + error_setg(&local_err, "server closed socket"); + } + +err: + for (i =3D 0; i < numfds; i++) { + close(fdp[i]); + } + if (isreply && msg !=3D NULL) { + /* force an error to keep sending thread from hanging */ + vfio_user_set_error(msg->hdr, EINVAL); + msg->complete =3D true; + qemu_cond_signal(&msg->cv); + } + error_prepend(&local_err, "vfio_user_recv_one: "); + error_report_err(local_err); + return -1; +} + static void vfio_user_cb(void *opaque) { VFIOUserProxy *proxy =3D opaque; @@ -54,6 +401,53 @@ static void vfio_user_cb(void *opaque) * Functions called by main or CPU threads */ =20 +/* + * Process incoming requests. + * + * The bus-specific callback has the form: + * request(opaque, msg) + * where 'opaque' was specified in vfio_user_set_handler + * and 'msg' is the inbound message. + * + * The callback is responsible for disposing of the message buffer, + * usually by re-using it when calling vfio_send_reply or vfio_send_error, + * both of which free their message buffer when the reply is sent. + * + * If the callback uses a new buffer, it needs to free the old one. + */ +static void vfio_user_request(void *opaque) +{ + VFIOUserProxy *proxy =3D opaque; + VFIOUserMsgQ new, free; + VFIOUserMsg *msg, *m1; + + /* reap all incoming */ + QTAILQ_INIT(&new); + WITH_QEMU_LOCK_GUARD(&proxy->lock) { + QTAILQ_FOREACH_SAFE(msg, &proxy->incoming, next, m1) { + QTAILQ_REMOVE(&proxy->incoming, msg, next); + QTAILQ_INSERT_TAIL(&new, msg, next); + } + } + + /* process list */ + QTAILQ_INIT(&free); + QTAILQ_FOREACH_SAFE(msg, &new, next, m1) { + QTAILQ_REMOVE(&new, msg, next); + trace_vfio_user_recv_request(msg->hdr->command); + proxy->request(proxy->req_arg, msg); + QTAILQ_INSERT_HEAD(&free, msg, next); + } + + /* free list */ + WITH_QEMU_LOCK_GUARD(&proxy->lock) { + QTAILQ_FOREACH_SAFE(msg, &free, next, m1) { + vfio_user_recycle(proxy, msg); + } + } +} + + static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); =20 @@ -92,6 +486,7 @@ VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr= , Error **errp) } =20 proxy->ctx =3D iothread_get_aio_context(vfio_user_iothread); + proxy->req_bh =3D qemu_bh_new(vfio_user_request, proxy); =20 QTAILQ_INIT(&proxy->outgoing); QTAILQ_INIT(&proxy->incoming); @@ -102,6 +497,18 @@ VFIOUserProxy *vfio_user_connect_dev(SocketAddress *ad= dr, Error **errp) return proxy; } =20 +void vfio_user_set_handler(VFIODevice *vbasedev, + void (*handler)(void *opaque, VFIOUserMsg *msg), + void *req_arg) +{ + VFIOUserProxy *proxy =3D vbasedev->proxy; + + proxy->request =3D handler; + proxy->req_arg =3D req_arg; + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, + vfio_user_recv, NULL, NULL, proxy); +} + void vfio_user_disconnect(VFIOUserProxy *proxy) { VFIOUserMsg *r1, *r2; @@ -117,6 +524,8 @@ void vfio_user_disconnect(VFIOUserProxy *proxy) } object_unref(OBJECT(proxy->ioc)); proxy->ioc =3D NULL; + qemu_bh_delete(proxy->req_bh); + proxy->req_bh =3D NULL; =20 proxy->state =3D VFIO_PROXY_CLOSING; QTAILQ_FOREACH_SAFE(r1, &proxy->outgoing, next, r2) { diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events new file mode 100644 index 0000000000..89d6c11c4c --- /dev/null +++ b/hw/vfio-user/trace-events @@ -0,0 +1,6 @@ +# See docs/devel/tracing.rst for syntax documentation. + +# common.c +vfio_user_recv_hdr(const char *name, uint16_t id, uint16_t cmd, uint32_t s= ize, uint32_t flags) " (%s) id 0x%x cmd 0x%x size 0x%x flags 0x%x" +vfio_user_recv_read(uint16_t id, int read) " id 0x%x read 0x%x" +vfio_user_recv_request(uint16_t cmd) " command 0x%x" --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255125; cv=pass; d=zohomail.com; s=zohoarc; b=j6rfDHV1HnFeVw9AR+iDlRNIGsXavQXIOctvqU6HgTZDV+t1dl4a/V4vp8V3W5Ru2c9uRdpiT0x2e3f1A11zhcs8CK1ZS1ucivksyaZ3OKKCsIe3yiHWDBAlQjrlwRCioVNOc2yFlLw9BuGTF2W4mIgN/rJw6TIl0r0XxyRqScM= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255125; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Lg0f18ItSZy8fyxg8r6nI4gWEiG7RAQGDVLx9207Jfc=; b=GJfmoRb/HuwXjONEwzYhfOWMg9/MlEKGs2zX04JYXpM2iA5YLlMhE2WD/UAYajBUPr3q/phpqD/bhwWeUYUsnHszKzlrSPMHJoctLi9N7qaMHdyoVukVBnbOmz2tXq/Fs+ZG8WwkRYAVbPVWz49GIZpMMzKQcuMlZKhCL5S5f4A= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255125147507.8520843365709; Fri, 6 Jun 2025 17:12:05 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9f-00030V-01; Fri, 06 Jun 2025 20:11:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9c-0002zB-Ck for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:20 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9Z-0006PS-8i for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:20 -0400 Received: from pps.filterd (m0127842.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556J65fB008460; Fri, 6 Jun 2025 17:11:14 -0700 Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11on2108.outbound.protection.outlook.com [40.107.223.108]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 471g843x7d-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:13 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:11 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=Lg0f18ItSZy8fyxg8r6nI4gWEiG7RAQGDVLx9207J fc=; b=kJt2+oz9DlOrvAbF3Ml8F6e+pxjaxQu9Wzjk9M8PO2j6FI6IbgSYzYDqU mcQ694rehQ/EclmgfEmNHqcHFSZ3FD1S5hwWldTvvyBppfVIZ/9WJverawfqTn5n b85REiZgLLEQ3M1Ss6yYFXHwJEnh5cCpA2RKgVkL4nRCalpoCuvYg2R/O9f/m8C9 gJzjTq1VHZWCqd8W0mH8ZEu7sB3VMEf6yei58KC2CKvnW0jjAw4D5QD25CvpxGsY jYoCl2ZJVfGiqoHkT6MjcgZzwmnFSeeJAc3tIl2Fy8X6NEixmNYixN6XpWVDtqmh lNIJTKOYNFnYfrPLLbWQz10hY56Cw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jjHCF3MX77EIPs1B69FvxJpFB+wlklWeX7w+e2Th3kSgg4gVx4X7NtW4sv5E75J8zFFGACNlBazGBwbmvuMIXOrBS7hQYqb5F9evXVGFodjpon/wgFy+H7XRDe/pvV9fk/fW2t5wbqPvb5FdQYOS43h0TE6E9NCW6Hz0VYoDxjNQbp0d7QVHg5O+FboWkFV4SExJkd7+FMKZRKdYJBM3FEjNxJqTohIpsDvTfc62XjMTwji1dMgZ6SOtDu6IJtRyTBPUQ6mPfLb6UzqQ/nM0tnzwLJmgdEqqi4T3HuLU8ecCOZ14C9NFtndp8FtUoVG4NtKcNsHu8SKLeu9I9sZw4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Lg0f18ItSZy8fyxg8r6nI4gWEiG7RAQGDVLx9207Jfc=; b=rBr/9P1dni/o0cNrALFHHWHhRgTm5RE7/NcGjBe9Zd1r6HsY+9Zl0TELjcCMXVmw31c92KIMhDoXkD4wmb0vFGfWz5BRg1NgrxNCaPHIJ98emWtIwfM+Ko10SZTf4Nc6wvVI0j0l0BD8L2gf5EsZ6DDOvTnNnEyC1vvDnJX+FLK1PuqJVU7RQDuOPVifKibEdCtxKxO5QdiCQWdZxWfjPkW+NfXUOJ3BjYvyqocjVgF/lCdl7NwFRugDWwwIAvwAVUa1MKc0PJzSjOqM/t/oCqc5bPoxjU+gMHWpZafi03j34uNrCiO1uGyjsD0krdf9h9wk5k4SxuhpaGwXLsO2fg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Lg0f18ItSZy8fyxg8r6nI4gWEiG7RAQGDVLx9207Jfc=; b=ffiya0ENxG8alLx1SsMDPZG8viFOqYM0+u4oXqtUZQ8Q9YtzGPCyGUkns4rvN80eZJzX3u3Js2ZHbPsRBm4hiZ9UQGsScYwRVp8Rta/iUNUie1j9Hn2IBDhsfV3elbXK/9DguT1k0kanGOrPovZv2ApSSeC0Wec5h4eofhuoHTwZJTNxtoWTr7dzu47wz/+6J1HHT2s84nxJ+iNPPtjidsh9JF0U18FRWuy0/yljCnar8wimhhMKGe+7wVJEcpkYP6z49gBzDJIWHyrwK3IpfAhKhKPbrLSk23kh3xtxyLiUv1/rorTTkWKFFKGEJyU4dtvpf0Ca0cAmkQszplmFDA== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Jagannathan Raman , Elena Ufimtseva Subject: [PATCH v3 09/23] vfio-user: implement message send infrastructure Date: Fri, 6 Jun 2025 17:10:41 -0700 Message-ID: <20250607001056.335310-10-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: c975a5cd-5d4b-4171-7733-08dda557cf4a x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?yTmbydjMZXAxpIy/F9nqsQRufi/s3KZ7OU21n+jr476dhGbaw5nhhVbvcbQq?= =?us-ascii?Q?usqLAoESIjb4GXdcW8pmU0oA8LWRlIgAmNY2u5wF6Lm5nf5tL7b21VIvFDGn?= =?us-ascii?Q?XAi3IVkNlz6pwuTXhXgufoYRPSmyinpJp3MYZrfDrxuP1H/m4IcFvUXRDaX9?= =?us-ascii?Q?7rTUx0EpS2AR4E74c7IFnpD5TbGqEfOplTcBv0rxFR6crfPxolE8ZqPbTGak?= =?us-ascii?Q?ez844Ik4ZDkXWaCIU9Hj3tUGmwJ6co4wSF7/Ln2cTllVDFgQCmWQx8KDcw9i?= =?us-ascii?Q?smJeRNBL8jWG3OConkeNwx2bqXuYfCIMLlzWyDo959MZd5Wnw07eDENbTJRp?= =?us-ascii?Q?V82t20ZchIrRpGYFwgd81xQg4/GvXE7kOiUaJBKBu7vadlw9sgAKUhqPFubk?= =?us-ascii?Q?54x3tFHDsWb6o/UKbbz7LfY4anrmFVAG/AdHZLhKsLu4k7LC/b8IKU+u2ubg?= =?us-ascii?Q?WsWgrtveKCdfdXivHW5Bg3XrTok27W0OJgjk5aeqitG2Kmpb3vmyDHTCTKGW?= =?us-ascii?Q?VjdrA/gyYUKWzakODWjCWWHDsVoBL3tNBYv3V/kl9sY2DgnlGbHJ6EBMGCGI?= =?us-ascii?Q?DE7gFtscgwyHf1r1JUhvSdU+NzDjL2YNidoncRbCZDyMtPkzBOjpLoZ0ZcBg?= =?us-ascii?Q?b7cZq7MQFjgoln0y9a67kGmjk5Ckf1cVsFD6jNipziK/20GAAL9x1Z2Dg12p?= =?us-ascii?Q?z8878+WyxisuvKuchZHOLRy83N4gwr2a8WvKIv8bJI3VLBREJs4YRr43PVl2?= =?us-ascii?Q?+Fn842rc7G0VpU9jAjiZeYaDCFJ5AfA0WuXtqfxlwn4ExRMOy+wGOgr4GlGP?= =?us-ascii?Q?KGDf49rWCT4j4Mga4tED1Lvs1bEYx4z4pAJN7kmnVU8jklCu0bWIK2Z9beEq?= =?us-ascii?Q?6VJmKOsUea58v6HeKK18KGAImbRTjKNVFPHqObPT/38Wmkhl2yKFbPcrXg/L?= =?us-ascii?Q?FhcdTjqekXp3XVmkWlP91ffF+cQX1ITEechLZZ5k75bqSM2PN3AdE2/NeoeM?= =?us-ascii?Q?L+E33CRDy1KpNtWeVsMgaPlews8Zi30Vi3CevV++8cfLOPAgrIfB9Kf+VGmE?= =?us-ascii?Q?MDnUQmFFkzfPctZ5gkA/iQCIxp7IdiNyKXnwY8Kf44ioxOihP4OGAg8kBqEA?= =?us-ascii?Q?naU6MGp+cBJ2X/RFxBveUsONqqmXWTc3vhFk4CDK+qXm7C/y6ZMwCPkQ1/WE?= =?us-ascii?Q?GKA9YVW/ao/1IZznDpXXDHDl9Xr1NNfo+rvKHDmI9zkbCuUCUZS+zGBuj2fJ?= =?us-ascii?Q?JduSaxmEChZx0dUF9MqGPcNow2hEt7nxhvNhBnyrvUeAu5a/GTUhDqQVWCTg?= =?us-ascii?Q?dTW7ryeyh9nykgZLSO3O9hbMGzLzcnf2LhKnrNCP/G6qpgJ34jeeUA5l/c70?= =?us-ascii?Q?E2SEIZNN5Tf7Dz1ArV9hDwCNolTmKACXhU1toyE1bep5TS49bZgzoQvsFbIe?= =?us-ascii?Q?31u6y6V8nHI=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?tnIzRGuZuHKVUs/ul1SOyy80a87K7qHNTlDTeyGtEBYI9YxIVbSg2Dpzh8ll?= =?us-ascii?Q?SDj1gXkyAT+0lXurge3VQn2Hsw7X/6F0nP6QBqCOJ9/5JyGRc5VT83oM1ZgR?= =?us-ascii?Q?1rtBujZ3OxmNklLIJc6JB6tHbcw2UmW1mzOoFmgcmmiya9x9TLwybkJuK98I?= =?us-ascii?Q?5ycFX2SLTLlzX36E0j7LCeWbkGrwbq4Si/mrSbP0ymrPQFacbbnhTS6imsU8?= =?us-ascii?Q?6zkCJZFsEljoWX7LkBygtm3e043bVysZjOEfuLZtC4dpPRnI+18NmaZqEGHz?= =?us-ascii?Q?js6y6AeCwTPzljiYg+3lAvcuT7u04vrxryDyccc7oOXEtkNt5piER2YLOUJC?= =?us-ascii?Q?oaNRlAk8x2jGJqsIublhNw1aeX95EHoh3oB8aT0K01kxDbN7ZfQKUQfFcAfw?= =?us-ascii?Q?kYmcyEZF+h+hQjRzCzju76oB/nS54d0oYsEB6UZ8bWD1IDwidwT9nGm1nbSb?= =?us-ascii?Q?eumekECqk7rZLwyYVFlfQsgvZdHgEfF+Er+og/xIJLBKPCLk91q1wGWjeTc2?= =?us-ascii?Q?VAW9Us8wNLFeOA1G4ORf/W7bPwrFlGeu5pVnNxbskIUI0m38PcOSvjP9H6/I?= =?us-ascii?Q?vFazfOrECwk7xwUO18HojOF8akyMSM74gVe6H2ZTVyOIIbdumc7nV062WSbL?= =?us-ascii?Q?TtJdVBiTwVsdNJD4zEEKBjpvfCE0Cd9vCoz5TxLP6A0OYdnSmK7znZqEl3jh?= =?us-ascii?Q?Fd2zwkxogjORitNja1jBccYgqVQKMDBM9M9M5W19zZGm/VNoP6VrvZV87Tn1?= =?us-ascii?Q?mKNmgxju/aAvOh8hjCkNEEzkpu2GI+8Rjlilkc1zje2U/gVJ+W7RzEPIVT0x?= =?us-ascii?Q?bJGuQtku5yRUW2XqlIM1tHBn0hMN8Fx/D+4pQRe8Qp9NsCw8VPEY1/SKCxc7?= =?us-ascii?Q?8SNojiygKF8qpfiSiPeDuTa/vwMA7YgbwT9ff+KyE9hogZjoglemzyPBQt5E?= =?us-ascii?Q?lZ81dDzL4MqY87WN6OzlnOjgHOkSv5OlJJbi7JpmYjZSh29i6F57kOp4TmEo?= =?us-ascii?Q?HnrYkc1g8T/XEkw15GdsBT3a/QNw2OV6DozyvTvagxrc4Es9D6c/+/WlJxxO?= =?us-ascii?Q?UL7HhLJAojbYN7T1BbWek7RYQzcWgeZnv51OQAaKt/mLEEdvN9Tkz5cG2lby?= =?us-ascii?Q?ijKUAaQHysy1nh8V+RFRio3Fw9MWo6HhRxuffF2vPc9Io5GXGeU06bLiNBlA?= =?us-ascii?Q?0iu0Lb8OuvbGeGhEinGz52Yiw/VOu1M/8rKcNHt/bb7UjF0CcT1KAHYcwDpp?= =?us-ascii?Q?d+zXFSuXCzN6gz9dirr7BqwplUP/XNKUAhgY6vPqIgSrlKHC9EKptucumSKQ?= =?us-ascii?Q?z5+8X+RoiCetsdBMClPE9UtGmjVjDztLPo28bGteH3w3nHsBERm7OywylAdE?= =?us-ascii?Q?Axev1a1milIUeJjd2JGsmthGwGhb7c0KA2LAuqGwgrB7tOe3O5HwjrvoSidW?= =?us-ascii?Q?TpsDJJyUy64QEcgplk6pCushaplyH7HGoXebQni3KBRx6CR5JphvXMILGdrf?= =?us-ascii?Q?4jl7YADzlWkGWyrUew5+WjNyFsUzSvk9BQbaSh5XEJb+RFiztf86SoFjn0Tp?= =?us-ascii?Q?QWYgBjiISRMmn8kbcdqQWJ+xwRsc1esB6uLA3xm6?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: c975a5cd-5d4b-4171-7733-08dda557cf4a X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:11.7290 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: doE0L/GiLY0HkXHUY0gDiBsaHVS3+8kF+bW15s1oYiuQT+pljTJ2h5M+pUYgY+bOyxWiWxUgR1vB2/txYv2QCg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfXy4JhsfRpY9g1 f5/ujXh9iMRZMkJjTYpJX8K1olJHhPC6sBZLIXnowqRayibziP361K1o4bAelSMR3HcHTbqk5m7 srDJ3cA7K0NO5V5/CkycU6DGlxUZMQH9TmZpu47rRmFfjUl3n0HpJPhGTkxyUNLskH39vC5Fw4v 9p//Tbcbg+3Idj0E5Hq8vWwDC0tLl/vXN3qtMw3nySrSdukMvfDJIGXnsUX+JTBein6Pim/Gxe5 xf+TkpDEMBqGJpQ4n8ARVsuLLp9B22y0RevDq+Kp0GHf0/KYiGDE9M3Yd2xTKpwEkmbXHDv6d+X GvWHyqTPsdKDkIn/1gfwG9FuJqgJjGd85rwEr4tq6PS9q0c1zEejUvm916Dir1rGA0j81liPmjQ 9ifsqAo547hW0s/3AmVJE5+WM3XxUUlF0xjhgMMVHlgkCPIwnzRC8+PpJdxL/80jrZSwFDQX X-Proofpoint-GUID: HoeGrYWw6qmC7GSiNGR8JhabBQW7Ij9F X-Authority-Analysis: v=2.4 cv=TvXmhCXh c=1 sm=1 tr=0 ts=684383a1 cx=c_pps a=A3LX+wu56ikakVX7ADtWTg==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=UnscCpd6-9sbi9RN3AUA:9 X-Proofpoint-ORIG-GUID: HoeGrYWw6qmC7GSiNGR8JhabBQW7Ij9F X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255126340116600 Content-Type: text/plain; charset="utf-8" Add plumbing for sending vfio-user messages on the control socket. Add initial version negotation on connection. Originally-by: John Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John Levon --- hw/vfio-user/protocol.h | 62 +++++ hw/vfio-user/proxy.h | 9 + hw/vfio-user/pci.c | 18 +- hw/vfio-user/proxy.c | 494 ++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 2 + 5 files changed, 583 insertions(+), 2 deletions(-) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 6300c681e7..74ea2a94e6 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -53,4 +53,66 @@ enum vfio_user_command { #define VFIO_USER_NO_REPLY 0x10 #define VFIO_USER_ERROR 0x20 =20 + +/* + * VFIO_USER_VERSION + */ +typedef struct { + VFIOUserHdr hdr; + uint16_t major; + uint16_t minor; + char capabilities[]; +} VFIOUserVersion; + +#define VFIO_USER_MAJOR_VER 0 +#define VFIO_USER_MINOR_VER 0 + +#define VFIO_USER_CAP "capabilities" + +/* "capabilities" members */ +#define VFIO_USER_CAP_MAX_FDS "max_msg_fds" +#define VFIO_USER_CAP_MAX_XFER "max_data_xfer_size" +#define VFIO_USER_CAP_PGSIZES "pgsizes" +#define VFIO_USER_CAP_MAP_MAX "max_dma_maps" +#define VFIO_USER_CAP_MIGR "migration" + +/* "migration" members */ +#define VFIO_USER_CAP_PGSIZE "pgsize" +#define VFIO_USER_CAP_MAX_BITMAP "max_bitmap_size" + +/* + * Max FDs mainly comes into play when a device supports multiple interrup= ts + * where each ones uses an eventfd to inject it into the guest. + * It is clamped by the the number of FDs the qio channel supports in a + * single message. + */ +#define VFIO_USER_DEF_MAX_FDS 8 +#define VFIO_USER_MAX_MAX_FDS 16 + +/* + * Max transfer limits the amount of data in region and DMA messages. + * Region R/W will be very small (limited by how much a single instruction + * can process) so just use a reasonable limit here. + */ +#define VFIO_USER_DEF_MAX_XFER (1024 * 1024) +#define VFIO_USER_MAX_MAX_XFER (64 * 1024 * 1024) + +/* + * Default pagesizes supported is 4k. + */ +#define VFIO_USER_DEF_PGSIZE 4096 + +/* + * Default max number of DMA mappings is stolen from the + * linux kernel "dma_entry_limit" + */ +#define VFIO_USER_DEF_MAP_MAX 65535 + +/* + * Default max bitmap size is also take from the linux kernel, + * where usage of signed ints limits the VA range to 2^31 bytes. + * Dividing that by the number of bits per byte yields 256MB + */ +#define VFIO_USER_DEF_MAX_BITMAP (256 * 1024 * 1024) + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 0105479d02..33e8a4d83b 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -40,6 +40,7 @@ typedef struct VFIOUserMsg { uint32_t id; QemuCond cv; bool complete; + bool pending; enum msg_type type; } VFIOUserMsg; =20 @@ -59,6 +60,12 @@ typedef struct VFIOUserProxy { struct QIOChannel *ioc; void (*request)(void *opaque, VFIOUserMsg *msg); void *req_arg; + uint64_t max_xfer_size; + uint64_t max_send_fds; + uint64_t max_dma; + uint64_t dma_pgsizes; + uint64_t max_bitmap; + uint64_t migr_pgsize; int flags; QemuCond close_cv; AioContext *ctx; @@ -81,6 +88,7 @@ typedef struct VFIOUserProxy { =20 /* VFIOProxy flags */ #define VFIO_PROXY_CLIENT 0x1 +#define VFIO_PROXY_FORCE_QUEUED 0x4 =20 typedef struct VFIODevice VFIODevice; =20 @@ -89,5 +97,6 @@ void vfio_user_disconnect(VFIOUserProxy *proxy); void vfio_user_set_handler(VFIODevice *vbasedev, void (*handler)(void *opaque, VFIOUserMsg *msg), void *reqarg); +bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp); =20 #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 93bd0e1361..bcef0ea341 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -22,6 +22,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserPCIDevice, VFIO_USER_P= CI) struct VFIOUserPCIDevice { VFIOPCIDevice device; char *sock_name; + bool send_queued; /* all sends are queued */ }; =20 /* @@ -90,6 +91,14 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error= **errp) vbasedev->proxy =3D proxy; vfio_user_set_handler(vbasedev, vfio_user_pci_process_req, vdev); =20 + if (udev->send_queued) { + proxy->flags |=3D VFIO_PROXY_FORCE_QUEUED; + } + + if (!vfio_user_validate_version(proxy, errp)) { + goto error; + } + vbasedev->name =3D g_strdup_printf("VFIO user <%s>", udev->sock_name); =20 /* @@ -101,9 +110,13 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Err= or **errp) if (!vfio_device_attach_by_iommu_type(TYPE_VFIO_IOMMU_USER, vbasedev->name, vbasedev, as, errp)) { - error_prepend(errp, VFIO_MSG_PREFIX, vbasedev->name); - return; + goto error; } + + return; + +error: + error_prepend(errp, VFIO_MSG_PREFIX, vdev->vbasedev.name); } =20 static void vfio_user_instance_init(Object *obj) @@ -154,6 +167,7 @@ static const Property vfio_user_pci_dev_properties[] = =3D { DEFINE_PROP_UINT32("x-pci-sub-device-id", VFIOPCIDevice, sub_device_id, PCI_ANY_ID), DEFINE_PROP_STRING("socket", VFIOUserPCIDevice, sock_name), + DEFINE_PROP_BOOL("x-send-queued", VFIOUserPCIDevice, send_queued, fals= e), }; =20 static void vfio_user_pci_dev_class_init(ObjectClass *klass, const void *d= ata) diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 4a654d7597..0bc12da865 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -16,14 +16,19 @@ #include "hw/vfio-user/proxy.h" #include "hw/vfio-user/trace.h" #include "qapi/error.h" +#include "qobject/qdict.h" +#include "qobject/qjson.h" +#include "qobject/qnum.h" #include "qemu/error-report.h" #include "qemu/lockable.h" #include "qemu/main-loop.h" #include "system/iothread.h" =20 +static int wait_time =3D 5000; /* wait up to 5 sec for busy servers */ static IOThread *vfio_user_iothread; =20 static void vfio_user_shutdown(VFIOUserProxy *proxy); +static int vfio_user_send_qio(VFIOUserProxy *proxy, VFIOUserMsg *msg); static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, VFIOUserFDs *fds); static VFIOUserFDs *vfio_user_getfds(int numfds); @@ -31,9 +36,16 @@ static void vfio_user_recycle(VFIOUserProxy *proxy, VFIO= UserMsg *msg); =20 static void vfio_user_recv(void *opaque); static int vfio_user_recv_one(VFIOUserProxy *proxy); +static void vfio_user_send(void *opaque); +static int vfio_user_send_one(VFIOUserProxy *proxy); static void vfio_user_cb(void *opaque); =20 static void vfio_user_request(void *opaque); +static int vfio_user_send_queued(VFIOUserProxy *proxy, VFIOUserMsg *msg); +static void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize); +static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags); =20 static inline void vfio_user_set_error(VFIOUserHdr *hdr, uint32_t err) { @@ -52,6 +64,35 @@ static void vfio_user_shutdown(VFIOUserProxy *proxy) proxy->ctx, NULL, NULL); } =20 +static int vfio_user_send_qio(VFIOUserProxy *proxy, VFIOUserMsg *msg) +{ + VFIOUserFDs *fds =3D msg->fds; + struct iovec iov =3D { + .iov_base =3D msg->hdr, + .iov_len =3D msg->hdr->size, + }; + size_t numfds =3D 0; + int ret, *fdp =3D NULL; + Error *local_err =3D NULL; + + if (fds !=3D NULL && fds->send_fds !=3D 0) { + numfds =3D fds->send_fds; + fdp =3D fds->fds; + } + + ret =3D qio_channel_writev_full(proxy->ioc, &iov, 1, fdp, numfds, 0, + &local_err); + + if (ret =3D=3D -1) { + vfio_user_set_error(msg->hdr, EIO); + vfio_user_shutdown(proxy); + error_report_err(local_err); + } + trace_vfio_user_send_write(msg->hdr->id, ret); + + return ret; +} + static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, VFIOUserFDs *fds) { @@ -92,6 +133,7 @@ static void vfio_user_recycle(VFIOUserProxy *proxy, VFIO= UserMsg *msg) msg->hdr =3D NULL; msg->fds =3D NULL; msg->complete =3D false; + msg->pending =3D false; QTAILQ_INSERT_HEAD(&proxy->free, msg, next); } =20 @@ -386,6 +428,54 @@ err: return -1; } =20 +/* + * Send messages from outgoing queue when the socket buffer has space. + * If we deplete 'outgoing', remove ourselves from the poll list. + */ +static void vfio_user_send(void *opaque) +{ + VFIOUserProxy *proxy =3D opaque; + + QEMU_LOCK_GUARD(&proxy->lock); + + if (proxy->state =3D=3D VFIO_PROXY_CONNECTED) { + while (!QTAILQ_EMPTY(&proxy->outgoing)) { + if (vfio_user_send_one(proxy) < 0) { + return; + } + } + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, + vfio_user_recv, NULL, NULL, proxy); + } +} + +/* + * Send a single message. + * + * Sent async messages are freed, others are moved to pending queue. + */ +static int vfio_user_send_one(VFIOUserProxy *proxy) +{ + VFIOUserMsg *msg; + int ret; + + msg =3D QTAILQ_FIRST(&proxy->outgoing); + ret =3D vfio_user_send_qio(proxy, msg); + if (ret < 0) { + return ret; + } + + QTAILQ_REMOVE(&proxy->outgoing, msg, next); + if (msg->type =3D=3D VFIO_MSG_ASYNC) { + vfio_user_recycle(proxy, msg); + } else { + QTAILQ_INSERT_TAIL(&proxy->pending, msg, next); + msg->pending =3D true; + } + + return 0; +} + static void vfio_user_cb(void *opaque) { VFIOUserProxy *proxy =3D opaque; @@ -447,6 +537,119 @@ static void vfio_user_request(void *opaque) } } =20 +/* + * Messages are queued onto the proxy's outgoing list. + * + * It handles 3 types of messages: + * + * async messages - replies and posted writes + * + * There will be no reply from the server, so message + * buffers are freed after they're sent. + * + * nowait messages - map/unmap during address space transactions + * + * These are also sent async, but a reply is expected so that + * vfio_wait_reqs() can wait for the youngest nowait request. + * They transition from the outgoing list to the pending list + * when sent, and are freed when the reply is received. + * + * wait messages - all other requests + * + * The reply to these messages is waited for by their caller. + * They also transition from outgoing to pending when sent, but + * the message buffer is returned to the caller with the reply + * contents. The caller is responsible for freeing these messages. + * + * As an optimization, if the outgoing list and the socket send + * buffer are empty, the message is sent inline instead of being + * added to the outgoing list. The rest of the transitions are + * unchanged. + * + * returns 0 if the message was sent or queued + * returns -1 on send error + */ +static int vfio_user_send_queued(VFIOUserProxy *proxy, VFIOUserMsg *msg) +{ + int ret; + + /* + * Unsent outgoing msgs - add to tail + */ + if (!QTAILQ_EMPTY(&proxy->outgoing)) { + QTAILQ_INSERT_TAIL(&proxy->outgoing, msg, next); + return 0; + } + + /* + * Try inline - if blocked, queue it and kick send poller + */ + if (proxy->flags & VFIO_PROXY_FORCE_QUEUED) { + ret =3D QIO_CHANNEL_ERR_BLOCK; + } else { + ret =3D vfio_user_send_qio(proxy, msg); + } + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + QTAILQ_INSERT_HEAD(&proxy->outgoing, msg, next); + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, + vfio_user_recv, proxy->ctx, + vfio_user_send, proxy); + return 0; + } + if (ret =3D=3D -1) { + return ret; + } + + /* + * Sent - free async, add others to pending + */ + if (msg->type =3D=3D VFIO_MSG_ASYNC) { + vfio_user_recycle(proxy, msg); + } else { + QTAILQ_INSERT_TAIL(&proxy->pending, msg, next); + msg->pending =3D true; + } + + return 0; +} + +static void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize) +{ + VFIOUserMsg *msg; + int ret; + + if (hdr->flags & VFIO_USER_NO_REPLY) { + error_printf("vfio_user_send_wait on async message\n"); + vfio_user_set_error(hdr, EINVAL); + return; + } + + qemu_mutex_lock(&proxy->lock); + + msg =3D vfio_user_getmsg(proxy, hdr, fds); + msg->id =3D hdr->id; + msg->rsize =3D rsize ? rsize : hdr->size; + msg->type =3D VFIO_MSG_WAIT; + + ret =3D vfio_user_send_queued(proxy, msg); + + if (ret =3D=3D 0) { + while (!msg->complete) { + if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, wait_time)) { + VFIOUserMsgQ *list; + + list =3D msg->pending ? &proxy->pending : &proxy->outgoing; + QTAILQ_REMOVE(list, msg, next); + vfio_user_set_error(hdr, ETIMEDOUT); + break; + } + } + } + vfio_user_recycle(proxy, msg); + + qemu_mutex_unlock(&proxy->lock); +} =20 static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); @@ -475,6 +678,15 @@ VFIOUserProxy *vfio_user_connect_dev(SocketAddress *ad= dr, Error **errp) proxy =3D g_malloc0(sizeof(VFIOUserProxy)); proxy->sockname =3D g_strdup_printf("unix:%s", sockname); proxy->ioc =3D ioc; + + /* init defaults */ + proxy->max_xfer_size =3D VFIO_USER_DEF_MAX_XFER; + proxy->max_send_fds =3D VFIO_USER_DEF_MAX_FDS; + proxy->max_dma =3D VFIO_USER_DEF_MAP_MAX; + proxy->dma_pgsizes =3D VFIO_USER_DEF_PGSIZE; + proxy->max_bitmap =3D VFIO_USER_DEF_MAX_BITMAP; + proxy->migr_pgsize =3D VFIO_USER_DEF_PGSIZE; + proxy->flags =3D VFIO_PROXY_CLIENT; proxy->state =3D VFIO_PROXY_CONNECTED; =20 @@ -572,3 +784,285 @@ void vfio_user_disconnect(VFIOUserProxy *proxy) g_free(proxy->sockname); g_free(proxy); } + +static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags) +{ + static uint16_t next_id; + + hdr->id =3D qatomic_fetch_inc(&next_id); + hdr->command =3D cmd; + hdr->size =3D size; + hdr->flags =3D (flags & ~VFIO_USER_TYPE) | VFIO_USER_REQUEST; + hdr->error_reply =3D 0; +} + +struct cap_entry { + const char *name; + bool (*check)(VFIOUserProxy *proxy, QObject *qobj, Error **errp); +}; + +static bool caps_parse(VFIOUserProxy *proxy, QDict *qdict, + struct cap_entry caps[], Error **errp) +{ + QObject *qobj; + struct cap_entry *p; + + for (p =3D caps; p->name !=3D NULL; p++) { + qobj =3D qdict_get(qdict, p->name); + if (qobj !=3D NULL) { + if (!p->check(proxy, qobj, errp)) { + return false; + } + qdict_del(qdict, p->name); + } + } + + /* warning, for now */ + if (qdict_size(qdict) !=3D 0) { + warn_report("spurious capabilities"); + } + return true; +} + +static bool check_migr_pgsize(VFIOUserProxy *proxy, QObject *qobj, Error *= *errp) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t pgsize; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &pgsize)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_PGSIZE); + return false; + } + + /* must be larger than default */ + if (pgsize & (VFIO_USER_DEF_PGSIZE - 1)) { + error_setg(errp, "pgsize 0x%"PRIx64" too small", pgsize); + return false; + } + + proxy->migr_pgsize =3D pgsize; + return true; +} + +static bool check_bitmap(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t bitmap_size; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &bitmap_size)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_BITMAP); + return false; + } + + /* can only lower it */ + if (bitmap_size > VFIO_USER_DEF_MAX_BITMAP) { + error_setg(errp, "%s too large", VFIO_USER_CAP_MAX_BITMAP); + return false; + } + + proxy->max_bitmap =3D bitmap_size; + return true; +} + +static struct cap_entry caps_migr[] =3D { + { VFIO_USER_CAP_PGSIZE, check_migr_pgsize }, + { VFIO_USER_CAP_MAX_BITMAP, check_bitmap }, + { NULL } +}; + +static bool check_max_fds(VFIOUserProxy *proxy, QObject *qobj, Error **err= p) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t max_send_fds; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &max_send_fds) || + max_send_fds > VFIO_USER_MAX_MAX_FDS) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_FDS); + return false; + } + proxy->max_send_fds =3D max_send_fds; + return true; +} + +static bool check_max_xfer(VFIOUserProxy *proxy, QObject *qobj, Error **er= rp) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t max_xfer_size; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &max_xfer_size) || + max_xfer_size > VFIO_USER_MAX_MAX_XFER) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_XFER); + return false; + } + proxy->max_xfer_size =3D max_xfer_size; + return true; +} + +static bool check_pgsizes(VFIOUserProxy *proxy, QObject *qobj, Error **err= p) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t pgsizes; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &pgsizes)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_PGSIZES); + return false; + } + + /* must be larger than default */ + if (pgsizes & (VFIO_USER_DEF_PGSIZE - 1)) { + error_setg(errp, "pgsize 0x%"PRIx64" too small", pgsizes); + return false; + } + + proxy->dma_pgsizes =3D pgsizes; + return true; +} + +static bool check_max_dma(VFIOUserProxy *proxy, QObject *qobj, Error **err= p) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t max_dma; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &max_dma)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAP_MAX); + return false; + } + + /* can only lower it */ + if (max_dma > VFIO_USER_DEF_MAP_MAX) { + error_setg(errp, "%s too large", VFIO_USER_CAP_MAP_MAX); + return false; + } + + proxy->max_dma =3D max_dma; + return true; +} + +static bool check_migr(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QDict *qdict =3D qobject_to(QDict, qobj); + + if (qdict =3D=3D NULL) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_FDS); + return true; + } + return caps_parse(proxy, qdict, caps_migr, errp); +} + +static struct cap_entry caps_cap[] =3D { + { VFIO_USER_CAP_MAX_FDS, check_max_fds }, + { VFIO_USER_CAP_MAX_XFER, check_max_xfer }, + { VFIO_USER_CAP_PGSIZES, check_pgsizes }, + { VFIO_USER_CAP_MAP_MAX, check_max_dma }, + { VFIO_USER_CAP_MIGR, check_migr }, + { NULL } +}; + +static bool check_cap(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QDict *qdict =3D qobject_to(QDict, qobj); + + if (qdict =3D=3D NULL) { + error_setg(errp, "malformed %s", VFIO_USER_CAP); + return false; + } + return caps_parse(proxy, qdict, caps_cap, errp); +} + +static struct cap_entry ver_0_0[] =3D { + { VFIO_USER_CAP, check_cap }, + { NULL } +}; + +static bool caps_check(VFIOUserProxy *proxy, int minor, const char *caps, + Error **errp) +{ + QObject *qobj; + QDict *qdict; + bool ret; + + qobj =3D qobject_from_json(caps, NULL); + if (qobj =3D=3D NULL) { + error_setg(errp, "malformed capabilities %s", caps); + return false; + } + qdict =3D qobject_to(QDict, qobj); + if (qdict =3D=3D NULL) { + error_setg(errp, "capabilities %s not an object", caps); + qobject_unref(qobj); + return false; + } + ret =3D caps_parse(proxy, qdict, ver_0_0, errp); + + qobject_unref(qobj); + return ret; +} + +static GString *caps_json(void) +{ + QDict *dict =3D qdict_new(); + QDict *capdict =3D qdict_new(); + QDict *migdict =3D qdict_new(); + GString *str; + + qdict_put_int(migdict, VFIO_USER_CAP_PGSIZE, VFIO_USER_DEF_PGSIZE); + qdict_put_int(migdict, VFIO_USER_CAP_MAX_BITMAP, VFIO_USER_DEF_MAX_BIT= MAP); + qdict_put_obj(capdict, VFIO_USER_CAP_MIGR, QOBJECT(migdict)); + + qdict_put_int(capdict, VFIO_USER_CAP_MAX_FDS, VFIO_USER_MAX_MAX_FDS); + qdict_put_int(capdict, VFIO_USER_CAP_MAX_XFER, VFIO_USER_DEF_MAX_XFER); + qdict_put_int(capdict, VFIO_USER_CAP_PGSIZES, VFIO_USER_DEF_PGSIZE); + qdict_put_int(capdict, VFIO_USER_CAP_MAP_MAX, VFIO_USER_DEF_MAP_MAX); + + qdict_put_obj(dict, VFIO_USER_CAP, QOBJECT(capdict)); + + str =3D qobject_to_json(QOBJECT(dict)); + qobject_unref(dict); + return str; +} + +bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp) +{ + g_autofree VFIOUserVersion *msgp =3D NULL; + GString *caps; + char *reply; + int size, caplen; + + caps =3D caps_json(); + caplen =3D caps->len + 1; + size =3D sizeof(*msgp) + caplen; + msgp =3D g_malloc0(size); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_VERSION, size, 0); + msgp->major =3D VFIO_USER_MAJOR_VER; + msgp->minor =3D VFIO_USER_MINOR_VER; + memcpy(&msgp->capabilities, caps->str, caplen); + g_string_free(caps, true); + trace_vfio_user_version(msgp->major, msgp->minor, msgp->capabilities); + + vfio_user_send_wait(proxy, &msgp->hdr, NULL, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + error_setg_errno(errp, msgp->hdr.error_reply, "version reply"); + return false; + } + + if (msgp->major !=3D VFIO_USER_MAJOR_VER || + msgp->minor > VFIO_USER_MINOR_VER) { + error_setg(errp, "incompatible server version"); + return false; + } + + reply =3D msgp->capabilities; + if (reply[msgp->hdr.size - sizeof(*msgp) - 1] !=3D '\0') { + error_setg(errp, "corrupt version reply"); + return false; + } + + if (!caps_check(proxy, msgp->minor, reply, errp)) { + return false; + } + + trace_vfio_user_version(msgp->major, msgp->minor, msgp->capabilities); + return true; +} diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 89d6c11c4c..7a3645024f 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -4,3 +4,5 @@ vfio_user_recv_hdr(const char *name, uint16_t id, uint16_t cmd, uint32_t s= ize, uint32_t flags) " (%s) id 0x%x cmd 0x%x size 0x%x flags 0x%x" vfio_user_recv_read(uint16_t id, int read) " id 0x%x read 0x%x" vfio_user_recv_request(uint16_t cmd) " command 0x%x" +vfio_user_send_write(uint16_t id, int wrote) " id 0x%x wrote 0x%x" +vfio_user_version(uint16_t major, uint16_t minor, const char *caps) " majo= r %d minor %d caps: %s" --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255192; cv=pass; d=zohomail.com; s=zohoarc; b=Lc6C4UtePKNRcEKV5nmoaxjCd8Q4FYxNcecmknFBnkJH2Jtm0+5ntzGq0OeujckjHOcxGlxcuBXU3fP3GgI+51IsNiM1O4P9naQMgLgR0ttLpuMGIj/nEOlY89mpVpIce7nx0M/NXH4bGlEXEvb15z7kFF/KEJ0oNZDIHZRZt9A= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255192; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=UH6f2tmI7ATg3KzwM9TyWxj1523guBD6/z5O7cFLNGY=; b=KJciV9YDJFE+Av2lTCQpLsI62AlvmLdM7LdPd7hq4fBnPEeAn7LsQkMMi4fQOd25IBOkFzCDZ4k2BQfw/d46AGy40J2ZlCFAF/crcfEQbWVlrDt+gxa4bfv+D7/TBebCnthfyVUVzA/svfSy6wmUed/837KFFy++OXTNc26yfIk= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255192294222.00905375123523; Fri, 6 Jun 2025 17:13:12 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9b-0002yr-OH; Fri, 06 Jun 2025 20:11:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9a-0002xX-U1 for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:19 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9Y-0006PH-R8 for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:18 -0400 Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556Kw9UM017955; Fri, 6 Jun 2025 17:11:15 -0700 Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11on2139.outbound.protection.outlook.com [40.107.223.139]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 4747u3r8fh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:15 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:13 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=UH6f2tmI7ATg3KzwM9TyWxj1523guBD6/z5O7cFLN GY=; b=0u55zSDIokWCwoS+LyOLpI/ieo13hQyUeKoIZjwcJpRF6LnL3+/A/YY1G WLKmja4IlnirIRYih8w13vNYrKiNjb18MSS7vvuYuHv8f3vJNflz5/w4BORQ1yq2 R1rwOhEdsmIKovy4mNUeD06X1t4wXMCFw85v1T9uAQdVfj8eJxHQrvTkqWLTjXyn Kyt91PtyvTNcK0WB32MSRIG0b3LXVdVo4UQLDx2bmKKRO89aPYOO6fRAbP3qZhwY /Pvzf1ZkOK5LNhnOqc8okBdzXo2o9yly0WNmuTbb8du/BJFCjyMQAptpaC0BmbAO lm/rGTbxS39xWJPA0Uw1hYDuQVJow== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=OformQ9m759/wjPfyY/jia/KJxBr+Yb0NByc4j/ZWPiA8IhuBD6mPFGU0ep9guuso7ay017HhqHMGkevjSB+lFAq9z55dSt0Jnh2onU+jg8g123RiqHz9RIOSN5m73oopS5/ajBItbbdDvv677bUg8UAhewd1YrgYgkAFpcMHLu3t8++YhreBtZ0Bmjvs46xKa0v+JglFTC4bCOC+A1X8MKT7Ccoj+tp4LIU5oFvOKHIeCICNscvbF2RYxixB8S2LXFeBfLC0QxDrvJWoOwhWdkycTVakLnYHdk0KcGkRimm2SERXGMW6N3k/o95O+sglQ5s1SQ86o1fec5N0NUXYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UH6f2tmI7ATg3KzwM9TyWxj1523guBD6/z5O7cFLNGY=; b=CFM40aVJAJXz2Q4ufHJrCiKjguMc8yaScJQQkCNwce55lj7sGyw3qu4WPrANlVkGXk25Zl6V/QWNufbiYTovhO4DNoSQ1Rvzy/ftWWaSsQ/fr8Jan0qb1NLMrW0ik9yaFsn5yTE2nfvhVP4X80m+b/5x5DqMDSVr2IX8SMzDzI8Hn/zfd524lPqyRx/JK8FheiSt8rCnnOk57hhGnffyqWk2AR62Ym4uu2NA7r3hYbr9XuxQLyYdYVnQE2/6IjKG6ZjLc9q8ySNqSu0UlBwDIdmwIRks6aTktHK0N8evz7wfClyYyPQK/DBeCPOm+DtmPTaobU8wo9glXyZ/s3yoww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UH6f2tmI7ATg3KzwM9TyWxj1523guBD6/z5O7cFLNGY=; b=n/Q5aAw1WscJBEM6HnyaB03emGIfmt1YKa7roXfz3munimJvu8tW2O7K8ryHxuE+txtq2nxtb8Vl5EEGMRIKkSWNyWVXhaSKzBfFYM1fopjR/Rls/7cogB8wN/g+FO2wRzxJwGjjwr4y/sRwiNyZmrnRB14QbDqy3ahHScDgObSbJVFVwV89sqt7fVU7PjGvXMvTZ0aK4QwWP0xSBbPk7m48kDHNXXXwzUZRRWPBPr3ZuET0Rc/KyFIlJxRCtvfkykG9usEtmafdJVunxlk4pkTg3rYEG31MR+HNxMLo/lhZYwAp3u0uTuLhLBfa/zKxUsamsaPNbPeIZW62E7MxbA== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 10/23] vfio-user: implement VFIO_USER_DEVICE_GET_INFO Date: Fri, 6 Jun 2025 17:10:42 -0700 Message-ID: <20250607001056.335310-11-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: d847f374-0dfa-486f-dbc9-08dda557d04e x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?M1M0ZzF3QVlLZXUzU1pIZ0Rad252clFqbEJmSkU2REdMOHpjU1RZT2c2cTNr?= =?utf-8?B?WENUMTkySmllWHRaRVF2d0ZlT3NzcnlEREh4dXd4TnNCZnUybitRbG5nb1p1?= =?utf-8?B?ZlZNbGNvQVpXeEVJYnAwSHJxZDhGVkRyTG0yQUNVaXgrZFNzWWJ5TXZTM2NT?= =?utf-8?B?RVNRMkhIaGt5dmdOY1owYUNNKzZxb2xKL000a0pWMjVManJQUUx6b2hWM0g2?= =?utf-8?B?TVhLK0NiY3NkY2gxY3ltRzZBdC9rVWxHaHlkM0JRR0hjVUZBNlRURnp3Z0c4?= =?utf-8?B?LzNaN1pKdXMwclIxWXRMQkJ0M3ZBWlBnbDRzSDZjR0V4dDlQSWRiRHc0T3Rj?= =?utf-8?B?Y2Rzd3VURVhpUDN0bk1YNlM3cTZRczg2WGFoS1RySnIyUCtid2VJSHRVRW9K?= =?utf-8?B?Z1l0akdESkVkRDdVVzZCbEtDQUFmaCtwMno1UjgweS9taVBMN0xtMENOVmti?= =?utf-8?B?MnBoNTcwRmRDUUw5dU5ZM2hTMGpSMW5scFlNbU1vT2Qxd1l2Zy8xVkpybExt?= =?utf-8?B?TEhwUU1wcDhSdlhDSno2TW5UdlNVOGJYVXMrZmRZZTBjemRVVlVNbEltSTM2?= =?utf-8?B?dUpoYm85MkVoU1NvcWJXcGlNd09KM2ZxOUd3TlZPUzl4WmdSZWZCSG5xM2NQ?= =?utf-8?B?UkZrbGowNGF0NkRjekNwaHNLeDlGYW5CV0UxQWRZUmRFaDZiU0dkT2JFT1h4?= =?utf-8?B?dWN5T1QzR0pyWXp2SWYxakVycXh1K29DQnJBWC9rMVVCTExLVmdJeDRkNmcr?= =?utf-8?B?WGlVRFpqMVUyRnlBZkVYdVFNeFBQcWQ0dkhMUzNUVWJUK2RlWG1yd0l0eldZ?= =?utf-8?B?Z25TL1ZUYWJabnFOMmt1SUdsR2s1VFo5S2dZczZRKzFsZGpvRWxpWDVHSk9Y?= =?utf-8?B?bUlHeDZGNXE3Y3RZd1ZtUXRDVE01bWswZFJMRFg5RVBJR29CdXpMMUVKUUNi?= =?utf-8?B?RXlxVkVEWDRvSmhER0lPNHVkT3JUS1JuRG1DMlJzNG5FaXRRTzNOdUYvZkN0?= =?utf-8?B?bVkrbDNITVBGMnFwK3NBOU1wU21NaWRDazlwMTVLREpvN3E4TDBmV2Zqem5l?= =?utf-8?B?c2NjZCtFcGtod2FZak9ITEFBci9WSEhDSWNzSW56ZzhPeW5NUjBCenVTRStM?= =?utf-8?B?YzFtZmNmeHlINnFycm9OMm9TUVJyL0NGV3R2cEZVc0RSd2RIZEtMQ21hVDY0?= =?utf-8?B?djFKQXlvS3RGSFNxZjNOdkZnUFp1OE9oQ3NQYkpxY0lrbHlkWUpyNmM3dGFv?= =?utf-8?B?Q29yVlZucW9ROVFTMGlKYUxEd3dxcFA2ZmJ4TE5kRjNQNnhJMFRKWXpqVGF3?= =?utf-8?B?RVEvVzFNZ2xFU3IxaXRQWTdadzl2K3lVTEVuaFh4NjN0dUdzUWlrRWlUeldu?= =?utf-8?B?OXpNakJwSHpiZ0NCM2NyZ3lmQzdjOVVGVUMzTGUvd0xZWlYvbGZ0bHFZUEJ4?= =?utf-8?B?VXZ2UUNHbElIWHE3R1ZickJpaFo5YWpUeVJnRVp3N3F4cndjRGJ6QmpUL1l0?= =?utf-8?B?TS9MSTZxZWtMVUEyajlmM2drbGFOODZCWWM0Zy9RU28rQTlDalZWWEhGUENG?= =?utf-8?B?MWFJc3FGdDNsd3U3eWgzQk04OTIxSVBnNDZGMjNlYVdyOW1FVnZKdWpSaWhl?= =?utf-8?B?SllIYW1lM3hESUkwcDRVcEhOanhCcWpldjF2a1ZPNG16MzM1V1BOWWR4MnMw?= =?utf-8?B?UU1yYWNNanhLQUR0TWJZeTdPbUpKb1JXMWFIaVZwUlRMYjV2L0tUQWN1bEl3?= =?utf-8?B?OVBRU1Mzei9ENlViNHdDQ1BmNjNiY2NNR0FpdkhTRlpSc0t0R0NvTWh1WUFm?= =?utf-8?B?MHRrcUM2RGRyNkpVaGVZRXNza3BwTDhLSVQvZDhUR2psNzZodkZJQ2t5dktk?= =?utf-8?B?dVhnUExaTWk3c21FUjk4ak9sdlorOU84MmlWWXZiRHlINjJHRHo4bmlTQ2tu?= =?utf-8?Q?m+9DRQSNxt8=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WEpvS2ZYSkt0eG93bldiQkhLUllGNDArZFUyenVZbDQzaFN5NkZyNmthWVhJ?= =?utf-8?B?dnN6WlZOeUFtWmVzQWZ4OThpTmFiZWpjdFI5VkNxUG1GYjAyUlV0dGNqWmFj?= =?utf-8?B?UFB2Ujk2UXdCdE1YM3NYNjJGUFhBN1dnUCszSW84MDBOb1NJVkc1WGZpMzlt?= =?utf-8?B?aU9lYW95WnBRWGNzbGllWVNIZ0lNczBFWXRXTDVZdDRvVWNOSEJLMEtBQ1hw?= =?utf-8?B?WnhZQzkraTJJV3hKMGx6OWFSUEVNN2M1MnVWOXFiMHE2WHRGcFlVaTVPSEU5?= =?utf-8?B?Q21CdW02TDYrU2paTVhEeEd0VmE0NEZEOWtubkpRMXMrbVVrbG54TW9sR0Y3?= =?utf-8?B?YXpZVWFwdllZejBmTWR5VWFOYzFwaHluakQ3SlRJSmNERFV6WFpUSnNGbVpj?= =?utf-8?B?anl2SDNRZG41Mmd2RTFzRnZVb3Y5TC9XTm9PZnIzOTMvcE5XSTNlOGV4aXpm?= =?utf-8?B?ZkFKRzBVcEJ4b285cHZRMlB5OEpLb0p0ZEQvc3pnb011azdxTXcxaXlLZ0tJ?= =?utf-8?B?VWxWS1NQYkZQejJiaGhYaW1ybVlVejJvMVZWcTYzUUd6QjhGMktVY3VMcm9Q?= =?utf-8?B?S2VpUG9Uei9xb25xeVdPS3RNdGhRc2pYYmNtcmJEYjdJeGtYV2hsZW5jUVVj?= =?utf-8?B?RStHc2Y2L29vR0dVS2E1UjBqTTRTcFNndTNwYVlxelVVN09qTFZoZnljNml1?= =?utf-8?B?a0xLQ2JtVnduditSMy9lSWZubjFyTTM2aFRQZnREeWxHM1BPTkRjNUk3aTAy?= =?utf-8?B?ZWs1eWJ3clZJbUZ2cGY5YnYzcjA3WFdqNXlhY1RuMW1IQ1BvTDJKempYNkJL?= =?utf-8?B?c0czbUhhVVp4a3J2QndpVitPU1NjS1NNS0F3elVOcFdxcUpHNjdaYm9zd1NF?= =?utf-8?B?RnRENnZiaFF5TWdkV1c4b1dkbHNsK0w4OXlzK2JXN1lTNkFHa1YyMmNyU1FI?= =?utf-8?B?emY0cURuY29HNC9MNTJQQmRWcnNOQ1k0NjZUMGk1TGZCOGJodEQ1dHE5RTlV?= =?utf-8?B?WTZEMFR2UEQ4eWRvUlA2T2FtZnJSSFBCM25jMWZTa215WHJCQncrR0lGZENM?= =?utf-8?B?aS9OcGZRUlFkZHp1aHduMVJLS3ZJVCs1eWlJb2lxODhCd1pVaHdlL05wUnU5?= =?utf-8?B?R2NFd3VkN2VUZForcGI5UU5LM0hSNnhXOTlpSkh4SlpseDFtWlVabXJRVnl3?= =?utf-8?B?WG1jeGxaMThVNkJPa0ptZit0UWVuK2NkZHZmcExYNTBXbkJxazEveXFJNm5N?= =?utf-8?B?eExmTWFDOHpHS0Q0dEc2ZklWRGpCUzVaQzY4RXAvQnhSVjhBY0NHUStYRitV?= =?utf-8?B?M3AxOXhPZ0t0b2VLY3JHSngrMlhKMjJGZ2NIQXIxeHRBYnVldU1BUm83RmEv?= =?utf-8?B?Y0k0cW5aZk5tZEdzeGR2V2F3alNSOEh6WER0UmRjVWRnamJ2dnNncFg2eEs4?= =?utf-8?B?VEw3SmY2eVppNDBIWU9taXNQTVkwRVZEZFdJckcwV0NBUG5jNW5WVXZNS3U0?= =?utf-8?B?RG4zS1QzMmFoT2FWRWZ2M0hqcnBQMEdOeG5CdzNuek9YdUlTNElIM2d0amxF?= =?utf-8?B?R2N6T2tBMnVaNGthaGpmbEFhMzdZM1ZTT0hwNDd0K01DaVAyVnV3My80OWxR?= =?utf-8?B?WXN1VGdpSGc4VmhZQW5BWVIrYmxwa1k2N05ETGpCVGNpclBMNll4dDFMUXBP?= =?utf-8?B?bEl0Z0I4UTk4WXJCOGsyNmxZNFdySll2b2l5ZklhL1FVRUZUdy9OczMzYlFD?= =?utf-8?B?TzBqZC9YN2ZzNUZRRlVGWld2RGdCd2VTd2pjdXJrcTFSeWNTckJOSkQzb0wr?= =?utf-8?B?eUZ2eXdzOEk3VkVXYVh3WlBEWS84NHpkMFFrRklBalZXd0RkK3d1R05NakxH?= =?utf-8?B?SHlsNnFrNTIrdUFNT09UTkNycTN2ZTJoTlAyc0VSNVVqVitLc0pTVHJmeFVY?= =?utf-8?B?cmpPcXA1cnhGVEt2MVlZTk16aGtWMTlEenVyMVI5ODFwTnBISDdqQ2JEQTdU?= =?utf-8?B?bng5MUJ4U0ErUDVyUDFJWHJrSVlqOWk1ZDlLZGRObGU0aGxRaVpEQzVlcVJD?= =?utf-8?B?bm9lZXQ3V1hDUUlhclk1YzczWDBPUVJoRVpoYVAyMnk4c0pwVmJwaDM1b3Vr?= =?utf-8?Q?0V1CTUhWak/t+QVrzRF+aoVVa?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: d847f374-0dfa-486f-dbc9-08dda557d04e X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:13.3411 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6gbcWBx0lQ/4ly+88mnj/maIwpwvRV7AIjoUWuobvzAx8iqKZEQmCjwwtF3cD9m5fNRpQklXiw1Wtm0KlgFV6A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Authority-Analysis: v=2.4 cv=cpSbk04i c=1 sm=1 tr=0 ts=684383a3 cx=c_pps a=Cb1kLmlYO8D/Z2wQFUco/w==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=9e9T-0rTUHAn1w8HAtsA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: wgSlH6HZY1mVlena1nczi9nohm0kaytw X-Proofpoint-GUID: wgSlH6HZY1mVlena1nczi9nohm0kaytw X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX/AcMiDzHkpaf qjdl21q9i8Fvp8mkE9np3hzqnLvAS+iGO2BCRnkS4Z8kOCZ50Di620aj3NESfrsQTks7xUAQmtC safxNepruNZC+9lrL/YWOpZAQNhUT9ZU9Eqo4j4YaOQRdaqHD/XkT4N+l3jfdDUJBIe/BxkGQ6Q 3m1tsCYzn1cCz3EIh64CDSGT3ThNlMXTmUoFht1g8lslxDydtWISEUH0IWZjBJWcWKpyEy2721T 6MdPTHvhqKxClJkXSwnqHKMadfd0hy6q7Ij2fsMSQdjiVuwVd/vpYhmgr48bWDx6cs0NFVaQovx QwgVfcmv7A0c2JdhtPD3fQ9C4bGh2WzYhr6hoGF7x+rG1dQ2l/Mu1UJbGTg/dFpE7NdJJ6y5cZ/ 6CJeu+sQ9gfwJZC7X6CHYPgd2hI4UhjSmTZqZp7tiqvMDtwXOBOc2hgorTsmjzqgP+QA7z/j X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255192884116600 Add support for getting basic device information. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/device.h | 23 ++++++++++++++++++ hw/vfio-user/protocol.h | 12 ++++++++++ hw/vfio-user/proxy.h | 7 ++++++ hw/vfio-user/container.c | 10 +++++++- hw/vfio-user/device.c | 50 +++++++++++++++++++++++++++++++++++++++ hw/vfio-user/proxy.c | 12 ++++------ hw/vfio-user/meson.build | 1 + hw/vfio-user/trace-events | 1 + 8 files changed, 107 insertions(+), 9 deletions(-) create mode 100644 hw/vfio-user/device.h create mode 100644 hw/vfio-user/device.c diff --git a/hw/vfio-user/device.h b/hw/vfio-user/device.h new file mode 100644 index 0000000000..f27f26b479 --- /dev/null +++ b/hw/vfio-user/device.h @@ -0,0 +1,23 @@ +#ifndef VFIO_USER_DEVICE_H +#define VFIO_USER_DEVICE_H + +/* + * vfio protocol over a UNIX socket device handling. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "linux/vfio.h" + +#include "hw/vfio-user/proxy.h" + +int vfio_user_get_device_info(VFIOUserProxy *proxy, + struct vfio_device_info *info); + +#endif /* VFIO_USER_DEVICE_H */ diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 74ea2a94e6..648badff46 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -115,4 +115,16 @@ typedef struct { */ #define VFIO_USER_DEF_MAX_BITMAP (256 * 1024 * 1024) =20 +/* + * VFIO_USER_DEVICE_GET_INFO + * imported from struct vfio_device_info + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t num_regions; + uint32_t num_irqs; +} VFIOUserDeviceInfo; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 33e8a4d83b..9c6b2cdf35 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -15,7 +15,9 @@ #include "io/channel.h" #include "io/channel-socket.h" =20 +#include "qemu/queue.h" #include "qemu/sockets.h" +#include "qemu/thread.h" #include "hw/vfio-user/protocol.h" =20 typedef struct { @@ -99,4 +101,9 @@ void vfio_user_set_handler(VFIODevice *vbasedev, void *reqarg); bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp); =20 +void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags); +void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize); + #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/container.c b/hw/vfio-user/container.c index 2892845b4f..6f0eb86a75 100644 --- a/hw/vfio-user/container.c +++ b/hw/vfio-user/container.c @@ -14,6 +14,7 @@ #include "qemu/osdep.h" =20 #include "hw/vfio-user/container.h" +#include "hw/vfio-user/device.h" #include "hw/vfio/vfio-cpr.h" #include "hw/vfio/vfio-device.h" #include "hw/vfio/vfio-listener.h" @@ -144,7 +145,14 @@ static void vfio_user_container_disconnect(VFIOUserCon= tainer *container) static bool vfio_user_device_get(VFIOUserContainer *container, VFIODevice *vbasedev, Error **errp) { - struct vfio_device_info info =3D { 0 }; + struct vfio_device_info info =3D { .argsz =3D sizeof(info) }; + int ret; + + ret =3D vfio_user_get_device_info(vbasedev->proxy, &info); + if (ret) { + error_setg_errno(errp, -ret, "get info failure"); + return ret; + } =20 vbasedev->fd =3D -1; =20 diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c new file mode 100644 index 0000000000..e081033ff1 --- /dev/null +++ b/hw/vfio-user/device.c @@ -0,0 +1,50 @@ +/* + * vfio protocol over a UNIX socket device handling. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" + +#include "hw/vfio-user/device.h" +#include "hw/vfio-user/trace.h" + +/* + * These are to defend against a malign server trying + * to force us to run out of memory. + */ +#define VFIO_USER_MAX_REGIONS 100 +#define VFIO_USER_MAX_IRQS 50 + +int vfio_user_get_device_info(VFIOUserProxy *proxy, + struct vfio_device_info *info) +{ + VFIOUserDeviceInfo msg; + uint32_t argsz =3D sizeof(msg) - sizeof(msg.hdr); + + memset(&msg, 0, sizeof(msg)); + vfio_user_request_msg(&msg.hdr, VFIO_USER_DEVICE_GET_INFO, sizeof(msg)= , 0); + msg.argsz =3D argsz; + + vfio_user_send_wait(proxy, &msg.hdr, NULL, 0); + if (msg.hdr.flags & VFIO_USER_ERROR) { + return -msg.hdr.error_reply; + } + trace_vfio_user_get_info(msg.num_regions, msg.num_irqs); + + memcpy(info, &msg.argsz, argsz); + + /* defend against a malicious server */ + if (info->num_regions > VFIO_USER_MAX_REGIONS || + info->num_irqs > VFIO_USER_MAX_IRQS) { + error_printf("%s: invalid reply\n", __func__); + return -EINVAL; + } + + return 0; +} diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 0bc12da865..34df9bba7b 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -42,10 +42,6 @@ static void vfio_user_cb(void *opaque); =20 static void vfio_user_request(void *opaque); static int vfio_user_send_queued(VFIOUserProxy *proxy, VFIOUserMsg *msg); -static void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, - VFIOUserFDs *fds, int rsize); -static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, - uint32_t size, uint32_t flags); =20 static inline void vfio_user_set_error(VFIOUserHdr *hdr, uint32_t err) { @@ -613,8 +609,8 @@ static int vfio_user_send_queued(VFIOUserProxy *proxy, = VFIOUserMsg *msg) return 0; } =20 -static void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, - VFIOUserFDs *fds, int rsize) +void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize) { VFIOUserMsg *msg; int ret; @@ -785,8 +781,8 @@ void vfio_user_disconnect(VFIOUserProxy *proxy) g_free(proxy); } =20 -static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, - uint32_t size, uint32_t flags) +void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags) { static uint16_t next_id; =20 diff --git a/hw/vfio-user/meson.build b/hw/vfio-user/meson.build index 695b341547..19bdee688c 100644 --- a/hw/vfio-user/meson.build +++ b/hw/vfio-user/meson.build @@ -1,6 +1,7 @@ vfio_user_ss =3D ss.source_set() vfio_user_ss.add(files( 'container.c', + 'device.c', 'pci.c', 'proxy.c', )) diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 7a3645024f..6b06a3ed82 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -6,3 +6,4 @@ vfio_user_recv_read(uint16_t id, int read) " id 0x%x read 0= x%x" vfio_user_recv_request(uint16_t cmd) " command 0x%x" vfio_user_send_write(uint16_t id, int wrote) " id 0x%x wrote 0x%x" vfio_user_version(uint16_t major, uint16_t minor, const char *caps) " majo= r %d minor %d caps: %s" +vfio_user_get_info(uint32_t nregions, uint32_t nirqs) " #regions %d #irqs = %d" --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255170; cv=pass; d=zohomail.com; s=zohoarc; b=erpMHZY56wXWligxbsYg0Vd39wN7gE308GAYk9TWV3D8RBcli2Ytsi7EPFuzVDNQPu7UVZbRA3YIhXM4ZdxhbkqW5i+9jnV8MniOxdvxx7DaSxG1uyolAqZbxEUWtZmDJAY291qOsHN4ZXV9pxGHHnV4cAxG13c4tfke5nAIRBE= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255170; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=JN2u1IJiIG9UH70AwmwkajUabBWu6GamPx886iAAWWw=; b=nu3L4AGuZJYRIGNd8J6tqjw3h65ejNs+hBHUHhL4AnpvNULXfr12t4dnjscry6oMxqeNLUebL0i7qcGl960pFuznaW2QV9czh/NpxP/mw/DJtNmn7NpdN+0orUaDvsVMp2wZBuPn+ySVopadPP3ZWvVvFy4XHMMBWkcle3c4bFQ= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255170795285.1233455047915; Fri, 6 Jun 2025 17:12:50 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9d-0002zw-Ou; Fri, 06 Jun 2025 20:11:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9b-0002yq-Mi for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:19 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9Z-0006PY-O9 for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:19 -0400 Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556Kw9UO017955; Fri, 6 Jun 2025 17:11:16 -0700 Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11on2139.outbound.protection.outlook.com [40.107.223.139]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 4747u3r8fh-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:16 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:14 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:14 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=JN2u1IJiIG9UH70AwmwkajUabBWu6GamPx886iAAW Ww=; b=PuggaElhhLDigxLyNOSZIbzgNonHjZWACyBWQy7qbr9EBUuSeXKStmm5C RK30OeCRiPhNKUPvFhp1zgqgozl54QvOWXOZNSlXH2PBYrJrvTmsdXdBCDh/kPm8 bsu3iYE/GsFxVjOMpPEYACL2B3aDsrMx7uKkPVqusKl1FN1Sbtl3TKzgNdiIJNil jbrTbhoER3WDf9dwBDapc6lvcoyjmlF1Bm5gBqyJCTMC2vlMEX3wJffFCjaLPYuM lguQLeohn/ytNjyUFlPWtMmVpcVRF/jmgvCtTR7jCVfmZK1cuP6e2CcQRASQ+FA5 JFIfj0EYOE1eG81uQbARbzAkjY1Zw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NQqTXSQgaQOcDnSr2wmPYkVkwdBQniwpeECUft8q9JNgpUpNIuX/RtAxG0smzPzcKBm/9Rgx/fbyIjvsRhSKj4tFEDdeR77FifGK5GJ4v4gnDZoGydwP1MfNhAfrldh6+OwK4xlTtznqnFDmAhMxEEi1AfOnjwGcbyFF8RK0Vfb/os8WVzpshbpY0UovD58szs0fAwiiZ4zQ9gsQUtfvIQnIxc1z7xFh1GRVK98Ld1NbqmU86huwu+T2qnca0oDB+JqZRrak/IzV0631tEgahYzbRAGbZQGByIg267PBPYeLabvPBAqLNF9uNI6a6irFRPYdQZuXi/xNcJc34oEkKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JN2u1IJiIG9UH70AwmwkajUabBWu6GamPx886iAAWWw=; b=bUsxmOsG56h4AYmUz4HRLKju3PkjX/YqGIZBMJG6yHxSDq6CC++bPPohxS3NV5Bo6Ju9xQVdon0CsdzeGoJYLVJCgzTWh2MFZu7s0ARDgsQlCyw5WIZuOzb7mJooSkTm4hvUFibXLxSLd4Y4AyCp4uGpOkrUx6uDU9JG+7GWaVfKjGzqX1+hG1DdlS3B/+KUvhoE94SZheAAjFhAspv8pTuy+t9zafczKFBXPVUxCoLWjXkzlsO9m+v8r2cWNoOzyM5eqse65gr8y7XWwxEUzMs20yO0M6QujOJWyWEHjJzMMmIT16En7vC4FJZ6GaG/QY6fUm9BRTV9iMpmX0nCJQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JN2u1IJiIG9UH70AwmwkajUabBWu6GamPx886iAAWWw=; b=DzVCpR0UuMPzKBDmTJ3AFGen0BzrkOeTIaKbS/zvP220+np5CQzrqJ/MsLEFy2Ftj8RFfQi0s4JGeS2I9FwUFlCzC0SvIl3QYUBdpKv+rR9gHhA3TLT47JJ3DvKN8kiwAhrkhMCS7gFWqehb6VOjjZljb6J89f84CVIv5SWDZCdBI6nWaTAf5UEUskS1FLbm4FZ7OyQC0MJo/12ASQxyzLrN6/ZjgnGML7+NkaVFmz0M/xhejMKtN3sRtfzurJhQAPceNsCITpVxtT0m61d/eL057EyZsgrcHPAmco7Y1CMSrOigVz8pJbp7fZYXJyurAHObJBrr4MRPTJKvPq3MJA== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 11/23] vfio-user: implement VFIO_USER_DEVICE_GET_REGION_INFO Date: Fri, 6 Jun 2025 17:10:43 -0700 Message-ID: <20250607001056.335310-12-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: d1e185ac-1c58-450f-0612-08dda557d119 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?x8BUzLDlZV83di7aMwxk+6FBVGQuuWF4CQRRctdnxISnHShBqZqPNBp2MisG?= =?us-ascii?Q?qQBksFPHw6a30b0s+5Y1FB0GSg/OIDKqLjU0T451byTOdxu7jNNuSGL9s5f8?= =?us-ascii?Q?6lAJco2ja1DjnHjywAx+luLQDW371iZjb1X6fD7oCfNEf1NP8eXgd500Xa0/?= =?us-ascii?Q?rhNz/p2CBBUair3tYk+ctNUGc8GDzViK2zGeOYTKjUTbj9MIf5YLVB8euBiG?= =?us-ascii?Q?3gJumf2ST6uPXXWZVqXKCsOcvIh22ApE6l13vICE6MLUe8NQaR9nhEq/H3Sq?= =?us-ascii?Q?4TYB5AVnCh1oEKBZS51i2VYHna+wRsjoq7UoD4FIX0moTqlvrvEpH9qLPQMj?= =?us-ascii?Q?ZIXXFulanvkuu6qpXCTnCmuM3Ie7qyeBVmQGDYxrlr2EqNqSGrbM2P+fB2UI?= =?us-ascii?Q?+4w9spLsDjechCVKOsmYxHIBIFWcqXAVXDmqFfdASveZsQsdTJ5OuNYncZt0?= =?us-ascii?Q?ZPSIbGxhz3NYBUfCU8xSEQnHsZ96Fxe99HH4evpysTsPqvAXUGtjQZX6K465?= =?us-ascii?Q?uLauz14+nFWgYIk+wdZU8M2b9MoqfB0hhnJliYj3sO3d+3S08wx8mOpg3wSa?= =?us-ascii?Q?Mub7U+l9p30LmMfSiC3IqEGhFdp3E4kB+GkIE69/9WgYV0kR4W4wDYSDfOXM?= =?us-ascii?Q?ur60v33nnsckwE7HWh7cmU1r3ABtHx9ZPpuALagyl3n/9Gw7MKzXUzhpsog2?= =?us-ascii?Q?3ST0KRJIK11951RkanSZFW5dsZJNyHrVJ+wNuJjm1EjrA7lil2U2V4G7DUCf?= =?us-ascii?Q?q525lXIgAGlwO6NTQk/6uk6kJ8b0fHUBU1AuFnWLi0NJltjXjtt6WI7DPO0X?= =?us-ascii?Q?KeL1ZBnn5GgRHAxeBBfn+BlzowtqXzbdpP0Eh6fD2cYQK3yn/8DXLBmD9IAx?= =?us-ascii?Q?dzGhtKSz8ADupvM3C+J/tQkkv74GIXPtadIz/u6kyNhwM+aGt9fPjivJV5ww?= =?us-ascii?Q?e5lWV6UWgiuX6Oy671DcRSt5WRrJ0uTY6LMe2xhA1as405VeFE+cDIIyc20e?= =?us-ascii?Q?Tcj1xW3geqwBBD89t/i6Y8O6dvO/+0Fw8luSPQYkCnnC/Dn1ug8FUyYQaLDk?= =?us-ascii?Q?8bPB34xrdKN6/JtxmcDVcQKYneGCTYG6HBF949d/OF+EXr97rkvwcmU+LoUm?= =?us-ascii?Q?kMwhNgJIeL5xWs1PACoUdY3ao0v2ebgcdrH1fdIllSGzLbmXrgNBE4M90qJO?= =?us-ascii?Q?iQEK1/CSkdF/dMm/2yq8sMJFm//awrFO7fUlYy+bEXup5wPQ0bO/bT7WAPHB?= =?us-ascii?Q?IsrVS16az+1qvFP3vsm0YbDx9l9yCBSgID1nf1AvGNPGPeFHMq6uVEcCddMB?= =?us-ascii?Q?VO7bASuCl5mr3gUNi6ECvXCTfANw7AOYb9fgtDYubcljPgTgqhsAh8yNcIOd?= =?us-ascii?Q?DQtRDKeiA9ekeOYqaaDufCZyLcTR9eZFkxOZ0fe3bM2vCF9qCI98Rru/2Jsp?= =?us-ascii?Q?xtLTnX/Zd4c=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?vYDhRYt+f+ia+PGRoGEz9Q/a4CVmavjULoYa5blP1SGLpGKB5DmvzlONW20Y?= =?us-ascii?Q?DMQJtFQzKkmnjwSZvwyRqefFsR7x7qHXP2bLqc2ZPelpGOFa+oWIUwme+Mer?= =?us-ascii?Q?OSVH1n322rj6h0CXcu6322ATLF+/fjS65mKsFrZGFKJltbpLayCafEiYvJDO?= =?us-ascii?Q?Dxg4kd5EME13z+W07zUVHsE/ctTBXLNUdam5Me68pcJn6SaQC/Xy+Z28Rmry?= =?us-ascii?Q?4187kuHninKoyL0fFsxjpEvNvf6IcCFY4SWjgPXRhfLiEKoUfAiiwPD/3rkw?= =?us-ascii?Q?KQoK/1/rQc0D9fnOvjIaeYQimx2NVfVEM9kyOiEVJr2oc8EAW5CMqBLIGX65?= =?us-ascii?Q?t5YZKVaCqZ7fJA3t2EZuDohqToQoywv+0jWuDEsl13pD2PraxqA6BZ/hVwV/?= =?us-ascii?Q?jJA2Oo/29PARjtvvCiWFK2mb5mqG5jyzGqFeQs0ostgl2JYwizHQ0rJbQA8h?= =?us-ascii?Q?+iVKXzsYeXZBS6Vow3MrAgMus9bzr2svB7q9gdB9BbXKUJll/ZH1DogK9bn9?= =?us-ascii?Q?+EzwNV8Tliy3e4Rg8Z8hx1G4dnOqAQGK0hrAj+7dyGzy4iXUBp4ewaQCjSm6?= =?us-ascii?Q?/AGFrcnHraykB/h6eCpNZIUYVPjb+Gzqi3XSLDZODLK7rSJ+GcFBpV8ndq+l?= =?us-ascii?Q?NEynzcyvEmnB0CrbxGlzXfcuDqF1CSbPFRyrxlWYSgEpQ3QijOkg4pID9/iS?= =?us-ascii?Q?MYRdnEsaH9GnXJn5JJHG0Tiv5ntAI44RcoH+TDhrL1qTRr/uTQjCNMoahRTv?= =?us-ascii?Q?5j+An+X3t3r2p0eqxyZQCqQfETscYLkH2i81kGg2TR82nx2tm1tY2f1lmPNB?= =?us-ascii?Q?RBOc/r5ULoQ1/M88pwFnyJIgLBV9MRl4+20DVGoSS5/0aB7z+fvGsRC3RE5t?= =?us-ascii?Q?6ML/aSmDYk9MhG54pU8Xhg53CPZqjzmDgFiOw5+t9S1aSHg+lQIFF2UQ3lJM?= =?us-ascii?Q?l6Z7Oik5L2h+jqP/5i+/2SQFOyQTcJMIf6qpAq7JMgih63o5oqM983MkXwhw?= =?us-ascii?Q?nQaxFiBWx6F5mtsU97zQ2sJmXG9n2vftBaRrX2Ec5sAM+LiYPJ7pwrhwP9xh?= =?us-ascii?Q?Tc9/GHFlaSJOz64InqsaMLXBokWTsmxlMRbFudJzQRM5Vtg+O6fI3u0JSNRA?= =?us-ascii?Q?qfrPb6CuyuPEOPBmsXuvYmNsg0pb1r0+OFcXMDS291mkGIgVhTmRS7c+a3Pl?= =?us-ascii?Q?U0io06p3MW3g1dWu7sGMBewuX5k8AlEAeo0tx9rXNYpHr7QzsQ9UksH3wqrZ?= =?us-ascii?Q?IeefUrBloEtxTvhtIBITlkBT7c7jNH0Og/0hjgpWjwF6tF6jKcagUbHK6es0?= =?us-ascii?Q?uLP4C+6tZ9GZmEcP5Fy+ziBTvdjgZW4pxqfPVx5+kBPA11ZkaazzSxI+TVWJ?= =?us-ascii?Q?8EgNpSMribSmjwImWfSuaB2QlLAj5xLch6hoWcwHi8iI+AeXL2zYH2v290wM?= =?us-ascii?Q?StPKZi3bZLHLYDgg2IVHlnBeccfi7pE+X6IeL8BnBiZnISjjK1yejCdossgZ?= =?us-ascii?Q?QRbhOJGVn0rdb42TqZcoTq7Vb3GiTGV04M7xrq886hwhJ6znGCnVNgi/NGQA?= =?us-ascii?Q?9uANlPd8fNdbQa0NTY9c8Z282rpPN5SHs02xAuTR?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: d1e185ac-1c58-450f-0612-08dda557d119 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:14.6889 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RqptReJnj9QmYcXxZua3AAyQJvShoe28EPMEiPfwQvfrxpxLR3kVBjtY4QihMhvrflpFMvkBtIylCbjYbYa7nw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Authority-Analysis: v=2.4 cv=cpSbk04i c=1 sm=1 tr=0 ts=684383a4 cx=c_pps a=Cb1kLmlYO8D/Z2wQFUco/w==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=u1fsugoMv0NfKDGiTLIA:9 X-Proofpoint-ORIG-GUID: grmvhtQ3h4tHs78IiYavJvXKQGq6R2AV X-Proofpoint-GUID: grmvhtQ3h4tHs78IiYavJvXKQGq6R2AV X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX0OCsQHBts3T/ BJfd6cU+nWzOy8g2d6eRyDDgcRNFdOb7+aPDMy+3bXAPJNKVu8rSIYMwzQbF489g2VM1tlqPobv Q0uQnnUMYB6zgqNGBWAEmoU7CJijvqadTJ6v5DuwZxLvNAPsiRx2B2CvNm7BldpKUTbOZhyRkDa Dlh2wkKgo9+AEzuJe54fgHdo/sZpPmsH0Ascxp7xB0NihL0x+U+XUJ5xVKvIcY5OHWcobAK5fIN JPL3uHqYRzkBP7hc/KtT4cgRRv891N3WWLAsdRe+Cn4dyGGvfA+GLPo2lyAPVY0uujuJx/IZILE j70QFL9W4Oi8Qx6P0BZVpyVLvIW3xqANGnvO2m6AdsMp7B8NfdXlE9hGNPIpRhBTyT+ijByOTy9 YzgYI98NePd2ktszPdIa6IW8UXH9oJs3mDObXc9a1eICDTpCvSGYsf/OnQtrbfxfVeS/GTw3 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255173082116600 Content-Type: text/plain; charset="utf-8" Add support for getting region info for vfio-user. As vfio-user has one fd per region, enable ->use_region_fds. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/device.h | 2 ++ hw/vfio-user/protocol.h | 14 ++++++++ hw/vfio-user/proxy.h | 1 + hw/vfio-user/device.c | 68 +++++++++++++++++++++++++++++++++++++++ hw/vfio-user/pci.c | 11 +++++++ hw/vfio-user/trace-events | 1 + 6 files changed, 97 insertions(+) diff --git a/hw/vfio-user/device.h b/hw/vfio-user/device.h index f27f26b479..a5a2aa9ba6 100644 --- a/hw/vfio-user/device.h +++ b/hw/vfio-user/device.h @@ -20,4 +20,6 @@ int vfio_user_get_device_info(VFIOUserProxy *proxy, struct vfio_device_info *info); =20 +extern VFIODeviceIOOps vfio_user_device_io_ops_sock; + #endif /* VFIO_USER_DEVICE_H */ diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 648badff46..1878d44c51 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -127,4 +127,18 @@ typedef struct { uint32_t num_irqs; } VFIOUserDeviceInfo; =20 +/* + * VFIO_USER_DEVICE_GET_REGION_INFO + * imported from struct vfio_region_info + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t index; + uint32_t cap_offset; + uint64_t size; + uint64_t offset; +} VFIOUserRegionInfo; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 9c6b2cdf35..e2cfaeead8 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -18,6 +18,7 @@ #include "qemu/queue.h" #include "qemu/sockets.h" #include "qemu/thread.h" +#include "hw/vfio/vfio-device.h" #include "hw/vfio-user/protocol.h" =20 typedef struct { diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index e081033ff1..57793e6b2b 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -48,3 +48,71 @@ int vfio_user_get_device_info(VFIOUserProxy *proxy, =20 return 0; } + +static int vfio_user_get_region_info(VFIOUserProxy *proxy, + struct vfio_region_info *info, + VFIOUserFDs *fds) +{ + g_autofree VFIOUserRegionInfo *msgp =3D NULL; + uint32_t size; + + /* data returned can be larger than vfio_region_info */ + if (info->argsz < sizeof(*info)) { + error_printf("vfio_user_get_region_info argsz too small\n"); + return -E2BIG; + } + if (fds !=3D NULL && fds->send_fds !=3D 0) { + error_printf("vfio_user_get_region_info can't send FDs\n"); + return -EINVAL; + } + + size =3D info->argsz + sizeof(VFIOUserHdr); + msgp =3D g_malloc0(size); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DEVICE_GET_REGION_INFO, + sizeof(*msgp), 0); + msgp->argsz =3D info->argsz; + msgp->index =3D info->index; + + vfio_user_send_wait(proxy, &msgp->hdr, fds, size); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + trace_vfio_user_get_region_info(msgp->index, msgp->flags, msgp->size); + + memcpy(info, &msgp->argsz, info->argsz); + return 0; +} + + +static int vfio_user_device_io_get_region_info(VFIODevice *vbasedev, + struct vfio_region_info *in= fo, + int *fd) +{ + VFIOUserFDs fds =3D { 0, 1, fd}; + int ret; + + if (info->index > vbasedev->num_regions) { + return -EINVAL; + } + + ret =3D vfio_user_get_region_info(vbasedev->proxy, info, &fds); + if (ret) { + return ret; + } + + /* cap_offset in valid area */ + if ((info->flags & VFIO_REGION_INFO_FLAG_CAPS) && + (info->cap_offset < sizeof(*info) || info->cap_offset > info->args= z)) { + return -EINVAL; + } + + return 0; +} + +/* + * Socket-based io_ops + */ +VFIODeviceIOOps vfio_user_device_io_ops_sock =3D { + .get_region_info =3D vfio_user_device_io_get_region_info, +}; diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index bcef0ea341..c2e7283489 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -14,6 +14,7 @@ =20 #include "hw/qdev-properties.h" #include "hw/vfio/pci.h" +#include "hw/vfio-user/device.h" #include "hw/vfio-user/proxy.h" =20 #define TYPE_VFIO_USER_PCI "vfio-user-pci" @@ -101,11 +102,21 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Er= ror **errp) =20 vbasedev->name =3D g_strdup_printf("VFIO user <%s>", udev->sock_name); =20 + /* + * Use socket-based device I/O instead of vfio kernel driver. + */ + vbasedev->io_ops =3D &vfio_user_device_io_ops_sock; + /* * vfio-user devices are effectively mdevs (don't use a host iommu). */ vbasedev->mdev =3D true; =20 + /* + * Enable per-region fds. + */ + vbasedev->use_region_fds =3D true; + as =3D pci_device_iommu_address_space(pdev); if (!vfio_device_attach_by_iommu_type(TYPE_VFIO_IOMMU_USER, vbasedev->name, vbasedev, diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 6b06a3ed82..1860430301 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -7,3 +7,4 @@ vfio_user_recv_request(uint16_t cmd) " command 0x%x" vfio_user_send_write(uint16_t id, int wrote) " id 0x%x wrote 0x%x" vfio_user_version(uint16_t major, uint16_t minor, const char *caps) " majo= r %d minor %d caps: %s" vfio_user_get_info(uint32_t nregions, uint32_t nirqs) " #regions %d #irqs = %d" +vfio_user_get_region_info(uint32_t index, uint32_t flags, uint64_t size) "= index %d flags 0x%x size 0x%"PRIx64 --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255154; cv=pass; d=zohomail.com; s=zohoarc; b=EA7uPItGjcVncvE33l5ZE72GzmYSNHgz3HyrYs1CK8AadIZ/DqeIDCPlyZwDUdvKQx7NNTMS+GSYQO1sy4OF6SkQnzy/9dRok0TFz+s+lYhb4R7gyUCB9gGwMfg9vm5qHGb4u6CJYn8ZlLg7hGWLMCpTonh6z23EKuh715h8cI8= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255154; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=zxCsFErulNbPOxCKbF3UavilB4wcwM+IwtZ/BHwWHH4=; b=KdCGC1YAoRHBG50Q9lesmQUn+HjfnpctYQWcShF7DD/zXJk7RlNQSpjYamsczjBJUG7253822o+WmDQ7YK5yZwCJIyE1U+6FK0dq8sPF84jPjLYl5t0UmNCN8898MGYH7Y+RZ3v1GSaNuaQodj6omPho0x83bLfMAsy7Qb01weg= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255154360205.75359000975425; Fri, 6 Jun 2025 17:12:34 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9k-00031w-8a; Fri, 06 Jun 2025 20:11:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9j-00031a-46 for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:27 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9c-0006Q0-CW for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:26 -0400 Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556Kvu6n017760; Fri, 6 Jun 2025 17:11:19 -0700 Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12on2137.outbound.protection.outlook.com [40.107.237.137]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 4747u3r8fj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:18 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:16 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:16 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=zxCsFErulNbPOxCKbF3UavilB4wcwM+IwtZ/BHwWH H4=; b=G2ChusJdhmH0nBqPeaEYmPiuHtvmn5sVQs5UNdQr7vzyG5lmntkIzFdNu QnExma9kEGCllsHEpwoYzh+6D1qYJu1KM/bYQebz0Bm5+nyChbzWGFCMYJYCBTmz Viml7hX97AaoSAFvQ3p6i527xWkP9T2nWGPw3k8AE63rdHyp74hKgdQXE6eabu19 gH4Kkbb4o2CJmx8YUETnkXDU9wgn2QbVcP2omOZCTpX2ICHSZYLp2c7mFvHP0GAq vN9leU+RMSk9Ia/lrOqPfVhLmwVrL/T8f7p64fMs9b8JIIN2UaEiK7kcMX38B+Za EIZkXvIhJPrgHJ/W53D8rPZfppJaQ== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=nJvACoM3BZW3/WXV3EfQRJrop0Tf1gvv4Vw60i6G+jA89gk/Vgldn8wIYglYXZn5jVXl/IGJqjBYnV6ONvqKfpS83VEK1gG6ZbQZa3BCyjHppSlu94PJjHMrSUAgekCqeCmUp7WXUCQzVevH5T+6oVRjQ266VPGI5+bfRmEvpEFCJZF9C+9KSf44arST4wCHEX3eD7wDuOj2d2p4qDSgMpbg4ze0Arh6jUzVlpeK9zout18MUCoYlW4IWYZzJO287YeV2xIzOj9tl9bvZs3VdVXDJ6OlpDslFr7NhjvyhF5wxBThIDF8or5H5NCgZnoIdWw7WZd+Auz5ZUI4Lvvm3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zxCsFErulNbPOxCKbF3UavilB4wcwM+IwtZ/BHwWHH4=; b=YhpUhV9Ff4YUOj5o0PoOFgFdE3qSARVxJ1bGdNKQKbI2vewqgjn3CZTacfarIcNRQX14nZUcDPRA8gQxzzNDjBwspDmajgEgzV+ZoOJBCvJSibwR0MFVoBBz78OnQLQtfsYz4VG2d2bHGGIGihITmqRiDhxzuhI8pbAmLw+G5+a4IfvwPIpS6WtGexL9NS3c4X0pYDQCVtK4DHuBi4up8IEsGOWbLljooekHUIYcd8xcnqKcj2lbPeS78VDP/XgdnKdIIu/j04E3ra0qGfBWbyS5JH8K+OuO/wmqkc3X+SJdToXYmR6kpYescS/LA3buna5JTR1rsQmZ2I06RM3JWQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zxCsFErulNbPOxCKbF3UavilB4wcwM+IwtZ/BHwWHH4=; b=Js5/7HqYShwXjneNtd826PZRaX8KnnSTK0pXw3I39NuksPk9ZDyASfZBkJBdlaCDaa+jW3mCBjFPA+boAWvJxXReL27nGbGwU1KMYPSScC4rZAmSu9iaOWuTSG2OAgB3uaapO8dLPbaKylOUUw8gCccbWe7tT3vcvHewqTZWbSC/D/8RXBcSrFe+OTjkJ3/5q2cT0j8r9oO3QC9fbvTnuQllitnJxGX8GhMDDO49HNbglWfoDEfMct06FfIsAqZYw8CqsERE33hb5t/vUa+EVMl6wxB9h7ih34EAJdMwu6DSxMXu1ulepbzJQqNuY/Lstk62cn5nEAh/L9Q/kHSQJg== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 12/23] vfio-user: implement VFIO_USER_REGION_READ/WRITE Date: Fri, 6 Jun 2025 17:10:44 -0700 Message-ID: <20250607001056.335310-13-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: 143caafb-a789-4d10-fd15-08dda557d1f1 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?T6JBd8TR9BFYjHwnptSlP07vsIEeO1KDsf2c69bEVgfKPOSmEjbIv2gt5xcF?= =?us-ascii?Q?GSbd1ZG4HIemgU95HaqCSHFh0NfdPPDDK55QuuDsB0m/64YKPIjJsjpkqHki?= =?us-ascii?Q?7s/haACjEbfTSYEX7uKqI4ffat1EyUN/pf+kCRii2YR8ZwjF+DSTjs0WN6f7?= =?us-ascii?Q?YRKy8F9eXSPQ+4prdv9wjZ7T+n0K4puIM8K5w7w28NauMW8/j848tpqarvUo?= =?us-ascii?Q?pd8aY5IUUxTXqXQ5e92ZHjsTE1draDncwpeXOlxL2hH7A/IfbIw5VZRWpPQF?= =?us-ascii?Q?GXFaT667xd1SikqKnQSPFMbqVrsBrUDv+tiRvCwk8JXBGGEZ/eWLNpcDyHYj?= =?us-ascii?Q?PeCauWVQ8oD+gk5HTPlE9j7pkGDficINTC1SK7qnkvCKoD3jIdFl1UlWQs+d?= =?us-ascii?Q?I1QDBi6rNScdQRx7u00VUrHck3nyYk2hkUW1flkmd4wzp9zXWwp4ok3aswW+?= =?us-ascii?Q?VDw7MWxbVTyCjD55uSkkIXy5st8CEyvTRqHME3ATbruFyTUxgZhi9lcFJ/a4?= =?us-ascii?Q?asMOVEBuGBrjp0vVmAJUZXsGMiEhr0oitCGM4/WQd/JTmuNxr62UMhl6wuRR?= =?us-ascii?Q?lnX6nETF1bIGlHKMxBcZn+QQV/j1jzpxjLAHfJ8Le31p3MTASPjfyED9S3aO?= =?us-ascii?Q?wfezAHnaZy79V9ZDF8g4NvbDZvQllQdZU5t5qKlZlucULqxRInfgHmSLDq5I?= =?us-ascii?Q?cW/Oz5Qk1poOG13f9quH72/6zO2zk7arSdFUQwmNEYKjAj3OYIgBTHX5xBPz?= =?us-ascii?Q?5R14+yY1GjpRZlnGE39cF2tSAbxFX5eqENc1nkFq2EFL/9wYJ4djW8urdzuH?= =?us-ascii?Q?wLlQ3H0L+2wxAoJYaYZJlj2Vh04osZ5oROMbpFSeazP16yGicBHGDQHM4f51?= =?us-ascii?Q?5OmJxRNYKxB+j9FLUYCNtw6tgUdLiDicRvBj4ixizWafordJpyI2UqsPCnh8?= =?us-ascii?Q?NrADEFGMeuCMBGKo15YZmLy7AshGUtT89IoSItEpPzvviMyQM+1LQfn64xJo?= =?us-ascii?Q?ypcC3hBtbwx7sed4VRtnULUt9QeonUxfki24OekJbjMVAhT1h5hkuYgstfbz?= =?us-ascii?Q?tO76f66fWQFXyoXXrq3o4K+WAINyPw51pT4+Mh8bH1ryA0qFNSWHmapyJFCf?= =?us-ascii?Q?mQGWVgCD1Gd4pG45i62x9rSpEfs+txKFfLPDYHOPKVv5m2TWe5BWpEswQ3Yu?= =?us-ascii?Q?GvPhrKE3y8w52yJvtgw5KWipq8NmKKVVgqM8dsQ5mm1m9kmgw5Dk/yFhyYhn?= =?us-ascii?Q?3r6MSrI6TcfF3YI8iZ/wkEiRcAqFOHwKVZedao2DzrT8m5URUsjkZMgN82YF?= =?us-ascii?Q?stYN3Bv1CNHMazl5rZPi9LmD5ZR/WwKdN1lqJii8ay0v25L0aEimNe/mBhJ0?= =?us-ascii?Q?uXEjfHtiJdgF6pWp+cGjoHwrLJCv+FRQmiaK5J8k5m8CEXdI19wo2p5RGrin?= =?us-ascii?Q?h2Kq7rc9kEs=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?kWOgkL9ZSJJFHMjQJ1Eggz2k3jF43CG47Zo736xfNgw4X5dA6aNnn8grZaCe?= =?us-ascii?Q?ga224wrZ1TzAgNiKo8WNs3Y23xuVoBDM8/hSqORFxEtZg5QuxDmw4PT+QbeC?= =?us-ascii?Q?RjYQdcRwQsgXt8v15DqzEx02cCswEoliW5220N04EkxgywNbm+nmND2xWiyB?= =?us-ascii?Q?alC2zxPAN/QVXyd1hJcPUPPQzbjMQRUJiHXVx3IC6ykWhp0RqQlojpuSSSWb?= =?us-ascii?Q?8wuj5xPkQd67aoQ0wmJeHJhY0WCPWixutsgMGGfau7ogDVH3FoSML4Sk+4ua?= =?us-ascii?Q?fdr2yBrq5EGdV3VmYHNOg9QZj2YUdjAsjjMzfRdzVKUxJv6dij0R1OuK694X?= =?us-ascii?Q?kZsuuBNQaBWGvGJ58pTZZNaYkcLz+pVywu/0yMC1jkMU7xhbjr64U36KD5vT?= =?us-ascii?Q?C6LT86ywP+jpWHVZMzQfxSc2aAbNogtmGn/4daJ2nX3QgzZNqKJuzxLOFQ1J?= =?us-ascii?Q?t2RK32lZVM1n2xSN0+1Zh3+6B3c9zySA7jhGiuy2MyO0BwRtOTsFQbY3TtU3?= =?us-ascii?Q?Zt9x4iF4vEAM16tPZpFY4U2DWwFUviHcV4EoaMJI1z0CNOWgjASQ1ERj/68e?= =?us-ascii?Q?eAghZvK5oGh3ZZ2YN/m6ackpGN6vcVce+svQuuCaa7vgJvD2oDxPeIwNgQXe?= =?us-ascii?Q?/QrEPW66xWN/guaITNynGICoBGPylWWq7B8aO+1FtieBH3C8tLr5g53Wl1fe?= =?us-ascii?Q?ncgMpNG951RZxikcRrEfO9MFToEP4j3ZPqX83kossw+0tVHXEcn6wCEw9p9H?= =?us-ascii?Q?chtIeenyX3IwX8RBAobQNPa32SO0LUSt27Lm1eExsTNpGYom6yvKU37+V72f?= =?us-ascii?Q?1g5bYpPqKn4ugnwLVZyy6wp69cueg+V1pz8J8HSgjo6W8tUJEWhGGY9NaUsZ?= =?us-ascii?Q?alSmbTyoAALw2fs0xWcoEGwbiASNemJolEAcoXUmF9lDAo4lT8PZF4n8zBRA?= =?us-ascii?Q?6D3paQpycWdKNmD+1cyT7dwiG5ObuIt8t8NlCgk04N6o8zlBPeV9RXNMRjMm?= =?us-ascii?Q?oJJCVfpZfhzQcAntX45llREhButds8NXkcIM6j15esqWOPK9mbBci1ccOiNc?= =?us-ascii?Q?RD55eqRB5faj/gVvliX+rzr3Xn7lehR4rLUCIsI6E+BRaolWOJ1bV+vcHioY?= =?us-ascii?Q?PufdIov+t2nO2LwsPdKG6sDxk6GFMRTn9B+Rtvz/fL4+K7JGw/ZSe6mI5HZh?= =?us-ascii?Q?a57tef3TaojYvi7cf0yHKrPARS9es+TOMkekXY5O4Qq+HQ7q2x8DFsFBEnSK?= =?us-ascii?Q?mBMvBNPbkXp5rdztuavyMkwCdtIJaJPUq83KeU3qA3gC+bRwYgGyvhxrB9qW?= =?us-ascii?Q?J93r/x2aSQVpQXu/14QaKwt5rTDO0JhZInhCkkOqi/aUtuwOZBJgXffFl4pR?= =?us-ascii?Q?ccA54G4zN2JRb+/5ZU3xWUFupIs3ExgY3/zHlALRY3tSUpLfXXTW7ApGLy8X?= =?us-ascii?Q?dKx2XmsA/eS928ntzGKZf5W7LH67lPJsM3PeP1nygA2ykgoh2NeKnU9bLvEO?= =?us-ascii?Q?vnLgUMf17JQKuABzKTA0D1zCCTCRUdEegY2aQwfMyq+fSgnRLI5vwFGbWQ4c?= =?us-ascii?Q?t+mnutcCy6CwNEukru2SZb6q1oXPqoQVEW2Pmbtr?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 143caafb-a789-4d10-fd15-08dda557d1f1 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:16.1033 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2n4ed4thEmDXFGsBMfBheVsybfazmgCWlfASrxqk1sJJIfiPhH9Ozo9ZfnOpWsdljRt01gdxzBWkgUWwwKdcbQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Authority-Analysis: v=2.4 cv=cpSbk04i c=1 sm=1 tr=0 ts=684383a7 cx=c_pps a=kojefl/8vcI7rOD5WQpBNw==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=lEmlVuBv_BsZ2DPd9E8A:9 X-Proofpoint-ORIG-GUID: TDq9rpSTV7F_B5DjjWccUbT5SuCzfjVt X-Proofpoint-GUID: TDq9rpSTV7F_B5DjjWccUbT5SuCzfjVt X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX5cv4JKSMVuJC HX2ARcyz/DUkP23ioXneAk/t3mi8xaqHFXxe4y5usXvi0D8WqLCw7TAWwYWzV0qfTlyiX3rrR6E 6jULGxfJAwDZ/xslepqQgST/mVRSu7YJuI1RJyAXIV4hiJhTR3/gjdy+7ZqIE5l5B3WkyfyiU8P 35au9Dm+kvDeDjVyjgx72VOAnNJWfJg422Co0/8cdh/8ZeconTJ1EUvXxzUlD6VYJwz+XbDG+W4 4idePTPwdDJm4xpV8oXXP4yPSmTjKbYCCEWJ3lJGYtKUk+wqSBhJcTzaQOR+0mIrvU7Imsg3y3b Q/47oYuuz5LgUFXVVOoiyYOHzbOUtGk+7Em7zZ/x6y2gs3vGz4vedIIbu4wcFoOgAZlv3IUHia8 vt6Y9bOoSuvJmjH0adN56+PPjuQVBIwXuNeF4L8rnuu7KFRfiR27H/4kur88Heqy5qEx0lol X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255156402116600 Content-Type: text/plain; charset="utf-8" Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/protocol.h | 12 +++++++ hw/vfio-user/device.c | 67 +++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 1 + 3 files changed, 80 insertions(+) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 1878d44c51..8e6ccbe9e8 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -141,4 +141,16 @@ typedef struct { uint64_t offset; } VFIOUserRegionInfo; =20 +/* + * VFIO_USER_REGION_READ + * VFIO_USER_REGION_WRITE + */ +typedef struct { + VFIOUserHdr hdr; + uint64_t offset; + uint32_t region; + uint32_t count; + char data[]; +} VFIOUserRegionRW; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index 57793e6b2b..a767e4fb01 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -110,9 +110,76 @@ static int vfio_user_device_io_get_region_info(VFIODev= ice *vbasedev, return 0; } =20 +static int vfio_user_device_io_region_read(VFIODevice *vbasedev, uint8_t i= ndex, + off_t off, uint32_t count, + void *data) +{ + g_autofree VFIOUserRegionRW *msgp =3D NULL; + VFIOUserProxy *proxy =3D vbasedev->proxy; + int size =3D sizeof(*msgp) + count; + + if (count > proxy->max_xfer_size) { + return -EINVAL; + } + + msgp =3D g_malloc0(size); + vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_READ, sizeof(*msgp)= , 0); + msgp->offset =3D off; + msgp->region =3D index; + msgp->count =3D count; + trace_vfio_user_region_rw(msgp->region, msgp->offset, msgp->count); + + vfio_user_send_wait(proxy, &msgp->hdr, NULL, size); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } else if (msgp->count > count) { + return -E2BIG; + } else { + memcpy(data, &msgp->data, msgp->count); + } + + return msgp->count; +} + +static int vfio_user_device_io_region_write(VFIODevice *vbasedev, uint8_t = index, + off_t off, unsigned count, + void *data, bool post) +{ + VFIOUserRegionRW *msgp =3D NULL; + VFIOUserProxy *proxy =3D vbasedev->proxy; + int size =3D sizeof(*msgp) + count; + int ret; + + if (count > proxy->max_xfer_size) { + return -EINVAL; + } + + msgp =3D g_malloc0(size); + vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_WRITE, size, 0); + msgp->offset =3D off; + msgp->region =3D index; + msgp->count =3D count; + memcpy(&msgp->data, data, count); + trace_vfio_user_region_rw(msgp->region, msgp->offset, msgp->count); + + /* Ignore post: all writes are synchronous/non-posted. */ + vfio_user_send_wait(proxy, &msgp->hdr, NULL, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + ret =3D -msgp->hdr.error_reply; + } else { + ret =3D count; + } + + g_free(msgp); + return ret; +} + /* * Socket-based io_ops */ VFIODeviceIOOps vfio_user_device_io_ops_sock =3D { .get_region_info =3D vfio_user_device_io_get_region_info, + .region_read =3D vfio_user_device_io_region_read, + .region_write =3D vfio_user_device_io_region_write, + }; diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 1860430301..3f5aebe7ac 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -8,3 +8,4 @@ vfio_user_send_write(uint16_t id, int wrote) " id 0x%x wrot= e 0x%x" vfio_user_version(uint16_t major, uint16_t minor, const char *caps) " majo= r %d minor %d caps: %s" vfio_user_get_info(uint32_t nregions, uint32_t nirqs) " #regions %d #irqs = %d" vfio_user_get_region_info(uint32_t index, uint32_t flags, uint64_t size) "= index %d flags 0x%x size 0x%"PRIx64 +vfio_user_region_rw(uint32_t region, uint64_t off, uint32_t count) " regio= n %d offset 0x%"PRIx64" count %d" --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255111; cv=pass; d=zohomail.com; s=zohoarc; b=MPaTG69TSQobV369DR4kEQ49+ZppWvsSP1EVNskOc8rpc/2FznIYXGTMw2vWhNhHUS11YdlWdRG5JGOwMTEy7qxrx/QuFfoeITQLZAXlVA6EX76Q+Xa44D0UzbNdE3p2Gu9Fh2RL2R+KVd+GFYWkBo+OVFxveViZXZLbjlFYvRI= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255111; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=f5mqVllGtPuPrRQDhSXEAGDYfQ9/kcL/55Za2ANYWTo=; b=hOOcuK78lwVO0USH595dzTsDL/arAWG/eKSosMTvnHQB1Sq6OCZu6/VVaZZ2r2g8ARPOpn5y56QQLIOYL9mse888unVrys34sVDBLWB9H1I5isKBQNuUpN1MFFXVrMUtgDiTXNtTZTPGWav4L2XJyI++woCezrG6Xo8tm+A2uKg= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255111743237.00754966665306; Fri, 6 Jun 2025 17:11:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9g-000316-TT; Fri, 06 Jun 2025 20:11:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9e-00030U-RC for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:22 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9d-0006QB-3O for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:22 -0400 Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556Kvu6o017760; Fri, 6 Jun 2025 17:11:19 -0700 Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12on2137.outbound.protection.outlook.com [40.107.237.137]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 4747u3r8fj-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:19 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:17 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:17 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=f5mqVllGtPuPrRQDhSXEAGDYfQ9/kcL/55Za2ANYW To=; b=XI1GajnZnFT/BToSgFB7k9f6C9qROZ7iX5NwdzrRERluPctdsraWj/ko/ timwYojME5Pax/91asT/uFQjL7GcWa5Tf0tKYl0M+ljx/o2dGN/+/WJOBCPu28kr kCWVS9lMCkavSIynUiEVVwYthk66mT/eE4+W/s0s8LEy4qJHu7QryBir9Mq8NTg9 qtK0uWlzwSrJBJ58sQzBRZHtek96ldafN5IgelAiPhnEiCNoSyMkSvCqWNcEXCUq 9G4jsylqfRWxSgGiRVTVAvbrsTVdqgnnkmzxe1TBvgtJaW7hF05jJGWlwgK1/8+U PYr23mIzHiwvj4R/yum8qViTgSTsw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=J+MwpVAaWRUqQ577GHjbFlJgs7zCEbFpSlaQiz1bzavYmQXAO8bxiHT12dEjoLEEV2cWTUzbBgBNSMf3MWUWjaCNY0daOBfhVejpG1hOR6/WSGfDU2H/dLkbInsw/qUlvugKPRHdDFUoz1HSZkoql2jhgzoCRrbveEi0FUwMS5VjO+rPDU+/aCneU5YueJ9BmLGzh5Gl7N5jcfttFd7PCLxpmtWYvy5chpoHYL4F+JyRR4tNHl/T/SHu0t7KUnYd61R2f3Ap6Ew8T8KPjJ4T2GNsi3GFi6gmz2R8DYBOS6zAWrB+r1m51fyWOJ5URUjM7Pf1qS8mnrYBm+KtuIJw8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=f5mqVllGtPuPrRQDhSXEAGDYfQ9/kcL/55Za2ANYWTo=; b=FkRLRwP3PfgxdVr3+NyGa8Ql7qPAuij9CPhLvaztX6Je2v13CgOmtCE0suLGD18HjvSShSY60t1vJr0oclC4ynQoyqufWnCqmSEszVyJN+CwPb6vdaVf4XMaPsc773A/hNtzHL6iNy/vYv64vIQ6GV/ciZinjkbeJLEwcVNgW/1Cu/r7Pb8sGt2lsOFMX1Qk5BEK5x/q8slg0bbhDzXj9TjM+7SNbv4T0VwVp/Yn+AEWWAqx1j8YhfKTgZiT2ecHUv22l91XSCPQ3erprIA1zXciko9Lp+Xni+AeLzabHb24TKBD5kEi6qiVKzlfBcbJLqvgDX9hZDDZaRYFP5B6ZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f5mqVllGtPuPrRQDhSXEAGDYfQ9/kcL/55Za2ANYWTo=; b=o7zfW/IvumeW4nhkJQUZjTFPzgajKnz1GtgUgexNjDvYPfW4OOtpCZ5ikOZ3JQXbo0PewkT0hHvcP0CdZvF5Aob3jM71vpVDGHbfM8JcZZ51Ox0zQWCg0gGI0nv/Mbv8ZE6q+BtkvLCAL5IoBi5iu4UiP8HSuBRKEioIuPqhyPgIZzpzuj+VBKRbzAo2qjSLoYOU0Xknz4JjzLCvQokPMjkfqNcJ6fqpea59ImvmHBPVAsPIj4OjHCsAecph6oV/ktLPLXcgA/b/Bb4q/sS1AX0t0kYkuew18d6RY1T3GVyi134hzDxUrGP6yZTV0SmK02ZrZdU2Q80uHiDxK1Ri0Q== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 13/23] vfio-user: set up PCI in vfio_user_pci_realize() Date: Fri, 6 Jun 2025 17:10:45 -0700 Message-ID: <20250607001056.335310-14-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: 7633b914-a42c-4c32-4ff4-08dda557d2c9 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?W+OPb4jc5fi7qcAgxf+5pfdQJsgvzNTN1Y0tQrESJo9b+d4k1IX6jKxcAPnZ?= =?us-ascii?Q?XErKkERxzoD0axiqqeZPc87Jc8ZESn6nWxpeYSi+xDEzYvbODQeQf1jzMksC?= =?us-ascii?Q?S6qqEJw1+fKr7Px2G12RG3dMy4R00NjzlCX82cRS5PsEJTiMWVlzSH+5xgIu?= =?us-ascii?Q?TOblncGzhTa4GT8n0ajYC6yLujk40fUQBP6w8foNEKx9S4MbRTE4QQ6+53GS?= =?us-ascii?Q?45kiwGZbRXe0dP1pZ4gAHqdkyjPUOyG3XDrHUcaA6RiUq0BHEfjE55bWMSc7?= =?us-ascii?Q?A/2+gwp0tG2UuZdNfw5zaRkxJ9EqwQ5c1HkiM++0dhhuw5vqfA/UfrBMxLtN?= =?us-ascii?Q?p9U63s0IpMSxL9sE7YMqjQIs1FtwgX90RQNqJaQZWhFa1qaXYx0tk+DVe1Nw?= =?us-ascii?Q?ani3uPgjiF6cEE4o0TpSTD2QqpYiS+a+1Xf2mlllcLSmGIdFujEijF+wmqkR?= =?us-ascii?Q?IBscMjpAmivSnr7SNbBqExOzeVRrThi50rrcvwjbEiPTTwsxzrWjCe9fbHia?= =?us-ascii?Q?2zKtY2TTACROpRXOUlbNOzeRJyLuU2J8penBDKLQccu7lef8ncd84NuTbHK9?= =?us-ascii?Q?5cRdihzsZ2hu8l4cjDdxCLhs81hBMjkwQpPfaJ0ArG56B3bhdl1U09HV8/Sy?= =?us-ascii?Q?Hc0EFiIoqqjGaniWAnj8h3qxwWmIt+JQrlbPNuxgGIJW3IEr1FdfjXkYG1/W?= =?us-ascii?Q?e66O0OC3hQgqXlJ8D1AidoHTDJaanwg6rlbgPA24xsfogkHtvIxEk88PjMcy?= =?us-ascii?Q?zhEIHPhhKC69iPvFWe0pkdJGtLup1eUtPGlfwacbnIN2e0OQaLzS8g0eYupi?= =?us-ascii?Q?g9fd1q3ZRg2J99iRIRwmpTw0GWkm/+mKnzlim7l4uQ+KueFyNjxJot1WlPZy?= =?us-ascii?Q?RBw2oUAQs/0xr5oJr5bspMwMHF1KqhU/GTFqbf1CdHuaCxxJJygPz4cmGWuz?= =?us-ascii?Q?4b2kq4IpK0su2fj3WXr8H6fXZyLeLfKyt8GCZQspnU+OUNpvIgvwmvbYs8od?= =?us-ascii?Q?xIH/sivDGZMGucAsV76TwdcvST118Pxq8wdX9xWLh8o54B7UwPqtAoRt2CBE?= =?us-ascii?Q?i83zcfzoPjZT4G6PudfhzZR7dXy93Ztv8AXz+he+cmH4mr9sH+Y9R1oPmZdA?= =?us-ascii?Q?NByjnrmG23pFQPLum/g/oMfqyymDn2HmrITmxE3D7JComnJHDS36CI10t/6o?= =?us-ascii?Q?auGptax5zzwM1PxaZeBC4oBNo8aRVd5lBkreQT1S70uTXh/N5ZjBBT1uhH/c?= =?us-ascii?Q?zRyEPBQB/5MU5mhwhy02Kn4k3BxKuSVimEicJOzF5H1TjT1Qys6jig6cTxEi?= =?us-ascii?Q?mcFZnLkxyrioaTch3qKGbscSxSdQujfxsFySWMNgOySm1jaTq/aQLzsgrVYZ?= =?us-ascii?Q?ZsKjTi+SzRIs20XfAspF2iyg2G/VDLfkpQgLLPJri48TF2eHZbL/SDzu5pXd?= =?us-ascii?Q?MUvLu06o7pE=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?uuAs1LoArNaRSqgYiL5x03TTBXhaMjuPa2apfdzyhtnThbf0pxrVPQ7vqA/y?= =?us-ascii?Q?HoNRoqdSsuh7NrfTkua9d6fZKAp2zGuQzae0bIhBcKr/dVYckXE3QPJNpm4A?= =?us-ascii?Q?uUW+L5QjqWHhrZ/FAyUjpeX7XwZCd+fcVq/PuSqdy1lfMvfWeEcCxcILFIiU?= =?us-ascii?Q?nNR40W4VZPqgUYVKUqll9KXaXQ0goJlddEvDh4xwWHcfBbOFZOuRxswqSgf1?= =?us-ascii?Q?fBzxt8FCamPqyPx4Tgp5PEBWOoAzC1TL24HnZLC4QH1sLrl+aJxr/a3L4Xdg?= =?us-ascii?Q?8W9eVnPo0LKpYnIyM5gRzbSDm2+KsKDKOr7mZNk5j5VjIRVt93aNkxR50Wr6?= =?us-ascii?Q?T5fNCYvTgaZ0EpXT2ekNAUjTXuyDN38ZYdxNTVsBBT94TOgnIOA+qOTEYc9z?= =?us-ascii?Q?ugr9dZSPpiid5qSg16ITc5NZTG5cmpPApX8XDcDCrslWkDkpBPM7he4O1PT2?= =?us-ascii?Q?n+6qp6pk5OqdWnnU9DEmrYvDSoQ2ZqX3QW0eFbnLE17sxfEl1oyKvSLSyyoA?= =?us-ascii?Q?tA5I3djNTiHiAuNmbjmafCT1Ewv7+DL9HFivUFeWKUPDfW8UaLDuCO/vpXrp?= =?us-ascii?Q?OKEheNEccliiDnXlcB5HA64sXH1rGMeUq7LTGXAI+9gGqBVGQ2JUJ5fpgXpa?= =?us-ascii?Q?U/j4r5XlnH9E3IWv1xZX5Q5HVr8ZZRTIF5U3Q+17dhcnrSlGG7ZW45boJur4?= =?us-ascii?Q?VAJrPc/jYFVQixv86uZTV3Q98X8RjPNHgOanucpPsJcIeZDO4kdOmqj7RPV9?= =?us-ascii?Q?yakWmq3p92UaF+AEEbeq20TVJoWhP1s2kbu44iDFPUvivP3qPdv5/QLMfR6f?= =?us-ascii?Q?v4K9bL8i7I41xymQsEFiE6DRuA8Q3sBszTCIpqL78ACmoi0w2YvbSNvx8a/3?= =?us-ascii?Q?4KMO1ez1Uh7yizxxHMeYOPgXb+3JGZB0+qJXDG9UfgQjQKgV7TvzwtKmt2/N?= =?us-ascii?Q?uQWiRLZjQfvZzP36DndhX6fcStwmMg1t8ngrUKq8x6RFye7J4x03lDC+dJZl?= =?us-ascii?Q?/ng35vHIyx5Koq6FOtPGFnXt4tqUacC4AOqArTBdwovATDXB6QlAPQY0tRGW?= =?us-ascii?Q?chj6qspDNfXZHdKErtPh5XzQyH+L3txj07AgLhb8ZlYXxH63+Gv6c2AMHxMN?= =?us-ascii?Q?zhRZwe6yeNKhsp6v0klq0bjPDQa6tG4VdlBg67ZL8ao3c8xD+/zKseWdD4Y8?= =?us-ascii?Q?mSIyZ8gRcrJ7736gYug8jCm15JAmKWbyso/SU4erARzruJgDN5ErXylo2nfe?= =?us-ascii?Q?Xruwpt5yed/us+uSxNH2RQuXxQWZ2f9KjIm9GnyAdVDYBGrzSYlHmQe8f3Zn?= =?us-ascii?Q?gPqlF1cz48AY7EQ5Mwfas15iXMeRhwxz+IWs1Bh82iZ7OdjeV7rTbb2wrpzg?= =?us-ascii?Q?hDVW5x1f1fomAoDtsN15DyeOXKkhFFZYlRmLIAbCFTQZ3pvjqSlyo+foKo2m?= =?us-ascii?Q?ScQ9Heih8L8fd08jXCku8+jQWsDE6aPtSuLrhAYARwkp7OxZtJ1UNXbk1k71?= =?us-ascii?Q?O1zhv1FatmkQzLSRNfbRkJMex52XTBxmh2nXYlJ2sYV2MAy0H5BM93J7nYzc?= =?us-ascii?Q?T3ce3aZI8HYVKhUtHNGQBq16XmOfaCA6Uv7/ZJdi?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7633b914-a42c-4c32-4ff4-08dda557d2c9 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:17.5076 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qSAfJzuPN4RvoGL9Ik5xQcrEJOZHYqBwJ46xZPjipSRAe3cl+SypqT4DqVE/o4AThsWHJMnA1w0nBel9ykWBVg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Authority-Analysis: v=2.4 cv=cpSbk04i c=1 sm=1 tr=0 ts=684383a7 cx=c_pps a=kojefl/8vcI7rOD5WQpBNw==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=uN1uQ0dK97G9gywtELIA:9 X-Proofpoint-ORIG-GUID: X9xdfSyrXr8hrlwNdDydDdzdrfUFhEVc X-Proofpoint-GUID: X9xdfSyrXr8hrlwNdDydDdzdrfUFhEVc X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX80DFTXUcf4im NwkHm0ko+XwJaMWVi4OwKtar9Gkt67lkl9Qz46R9BnUWsRBX7UO/7WTTu8Y3lImsfiaTTaQVgB9 0JIQTZgs4fcT7C4lFKNhdKs/TyduQAwmi/nbqvxlfVuI77vc+6SVdHC2aOJC8unKzz5SIWRS33V nrwi/vDqT4v0WIPRtSMenGti+g79kLtBy7NGDbd5nfFfLswysUFl/hmaOpMS4MdaRwvFkCcyEf3 tb3oJqRMBebOrU7Dy3dIdylS1ilXH7iEB6gUwhYs1SD+mvGpdWXoLrsmqyvVI/GpgMw4PzmzFBS 2XFZsfbs1Af02XfUMqQtC+2UmPgXrhF0i4rh5LAGY4YEbCh6d+Xvv9YX5lmwS4hMSfhKWB1r8Uq hRtj3EFunfIeV20Cuv0Mji2uVm0IHWGnhsF+fFz5WtDR5qUBR8nNH9sq5Rd2PYs3GQOs5TqU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255114101116600 Content-Type: text/plain; charset="utf-8" Re-use PCI setup functions from hw/vfio/pci.c to realize the vfio-user PCI device. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/pci.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index c2e7283489..5ac3fb087f 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -124,10 +124,39 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Er= ror **errp) goto error; } =20 + if (!vfio_pci_populate_device(vdev, errp)) { + goto error; + } + + if (!vfio_pci_config_setup(vdev, errp)) { + goto error; + } + + /* + * vfio_pci_config_setup will have registered the device's BARs + * and setup any MSIX BARs, so errors after it succeeds must + * use out_teardown + */ + + if (!vfio_pci_add_capabilities(vdev, errp)) { + goto out_teardown; + } + + if (!vfio_pci_interrupt_setup(vdev, errp)) { + goto out_teardown; + } + + vfio_pci_register_err_notifier(vdev); + vfio_pci_register_req_notifier(vdev); + return; =20 +out_teardown: + vfio_pci_teardown_msi(vdev); + vfio_pci_bars_exit(vdev); error: error_prepend(errp, VFIO_MSG_PREFIX, vdev->vbasedev.name); + vfio_pci_put_device(vdev); } =20 static void vfio_user_instance_init(Object *obj) --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255152; cv=pass; d=zohomail.com; s=zohoarc; b=cxPZ3A4agenTIcp2ZvSU9cLC4tqvblOFdoVYtHfMXnNcYzwexPzs/5ICizjHBf3ekH/gakcireQa3MQDJb9IShcZKlakVQAp0egnQQCLKEKA3JXp1Vgl6+aptXy6Mg54szOqgsLIGic2gYYHpi/ekedW/L+bNbOwRRXv5Jjqzvs= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255152; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=gLSKAir9CownbVZp50Jw3pkCn369NuhHnd0TmbNavzo=; b=fFqZNkrjX+vBZakXm62sIGe5EvgwaJxgv/YJsDCe7QPOOs+/ctExd2twkUtgHUHUQ5rtYXa6aC8m8kRRbCweBjnSr/lhHtCr5+TkwbLLAEOphITwLfjPrewFjWX7aW3TBipBnt2jvRjFpDairCLMTr3mqF5CcHZIv7HMYSpFGKc= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255152316409.58002303959256; Fri, 6 Jun 2025 17:12:32 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9h-000319-MQ; Fri, 06 Jun 2025 20:11:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9g-00030o-Gz for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:24 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9e-0006QI-DJ for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:24 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556Lu6GA022566; Fri, 6 Jun 2025 17:11:21 -0700 Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12on2114.outbound.protection.outlook.com [40.107.237.114]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 472njsxqau-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:20 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:19 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:19 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=gLSKAir9CownbVZp50Jw3pkCn369NuhHnd0TmbNav zo=; b=G3hZSBhd47wkcnLR66KPStQAwYsXXpkuhzDkNdRz2eZDOafaFkr8sHJ7J LW22aSg8md8JKaPvbPsAcWzeRoH9IQ5eddJq56b0NVn5bOVnjVDs8k9nhGAmZyJc PEadgiIno1ZEiI2asKCMKq2nQuQhd3zekWTzBbnnZ/txE3RIjKm0b99bL1ljRJQR QXYn9CxLxjj4hwAk28mxTPToPB9H5GhWhjl7zeB0NL9JYvHb09fgS3bix4F+GMCH V4v7cCqb8PmTXWxlS4jtWROg1aGbo9qIBHb//YQgSqmhoFYXZQ4xc8zJ5dhD2rw2 Q6sUFfSTA/Dwv0LpWRIR2e1Ez8hAQ== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=aWY7qLc5pCu3q9H9Fq3RDvBgjdC4y+1UW7cft6psOY4Kz94ns90LNj+GlwVOowmkCTHvDkG8TRak/iHNgx9cam/4lVbZs4SxA9LZwu3r5RNbMIlxiVeChodvu0UkF0Tp9gq8WwcqVfkVIbkJ0XpGQVjPw9V7gPcD4D5nOp1alxjTlPVCJSqtESC1TbCtzPz3UQ8tP7H9dJhqn7Q+9LJv6v95N82Nmr8IZTahXMpYD1As+o9i0GVlw04+3bIjW5zvjt/QiaiUD/AQxAPfSh5Xv/dIAX6Gb+ppTubgShc4+v8rHt8Hma8owpI11NUWQ/GLlQNmWWSdBwH/5Tfzxghi7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gLSKAir9CownbVZp50Jw3pkCn369NuhHnd0TmbNavzo=; b=LNndGCXvdM9VA533DeGZX6wYJFp8h43ZUu9XU76tXKlbdX5NaKe7m7igInRz4ndB9ij6O4aEJAkne55XGi+Wu9M0MRBLBmp50xQhmaE7efhw7XNpygigjSCZCD91/aJg6ka4Szf7bBpfj3ULjswCF5CgNS6y0TP8gNjuCclhZHheubU+Wvm5JXKIx/6Ce7MDjYoKO5sHc4TRXuUw1MgszN9nMYcCNfsyqJeAeklC9AAUGqTcCnXrFKy7KGNOkTyXYTiu6OgZUkn1B+WBLQqoRWyLmAnTRUd/pSFhZgP9oz6MG6gOPLq5Jr7qVFjiwRuRUHh0ZeDZE7tCS8Njs8GKKQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gLSKAir9CownbVZp50Jw3pkCn369NuhHnd0TmbNavzo=; b=aNd5DShztgv8qNmebAwUqN8/mK/2l/fyf9jHLU0d8VTMMYnURi4kZAlz8SPkV89ywDz2AR8bJioLmgECfOc18/bYn+Nj8GAIQCmyURUkvoS2b/pRzmOTYoUF1aFA/In9mutYpxICup0FLzSubrhhjlUGdEdMjbcqlyUD/ntz+t4FWRF6RJxzAm52t//WzwKcvx8uH1Y8NfTwjxajLcXpZFHnUFUa8UYUSQeueDTlOmVkO89P2iZebgfnuxNJ1TO/suA4dQXw4jZK506A86/PYqw3kVB0mmqbGn6WFKu1BYbfLyuGwnX5BEr5XEOAQ+/0qZh1Ysrs3UYerc7pvkN3gA== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 14/23] vfio-user: implement VFIO_USER_DEVICE_GET/SET_IRQ* Date: Fri, 6 Jun 2025 17:10:46 -0700 Message-ID: <20250607001056.335310-15-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: 7cabdbca-4645-4725-e6e9-08dda557d3a9 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?vbWPYsarFo0gRCRcuYwwi++T1r8SSvrgBtU7wVmGtNVYWsdx30hRVy5ACAeb?= =?us-ascii?Q?QqR/jag0YNAPHcJkkM2ZAaZZ88feVQQe5cZj1lsV0QCfjHzcPHd/SefsLMYP?= =?us-ascii?Q?C2TAs7w2vsFWwHEVZenYmBbN3iNJG+evrPfM7S58qXeM4y7wZVX8bPGIUg8Q?= =?us-ascii?Q?r3DkkU1rt6s9aSr1xOi5IzaP9U+AD4zkSbm4nWnl47HBHdO4aGvImNQNE3Y7?= =?us-ascii?Q?W96MqNw3pMBA4S1Nb5CqtPjr9W2ZdL+3O5WaC3H28qWyxO9qHDc12tr75K6i?= =?us-ascii?Q?EmUHWiIJMarHbaJJiUrQa6yAg+i1iyHopE8eIHl2gBBQaeFGUaguM5+uB0Yk?= =?us-ascii?Q?jjhdKOcqUSclwnLN7PrgjyxoJHpYLtMEOrwmokxsn0Q0y0JaGjJyUZdcW3Pt?= =?us-ascii?Q?Iiyx49d/++9+sbSYoGZ7+CuLriGfLjNNPPx3Iq2zWODCgORD5Ak8G9ljUkJM?= =?us-ascii?Q?++6JWvSsTXZKkKTfd+DcShalKE9uxxbVNMVJH6T/U2QYHuixY4zdX9D+mJnu?= =?us-ascii?Q?NXrJzdy18gHsswz7utmzX0cICSSeELhWwAlxZUlyB4vRBHmSwtpHZOyobd/C?= =?us-ascii?Q?dNhTqF+WhCWGhKUVdnWzVeKy2d92XiR3GkEhsezAA6N4q2ssKM0l+zHAKw2o?= =?us-ascii?Q?TKQAZZ9jORJeBuxkaIrcFZO2yD06MSJZmwYKRBCia739+rytTIIZV64zumHg?= =?us-ascii?Q?AE0+/GrKm+XcBSy0HB+6RcTpMNI7+KaJEeMoNlwmtXYPMI8Rp5+nFKvXRHFe?= =?us-ascii?Q?lbnHApKaCpu5zVhndrOP5IPdtI2/RuuEYympt4Ezz9jM+MazScn3vYf2IumB?= =?us-ascii?Q?PZFnDUdxtNHXnRzePeYMt3G8U/eILRebI2EZ6SdGWu8+ZcIFK8wCKaOwwYwQ?= =?us-ascii?Q?9nhDOCI7WKaMpgJdvRv9oDz9VN7jSXwnrkz23wGzS6TCszdjirTxUrqaWJnI?= =?us-ascii?Q?TK4Wixr0i2J3XM3I3Oq5Sq22KsUrDUe06klwyd7L5G9YG7mOWJTe44K0LKaI?= =?us-ascii?Q?aZrJ8tK2v0Xp/d5/aKDynFDHvbcR1wiC627LI4Pp3qeywzg/SrVcnLAnfPQE?= =?us-ascii?Q?PlU8TR4m1iXUgTtscUliulxG7wQ0rn/riGNx/RzkzWMAg3zjYE8OZz496Nre?= =?us-ascii?Q?VFh6uEqh9Hdt1FUlnEA5gFqkmiYbxlyq9IQVl7GEi/e9/oP6gqL7i+2uP5MG?= =?us-ascii?Q?EigjvDqiYlv6FCRUGG2O+jE49CrjYhGv8nTh4J4lqnT+4jIOPhqBrhVH35J5?= =?us-ascii?Q?iAvo3iZ4Jcl2QyhZHiM2kgAT3o1qJVEpDcYswFgMVztaplKHLRUwO9epSxeT?= =?us-ascii?Q?fb4r60rvdo4wioPLM1zhIojJcwkyKA1bT5Y1Tk7zuPxG7G2v0AYCQk4nu/pi?= =?us-ascii?Q?VcNs14pAN0LTIP8HfrKLrJOBHd5hS1d7slNu0z9QOwRXtikYqatwh5/kaxK8?= =?us-ascii?Q?G4MQRl11pdw=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?a1m3emT82LtMQ7N94rrCfZYJnYK8X8wJpGjByLYAjHZw/4LrSYCPfZhxDLbx?= =?us-ascii?Q?WLUwOBsFC/n3ma53KmcyCQ8wd3qDxBBwxVXiKoIUCfTTefGwEd0ckrxWb63z?= =?us-ascii?Q?FWHvFVVAdySQgBneA0hf6G+q1yChPkB23vDILUhxbro9lCp/2sVq+ufrXrND?= =?us-ascii?Q?FSD8mTpyVK/K8jNOf1HpzpMCWuuOHn/HO0HS6dQobN2SXaAQM3iHKR3TWc9f?= =?us-ascii?Q?0HODYGSiuPjO2VgnS5sTNz0AfbJlvC4kDsGQzqLcJSvgLQ5kAJilWXEtpYlW?= =?us-ascii?Q?Y01yp9wpyvjcrXFsSoYw0+ep6ddL+gsIkQFTwM8iQ7uw+xndDGyMvw799ECc?= =?us-ascii?Q?+UtsbFIoS1q7Xu3Btqq3zsnCwcIg/cxLwPs8c7vO9TigRslH5qrXFfECcyjQ?= =?us-ascii?Q?2UhpF8MNMru5cxDYlUvyk0VSyOidxT8Sg3U0RlcmpE9gYhDYMa45vbcYErbr?= =?us-ascii?Q?kcw0JjG22UGcVbWtUr8VEZFOwKzCqphxhNh8U20hvsfHH33zbJBrnFnY/1iz?= =?us-ascii?Q?b7epfopfyvs+vqeCAz9nmyqXt+X2n1ClwLuzByRUX0xl970sqfNjd26OEUbK?= =?us-ascii?Q?8+/8OjRJa5Yh6bqrgL0IO32HNUtzRfg3jXslmCXiGk4fgTIc3uwUHQQGnye7?= =?us-ascii?Q?lpZ0iwObT37wfpB0KERS3baQ2p1Na0MEJJiU0lInWCcrR0PNZW0PENqC7Jot?= =?us-ascii?Q?lrj96LBBVrcJ4vXlrdyxap32bgTAK039FW+GJGb+KIeWb/tJ+lDxGs+emC25?= =?us-ascii?Q?1siw022w4wAktFa3dj7XtjUCPcd7uP8JYCUL94uFKF1ntbK0XQork+KAjD61?= =?us-ascii?Q?UXdsovm/dOdUMe84hzxPhrbTLqA7jIuKo5VxufNzAxtljCT5Wzfr5YQCHlFB?= =?us-ascii?Q?Rlg+Uf76BphRnAOvNvD+ajOOIhx4jOtNtBSVLhPrzk6Zwf9bsM37AI/OBe+z?= =?us-ascii?Q?EisXQsCg8D7Rhl5/6DOY9DW5TZSmwgCtJ9/Au8jAnbjgqFGLMzgt8wqS8Hqo?= =?us-ascii?Q?RYYMxtv+pFbR33taQOlFV7MgTx3Q/7aOLVuPlSFaYE5L4toaa37ayQTgm8r3?= =?us-ascii?Q?B/9oILYcK2AxFLcOHKcAMWHEgepgKBnM0Owd8yRDsySh5WSnWdn5sBrTOvVx?= =?us-ascii?Q?37Rw2LW70/4RAsq9ASITXsRvZidkqXiVTGFUJjDpZemCNNAPgpJJGBhVWhGz?= =?us-ascii?Q?Vj4chwUw65868VcX0pL1Lvna3nri1k5GawjtdyR8yL4RAbYwuaVC5B8PjrcA?= =?us-ascii?Q?n9j9klBhFZk9w2rZYIFCrGCMiD88Xo0EtwqlAXH9EmBOdTA+QdCA43GioMqz?= =?us-ascii?Q?NMwjF7sJNDvORhhqjSUSKvAA58h4FS8U7T1N1gnX/KCTHhQYWMnGiMdR0W1T?= =?us-ascii?Q?0vc1qkn7Fsm8zImyf/UAsk525CNUhd9bOfBEM/DaLR0afVT3Zv7T09zmndOJ?= =?us-ascii?Q?aQAJ3WS39ioL6H/jni24TBsMPodaTBbrN84tI1fHQfE4G7bW5jiTdqWyPjJD?= =?us-ascii?Q?pMWGrIrBsW9JK78O30Y6NNkDday6ONfVwBuBFHpzl+41a1B3eiuuzzb+g7o7?= =?us-ascii?Q?Vspo0zeDbTA383eIgtNlMd4kLAEXI+t/vWZor9uH?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7cabdbca-4645-4725-e6e9-08dda557d3a9 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:19.0109 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jFHRoWoN0PW5tP90A1JG233ktJCZiH08KcDdYBc3AfJi2JRzq0FQMF23nUs1SM2Fxud6X9+4DcUQCVq7UjEa4Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Authority-Analysis: v=2.4 cv=f9JIBPyM c=1 sm=1 tr=0 ts=684383a8 cx=c_pps a=/VclOIiYbL64yKigy/WxPQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=03k_OfxykpieI8sgW_IA:9 X-Proofpoint-GUID: hDBCO5W3rPPIH1OdKb1SCZoxIRdWkhU5 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX8z3QcqNcxwMz eT3ALa0GT3//Hv9UJfk+kfPkAAwGosFze/66AfPuEIKqCT8Xhouu5QYPbCadTl+zJpcaQ0dN/LU beCbsfGpv9/mlH0rD5mE8StmyyY1r+WPPvW0lzU+Mp2Lyv24tdyVaGiifJkF0zjQRpXKKgdT15C kQHd2W+6cDX7f+pqPtWK7VNVSD7zBPaRYBWQcbyW6b8s6IYG5mWhflE5qZCIhb1lGCq3f2nERED 5BPD6JoqrqEgo72ISsTvfwNDxmceex7NPyZp0YPEOuSVNVu1XTZaxWiDZrgo8qijs1JDZ/Vc3e4 E1hgvPXy4y3twrVVae8uM6qmQVh/K8OY0/i3pPxkfnffSIrI+72S09iRyHBZHvGuGpAWCS7REZ9 DdY47FkGrLZpRWDtTEeXUB6IOmjn86rCVeVfzdk4+d5s1gjY8UwQzK41gszhGnMYvWUp8vwQ X-Proofpoint-ORIG-GUID: hDBCO5W3rPPIH1OdKb1SCZoxIRdWkhU5 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255154577116600 Content-Type: text/plain; charset="utf-8" IRQ setup uses the same semantics as the traditional vfio path, but we need to share the corresponding file descriptors with the server as necessary. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/protocol.h | 25 ++++++++ hw/vfio-user/device.c | 121 ++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 2 + 3 files changed, 148 insertions(+) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 8e6ccbe9e8..920b9c44ec 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -141,6 +141,31 @@ typedef struct { uint64_t offset; } VFIOUserRegionInfo; =20 +/* + * VFIO_USER_DEVICE_GET_IRQ_INFO + * imported from struct vfio_irq_info + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t index; + uint32_t count; +} VFIOUserIRQInfo; + +/* + * VFIO_USER_DEVICE_SET_IRQS + * imported from struct vfio_irq_set + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t index; + uint32_t start; + uint32_t count; +} VFIOUserIRQSet; + /* * VFIO_USER_REGION_READ * VFIO_USER_REGION_WRITE diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index a767e4fb01..45b91d22f0 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -110,6 +110,125 @@ static int vfio_user_device_io_get_region_info(VFIODe= vice *vbasedev, return 0; } =20 +static int vfio_user_device_io_get_irq_info(VFIODevice *vbasedev, + struct vfio_irq_info *info) +{ + VFIOUserProxy *proxy =3D vbasedev->proxy; + VFIOUserIRQInfo msg; + + memset(&msg, 0, sizeof(msg)); + vfio_user_request_msg(&msg.hdr, VFIO_USER_DEVICE_GET_IRQ_INFO, + sizeof(msg), 0); + msg.argsz =3D info->argsz; + msg.index =3D info->index; + + vfio_user_send_wait(proxy, &msg.hdr, NULL, 0); + if (msg.hdr.flags & VFIO_USER_ERROR) { + return -msg.hdr.error_reply; + } + trace_vfio_user_get_irq_info(msg.index, msg.flags, msg.count); + + memcpy(info, &msg.argsz, sizeof(*info)); + return 0; +} + +static int irq_howmany(int *fdp, uint32_t cur, uint32_t max) +{ + int n =3D 0; + + if (fdp[cur] !=3D -1) { + do { + n++; + } while (n < max && fdp[cur + n] !=3D -1); + } else { + do { + n++; + } while (n < max && fdp[cur + n] =3D=3D -1); + } + + return n; +} + +static int vfio_user_device_io_set_irqs(VFIODevice *vbasedev, + struct vfio_irq_set *irq) +{ + VFIOUserProxy *proxy =3D vbasedev->proxy; + g_autofree VFIOUserIRQSet *msgp =3D NULL; + uint32_t size, nfds, send_fds, sent_fds, max; + + if (irq->argsz < sizeof(*irq)) { + error_printf("vfio_user_set_irqs argsz too small\n"); + return -EINVAL; + } + + /* + * Handle simple case + */ + if ((irq->flags & VFIO_IRQ_SET_DATA_EVENTFD) =3D=3D 0) { + size =3D sizeof(VFIOUserHdr) + irq->argsz; + msgp =3D g_malloc0(size); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DEVICE_SET_IRQS, size,= 0); + msgp->argsz =3D irq->argsz; + msgp->flags =3D irq->flags; + msgp->index =3D irq->index; + msgp->start =3D irq->start; + msgp->count =3D irq->count; + trace_vfio_user_set_irqs(msgp->index, msgp->start, msgp->count, + msgp->flags); + + vfio_user_send_wait(proxy, &msgp->hdr, NULL, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + + return 0; + } + + /* + * Calculate the number of FDs to send + * and adjust argsz + */ + nfds =3D (irq->argsz - sizeof(*irq)) / sizeof(int); + irq->argsz =3D sizeof(*irq); + msgp =3D g_malloc0(sizeof(*msgp)); + /* + * Send in chunks if over max_send_fds + */ + for (sent_fds =3D 0; nfds > sent_fds; sent_fds +=3D send_fds) { + VFIOUserFDs *arg_fds, loop_fds; + + /* must send all valid FDs or all invalid FDs in single msg */ + max =3D nfds - sent_fds; + if (max > proxy->max_send_fds) { + max =3D proxy->max_send_fds; + } + send_fds =3D irq_howmany((int *)irq->data, sent_fds, max); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DEVICE_SET_IRQS, + sizeof(*msgp), 0); + msgp->argsz =3D irq->argsz; + msgp->flags =3D irq->flags; + msgp->index =3D irq->index; + msgp->start =3D irq->start + sent_fds; + msgp->count =3D send_fds; + trace_vfio_user_set_irqs(msgp->index, msgp->start, msgp->count, + msgp->flags); + + loop_fds.send_fds =3D send_fds; + loop_fds.recv_fds =3D 0; + loop_fds.fds =3D (int *)irq->data + sent_fds; + arg_fds =3D loop_fds.fds[0] !=3D -1 ? &loop_fds : NULL; + + vfio_user_send_wait(proxy, &msgp->hdr, arg_fds, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + } + + return 0; +} + static int vfio_user_device_io_region_read(VFIODevice *vbasedev, uint8_t i= ndex, off_t off, uint32_t count, void *data) @@ -179,6 +298,8 @@ static int vfio_user_device_io_region_write(VFIODevice = *vbasedev, uint8_t index, */ VFIODeviceIOOps vfio_user_device_io_ops_sock =3D { .get_region_info =3D vfio_user_device_io_get_region_info, + .get_irq_info =3D vfio_user_device_io_get_irq_info, + .set_irqs =3D vfio_user_device_io_set_irqs, .region_read =3D vfio_user_device_io_region_read, .region_write =3D vfio_user_device_io_region_write, =20 diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 3f5aebe7ac..053f5932eb 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -9,3 +9,5 @@ vfio_user_version(uint16_t major, uint16_t minor, const cha= r *caps) " major %d m vfio_user_get_info(uint32_t nregions, uint32_t nirqs) " #regions %d #irqs = %d" vfio_user_get_region_info(uint32_t index, uint32_t flags, uint64_t size) "= index %d flags 0x%x size 0x%"PRIx64 vfio_user_region_rw(uint32_t region, uint64_t off, uint32_t count) " regio= n %d offset 0x%"PRIx64" count %d" +vfio_user_get_irq_info(uint32_t index, uint32_t flags, uint32_t count) " i= ndex %d flags 0x%x count %d" +vfio_user_set_irqs(uint32_t index, uint32_t start, uint32_t count, uint32_= t flags) " index %d start %d count %d flags 0x%x" --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255141; cv=pass; d=zohomail.com; s=zohoarc; b=E+wkDqI6an+eKahRJYAMZWU3JH4gapoN7LT1opoKg5lmo4Wl4rBpoUEh8SXFzSy0WWxucWr4ju7TeJGF48IPlmeL/kRmOhDruodeSz1nh/oYuP45g78TwQhxwxYrdRrzpnV9U/nDB0XnLvBjLzoNpGJgRcW1ZcfnUmvmAw/PAtA= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255141; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=FzuK4DjHNkzel/Y1hX5O+J/iIuxZYuPD3fBEnqEQGVE=; b=TOZ2lScjtOEhOm+U4bydfLJX4pDPPDYxSErE7GKY7FHtJR5HvpVqINaCLZpbCRCacDaiYqi18Va5iXN36LQ9PkOSJlV73ZB2kYaXaDSIR/VAgoDA2sZ2PQqrU5BUSn+ekgl4I79FeXGpa2Vrt62lghaxvqjBbmd5mpOZQg36Z+o= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255141851429.78549024727454; Fri, 6 Jun 2025 17:12:21 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9u-000332-OO; Fri, 06 Jun 2025 20:11:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9s-00032S-1j for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:36 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9p-0006QS-Gq for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:35 -0400 Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556Kwc12019117; Fri, 6 Jun 2025 17:11:22 -0700 Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12on2096.outbound.protection.outlook.com [40.107.237.96]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 4747u3r8fn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:21 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:20 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=FzuK4DjHNkzel/Y1hX5O+J/iIuxZYuPD3fBEnqEQG VE=; b=m37PnhJ17y9PM06Jo4Dahk2xUG1jDxth9gdFU0rcZvMD78bUH3fGY+XWe t2bEqtDZsKK9isYQ3lN8dmWm7iP2rIegkPHv3oVEuR+f1qtqLfwpT2Zh7BV9aPQw YrdZxLRR1bJUPf7aoG1zeqydSm7RCQ7FNcnQaJseVww6Cv3OfWh2snbluFzZzZXi QtdOf3WqxebZoNohILTXr0kyy7mkUsGutjjtTp1XUJDYOYovbr0nyQGY09KpdEDM cHP72W62UFgMPVRz325LxN0qBZxm3urSBm/y5XecqUGaSSKTjODlE3ZtXrLkFFna 1REwxSaqLb0pTS/lPFwUWAU26Cpvg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=AzCUW/rMFmYJDpNPDe4L7atKHpCu1I9EzSPpP3aUjD4itqysnzqhp1OzcHfmlfHNtKNj8/vGWkh6of8qkpM/d2E0fjN62EYHMuVRPPxntucdxfcSdFFEgXBnvJtLcgC7JKIjjV345B99YOqqzi6ssCcksxcItSoJw2bIkK+BtiInd7HNY/l8/taxZl9nzOmk6raBwx7EqdTr2bXZIw8X+YIuEE7uA/A2aIpIWsdd1bd6/a63oxLGDGSkWXLMVZEDRxrWC46CyamBtkSGmAuc9bkwYi2kZbo3w1b9ND9F4olYl71722mFigJJVjt9Mss17GetmYL7bxS1QCH1l6ogRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FzuK4DjHNkzel/Y1hX5O+J/iIuxZYuPD3fBEnqEQGVE=; b=XYKJYVIMhEHXkNqi9Unx++F0SjYffDDp7vdLN+XL4ZamIL1eulO0SZ4qNTgX0E2UTqVzACHA87pI+7G6PbdxXgMlmlvp7722RhMODLicqDoYTvIrjQcB9uFXZQGEJ5w1unc1WV2pjZmCfT+FAkrdXJg73x2U5BIVoEAFxQLYqAGP3kpqVzmbSwIx7iOWzb5MdpEQonElfm6ug+c5glXETZO/8OF1qJVNyiFK2ASCThRDFtyDP+zhdKPymFGiEeHGm8hx6Zh1aoc+CU8ajuoG0j3xs/jbVyb6ZIngoZzlS811DU5r1U2SZRU9tVsitD3R3fgpNHuZpjoOUnDqCDyCAg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FzuK4DjHNkzel/Y1hX5O+J/iIuxZYuPD3fBEnqEQGVE=; b=lk673riy11F5sTkql80uFNy2jy1/OJdTXzizrom73OMhqn3u2Yl9DYZ4/ofBfxiNFsQCM66EZYdSVnPRaHoAJHpEAMkkIn8uO9Konz4Qd/ajyS4oQU9+h9D28TLT3rD1YZ4nq2l7mGu0H7ZGLzqAV5c5hsaQ1Z+LKVK/pLytkxqvzMAaZx5GrRuznuekRNbR1tXa1Bqi0FVE/WoQ3e0h9iqul2wMiVuvhinwGkV5XFIrsnZjOm9e5r2otXA3MOKMmRgP1sBychYQEZTmgFNaokL3V1w7Tnvqs66xhCZHO32GBSYItEj8SYbZYNoh6fdYdzR/i15C3ZE7/wZSSlqt/A== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 15/23] vfio-user: forward MSI-X PBA BAR accesses to server Date: Fri, 6 Jun 2025 17:10:47 -0700 Message-ID: <20250607001056.335310-16-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: 56f0bf15-132a-48df-e96c-08dda557d484 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?qeZ/k3yQ2VJCHYV0FY1NgFqYB3OvYUqtdtVZ1zc5k605Nxg8CEM3uv++1YkJ?= =?us-ascii?Q?Sha6bGHOBmgQj5wp/Ccy5cn+UOUrJ9Ln0w3in487JsTGX+PiLrZRLI5VTIE4?= =?us-ascii?Q?ubFWAJXvsmlNcfUtvhT9Ovzx+ddC8BAk8LbOxhEz7Ki+qzNtub+PENF7kuvq?= =?us-ascii?Q?eMJCF6IOZlXvwDRdvh7M/qcH3iNWDq42ypOaiqCsKK+elOffQB+0aceDTnSm?= =?us-ascii?Q?kmx5KUId2DLcKeKbUkbcPkAoO9Ztu8iBK1/meXm8WTDyCktm0kgK7dZL9AS6?= =?us-ascii?Q?q0MjD+iCUdsIMEZWDgpJ12qIsDq8VP1/OHgATIqVQrJZMoNuBUCzutwtlxp5?= =?us-ascii?Q?TJcBj218kFkQllB2FAr8gXu29cMO02h+beE36n5H1rR0SMTR93ZDDZUZgfOD?= =?us-ascii?Q?UdkLYXC39sieC+kKROOOgr09JlRADgy2aqEY6MAVMpmy0jsTxCTzZRaZIOMP?= =?us-ascii?Q?k5WA7cKu9lGRIIgzp1GOoYmeV6WrsDrXkVHMaqc+WpYDOBQPSFg9IkUdjQ4x?= =?us-ascii?Q?MOPxof+8nBTOJQhY05iwV62sssfhS6qrKryNtD/Yowu0O7u1XwoFFQfxMs29?= =?us-ascii?Q?VLPT0mZ4jcM9n1Z5ifcd9ilFD0KvRI/4AlDcJMk1C1pH29mOXNp1aqWJUWF9?= =?us-ascii?Q?30Pvwro9Zic0VGhBpDQ0K4u5WuiRRCKuuAAZsQehaLXobagTEWpUHq7MZiy6?= =?us-ascii?Q?Oxidxj2L4gpYQD/sJWrifSfxEfpazu6uoeqGBA+xlnfviM5TeAL4Xy0iea3c?= =?us-ascii?Q?Osc057OohIZogxSs7khyMpm3rRK86OtXKxH1ev/bcRVQRxWsC9Op1NAkhFyA?= =?us-ascii?Q?KweYwJdvZfi6DtlYGgP68IiiZWRpzcr1DkyCVOzup+Rn8P7Bac3uRakTvvYh?= =?us-ascii?Q?7iymLv2zO7/d7aWYD+VTIzHo3DqaSW5tiX2YGkY4DRVSKPL6uCvcxNFyok/N?= =?us-ascii?Q?pXkFTnkUKHteQlPWmHZCrfv5A7jrP/6gubnbMrxIJxJ7r99BXCvx3qbc1Fgs?= =?us-ascii?Q?b/Jv/S9RG4gs7rIcbPV1N1gFcBGUMtDcF7jZYeVDXuZDvhuxVr8oEjfJfTLM?= =?us-ascii?Q?XYBK3ArmWafGWfRUa9NDjihGFzIa+tZA+1I01JbWIufaV9e5kA/ckK1JzSah?= =?us-ascii?Q?ypYRhQsP4u5HF0GK9hTJMT5iqgLA20M7+Lp/+GUl5NoUwPYIxF/a/IfcgcpM?= =?us-ascii?Q?5S/Q3ex3jo2zrg9YQEBDMhs7VchPCTMlYvy9KFxaFlS5QRCk57jx2WYqmxZy?= =?us-ascii?Q?X9W8ng0d+qre7VHTJsZ9hBdsTRQ3gsOqhUGY+IJux2TyS6G0pu8ZabUjm1kK?= =?us-ascii?Q?fC5jysSZsQmLMn3xMOuW5qABOVIpcuv32UpF8uXq3T1F/6eSJ/bByH4B5KBv?= =?us-ascii?Q?aGgHknvwFEax7Sb9JYm2pW0DrE/Gln0xNA4RdH8eGsvN22cU2bArmz1naVtp?= =?us-ascii?Q?mSC2xQj0xy0=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?JslIefZOxPzokjaAHDGozDT3cw41M9xQXoVDk2ECm4wRY7tC4AdA7yqP6dZa?= =?us-ascii?Q?Kh/y+tsEoSM77KIqyl5+1jQNQ2wDRaOm9NICXlu62Omy6h3wGKrZPmaz7uPd?= =?us-ascii?Q?codgfMwHS6HgddOAnuXeEuK4yoBFD2od1MKh5RfVlEHroL7AViHbeXQ4i/Js?= =?us-ascii?Q?j0TITnuRbfwJFl2EaSPppmECMROUAKnUm/n/JVf2SpG84OhcfWEG+zzlj4IM?= =?us-ascii?Q?eq9K8VtcoP10ITH6n+W6gkF7Yg6ngXWIIPL/7Q6ZyfLuqutBfnwGeiRCrEQW?= =?us-ascii?Q?59S/hrJWb6D8ucUS856ahQSHLn/mEX3gTEPjF5GVEAtbtGOFTw9pqLYlpYqA?= =?us-ascii?Q?ipTYdzw37ES+/VMsK67KSi6MmjgtjjK5r1G2W8oQ+j77uhwO473QxiECXFp2?= =?us-ascii?Q?5Qt6AgtyptMV4x5W/VLnKewQDUOTW/xkjqPSt1P9SxijaFgIbM03htnApWqN?= =?us-ascii?Q?AClS9Iq89KyCA++fF91YvuVWeJCce1JIcwFVE3qUxggWX7F3TJMC14aH5rUt?= =?us-ascii?Q?PqRS8UnwWR7qSFKvAunvYSycxHMDyL10B1rPDci4ILr5KRNaRmoafkMXy345?= =?us-ascii?Q?fHWKnxevJtb8cH8sQXoRSSVtinaJtf2zbtdqakUN6AbmBvUGwhG1FJfQEgaT?= =?us-ascii?Q?zCUcHTgtsK1BOjSlSBLFaxzXaeQNNaCyvE12T1wugSSsN9NrR0khcZ17NYed?= =?us-ascii?Q?ysKQKhRIY2Oi/Nv1f0stMD11IqritcZm/x0rCQs8w95lUQue5/YmB2AyGdGy?= =?us-ascii?Q?RAGoRD8eWp5hSV82ZnkL5+79KnW3ibNGo2aF+S44j4wPwX8hBbN49adMipF6?= =?us-ascii?Q?4K4bSCqxSFZt7iIMWco4Gufqg/4xx/0/6pENMuiQphtfBrTe+pnt7Ielfsmq?= =?us-ascii?Q?gW87F0mRnDflKiS6CqifsDXuIlPtn1n3tF9NsFKOTR5OvxgojAhibYdJVnLl?= =?us-ascii?Q?KjhYKq3r8i/y0IrHxLzjYGIknvsbLW+NRfasMqpcXcA2v7Y10JsS0i2QzfKQ?= =?us-ascii?Q?1ZGN3SJDDXcPlN2M4eY5bwgvUqOqcb3ddAXLigZsdWeFi+QccGgF6rruEJNX?= =?us-ascii?Q?po+hxR7kx4o4nIgfcg3mfE6VtcCdAAhgJbrM3/C8SrNoZA72US0Fk58xm0lg?= =?us-ascii?Q?xb2W3CmcnQkeU+UJXXnoSr6RMc/+WXYH+9q0aywOKsPKLjKuVG1ktrj+GDWl?= =?us-ascii?Q?mE2a/XV6SMULO7IHwLW0ZH4/9z5JI1tGkh6QLdvkzpBr0gZXnj+cFfeduWfV?= =?us-ascii?Q?iUljZecyXt1bRTP2XJi/tBL2QTNQ/sXl+MIBDBPCbmzrLeSvXVOmH8NfglaX?= =?us-ascii?Q?vy6XVAujWu6CCFKCx/vpMr9ZlVPj3TVxEfqgRHLvkcyXa5Hnhjo/cdf4A2XD?= =?us-ascii?Q?zVIo1B1ch7WE/BhsxMnFe/xQzakgWFLeyB50mNDG104erHDjURwl/LwC7HZg?= =?us-ascii?Q?XUiQ++bw68OmS2nLoNfYLugHdJQK8bLQGKiD7iWcduGeEo0XLX9k0oUwYsvt?= =?us-ascii?Q?MlAeewGo4UceB3SMHVh5ZKDvgBcqsfZ8L3QkLb9F6orI3LMAlf+XWKDr7pIR?= =?us-ascii?Q?VlwxIOU9e/iG4cdxBB+668F1UHKm4kfw+L74MgCZ?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 56f0bf15-132a-48df-e96c-08dda557d484 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:20.4040 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rpjumog2H1SH4+5OMMkl50qgMel98oiKaXmOGfRNkGocwN9qlX3x30KACuzmqm64QRVARdVJ/ILyb7eaCHMv7A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Authority-Analysis: v=2.4 cv=cpSbk04i c=1 sm=1 tr=0 ts=684383aa cx=c_pps a=XYC6++eb3LU7Tn/YozfASw==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=P96-NN1LHiVJMZ2xip4A:9 X-Proofpoint-ORIG-GUID: ysfLDn_WGXP4Vh698zuz2CUlTDQCpQBs X-Proofpoint-GUID: ysfLDn_WGXP4Vh698zuz2CUlTDQCpQBs X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX2A7fm+JIpXty L7b9eO+lG4dIv6vmJbNOFHkypz8P18nvQDGsjYSqmDx+p3wEk/W/a4irUVjWS8TOdgPBMdVJJ4n H1uofTe8Ce+/WRf2OHMUm6cLT0WyKIToyh6bewfRw0F1/At8CKcttht+BJS0u6X2KliP2BdPmEA 8VnEeD+eBJwM3ncl239V2PH9YRMNx8RfR03Pc+D5Jo4LJEnj3pm+ivRAwjIEqgzOu4uDIcRDTUn uBMzY2oqT4TuhCSCzz3a19HGJJHV6bDzl6nruKYtqUwXROIZpHPInKU30I2I9wokdDTK4LaPF74 SSiWtcfa24/nR6CwxmWKjMoiMHmVUlw5uZeCrXMMfNe0duHjvzwLRV7KQEM/i8/Gpj/T/ZkG+M0 9+bVOTOueEQ/QoqrSV/2F3eYDHSu0QwhyrvC3Rs6W2fPW3JEBdxRJ1Et4uhlFXCoaf6qAPJ+ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255142334116600 Content-Type: text/plain; charset="utf-8" For vfio-user, the server holds the pending IRQ state; set up an I/O region for the MSI-X PBA so we can ask the server for this state on a PBA read. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio/pci.h | 1 + hw/vfio-user/pci.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index e3a7d7bdca..47cf50a4bc 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -116,6 +116,7 @@ typedef struct VFIOMSIXInfo { uint32_t pba_offset; unsigned long *pending; bool noresize; + MemoryRegion *pba_region; } VFIOMSIXInfo; =20 /* diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 5ac3fb087f..e055d5ffec 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -26,6 +26,62 @@ struct VFIOUserPCIDevice { bool send_queued; /* all sends are queued */ }; =20 +/* + * The server maintains the device's pending interrupts, + * via its MSIX table and PBA, so we treat these accesses + * like PCI config space and forward them. + */ +static uint64_t vfio_user_pba_read(void *opaque, hwaddr addr, + unsigned size) +{ + VFIOPCIDevice *vdev =3D opaque; + VFIORegion *region =3D &vdev->bars[vdev->msix->pba_bar].region; + uint64_t data; + + /* server copy is what matters */ + data =3D vfio_region_read(region, addr + vdev->msix->pba_offset, size); + return data; +} + +static void vfio_user_pba_write(void *opaque, hwaddr addr, + uint64_t data, unsigned size) +{ + /* dropped */ +} + +static const MemoryRegionOps vfio_user_pba_ops =3D { + .read =3D vfio_user_pba_read, + .write =3D vfio_user_pba_write, + .endianness =3D DEVICE_LITTLE_ENDIAN, +}; + +static void vfio_user_msix_setup(VFIOPCIDevice *vdev) +{ + MemoryRegion *vfio_reg, *msix_reg, *pba_reg; + + pba_reg =3D g_new0(MemoryRegion, 1); + vdev->msix->pba_region =3D pba_reg; + + vfio_reg =3D vdev->bars[vdev->msix->pba_bar].mr; + msix_reg =3D &vdev->pdev.msix_pba_mmio; + memory_region_init_io(pba_reg, OBJECT(vdev), &vfio_user_pba_ops, vdev, + "VFIO MSIX PBA", int128_get64(msix_reg->size)); + memory_region_add_subregion_overlap(vfio_reg, vdev->msix->pba_offset, + pba_reg, 1); +} + +static void vfio_user_msix_teardown(VFIOPCIDevice *vdev) +{ + MemoryRegion *mr, *sub; + + mr =3D vdev->bars[vdev->msix->pba_bar].mr; + sub =3D vdev->msix->pba_region; + memory_region_del_subregion(mr, sub); + + g_free(vdev->msix->pba_region); + vdev->msix->pba_region =3D NULL; +} + /* * Incoming request message callback. * @@ -142,6 +198,10 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Err= or **errp) goto out_teardown; } =20 + if (vdev->msix !=3D NULL) { + vfio_user_msix_setup(vdev); + } + if (!vfio_pci_interrupt_setup(vdev, errp)) { goto out_teardown; } @@ -190,6 +250,10 @@ static void vfio_user_instance_finalize(Object *obj) VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); VFIODevice *vbasedev =3D &vdev->vbasedev; =20 + if (vdev->msix !=3D NULL) { + vfio_user_msix_teardown(vdev); + } + vfio_pci_put_device(vdev); =20 if (vbasedev->proxy !=3D NULL) { --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255120; cv=pass; d=zohomail.com; s=zohoarc; b=oFWwQZzAjB7rUQDlsf6lnUYfxOTsfL0yxABOyy3md5ArhRT81s//a0hWZnSsckDh/aoymvsd0cuBxFa934ojqVjo+0d08i2wxK9vGVRv8T9sGgMD5wOrwuSQTAe+hcxQNmq39fzNbZdbflVH+25XPT4c0rdTZ2gjfZYMVq8RDRM= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255120; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=le0NKHZtXWjCySeTO7S61i/syVWzvme3vUpWwRiOyn4=; b=SukY03q/W2DkouqqDIrfRKQ17mbtgoFe40WmzaJ2aLW91b90iDjySJrQtx0UPRJQGECdbRVdz0sNeWaUlw+Vl7/158op2C6dOB9sSCiwwvlkhEo1DMKe1L4UFI4VzICxaAjSHvOeryo27+Hy0BChqII9KpEXRqFq8JrEXL5tRq0= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255120131887.4778184511795; Fri, 6 Jun 2025 17:12:00 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9w-00033F-VR; Fri, 06 Jun 2025 20:11:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9u-00032p-4E for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:38 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9q-0006QY-QM for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:37 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556NqBwV023142; Fri, 6 Jun 2025 17:11:23 -0700 Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12on2137.outbound.protection.outlook.com [40.107.237.137]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 472njsxqax-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:23 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8715.namprd02.prod.outlook.com (2603:10b6:510:da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.21; Sat, 7 Jun 2025 00:11:21 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:21 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=le0NKHZtXWjCySeTO7S61i/syVWzvme3vUpWwRiOy n4=; b=e5lGYuSlR7pF5cGUn4QuN0PUDA2TqLYLUwcM5VUm70e2r/x+kW5EU8iKR 6V4B8nAkVNN/EBDTnbGcqM6eNalSCCfddnce9UP4hs7QT8YZdJptCafqGTp0xwAb 1CNZcUR10Y15MjkLDaq89DcQvjKLbK9qssgfKFNr9t0b4grnSzrvYjTDcS0GACl1 8hmNPS55iiZktCskQxtIfXDBhjubqPTJ5EyP9xmqLlRHlnxSjMsbAP3N7XKTVe0P W0KKzBlcob8SdGcz/BRKaLhty/WMnUjU283+R16vNshc4iLAtmidgdt4XQB9mghs EHmSEANWbK8pVoCepIkDUktDpy5OQ== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ahOA68HBisb+Ks4KMnhqI5w+EfEUXXR2CTLUqsGRcieE/zoqZh+ALBPCDSgGkca0v3nl3n4Jpfp3Ay5gnMcP5x1M/JxM0xtclqANz9LJ2d58VFDSo2VMejGbMJ+j7E6nNva1vtlurJ/3fOK+UxN8g7Om8oroqk5Il3FR61+H3Q5RDCU7U0i/kzDaOkCI3x4mHau4DYaiL0wMIf9okRAVIIgho+oOcpFx/wqQEsORzbW+G9F5p+Hw67CkbHBYXxKBwpy2cmKJfhEdUmQFf2JCJheEOtW0HZiTTbWaZM/dmmVQFr0yfQIn+JeGEuuYBF8vU6H3J4srem9f1Z8OYiA6SA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=le0NKHZtXWjCySeTO7S61i/syVWzvme3vUpWwRiOyn4=; b=JpWQN5R7Amf5dqUp1uQNKJfCkryGKPDMWCqLwT7tCqdlqFww/3HganHXCBRQ51HjWk93enA5cViMlnbT2gZewDGWw03VOoS/emW4RC5YSNQXFk6LqNaUExPGBgz8xRkYtb6onLUwQQCDF+M9uKVzLecwRvMeDNM/F5kJ8LALbRhAfjWjLcxs0sqoXZctphagRmBWJnacgKN62jORl7kU8SjWm9bVyg83Z24xBubmbMfzk6tBMTka/dO6jrxv+KVYh6nos7NABbdeWhzJtaq9hK18WZmbx3CUmLwln645W3vA/A/tsMfXIM03kEME64pobGPKZJnF4iASOnH30MNAsA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=le0NKHZtXWjCySeTO7S61i/syVWzvme3vUpWwRiOyn4=; b=kZ5myy2PaY4nLPz9NllP2CdKBR30WppkjNyHhyiBrVThnxZM1YTco9IJHxfWZHmVZ+I6ncTztnAQg7QcMVijPtDEgOD9xpJw9xnR6vWBV9XOrUFKq+kevoWpfgB3AoTROaOWbATA3yse4DFOOna7AhJc2J8vPTc40pHSXDjZ4MN2xIo6MDBxEr7JcX1w7e57bf0s6d7vxizrAutNf3/gWMY8vAsZHAVcNhTtmJDTS84ScGCaCnm0vxLZozQ2r2VQP/SkcD85FnnXSbpCJs5lWyrgdm11qR+7bbnFLbSDtZNKjZqDFTntY27c04btwDtLpgLHPbD5QS7f6uIkXwRnLg== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 16/23] vfio-user: set up container access to the proxy Date: Fri, 6 Jun 2025 17:10:48 -0700 Message-ID: <20250607001056.335310-17-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8715:EE_ X-MS-Office365-Filtering-Correlation-Id: 2af1a3e2-fc01-4c8d-a6f9-08dda557d555 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?MkelGuIKg3YqAP2HAS0eTJxsEZ8O+V3E13impj/s1OlkwfjKx+yM3VqLl7GB?= =?us-ascii?Q?p8qqBZDKVVqKRi16MY7NGB1lRwSBU+A3Gn0o9UaHBif8HkIygApOrl26HtPr?= =?us-ascii?Q?FyPEapx10LcK/MNIQnYe8Is8q9O+2//ZaxMls6ya36saNtdv8LKoBIBeF7D7?= =?us-ascii?Q?DrYkbj0aAhVgA3C0dImJacvNC0GuyY+SxtdouyGTFH5sJUE7FpUOelz5+9Rv?= =?us-ascii?Q?nKslMvde+WTh2nsXduSiuqtUensz1xTXoO2LxRzPyaJSsBwjt59ylC5iD8rb?= =?us-ascii?Q?CbObWdoMFkGOuPcCdKjcsdvlihe7LV/ajWRv4/AkB3KmzsEFd3D0mX3RqHY7?= =?us-ascii?Q?DO8wUfPEKnIAC6OZNSwfiHT68sNpstA+PKNruoD0V2YU/qodwCGwhyB5+j+S?= =?us-ascii?Q?c4LmXG2ZrKNaoH6k6PlvZjMZRnsSJf1P/Sb3PUOLUo6z6HzS63XTIxQmE+ZI?= =?us-ascii?Q?Ioo5PDDkYj5d9+CqNzTgRo0EseZUrk9eXg0trTNAHDBgxgbfMrF8oiUz1Paw?= =?us-ascii?Q?z2MKRB3b87W/PRPZ5rjZyWnyp0lnhCuH+dSflNTB6O2mzndrZx+GnAlQofd0?= =?us-ascii?Q?8FwNlsQWMq6BRjhDfByQ8KQ3CuKZD6mPHO9rLV816Ea50Uu7uSbb3sdVqm93?= =?us-ascii?Q?s+c6sAU0cZVEWFwXjoCUwV9nHwIunxc8lyJA9W7so067dcH6JkvwrTx8S8qY?= =?us-ascii?Q?vXrfz4aTLer9F2LVlNX1KO0kswbZslmZnQvOx63MFys0BkLKbeA2oTTcD2uC?= =?us-ascii?Q?SHLXapUsTRpm8gSHVL+OyutLr2kli4eYNNn9WKwjKAYbIpOyPf6qtwr1wDgi?= =?us-ascii?Q?UjT0uj9PouGmkZgyYd70PEJ3YSHO5uVbdCi5O9CnlAtLIn/XMjxdgRqgew+1?= =?us-ascii?Q?LoRDj/1+UQ3D/rSXDmWBVuU4SNsyhMOyAc4d24VqSCHCFLHinCiiX3K9+Qb4?= =?us-ascii?Q?xHNtg4zenFra36CyJcuo6Q3iJYi+Ay81ntBzEJiU1rJLByq48UzHWg4sRlZ6?= =?us-ascii?Q?inFcZEJnfIvAfqn8SV3IBhCng10E3nRa4kchfnagIs9uEeltx/+TqG0WAg7/?= =?us-ascii?Q?pjaHTlrZtopgXE4nskvnj/k4QqcVWls/sgBTew/ssOzuO6qMVP5kK9o75JwI?= =?us-ascii?Q?Rw9qQwM+As/9tfRhXi4yY5CM0cUSRwF6K1K3WRTeRFaM3SCbxwdfCb64f2f/?= =?us-ascii?Q?WQd283G4HrbKhGoDwIvGnOiyQg6d+Vt/s4DO7qtokZSS+zAh4dfoDEL9FXoY?= =?us-ascii?Q?BQHZoDJzel2e0wDMpivtRxHZWCy8IetfELOjy7+RlhDDmTq26FlSykS9i4pY?= =?us-ascii?Q?G9Rray7XBhHNjZRhBujr5ySb803u44Xj6kxxu9Q/SHe89Ilx41deMcVjJ2QW?= =?us-ascii?Q?LCbIkHEbXHC0JS0aoHhAxRKhbiKb5GbYNm8ylVFVEk/pM3KTornUefQYp/9V?= =?us-ascii?Q?oxldU2T/lTA=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?kB9+C7b1XnaMSw1xE+3WC9EJngYkCAZ2DVQH3kuEF8OV/ypYvnlsmSh/iCAo?= =?us-ascii?Q?dbhxjfoHAoOWVxVUkVDeqwolpEh/toqn0Ps1cUxcYONW8lXKvaIpVaMZJc7a?= =?us-ascii?Q?SHd7hK4pqFzb4an8AK3oVDUZY26u5gc1OzEWnovNskOji+31aIT2VZ4BaBgM?= =?us-ascii?Q?EFJwzdGAGLskqZWIFmNCTTwlSBPhkR8Z7+1hKXc5y2Vtsx6A+J22QoocrwEb?= =?us-ascii?Q?H5IH/ZW0gwAcRgiMME1QBLc4ueFkEOCg5PxULsifmhnkYMbQM1xrAWKZ6Y7w?= =?us-ascii?Q?BJY1NoPxO5OGAElFfEzLdUyD5iBTDXjHofxcD6FLhDXVeuaHUBJ6qMwEsU1c?= =?us-ascii?Q?Yo/qG+pO30R9kTSKPJzjv8lWlDcwLA8PNYnItbw/OjvqyswTKBRf3FJYpP5z?= =?us-ascii?Q?Zedw/uvzAKdegC2d9YqTPlpG6WIhcFhrZhtd0n+m5pHGZWZ/Gm4N888hDpe9?= =?us-ascii?Q?iunar+pCV7uurYLdOaFm2d03vHcw/OoOFhkAGUFjXvL+n9b7+C+eCoNoXmOe?= =?us-ascii?Q?WwIFAyk+wIlegstQabJhe9SqNsdF2oUa/EUKuRejbMhoYuea5PgwlfM/dRoV?= =?us-ascii?Q?LLyI4+vgomRV/LeIzytoaYY5/V2luhmHaTndXIgVxlM7p5ILbaLFPAYEWrEO?= =?us-ascii?Q?BGatJ6KTRYTbquz4taPlsyU8ZX1B8OwF8pK4ppi+qM8s/VufyFYwO3ujVUuK?= =?us-ascii?Q?7G2OsTPgtJ3cNHq4RZxXYuh3Z+A5FPs7JWhTZFDRaJ843Mo4bQSHNpfBxZJy?= =?us-ascii?Q?l8JukP+m8D1onamYJlEy/TBv9jncJgm8nYEOQO+oYcUhbiuZIZgyP7tf1Fdx?= =?us-ascii?Q?ivgCu6/YLGPh0lNX2nvL07zbRvDEh2X2PDtwqKq5Yl0C5ayPHrAKrqOmqibz?= =?us-ascii?Q?gH008vgeNberrFJA8KcZWLxtFiGLAnRklnx9RIrpu3T4tL04BZniLt6cRhhx?= =?us-ascii?Q?Bjg02JCkh8N8RyNTE5mHw09Td67dpZ4g/mwXgoMJd73rcNRQw/dP4RvbY4M6?= =?us-ascii?Q?phZOyuc0a+u20o9X+r1eNNl5dDghIOXQYboTRvsq/pY/2Jw9GilwU04IcPoI?= =?us-ascii?Q?W1qAg3gB6qAsjEU2r5rfma6S5LAJ5bVMqeS48ukduqkng1POT2Ex3w8iW6yz?= =?us-ascii?Q?z1oxBYMmiyaVXCxB69uVKkOUJ+Ac5OKW3ScfPxLB99p0PIj/WQ3p2KVOfCYB?= =?us-ascii?Q?SCnIPL0vC8g/7p6DrTc+5jfm557pRIou6jKSsa+mF6psXTCFGj95zyLa618o?= =?us-ascii?Q?6awHA7ZbQWyEm/bKsQkyw2zMwQfagVmiPdueQBu55YAsKjpTS2WyDKItl8sK?= =?us-ascii?Q?Kpjo0F+d34ze8r3EWvkrpI9TjQ7t/EkK3urFUO5KeDJaX7qnIItEO3XowwDV?= =?us-ascii?Q?xWHIiFkLy0MXSi6hOXQY18ZCxY2gOGcDuIm8sc7/+Nk1IDIXM8+AajGcqpkJ?= =?us-ascii?Q?E5THJwdlCFdt9IY18eAxDiEobsbD+sDdNntBAsC44oK5JXK6cM8tyDfufOeC?= =?us-ascii?Q?9BVlELqaxEtwWFeGte2OHbTHm8h1625H+Tl/aqXtZ2g8V6DyKK+3+bZnHVe4?= =?us-ascii?Q?gk1i31EcpwZEYinTkU2k62bfccJn4s8g+usSgG8e?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2af1a3e2-fc01-4c8d-a6f9-08dda557d555 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:21.7727 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yDInLkJhDhzvsHljlXq7NZFmluURjNIpJtwdFv1Ts/Wp0N+jLzhdAkYJaPdG9lr9moDHy+zcRjyqI29yT8n+rw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8715 X-Authority-Analysis: v=2.4 cv=f9JIBPyM c=1 sm=1 tr=0 ts=684383ab cx=c_pps a=kojefl/8vcI7rOD5WQpBNw==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=eNiANMbe5KInMaSQXcgA:9 X-Proofpoint-GUID: nM62GD2398d-UGQ9uWgpiGklQGiTWB6W X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfXzyTLl85LnfFW J7e33uE6lXQhIGo4+HLbUH4zy5bo+65OY15Bhv/I8Fxg0u0FYly8y3f4L3IWEe0PDdQE66nIlcW TD0YrTRVp9k3eeQiTkI7SLkHd12fmwd4+JalPt+VKDKGhnQryZpIjo2qzOVLbxBX46975KK00hI rVF9axRlOB7sGbuvI7/zVVm+ahLQ76xtUxiKuuyLUDu/pqclc+xK17fXh5QzUZ8i/1zV7u96TUG QjY08/HQhzQSzNPB2gLbeOGTLPcliKhuQBVI1bbh4a5IMHVsgOQN0R7IQ1O1eRCErI042dyMsye PbHMcjOVDJXkCUKVGMTbCLa75Ibc+XNd8DG9zn0//P/cmamn0vQL8Sqa6naD6LlqJnPE9Md2PHQ /3nh4IqRNbFw2TdB/sXyKejCDgdq8+L64PQhAGzgm/zkBwhL2p4+XnO/TyMxbelUhlzfkFmR X-Proofpoint-ORIG-GUID: nM62GD2398d-UGQ9uWgpiGklQGiTWB6W X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255122169116600 Content-Type: text/plain; charset="utf-8" The user container will shortly need access to the underlying vfio-user proxy; set this up. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/container.h | 2 ++ hw/vfio-user/container.c | 43 +++++++++++++++++++++++++++++++--------- 2 files changed, 36 insertions(+), 9 deletions(-) diff --git a/hw/vfio-user/container.h b/hw/vfio-user/container.h index 3cd3303e68..59e14cf5b3 100644 --- a/hw/vfio-user/container.h +++ b/hw/vfio-user/container.h @@ -14,10 +14,12 @@ #include =20 #include "hw/vfio/vfio-container-base.h" +#include "hw/vfio-user/proxy.h" =20 /* MMU container sub-class for vfio-user. */ typedef struct VFIOUserContainer { VFIOContainerBase bcontainer; + VFIOUserProxy *proxy; } VFIOUserContainer; =20 OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserContainer, VFIO_IOMMU_USER); diff --git a/hw/vfio-user/container.c b/hw/vfio-user/container.c index 6f0eb86a75..38e9cf8d8a 100644 --- a/hw/vfio-user/container.c +++ b/hw/vfio-user/container.c @@ -53,15 +53,28 @@ static int vfio_user_query_dirty_bitmap(const VFIOConta= inerBase *bcontainer, =20 static bool vfio_user_setup(VFIOContainerBase *bcontainer, Error **errp) { - error_setg_errno(errp, ENOTSUP, "Not supported"); - return -ENOTSUP; + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + + assert(container->proxy->dma_pgsizes !=3D 0); + bcontainer->pgsizes =3D container->proxy->dma_pgsizes; + bcontainer->dma_max_mappings =3D container->proxy->max_dma; + + /* No live migration support yet. */ + bcontainer->dirty_pages_supported =3D false; + bcontainer->max_dirty_bitmap_size =3D container->proxy->max_bitmap; + bcontainer->dirty_pgsizes =3D container->proxy->migr_pgsize; + + return true; } =20 -static VFIOUserContainer *vfio_user_create_container(Error **errp) +static VFIOUserContainer *vfio_user_create_container(VFIODevice *vbasedev, + Error **errp) { VFIOUserContainer *container; =20 container =3D VFIO_IOMMU_USER(object_new(TYPE_VFIO_IOMMU_USER)); + container->proxy =3D vbasedev->proxy; return container; } =20 @@ -69,16 +82,18 @@ static VFIOUserContainer *vfio_user_create_container(Er= ror **errp) * Try to mirror vfio_container_connect() as much as possible. */ static VFIOUserContainer * -vfio_user_container_connect(AddressSpace *as, Error **errp) +vfio_user_container_connect(AddressSpace *as, VFIODevice *vbasedev, + Error **errp) { VFIOContainerBase *bcontainer; VFIOUserContainer *container; VFIOAddressSpace *space; VFIOIOMMUClass *vioc; + int ret; =20 space =3D vfio_address_space_get(as); =20 - container =3D vfio_user_create_container(errp); + container =3D vfio_user_create_container(vbasedev, errp); if (!container) { goto put_space_exit; } @@ -89,11 +104,17 @@ vfio_user_container_connect(AddressSpace *as, Error **= errp) goto free_container_exit; } =20 + ret =3D ram_block_uncoordinated_discard_disable(true); + if (ret) { + error_setg_errno(errp, -ret, "Cannot set discarding of RAM broken"= ); + goto unregister_container_exit; + } + vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); assert(vioc->setup); =20 if (!vioc->setup(bcontainer, errp)) { - goto unregister_container_exit; + goto enable_discards_exit; } =20 vfio_address_space_insert(space, bcontainer); @@ -112,6 +133,9 @@ listener_release_exit: vioc->release(bcontainer); } =20 +enable_discards_exit: + ram_block_uncoordinated_discard_disable(false); + unregister_container_exit: vfio_cpr_unregister_container(bcontainer); =20 @@ -128,14 +152,15 @@ static void vfio_user_container_disconnect(VFIOUserCo= ntainer *container) { VFIOContainerBase *bcontainer =3D &container->bcontainer; VFIOIOMMUClass *vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); + VFIOAddressSpace *space =3D bcontainer->space; + + ram_block_uncoordinated_discard_disable(false); =20 vfio_listener_unregister(bcontainer); if (vioc->release) { vioc->release(bcontainer); } =20 - VFIOAddressSpace *space =3D bcontainer->space; - vfio_cpr_unregister_container(bcontainer); object_unref(container); =20 @@ -169,7 +194,7 @@ static bool vfio_user_device_attach(const char *name, V= FIODevice *vbasedev, { VFIOUserContainer *container; =20 - container =3D vfio_user_container_connect(as, errp); + container =3D vfio_user_container_connect(as, vbasedev, errp); if (container =3D=3D NULL) { error_prepend(errp, "failed to connect proxy"); return false; --=20 2.43.0 From nobody Sat Nov 15 15:29:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255172; cv=pass; d=zohomail.com; s=zohoarc; b=D7xniGbnha0qJBSviKZbtVcEgBpuQzXGqeCnOHJpAqxQZqMC12I/Gvw/hgGU9JdPgXYvhKgt+VTbtsjwTYHDOgYiKWHReg3HwI3tqInItcC0oVJxuX11H7ZfNi/5YV0u7Dc7ONxrKK97TH8uAPh+cGfVGxZk0jHDeE5N8Ky0x8Q= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255172; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=bBSUnVU/HbOuDO4pgq6VsgTv/BA1EDII25SIHoHEfgE=; b=Lt63gvRhZHcwWjVz7W7Xzapsot4E26/H7Ym5JD78tNeGMUPIZZHU99XBQSMuSXDoUOxSC5ffL/CV16zZqGlJwZUcIDKSLlnjwJTnAxb4k9c/IAEMFhooORBQu59t1MaPt66BeSNeKGm6Oku97C/al6lEFFGaGl0GWmhvzupOcIQ= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255171994679.9087760705474; Fri, 6 Jun 2025 17:12:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNh9z-00033t-9w; Fri, 06 Jun 2025 20:11:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9x-00033T-1c for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:41 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9v-0006Qo-Fw for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:40 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556N0Rt6023133; Fri, 6 Jun 2025 17:11:28 -0700 Received: from nam02-bn1-obe.outbound.protection.outlook.com (mail-bn1nam02on2126.outbound.protection.outlook.com [40.107.212.126]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 472njsxqb2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:27 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by CO1PR02MB8441.namprd02.prod.outlook.com (2603:10b6:303:154::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.17; Sat, 7 Jun 2025 00:11:23 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:23 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=bBSUnVU/HbOuDO4pgq6VsgTv/BA1EDII25SIHoHEf gE=; b=jfsCnA34hmHgT1vgpqA2kWu/LgrsvOlB0/7HUMJDpZ69glUPvHAOnu08M Tb8NAwLc3pGj9mQ+0GDgeSzgO7A1aQJtltTQtV+Go4xRGDIbE+Dsmt4UVkK//jTb zTIDeaK7go+SD+JNTvvjH3ooC6TwzDylK54GlBI92HKvj7W3f26Cbf2za7hjeJQD FjNXkmzmSnuXiwG6P2mXCXFDfEEdaEpzO3yxJPsD+9nFaggB+e/iI8vJZVMNsiOe YBB1YBfqwREVczaQ4yA8dXKMindt2v1+cbeVP6BmlRZgs8LMwKxtXOPx8lLcQjId CKXM9dGckoovBICv5e9CklfrYrPVw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Xi1UGm+7hZ0z/IdrBynhxqWQk5Ef+fxWA7IeeLqgqapPkVmnxqrx5YkXCAFUK3JXbKVd5JyHP+iVFRYzDyFmcK2MR6Rh+Fd+oOUCjUdShKqhN4jqDXrRLdXNNEcQYRF0D+5IzVCi+pTvjcEegbUe6hafPZzHIzs6UxxVve177XcD9K1eVr3I8deDvhNsAMzfJLQnWhJe+qbMnYRpJGMeNutr/Wmtl/gUpDRSq7lFdfi+Q05+EgVvkXtN7oAbbm323WTb+gnaG4yGBAGk2ZWrG7rEpxjKIHJppQqwSBb9odJcneoKSNoJCM+f0EBCOfvfDxSKCcEESGjhIHpO6Ly7Og== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bBSUnVU/HbOuDO4pgq6VsgTv/BA1EDII25SIHoHEfgE=; b=Ta2C9Ty/t1YFi19z78kFVnMTGaPKZKCOSm0jILWXuQYOPVxyiO98olejSP04dCMaRv+UzOomg2tdv4E98hiOoZ0CWqEDAaELfKUrRLEPyftKqBMuHNznCAWl2wvOw/KoHYUkFD6ujLbKqnzENxJfeYaVOtROLcKjPOQSiHKMGo8fvVD5EpvDDecqk+vxiRnX9Xnj/3jav/Q0B1k6TGIzlaPJ39C7nsX4FcxyH/CQ0IByWjb3lAUdj7r6cUlKCAFKy7nLaAtfpm6oy3Nsd59BETiJGA7Ez9VjUrup+W/W17mKJtfFjrzcpX2iHgZFKNeP0Yxfl7gVTKVihjNm+p5WuA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bBSUnVU/HbOuDO4pgq6VsgTv/BA1EDII25SIHoHEfgE=; b=ZyhRUeKmObNWPthx4gDR5cEUBeRzBj09YwQ9ivf0pK3rnLKN681TvnQoueqd9ajsMuLE6B0iyMausqA1Kb5nnAHb+68hlU53sD3mjonGvg+79g56oFtRSePQU6uBRaocilRMDvhmY2DSfJKZhCnma/dghuFlxYNC7QN6RxV78gnT/l5vxtxLhxqE9qfdX0kcwzcn7eXTdOZlPpDqJ0mZdeYuLNqOB/J4zHqzAOLWQ47f4hIKxqAIzLYyE5mQp8VOjZ1ECAj5bWon5f1KstVOs+YwgTIJ8wp7DPv2q6jK5LZs+QzJTta7IdS7afMLyFfrTi9cHqtkQ6Q+e92M5TfVcQ== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 17/23] vfio-user: implement VFIO_USER_DEVICE_RESET Date: Fri, 6 Jun 2025 17:10:49 -0700 Message-ID: <20250607001056.335310-18-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|CO1PR02MB8441:EE_ X-MS-Office365-Filtering-Correlation-Id: d6478179-2999-44c6-ffac-08dda557d62c x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?dxwHbjU9sqYFZLdj6yCZtu9u1RJoQJqIB/Rk0T8Clyooxo519Zu6f1WIb8UU?= =?us-ascii?Q?EU9557UwEs+WlhV7m3FuXOaIRNSLU6evxFmBoorT7NwEn0gxkePg84+DXMKo?= =?us-ascii?Q?hgfGEBIpsk9GiqRVn+Vpsj4CSceFbRK9Xq7xkeZxj+OQAS1SteguU1dMnUT0?= =?us-ascii?Q?X3C+SarayksAYt8d7IrVOleOKAueCT0mos5QZlPygao1XivSMir4ckLdXzxJ?= =?us-ascii?Q?32CAIooQqBwxsNtxebHrwzeBQCxzbXOLPLRc+SW6sVdXylLZvvuwmto8xkjM?= =?us-ascii?Q?TyUt3OARQ18IyMAdhnm6XRDG6zPxf1d+5MCWX7hN7UDY5ETK5QQRDP9lmhZF?= =?us-ascii?Q?8QuFyrZ4A4vwFbXrnrtkcMQ1zNabclSKrqkaTydnjETVw0b4shemTesYK2bg?= =?us-ascii?Q?zbXCS2NZzf9uyClxio3RXv299+Xf4DdjRRaHG2R/nB7X+jzoI3NtKBV5sq1z?= =?us-ascii?Q?ah7atqxudenLnx5vV50qixPH82OQ1yEXN5AK2h/5d+DBrCKsYys10U9MVCNj?= =?us-ascii?Q?7BJG0DlrNIgjJzl4CIaZT3HnYI+8ZAXT7OhTLxcnQIar7z6bLkdvn8IYfF7K?= =?us-ascii?Q?xU1msAQkLt8s9lyjDArp8RaG4e7akP6gsqn+EGIogqmdErgH51f2sG9UjfA0?= =?us-ascii?Q?Kahe7cr3nRc3EXapkUK11MHLiMWV6wtf1T32ntg/rnNyol/o7259G/Q8xTqf?= =?us-ascii?Q?2ci5oDveOVuElBvR/BYHfZdlBockAiVNQy77oT4hKXariVsQUqgswJkZ2rn2?= =?us-ascii?Q?82NYfDVoWEz7dVxc65OQHe7TLBq0AyuYSSuNKkC+4Q3NSHyA6gz+vTPHfKYG?= =?us-ascii?Q?Swx/1cZhSY07lhAJU6XepFvKBtv5IdrKJqnY8HDNm1tX8RobfmAcd8h4Q5Si?= =?us-ascii?Q?tQRSGTty+EqYKa0/SkIKCAELfxWKh4MmckL9P+tHSZdWRoh37R+LL0d+sRrL?= =?us-ascii?Q?oSuKYP0hiHxZg+15TjtvAxH9/LVm3stAuMnbbaYXms/qb1kV7hUlarPkoJqE?= =?us-ascii?Q?l8+aSr/Hyed4+TB5rnVE3M6GZMktKjY4dBo7IGvWNtD9Hiv79ILRKRpquULb?= =?us-ascii?Q?mCk7WDucdaGkpmRH1mw32cIg6QDypP2nUm+tu5sqa7MSlKfEw0Bc5M8tQ/Wb?= =?us-ascii?Q?VLhXt8JA4r6C3+09enIst22/m4zq9JN2LBWUrnWWCBYI+VpycbBEuCNN2clS?= =?us-ascii?Q?e2MMk1aZvBu/wolIchNFQ93Nsj+sri67Dd65ECbw8qc2KCMz7F6y2d9yI45e?= =?us-ascii?Q?6ykAvcUoLErc2AOEEyF6LJo8Uag0mSz1P5GnS73zYkBhhRjB+zWGcJi7YSgR?= =?us-ascii?Q?gLlQnqZGRtZOjptPlyMzY5GSaDUYtmTJfoXe+bJBBSnFNnxVlBlSPggm2vm+?= =?us-ascii?Q?iqO31rycvEOZc1lAx2DWkYn/KbF7ykA03TsFV2HpP4AtXYIw9bBrHr08wWFg?= =?us-ascii?Q?TIh6urX/ZIs=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(366016)(376014)(7416014); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ltaVVVt1AK1j9bbgoZ+4NMZa2ndJvwqyvhOKThd0K3rKuk2R4Xc/0aP6jcpc?= =?us-ascii?Q?8QEluwiX1Z/YTvP1HboCI++b8JApm0R8n9KwBXGkPnQMtIcTpm4b8/KhxH9D?= =?us-ascii?Q?2uuk1B6OELclY4Ep4LYwRUjGqYHci8MP9J9lLqKkMm7wxIZp17vxAzRtd0/Y?= =?us-ascii?Q?t+90x8t5Adkzygtwt2cw/PT6acy8s6SjT13t5DHValyf5J6h8kDjaiGSwc2y?= =?us-ascii?Q?JguPCAgyp3CXVahmqFRJoN2a1lSSLbbrPO3w6/3feVNAzDFpLU4Qjp9Bcxae?= =?us-ascii?Q?MaiVTIaJ+6p1YK5yssLncDvoQLavvmDNRoC+xCnHTcCDRSkW1SJLmiUNEQ/x?= =?us-ascii?Q?uhq5KSBCWxXZWpyxL4oMWx+MqZy5xoWQCwnD+LcgXp7CbtN3z6IUvJ/RNUE7?= =?us-ascii?Q?VopNtRHbYuXEs/GTMfVrqN+AcyVO8ez4BrkjK+DgZ9EtjKB/qQGSn99ikxG1?= =?us-ascii?Q?cL7aMFlfPR76aiGm4YlG/fIx1DtplszsWNHsp2AskVmk09+SzFyfD/npCUHO?= =?us-ascii?Q?k4O+EwxOCrlQP9CNOBTbplW3qaeWQrX7kOAlrDVfhG5GfLA065EA+quljGZi?= =?us-ascii?Q?sM432jqVq7G7/AmYzeOwiZZkjXzdgWZ6h8DkPFKjxWdR7SAs7fmaJe7tHiX4?= =?us-ascii?Q?Kf5b16XYMrNz1P3y8Wr85U9euAo9jwZqUu47Rf0tsYp7SVyNfm0QjoSzSJj0?= =?us-ascii?Q?7pc32mm1WoAVuPCGYS77+H+Rv+9+cqf7yTQgzRSzxPA9AOYgwbriirIiaXDl?= =?us-ascii?Q?hhIj3RMJ//9JZ0C8o6EXWCZYaYcbtbBUJ2DJFaRdwQPYkszfF+SMp+fYXVJV?= =?us-ascii?Q?zBpe5flzPQxcAHjPKOIhinaH1W0TXqtXXCbGsRsyBsVu7iwTccHETqOcP2LR?= =?us-ascii?Q?6f4Zz8ohhxCc7zEp0toUEACFgQ2QX9ztMbzMGkIC61E4Zv+zUbXHQ6ofK/3T?= =?us-ascii?Q?OvLZqRld1jybvXY+K8Dprsfjxr77hgNuhsZ8/1rkcPSZtYT5d9OKo4eyBgZm?= =?us-ascii?Q?jh59NE35Kl2tfqjngvWYNPe9G3tDAQAvQsP4eATe+/bNlGF23zcj7SN9Of3P?= =?us-ascii?Q?1mshBgtmVaKxo81n+7GTa8vIQSBzRP1iJTQb+1E3NiblxMPk/QFxKvb6OCNk?= =?us-ascii?Q?faGeZNzRoC3CdkBfWIscSOw48qChF5hfhauwkUFhv3zdO3r7O5dJaSC4HRKT?= =?us-ascii?Q?tvDTMiM+RNCaqkxYVvRlAKodBr6ErEW1c4f7CsGSp9mSNyg9bY+IyW/ftwwj?= =?us-ascii?Q?4yxXyiTKJ85rmODQiM6lMR6ZAZ9r2SgAwweTSV4bZDoFb+kdKYnuFO6cX1dX?= =?us-ascii?Q?3d78XSUmx/0kCvet5fMslTSWTY5/vWUW3XTQaCKSnKrTqQq+mR/QtJObazDR?= =?us-ascii?Q?2uhVQ1aj8LwjHF2sI9q1ZPgqteacfPu11K5UlA2nP/kIeVIlB8wWFFn4UPzZ?= =?us-ascii?Q?XtXR3xx/TdfxLhjCgV0CbR18+3CO0v9ZVUEGjfYwUbkYw7Dzlx1vdOVHzGgn?= =?us-ascii?Q?NKDqHQWYQM001nVPUguw2LJ9PvnB2ya58UQUFHt8E9RBfo39FBTueN0A/KTD?= =?us-ascii?Q?yJZR1PeiaDQq7kzwfYEmg5KqaTANokmCV8aQMwIp?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: d6478179-2999-44c6-ffac-08dda557d62c X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:23.2114 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Eh7JFakykTM0hvXnZSXnvrk1qzJvo3n37spGs3c1wpQAqDIpnu9AWfOKRQem7KDJ+BS5gqZuDYuaFArvQ0MHiQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR02MB8441 X-Authority-Analysis: v=2.4 cv=f9JIBPyM c=1 sm=1 tr=0 ts=684383af cx=c_pps a=JomqzcHDjT5Ni/tnCvtrfQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=70NfNXkHNs9el5T2m-kA:9 X-Proofpoint-GUID: nb6y1REpUdZnXUpk4NPxc7WqiLHqy8Y4 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX8vVK0EQwrll0 Vi88pAK1TcYhN3IgUABM45sJO+kTprYxAwkxeimxgJpUzYmkMSJOowZJkyqh7sR7YjNUlzeYBVu zZ9vIfeHDUy6nTKCHAiLNu55QL0rQ8dqoeWEje4pRuF++5l7vzqO+vzdDUELVl/PG70nE1LTw6R HRPMKntlF3N8a4BzTS4tY6afDf6PRtbUmIv6nKQFd8Jcl0Wnqw60Ov4BW08mqi71Q3jSvbZGwQA SQaqvbkGWD4oCEaXQ4cNCpwXGel40YuAwSkXmX+58/27rEAxAH2K4LvoC05jlQ10SwyfOfzvzos 0Q5Cu8CoNrMfOthk6+AX1I8RVS4xmnOiu4COJNi+hwkAiOpMCLO5gLKWNbg973qfu5Gk6XcUzsW 6Olksr4gG1kGD99lBQeOXrvgSs4rb9AyCZAU+wL+vn+AUlIvbUhEnKnuxn8O5hHFpKYG74sR X-Proofpoint-ORIG-GUID: nb6y1REpUdZnXUpk4NPxc7WqiLHqy8Y4 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255172965116600 Content-Type: text/plain; charset="utf-8" Hook this call up to the legacy reset handler for vfio-user-pci. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/device.h | 2 ++ hw/vfio-user/device.c | 12 ++++++++++++ hw/vfio-user/pci.c | 15 +++++++++++++++ 3 files changed, 29 insertions(+) diff --git a/hw/vfio-user/device.h b/hw/vfio-user/device.h index a5a2aa9ba6..25cd4c8a00 100644 --- a/hw/vfio-user/device.h +++ b/hw/vfio-user/device.h @@ -20,6 +20,8 @@ int vfio_user_get_device_info(VFIOUserProxy *proxy, struct vfio_device_info *info); =20 +void vfio_user_device_reset(VFIOUserProxy *proxy); + extern VFIODeviceIOOps vfio_user_device_io_ops_sock; =20 #endif /* VFIO_USER_DEVICE_H */ diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index 45b91d22f0..b37f7329a1 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -49,6 +49,18 @@ int vfio_user_get_device_info(VFIOUserProxy *proxy, return 0; } =20 +void vfio_user_device_reset(VFIOUserProxy *proxy) +{ + VFIOUserHdr msg; + + vfio_user_request_msg(&msg, VFIO_USER_DEVICE_RESET, sizeof(msg), 0); + + vfio_user_send_wait(proxy, &msg, NULL, 0); + if (msg.flags & VFIO_USER_ERROR) { + error_printf("reset reply error %d\n", msg.error_reply); + } +} + static int vfio_user_get_region_info(VFIOUserProxy *proxy, struct vfio_region_info *info, VFIOUserFDs *fds) diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index e055d5ffec..2bc71b9b54 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -261,6 +261,20 @@ static void vfio_user_instance_finalize(Object *obj) } } =20 +static void vfio_user_pci_reset(DeviceState *dev) +{ + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(dev); + VFIODevice *vbasedev =3D &vdev->vbasedev; + + vfio_pci_pre_reset(vdev); + + if (vbasedev->reset_works) { + vfio_user_device_reset(vbasedev->proxy); + } + + vfio_pci_post_reset(vdev); +} + static const Property vfio_user_pci_dev_properties[] =3D { DEFINE_PROP_UINT32("x-pci-vendor-id", VFIOPCIDevice, vendor_id, PCI_ANY_ID), @@ -279,6 +293,7 @@ static void vfio_user_pci_dev_class_init(ObjectClass *k= lass, const void *data) DeviceClass *dc =3D DEVICE_CLASS(klass); PCIDeviceClass *pdc =3D PCI_DEVICE_CLASS(klass); =20 + device_class_set_legacy_reset(dc, vfio_user_pci_reset); device_class_set_props(dc, vfio_user_pci_dev_properties); dc->desc =3D "VFIO over socket PCI device assignment"; pdc->realize =3D vfio_user_pci_realize; --=20 2.43.0 From nobody Sat Nov 15 15:29:49 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255235; cv=pass; d=zohomail.com; s=zohoarc; b=FrTWDQu2DU780ZmOwJPeUoXDBcMrS9dk20o2mUpb2YJiy3eXoSBFd9xcGXUSfhwfxMbaUuV0UVs2I4R8zZI1FS4YXZgN2a2xD+pjtp9eTeN0VFPUz0nPzr0VmmfIPm0BT37mlTLvFsep1o8hAYrp9LN5wX2sIN7EqquTsnOFj0I= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255235; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9s8GGmvHd7YbLvsT1cNsRFjqwTXQbF/N8tWvGuRlBX4=; b=IRnzgUAPagTRNoSzXbPLRUBcsn5x52+cgIgjmlxevc+vp+is4eRNDeqIp4E/rV7B2vPL4GnlM5MFElk8STFYV5hmx4a3HsUqv5iS5g/xw/PdnFHy9kJc95jk4T3e+5INcA7N6IcBKEqzuZwo1Y3TDr95AwNLhaLkvpyu3uumAjQ= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255235364182.6119248547709; Fri, 6 Jun 2025 17:13:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNhA1-00034O-6g; Fri, 06 Jun 2025 20:11:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9z-000348-Sl for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:43 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9x-0006R2-P9 for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:43 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556N0Rt7023133; Fri, 6 Jun 2025 17:11:28 -0700 Received: from nam02-bn1-obe.outbound.protection.outlook.com (mail-bn1nam02on2126.outbound.protection.outlook.com [40.107.212.126]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 472njsxqb2-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:28 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by CO1PR02MB8441.namprd02.prod.outlook.com (2603:10b6:303:154::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.17; Sat, 7 Jun 2025 00:11:25 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:25 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=9s8GGmvHd7YbLvsT1cNsRFjqwTXQbF/N8tWvGuRlB X4=; b=Y2t+ZSEBtZUmnmGIwot7rDasOPywYpZn3nWhNG/N4xwjlt+tTfZa9TlLr sWJJenq9SHz2TNF0rlCofb++w2UOzJSXxClfobYduHI3wJ1n3GhbWsawpInyA+nx 8UWC20iXV8ax2V+be1nhiMx1TU/8CJC9jQ5xCXlYko2Xq3jpSXrxa45dPGkSjdDJ 5uA2zVN4zTXI+YxDbm9IsPr8kfI2FGFvtMUkux57JeRDTIi93uQ3w03ZnVlJGSEF +3gpDlNQBlKamURzoKQcTIN7HPztgGJByRqkde4e+WOhgRyEd1FI6jqAFL8orkXa UTa/P3bFdXUFg/J/tiqFY7Zu/LNPA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=yRyqt9KFcA1e3K9spFbi5dlaxBolS/VRDx/2sKl5r92R1ejgauSANdORV4gHjYdDpEc1FdxSC9ZYX2kqp45HcdozIpE5M1P8T9rv4557e41cGFHIepr0wb9WaIspZBIR+bzY0mqMvHzbr3nK4X7TNx+U3qx0929j4SU7NjSJoohvRRr6onXFWD6nu8u5dz5NUSz5WbfsEdfolTDGyGoHP4X+tQpgxnbwNzrH+FZFe2KdrXoFU0bRaucG7QZ8bKCnhhiAIFNaMsXjMk1DFJ8G4bhzXcH3est6I6WaISE2St2VNHaP6lQ4q6bD77eE1ANqBe5kvtBrwLMZIoiyh1OXZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9s8GGmvHd7YbLvsT1cNsRFjqwTXQbF/N8tWvGuRlBX4=; b=csawmvjo8AdFpzNVAVpJM7YxJyfKVQKKnztZZe2+7H4ZtBhxFX3AnBGbUOrmT9xZnQXAZf6ybdhZNgXNxbe7ctF9wUc9IeIGHCMN43maLj3E+/JUegeLFRfc57LC6OIiT9PHBvdU5ifZS5poOjXE7fEaan3YH49DGM58VMFPmxVy2FTrQk6/aywo1Pl28dw10ORm0FcAwEXdIbVi9BoxWFjJeky987+QGW7A4mRb6WKYIKoIzeN3wbh8fe9uXx85wnXCZXu875hnaLojoQ4AzDmVYctnHS5/n/bUQ9mh9cBL4qXj6zv+FXwCANn/G9owv/3e+1agZgqHpq3kuPmP5A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9s8GGmvHd7YbLvsT1cNsRFjqwTXQbF/N8tWvGuRlBX4=; b=yZvc7sRCG4B5CYlPGTdMAY/mRrmSLQQLEbkI2SIwPlewDU2kb5bQeAZ8r7oqnsJK7WdQV8p8SeTGHuRDCvpcomp96eFan7sYyauNHtRBDWMzb3qlRfz2tBfinBU9DCTKu45sWh6GMgXdDCOb5AtYJ/0M+XEQqEYtWcc6eSKV2LRYkeFHzKuSoWXzXcyaExXuPtHoBRtoQR8XIAVMJtMhLPyxx0qJlYzUBcQzlzsTiFF2XoYEo3IL6l/EludXdAd7V9TEXLy1IJyaTBo7TX9TsFZYhg2Z0RKxjfbFul8nY7ABAqHDpJIHgPOeYh2ePwjH6aIE6U6rgx5J2Wqy66Ak/w== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Levon , John Johnson , Jagannathan Raman , Elena Ufimtseva Subject: [PATCH v3 18/23] vfio-user: implement VFIO_USER_DMA_MAP/UNMAP Date: Fri, 6 Jun 2025 17:10:50 -0700 Message-ID: <20250607001056.335310-19-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|CO1PR02MB8441:EE_ X-MS-Office365-Filtering-Correlation-Id: acf2b94b-b746-4a3d-3dd8-08dda557d73b x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?KHOm3rgVa41x8vlS+uZHqF+pQ0BcDOgStXHJyAeqAxR36N+tO376olDOYtAf?= =?us-ascii?Q?nI1JeC+P1xFDFJnAMCvjD25VAgSMcA39+goZCEb7qRYOKmvIzEQ4vpMsKfHn?= =?us-ascii?Q?0A+3Z6hu3iL9JpZ+muIxhsG68Dg/QpxlR3vWBsvfCRcw21uGkzBMcyKe8a0Q?= =?us-ascii?Q?a+qwa8bseUoV3KrSfHKzYUdG+R3ndfRkWUA8LozuKpQJEOQdReq7v3f/yhDG?= =?us-ascii?Q?09gZ6nAVjvlkIa0KfQ8NyxiOHUge0gvhAqvOJVbBAmyzj/W+XmcOQGU/Za5/?= =?us-ascii?Q?uYQdUiyfLHqTr0u6gWQ5EMexxGRIl+ujPzUaBZjZc+pdTatyqv67eZx7Q38j?= =?us-ascii?Q?4I2gjr0WcqFVZ5p3NUa1SkdMBi8lJsX8hN86yqumWkSbfuHzKEfUoADYTpFG?= =?us-ascii?Q?iJLoEyaF5RuHhSxxv8W079bCALwTMtkS8q9XWg0w3xMKsxRLSdhOoyDl2iai?= =?us-ascii?Q?bwkMIRXOAMW7REuCbJaBRwz+/1qSRkYMj78NxzZ41IVK3l4fxJUoQsPXtjws?= =?us-ascii?Q?gPl7B09PU4ARmBBdM7LDxHHKWviZNL6x6PmYpmWDXehjVGorLsqz+ahPVXos?= =?us-ascii?Q?5Dy2liapfZlrBrqjcj7+LmT2uqrlLU8YHcNYuJDugN9W9K/yTrHpeFpBHGbL?= =?us-ascii?Q?/XNMRC5QaxGIuSNPAoejmPTEsccKG/2XF07SqoB09uZ+1Pql2ou9yCUxEAOk?= =?us-ascii?Q?bTYefH0fTjc5hVYMjOq3UoxRs04s8sWHk3tUZq1uWYXoQf2K7aQ1KT23wqMz?= =?us-ascii?Q?XUTW17QiBafbW8059cIr1v3xaOf9ZqVNVS2fE0L+ryH0k582F8ZN+313MBf7?= =?us-ascii?Q?ljySxQsHDnctxdD3Wp4KwKqoTF7HEgGPnDDkuMk6IuExUFXtzgLFsbQn42Bd?= =?us-ascii?Q?ddI0i4m0HedKMZ/7KnhyY28FW0tTtV91pxEGxRibTKW0IMgPxFiLFZT1s7SJ?= =?us-ascii?Q?ZI2X4iEepDi4dXs1GVrZNTfdutyf88z6UAFRGNEeU2X9784BNz6SUUd+P1QB?= =?us-ascii?Q?YhJwpygWivYNvQmJAOJlmhYmWhu4Fr3Nb/b5YLPd+zQdUYdP8/iw/M3xbCu6?= =?us-ascii?Q?JY9bPToEvZFK+pOVlJMpyvzCOtsKLOAo2hNitT/dywyCz7HxcpTqkHFbtMhk?= =?us-ascii?Q?HGeB1PKZnY+xZpHwZUpkKLyucn7/cVlUBDmRSWHhPAd/UGnBxXOE3eDg/b0n?= =?us-ascii?Q?BbUVp1EufQ1l0to+z/cREIAWuM2W+LoBMVbGppyfu0yTSrC2WO0V4EICUNbx?= =?us-ascii?Q?Y+boQf2F5iNmVRlIt7UUX3SXiC8KKLGRfH6uDgDaW9x9F/iGN2Xj7iq0441e?= =?us-ascii?Q?W/PRSOE8ZFoXDKNrQG5XlAQ/SwWtKm1HbnafsuCnGhyXnvGlC4zK7n7I469m?= =?us-ascii?Q?z258PYKpXjTmPcR2vpekrBCazmCtrk39azMvIItyf9S16WcmBZfN1rZmf7Of?= =?us-ascii?Q?9cOF8RDEp1A=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(366016)(376014)(7416014); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?bykdDqr2knG2iPXfKC/lUpr9Hb+7sMfH4StnU5cT78ZOZH3XbrDqhjGMt0g+?= =?us-ascii?Q?w1+r/SmqoYYLTS159ZttN50O9uRy33UAUti30b9+tjHw1VRPrW3sdY7iBrw3?= =?us-ascii?Q?jhajQl+J1AdKi+UKpTmT5hpKxSW9oh2d/cOCDgqfgHPn3jGOnlxm1ho0VD91?= =?us-ascii?Q?gE3Er7Nrbx4WNNHKOIi7giWWjVbMm37SK3wXHfkw1WwkwXmpF5TQ9Aj1YraS?= =?us-ascii?Q?84GrE0o8rw6I4M1wcFFahF0vL8/WOlhp2X+CZLMkAdZCDrV3+Mop1wWfztOv?= =?us-ascii?Q?XGsX9DlYqNoElCsOz21HHFlP8uKV38+02K1mYClad3dd3eA5gz+8QUak55ZM?= =?us-ascii?Q?4bv1NCDwyZ3Xpv9Pz9Zy5dCb+SU5vTrDSYtXYjJH0IJ89BAVhxl0g4hFg7IR?= =?us-ascii?Q?tJZqwm+FzGpybVVdAF7Gtsw39afc38en8hE5SCIN42NrgANdKPRCc5qU26iP?= =?us-ascii?Q?ePU7cZjOxOe9A8Okj4fUaWhBmKBa1jG+QFNCZ7sf6sSF7IDCnDh9vFGv7oHa?= =?us-ascii?Q?Wq9hl/eoJQEAECbPVt6ZWzosLli118IgUQmptiyM3Tc2qXRd0/bj1TcMcq1O?= =?us-ascii?Q?bwbBRLdhiVjvnveP6/Jxpn5MikrQp5S4KUnvgVO/Y6B5RheFmYcZsOgdFoCD?= =?us-ascii?Q?covTrgzA+49DK266/RnR4g/cdwaTyoeaHfUM/ps+g+lHrXe95A+NEMVQTNE5?= =?us-ascii?Q?NvT8Z6kb4FylVzcRfHQJZOlA9dvoMMNw6iU+5+ioD2shyLb3JtMwaAmlaSi2?= =?us-ascii?Q?ecB8se+Qp8xRQiksiyMV4Bk+R6w0OPl1XhPO1pOZSG574ih49MVedWipzatR?= =?us-ascii?Q?pvSHMn802nPC0XETcwE8Li4zVdMF1SCv5/rJvXT6npj2szRBTi616M74ADcS?= =?us-ascii?Q?xC4CxMwvlneJ114nwsb+lMK1sbgqJxCQ2b2JL3g/vyZnOY5fF8rHyT8Cjgfk?= =?us-ascii?Q?a6meB0TBE1tYMXrpegO8rGKkeC/vwqSnutQB1akltwTWlrTZruyIkwYD/8Zn?= =?us-ascii?Q?2gktJgtBk+QwL8m/QLZGby4MAtmqdGlQkzUZ0FVaJ6Tnnoa8j3JMCq9h4Q4w?= =?us-ascii?Q?f3oirpo+fKgVK5rIeYaqYA6kwTAfiZhJ+us6NUpu89gfIEXgJTzCerPIfkfE?= =?us-ascii?Q?xlG3rSKPZeQo1C9OjAZBHE329VXOe9XlP+nmSTmtDlcJAJCJNsN85wJGO9U8?= =?us-ascii?Q?ICUNpNJe66S910MLyPgFpNZf+iHSVYR1SDdW0mE4sIqo9xheJGThYcxer+2m?= =?us-ascii?Q?+obTv3rcGtX4HVkrh9BLqRh8wPicW9AU0OKxm2rmd1JTcrvjtnomxcb45+C0?= =?us-ascii?Q?175zC2+qFL47BupPuluxLpFeSrCu/IqDwO6bk3ZHsedoJ+TuqIH38r7GD9IH?= =?us-ascii?Q?bRfsu/n0KKQfLhNy3MegkKn7DM95SEuCbZAKg+TLhTKizumE6H7f1hLcV/jM?= =?us-ascii?Q?GADsbCGF0Z41QSteMH/7tDg/6bDsDGFbWv3NO426FqHJKtyeIW+g2oO5jJe3?= =?us-ascii?Q?6li1kJg9kHp6GXJ+JUYyIJAVI1aEghB/qo1CWYJqtfw/CkYWhDhtQy6SQ7Y7?= =?us-ascii?Q?baXIpEDk+J9Qe037/bpQh8Zi7BVtzdGLUIfog3aw?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: acf2b94b-b746-4a3d-3dd8-08dda557d73b X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:25.0387 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: p7y1n1dgQK4BaNMh2UtdkT82L443aCnDnQTN2c2qTcoIK35JKwVxUwR6Rh5QXSumMzbpNoBH/T7yG7caO63j6A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR02MB8441 X-Authority-Analysis: v=2.4 cv=f9JIBPyM c=1 sm=1 tr=0 ts=684383b0 cx=c_pps a=JomqzcHDjT5Ni/tnCvtrfQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=RQZ_2NmkAAAA:8 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=QltQKaWwH6ua0DdkohUA:9 a=46pEW5UW3zrkaSsnLxuo:22 X-Proofpoint-GUID: -un85e7K5-t7RPu05QAYDX7gTiOVlCcl X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX5Tm+xCAeihnY ECRrNfrQ1QE6EMEFBY4iML/oH/fCjNMehHD0tE+TyXlDCyVdm9S1wL/Hu7MErPtc78G551DNwUT Y4jImeO1JfPSh4DXb8HlAw/lbjWV0oTF5ihtHjtJd0rdL1ezftXGfGu5y6zopoa4oyl/MJtz2NH t3kcek6Fgt51hDY/7PH9Z2wXRoCquEtZ4eYmGYcRk0HpHQqYcshmN0FmAC+eGTbXGZMwIQpMYS6 /RHOxZA9dSr5lCXNk2VYfxDYYFpwlR2SukhzPYyQ3rjRP4Lw4oBCijipzRlo0aEjrGXW2Wnn3IK NBHWr40ofxOlGxErZpApqxIDo+IlnOrdQJ9QfkmBA0uKHFi/nE1Wilgyjm9hf5fzSL9cbuZMs1j ce/5+o7arC8CjyQ0X8XghUIYnDBRlN3G/Rx60ilTdBqYcpKSa5HDDp9aQutGRc6YDbBHzYK9 X-Proofpoint-ORIG-GUID: -un85e7K5-t7RPu05QAYDX7gTiOVlCcl X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255237472116600 Content-Type: text/plain; charset="utf-8" From: John Levon When the vfio-user container gets mapping updates, share them with the vfio-user by sending a message; this can include the region fd, allowing the server to directly mmap() the region as needed. For performance, we only wait for the message responses when we're doing with a series of updates via the listener_commit() callback. Originally-by: John Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John Levon --- hw/vfio-user/protocol.h | 32 ++++++++++++ hw/vfio-user/proxy.h | 6 +++ hw/vfio-user/container.c | 107 +++++++++++++++++++++++++++++++++++++- hw/vfio-user/proxy.c | 77 ++++++++++++++++++++++++++- hw/vfio-user/trace-events | 4 ++ 5 files changed, 222 insertions(+), 4 deletions(-) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 920b9c44ec..4fa6d03b3f 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -115,6 +115,31 @@ typedef struct { */ #define VFIO_USER_DEF_MAX_BITMAP (256 * 1024 * 1024) =20 +/* + * VFIO_USER_DMA_MAP + * imported from struct vfio_iommu_type1_dma_map + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint64_t offset; /* FD offset */ + uint64_t iova; + uint64_t size; +} VFIOUserDMAMap; + +/* + * VFIO_USER_DMA_UNMAP + * imported from struct vfio_iommu_type1_dma_unmap + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint64_t iova; + uint64_t size; +} VFIOUserDMAUnmap; + /* * VFIO_USER_DEVICE_GET_INFO * imported from struct vfio_device_info @@ -178,4 +203,11 @@ typedef struct { char data[]; } VFIOUserRegionRW; =20 +/*imported from struct vfio_bitmap */ +typedef struct { + uint64_t pgsize; + uint64_t size; + char data[]; +} VFIOUserBitmap; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index e2cfaeead8..01fed3f17d 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -73,6 +73,7 @@ typedef struct VFIOUserProxy { QemuCond close_cv; AioContext *ctx; QEMUBH *req_bh; + bool async_ops; =20 /* * above only changed when BQL is held @@ -102,9 +103,14 @@ void vfio_user_set_handler(VFIODevice *vbasedev, void *reqarg); bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp); =20 +VFIOUserFDs *vfio_user_getfds(int numfds); + void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, uint32_t size, uint32_t flags); +void vfio_user_wait_reqs(VFIOUserProxy *proxy); void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, VFIOUserFDs *fds, int rsize); +void vfio_user_send_nowait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize); =20 #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/container.c b/hw/vfio-user/container.c index 38e9cf8d8a..5a109c591e 100644 --- a/hw/vfio-user/container.c +++ b/hw/vfio-user/container.c @@ -21,18 +21,119 @@ #include "qapi/error.h" #include "trace.h" =20 +/* + * When DMA space is the physical address space, the region add/del listen= ers + * will fire during memory update transactions. These depend on BQL being= held, + * so do any resulting map/demap ops async while keeping BQL. + */ +static void vfio_user_listener_begin(VFIOContainerBase *bcontainer) +{ + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + + container->proxy->async_ops =3D true; +} + +static void vfio_user_listener_commit(VFIOContainerBase *bcontainer) +{ + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + + /* wait here for any async requests sent during the transaction */ + container->proxy->async_ops =3D false; + vfio_user_wait_reqs(container->proxy); +} + static int vfio_user_dma_unmap(const VFIOContainerBase *bcontainer, hwaddr iova, ram_addr_t size, IOMMUTLBEntry *iotlb, bool unmap_all) { - return -ENOTSUP; + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + + VFIOUserDMAUnmap *msgp =3D g_malloc(sizeof(*msgp)); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DMA_UNMAP, sizeof(*msgp), = 0); + msgp->argsz =3D sizeof(struct vfio_iommu_type1_dma_unmap); + msgp->flags =3D unmap_all ? VFIO_DMA_UNMAP_FLAG_ALL : 0; + msgp->iova =3D iova; + msgp->size =3D size; + trace_vfio_user_dma_unmap(msgp->iova, msgp->size, msgp->flags, + container->proxy->async_ops); + + if (container->proxy->async_ops) { + vfio_user_send_nowait(container->proxy, &msgp->hdr, NULL, 0); + return 0; + } + + vfio_user_send_wait(container->proxy, &msgp->hdr, NULL, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + + g_free(msgp); + return 0; } =20 static int vfio_user_dma_map(const VFIOContainerBase *bcontainer, hwaddr i= ova, ram_addr_t size, void *vaddr, bool readonly, MemoryRegion *mrp) { - return -ENOTSUP; + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + + VFIOUserProxy *proxy =3D container->proxy; + int fd =3D memory_region_get_fd(mrp); + int ret; + + VFIOUserFDs *fds =3D NULL; + VFIOUserDMAMap *msgp =3D g_malloc0(sizeof(*msgp)); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DMA_MAP, sizeof(*msgp), 0); + msgp->argsz =3D sizeof(struct vfio_iommu_type1_dma_map); + msgp->flags =3D VFIO_DMA_MAP_FLAG_READ; + msgp->offset =3D 0; + msgp->iova =3D iova; + msgp->size =3D size; + + /* + * vaddr enters as a QEMU process address; make it either a file offset + * for mapped areas or leave as 0. + */ + if (fd !=3D -1) { + msgp->offset =3D qemu_ram_block_host_offset(mrp->ram_block, vaddr); + } + + if (!readonly) { + msgp->flags |=3D VFIO_DMA_MAP_FLAG_WRITE; + } + + trace_vfio_user_dma_map(msgp->iova, msgp->size, msgp->offset, msgp->fl= ags, + container->proxy->async_ops); + + /* + * The async_ops case sends without blocking. They're later waited for= in + * vfio_send_wait_reqs. + */ + if (container->proxy->async_ops) { + /* can't use auto variable since we don't block */ + if (fd !=3D -1) { + fds =3D vfio_user_getfds(1); + fds->send_fds =3D 1; + fds->fds[0] =3D fd; + } + vfio_user_send_nowait(proxy, &msgp->hdr, fds, 0); + ret =3D 0; + } else { + VFIOUserFDs local_fds =3D { 1, 0, &fd }; + + fds =3D fd !=3D -1 ? &local_fds : NULL; + vfio_user_send_wait(proxy, &msgp->hdr, fds, 0); + ret =3D (msgp->hdr.flags & VFIO_USER_ERROR) ? -msgp->hdr.error_rep= ly : 0; + g_free(msgp); + } + + return ret; } =20 static int @@ -224,6 +325,8 @@ static void vfio_iommu_user_class_init(ObjectClass *kla= ss, const void *data) VFIOIOMMUClass *vioc =3D VFIO_IOMMU_CLASS(klass); =20 vioc->setup =3D vfio_user_setup; + vioc->listener_begin =3D vfio_user_listener_begin, + vioc->listener_commit =3D vfio_user_listener_commit, vioc->dma_map =3D vfio_user_dma_map; vioc->dma_unmap =3D vfio_user_dma_unmap; vioc->attach_device =3D vfio_user_device_attach; diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 34df9bba7b..6d1446c38b 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -31,7 +31,6 @@ static void vfio_user_shutdown(VFIOUserProxy *proxy); static int vfio_user_send_qio(VFIOUserProxy *proxy, VFIOUserMsg *msg); static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, VFIOUserFDs *fds); -static VFIOUserFDs *vfio_user_getfds(int numfds); static void vfio_user_recycle(VFIOUserProxy *proxy, VFIOUserMsg *msg); =20 static void vfio_user_recv(void *opaque); @@ -133,7 +132,7 @@ static void vfio_user_recycle(VFIOUserProxy *proxy, VFI= OUserMsg *msg) QTAILQ_INSERT_HEAD(&proxy->free, msg, next); } =20 -static VFIOUserFDs *vfio_user_getfds(int numfds) +VFIOUserFDs *vfio_user_getfds(int numfds) { VFIOUserFDs *fds =3D g_malloc0(sizeof(*fds) + (numfds * sizeof(int))); =20 @@ -609,6 +608,36 @@ static int vfio_user_send_queued(VFIOUserProxy *proxy,= VFIOUserMsg *msg) return 0; } =20 +/* + * nowait send - vfio_wait_reqs() can wait for it later + */ +void vfio_user_send_nowait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize) +{ + VFIOUserMsg *msg; + int ret; + + if (hdr->flags & VFIO_USER_NO_REPLY) { + error_printf("vfio_user_send_nowait on async message\n"); + return; + } + + QEMU_LOCK_GUARD(&proxy->lock); + + msg =3D vfio_user_getmsg(proxy, hdr, fds); + msg->id =3D hdr->id; + msg->rsize =3D rsize ? rsize : hdr->size; + msg->type =3D VFIO_MSG_NOWAIT; + + ret =3D vfio_user_send_queued(proxy, msg); + if (ret < 0) { + vfio_user_recycle(proxy, msg); + return; + } + + proxy->last_nowait =3D msg; +} + void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, VFIOUserFDs *fds, int rsize) { @@ -647,6 +676,50 @@ void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUse= rHdr *hdr, qemu_mutex_unlock(&proxy->lock); } =20 +void vfio_user_wait_reqs(VFIOUserProxy *proxy) +{ + VFIOUserMsg *msg; + + /* + * Any DMA map/unmap requests sent in the middle + * of a memory region transaction were sent nowait. + * Wait for them here. + */ + qemu_mutex_lock(&proxy->lock); + if (proxy->last_nowait !=3D NULL) { + /* + * Change type to WAIT to wait for reply + */ + msg =3D proxy->last_nowait; + msg->type =3D VFIO_MSG_WAIT; + proxy->last_nowait =3D NULL; + while (!msg->complete) { + if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, wait_time)) { + VFIOUserMsgQ *list; + + list =3D msg->pending ? &proxy->pending : &proxy->outgoing; + QTAILQ_REMOVE(list, msg, next); + error_printf("vfio_wait_reqs - timed out\n"); + break; + } + } + + if (msg->hdr->flags & VFIO_USER_ERROR) { + error_printf("vfio_user_wait_reqs - error reply on async "); + error_printf("request: command %x error %s\n", msg->hdr->comma= nd, + strerror(msg->hdr->error_reply)); + } + + /* + * Change type back to NOWAIT to free + */ + msg->type =3D VFIO_MSG_NOWAIT; + vfio_user_recycle(proxy, msg); + } + + qemu_mutex_unlock(&proxy->lock); +} + static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); =20 diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 053f5932eb..7ef98813b3 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -11,3 +11,7 @@ vfio_user_get_region_info(uint32_t index, uint32_t flags,= uint64_t size) " index vfio_user_region_rw(uint32_t region, uint64_t off, uint32_t count) " regio= n %d offset 0x%"PRIx64" count %d" vfio_user_get_irq_info(uint32_t index, uint32_t flags, uint32_t count) " i= ndex %d flags 0x%x count %d" vfio_user_set_irqs(uint32_t index, uint32_t start, uint32_t count, uint32_= t flags) " index %d start %d count %d flags 0x%x" + +# container.c +vfio_user_dma_map(uint64_t iova, uint64_t size, uint64_t off, uint32_t fla= gs, bool async_ops) " iova 0x%"PRIx64" size 0x%"PRIx64" off 0x%"PRIx64" fla= gs 0x%x async_ops %d" +vfio_user_dma_unmap(uint64_t iova, uint64_t size, uint32_t flags, bool as= ync_ops) " iova 0x%"PRIx64" size 0x%"PRIx64" flags 0x%x async_ops %d" --=20 2.43.0 From nobody Sat Nov 15 15:29:49 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255304; cv=pass; d=zohomail.com; s=zohoarc; b=OUhtO6q5TXuJ9X4pVeiVcXZoXkfNmBjC/Bz//SWwmB1Ji/F/Fqyez+fpZ84RTkn1PwkwWHQRxs3QYArihykOamVgOWqX2xFcbdWwKIFA1BzktIRAdk43XP4Q6qEwqSn3VpxhgExkyMhSS/R35tYZL1c/55pJnAsyGK/uXMm88MI= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255304; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=H+XM4/n1ESevLcnSCydVtUso4uc22MDK1snAtVmW6KU=; b=QbjqxWeaw1fsqWGAXvzkbns3O/6gg81UUm/sUNmOo/p5CLEcWsoq1qjVP9wKPB2Zr99Ph/kkS9+AJRklPIwJPaCe/PpDcRwTYSVW6DOvlAE7eXhHEXXbr34KGsbriPJ8G5tYavRJiOYTangXrEv5tuv0ZC6vVTnSx8gl9RE5X3Y= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255304648692.6569800096335; Fri, 6 Jun 2025 17:15:04 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNhA1-00034T-Ey; Fri, 06 Jun 2025 20:11:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9z-000347-SI for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:43 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9w-0006Qu-Je for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:42 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556N0Rt8023133; Fri, 6 Jun 2025 17:11:29 -0700 Received: from nam02-bn1-obe.outbound.protection.outlook.com (mail-bn1nam02on2126.outbound.protection.outlook.com [40.107.212.126]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 472njsxqb2-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:29 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by CO1PR02MB8441.namprd02.prod.outlook.com (2603:10b6:303:154::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.17; Sat, 7 Jun 2025 00:11:26 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:26 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=H+XM4/n1ESevLcnSCydVtUso4uc22MDK1snAtVmW6 KU=; b=t8UApXzU25K3U+f/iGiO3sr0KyVxAeH68AHpV+1NkX0enqSx7SEvY/0qE eX+RnTzPcvn0xpigy4A7avERpVUXZPgW/1xrUUBddWOY5Z2eLlRD/GZYUHRE2juJ HoQDKmSAzHnrAmnihOJfm68U16k/FD1NmGKnD3EKVC/lJ3LfqSd29c71iCkTU/gF nNGAxBzFOb+SzUQbDlRNbx9sS92BjmFPDYg2nLQ2c/NlJgqLffyuKO3xamMkWnaM A/lal5d6z/LzOH/zU1FZ4Nxese1QPzqI2AvFrPEAvExAGqu1M9FNLJlOfO74Pa2C RNIfFWZiFEq5sNip7Ym58+Hfeap/w== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=eyK9+VGvMIETB7TWPrt4LPMYnFGIM3sdTjhvatxWjOe44Y4OOJT/2zW7UP9fUbqaeP0DD960iFz+ry665jbJcOor1tFXf/s9deskFWZJ4J6vEBQt9ATItR/KayAtX7lpoNUFYkmU+3sIKWtWTIgF5lBmjzizpAdC/AWn/84m6JMU6PJPnGOw/Pt9JbdjWwSSwSii6BtI7Z77JyjZ510BU6G8Y3PJ26J+yNo4o2ZD26aoA9etDU0SsnkJHYKfBnNERYvF5TZJg0kMfJA9K/7mZ9fHbpZjPdP04vnFYLP1N5xuQ4bVb9mEvB7QU0IDYxQUxzJd6DZ+dHorb7394QupkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=H+XM4/n1ESevLcnSCydVtUso4uc22MDK1snAtVmW6KU=; b=fqLtvPFU3ysDMZQLz7lyfTwskk1vOei5sgGXZkqYVAERTF9d2yUdyGAkfBFPiaL4HumOVPDgdjRGqXVM3re6nmD0F609WhzBM5KT8pnGihwFSH/tV867ixQMi1yWgyDj75eVkvvGPamz7Asn+db0B7NRnfrARN4bSFor4AWoPzJxP6yARDV+fn0L6Dty06PjG4HXmkIewcaAHPLupMDkNSnEBvfeMjtUj8bcHyJWWbfGtcb56G+ErXL7KM6GKOKxG49+lCNhn2Ry3l0Y2PqvQVpxcmhONK6KDy43G87lM4dGrV3TUfoIxKShBmA9XBZJB+L6pQDGMZPZWHKN8IFkhw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H+XM4/n1ESevLcnSCydVtUso4uc22MDK1snAtVmW6KU=; b=QpcndXHRdisuh7K36dVM9mxELGAUiNsF0/C7xBqyHWwIkU+Oh4wLE3E54UMWJUs/90oQuA01lLbjspGYS0gyy/Nt95G9PSo5OLSLkS/9Gq7EBJ19wo9K2XhZgL7XiIEYoGtAVrljic0y2rkQxW+FZwiA6xvRQ6Azp62VwfL4R6qy3xLOqnYiCp13zysJQ29Xnvz1WhAxKb4Rs0TY/3N4QCKyz45NkMWWb50TAPtTtdemakrAxP899N3QNvsX1OikZ6cCqiQjOHrybWYPwubsRPGt6SVWJJk2mcatEcwKQkZMmt6PC/9yxKemze8a4JVeompEPn+3tIBw6bUYjd/ztw== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 19/23] vfio-user: implement VFIO_USER_DMA_READ/WRITE Date: Fri, 6 Jun 2025 17:10:51 -0700 Message-ID: <20250607001056.335310-20-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|CO1PR02MB8441:EE_ X-MS-Office365-Filtering-Correlation-Id: f926a1f2-69ee-450f-26b1-08dda557d818 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?MQoZ73ZJrrDruM+3l76JyqiwiprCehFOCLKwI8Wqj+LYF/sIuPlaN8xIxKT8?= =?us-ascii?Q?U8msPO4rpwMyRqW/oczLiLMGZ96oixy3CXbI415h5bhDw7yBHgUK86jUDQDq?= =?us-ascii?Q?sYKUJVLX45Vg7B7T0rR+i+zG0px3ULEm07cTI5YIkT/C+OooFG9Q5Lx9O+hj?= =?us-ascii?Q?87GsZr6gVZG4Pw2tPhyVcDek1E7JU+M7+C2IEusTkt2j0oGsLGSFvTD7nvpE?= =?us-ascii?Q?GYYBqIuE6vu7SYMLEepFhgtgxJT+Sm5gW7dwFnvy9AfzuJlfOMwwITB6rv/r?= =?us-ascii?Q?1E0/I6VLpLEfvUv5pdQSjsxAoOOIirKY1Dg+Qux37xzTWWKEkIC2bd30LVhU?= =?us-ascii?Q?rUov8AUhYJ8gNVTh0F+8/eST/HAmG2peuZ+1o4v+0CU8a8f/rEnScmAxnJbz?= =?us-ascii?Q?7s7JY/H7uT51nf2YJyUyTy4/4DyRyZFssBZV13vHxuk7hDXLiHt351wRldfc?= =?us-ascii?Q?QGOsVVi7SabBV4LbLcdhdPctANDHez50ospYDTcIMNmqeW22ZE2ljS0zZnaK?= =?us-ascii?Q?SGW80aB2y+v9NxOY7MfCCGSfWct02U6uu7A2nYQB/KZ50DOoM6mrNlLArRp5?= =?us-ascii?Q?lXoCFLipbYRCRrBPQINJlREtUq0PtIXv7qrgVWRDV+mJ7XHKS6jgZUxLdPf4?= =?us-ascii?Q?qBxl9X+2wjQ42LasvPGGVbfcXN+QqOnbslvHy95R8Mhm6IcX7wpFG0eGz7c/?= =?us-ascii?Q?5CDZ6D1QoqSLMZqNQXusfKQ8UJahSnhbhYsXlkfDjYzaKA0mRRk+KdgHGp0S?= =?us-ascii?Q?LQziLNpjpcg81yOufb87wed2UNoIWvc6oaNItGnbU5NYlG1Ikvgz2KkTJhjK?= =?us-ascii?Q?qRXZNs4l655/xQ2ULAax5nDYbsL19syozKmXkc1pFCHvwAVpN0aempkobA4K?= =?us-ascii?Q?cfoGJfQpakKPkztpaur13dsbVp6JsCBepvPGx75uRBWtyemMWu/gseRSmO26?= =?us-ascii?Q?0CbOxK6MHmAvNMml7e+gnjXaWVh5eckpuJFmtHfkxVveqb3TramZPmqTbi7n?= =?us-ascii?Q?WmyQtovMffCqxpAAu9iW96dq5SJKJ3W57vbIiZi2UKTDmtogQnerNmXSCoP4?= =?us-ascii?Q?4imgnMtE637shlpJhYly5o9KlscdNd/u4zzYkJBTHuBX1OgrSmgLkJVJTNi9?= =?us-ascii?Q?5hcPUHE6Boc+810sJV/kOq1O5o6uLKFBPmKmxFj8sRHcjbLL2GoiOphJxi6z?= =?us-ascii?Q?IIT9dPkFAbeu2TxoQuuH6ZX8AKebwy5tVFXeuq/AukVsg0nc2m9ZrqTsRmMx?= =?us-ascii?Q?7loQDCibgz0wl0bUuGEa/toEvBNML6jcO3J8IiQrVpqxsZbrEEJQqzrI9rWX?= =?us-ascii?Q?hUO0nqlxexqjkI04UahQm8W5Jl+Od31toTT1IsH9qRVkpwtBi+6318TgtL2f?= =?us-ascii?Q?yZ9QqyUx2km50KxO1b3oABUwX/sMAG67FDoeuhaUv4wQD/4DEg=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(366016)(376014)(7416014); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?/4Wyywoqk6kxSEJW/baCrP5KuEbU4H+KB791X8zCgB3lrcpsQArHa4CwKw/2?= =?us-ascii?Q?5WEThMTxXqnFRJEll2EqcRrRk11cUf0uDmo+Q3bf6MpEfIQqI52JB6Uev5sW?= =?us-ascii?Q?QreLtO12k7vfpJi3N6j6wkxf5or8Egh1+rpwSs+vaLdBmdqZDqmJXxEwJqgU?= =?us-ascii?Q?gpyh/Nm0k/+tJnRXEyCipI7+t+FCFgzfN5ifZs7J0WA18KvmLga5GR74tTwO?= =?us-ascii?Q?S7yRljIfFegLoY+nMHFD1MkNLaiX38ENr2/NTkcYH+jnuMyCcquVuf95Ddc9?= =?us-ascii?Q?OtOngRqKEoGVTtMYdCDbUrhkRtIYaOAuKadJyUaEfcusmYjnzkZGR7I9QKym?= =?us-ascii?Q?vSYdkurdh7On3NJBDhMj7GTuf8HeMeAymW1tYZJ2AYjgmiHm9VVpfjpGG7uX?= =?us-ascii?Q?e1XIyqyRKVUOkOStcD42BI4ZM9gQqhZlaCq8sIOEEu5h5OEaIzHNJMk4zcwE?= =?us-ascii?Q?ksnoBiJ8kk7sFgONi/o5WRCAc4e9MUILEwmUGXVsFgluDZR3DdlkKJy6Deam?= =?us-ascii?Q?qbWrwseIP+CQs2BfQD/MCfT5IZ5miTXAwmXLnWP7uaOC1mDjAGyDvUU6kn9J?= =?us-ascii?Q?YZTuXQx3CPT7RfdB0vnlRg+CC7qDZLFJiy+8Em9NH/2pIRv27bj/F5A3hWVz?= =?us-ascii?Q?sywZVNkn0V2+1laSBKiDScyj5r+sl2XBflY0oepXTL/SoVuoQBZmZwJ+9Rug?= =?us-ascii?Q?5PZN+UZnE657qaUy6cIx6x1RjT+kEqO71p5gViXiZallncRYgrg+13YhcSoa?= =?us-ascii?Q?uGdxr/tu1OM+6C1Xgz6zsvEK5mn2jWO6d1odyxV06pVFH+VgGUI4YIRPPtcp?= =?us-ascii?Q?dRFVqdVT0nUf58dD/snWREnprWLLlQCaVwMMxv4bHgMmrFvDFQL4Rn6vMEw6?= =?us-ascii?Q?n4fkrb6NBNnb+qVSA2d4gS5//em3pQvCSGURz6FrUXVJDPay7o9n1OMY97m1?= =?us-ascii?Q?kLMNB7yV2m6Ssj+70bfm4t6BBDiIV/kS6TK75l477DCDZwhlMl9X0xZKzmhN?= =?us-ascii?Q?hB4grXxAbS6AlKnTPs/irdhXI6RuqnX/VuaPBpgc6Rl/d87mzpUifE8azLsK?= =?us-ascii?Q?mXf04A7HQPrXqy2IrvqZPv/ty/rdWkBwidw1iioXweggDIPbX8rMj+eii5Yt?= =?us-ascii?Q?Pukvda3HleMoP1JadxMA9rk8Hak8i9LviHDq4dXGv3gIF/ux5OLHBm063mVj?= =?us-ascii?Q?lt9Pl6I7T2pJ3x2KdVxXHPx1TcyzljPWY+0Q1EOV0Ge6dvBuIx6OVIjFZM6g?= =?us-ascii?Q?TluQme4rj9SMl+1A5tnZhFUlvcSX9JbYQHxSRdh/2UtHQvHrDqyLRDFzj0ub?= =?us-ascii?Q?E/7HfhPm57lPGHg4rzr5eaL5N9UiQZfNGGMlONLAu1BKPkMyNxRoCyhrI0PB?= =?us-ascii?Q?HeAthUzIFed7MtZUCQCJdSgP+IBF4j+3oQQ0cbWG8LDAQLpVVhnS2jzYYD3V?= =?us-ascii?Q?q8BdIZEoTQDzVe97+g7D5Engk7l2l3dNh2UXzanF88YK5He2q6S53Mll/I3T?= =?us-ascii?Q?0zBSNFU3hHRpRLOPOQnrtw1TUjpiQhenMovg0cKjvsvG9Ru4ahNE+A8HU87d?= =?us-ascii?Q?wYOQc8mP+B7CBzB3exo+VE/VMHNhaNaZ5bkC4WJ9?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: f926a1f2-69ee-450f-26b1-08dda557d818 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:26.4248 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zEiHmPe5mTt80KoddCTipnd/+rhcRqnL8tcyX2ursboMjSQJLX0hgQrURiPyulW9VH7tDpNlqgWX+UC/fckfIg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR02MB8441 X-Authority-Analysis: v=2.4 cv=f9JIBPyM c=1 sm=1 tr=0 ts=684383b1 cx=c_pps a=JomqzcHDjT5Ni/tnCvtrfQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=qTvU7B1ccVJYr7cy3AMA:9 X-Proofpoint-GUID: eQkxP92-Yq8lGAZvZgPkf5ut6Q9fyEbW X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX9nILEL4K9GMh 1EdNSs5o6GabZdRqFXymaFFVPce87jkRupEbv80JjbTzhepQtJttJBTHk8sSE+Ai9mag15KwKYH K2/GkHWeR323X8S/1oGmyQAOOG9Za41KC/uK0BMvcFlYWnVe63qSbUqDsw0aQBgm7HV0RJVk89r vvvzqkuTmsewsSwWRVRyBmxFLV/7K/fuFkWCaYaA2tRk6qXPtO+/+wyjihcsSNO7mqbjSJrpquX orEAREnswUBbmogNOjWcNL6EKO9cF1dHbwDgeqfXipEWPYpGEbzqHAUQXGHyT8uScZqxYKN3W5H mSLo6St9BpNKCsVj22K0pyTK1uqGgkEe9YneSpFCXGgLvtndhzO4u0iSEKZR4VAqjjEzrC7qw1j 1LPurT+ZE2rzYO2mUlH8lZIm9XO+zH+asfhpmusIr+eeGlvHgJlmDlyr/TxGlFOfqtbgQibI X-Proofpoint-ORIG-GUID: eQkxP92-Yq8lGAZvZgPkf5ut6Q9fyEbW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255306151116600 Content-Type: text/plain; charset="utf-8" Unlike most other messages, this is a server->client message, for when a server wants to do "DMA"; this is slow, so normally the server has memory directly mapped instead. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/protocol.h | 13 ++++- hw/vfio-user/proxy.h | 3 ++ hw/vfio-user/pci.c | 111 ++++++++++++++++++++++++++++++++++++++++ hw/vfio-user/proxy.c | 84 ++++++++++++++++++++++++++++++ 4 files changed, 210 insertions(+), 1 deletion(-) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 4fa6d03b3f..8f589faef4 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -203,7 +203,18 @@ typedef struct { char data[]; } VFIOUserRegionRW; =20 -/*imported from struct vfio_bitmap */ +/* + * VFIO_USER_DMA_READ + * VFIO_USER_DMA_WRITE + */ +typedef struct { + VFIOUserHdr hdr; + uint64_t offset; + uint32_t count; + char data[]; +} VFIOUserDMARW; + +/* imported from struct vfio_bitmap */ typedef struct { uint64_t pgsize; uint64_t size; diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 01fed3f17d..8f2d529846 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -104,6 +104,7 @@ void vfio_user_set_handler(VFIODevice *vbasedev, bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp); =20 VFIOUserFDs *vfio_user_getfds(int numfds); +void vfio_user_putfds(VFIOUserMsg *msg); =20 void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, uint32_t size, uint32_t flags); @@ -112,5 +113,7 @@ void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUser= Hdr *hdr, VFIOUserFDs *fds, int rsize); void vfio_user_send_nowait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, VFIOUserFDs *fds, int rsize); +void vfio_user_send_reply(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int size= ); +void vfio_user_send_error(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int erro= r); =20 #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 2bc71b9b54..095f32604d 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -11,6 +11,7 @@ =20 #include #include "qemu/osdep.h" +#include "qemu/error-report.h" =20 #include "hw/qdev-properties.h" #include "hw/vfio/pci.h" @@ -82,6 +83,95 @@ static void vfio_user_msix_teardown(VFIOPCIDevice *vdev) vdev->msix->pba_region =3D NULL; } =20 +static void vfio_user_dma_read(VFIOPCIDevice *vdev, VFIOUserDMARW *msg) +{ + PCIDevice *pdev =3D &vdev->pdev; + VFIOUserProxy *proxy =3D vdev->vbasedev.proxy; + VFIOUserDMARW *res; + MemTxResult r; + size_t size; + + if (msg->hdr.size < sizeof(*msg)) { + vfio_user_send_error(proxy, &msg->hdr, EINVAL); + return; + } + if (msg->count > proxy->max_xfer_size) { + vfio_user_send_error(proxy, &msg->hdr, E2BIG); + return; + } + + /* switch to our own message buffer */ + size =3D msg->count + sizeof(VFIOUserDMARW); + res =3D g_malloc0(size); + memcpy(res, msg, sizeof(*res)); + g_free(msg); + + r =3D pci_dma_read(pdev, res->offset, &res->data, res->count); + + switch (r) { + case MEMTX_OK: + if (res->hdr.flags & VFIO_USER_NO_REPLY) { + g_free(res); + return; + } + vfio_user_send_reply(proxy, &res->hdr, size); + break; + case MEMTX_ERROR: + vfio_user_send_error(proxy, &res->hdr, EFAULT); + break; + case MEMTX_DECODE_ERROR: + vfio_user_send_error(proxy, &res->hdr, ENODEV); + break; + case MEMTX_ACCESS_ERROR: + vfio_user_send_error(proxy, &res->hdr, EPERM); + break; + default: + error_printf("vfio_user_dma_read unknown error %d\n", r); + vfio_user_send_error(vdev->vbasedev.proxy, &res->hdr, EINVAL); + } +} + +static void vfio_user_dma_write(VFIOPCIDevice *vdev, VFIOUserDMARW *msg) +{ + PCIDevice *pdev =3D &vdev->pdev; + VFIOUserProxy *proxy =3D vdev->vbasedev.proxy; + MemTxResult r; + + if (msg->hdr.size < sizeof(*msg)) { + vfio_user_send_error(proxy, &msg->hdr, EINVAL); + return; + } + /* make sure transfer count isn't larger than the message data */ + if (msg->count > msg->hdr.size - sizeof(*msg)) { + vfio_user_send_error(proxy, &msg->hdr, E2BIG); + return; + } + + r =3D pci_dma_write(pdev, msg->offset, &msg->data, msg->count); + + switch (r) { + case MEMTX_OK: + if ((msg->hdr.flags & VFIO_USER_NO_REPLY) =3D=3D 0) { + vfio_user_send_reply(proxy, &msg->hdr, sizeof(msg->hdr)); + } else { + g_free(msg); + } + break; + case MEMTX_ERROR: + vfio_user_send_error(proxy, &msg->hdr, EFAULT); + break; + case MEMTX_DECODE_ERROR: + vfio_user_send_error(proxy, &msg->hdr, ENODEV); + break; + case MEMTX_ACCESS_ERROR: + vfio_user_send_error(proxy, &msg->hdr, EPERM); + break; + default: + error_printf("vfio_user_dma_write unknown error %d\n", r); + vfio_user_send_error(vdev->vbasedev.proxy, &msg->hdr, EINVAL); + } +} + /* * Incoming request message callback. * @@ -89,7 +179,28 @@ static void vfio_user_msix_teardown(VFIOPCIDevice *vdev) */ static void vfio_user_pci_process_req(void *opaque, VFIOUserMsg *msg) { + VFIOPCIDevice *vdev =3D opaque; + VFIOUserHdr *hdr =3D msg->hdr; + + /* no incoming PCI requests pass FDs */ + if (msg->fds !=3D NULL) { + vfio_user_send_error(vdev->vbasedev.proxy, hdr, EINVAL); + vfio_user_putfds(msg); + return; + } =20 + switch (hdr->command) { + case VFIO_USER_DMA_READ: + vfio_user_dma_read(vdev, (VFIOUserDMARW *)hdr); + break; + case VFIO_USER_DMA_WRITE: + vfio_user_dma_write(vdev, (VFIOUserDMARW *)hdr); + break; + default: + error_printf("vfio_user_pci_process_req unknown cmd %d\n", + hdr->command); + vfio_user_send_error(vdev->vbasedev.proxy, hdr, ENOSYS); + } } =20 /* diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 6d1446c38b..0935b72e9f 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -362,6 +362,10 @@ static int vfio_user_recv_one(VFIOUserProxy *proxy) *msg->hdr =3D hdr; data =3D (char *)msg->hdr + sizeof(hdr); } else { + if (hdr.size > proxy->max_xfer_size + sizeof(VFIOUserDMARW)) { + error_setg(&local_err, "vfio_user_recv request larger than max= "); + goto err; + } buf =3D g_malloc0(hdr.size); memcpy(buf, &hdr, sizeof(hdr)); data =3D buf + sizeof(hdr); @@ -676,6 +680,33 @@ void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUse= rHdr *hdr, qemu_mutex_unlock(&proxy->lock); } =20 +/* + * async send - msg can be queued, but will be freed when sent + */ +static void vfio_user_send_async(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds) +{ + VFIOUserMsg *msg; + int ret; + + if (!(hdr->flags & (VFIO_USER_NO_REPLY | VFIO_USER_REPLY))) { + error_printf("vfio_user_send_async on sync message\n"); + return; + } + + QEMU_LOCK_GUARD(&proxy->lock); + + msg =3D vfio_user_getmsg(proxy, hdr, fds); + msg->id =3D hdr->id; + msg->rsize =3D 0; + msg->type =3D VFIO_MSG_ASYNC; + + ret =3D vfio_user_send_queued(proxy, msg); + if (ret < 0) { + vfio_user_recycle(proxy, msg); + } +} + void vfio_user_wait_reqs(VFIOUserProxy *proxy) { VFIOUserMsg *msg; @@ -720,6 +751,59 @@ void vfio_user_wait_reqs(VFIOUserProxy *proxy) qemu_mutex_unlock(&proxy->lock); } =20 +/* + * Reply to an incoming request. + */ +void vfio_user_send_reply(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int size) +{ + + if (size < sizeof(VFIOUserHdr)) { + error_printf("vfio_user_send_reply - size too small\n"); + g_free(hdr); + return; + } + + /* + * convert header to associated reply + */ + hdr->flags =3D VFIO_USER_REPLY; + hdr->size =3D size; + + vfio_user_send_async(proxy, hdr, NULL); +} + +/* + * Send an error reply to an incoming request. + */ +void vfio_user_send_error(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int erro= r) +{ + + /* + * convert header to associated reply + */ + hdr->flags =3D VFIO_USER_REPLY; + hdr->flags |=3D VFIO_USER_ERROR; + hdr->error_reply =3D error; + hdr->size =3D sizeof(*hdr); + + vfio_user_send_async(proxy, hdr, NULL); +} + +/* + * Close FDs erroneously received in an incoming request. + */ +void vfio_user_putfds(VFIOUserMsg *msg) +{ + VFIOUserFDs *fds =3D msg->fds; + int i; + + for (i =3D 0; i < fds->recv_fds; i++) { + close(fds->fds[i]); + } + g_free(fds); + msg->fds =3D NULL; +} + static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); =20 --=20 2.43.0 From nobody Sat Nov 15 15:29:49 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255225; cv=pass; d=zohomail.com; s=zohoarc; b=LcbXCnN75z7Mieiu/jRZQcE43YQyQ3b8KXD2ffGGLsjjA/RtvXmqUWTPpP4vMvK90qZbsPynI++OmN5hwy8767x7SkcPDJD08H/GAtFLVLoL5I9L7AqZ0jhZPZRHkFn6lyVi1jQbjIn37qgJFsxFyxvoJNmFvr3gWssYpDhbLAI= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255225; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=cTA0UaGQyjYS0LKGNn3//eAGxWer1OE7mkfqMKrWPiU=; b=LKHML7gPGLh5fdFqXazzlr5yooL/DyrbrMuZooUfecRe73ikx3ocZjjKoula9O320K9ClzJPGYGzssJjihwAWwPMqGcHmuiWz3UDyPatwnYmwiRIQzxCIrVpW0decVbGY80qgzu+ZkeQEsV3K/xxl/yozLUru63y2aixUtq1Wh4= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255224991381.1060500945555; Fri, 6 Jun 2025 17:13:44 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNhA3-00035m-3s; Fri, 06 Jun 2025 20:11:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNhA1-00034S-CB for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:45 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9y-0006R5-7W for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:45 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556N0Rt9023133; Fri, 6 Jun 2025 17:11:30 -0700 Received: from nam02-bn1-obe.outbound.protection.outlook.com (mail-bn1nam02on2126.outbound.protection.outlook.com [40.107.212.126]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 472njsxqb2-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:30 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by CO1PR02MB8441.namprd02.prod.outlook.com (2603:10b6:303:154::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.17; Sat, 7 Jun 2025 00:11:27 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:27 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=cTA0UaGQyjYS0LKGNn3//eAGxWer1OE7mkfqMKrWP iU=; b=mglQP8W8QCuIhl47qgsUPOwD5POTfs4jGG0U9+o64d8GOuaRhtK9drB0x ZSlw1SwEQXHzBWuLdpXe4HpMeAMHDVNB/aWuUBgBumFgSgxpE/rPwA3hHRQZGOrY eTE+hKXxfmTYMWR3aOuo03+35hPufB2TJjFcUMzl2HPvo0pxY/bBraL0F6SmuzLD +FTk9CoyVHBwUOUTKC0m8yVUXDuigiFVoWMZ3vZ7UCq9ZhPWEAXd1LNlCi7sUqkv nPs5mmvYXLUhIpnDKMEr2dBPsTPPLS2FcaD6W+pgsODiC7UvdM8mXVA3cincKUDG HFhe2KUdwILZAuvhaw4pcbVhX7fqw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WNOdqsjVhmwhAXJDiazITp9bicLKEjeFjr72Q1m9VLZvDBd6G6Vbs+O5/A7zW++f7fepbREcf23IefckXuA/s9rX5y98P4rnL8duR0kU3WXB6/w7Arn+0O7896CPQ0BFr5EdJ4gWS+sMNebskNuf/1auJ/+11evxEoksqrvfeObYMTOvrNLF79e34XTDmE7b0R9iFQ3kP4V59uVcLCKxNYG+miXkBb6W6xLw3hlnIsbqaNPSkB/f6SORIQ9dFqHivzoCSlRqih5pqRKghQkcaWoPC1ksYgDDtkmO+r7efNSKgQWXfPOT9lnQ2Oq6OZyfXzguUufpYONJ6bRp6mcoog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cTA0UaGQyjYS0LKGNn3//eAGxWer1OE7mkfqMKrWPiU=; b=PydSReU8nuD6FtTFu1c6euRW6pYWj/6kKsBtA2T2/xdDndaqx0WltYJ2iMHX89JmJ6P9MiovjmVZX5/2yroahJz3+2ngDnr+IdLVjxNuekExeAmAf8CKkMjLvfWaU7tqM2L6b9nveBOOlP97NFbLLhGDrnqdcVVSlbcVKwo0NfrfhMlI5B2FEQvztLOI9kskG883hnSgu1OnDWiv4UkEr6f0/rlzt0s7waxq4g7lWCBy1oLEKxt7Ikj/bD+ou9EGN5PhcosakUKSLPpm+ZrtENvdg/OUnGNDeWRm2RdvIsFQMTyFvTlk18ea7urvMhGB7gk7y4grsP8Ul2kZVpNy2w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cTA0UaGQyjYS0LKGNn3//eAGxWer1OE7mkfqMKrWPiU=; b=lQn/JS012gWABRgm7yBT0D8Be6cOMNSGXo5FDJjwPYnN1nnbKdSIyVD4zdV251YjjeIbuHfl190j4HSvVFUw2QgdtED/jq78wBoYgi9is0xt52sCqz9aIWTkWLSuGfBnFQKkaMEMn+EqONWUyO7uBKPvBy9isjoXBetV1LEtKDtvUrOb4868fG6pa7L74IOSLRjYfvhJ6cyyu0FPU+9hA1H7r+rq8u3ZHNcpvFJ75ZizT8pJmbb4HFtfrs72WOg95JpZ9VNSw2qN7FyYo2+gwRGFXINjijWaM2p9F8ZEm2XlkNbU9dOn4ahxBpRLtiaY3TWDeiGxAvcyZnIRE3rtYg== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 20/23] vfio-user: add 'x-msg-timeout' option Date: Fri, 6 Jun 2025 17:10:52 -0700 Message-ID: <20250607001056.335310-21-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|CO1PR02MB8441:EE_ X-MS-Office365-Filtering-Correlation-Id: b213150d-49e1-4e57-c888-08dda557d8e7 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?5Z58ff8vLwl7yIg/+kjnDnBxvDRqA76tucVbdxG2ZyU9982/VetE3Si8d1vt?= =?us-ascii?Q?iXF/FmbQnLsOHvRDE0T6VcWSw8GKMiy1Uof53nkyAJR2/vJZzM+YRXA2G1ip?= =?us-ascii?Q?Zs67+gQHCl5xH8nWk9Q6Yj04bZur1W4LwtXggFr7EVZbanmfLMDHVjMCrFrw?= =?us-ascii?Q?SQn+H/78KyuDZbdKpUDv8krVZ6I9kuOz9ZiJAn08MZ/PfuIo3D4/K2SYrDJ1?= =?us-ascii?Q?tv5FkEt/bl0eJo3o3l0nZMWim+ri9tgRB+VuoAfNnjfq+INzDLvlRWQhLuYs?= =?us-ascii?Q?rjHASUNX8bvIqHFXEMDaQtMxmlRVMOaohVotyhAUgySU/odKJ4ReOhS9jD6e?= =?us-ascii?Q?kfH8m23UYaCA3pu55zzNTfAF5NANe0Wxjy97Zm7wxaLVNmyCfOortwA763vF?= =?us-ascii?Q?niQybJ+IJV/Qi5R5pcA4h+1padYVOLUqxBhjt0H+yegAy3+lz1dhttooUa+J?= =?us-ascii?Q?rmWIr37E3bysecRJndQrWRQrZP7FvPhcIlJlj7bylZBmv9zHHcjNkaG7RnUi?= =?us-ascii?Q?BgSwukdHisFg4Fk0hdatypJsLuK56oHUFGFuY26Sm4AfnKQabcF5uIt0CiuZ?= =?us-ascii?Q?4gfSGfdf68RE0Z6Wjgjizsz+v5TWc5elVsDq6kDqJCxYc22/rMc+98jJTi6N?= =?us-ascii?Q?gU+o1qzsTSWrNqgccq7lM9OzqNjGsXPRLM5nqHxVw/8In5Z+dOBL5vaDKfIg?= =?us-ascii?Q?6rVx4d3B1kYeGtW/8uDa9jnid+RQ5enySL/QdAaEpsOzx0T/+fEhWHVtVGNt?= =?us-ascii?Q?qR0EUI5GrzfOnaRxcjLkS+i5lOvN+BKD5HaGOYH5vmqN0E4e1frLA7lEigU/?= =?us-ascii?Q?Vodt2aiA9DhTdwbUieVNyZwDsloc1DF/Oxoiqe7oz4e52/tSYK4dzLRTCWpB?= =?us-ascii?Q?4pHqlolHYeSwOMbfWPpWfEJX7Ikr5HAJDB6o9CY5QgrpxdL9xhFfET7qVooA?= =?us-ascii?Q?XKID5HtE9qplXe/bNSHrDtgGpsPBE/jFqBaT4ZyT9JVQgbSmDdcgZ5PlvibZ?= =?us-ascii?Q?SMzJRMVVcD5B1tFmytHSz+FOouKIE/0cTUIA0ncA7LRNnJqY3Q68m/6YY6Uj?= =?us-ascii?Q?pXY+IDnu5+pldhBFFXL5+9er5XzcNeKEEWnpfOZfT8hd+tJqoYeMgswDBt4N?= =?us-ascii?Q?FuEz7A/Yv+anDuDMEQ3ODjrEBHpHXpIzq8Lf280dBHmL6yak6tZZEEwu0snF?= =?us-ascii?Q?aUQnxrTBFwQZ+7UfDKn7UaFDPILYNFOy3bi3YCI2p2HwJKSYgcGDsAPcQpbC?= =?us-ascii?Q?Fknn1NxuNfYRPct+BmdJfaLbabbfgf2Tdxe5ME98DSg5ij9y7jg/82hgnZzX?= =?us-ascii?Q?Zck7v6U3ja6CE6ALDdOM+33Xo8sbX1xGw+5kvpJxTgk7iC3yJJz3WqXiBxkl?= =?us-ascii?Q?qZHFsvAlY0hkSZOIqGOAc2w67l4dNp81pOZFFndH4LcGpFfaJLSpH1DLLxug?= =?us-ascii?Q?V38/bCMSM1c=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(366016)(376014)(7416014); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Op7o//2klBnxBGxn6wn6V5H+RIXW0Ak6Hmf1fgHYJDdIL7KL9Q0flvLkNjlw?= =?us-ascii?Q?2/QfF+gDSSGc+3owqJwp3L36Q7UBUkI7kwEcJRyWZXHosZknaVF+C8hQ4vsc?= =?us-ascii?Q?thmnD61iPQmMcO+BPM8Pc6Lm0D24SuxwJ/ItyV3qrgLYyyn8lHTM0xLXjBjH?= =?us-ascii?Q?rNcOynuI8ZEXLyAK1H8rfzwJVPeAk+u8P1Kdh7A5EqqK5xpK3Kk3+ioLWZ1Q?= =?us-ascii?Q?1wzpaZYmxBJystn9RXM/PyxAEU3PDJw0Y7hfSt/pyXxmHP9RXd+O6d50Z4el?= =?us-ascii?Q?A48PpHQ4x3Gre4D9KOOo8LZgvAG2TDYtll9u9pMoS4dwyoHF6BNlWOHobvXq?= =?us-ascii?Q?fXOhLQgtc33zSi2SCUkWi3doIYIEY5jopD9Ly1w4LWw3U0tiAyULjsZkeBDw?= =?us-ascii?Q?8wI7kLF96bnVNcftr4qikAhBg5kgK9MfyIfBNBbWvKMBzzSLgV74BQ+FNpvN?= =?us-ascii?Q?lwXUEZnKg4GrKg5Yem2pRpwvyJsr1FT3sWM5YVEm7vt+Xm46mhb9Iy4chOqX?= =?us-ascii?Q?6BYw6VHsJ7c4gwzswQvItRp6jvoe/h3jo6n90HahlGjWAVRCIN4hPoWWwFvN?= =?us-ascii?Q?7aHB4sMvLzH85Rp0GBRvAe2G3EHKc+JflhUU6qof3dlA4xizQgOFSopw0VvK?= =?us-ascii?Q?5RzrBwExYkMjspHULSXJr465W33lO0BgL+Wq+wMukq3VmmDnekLYeV2WqPOp?= =?us-ascii?Q?YELSPjvpl68BHTGfTtdQuaXPwEt1oFRZWRqMi3H4tu8J/RcjgGgzZGDxkQUS?= =?us-ascii?Q?roKdWo4Wt2iElzCXqlhasbrj2lXFwEH7YAuuR1EG9+4dBnvFyEeS8qlqXG5i?= =?us-ascii?Q?3VVZNKnGKBiajgIEewU+FIt6R+vNLLsNR/cDLzfjlBscLVi2/EUxxMI4CCUv?= =?us-ascii?Q?dCLooRzpT9RiN/UjxI6z08hKqR2aX9LYjnqbbA3EdRxFD/SXpNbVeYeCPQQh?= =?us-ascii?Q?AbRpfXkOhiuPP07MQCKXOdBJDUAhl+zsCAvwSvUNLKE8NFwPBtKq/8jw49ac?= =?us-ascii?Q?Pn2XqIBxDQS7ksOf+61yng0FJs1AkPRvSnzTD2AnTtJdiDcjzjzVjNnNQXPC?= =?us-ascii?Q?GjLZmyPb/erksD8q7w35+G8xMLWqz5Y7xtcE5A+KxEGZGx2nEXDku+uxqWIZ?= =?us-ascii?Q?dmAT0KKdKXZAbvkJL5qKasRx62iLOfZuopxGLO1cOM6HWKpeEHxEU/OP173s?= =?us-ascii?Q?V2M/fuAQCc0l+Xk7LFSkGui7fVPUcH1/2Plrszf0eb8AAGDoOz+ae4HV+ApP?= =?us-ascii?Q?xDysjtQLrdM9amDUlGpLhs7QBmeEu1CefsUz8RjGDF+wOe9HYKFCzpF8v55N?= =?us-ascii?Q?8I/yYibA3onUqBrPpYhBNZOOkC5P6rwICUGITEdGAH3Qvou9IqCyvdCq5sJ/?= =?us-ascii?Q?UyFIbYTIVHYwk6LBdD8OyfWugDg+g6mfjrtkKBbWPqmF3HzWTLHqRrY5AMpv?= =?us-ascii?Q?BKSQ1xolIhz90EAh5X4HwX6TukcAH/WVVxdnhrkd7cBxHtcXHJmL2u/dN8JU?= =?us-ascii?Q?El1U7VHyPi7DeAMXc+EEy4HPpd1ZJtA+Byq13reVHNO+Rt2eZWFtNoBMH6wb?= =?us-ascii?Q?VbDRfNIbfIjr64OVfirsSS7RLeHTiTY6MNRPDllv?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: b213150d-49e1-4e57-c888-08dda557d8e7 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:27.7606 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FieD2QTozSysIkJ2FhA0k99h3JiW2m1JSMPJUtxaDOtnV0tqiOTA18FAc0uDDAYWOg8RTe3H41uSSd9qyBh+Bg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR02MB8441 X-Authority-Analysis: v=2.4 cv=f9JIBPyM c=1 sm=1 tr=0 ts=684383b2 cx=c_pps a=JomqzcHDjT5Ni/tnCvtrfQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=Ql6ANm6PklPDYutWWPYA:9 X-Proofpoint-GUID: sM1w5dSBsSozwz6TbkIp_8AjEn9V0O8m X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX16aR1Lt/uhvp Lkr1qB/4vthXiWiGB5UdNxHgcFHz4zRdWF2ezzeYOB8lwFjygJcJsNGIGHdOwvJIoZPnUbhyMWq I/DN7Fe2Pr+/CkSmvD4yQ1f2eiIHTHomBx9khOhNIcaNV2bCHxMXe+1YPOd4BU0efHKRu5mRegB Zhq8mxzHHuK+9C/r7vEuVQW10QcY9FTsc9/Tnr/9JvJXE73gxTBIOdP99pK+nVsjIcwZ0YizCNK YYWM4wduirBUnwcrWF/dbQWk3csdk+RZsgLnxl/zBfO/EyPqREYgrmq7hConGdgRkR+uJyFIl8H ESL8N3SYbZQAD47rC4/rnTGZBEqIeAvQDlCtu65VGaOhnwyj8sJhm+YfniaVS2ZPty9rKWtlf7U iAqt7/gbwrzfE+PnHhqo52jXYb+p2p193Hq1YiP+RSRnIgu0ut5K2irwtA/lOVjVFhYuqDt8 X-Proofpoint-ORIG-GUID: sM1w5dSBsSozwz6TbkIp_8AjEn9V0O8m X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255227193116600 Content-Type: text/plain; charset="utf-8" By default, the vfio-user subsystem will wait 5 seconds for a message reply from the server. Add an option to allow this to be configurable. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/proxy.h | 1 + hw/vfio-user/pci.c | 5 +++++ hw/vfio-user/proxy.c | 7 ++++--- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 8f2d529846..da29674fdc 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -70,6 +70,7 @@ typedef struct VFIOUserProxy { uint64_t max_bitmap; uint64_t migr_pgsize; int flags; + uint32_t wait_time; QemuCond close_cv; AioContext *ctx; QEMUBH *req_bh; diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 095f32604d..769dd9b810 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -25,6 +25,7 @@ struct VFIOUserPCIDevice { VFIOPCIDevice device; char *sock_name; bool send_queued; /* all sends are queued */ + uint32_t wait_time; /* timeout for message replies */ }; =20 /* @@ -263,6 +264,9 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Erro= r **errp) proxy->flags |=3D VFIO_PROXY_FORCE_QUEUED; } =20 + /* user specified or 5 sec default */ + proxy->wait_time =3D udev->wait_time; + if (!vfio_user_validate_version(proxy, errp)) { goto error; } @@ -397,6 +401,7 @@ static const Property vfio_user_pci_dev_properties[] = =3D { sub_device_id, PCI_ANY_ID), DEFINE_PROP_STRING("socket", VFIOUserPCIDevice, sock_name), DEFINE_PROP_BOOL("x-send-queued", VFIOUserPCIDevice, send_queued, fals= e), + DEFINE_PROP_UINT32("x-msg-timeout", VFIOUserPCIDevice, wait_time, 5000= ), }; =20 static void vfio_user_pci_dev_class_init(ObjectClass *klass, const void *d= ata) diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 0935b72e9f..74cfaff6fa 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -24,7 +24,6 @@ #include "qemu/main-loop.h" #include "system/iothread.h" =20 -static int wait_time =3D 5000; /* wait up to 5 sec for busy servers */ static IOThread *vfio_user_iothread; =20 static void vfio_user_shutdown(VFIOUserProxy *proxy); @@ -665,7 +664,8 @@ void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUser= Hdr *hdr, =20 if (ret =3D=3D 0) { while (!msg->complete) { - if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, wait_time)) { + if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, + proxy->wait_time)) { VFIOUserMsgQ *list; =20 list =3D msg->pending ? &proxy->pending : &proxy->outgoing; @@ -725,7 +725,8 @@ void vfio_user_wait_reqs(VFIOUserProxy *proxy) msg->type =3D VFIO_MSG_WAIT; proxy->last_nowait =3D NULL; while (!msg->complete) { - if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, wait_time)) { + if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, + proxy->wait_time)) { VFIOUserMsgQ *list; =20 list =3D msg->pending ? &proxy->pending : &proxy->outgoing; --=20 2.43.0 From nobody Sat Nov 15 15:29:49 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255269; cv=pass; d=zohomail.com; s=zohoarc; b=DUwW+8u2vNU9KrvzWPY15RoivSqQVqJXVshE2M9uC4gJXUtI78Jet0FSJvWqmTZElMXtd5NLIME3YePDvAm8bkhkbqdBU/bj5i7gbfk6lRJPnqNyFaLwViFDrTtxZ35nXiB0eL/WvZGZruZIDToVNbR5Fpus/Rzangpn0r4i93Q= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255269; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=YgOnQTNyG83d9gdChM10EEgQ5OT4pK5TkYiCd56uWD8=; b=ML1JwGWL+5+1ZCkWikl0roQEwegNqefvJiqVgMVSIGDTg/WlmtoWjesMfwSesXxndrbXe5eV3iOOwhK6iC6xxaQBVO9g9oArrlQOfE6uH2JTBgLEv8M0AnYz1bFKv4GA34bbwnZBNjbsAaIqFBMn0csnSQS7NGBHVg/GC8ruCtE= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255269139821.6875664945267; Fri, 6 Jun 2025 17:14:29 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNhA4-00036H-GJ; Fri, 06 Jun 2025 20:11:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNhA1-00034l-Jc for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:45 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9y-0006RA-St for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:45 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556N0RtA023133; Fri, 6 Jun 2025 17:11:31 -0700 Received: from nam02-bn1-obe.outbound.protection.outlook.com (mail-bn1nam02on2126.outbound.protection.outlook.com [40.107.212.126]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 472njsxqb2-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:31 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by CO1PR02MB8441.namprd02.prod.outlook.com (2603:10b6:303:154::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.17; Sat, 7 Jun 2025 00:11:29 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:28 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=YgOnQTNyG83d9gdChM10EEgQ5OT4pK5TkYiCd56uW D8=; b=KdB26dpra3vh6vQ0lELOxY+Mhf/I4XfZHI4sEGNRxqGv40vYvwNWsbnzi N5hESByHjz2htXgQ8dGW0xJIuMnMwtA7pJHLZucmtp2010lwSnhHuZPy3JuKRk81 ODPouUIrScW71x2sKOpLtDAMd9LNi4Ydfd+r7XGp+floYKk1JNfQP3jzW3KyHBg1 VE3DTK0oLmkQs8izFfyvbJT1f2PJjzk/Zk6zd2IAZGUvHqPnDwliC1FYmXCmNILq U1mn5dWB52osdR/orBomHIZf1IPViB1st7uy5JxOMo0vf4GvbKX8cqimc+ITdoOk Qo7LV+NayYThZsgN7Z75TV/Xd2cKw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=DqnMQEbWhu8QEKsupshz9sh/hZXTAlGy1+yb9gkZHNeCDd4lk3ve1pWPONUi5oYJlgE9p8ts5QI+8BxMAy2vzgy9xdnFy6rz4HFUiLiK/IcdqQbNNW48biDAaXEM8mK+SA/xm1KVmJEDfLsPGfAe9IGBoQFj+C+EXytmih526xZg0Gz18J79imHpX34TQk9o++VIUALhmFlm0pprvNMaSD146E0G1K349yjR6T2ynctiWqAGyicLfm5Fz/JbLtSa+/5seApCHpTPG/YGQqLVGB+HWGBoOiHhcKndirTbLrhbQs9a/6UhvjwLptZ2hzaLoXacDecDboIOOlTW1qub7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YgOnQTNyG83d9gdChM10EEgQ5OT4pK5TkYiCd56uWD8=; b=XfURr0dC4gmNduxoHT4HNvxt82SGChoYC+2jyDWy3LpctaUNvSgrvyIX4Cz4g3yk5mcdC4o5/Jfk33YXiRGCZaLXKX2XBCMs6sYzOgRzo/Cf3s+SbmwzqOPcrDiz0qSQNmSGekDB/h9Q0eOtO8MsHOc+mCxlRvxz7BjEAKSIlDfvXc4wMJO2FpaGhAdBksEqHBu6Au5xPnFtKeVvvx8KgX1N2f8bdMQu9wFX+Ffk5Bz2GGKtnbam55vwN3IH0a6ZZZs5bBIyVQsScCD4iZLyvkfb4bx7FlFxKFWjowDLqgXynenhkisCaU5kZifPAR5tGS5eVfYK18Jf0GIa6KN6iw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YgOnQTNyG83d9gdChM10EEgQ5OT4pK5TkYiCd56uWD8=; b=h2tRO9SbixFeMjm2gtdbrf+pOBbsMWUBNX6zlVSYb2w0Y94Y/TKB65TClwMjMtn4D+JBALcWwE8EZugpG4mujUkE9970oqCIJPtFeaB8KtSrzabMxOxWslnHnZiIIgrDhgCraQ/3sfhfDQDLURvwu9uvy6ZpPy1o9f7O5UvJkoxo8Jo9XWoZuxellBlx+eGw+5mRPTYVsR/BaWIqmEZIlJhJ+/D3dVxmXAGID3WVrmH7RXGOCQOAtLFDh77cXJsdpYSTCT7KislfoIOgAFfiqoTJU/LXUgHAS/qQ5AFoxyCS3MciNv4uO1pNhYQxsKoTxLvv4yhaTesfoUzVEaf6hQ== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon Subject: [PATCH v3 21/23] vfio-user: support posted writes Date: Fri, 6 Jun 2025 17:10:53 -0700 Message-ID: <20250607001056.335310-22-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|CO1PR02MB8441:EE_ X-MS-Office365-Filtering-Correlation-Id: d6fc7839-6794-4f6a-36c5-08dda557d995 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?QRwEIeXhfrWWuDXOQVicgvk4PxK6E05xP7IO55HEDAkKiLri1cOmn+jD+tTc?= =?us-ascii?Q?Jdm1QsBtG+ujEYUl98B5YGoO19gIC1R54DXR7VzAqsEfr5FSOFEAtq94Qgtt?= =?us-ascii?Q?vNWuApK4A1GQiyvuO31ZNQXkw3qxwC2/wg9KlHeK4FJQ3nhOvtnFuyD865Fk?= =?us-ascii?Q?/yWqdGagf/rI8rbpQD2UMtK2yBkOLAE83YOHWX8G6Yt81KwuhMlX+aNthZ8a?= =?us-ascii?Q?5tcK0tmXIBtjQSoJQNlNgl90MF/0M+hGGZ0u7tXCGzf46YbnmC9fRhxzepp5?= =?us-ascii?Q?KuFTa93+aGKL9pH4K5X0LUyhE3M96Wme6Spw1Q8oCQ7t6/FpbhUxEO/lBba6?= =?us-ascii?Q?ZeIDvfwKQYI07T1IOezVLETWfQOGKaq7MQl81By5ZzSXJ/R7bZOET0F1dpEs?= =?us-ascii?Q?EG6yYO1Hh8RVwX4Viy6sSgwBkZNB+0mc8LOAbglDGYKBkxzX/rVOcvbCoVLS?= =?us-ascii?Q?T6KIA8CdHRQZFi5ARmQqwasmLPEY4k9tuN8wlDLmfSDOP6zTjXY4hR8LpeLA?= =?us-ascii?Q?LiUz5DOiNWdcPSPkVH8ptl9Fu5DYbBqTBltem1WhGKKxQgO47FfmqRwVBIrh?= =?us-ascii?Q?YLAvVtAYBjqbv03jxAoJrAtGjQT7erRN6dSB2vxZylpWs5rdvZW57VohB4Cn?= =?us-ascii?Q?ufB8/Q0MPyVdc9iTbhS4ntLZFKM/405TJ1G82mNZuwwcDJMOiW9tEeFOw7rF?= =?us-ascii?Q?Ivtn9G/PC9HDMpMsAxTDPhQZvJLhIqbN4VwuJpr3jfhzLNf5nMSqXc0gtzz9?= =?us-ascii?Q?klCQW0kybP6b2y437QSqfmUfV1DxEEiYND155okNEVYahmmvjEghUUEO8DNb?= =?us-ascii?Q?0Vst4qiqUFBNLTyPxT/lizpcKjU1WmBt5SA0t+pL1S9XQWu8L0K5Uu/fOWT5?= =?us-ascii?Q?Y8F9/eg7bhDSQA5NChLJizChVdZArPhl6R4MyOH1pUB3rbbR3qvvZxg0DU4p?= =?us-ascii?Q?1BLp+pAT8S7wxAocxmHg8wZARa1CbswQINkXCoy7Zv1UEV/wVVSUxRzas1IT?= =?us-ascii?Q?MOPGxKghsvX2npjN8TlwJJ92eJMO3/XYl/WhD96svUykbpKmg8UeTIx1bich?= =?us-ascii?Q?LQ6Aoyex1rTA105ZPSyTR9mS+OoeRtRsb+pXP8yZWpvGj0SrBxpD1A/NTw/9?= =?us-ascii?Q?pk4Vvv3e/M2z4uo3TYW1KHocmHvXqn96IOndn0s+Kgz9Iqaim5zXX4B8AN1D?= =?us-ascii?Q?n7ctD2tHsD1ZKS6oCf7KXhimYvx3rzD3fyF11a6gejQhC/BJi1l8OvjSeLpm?= =?us-ascii?Q?UZ9PXxq9rfAIV2077D9+eWz3tKnbXoA1NaQ7xlXyDBiIvkw5q/JD6JUDPIMI?= =?us-ascii?Q?NM1bFkMPG+TWK2sVAvoQ7S8+sQmMpovZfbLS665yR4N7QFKFhNYoP0PtLUu5?= =?us-ascii?Q?qS1jwtwTytl51vGTjcwog04qw5Z3VN50x5UE6zWcEBEUzXWbIrubbk+sD8fu?= =?us-ascii?Q?F+cFIgqEfHE=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(366016)(376014); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?0gwSd1I83IXpz6yrqRsD5dCVQxr+UQ7Jh4RiELMS96AYSDuAJ5XjtbixOO+D?= =?us-ascii?Q?2AwLUk3IonI9MD4ZiHNmPdyB9T/dlr5l5ZH449RaM1s+aCftjfejvW9PwL1o?= =?us-ascii?Q?myIfU36cagUveDVt+WoQQS/LV0S/wGRqckWSUXYLuDV23Lk0AcIyZfxrSUuk?= =?us-ascii?Q?thkN0KQ8LjyoYwYbHrbhandHhj4mZaydsBOy0XrObsRQDCXNdUA4Hosm19Dp?= =?us-ascii?Q?xH23jde1BjUG6aroqyofiS009kVtUEDeVO3ty2D5Mbx+ehboFbKoB/GqCntF?= =?us-ascii?Q?bGAidYeEkxHRUg0UJufiO7kGOefJ7GTlzcsHhjeUBC3QQV3azbosA4Ajh6mZ?= =?us-ascii?Q?IWgSEDU8dj6XX84AVhY99y2UHt8y4cQF+b+MWnsei0Ajdye83xFyw1HyAHfW?= =?us-ascii?Q?G9OCFlQd+6wsX+mcYLQ/yIVCeNZYOUTXm2c3boExaQ/7Y8M059TOA9nrkTYm?= =?us-ascii?Q?1m7QN2dSsav++1th0b4X0CEhiQI6S8wbCYP2IOA5gRB9sXZczC0GrJn4KlUD?= =?us-ascii?Q?KMEG5/fXFLTossZ8YFxmFqLGIzcYxthjJzxvpIn2tMXkhNWjCPfsw2p00OHv?= =?us-ascii?Q?6Md6D+/h2cV66lVpmCwoqEPCTigwW0sDVwGGHLAhyco9p39oDjljW0YE1idJ?= =?us-ascii?Q?wOen2TfVq+scE/gzLriircLhM0BtQSyTxi056ulZnv6EMC9fh7yO6tW3aRi0?= =?us-ascii?Q?umb26if6dcmTJlLzgSFSyv8uaDHe8DO/7xqTAxuduZ5ydlaZRF9Dbp2zSMWv?= =?us-ascii?Q?749p3bW549eli5lLQcBjlrtIE3+HTfMQQiXIhNah3CrwlmXkHWb+TkWd9IXy?= =?us-ascii?Q?BCAs0eD5aGpUlH3M4of8JojkF6RL+Yx1NIAjS2eLlWGP/R8OdjNb3Cnri01T?= =?us-ascii?Q?QRZ7xZ2oiYbsL8VxBc3lyYiPObAZMipOcdCasqAqwkstwXgRah1JjKWaZM1V?= =?us-ascii?Q?N42p1WIjVd0BmKMRUyOvZb2a2c0Skwoe2GVV+Q9aPzbLk/MhGv3/Bu1bHEGB?= =?us-ascii?Q?+g1O41D0/2lqHZrEnCFFuvy9gVkNmVHn6BuecbQohZubqsa/vdpTn1os0Koz?= =?us-ascii?Q?dsCxSux+93RG/s/yod13XQ8BEYlZQAtdH0lfkYF95nW+NonZx9T4veLL4cDu?= =?us-ascii?Q?5esexIoyDut4s0pOJM6G+mOEWfiKt02xQ9Cg56qQbxxT8IJcaSBs/nViQ163?= =?us-ascii?Q?r2sIlqI+N4wa1jRKexB9uzsu7X6jjNUYxsh43KMUTZxW2VuU16kQXFJKIRuI?= =?us-ascii?Q?ss8BYSuf4am7CZQas/U2+hQVFYu8r88iKguFi6JifHxCohuLI7Jgz+5Q8fCf?= =?us-ascii?Q?RCKpk6p1x7yxwCbVJDaZqsvvCVqs5OKExmah6aNZYis09hR+gXRNdDeRBWXC?= =?us-ascii?Q?y7XJjpDJ3A4jyxaqQbvK9NLPt1jCh32GW/UNF+IyYElgaNSC99eBDRwyCNPZ?= =?us-ascii?Q?DYLUpLMZSwgOBVDmoCisVbynbU7dRRQJvXZrYRgFlbgk/h5gU4n0BcjSqeV2?= =?us-ascii?Q?KfLM/DUTmFlVxj94JKpjQNZJ6rbtcqi6gU83fxQEQ304w2tRHWIsC1aDVBTF?= =?us-ascii?Q?Z50xz5fw999e64Bdn7np0H1uI/7q60G1CpFGLV1O?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: d6fc7839-6794-4f6a-36c5-08dda557d995 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:28.8813 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: e1h4vt3Ioj8DnBEE9YmzBG8odgZc8yXnLP5lYSvubLPa5HJQApt1M8Tq4IkkjmVU0H/znFyMOkQDCfC/QmkQ0w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR02MB8441 X-Authority-Analysis: v=2.4 cv=f9JIBPyM c=1 sm=1 tr=0 ts=684383b3 cx=c_pps a=JomqzcHDjT5Ni/tnCvtrfQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=64Cc0HZtAAAA:8 a=hLZCMK2IaNhFlHKNMJgA:9 X-Proofpoint-GUID: dTItlo1qdEwqSdaImfYbL8Ft0ocuzmk4 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX14PS/7bO9r90 9YBZQw8norxgbw6qoeKxzxEvEkVPONGQJ1Ytyc1NE8SMDWtxuhhAQzIXK2+01wfB3Hddp9Rj7JW u3vFRxZG0IJ3NneBCHYlJy14Djoe2diwmclNaiGhK7t2CTQDUWkmATfWztKx7sC9X88e20J4Pyn NZ9l6fLitBy/O+VMJntCIMiBW7EKXi57FiMfViZdAmThl6WfTBvx6pBdPXXy9Gd4KDtn1a8jlfp OITY8GLQwdwwKA9IqgkfHlAIdO9Unw77ah0C3l5aDw7w2BqbayaCSIAljgzm9YiDX1AQv5CCnY0 ifoah4EnQICGNDczZxikDswfKla9pI/IjtH4WL16NXrlSaPOPrTpJG8V1ODGevteH6/YcDhFkLm ei07nVZPIyAWKVjUci9ZS3NuTm/7UDNGdP15H0JVf/4fAquo1/WV7gFkeNiwxtH3cupxH3Oz X-Proofpoint-ORIG-GUID: dTItlo1qdEwqSdaImfYbL8Ft0ocuzmk4 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255269661116600 Content-Type: text/plain; charset="utf-8" Support an asynchronous send of a vfio-user socket message (no wait for a reply) when the write is posted. This is only safe when no regions are mappable by the VM. Add an option to explicitly disable this as well. Signed-off-by: John Levon --- hw/vfio-user/proxy.h | 5 +++++ hw/vfio-user/device.c | 37 ++++++++++++++++++++++++++++++++++--- hw/vfio-user/pci.c | 6 ++++++ hw/vfio-user/proxy.c | 12 ++++++++++-- 4 files changed, 55 insertions(+), 5 deletions(-) diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index da29674fdc..22ed66c54f 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -94,6 +94,7 @@ typedef struct VFIOUserProxy { /* VFIOProxy flags */ #define VFIO_PROXY_CLIENT 0x1 #define VFIO_PROXY_FORCE_QUEUED 0x4 +#define VFIO_PROXY_NO_POST 0x8 =20 typedef struct VFIODevice VFIODevice; =20 @@ -107,6 +108,8 @@ bool vfio_user_validate_version(VFIOUserProxy *proxy, E= rror **errp); VFIOUserFDs *vfio_user_getfds(int numfds); void vfio_user_putfds(VFIOUserMsg *msg); =20 +void vfio_user_disable_posted_writes(VFIOUserProxy *proxy); + void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, uint32_t size, uint32_t flags); void vfio_user_wait_reqs(VFIOUserProxy *proxy); @@ -114,6 +117,8 @@ void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUser= Hdr *hdr, VFIOUserFDs *fds, int rsize); void vfio_user_send_nowait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, VFIOUserFDs *fds, int rsize); +void vfio_user_send_async(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds); void vfio_user_send_reply(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int size= ); void vfio_user_send_error(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int erro= r); =20 diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index b37f7329a1..eb2194c0eb 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -93,10 +93,21 @@ static int vfio_user_get_region_info(VFIOUserProxy *pro= xy, trace_vfio_user_get_region_info(msgp->index, msgp->flags, msgp->size); =20 memcpy(info, &msgp->argsz, info->argsz); + + /* + * If at least one region is directly mapped into the VM, then we can = no + * longer rely on the sequential nature of vfio-user request handling = to + * ensure that posted writes are completed before a subsequent read. I= n this + * case, disable posted write support. This is a per-device property, = not + * per-region. + */ + if (info->flags & VFIO_REGION_INFO_FLAG_MMAP) { + vfio_user_disable_posted_writes(proxy); + } + return 0; } =20 - static int vfio_user_device_io_get_region_info(VFIODevice *vbasedev, struct vfio_region_info *in= fo, int *fd) @@ -272,6 +283,12 @@ static int vfio_user_device_io_region_read(VFIODevice = *vbasedev, uint8_t index, return msgp->count; } =20 +/* + * If this is a posted write, and VFIO_PROXY_NO_POST is not set, then we a= re OK + * to send the write to the socket without waiting for the server's reply: + * a subsequent read (of any region) will not pass the posted write, as all + * messages are handled sequentially. + */ static int vfio_user_device_io_region_write(VFIODevice *vbasedev, uint8_t = index, off_t off, unsigned count, void *data, bool post) @@ -279,21 +296,35 @@ static int vfio_user_device_io_region_write(VFIODevic= e *vbasedev, uint8_t index, VFIOUserRegionRW *msgp =3D NULL; VFIOUserProxy *proxy =3D vbasedev->proxy; int size =3D sizeof(*msgp) + count; + int flags =3D 0; int ret; =20 if (count > proxy->max_xfer_size) { return -EINVAL; } =20 + if (proxy->flags & VFIO_PROXY_NO_POST) { + post =3D false; + } + + if (post) { + flags |=3D VFIO_USER_NO_REPLY; + } + msgp =3D g_malloc0(size); - vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_WRITE, size, 0); + vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_WRITE, size, flags); msgp->offset =3D off; msgp->region =3D index; msgp->count =3D count; memcpy(&msgp->data, data, count); trace_vfio_user_region_rw(msgp->region, msgp->offset, msgp->count); =20 - /* Ignore post: all writes are synchronous/non-posted. */ + /* async send will free msg after it's sent */ + if (post) { + vfio_user_send_async(proxy, &msgp->hdr, NULL); + return count; + } + vfio_user_send_wait(proxy, &msgp->hdr, NULL, 0); if (msgp->hdr.flags & VFIO_USER_ERROR) { ret =3D -msgp->hdr.error_reply; diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 769dd9b810..e59fa27039 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -26,6 +26,7 @@ struct VFIOUserPCIDevice { char *sock_name; bool send_queued; /* all sends are queued */ uint32_t wait_time; /* timeout for message replies */ + bool no_post; /* all region writes are sync */ }; =20 /* @@ -264,6 +265,10 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Err= or **errp) proxy->flags |=3D VFIO_PROXY_FORCE_QUEUED; } =20 + if (udev->no_post) { + proxy->flags |=3D VFIO_PROXY_NO_POST; + } + /* user specified or 5 sec default */ proxy->wait_time =3D udev->wait_time; =20 @@ -402,6 +407,7 @@ static const Property vfio_user_pci_dev_properties[] = =3D { DEFINE_PROP_STRING("socket", VFIOUserPCIDevice, sock_name), DEFINE_PROP_BOOL("x-send-queued", VFIOUserPCIDevice, send_queued, fals= e), DEFINE_PROP_UINT32("x-msg-timeout", VFIOUserPCIDevice, wait_time, 5000= ), + DEFINE_PROP_BOOL("x-no-posted-writes", VFIOUserPCIDevice, no_post, fal= se), }; =20 static void vfio_user_pci_dev_class_init(ObjectClass *klass, const void *d= ata) diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 74cfaff6fa..13f2407845 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -683,8 +683,8 @@ void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUser= Hdr *hdr, /* * async send - msg can be queued, but will be freed when sent */ -static void vfio_user_send_async(VFIOUserProxy *proxy, VFIOUserHdr *hdr, - VFIOUserFDs *fds) +void vfio_user_send_async(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds) { VFIOUserMsg *msg; int ret; @@ -805,6 +805,14 @@ void vfio_user_putfds(VFIOUserMsg *msg) msg->fds =3D NULL; } =20 +void +vfio_user_disable_posted_writes(VFIOUserProxy *proxy) +{ + WITH_QEMU_LOCK_GUARD(&proxy->lock) { + proxy->flags |=3D VFIO_PROXY_NO_POST; + } +} + static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); =20 --=20 2.43.0 From nobody Sat Nov 15 15:29:49 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255299; cv=pass; d=zohomail.com; s=zohoarc; b=cGxTC7P1KxH7A14tcoipDH19IO77Z+Fej98qPfYawo1E+y+JtCyu3sbQqdloJkgGswUkozEq3BlZDv02wqaX+sves/XNbCI2gQB7RSZWt2KVoGvOr9EsE8yw0nuw6AZY7KAoy1MyxhrmocmnYtfG/SGPFUw7yevs8TRvKrz1DW0= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255299; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=CXd2NeK/ET6uD8gmkhbk2vh4nYzQVltKbqv/Wx4cyuA=; b=mZAaMUZiw3GpctSysciVeMcARfDnWDwXjx/LACr5ENivbuQjZxGsL9jiGLjMSAeYjYJT+jwzh3/Z8iZfTZmCb+NLwPlOK9i459O7IWTBTMVByjqvCyPyveGvd8SkUYz7HpJLPf+t44t+iaSucnoG+pvQHEDUJJiaC5ueS+qUgYc= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 174925529940268.9515714442764; Fri, 6 Jun 2025 17:14:59 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNhA4-000365-90; Fri, 06 Jun 2025 20:11:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNhA2-00035K-Dd for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:46 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNh9z-0006RE-VY for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:46 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556N0RtB023133; Fri, 6 Jun 2025 17:11:32 -0700 Received: from nam02-bn1-obe.outbound.protection.outlook.com (mail-bn1nam02on2126.outbound.protection.outlook.com [40.107.212.126]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 472njsxqb2-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:31 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by CO1PR02MB8441.namprd02.prod.outlook.com (2603:10b6:303:154::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.17; Sat, 7 Jun 2025 00:11:30 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:30 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=CXd2NeK/ET6uD8gmkhbk2vh4nYzQVltKbqv/Wx4cy uA=; b=mzXqVpK6NozzyFqNGxD8Gre9vWglDRDC8JQeer0EOb3KStirwgwPEa42r 3Ek7l4Sp+6XWFiCC3wLoegwvq6/g9c9q1OYTmDIC4W0u1qHhgKO+Kwl8063Tef1h Uq9J+PUPXchn7sjTYUL6xhWOh5yj6UfcvsoOsc75E4YUm88dQZRpkJKrkUNsu8Jd d75cfFFE3i18V++dcsRnqFZCpPFv0MYGjUYfTsqzqMIR1I8ZRcAc8Vzc54vVa5wo 6EIq8iCa/D4Y/ffv4yqYXrJHsyCK8RMs3PiGrpIsBVxWMJhp2VeC1BYlah1JbvlF hsoVbGrx5RVg/lxcCRmCYoedsuGfg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rTu+34QADzrjUa3+3hS16AHH/3spXpeNlHzx7trZtqPGczd4oR9+kR8FEld7aq/s+tq8dkCMrghSqhLec+KZoyvjezAq4F5JEr01GPwsXn29GgikJasXpOZP7F/pMjK4udbTEfyoYfFDTm+Z96IHh9VeNR557v9EUv/asz8zImDULqCgv/SR6Ol9ve1/gmtKlb0hXhmXCLD1y/ykZYLTTYIIwGXqRBIXbAk/fA9vLvYYN/Wakfr9rKK6GKsnHdVCWJvy3A9WNV1FNu/CzB+5qo4CFmb047IjkTB1Mq6sKcOOcusI2I0Resjo9jGf9TULXRI4WdAnyMkUzcuZjIVacw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CXd2NeK/ET6uD8gmkhbk2vh4nYzQVltKbqv/Wx4cyuA=; b=a+ThgQOrBpbduCQyapJZuzKaRSpGH0lUwEjG+G4PvPRL2TA99u/PVSVLnaZAuICqMqZU8yy+RADmC3O6uN+bLNJT/uvTGr8t/irgnrBTcAU0v6NcCQii8r2DyKXvvRFruAoZfRgW3Tdl4npobXOizbyqvIjYrSXbyWepglAEFCgCIVZt4vfVbEJe9Sqdc4hCMwfOaXGCjMhtXSzo8wg5XyDl4B4PVkxKTqcaj+sx+A8cogXuKTUIj+BD13D9DDs27ZHgsRoB50+y5ak+uKiQXsB/wHVUvH/vQGYOpYVjgJzBYwQSdLIOBR04DYUArBtvUFgRROMdCGtuqBwF3jU93g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CXd2NeK/ET6uD8gmkhbk2vh4nYzQVltKbqv/Wx4cyuA=; b=g2rZCOLgUR/zS20nGGF2lFpK06cwk+nZmmXOs/raKdOlJlXjt3M/HJ2XAFfLorvAA3dH77aOTw9dvJGJSZRe4OGcuBNiLzzOQshjRsjAHH/bKeyJglllQE9BQUnwKzXRl9TgphZO7SZUXBY0jw95/xHkN3pF+J5LWKb680+SP+oBRPX126AP4T9ydHcRi/ODnUuacuYLqLNY7+HldonZWbbVLmq6Tkcz700ZolrXqeqxaBGDTOw06GXAsQgIvm7Nwp+tD1fRtgKIPKjnn1bPmRzlj6nYVf2arjvi5X6KqHgk+dhLkwPXtZAkdl6AuEDwrKm7TZA0SOlmEa2Wvfg05w== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v3 22/23] vfio-user: add coalesced posted writes Date: Fri, 6 Jun 2025 17:10:54 -0700 Message-ID: <20250607001056.335310-23-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|CO1PR02MB8441:EE_ X-MS-Office365-Filtering-Correlation-Id: 0127dd35-a6c5-4c23-5030-08dda557da69 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?4S0Un5bbLljfBOLw6Q3BrwigCWqpkrrWDr67oXFzyK2Ufi209QitUx30cWhf?= =?us-ascii?Q?ijtrngGKg2bTyGvXcmnplsWGSV5hTmlqakKu2B2ihkunRJTIcQnquVQEspt1?= =?us-ascii?Q?pYp0s8jUX6ToCJEm1uB9SEWAndVMll1nyxMACarNSsaL5YQ1UU91JZs+YJpy?= =?us-ascii?Q?pEy5KQgvGuZMgWjOp9+0AOABDeKMgdF91oWvetY8WP61Hrrl+w/EccwOHkAy?= =?us-ascii?Q?UZa3YUZqAKkDl8RPE9a+PpbLbioJL+7rRsAX8hNd0oo50AFbSgjkHKkC53pE?= =?us-ascii?Q?VWFGwvZj66p3vMrq3JS/8Cp7yb8+TwoIh+s9h6tR5+og0YQd4GR9dBSBQ2YY?= =?us-ascii?Q?5nxCgQSpTiOap2X2zKVUZq97h6Kono+kbJDePgGVNDM4lrjLO/i0+HnTGp4r?= =?us-ascii?Q?qkx/yAt6fgf+PcABaX68Nr8u5gjg+24J8RXKQUEWqgzWOX/u01Fh1laVLLbd?= =?us-ascii?Q?Zsh5MvwjHKL3HUCDKt0gsw6+tbZUqi+vJIgOLORV4PXlY6V9pdM+7m6RIdfb?= =?us-ascii?Q?2foKFoWOo8/XPj+XtBSDD1hTVP/z5y8qrPuFlc2V97M9fS5cqkat+hTeSWGD?= =?us-ascii?Q?dA8/wOuI05EAEWfNEmszkmhOVvSeBt1kSnMnMW/cIfpkHtgw36dELtfHEiMl?= =?us-ascii?Q?TKHJdheo1CPt3jf6Ynj/LRSaPrXewB8qWDOqck9sutAmLXlxZ5mj/yVpZ7AP?= =?us-ascii?Q?OPcSZnnfqRNkkXCoLld4wXU6uzwihlsZM1UBBEOABl7dxxuC+fwEnTJvK+u8?= =?us-ascii?Q?75ASvxi6FQ6wrj2JU7RJqdx62glgLGHZYjVJM1mgstNRydncvlz4gaXZOrgf?= =?us-ascii?Q?TcWgm9XR8lNiO1PU3KSxW/qoUQXgTnW5Wx1HQqjRb+icheHOTm4R38h9b6Nm?= =?us-ascii?Q?Ppe8NgLKT3kjIjw3V1PNfqYazCl0EAFyxkrRb1xtSr3VuNSTPrINw6RJPBUR?= =?us-ascii?Q?IlZjY8QYTy2qCVM5G6arcXsn9pTv4v5crPqTsXiaRJCghBVU1SPh91uqmF6E?= =?us-ascii?Q?Gvl1s6lcsHY7eG7yFcVBkdmAoJ/XaoduXTdwJ6U0w2WI5NQV65WtoV52spzf?= =?us-ascii?Q?6GPNtyeKN515fp0fhbD5eE6TIJVcqn5KkIA5GMBRl82thHONXZICHfwl3HyT?= =?us-ascii?Q?1S3FMRqzlsVv5qxX3Ixu/G2GRpTUBhJ7u516Teog/6pyEUcRG3K3k9mNjv3q?= =?us-ascii?Q?nGe6lQPe9RjeCSiz+RYTojAWYO228BeelPA4PCcYnHVmJ/GsEyZZifURrKdQ?= =?us-ascii?Q?peSyahpyn6fxzVCxBkaxNgmh8lRR9AL5j3EZfRTcgWrD0qIEsNZvrGNYEIOE?= =?us-ascii?Q?2H+mPBjMhbYwdVxle9oS06mMB1I/unpEwx/+tlvSsjfwZX8CRc3fRE2RhIDo?= =?us-ascii?Q?X9gsVZUSDUHvUX63zRmbakkD58H+dLs0w8Z6JlP2kZ82nKvacw=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(366016)(376014)(7416014); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?VTvnWGz4HhWroV03B84BQFp2SfDbgyFVB4PpzfZLRBkMbPx7VCSFMgYJ1YQ0?= =?us-ascii?Q?ORvgvOsSoVhieV4zRZrqA9AahlF8UXXzxZ18CXGWWFYuvV5igIuS7oYmL+C7?= =?us-ascii?Q?j5WY51HM3FIIAOEtpsNzsd9WNq/VKasWzz8Qb6xA9axBwgiXN80sQ+e6WCy1?= =?us-ascii?Q?9jh0lTcHBh9DD1Q/uf0KCClgvDqIxQsAKKRhEahFepFIyARit4nuZrnUHCf0?= =?us-ascii?Q?iJ2+V66SL8y6pPv188oh82UBvkUsn+IeVitggrTMZn83KQtRTn7VELtFDSdk?= =?us-ascii?Q?ScOu1IpTY7fBRYwPS2w0ydp6FD3VkqDS2Zg2Rfe+yYsVFlT2G0LMoSLxPefm?= =?us-ascii?Q?togwWqFbQQyti5ZApsp9RxI6TTTJrxMpumYVKNjfT75K2uNycyZvWYvjXdFj?= =?us-ascii?Q?MQcvZQEeWeIwjPZrg0D7IdND8S/ILtRXwihs5zNxYhqtliGKUf9dsWbYxZjz?= =?us-ascii?Q?NDmWhHtwdVUAHebhYsrgS7/XH9uz98ZgqAKYJcjXP9MLU1HoxYldw5pX5zw2?= =?us-ascii?Q?BkrWopqXLTZC4FcBMsBKqLcu0/EfnuC2SFP0MNlGRAE0qJBGVJ/SledHKUSW?= =?us-ascii?Q?jzRh+9g/mEOShG90PjTzjmOVfEaFBWb6R18uLCQ9XFU9vxlxYXlgAacqOFGp?= =?us-ascii?Q?rzIz4d26Id0DQ4nGtmx5Nvy69mk7cm7lSleDdgQYctXgqgoXfytze1/kfcrs?= =?us-ascii?Q?8yza/rXq/Yx3LXi/gpqCPcv9U5VN/9WLl6RUtztB5/On8F9MBCV0YT5dORed?= =?us-ascii?Q?JRzCDLpgy5T30OrZRAvni8En2cZK+CGRZ98lUJlHqkMVl3e+lju/xz+tF4k9?= =?us-ascii?Q?PUeU/HD2et9WZ6/GLayZ+93NfMs/k62yYsitJRdDqppTuO2JcMsqR7VbVi6j?= =?us-ascii?Q?2DmIRN11XcrcGb/9RkronacrbOJf6zQKGm7qbGIwVM7UHanrUZ0jZ6sJHCtu?= =?us-ascii?Q?zjKmUPO7vnvfP5Zf5rn0N7xCiryvhgW50IauncTIkbrl24WHYxTtKLXWH6Ix?= =?us-ascii?Q?qqisvvSfvR47yLTdlIkvj3X326y8ZGpAyXX2XKYzJ0XgPMs3veZ2X/vFpAiz?= =?us-ascii?Q?4TI6AuH0hlGWLIlSw4YO//q6lWqeDrMMSodeG2VKpzD+/gyvvb7vNhBaxiBe?= =?us-ascii?Q?4Ykwb4KgLUi5jmPnC5P2f/aJrM3zEid6w2nG9l/0a5gOUfsGX/5EmsuN2x+i?= =?us-ascii?Q?3h6UOr6tHO4PId9PXJPsZ6PokOUvwJyNbnfrwKotL4J2pdn9diMO/13Si8Km?= =?us-ascii?Q?o1wOcWGFFyXNTXqUI2zpJF+8KxSqmoj2omtiQfrmy4uKY6Iao8JXeeKoOf68?= =?us-ascii?Q?RyU1LNNU/ygzs8cncG+ErZ4sC1aJNkhFbZaGS7PCcOs2b1OSC6nLYDmuEJux?= =?us-ascii?Q?sv8eKO+muqv73t0/kyNKtyxZBTVGpl/hVALxLKvPTzVdhO1w+sDKsZDXydk8?= =?us-ascii?Q?LyBoZpqDxBG7ABii895gf0sOR8mLQfI45KxxjSXE1Xt7gIkX7qOq535GmAiz?= =?us-ascii?Q?M/khvV9i17S/xgGKjLLTNEHiPo008oklNXt9dhDo3lRw3XRTDVdVhx/bPfcx?= =?us-ascii?Q?t4hKXVpwjRJk4yK8gAnbGgzhPc+lN0vPGg2+XuyU?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0127dd35-a6c5-4c23-5030-08dda557da69 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:30.3615 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: cNZcYoQy6JHPjHG3GCMYBak4/63oeG7gF7YNfHsZzxh0/4ft+zyXZlENmX2JMvsduboPUEgAWvP3Mom+oYtwQg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR02MB8441 X-Authority-Analysis: v=2.4 cv=f9JIBPyM c=1 sm=1 tr=0 ts=684383b3 cx=c_pps a=JomqzcHDjT5Ni/tnCvtrfQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=pwxC0n7aooic8zlvp-IA:9 X-Proofpoint-GUID: 8rZo5teVQRLoszLzSTIQlTXrS9KqCpDq X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX/EP+BRcMYpMw lwtDhph20WPzjtfmhyWg+qUkePP0tKxycqVLesNJfpfc9sM0f0yAdFtJ1B+2LCsJGqqBTiOzGQ+ gUvmOezpWUPvc2UDiPf5/A3upgV9Erz8bCA6CUGdwUUdbOLJW4nYrzFtybigpZmgwAmDZr/NgNc trQhduNRPBTPQo4uyXrc0k0RG1bNGfkPamCLgDGnRyBB5oFEUItuJ+/YSvPXp809mRGOGoIMvTg JRS493aTk3HbA16XeQ+VSL0gLqOLqiFDXf7iox8inX3ucqsLzyVAHICgfzvpWxNWAHs6XWpEVDT DfI423fLYbwM6WSRQVoSk/ftYxDbFGGhc44HWZunOXxE+DXTdAM7/KXOOWFWKb58qWOBy5XDtrB tv0jh6An0xjNPwJjOkCS5HM+jn8AYJZpickfk5uGMmSd+ZcL/dOjF2T15UoQD+UtQXTYL0rA X-Proofpoint-ORIG-GUID: 8rZo5teVQRLoszLzSTIQlTXrS9KqCpDq X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255302070116600 Content-Type: text/plain; charset="utf-8" Add new message to send multiple writes to server in a single message. Prevents the outgoing queue from overflowing when a long latency operation is followed by a series of posted writes. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/protocol.h | 21 ++++++++++ hw/vfio-user/proxy.h | 12 ++++++ hw/vfio-user/device.c | 40 +++++++++++++++++++ hw/vfio-user/proxy.c | 84 +++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 1 + 5 files changed, 158 insertions(+) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 8f589faef4..f747e70e50 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -42,6 +42,7 @@ enum vfio_user_command { VFIO_USER_DMA_WRITE =3D 12, VFIO_USER_DEVICE_RESET =3D 13, VFIO_USER_DIRTY_PAGES =3D 14, + VFIO_USER_REGION_WRITE_MULTI =3D 15, VFIO_USER_MAX, }; =20 @@ -75,6 +76,7 @@ typedef struct { #define VFIO_USER_CAP_PGSIZES "pgsizes" #define VFIO_USER_CAP_MAP_MAX "max_dma_maps" #define VFIO_USER_CAP_MIGR "migration" +#define VFIO_USER_CAP_MULTI "write_multiple" =20 /* "migration" members */ #define VFIO_USER_CAP_PGSIZE "pgsize" @@ -221,4 +223,23 @@ typedef struct { char data[]; } VFIOUserBitmap; =20 +/* + * VFIO_USER_REGION_WRITE_MULTI + */ +#define VFIO_USER_MULTI_DATA 8 +#define VFIO_USER_MULTI_MAX 200 + +typedef struct { + uint64_t offset; + uint32_t region; + uint32_t count; + char data[VFIO_USER_MULTI_DATA]; +} VFIOUserWROne; + +typedef struct { + VFIOUserHdr hdr; + uint64_t wr_cnt; + VFIOUserWROne wrs[VFIO_USER_MULTI_MAX]; +} VFIOUserWRMulti; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 22ed66c54f..ae09b9cc60 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -88,6 +88,8 @@ typedef struct VFIOUserProxy { VFIOUserMsg *last_nowait; VFIOUserMsg *part_recv; size_t recv_left; + VFIOUserWRMulti *wr_multi; + int num_outgoing; enum proxy_state state; } VFIOUserProxy; =20 @@ -95,6 +97,11 @@ typedef struct VFIOUserProxy { #define VFIO_PROXY_CLIENT 0x1 #define VFIO_PROXY_FORCE_QUEUED 0x4 #define VFIO_PROXY_NO_POST 0x8 +#define VFIO_PROXY_USE_MULTI 0x16 + +/* coalescing high and low water marks for VFIOProxy num_outgoing */ +#define VFIO_USER_OUT_HIGH 1024 +#define VFIO_USER_OUT_LOW 128 =20 typedef struct VFIODevice VFIODevice; =20 @@ -122,4 +129,9 @@ void vfio_user_send_async(VFIOUserProxy *proxy, VFIOUse= rHdr *hdr, void vfio_user_send_reply(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int size= ); void vfio_user_send_error(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int erro= r); =20 +void vfio_user_flush_multi(VFIOUserProxy *proxy); +void vfio_user_create_multi(VFIOUserProxy *proxy); +void vfio_user_add_multi(VFIOUserProxy *proxy, uint8_t index, + off_t offset, uint32_t count, void *data); + #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index eb2194c0eb..79375ddc96 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -10,6 +10,8 @@ */ =20 #include "qemu/osdep.h" +#include "qemu/lockable.h" +#include "qemu/thread.h" =20 #include "hw/vfio-user/device.h" #include "hw/vfio-user/trace.h" @@ -296,6 +298,7 @@ static int vfio_user_device_io_region_write(VFIODevice = *vbasedev, uint8_t index, VFIOUserRegionRW *msgp =3D NULL; VFIOUserProxy *proxy =3D vbasedev->proxy; int size =3D sizeof(*msgp) + count; + bool can_multi; int flags =3D 0; int ret; =20 @@ -311,6 +314,43 @@ static int vfio_user_device_io_region_write(VFIODevice= *vbasedev, uint8_t index, flags |=3D VFIO_USER_NO_REPLY; } =20 + /* write eligible to be in a WRITE_MULTI msg ? */ + can_multi =3D (proxy->flags & VFIO_PROXY_USE_MULTI) && post && + count <=3D VFIO_USER_MULTI_DATA; + + /* + * This should be a rare case, so first check without the lock, + * if we're wrong, vfio_send_queued() will flush any posted writes + * we missed here + */ + if (proxy->wr_multi !=3D NULL || + (proxy->num_outgoing > VFIO_USER_OUT_HIGH && can_multi)) { + + /* + * re-check with lock + * + * if already building a WRITE_MULTI msg, + * add this one if possible else flush pending before + * sending the current one + * + * else if outgoing queue is over the highwater, + * start a new WRITE_MULTI message + */ + WITH_QEMU_LOCK_GUARD(&proxy->lock) { + if (proxy->wr_multi !=3D NULL) { + if (can_multi) { + vfio_user_add_multi(proxy, index, off, count, data); + return count; + } + vfio_user_flush_multi(proxy); + } else if (proxy->num_outgoing > VFIO_USER_OUT_HIGH && can_mul= ti) { + vfio_user_create_multi(proxy); + vfio_user_add_multi(proxy, index, off, count, data); + return count; + } + } + } + msgp =3D g_malloc0(size); vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_WRITE, size, flags); msgp->offset =3D off; diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 13f2407845..dbaa322952 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -16,12 +16,14 @@ #include "hw/vfio-user/proxy.h" #include "hw/vfio-user/trace.h" #include "qapi/error.h" +#include "qobject/qbool.h" #include "qobject/qdict.h" #include "qobject/qjson.h" #include "qobject/qnum.h" #include "qemu/error-report.h" #include "qemu/lockable.h" #include "qemu/main-loop.h" +#include "qemu/thread.h" #include "system/iothread.h" =20 static IOThread *vfio_user_iothread; @@ -444,6 +446,11 @@ static void vfio_user_send(void *opaque) } qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, vfio_user_recv, NULL, NULL, proxy); + + /* queue empty - send any pending multi write msgs */ + if (proxy->wr_multi !=3D NULL) { + vfio_user_flush_multi(proxy); + } } } =20 @@ -464,6 +471,7 @@ static int vfio_user_send_one(VFIOUserProxy *proxy) } =20 QTAILQ_REMOVE(&proxy->outgoing, msg, next); + proxy->num_outgoing--; if (msg->type =3D=3D VFIO_MSG_ASYNC) { vfio_user_recycle(proxy, msg); } else { @@ -571,11 +579,18 @@ static int vfio_user_send_queued(VFIOUserProxy *proxy= , VFIOUserMsg *msg) { int ret; =20 + /* older coalesced writes go first */ + if (proxy->wr_multi !=3D NULL && + ((msg->hdr->flags & VFIO_USER_TYPE) =3D=3D VFIO_USER_REQUEST)) { + vfio_user_flush_multi(proxy); + } + /* * Unsent outgoing msgs - add to tail */ if (!QTAILQ_EMPTY(&proxy->outgoing)) { QTAILQ_INSERT_TAIL(&proxy->outgoing, msg, next); + proxy->num_outgoing++; return 0; } =20 @@ -589,6 +604,7 @@ static int vfio_user_send_queued(VFIOUserProxy *proxy, = VFIOUserMsg *msg) } if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { QTAILQ_INSERT_HEAD(&proxy->outgoing, msg, next); + proxy->num_outgoing =3D 1; qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, vfio_user_recv, proxy->ctx, vfio_user_send, proxy); @@ -1112,12 +1128,27 @@ static bool check_migr(VFIOUserProxy *proxy, QObjec= t *qobj, Error **errp) return caps_parse(proxy, qdict, caps_migr, errp); } =20 +static bool check_multi(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QBool *qb =3D qobject_to(QBool, qobj); + + if (qb =3D=3D NULL) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MULTI); + return false; + } + if (qbool_get_bool(qb)) { + proxy->flags |=3D VFIO_PROXY_USE_MULTI; + } + return true; +} + static struct cap_entry caps_cap[] =3D { { VFIO_USER_CAP_MAX_FDS, check_max_fds }, { VFIO_USER_CAP_MAX_XFER, check_max_xfer }, { VFIO_USER_CAP_PGSIZES, check_pgsizes }, { VFIO_USER_CAP_MAP_MAX, check_max_dma }, { VFIO_USER_CAP_MIGR, check_migr }, + { VFIO_USER_CAP_MULTI, check_multi }, { NULL } }; =20 @@ -1176,6 +1207,7 @@ static GString *caps_json(void) qdict_put_int(capdict, VFIO_USER_CAP_MAX_XFER, VFIO_USER_DEF_MAX_XFER); qdict_put_int(capdict, VFIO_USER_CAP_PGSIZES, VFIO_USER_DEF_PGSIZE); qdict_put_int(capdict, VFIO_USER_CAP_MAP_MAX, VFIO_USER_DEF_MAP_MAX); + qdict_put_bool(capdict, VFIO_USER_CAP_MULTI, true); =20 qdict_put_obj(dict, VFIO_USER_CAP, QOBJECT(capdict)); =20 @@ -1228,3 +1260,55 @@ bool vfio_user_validate_version(VFIOUserProxy *proxy= , Error **errp) trace_vfio_user_version(msgp->major, msgp->minor, msgp->capabilities); return true; } + +void vfio_user_flush_multi(VFIOUserProxy *proxy) +{ + VFIOUserMsg *msg; + VFIOUserWRMulti *wm =3D proxy->wr_multi; + int ret; + + proxy->wr_multi =3D NULL; + + /* adjust size for actual # of writes */ + wm->hdr.size -=3D (VFIO_USER_MULTI_MAX - wm->wr_cnt) * sizeof(VFIOUser= WROne); + + msg =3D vfio_user_getmsg(proxy, &wm->hdr, NULL); + msg->id =3D wm->hdr.id; + msg->rsize =3D 0; + msg->type =3D VFIO_MSG_ASYNC; + trace_vfio_user_wrmulti("flush", wm->wr_cnt); + + ret =3D vfio_user_send_queued(proxy, msg); + if (ret < 0) { + vfio_user_recycle(proxy, msg); + } +} + +void vfio_user_create_multi(VFIOUserProxy *proxy) +{ + VFIOUserWRMulti *wm; + + wm =3D g_malloc0(sizeof(*wm)); + vfio_user_request_msg(&wm->hdr, VFIO_USER_REGION_WRITE_MULTI, + sizeof(*wm), VFIO_USER_NO_REPLY); + proxy->wr_multi =3D wm; +} + +void vfio_user_add_multi(VFIOUserProxy *proxy, uint8_t index, + off_t offset, uint32_t count, void *data) +{ + VFIOUserWRMulti *wm =3D proxy->wr_multi; + VFIOUserWROne *w1 =3D &wm->wrs[wm->wr_cnt]; + + w1->offset =3D offset; + w1->region =3D index; + w1->count =3D count; + memcpy(&w1->data, data, count); + + wm->wr_cnt++; + trace_vfio_user_wrmulti("add", wm->wr_cnt); + if (wm->wr_cnt =3D=3D VFIO_USER_MULTI_MAX || + proxy->num_outgoing < VFIO_USER_OUT_LOW) { + vfio_user_flush_multi(proxy); + } +} diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 7ef98813b3..64fac9137f 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -11,6 +11,7 @@ vfio_user_get_region_info(uint32_t index, uint32_t flags,= uint64_t size) " index vfio_user_region_rw(uint32_t region, uint64_t off, uint32_t count) " regio= n %d offset 0x%"PRIx64" count %d" vfio_user_get_irq_info(uint32_t index, uint32_t flags, uint32_t count) " i= ndex %d flags 0x%x count %d" vfio_user_set_irqs(uint32_t index, uint32_t start, uint32_t count, uint32_= t flags) " index %d start %d count %d flags 0x%x" +vfio_user_wrmulti(const char *s, uint64_t wr_cnt) " %s count 0x%"PRIx64 =20 # container.c vfio_user_dma_map(uint64_t iova, uint64_t size, uint64_t off, uint32_t fla= gs, bool async_ops) " iova 0x%"PRIx64" size 0x%"PRIx64" off 0x%"PRIx64" fla= gs 0x%x async_ops %d" --=20 2.43.0 From nobody Sat Nov 15 15:29:49 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1749255235; cv=pass; d=zohomail.com; s=zohoarc; b=K0dZkjSw0S0flfUIdDucYA3f6HosCRfwqwi1LIndrFNzJ95nDkGj4BEMJqf4jZqFgjwrlkpZD0a0/i4ZQyHHPiE3TI66qLmzh2/ttVXjSDggQamlOfXUXam40cscpKfOwutE0wq/XTDdVqpMx8iaZq4P2BDuYGalC5eQFDPnLgo= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1749255235; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=NYHfEYolJYtB7ztdkTtSBJXJfFu8OJVZtPxqxjVBUgw=; b=DMRxcyYm/zEZpw+X8FHEJYX5EmgKOQ+zIlxJQJta31ciJ1GegbTnSShnHS3NciHVoMnMphlHzvI1Yeb925jL/b4nzrDZ5fruamUDGZxWH6cxuwwW2tjntRiOjrFTgZLXfRrek1Pp2Wyygu/7XBtcSyQ8u3E7UMoOlKMVYl2CMOU= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1749255235714166.2714490988801; Fri, 6 Jun 2025 17:13:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uNhA7-0003Am-TH; Fri, 06 Jun 2025 20:11:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNhA6-00036U-0M for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:50 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uNhA3-0006Ri-V7 for qemu-devel@nongnu.org; Fri, 06 Jun 2025 20:11:49 -0400 Received: from pps.filterd (m0127843.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 556EFBe0024977; Fri, 6 Jun 2025 17:11:36 -0700 Received: from nam02-bn1-obe.outbound.protection.outlook.com (mail-bn1nam02on2099.outbound.protection.outlook.com [40.107.212.99]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 471g88usth-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 06 Jun 2025 17:11:35 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by CO1PR02MB8441.namprd02.prod.outlook.com (2603:10b6:303:154::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.17; Sat, 7 Jun 2025 00:11:31 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8813.016; Sat, 7 Jun 2025 00:11:31 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=NYHfEYolJYtB7ztdkTtSBJXJfFu8OJVZtPxqxjVBU gw=; b=21TdSuFBeITNP8J32VP59eluNZObuO/zBt1Ymev3rVj+J0z5lep0qlVV/ vVnoTSssn5jVOLyBjz+8Jk9URGdW2aWEVgGGmPhMTF1egN8oT2RWAZGxe/835l/K f/bDDuazTobcYbhTxFzbEuKLwW8EPBelhvmHbAknl147JCmha+SCw8nzqBDcYZ0L 4RvX5Qi6ZAwdTDh3kvnWuTxaIDWFGLpRuaQGl+n2hPUMc5BtvEJWdQ+6nhajn2WS eVuOxfH5XRuKgwqWdyVbpMLMKw1JGHqgdd3KLovn8EW55Kk+r+9XayE/iZBLflp6 t8ySHQ5SLJVv5tKUHcCqWDERzKKSg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=F0lAdfX2soRZEGNok+98eRAEA8YThac+NaVkCDkjlLDezuFG5RFZ9x+ThrIPie4fn+cNi0fL314NVipMe2v3Ns3lVg+WUX3ywBu03ktIcIA62dClZxvo59+foEDTC5mZYUMMesRzqKBH6x7j7xP2vfyefAXWvoU2V9kptET8ys+hdvOBSfGg2ZQvMVXj/V80/K85R0fOgsw42QQxk6lqGm3oXY5kNW5uY9hQ8NlXtocHT0N/oXqfe/xInlW3BirbB3z5G7hlys2kxw4jTa1x3JnFWvjXEVp2em4v2sjXMigaEr4SKhXbLOWkrjsXSV+5CKvQw0XW9Ux3w6pNK3iJHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NYHfEYolJYtB7ztdkTtSBJXJfFu8OJVZtPxqxjVBUgw=; b=WlopAGm+g9T+MOfFc7do/5d1ceTluUR6g/a261Bx/DWwtVwUERJLRr5Fiur7EwYdCoEv9+OmMeWS7veTDBgz7alSmqDSHrjm8DGvppFxRYrCwELU0scmfDCGMbK1/ZVCjtqNIU+OJxmdCDXyJQqVPlT0XyAOyCzBno8TmKrbeEqHFIlMWZYMpda3hFiQPP/UK1wqdQ3+O9Zrpowcnr0zSCPJ30KZ9VjHvUQqs9IJbmRzQnC9c9TeRLp+iTkTE74qH9RY1dUrlsrDs2tvj8b2gjHo0EiW2EqlkuUtErjOsFerIWHH1qOFghGI0dg4Fv/P/E+ICPGduOigzAl+STt1KQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NYHfEYolJYtB7ztdkTtSBJXJfFu8OJVZtPxqxjVBUgw=; b=F/hObrJ1a1quQpO8GzcPzrlOuYAvSQeTqsNmY4XzELRcSnKW7x3t46heD+ZGfc9sEra5og/I3+mkVzGQCVsI/FRWw3A+5VhlFDvGGgcFHWUHLCGOg6pzImmBe8s3ZYjeu+kAVwbnX4p7oZMjzFWOA2lq+mOeqAJr4mMDFgAUD8pztZKUFjo15s86oVZxAzJuSGm0ZpU/gBcpxvfmBXgYHVicmwibRTH7FRr5sn5jfwa7t7x7nYtGMQ+61GCc6Nzxw44BfNYcevaXVXeJ9mp1WBD+vJzdMyyV79yHWo09/0LIK0uyXjEubC6oO94kUAJ5FEfvOWe/UI+JMLucyeUW5w== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Thanos Makatos , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Alex Williamson , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon Subject: [PATCH v3 23/23] docs: add vfio-user documentation Date: Fri, 6 Jun 2025 17:10:55 -0700 Message-ID: <20250607001056.335310-24-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250607001056.335310-1-john.levon@nutanix.com> References: <20250607001056.335310-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: PH8PR22CA0004.namprd22.prod.outlook.com (2603:10b6:510:2d1::19) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|CO1PR02MB8441:EE_ X-MS-Office365-Filtering-Correlation-Id: a550d208-a55d-4823-cee2-08dda557db28 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?JNBLrWJPvmUwXttaja0d5z03BlYXsNQ+bXDRjHpP9uIH4+Z9fVqygN8EgUMW?= =?us-ascii?Q?zQJ/UZeIc+2I5PamQjeztiEhZxjqiO2z4VHiV/sj8V3+T2EC0285GCKSFAzJ?= =?us-ascii?Q?wsPmLg3p1B/RmUFUJt8nTPe4wXCELW/8eYVjm9ubPT9AoQKXpNlawrFNsgk8?= =?us-ascii?Q?0ZnDko2EDcdkCSuNOYrspOmFzv/JJo1x3vvcxzelS0svXOTgqVdJRbo7gg22?= =?us-ascii?Q?zI7FiZq37TOriuwlv+5dH6aTVjbGfGteCQy5j8nP+N3SSJVNfySCv1j7dGtD?= =?us-ascii?Q?w5sG+ahIVdp2anxFao2YsPhZyLQyxN930S3k0TvpKSiMFp4nbAgIEIH4Ii6Q?= =?us-ascii?Q?5KFjVjjS3Ktbe99CxOEzbtv7T6uiFV6fptfJB72wS2NnxXSqt1dD9caA2cK7?= =?us-ascii?Q?CH2812h54Plf6RfBf8d+/2iThf7k2GCrPWWUEhIq3FMRxfmMrjc+Y9Zt1FuS?= =?us-ascii?Q?dmZlW8cN7zrXuVJ+na69DE5ZDl/co9Jr+e6Evq7jPTCq1PEAwf61kohodVgA?= =?us-ascii?Q?fiVr3UbUKB3mrYCPCS5sOCwuJf4GnbDa4XTXpUpId0ChR+u68fZeXDUpyKCZ?= =?us-ascii?Q?4PHSgFIe4PCc2OiZQAd8BdfVSSjHHxE7l59fOixnRcB6l0CXUeWNR9xWZ7/W?= =?us-ascii?Q?hrRizwUx8RfFeqFxSDjG6J4ocJ2I11HAAmRHXQ68MKoea+kR3jCESjrxvlrl?= =?us-ascii?Q?fdmnj/y/WRlBMmtbBpXqiSneHww+n4kRbnw5sxgTmz5ScNVeO4k3pgJPh4bT?= =?us-ascii?Q?wTaY2n0HQfcy+2WMTqe0DtPcmncrf75EyT1CSMJzC20GLuLBS09hUdL72I8C?= =?us-ascii?Q?lCLvl+k1JDXfTtIhg1oY7Vzoizo7w5zm9AjDhCP1VF/vNIJHA92Du6022LoO?= =?us-ascii?Q?EzIc+3Qttt0GYh9QSYmamGekxajf3jjXF0r4Du/Nrk4WlUSCY5qPvsExO2hs?= =?us-ascii?Q?vkkYJyYbT/AcX0mdJrGea18M4gCA9IANDLInjAvbzvld8n9MZACgUIFbC4ZL?= =?us-ascii?Q?27uqsk2mMJs/zSPH1oIbUT9+aT2jbwrxBNEIFaPKVhHj+Kx/VApynV1tl2mS?= =?us-ascii?Q?L50lFamGZQKkj94PjSM1RtweWQy5tTCKSizYQrDN8iN0sfy+1M+HmD6MFSs8?= =?us-ascii?Q?CxT/jiNc7TnMzyscv4WPWuSbR5emtUMub/Cht2vNG2ceGyuK3JyujHjukH6J?= =?us-ascii?Q?SEzWYsFEcTP8DCBVYQEmDMZ8inQmlcPqlXFfbhZNnQ4jezJJ41Dn0pJquXlE?= =?us-ascii?Q?tipyu5FLDA4L+ZYv8yi1RYg34g4Y/pxsFzt0qEprsjTAXL/EswoDblbugz+T?= =?us-ascii?Q?b7hssVGvdet7kCeDqnXZQ8DnmDAxWMNEvnTA95ZHAAGVxwopp2bSjcnLVEz3?= =?us-ascii?Q?JQ69Nh+4xyYZDAQ8IJQzt8IxNlYvOPXSjPK+9UMa3xBq1v+5uvAfy75yn5FP?= =?us-ascii?Q?Hl6GkYY93hg=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(366016)(376014); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?WGG4nTgl1ok0X1HrAB0bpssHZ/OaKvE9Mz2pfmKMvdYfyp7wqqQ4cLEz15mo?= =?us-ascii?Q?uoewugm0CUnaQOBnHBr6T5kM07ExZ9YDcaXqaJcvY+ef37McZcfDR/kYIn6h?= =?us-ascii?Q?2xNTMg52QSUfcqlTDESKMmBGq8Ln6yhPgADwc8g5wnrwa22or4zZVW8nVfOm?= =?us-ascii?Q?VIzRIl0NxKVaKdROD4cSNGHM86c/v2NeKZ4h3LAbDs0Q9vEV5RqwLj5vd0Z5?= =?us-ascii?Q?7VLUggvOdhQNBZvxGnXcFuxBLv+VDylQuP4oPghNLyQ9Iw1odiJnaTQj822Y?= =?us-ascii?Q?+Y2q/cLBytexFBS5AVRhcKxaQrV6D0XGB6FXs2Cy6bAdG1rALmktNxgKusPC?= =?us-ascii?Q?uoZxM4i7qj0Gu3bjylBc8Ww46wGrnnUC8kKWs1DqP9r0MRm2KJ9xf0QKruIl?= =?us-ascii?Q?JaB80PpeQS67wJdera7QrfwRiCCvqRcQ+xfJnmNxjP1F8Ym23m3qBl+/h+7a?= =?us-ascii?Q?K8grY6A8QMppGoxfpFZxf2gXSLYs+mS39KtsFOAYiKX3Wr2Yh/7rZJYL9pK8?= =?us-ascii?Q?NeDqZKN660NdqKWkxWqMi+J2XCSW3A7ooeMB+PqxIp8FHNCTchLqBOeNZ19l?= =?us-ascii?Q?yXHUhNYbnfblO8M4R6N9mePrf1SVX/PG8pBzuV6mP/yqz5jE0y0FpLVYqiL7?= =?us-ascii?Q?7hmemJ2VONkkfxS5+dj3yFpKwoN4HCE/ny8Ri6x0ZWKSPmDM5Pc0AN8nL8Ym?= =?us-ascii?Q?Dbm57qeK3md1terQiEQhIvCDd+AoYC12JXCyERerQGlxQnz83/qY+2mrbhFu?= =?us-ascii?Q?rd0iBxQGUbmPW1e+o4bhkq6c3qkJiG0nU/6ke/GTZndjplBu3T0V6gbOlEjn?= =?us-ascii?Q?ZD9rP7n7+R27oxOOfhPoH3DzAqNvZprifeNcccwlCv/SOf6JcQs7Vsl7k3FI?= =?us-ascii?Q?qlFfs2n3/7YbIG5NEfTC4vtlTLm0Uq38Tkg+sqaZmswc5UP8l2ACW+2WMy6m?= =?us-ascii?Q?0gyw4blUuua1UIyn2Kyi313w5v4pqC0XBhrEgP2OiNQcCdMIn2S2OZ505N7f?= =?us-ascii?Q?hTce2gHFbqg3O96FydEkMcSePiv3AHYc4Ugx8xDr1tWnexS+L42hV6bTcTU1?= =?us-ascii?Q?fqUwx8NKyBzPQYiLm2yDj0mFfAw9cu2kM74+y8e84z7dyqNEX7LCgAFuAVb4?= =?us-ascii?Q?ra2YxbJiPXIxb6XINLNn65HD1QHxER6Kxd7zc6mpoVOkhhBp1l5o6sHP96vN?= =?us-ascii?Q?3mjMJ9FoaUyvE2kCT5CQ0bQikGmDvb4QQGlCwuh/pTSpsEcE5Pi92Lg1pqYH?= =?us-ascii?Q?m09fbgnKPKrgu8m1dtRkthAbsDtOhuy27UXQZwsZceUorJFz2ubC1SVF9Qqi?= =?us-ascii?Q?+CuBc3lFEHLz1PDJCND9enMc2kbUrtQx9dtgcOjwufFcJeWbgVjZvJchXiPy?= =?us-ascii?Q?9A6XTbRRPb2Yp5EYYZFz9icecJfvMnHCaaGpTRgBWQ/M9XtWHtuLLNMxu/NG?= =?us-ascii?Q?GfZBG1akDxE37+joHP9kKnGA/2kf5XX3Ofu7gSfv+a/ehTzSWFvInJbG+lkN?= =?us-ascii?Q?+nw2K/ph6pqIkNZAVpWxZ5bHegzIl7GNu0uvbGWLYd6kPoXT1NTVrS7toc6k?= =?us-ascii?Q?uXNVY5SlwOQ1CD1LroRiM+NZ8VIEhMJkDjQ7dM+f?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: a550d208-a55d-4823-cee2-08dda557db28 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2025 00:11:31.5880 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ov7lhJ7S/j4OFgQsk7Y2Q/HtUiKo3I5DGc7gsFVa0LzfIH1EnNXE/vywxp34MaueXmiYMAo0EsEaWHgZ3/pt8A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR02MB8441 X-Authority-Analysis: v=2.4 cv=K5kiHzWI c=1 sm=1 tr=0 ts=684383b7 cx=c_pps a=XWLsCJYX8W/xo5I+dSU+/A==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=NEAV23lmAAAA:8 a=64Cc0HZtAAAA:8 a=LOSiUy74ef7l4vre4JwA:9 X-Proofpoint-ORIG-GUID: XMFr4WMXYlmMY1hYjim5pNWPB4_RdRmP X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjA2MDE5OSBTYWx0ZWRfX8lFvaUtWGXcw Q5NXgdhOa9creKi3r5ZzrZFsbGS6/A9OaKg9yjTxP/eEw5keBf7otslMAU9phMXGgUDinrjYqx4 AxolqcZsJkn5rfMKWOmjGSv/JXOK+0uNJHd7+rzmn0bLaWjGLJv3HQw+jzxBRNLE6ZVe34idQrJ tUUw62ToCx3DnC/TrxtkrohKNcIEfBZGOl985modG62qfFoj8e2MKW0wNSTTAVM/RyEJRTO3eJ9 c1OSpxC2yU+RmdNYT0NBXrvDjtCgv1mNKeh/d9BzxMZWB8XUOzX5G00vgOLtt7IHZ6zVLv6Y5gi R6yxWyilUUQazi4sDqqiuJla0dCmRqZOQpjb2czzLStB7/07UIIM/pbMW4Srrp+xhSwjynlj6bw rBSbcznpmNWrmIWd0cB/8mG5JIfIek5l4rD3KJ/w8jmV1Ju4lMMN4QgikT8l2KB5bh0btAR+ X-Proofpoint-GUID: XMFr4WMXYlmMY1hYjim5pNWPB4_RdRmP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-06_09,2025-06-05_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.104, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1749255237230116600 Content-Type: text/plain; charset="utf-8" Add some basic documentation on vfio-user usage. Signed-off-by: John Levon --- docs/system/device-emulation.rst | 1 + docs/system/devices/vfio-user.rst | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+) create mode 100644 docs/system/devices/vfio-user.rst diff --git a/docs/system/device-emulation.rst b/docs/system/device-emulatio= n.rst index a1b0d7997e..911381643f 100644 --- a/docs/system/device-emulation.rst +++ b/docs/system/device-emulation.rst @@ -85,6 +85,7 @@ Emulated Devices devices/can.rst devices/ccid.rst devices/cxl.rst + devices/vfio-user.rst devices/ivshmem.rst devices/ivshmem-flat.rst devices/keyboard.rst diff --git a/docs/system/devices/vfio-user.rst b/docs/system/devices/vfio-u= ser.rst new file mode 100644 index 0000000000..e33e49d283 --- /dev/null +++ b/docs/system/devices/vfio-user.rst @@ -0,0 +1,24 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D +vfio-user +=3D=3D=3D=3D=3D=3D=3D=3D=3D + +QEMU includes a ``vfio-user`` client. The ``vfio-user`` specification allo= ws for +implementing (PCI) devices in userspace outside of QEMU; it is similar to +``vhost-user`` in this respect (see :doc:`vhost-user`), but can emulate ar= bitrary +PCI devices, not just ``virtio``. Whereas ``vfio`` is handled by the host +kernel, ``vfio-user``, while similar in implementation, is handled entirel= y in +userspace. + +For example, SPDK includes a virtual PCI NVMe controller implementation; by +setting up a ``vfio-user`` UNIX socket between QEMU and SPDK, a VM can sen= d NVMe +I/O to the SPDK process. + +Presuming a suitable ``vfio-user`` server has opened a socket at +``/tmp/vfio-user.sock``, a device can be configured with for example: + +.. code-block:: console + +-device vfio-user-pci,socket=3D/tmp/vfio-user.sock,bus=3Dpcie.0,addr=3D0x9 + +See `libvfio-user `_ for further +information. --=20 2.43.0