From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340211; cv=pass; d=zohomail.com; s=zohoarc; b=fBXBLrl1heGX1coW1neI2zURQQRCuJs93g0STnDkpakh8u/OXw4GqfiF+LYcG+pD5fzjrBrJi0889vEahduUHMMTzvSFVFrCDqv4FrCZ91M77WgGVgrk8zRJeIUoX/n4CYgAbfcabzqqVAALpiRzVAZ4nNp21dhGGUmqLNIVLKA= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340211; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=TBLpsxtYnKCZGPa6nWZU0WtIG+dmzl9JSrFKnvaCKpA=; b=HWqMqUXqMyekFzMjTBId/pMqeLsHO62YeFUbbOvtWAm3Rpcrqij+CuPnz9K2njRn9Yex38yRjEyPkUX8BAmYTTYNItWs0FnFF0142vTZCtEUt7CgrL0XTfPXZ+WYAPH03uQ4K+sjrTeAf9aUdyVAvd567AGxSC2GbnvgtyVY4N4= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340211217696.1991137369174; Thu, 19 Jun 2025 06:36:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFNe-0004Ey-39; Thu, 19 Jun 2025 09:32:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNc-0004EA-Ex for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:36 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNX-0008Dk-VM for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:36 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55J7OixC013270; Thu, 19 Jun 2025 06:32:28 -0700 Received: from byapr05cu005.outbound.protection.outlook.com (mail-westusazon11020091.outbound.protection.outlook.com [52.101.85.91]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 4798mykwpg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:26 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:02 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:01 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=TBLpsxtYnKCZGPa6nWZU0WtIG+dmzl9JSrFKnvaCK pA=; b=MM8E0UHjIR+os5eSKAkyKdHamQS45QcrNorlzdX3QsIsn9hg6je+GV0uR 2z7ayPcY4MegvQ3GRLFj8L1ve+ZW8FSNO8xIvnpMNWTLjKVsVc7OLUjx/qki2V+i xGr+JjvHtDgkOUnrxJ+gdxJP5voyZWyqKPBe/YMBcq8MkWVc7DW2BY8pz9AKIfhx 6NP5KCmtPHpwBadz/HM7tbF+4vgScXIyifePRYjCwGN3l8o0c9WVurmy/WpgguTp gkEHMUkkmXpCCyPnLg+ghAn52FJQnVaL3AAPDKjnv8yZiYhv4nUK/xVqob3FHjdJ DQvOPhyof1ntL1yTH4hazcdjkcjsw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vU8Oy5yGLI9+sdWtZuI+jkmsBKrYLZn8uJ8qtOeMao5L2A6PiVAdd+/h82lmmVJ2ydlZj+fwnAZnMRsi0MNuni5e0qhvpcE//oYxXfyWKAb0u+A3HZUatwqJE3HRjaKe6buJP5SKjHJMNSXBO3BdOdLMwUzE0V+TeGPR+IcF5RfaSZSwvzRLGTaVvX4VZd7HjX6TTZErJ7kZw44iHLI1Grf/Nc9hfqhvxcu4YRnhwkC9r2lDXhPj2/CNcKar6IlUcE2h/l7vWIpnmXjNMwLq1D2zg+6VU57BA2rNa4lBkrCqTgRYDYt+H6SnUX1f9w+LZ80U8J45wJdJcBqjIvpDuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TBLpsxtYnKCZGPa6nWZU0WtIG+dmzl9JSrFKnvaCKpA=; b=CEqR5kjY1i2K+HZI/CS1YLZnxIJkUKRxhTZ5Oh8wfuPqyGtmRCnlHjT2ob0945mZulluZZiNsr8kK9BBfquMB5IiSII7wa0C9LHJ2s60diZVyUhqFVaLvGT6lanNkfjm42cmzHln6tdBqGQKzKMsMI9XibKI91VDlW795Gz6pnhjwvu1iH2Iu+pOFGiuf/d6TPUdRojea48NF2zf36Gcvvc8HTC2k/aFdmXvrwY/6lsMSHffc7Y7ZH6AV2axcYLGCwuyznawTb51HMe20b56WLHQLWnd91/7Or0Lv3Z5YmhaLqRKo6nBayWFqgx2iuX+UTyFk4/OibKYy/hx4bi87w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TBLpsxtYnKCZGPa6nWZU0WtIG+dmzl9JSrFKnvaCKpA=; b=MfxvGguwF8oa0nLweahVhMsJxR/RpFwRsZySvn7TIuJ8TMzZ0M8NFFPrlLUBME7R76YdDzTmYiJg+e6t2S6ZvxSYovrhmiqu5ziWA+vml+CCa9pPS/dsSwuOilo04dwwXyoEZbSFuZiNTdMqUWbgwUNRLcoUC2wAI1AgDuXGtLMM+AReZ2i5I20RbSrMR8cTvPF5du5syRPxyl0VE94T5WT3S2FuRt9Z23liENROhI7DfyuLoFMRRoYcHPboPVIsOT6C3Ge5caZKlpARO0ENcOF/kKfia+AdnL9TRFfnlkStuafwdqX9DuVpbzP61YYn07CMpyMo7/CA0IyWreuT9g== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 01/19] vfio-user: add vfio-user class and container Date: Thu, 19 Jun 2025 06:31:35 -0700 Message-ID: <20250619133154.264786-2-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 59e7245a-a726-4bac-e290-08ddaf35abe9 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?QzVrUXZnS2xKUzg2SVB3Y0RKY05yUkVsZDhrTTRFNTBPRGRDam55cWRwSFBw?= =?utf-8?B?TXQwN0plMjU3ZzhRVU44UXRVN3gyZURzUHlJUWI0czZBOXhuMVhWYmJDYUs2?= =?utf-8?B?U2RSMS9rNUVqRnNib2ZwdFBoZG1RSGhUb0gwbloxS1lhNjRmaVEvWDFVTGc0?= =?utf-8?B?Ynk3TzVEcHFLRWJUcU53SGVDb0RNTmo0eitEbmJyQlFYWnhmdEFRRGlsaUF4?= =?utf-8?B?eGlzeEF2TXBVRzNkdXdUL05wNnFkbmFMemg2eUNTeWt3MTlwUXZ4dnFhS3Zj?= =?utf-8?B?MEN2Q09EdGpqUG13a0xrcWdVMXF2b3ZxTllCZHBscE9uejZ6RzArbDYzWVg2?= =?utf-8?B?ZG53RUpCQi92dXVkWktQWDBoOE56TjhUVkNqaUp0MEZVSzMwd3ljUWFVN2Rj?= =?utf-8?B?cUtqTkxSbTU0dE9xUmFkN1dGZEdONDJzaVcycjNadmxKNktadFVtMVcyaUZ1?= =?utf-8?B?c3hIZVcwb1FIZXNrQm1ERy82d0drWlJoWVVpRTlFMmxvWUUyZ0kyeDNPMkxa?= =?utf-8?B?dXRLcFlLTldUdWo4TnZmSStJVmJNU25RVC9OYlg5K3RaMmxUVG92eERzZklB?= =?utf-8?B?c2hoYUhWdDVISG5BZ0dqbDhjdWgrbEtOeGJGc2hkN09USjlaZlIvYjRlWlFK?= =?utf-8?B?NWkvWko4dXlCMTJYbDloZHVVQTJ2WmRRTEVuTlQ2RGh6M05VOGREZUY5QzNt?= =?utf-8?B?TDd6U0hNZXZPckFOU0QrYXdTbDhpM0xEN1VMR3pXTCtBVHZVWEdhNFFLMUVS?= =?utf-8?B?RGsrN2JaYTBkYlZqN2Jpd21iK1EySk9IUjVsRi9UdnhkS1pnSVJuRVJmV0Jt?= =?utf-8?B?UzVmc29uUXFiWW4yQ0VScGd0SlY1T3IvdmZjTlY4Zjg2RU5XaHlNV2thMGlZ?= =?utf-8?B?bndrWkNtYUZLTVhSRGQ4UjJlRGh6ODNQSVhhUUhjb2tSanZvUjlVcGxGQ2ls?= =?utf-8?B?cXNlUE1LYVlNbFRoWDc2RmVyM2ZHejFhR0hWcW9kVCtvdjRNbnpCMy9IWmx1?= =?utf-8?B?SW9SQUdGZC91c3MwUTIwaVlnVW5aK2pzOXcraXdxNUc1VjJNZnl1TkNHdG9X?= =?utf-8?B?VnBCdkx2OXkweU9hTUVEeUJRQXF4WFphWktlUlFsZUdrOXpZT1RkR3dncU9x?= =?utf-8?B?TkVha1hVU01sc29ITHpaTXFhcThyWk5GUGM4dkZnbmxNMnZLZUxPYmE1ekd0?= =?utf-8?B?Z3F4MzA3ZTBKbVpSMGlhV21hRU9FRUdVeXk0bzd0RjNiaE5kRmlaR0h0VG1i?= =?utf-8?B?bW9ZbzM0b0hnRUNQeG5BRWtYaHZoWWs2L2VYM29ZM2w5Z0pMR096azJJcGFY?= =?utf-8?B?djYxNFA3bDcycGswM0MvNDkvQU5UOWlkNzhJN3NaOVl6KytrMDFaNnlKYzZS?= =?utf-8?B?ekRqYTY3Q01iaDVmTlUrSXBqRmsyNWRmS2FpQ3E2eGwvOTdIazFoUzN2T0FY?= =?utf-8?B?UklGNGhlNjJodjF6SjlXbWlGY3RFS1NtWHR3Y2o1UUIwMHJ0WWNReHdzMldq?= =?utf-8?B?TElmL3pGZGhXK2tkSW5PSlhRdkFOQm5RWDNQUlVMeElMbzF0Wm1uZWw0bmxs?= =?utf-8?B?WVIzbzdDVGE2U0NlZ083N20xV0c0UjZMQXNwT1krSzdITmd4NWc3K2tmZkhR?= =?utf-8?B?MTVRNThrUmVWTWNMcm5KSEYzcjZrSWpJMURCS01JanZXMVNBSE1CSDZIcTRh?= =?utf-8?B?VENlNjBDTUwveEdHY3hCSFFCOTZGc2V5eEUrbVdLQTcxWmlybjNyZXN3ajlr?= =?utf-8?B?VjlDalRJRWFwSDdIeFY0WitFbzBDYnFRQXFnZzkzZThuemphOE1DT0FrZFhP?= =?utf-8?B?Yks1WmNDTHF0Ui9DTHJmWDhSVnRBRjZzNks2aGs5K3crbmFLOXF1MHpmWnRi?= =?utf-8?B?d1ROU3d3UUJ1SXBvNk41Q0JJWjJkNCtKV3VKcEd1UGlIZm5FUmNrWGRDQmVp?= =?utf-8?Q?iSju/NunLQc=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bnJpRDBGTFc3cmk0RTVMN3p4eWdja1kxTyszNjc0T3QrOGZibHVNUWVlZDVP?= =?utf-8?B?OUVkNW1BOVZ0djVDT05sOER3czJ1Y2RTRUlFbWI1U3h0RFJCWEg2bWtHL1Vq?= =?utf-8?B?dHNWMUxQcHY4U2w0MktBQ3FZWW9sVGx0aWI1WHRFdHZrSHRiT2JiakwwS1dr?= =?utf-8?B?VkpqTlhhZGpYT3NtVDR0V3lkMUF6TFNFbE1OZWJRRWs3RTc0cS8xdlZrOFEr?= =?utf-8?B?WWJFLys0UlZMNUJ0RHRMMDAzWWQwelN0a00zaDFQelNhNGlSNVQzYWNkRW84?= =?utf-8?B?SWgwVVEwNFdqYjRyOCtsanppdUtDVHU5a2VtRnR3QnNVWTh4SmZpdVRXWThG?= =?utf-8?B?dFdBMG5jVjNXUHd1bC8xczdhQWtJYUVmM3JwaVdIYitJZ3NlZkdmWjFoRHho?= =?utf-8?B?OTFWeEdWOWZRbHpPNythczZGaGRRM1ZXZm1jZjNPcUdiRXNhMkM4SlBaalRB?= =?utf-8?B?Z1FqSnl2T0RGcVU0THRBdGE5a1hPRGVPcnNtRW1mMmN4cEIzUHJSblA1N2ta?= =?utf-8?B?WGpnYU1xaHFFWExhOW9saGZ5cStSWE16bjdZcDFEdVRRdFphSzh3cWhUazJl?= =?utf-8?B?UHI5MjhCMUpVSWpORTcyZFpLYWRZVERINWVYTzJtNmJ3Z2JXalBMWHYrQ2Qy?= =?utf-8?B?TFJISWh3ejFZNm1BbFY1UWsvOER2NHVKVlpCeGREclZ4U3J0UC8xQjV3bzl2?= =?utf-8?B?K2RsajBsRkVqbzVIWjgvYnFXaTA5a0VnQ3JtOWpwbWdOUGU3ckJzWC9jQ01q?= =?utf-8?B?S2VUYzJaYWViQmlXSytBNzZkSUwwSCtvbDdWb3ZhUnRtcDNlckt1R21EWmFN?= =?utf-8?B?WGZlYkI3cU1jbU0vZU5zRU5NUElkb1RvUytVRWZFOWtUT2ZJM2xFZFU5d3I1?= =?utf-8?B?dEduSWtEZUJpaHNIdDR2MkVxd0hYdkNEcndoVHZ6THFjS1d6N3RkWEp6dThP?= =?utf-8?B?NHY2eXZnRGRFbnRLMFdOak1MaW1Ib1dsYjBOUkhwL1E4UXRTRnJrd21xbSsw?= =?utf-8?B?ZFI2WnNoQ3JhbHBXUXFNMEZrMnpIdzYrZ2lVZ0djVGhVZS9vTndLVXdPcE5Q?= =?utf-8?B?TkoxQVB1YUxFa3d2RUM2a3VKdVJMN1JwbzA3dVRBbkhFQitvT1FsZFdTVml1?= =?utf-8?B?TUV4SWZ3azdvL2s5a0xrWjNuTEwwNllpb3VJaUkrQiszSWpTbS9JOE9SQnRD?= =?utf-8?B?Y3AyRys0amZWYkJPSTdKS3NKSFk3WjV4NHJPa0FNNmVWTWdqRmlRTUpQcVlM?= =?utf-8?B?UlRLd3dIaDVkd1IxTkczelFRK0NCRVRMUDkzNlNZSVBJdHJ0R0RLUnZoUlI2?= =?utf-8?B?Tm1kT0dCc25ZUjZ6SDNabC92L1FibkNaRnhFZGdEN1hiaUVTVm5KQmM0SitZ?= =?utf-8?B?OGxwNzVGVDU3SzZxc1VTd0R6Z09mTkpVbk92RGRJTW56RDkxSmJyNUw4ZVVK?= =?utf-8?B?eS9haFBvK0dUTllZN3hFWlhyQWdrQVBmTWFXenlVa2NUSTNMME1mcm5hS2sv?= =?utf-8?B?cTA4RXZvMHpZRHVPd0NpMThhOXFXYUk4dVlUbzEyekdMS05KTVRENU5lZnlW?= =?utf-8?B?Z3lQQ0U1OWpsZE1qcmY2Rm5oOW1rYzFjVGZXM0YyaFpzaUhPbU81eXdhM3NQ?= =?utf-8?B?SjBMdUFySkpuM3phSUlSWEVSYzZyZFNIT3dkSDQwaW9GcTJLc3RYaEd0OWd0?= =?utf-8?B?Wnk3bjl6V1p5Q2pZSXRCYTg3MnEyUVp5cDZjNlZPcy9uTWsxUDVuNDF5bXBL?= =?utf-8?B?Mnk3bm1GeHNCQUwyKzltQ0kyNmNFYWNlbnpvcDNsejhYYzMvYmJVTDRmUVE4?= =?utf-8?B?aTU2UC9XRlYvU3k4cURKL1lpc1pJY3pSalFtUll3azVqWkUyYVFpbzJoLzFz?= =?utf-8?B?MkhieDVGbEtFQ2lMWWNZdXRBakk4NmE3NnVzeU1oSitCWjNqRXpsN1RrMVBj?= =?utf-8?B?Sk9kYnNCMEI0VmliZitmSWNCaHA5OXNBZmlVbm8xeitPNWFUb2plSkxOb0Zy?= =?utf-8?B?a0xVZCtTb0pTcTByTDJ1K0tVQ2FrVXJ2L0NjelI5TytHaUI4OGljZ1VFWjhM?= =?utf-8?B?OU1UTDBINU9hT2dQMXpPNHpIWW5NZG9kdlFwcmJQOUhIdzJZMjdzSUdNa3Iz?= =?utf-8?Q?1l16P6dZu8rqt1VeF4MGkc8yE?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 59e7245a-a726-4bac-e290-08ddaf35abe9 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:01.2416 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Lbg6euvmcgpS8jKrlBTRF5+RnBcQ/elK0X6M01XTfIrgL2/Zc42vDZjSpCmru3kwlpKwZvrdHJDjFCK9ilMsow== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMiBTYWx0ZWRfX7wlzT5TqWtIk UAUYeMUKtcSsqxQ4iYQTzgOsL9i6tFLD55qwuDN08dh9qDNzgpbMqvFX3PlkdEpMoe1bKmsSYiU bYqya7Pt21ANOofG/VWtqnHS+cOZd/qlBmiXrxJVT2SMamIutaxAE7enGfQMEDUcSrpRSZ5+7aQ Is5G6i5WfVdsmPY6nPLSKiSoRIY0GUe5z2mnPz/h9fNB5CoJw6xVZqYie1XydiEtVr/tDL633e2 XK31E6KrPUZe8rqiqLJi/hVXf63na+sK7c1SLkkAbwRVH+BZH6y2opjY92I14dDME1FCOu8nm41 BKRGp670U9KE3TeC+en1P+Set4xhtJgFLPAFdK8TCojlHaVcdyqCfgGvud0rZaTp2fhSKLo2k0Y Bv5YV1wF35X8IA9hrhjmQpQ3/dHLTVLj8JPB9E/sUvNInkEp1kCWbPecgCEVYCe9OYKv9xet X-Proofpoint-ORIG-GUID: fqNH5t3eKzaCS41BkZH8CGHJfRgJQAoZ X-Proofpoint-GUID: fqNH5t3eKzaCS41BkZH8CGHJfRgJQAoZ X-Authority-Analysis: v=2.4 cv=LIxmQIW9 c=1 sm=1 tr=0 ts=6854116b cx=c_pps a=kkQK6hjpUACXm7ua2zl0Pg==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=20KFwNOVAAAA:8 a=z4glEzOvAAAA:8 a=u7rQ6wk7R_5OE3_9H9YA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 a=92dS5hN0c3Q7EetK7xW5:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340212662116600 Introduce basic plumbing for vfio-user with CONFIG_VFIO_USER. We introduce VFIOUserContainer in hw/vfio-user/container.c, which is a container type for the "IOMMU" type "vfio-iommu-user", and share some common container code from hw/vfio/container.c. Add hw/vfio-user/pci.c for instantiating VFIOUserPCIDevice objects, sharing some common code from hw/vfio/pci.c. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- MAINTAINERS | 8 + hw/vfio-user/container.h | 21 +++ include/hw/vfio/vfio-container-base.h | 1 + hw/vfio-user/container.c | 208 ++++++++++++++++++++++++++ hw/vfio-user/pci.c | 185 +++++++++++++++++++++++ hw/Kconfig | 1 + hw/meson.build | 1 + hw/vfio-user/Kconfig | 5 + hw/vfio-user/meson.build | 7 + 9 files changed, 437 insertions(+) create mode 100644 hw/vfio-user/container.h create mode 100644 hw/vfio-user/container.c create mode 100644 hw/vfio-user/pci.c create mode 100644 hw/vfio-user/Kconfig create mode 100644 hw/vfio-user/meson.build diff --git a/MAINTAINERS b/MAINTAINERS index 28b3dd2684..9bc1005b07 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -4250,6 +4250,14 @@ F: hw/remote/iommu.c F: include/hw/remote/iommu.h F: tests/functional/test_multiprocess.py =20 +VFIO-USER: +M: John Levon +M: Thanos Makatos +S: Supported +F: hw/vfio-user/* +F: include/hw/vfio-user/* +F: subprojects/libvfio-user + EBPF: M: Jason Wang R: Andrew Melnychenko diff --git a/hw/vfio-user/container.h b/hw/vfio-user/container.h new file mode 100644 index 0000000000..e4a46d2c1b --- /dev/null +++ b/hw/vfio-user/container.h @@ -0,0 +1,21 @@ +/* + * vfio-user specific definitions. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef HW_VFIO_USER_CONTAINER_H +#define HW_VFIO_USER_CONTAINER_H + +#include "qemu/osdep.h" + +#include "hw/vfio/vfio-container-base.h" + +/* MMU container sub-class for vfio-user. */ +typedef struct VFIOUserContainer { + VFIOContainerBase bcontainer; +} VFIOUserContainer; + +OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserContainer, VFIO_IOMMU_USER); + +#endif /* HW_VFIO_USER_CONTAINER_H */ diff --git a/include/hw/vfio/vfio-container-base.h b/include/hw/vfio/vfio-c= ontainer-base.h index f0232654ee..3cd86ec59e 100644 --- a/include/hw/vfio/vfio-container-base.h +++ b/include/hw/vfio/vfio-container-base.h @@ -109,6 +109,7 @@ vfio_container_get_page_size_mask(const VFIOContainerBa= se *bcontainer) #define TYPE_VFIO_IOMMU_LEGACY TYPE_VFIO_IOMMU "-legacy" #define TYPE_VFIO_IOMMU_SPAPR TYPE_VFIO_IOMMU "-spapr" #define TYPE_VFIO_IOMMU_IOMMUFD TYPE_VFIO_IOMMU "-iommufd" +#define TYPE_VFIO_IOMMU_USER TYPE_VFIO_IOMMU "-user" =20 OBJECT_DECLARE_TYPE(VFIOContainerBase, VFIOIOMMUClass, VFIO_IOMMU) =20 diff --git a/hw/vfio-user/container.c b/hw/vfio-user/container.c new file mode 100644 index 0000000000..2367332177 --- /dev/null +++ b/hw/vfio-user/container.c @@ -0,0 +1,208 @@ +/* + * Container for vfio-user IOMMU type: rather than communicating with the = kernel + * vfio driver, we communicate over a socket to a server using the vfio-us= er + * protocol. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include +#include +#include "qemu/osdep.h" + +#include "hw/vfio-user/container.h" +#include "hw/vfio/vfio-cpr.h" +#include "hw/vfio/vfio-device.h" +#include "hw/vfio/vfio-listener.h" +#include "qapi/error.h" + +static int vfio_user_dma_unmap(const VFIOContainerBase *bcontainer, + hwaddr iova, ram_addr_t size, + IOMMUTLBEntry *iotlb, bool unmap_all) +{ + return -ENOTSUP; +} + +static int vfio_user_dma_map(const VFIOContainerBase *bcontainer, hwaddr i= ova, + ram_addr_t size, void *vaddr, bool readonly, + MemoryRegion *mrp) +{ + return -ENOTSUP; +} + +static int +vfio_user_set_dirty_page_tracking(const VFIOContainerBase *bcontainer, + bool start, Error **errp) +{ + error_setg_errno(errp, ENOTSUP, "Not supported"); + return -ENOTSUP; +} + +static int vfio_user_query_dirty_bitmap(const VFIOContainerBase *bcontaine= r, + VFIOBitmap *vbmap, hwaddr iova, + hwaddr size, Error **errp) +{ + error_setg_errno(errp, ENOTSUP, "Not supported"); + return -ENOTSUP; +} + +static bool vfio_user_setup(VFIOContainerBase *bcontainer, Error **errp) +{ + error_setg_errno(errp, ENOTSUP, "Not supported"); + return -ENOTSUP; +} + +static VFIOUserContainer *vfio_user_create_container(Error **errp) +{ + VFIOUserContainer *container; + + container =3D VFIO_IOMMU_USER(object_new(TYPE_VFIO_IOMMU_USER)); + return container; +} + +/* + * Try to mirror vfio_container_connect() as much as possible. + */ +static VFIOUserContainer * +vfio_user_container_connect(AddressSpace *as, Error **errp) +{ + VFIOContainerBase *bcontainer; + VFIOUserContainer *container; + VFIOAddressSpace *space; + VFIOIOMMUClass *vioc; + + space =3D vfio_address_space_get(as); + + container =3D vfio_user_create_container(errp); + if (!container) { + goto put_space_exit; + } + + bcontainer =3D &container->bcontainer; + + if (!vfio_cpr_register_container(bcontainer, errp)) { + goto free_container_exit; + } + + vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); + assert(vioc->setup); + + if (!vioc->setup(bcontainer, errp)) { + goto unregister_container_exit; + } + + vfio_address_space_insert(space, bcontainer); + + if (!vfio_listener_register(bcontainer, errp)) { + goto listener_release_exit; + } + + bcontainer->initialized =3D true; + + return container; + +listener_release_exit: + vfio_listener_unregister(bcontainer); + if (vioc->release) { + vioc->release(bcontainer); + } + +unregister_container_exit: + vfio_cpr_unregister_container(bcontainer); + +free_container_exit: + object_unref(container); + +put_space_exit: + vfio_address_space_put(space); + + return NULL; +} + +static void vfio_user_container_disconnect(VFIOUserContainer *container) +{ + VFIOContainerBase *bcontainer =3D &container->bcontainer; + VFIOIOMMUClass *vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); + + vfio_listener_unregister(bcontainer); + if (vioc->release) { + vioc->release(bcontainer); + } + + VFIOAddressSpace *space =3D bcontainer->space; + + vfio_cpr_unregister_container(bcontainer); + object_unref(container); + + vfio_address_space_put(space); +} + +static bool vfio_user_device_get(VFIOUserContainer *container, + VFIODevice *vbasedev, Error **errp) +{ + struct vfio_device_info info =3D { 0 }; + + vbasedev->fd =3D -1; + + vfio_device_prepare(vbasedev, &container->bcontainer, &info); + + return true; +} + +/* + * vfio_user_device_attach: attach a device to a new container. + */ +static bool vfio_user_device_attach(const char *name, VFIODevice *vbasedev, + AddressSpace *as, Error **errp) +{ + VFIOUserContainer *container; + + container =3D vfio_user_container_connect(as, errp); + if (container =3D=3D NULL) { + error_prepend(errp, "failed to connect proxy"); + return false; + } + + return vfio_user_device_get(container, vbasedev, errp); +} + +static void vfio_user_device_detach(VFIODevice *vbasedev) +{ + VFIOUserContainer *container =3D container_of(vbasedev->bcontainer, + VFIOUserContainer, bcontai= ner); + + vfio_device_unprepare(vbasedev); + + vfio_user_container_disconnect(container); +} + +static int vfio_user_pci_hot_reset(VFIODevice *vbasedev, bool single) +{ + /* ->needs_reset is always false for vfio-user. */ + return 0; +} + +static void vfio_iommu_user_class_init(ObjectClass *klass, const void *dat= a) +{ + VFIOIOMMUClass *vioc =3D VFIO_IOMMU_CLASS(klass); + + vioc->setup =3D vfio_user_setup; + vioc->dma_map =3D vfio_user_dma_map; + vioc->dma_unmap =3D vfio_user_dma_unmap; + vioc->attach_device =3D vfio_user_device_attach; + vioc->detach_device =3D vfio_user_device_detach; + vioc->set_dirty_page_tracking =3D vfio_user_set_dirty_page_tracking; + vioc->query_dirty_bitmap =3D vfio_user_query_dirty_bitmap; + vioc->pci_hot_reset =3D vfio_user_pci_hot_reset; +}; + +static const TypeInfo types[] =3D { + { + .name =3D TYPE_VFIO_IOMMU_USER, + .parent =3D TYPE_VFIO_IOMMU, + .instance_size =3D sizeof(VFIOUserContainer), + .class_init =3D vfio_iommu_user_class_init, + }, +}; + +DEFINE_TYPES(types) diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c new file mode 100644 index 0000000000..86d7055747 --- /dev/null +++ b/hw/vfio-user/pci.c @@ -0,0 +1,185 @@ +/* + * vfio PCI device over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include +#include "qemu/osdep.h" +#include "qapi-visit-sockets.h" + +#include "hw/qdev-properties.h" +#include "hw/vfio/pci.h" + +#define TYPE_VFIO_USER_PCI "vfio-user-pci" +OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserPCIDevice, VFIO_USER_PCI) + +struct VFIOUserPCIDevice { + VFIOPCIDevice device; + SocketAddress *socket; +}; + +/* + * Emulated devices don't use host hot reset + */ +static void vfio_user_compute_needs_reset(VFIODevice *vbasedev) +{ + vbasedev->needs_reset =3D false; +} + +static Object *vfio_user_pci_get_object(VFIODevice *vbasedev) +{ + VFIOUserPCIDevice *vdev =3D container_of(vbasedev, VFIOUserPCIDevice, + device.vbasedev); + + return OBJECT(vdev); +} + +static VFIODeviceOps vfio_user_pci_ops =3D { + .vfio_compute_needs_reset =3D vfio_user_compute_needs_reset, + .vfio_eoi =3D vfio_pci_intx_eoi, + .vfio_get_object =3D vfio_user_pci_get_object, + /* No live migration support yet. */ + .vfio_save_config =3D NULL, + .vfio_load_config =3D NULL, +}; + +static void vfio_user_pci_realize(PCIDevice *pdev, Error **errp) +{ + ERRP_GUARD(); + VFIOUserPCIDevice *udev =3D VFIO_USER_PCI(pdev); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); + VFIODevice *vbasedev =3D &vdev->vbasedev; + const char *sock_name; + AddressSpace *as; + + if (!udev->socket) { + error_setg(errp, "No socket specified"); + error_append_hint(errp, "e.g. -device '{" + "\"driver\":\"vfio-user-pci\", " + "\"socket\": {\"path\": \"/tmp/vfio-user.sock\", " + "\"type\": \"unix\"}'" + "}'\n"); + return; + } + + sock_name =3D udev->socket->u.q_unix.path; + + vbasedev->name =3D g_strdup_printf("vfio-user:%s", sock_name); + + /* + * vfio-user devices are effectively mdevs (don't use a host iommu). + */ + vbasedev->mdev =3D true; + + as =3D pci_device_iommu_address_space(pdev); + if (!vfio_device_attach_by_iommu_type(TYPE_VFIO_IOMMU_USER, + vbasedev->name, vbasedev, + as, errp)) { + error_prepend(errp, VFIO_MSG_PREFIX, vbasedev->name); + return; + } +} + +static void vfio_user_instance_init(Object *obj) +{ + PCIDevice *pci_dev =3D PCI_DEVICE(obj); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); + VFIODevice *vbasedev =3D &vdev->vbasedev; + + device_add_bootindex_property(obj, &vdev->bootindex, + "bootindex", NULL, + &pci_dev->qdev); + vdev->host.domain =3D ~0U; + vdev->host.bus =3D ~0U; + vdev->host.slot =3D ~0U; + vdev->host.function =3D ~0U; + + vfio_device_init(vbasedev, VFIO_DEVICE_TYPE_PCI, &vfio_user_pci_ops, + DEVICE(vdev), false); + + vdev->nv_gpudirect_clique =3D 0xFF; + + /* + * QEMU_PCI_CAP_EXPRESS initialization does not depend on QEMU command + * line, therefore, no need to wait to realize like other devices. + */ + pci_dev->cap_present |=3D QEMU_PCI_CAP_EXPRESS; +} + +static void vfio_user_instance_finalize(Object *obj) +{ + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); + + vfio_pci_put_device(vdev); +} + +static const Property vfio_user_pci_dev_properties[] =3D { + DEFINE_PROP_UINT32("x-pci-vendor-id", VFIOPCIDevice, + vendor_id, PCI_ANY_ID), + DEFINE_PROP_UINT32("x-pci-device-id", VFIOPCIDevice, + device_id, PCI_ANY_ID), + DEFINE_PROP_UINT32("x-pci-sub-vendor-id", VFIOPCIDevice, + sub_vendor_id, PCI_ANY_ID), + DEFINE_PROP_UINT32("x-pci-sub-device-id", VFIOPCIDevice, + sub_device_id, PCI_ANY_ID), +}; + +static void vfio_user_pci_set_socket(Object *obj, Visitor *v, const char *= name, + void *opaque, Error **errp) +{ + VFIOUserPCIDevice *udev =3D VFIO_USER_PCI(obj); + bool success; + + qapi_free_SocketAddress(udev->socket); + + udev->socket =3D NULL; + + success =3D visit_type_SocketAddress(v, name, &udev->socket, errp); + + if (!success) { + return; + } + + if (udev->socket->type !=3D SOCKET_ADDRESS_TYPE_UNIX) { + error_setg(errp, "Unsupported socket type %s", + SocketAddressType_str(udev->socket->type)); + qapi_free_SocketAddress(udev->socket); + udev->socket =3D NULL; + return; + } +} + +static void vfio_user_pci_dev_class_init(ObjectClass *klass, const void *d= ata) +{ + DeviceClass *dc =3D DEVICE_CLASS(klass); + PCIDeviceClass *pdc =3D PCI_DEVICE_CLASS(klass); + + device_class_set_props(dc, vfio_user_pci_dev_properties); + + object_class_property_add(klass, "socket", "SocketAddress", NULL, + vfio_user_pci_set_socket, NULL, NULL); + object_class_property_set_description(klass, "socket", + "SocketAddress (UNIX sockets onl= y)"); + + dc->desc =3D "VFIO over socket PCI device assignment"; + pdc->realize =3D vfio_user_pci_realize; +} + +static const TypeInfo vfio_user_pci_dev_info =3D { + .name =3D TYPE_VFIO_USER_PCI, + .parent =3D TYPE_VFIO_PCI_BASE, + .instance_size =3D sizeof(VFIOUserPCIDevice), + .class_init =3D vfio_user_pci_dev_class_init, + .instance_init =3D vfio_user_instance_init, + .instance_finalize =3D vfio_user_instance_finalize, +}; + +static void register_vfio_user_dev_type(void) +{ + type_register_static(&vfio_user_pci_dev_info); +} + + type_init(register_vfio_user_dev_type) diff --git a/hw/Kconfig b/hw/Kconfig index 9a86a6a28a..9e6c789ae7 100644 --- a/hw/Kconfig +++ b/hw/Kconfig @@ -42,6 +42,7 @@ source ufs/Kconfig source usb/Kconfig source virtio/Kconfig source vfio/Kconfig +source vfio-user/Kconfig source vmapple/Kconfig source xen/Kconfig source watchdog/Kconfig diff --git a/hw/meson.build b/hw/meson.build index b91f761fe0..791ce21ab4 100644 --- a/hw/meson.build +++ b/hw/meson.build @@ -39,6 +39,7 @@ subdir('uefi') subdir('ufs') subdir('usb') subdir('vfio') +subdir('vfio-user') subdir('virtio') subdir('vmapple') subdir('watchdog') diff --git a/hw/vfio-user/Kconfig b/hw/vfio-user/Kconfig new file mode 100644 index 0000000000..0bf6f1191a --- /dev/null +++ b/hw/vfio-user/Kconfig @@ -0,0 +1,5 @@ +config VFIO_USER + bool + default y + depends on VFIO_PCI + diff --git a/hw/vfio-user/meson.build b/hw/vfio-user/meson.build new file mode 100644 index 0000000000..8749526cd0 --- /dev/null +++ b/hw/vfio-user/meson.build @@ -0,0 +1,7 @@ +vfio_user_ss =3D ss.source_set() +vfio_user_ss.add(files( + 'container.c', + 'pci.c', +)) + +system_ss.add_all(when: 'CONFIG_VFIO_USER', if_true: vfio_user_ss) --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750339973; cv=pass; d=zohomail.com; s=zohoarc; b=hpeg+IPJXyhm3tq2iOBJ5NCeErCz8wg39+j/YuBsZeEoVm3GRPS44cT3m1o8u0UtXYYSxbne41FicWJHlCcIIiuFWJdB2TWdfCAx9IN7Tj4Pkz2pRnslGANJWojZ2lyZMgcDt6VsUP/JkLpbQXWyLJ5lsuWbqcu1BaNBJb60me0= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750339973; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=I+gaiIVzsQ+G7ssGyHh2v17wDdt6HDwsIqmTKtnM7qc=; b=g0WdTqKiywKuCLItunT29NLfJ0bwwLPfIOqxvQeSmQ09+s7/eEHrRXMWyPTHqvhnEo2sHPW60eigDdEp/VTuVrdf0sBzit16/e9RnEWqO9206PZtyNuBiC4rDNoAPvPKgEPsjtI9haMp/P2wStBDrmg2Oh/SIl357Q2IgSVLuO4= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750339973230980.227662683846; Thu, 19 Jun 2025 06:32:53 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFNX-0004CK-EN; Thu, 19 Jun 2025 09:32:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNT-0004BS-5R for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:27 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNP-0008CY-6A for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:25 -0400 Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55JBwK8m024158; Thu, 19 Jun 2025 06:32:20 -0700 Received: from dm1pr04cu001.outbound.protection.outlook.com (mail-centralusazon11020080.outbound.protection.outlook.com [52.101.61.80]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47c0vfjedr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:20 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:03 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=I+gaiIVzsQ+G7ssGyHh2v17wDdt6HDwsIqmTKtnM7 qc=; b=YULU1reXVWFJs+U5AIluGuG2btpogxUeXWli+wSbdj/ZEah5QEjyq8olJ haXbAGuHaxNZ/AW70xXM/3seP5DrP++j2BeSwnUKBObtfDFmdWEZYsOwswQ7hEfo iavmFRAX7NQqqgUI6u5Cq/3oclKMFA0dtejhcjvNK9ptGSUGcOYMnVYwC8PdQR8h 37uF9QPlHxvX/YlD1ck8l2v/coCNYb90fSstGWLBTcEr3zXwEn1I1DknLmKvjMYE J7mRzDwoCqRX6wd4/JFiR2yiGfjROjWrRNGS/uAWlFZ/W/dbu2FRRVsvhw/Zt2QI Z/U2MmJqtWotVoYyRm9ZeU0wsI4RA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Q/H/P8F8RzW4Y8mMHdWHBmCnV3XWIWGof4625M+B52nqgylKmwgV/m/VP9fbsGOoKvjB0sjfFzMWnAT1M6ugl3XQ+T8ZUFnFBjxt59QG/oqKyjJWIVvyR3WGndZPlCc86+w9lWAPopLEmo20hQsxKr0XyuUY6uKhk57Y2BmEobz/T6XWaCuB5tMzlB1pKsN/ICCebLUe2JYk8gx3xv8OYYz01kYMZteRtTu2ClPw2CroUouela8vTrDSs5PAVbrXqxxTlZbyiGXjPDdooKVj+zoOpmEDRZkfpVicRmEUa8mxoYeXRRashLj9fDd6cZ2W6uj0Zvbi0OGQybok8fNUeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=I+gaiIVzsQ+G7ssGyHh2v17wDdt6HDwsIqmTKtnM7qc=; b=Lnc+9xLkoxIslcL9i1pnjvqiqr42/nmROQF119CeTKGOMGWjPRDeArGBDVavo6WR+nNtXJ9i5EJquXH+PnEYtKh3VDLKhYe01HL31QE4oaUJCFGtvYlVNRXI/IruFDhW7O3OLm3gRB85Uv/H/j8nlmQvQM0a5boWenc2RDVq3qnlS4UgeHRJ0rXMfgltzNt4r+Dueptv+e+V4CHaBYpIjG5ahV41hZYSBYTXdzwg6t6JfPwRvX+jqHG61Bp7nkdx5RrkHJCFsbc7IPoyyg2W3rPqU/85DLMA8hZxGbkKIHaYkmL3jAWKR76sKxUd96kB4+LjxhGfEUtjKgF15niVSA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I+gaiIVzsQ+G7ssGyHh2v17wDdt6HDwsIqmTKtnM7qc=; b=bPjgRYKWrAZaSG6De824HveoPutKU4ZqPXj9o3PuQlKyxpJ9+zYmdMmSIEk22BhYldnDwcRCMoM0CTA6o9NP0kVoGg9OIm9vewK3a+cX0NBfDSWPVfdIROM26ymNEOesS3DNyfDpa79WZxtF2/f+z8tPucVnr6bOlUw7DvA8CvGMWg7DqZQq4s89kkkJWWjP+LeuUA4IjRYN2c9m1xArfsaPWsEedRrW3JmHZUZ5Fvilc1gIXNyhKXH3hIFvEab7C3TOpKgs+UpspIaqLsJy1twjl6eDiUz2Z1cZ7p3n0PR/D6gWiLi/lcbK5ks2aKol3zi89evf/K2Wyvkriqybyg== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 02/19] vfio-user: connect vfio proxy to remote server Date: Thu, 19 Jun 2025 06:31:36 -0700 Message-ID: <20250619133154.264786-3-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 4dfe2141-6739-4793-ed00-08ddaf35ad58 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?dkVOM0Z4YWI0d3BSV1NRTUg4VDZHOWQvTjFIWUhRam1DYUprcHVyQnNlMTNv?= =?utf-8?B?NVNLenlSbDIxVGRCenIwa0JneG9ab3d4RzlzYmJKaXNmQlpMY2JELzJiZnFD?= =?utf-8?B?QUpwSENCeGNlVmo5ZGlPeGpFbVBUT09EaWN2WWxrTXQyaFMwaENoNmpkU0Mv?= =?utf-8?B?UjdGcStaR2tKWmM5Sk9sVVFzTmVONzVJVGRlUlFITEl5U0JNRlQ1cmtCQUFK?= =?utf-8?B?ZHhUNHRCd09tZTJFclB2N0hOK0MzRVMyempqSklYT2JEcmNudHVoUk9wQXpN?= =?utf-8?B?eldjdTdlSCtPdlRndHVadW4yckJqVHhoZDhBYTh1TlVndG83MlZhd21HN3RP?= =?utf-8?B?SVVFbktRTFA4UTdqRDYyaWFtY2Y0YmZycVJaUnFqZ21Wek5Ra3NVQUR1WDF0?= =?utf-8?B?dURHbi83OWNLcWFQdGZna0JMMmR5MmpYNGxGNStzTW9CWDZielYrT1Y2TC9D?= =?utf-8?B?ZUF3dTRBaGhUMzBpWEp0bTVPL3pKc1IybGRYMW1FM08yTE5LTEdtVUtWTTFv?= =?utf-8?B?bXI1SWw5dnBPNlFid0F4MjgrTFIrR01IRTl1Ky9ZTXJUOVN1aTNlOFBWTjlF?= =?utf-8?B?dlNxeUloNU45REs1aEpITUVpV25ObmRKY0ZYM0ZZdTE4Zk5rRVRuYVBCVEtT?= =?utf-8?B?WlJRWjVsTm1OdDNLNnd3Zmtkem1IT3g4YkN3VS9DN1lycEFiZFpNMFBVNmpY?= =?utf-8?B?Q2JEMXNaamVPendQWTluQnRsbmJHV1VNODl5WUNYcGM1blBNdm1HSmlCMUJo?= =?utf-8?B?QndHdlJ5dmYrakV6bWJsbVR2U3dYMVdZZGhDTEdPb3N5ZFVNbkpxdUZDV0E0?= =?utf-8?B?WjVtUlg1TlM0YjBwWUIzeTF6d3J4Y1MxNlJWM2VzTzVTZHI2ZVlOWUhpdmph?= =?utf-8?B?Wkhqdkd6VEF1R1I2cEYzbGMva0Z1VFJwV0NyVUhadVFpbEdyaWl1YU8zMHdJ?= =?utf-8?B?VWF1NExoYXlsSkkxZTRmWkdVbDJOcElLOGhJaFhrWmNKMHJxc0RvZ2NibWti?= =?utf-8?B?SGdVbFhsT0pEanpkTlcyQzVuMlFYbDZPSC94cmxLNWcvejVtb0VsSVRrNkpu?= =?utf-8?B?dlVIWTA1cjROWUJhYllNRDFEN0ZUSmEvUlZZNFo0Ykkzc1JNTHI5M2JaRXg4?= =?utf-8?B?UWZwWEZBNWVzTmxzbDR2VlMzN2xkTmJURkZXSEdaeTFUZFNkVkhrN0tSOEFo?= =?utf-8?B?Z2lKV1JTZ1k0NzI3d0Rwd3NPMUhCZnloMHo5MDY3ZEIvcnIrUWJncmorU2tE?= =?utf-8?B?TFQ1NVU1blFWOU9Qbmd1SVRxVHVlRG5xNlNSUjNBZFFyZ3dPTHNET3A2aFhC?= =?utf-8?B?emlLVjgxdUFmck5kZUZoY2NSV0dlOGp4NVFXOUZHcGVzYnVqMWVwUFM3MUp5?= =?utf-8?B?QjFKaUIwRUdzS0ZIN1BlWGVJSC81dzZKZDYrUFh4aXM1SWVHUmdBQUovR2JO?= =?utf-8?B?L3ZDRjVKelRVY3YzVy9VeUQ0YS9XbVZqSkZXT2EwTTdydFlwR1o1ZG93OGRl?= =?utf-8?B?YUkwRk5sUmZSMWN3NDV5d2IxUXhlLzdMU04xdi95R1ovZjllTFlDejJzOUsy?= =?utf-8?B?LzR0R1lBSjBzRDdwWGhESTF0dmQrVlQ4UjZyMkpmUnVSU0YyWm52UmJab2VT?= =?utf-8?B?RXJZUS9hMlpSQVJMNzI3MjVQazlwd1pYSnNrazR4UWlnVTNOK1BrSUlxVFQ2?= =?utf-8?B?L0E0ejJ4cVBDU2JheTJDdDBrZkVmamYzQmcvOXlWTUxmSFdJLzhDWWM0MEp5?= =?utf-8?B?UEd2bTRsZi9NTzlXako1M2tNZnVLOUxIdnhZT1lZZ1R0M0Jha1hUOTFpSlQ4?= =?utf-8?B?QTBtNCtGSXdKYVZIajRyaDBrTTNrc3V2aWo2VjkvR1lVSXY0U0xkRHlVTFU1?= =?utf-8?B?bWpsN2VTbTVaZlR1NFI1TmZmWGFGcy9mbTdPVXBpcW1NQW9pOExLaW5JeTBZ?= =?utf-8?Q?faT7LX+WiQQ=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZlhFRzh3TmNjVHZZRlJXTmtYcXJYM1FhbEFsU0ZtMW9GclgvN095d0dYK0NN?= =?utf-8?B?RG9JeURkT2FYZ3pmTFpyL3N4Z01WczcrWkFhcXRLQU4xZDJBRkN5WDNOSThG?= =?utf-8?B?SzhaRmxYbElQOGkvazQ4M0FmMXVzdjU5SHRKaUtnZERVQVMvaGVib1hjRlhE?= =?utf-8?B?R3Q5ZVVxVUx5dEhUcDNCZTErcDBUUWRGM1VTdGhLbnpuZGRGZk9sZFZMcElk?= =?utf-8?B?YjhBaVJpRHFyQWJRYVVFVXVHZDg2RnVsU09aYlhYYStmVVN0QTdqMURRM0dN?= =?utf-8?B?ZkVPdVF4eHI2L3pNb3pyUHBjUC9WbTN5TXZpWk5hYytRRzlDTVozU2hzREpY?= =?utf-8?B?S21KKzRFcnRob2d1cC8vdkI2MENBNThyUUxMbU50WlQra2JKRlAzcVdzUENN?= =?utf-8?B?QnJCYWEybGNKSzExQmJKSk1YU1ZPK2k2RXVhM2ZmRG9Wdk5rUCsyU0VTSkpr?= =?utf-8?B?d3JyUktrUUpaQ3E4NjYrRW1KVS93eHdHY3l3dXVtdGxKZUtpdUFIeGh0MmNy?= =?utf-8?B?RjY5WTBTNXpmUzBxSFYxMTV5UERqZ1ZIRGZTOHkyZDNlRUd1a2hFSjhmTFhX?= =?utf-8?B?KzdqeFNyYnNSK2tTYmhRbFoxZVRCVUIvUCtPWStPcmFwaGpzTFZqQlhHZldy?= =?utf-8?B?Q2FSRWhmZTUwemdtZFJzV1dlcjlOZVJxbWs2OFZlai9DdGVOVW16bmUyaW1Z?= =?utf-8?B?WW9PeG9Xc3ZvY2c3THJkODkrTzAyYzMybit0WmtRTXRsK2hRblJ0L1o5VUhI?= =?utf-8?B?Qm5sK0ZmL3lXZ0MzZERlRC9lRUlTamRRdjlnckR2WTNLUU4yQ3ZIQTRlWjEz?= =?utf-8?B?WXJTdmUrUS9nenN4YlBDK1loY2NkZlNVQkh5QzJPSEVsZGQvTW9DQXhyTHNz?= =?utf-8?B?c2hMd1FIUjV5MHlnaENtWUE3cnZRMGtYYWsyRUhYZ1J1SU1oR0NQbHRGMkdi?= =?utf-8?B?OGh3bERHU2pSQXJ1ZXBoeXRKTEg0b0ROY1pMa1N3OXNLWlBsWHhqVnlTNTdz?= =?utf-8?B?NVhROG9UUXJmV0JPQXg0ZHZUQ0ZmWGhwenNYRk1PYXVWcm5teDMxQjQ4dEFB?= =?utf-8?B?V3dtTGltRWMwc2ZhSDRsWklMVUxzTks1Z2RYN2paSndRS3g1a3QwUk1haUVj?= =?utf-8?B?cXNybHN1MmlmbStrN0lGejc2Vzc1NllGaTNCY1VONDVEd1p0cWZsWUxSakJQ?= =?utf-8?B?Yi9QNU9sNWpuS1JzZG1FOHY4Z0M4bkRHZkU3b1BrNVhPeWF6Wkl1Wkk0LzBW?= =?utf-8?B?NHcvS1E4TXdqKzBZQ1RUZjE2NHhYa3haODh1WVZCcFc0N3V3M0Z3ZjVoUjlV?= =?utf-8?B?QVkvbWhYWWFIVkY4dUVmNXpHK2JqUXhDQ2tiYWZRbHZTVCtMK1VWQ0xvSDQ2?= =?utf-8?B?STdoTStHUzdibTVnSTJoTXcreS9VWXRsNDNqdHVYR2ZiK2x6dXRld1ZSUmZJ?= =?utf-8?B?SzNOajJrVlRLK3N3aXlRNFhmeGFlbU5HbXNlNXV5VGlucC9kbkxpbGp1VndE?= =?utf-8?B?ajB4ekRNRE5zWmFWTmUrQ2xwUGV0NHdCL09SV3dzbTNSUzhmY0Q0cDhHRGo2?= =?utf-8?B?YTVrN3JnY1p2cFpEM0ZGR1V2YkJVaHl5M3BYL1pqMkhTd3V4dVBHM0s1aWF1?= =?utf-8?B?eDF1SU1pemhtTWJuZTZZTFdQM3Zkb0VWcmRuRzYwWlIrWlFGM3F1WDllejdV?= =?utf-8?B?L3FTTlFEQ3JnV1hFcGtRLzA3djBUUHhnSUtMQ2NpOFN5a0NZSW4xTTQrS1FS?= =?utf-8?B?Z2YrUXRRZDU5QmI2dmJYcnBacVlVUmVycVY0akkxVjRNUjFjRGJaM3U3T0Jm?= =?utf-8?B?V3RWaW1wNktCcjJxZEtpQkJQV3NHN1ZHbEVsYm40YkQwVHYxaUdFeWhtenV6?= =?utf-8?B?d3VkTExtRlY3ek4yZStQOWV2Y3FNQ2Q3WDltOG1WNDFyUVBKRURqcm0wNHRH?= =?utf-8?B?U1RvWm5nYzhCM2RuamxiR1JWSEpGVHBOZnlGK1pwVXA1RXpYREFSbHNZdGhz?= =?utf-8?B?d3BpdnAxdzVWVWRLbGxzcm0zb1RvRTByL01xWnMzMGl5Q2dYbWkzV2hpVW9F?= =?utf-8?B?bXgvcTIxRExoVk9JbTdNTXJ3eGd1b1lXQVFSTjBKNEIyOWtkQVVBWlR1alVO?= =?utf-8?Q?3BPsUhi4jqLm5m+f8U/QXK2fn?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4dfe2141-6739-4793-ed00-08ddaf35ad58 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:03.6768 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zTmCuS1QgCoCk1kEdmOI/YPbwjAfVgav15afOQDiNw1RGAjdjkK6LcyYq58aM5E12JSCnUznFOdrwQQHQyWolw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-GUID: JPrldQ6nu35vA9ziDshJ-cmAvyeDiBz2 X-Authority-Analysis: v=2.4 cv=Tc2WtQQh c=1 sm=1 tr=0 ts=68541164 cx=c_pps a=UMsKb9OrGFB5W2EpWsGVZg==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=Fsi43M_n0QUer9rQu0sA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMyBTYWx0ZWRfX81KPE82jWCuB o5c7VprAlYVxWunw1R4xmhqx2N25oq/py6NDZ/VS2zbBLwRF+dslDrrX74XtW11iUUiA6xOaW86 Za0cDQrXxPf2rZbu09s8WnQtxZOZYacVX1a4XapbQeu5vOD/cWL+kLGzOr5H6PXkoEeXABqLE7W 0uzmMRj2RhV7pIjLmq2RExU+D4A5l9Cwg0rH1CkIQY+ucvQ1n30fC/Wi6/1EAzNU7XcbksPKMug UAXbfqUWLYfKtLZphwg/rfb2N7bBzurfaU3xNi9yhpB6YktYvvY9bslW3pfi4Hxr1nVMzszg1yu LT+LNXeO/40EFxJ6/VVUQkWnjGBc4yrMwzrup0j9NGvJ5eH5M4mglN3jTcdnYMJOVSqA9JuZEqV W2NWslb8y04QLVKXD9XOjnEC7OkdRaRaREw5tQVG8jjxfCEJG+WQnVVhtXSJmlWW8gwTm7yN X-Proofpoint-ORIG-GUID: JPrldQ6nu35vA9ziDshJ-cmAvyeDiBz2 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750339975508116600 Introduce the vfio-user "proxy": this is the client code responsible for sending and receiving vfio-user messages across the control socket. The new files hw/vfio-user/proxy.[ch] contain some basic plumbing for managing the proxy; initialize the proxy during realization of the VFIOUserPCIDevice instance. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/proxy.h | 79 +++++++++++++++++ include/hw/vfio/vfio-device.h | 2 + hw/vfio-user/pci.c | 22 +++++ hw/vfio-user/proxy.c | 162 ++++++++++++++++++++++++++++++++++ hw/vfio-user/meson.build | 1 + 5 files changed, 266 insertions(+) create mode 100644 hw/vfio-user/proxy.h create mode 100644 hw/vfio-user/proxy.c diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h new file mode 100644 index 0000000000..a9bce82239 --- /dev/null +++ b/hw/vfio-user/proxy.h @@ -0,0 +1,79 @@ +#ifndef VFIO_USER_PROXY_H +#define VFIO_USER_PROXY_H + +/* + * vfio protocol over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "io/channel.h" +#include "io/channel-socket.h" + +typedef struct { + int send_fds; + int recv_fds; + int *fds; +} VFIOUserFDs; + +enum msg_type { + VFIO_MSG_NONE, + VFIO_MSG_ASYNC, + VFIO_MSG_WAIT, + VFIO_MSG_NOWAIT, + VFIO_MSG_REQ, +}; + +typedef struct VFIOUserMsg { + QTAILQ_ENTRY(VFIOUserMsg) next; + VFIOUserFDs *fds; + uint32_t rsize; + uint32_t id; + QemuCond cv; + bool complete; + enum msg_type type; +} VFIOUserMsg; + + +enum proxy_state { + VFIO_PROXY_CONNECTED =3D 1, + VFIO_PROXY_ERROR =3D 2, + VFIO_PROXY_CLOSING =3D 3, + VFIO_PROXY_CLOSED =3D 4, +}; + +typedef QTAILQ_HEAD(VFIOUserMsgQ, VFIOUserMsg) VFIOUserMsgQ; + +typedef struct VFIOUserProxy { + QLIST_ENTRY(VFIOUserProxy) next; + char *sockname; + struct QIOChannel *ioc; + void (*request)(void *opaque, VFIOUserMsg *msg); + void *req_arg; + int flags; + QemuCond close_cv; + AioContext *ctx; + QEMUBH *req_bh; + + /* + * above only changed when BQL is held + * below are protected by per-proxy lock + */ + QemuMutex lock; + VFIOUserMsgQ free; + VFIOUserMsgQ pending; + VFIOUserMsgQ incoming; + VFIOUserMsgQ outgoing; + VFIOUserMsg *last_nowait; + enum proxy_state state; +} VFIOUserProxy; + +/* VFIOProxy flags */ +#define VFIO_PROXY_CLIENT 0x1 + +VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp); +void vfio_user_disconnect(VFIOUserProxy *proxy); + +#endif /* VFIO_USER_PROXY_H */ diff --git a/include/hw/vfio/vfio-device.h b/include/hw/vfio/vfio-device.h index 959e458d7f..c616652ee7 100644 --- a/include/hw/vfio/vfio-device.h +++ b/include/hw/vfio/vfio-device.h @@ -47,6 +47,7 @@ typedef struct VFIOMigration VFIOMigration; =20 typedef struct IOMMUFDBackend IOMMUFDBackend; typedef struct VFIOIOASHwpt VFIOIOASHwpt; +typedef struct VFIOUserProxy VFIOUserProxy; =20 typedef struct VFIODevice { QLIST_ENTRY(VFIODevice) next; @@ -88,6 +89,7 @@ typedef struct VFIODevice { struct vfio_region_info **reginfo; int *region_fds; VFIODeviceCPR cpr; + VFIOUserProxy *proxy; } VFIODevice; =20 struct VFIODeviceOps { diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 86d7055747..642421e791 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -12,6 +12,7 @@ =20 #include "hw/qdev-properties.h" #include "hw/vfio/pci.h" +#include "hw/vfio-user/proxy.h" =20 #define TYPE_VFIO_USER_PCI "vfio-user-pci" OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserPCIDevice, VFIO_USER_PCI) @@ -54,6 +55,8 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error = **errp) VFIODevice *vbasedev =3D &vdev->vbasedev; const char *sock_name; AddressSpace *as; + SocketAddress addr; + VFIOUserProxy *proxy; =20 if (!udev->socket) { error_setg(errp, "No socket specified"); @@ -69,6 +72,15 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error= **errp) =20 vbasedev->name =3D g_strdup_printf("vfio-user:%s", sock_name); =20 + memset(&addr, 0, sizeof(addr)); + addr.type =3D SOCKET_ADDRESS_TYPE_UNIX; + addr.u.q_unix.path =3D (char *)sock_name; + proxy =3D vfio_user_connect_dev(&addr, errp); + if (!proxy) { + return; + } + vbasedev->proxy =3D proxy; + /* * vfio-user devices are effectively mdevs (don't use a host iommu). */ @@ -112,8 +124,13 @@ static void vfio_user_instance_init(Object *obj) static void vfio_user_instance_finalize(Object *obj) { VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); + VFIODevice *vbasedev =3D &vdev->vbasedev; =20 vfio_pci_put_device(vdev); + + if (vbasedev->proxy !=3D NULL) { + vfio_user_disconnect(vbasedev->proxy); + } } =20 static const Property vfio_user_pci_dev_properties[] =3D { @@ -133,6 +150,11 @@ static void vfio_user_pci_set_socket(Object *obj, Visi= tor *v, const char *name, VFIOUserPCIDevice *udev =3D VFIO_USER_PCI(obj); bool success; =20 + if (udev->device.vbasedev.proxy) { + error_setg(errp, "Proxy is connected"); + return; + } + qapi_free_SocketAddress(udev->socket); =20 udev->socket =3D NULL; diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c new file mode 100644 index 0000000000..bb436c9db9 --- /dev/null +++ b/hw/vfio-user/proxy.c @@ -0,0 +1,162 @@ +/* + * vfio protocol over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include + +#include "hw/vfio/vfio-device.h" +#include "hw/vfio-user/proxy.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "qemu/lockable.h" +#include "system/iothread.h" + +static IOThread *vfio_user_iothread; + +static void vfio_user_shutdown(VFIOUserProxy *proxy); + + +/* + * Functions called by main, CPU, or iothread threads + */ + +static void vfio_user_shutdown(VFIOUserProxy *proxy) +{ + qio_channel_shutdown(proxy->ioc, QIO_CHANNEL_SHUTDOWN_READ, NULL); + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, NULL, + proxy->ctx, NULL, NULL); +} + +/* + * Functions only called by iothread + */ + +static void vfio_user_cb(void *opaque) +{ + VFIOUserProxy *proxy =3D opaque; + + QEMU_LOCK_GUARD(&proxy->lock); + + proxy->state =3D VFIO_PROXY_CLOSED; + qemu_cond_signal(&proxy->close_cv); +} + + +/* + * Functions called by main or CPU threads + */ + +static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D + QLIST_HEAD_INITIALIZER(vfio_user_sockets); + +VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp) +{ + VFIOUserProxy *proxy; + QIOChannelSocket *sioc; + QIOChannel *ioc; + char *sockname; + + if (addr->type !=3D SOCKET_ADDRESS_TYPE_UNIX) { + error_setg(errp, "vfio_user_connect - bad address family"); + return NULL; + } + sockname =3D addr->u.q_unix.path; + + sioc =3D qio_channel_socket_new(); + ioc =3D QIO_CHANNEL(sioc); + if (qio_channel_socket_connect_sync(sioc, addr, errp)) { + object_unref(OBJECT(ioc)); + return NULL; + } + qio_channel_set_blocking(ioc, false, NULL); + + proxy =3D g_malloc0(sizeof(VFIOUserProxy)); + proxy->sockname =3D g_strdup_printf("unix:%s", sockname); + proxy->ioc =3D ioc; + proxy->flags =3D VFIO_PROXY_CLIENT; + proxy->state =3D VFIO_PROXY_CONNECTED; + + qemu_mutex_init(&proxy->lock); + qemu_cond_init(&proxy->close_cv); + + if (vfio_user_iothread =3D=3D NULL) { + vfio_user_iothread =3D iothread_create("VFIO user", errp); + } + + proxy->ctx =3D iothread_get_aio_context(vfio_user_iothread); + + QTAILQ_INIT(&proxy->outgoing); + QTAILQ_INIT(&proxy->incoming); + QTAILQ_INIT(&proxy->free); + QTAILQ_INIT(&proxy->pending); + QLIST_INSERT_HEAD(&vfio_user_sockets, proxy, next); + + return proxy; +} + +void vfio_user_disconnect(VFIOUserProxy *proxy) +{ + VFIOUserMsg *r1, *r2; + + qemu_mutex_lock(&proxy->lock); + + /* our side is quitting */ + if (proxy->state =3D=3D VFIO_PROXY_CONNECTED) { + vfio_user_shutdown(proxy); + if (!QTAILQ_EMPTY(&proxy->pending)) { + error_printf("vfio_user_disconnect: outstanding requests\n"); + } + } + object_unref(OBJECT(proxy->ioc)); + proxy->ioc =3D NULL; + + proxy->state =3D VFIO_PROXY_CLOSING; + QTAILQ_FOREACH_SAFE(r1, &proxy->outgoing, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->outgoing, r1, next); + g_free(r1); + } + QTAILQ_FOREACH_SAFE(r1, &proxy->incoming, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->incoming, r1, next); + g_free(r1); + } + QTAILQ_FOREACH_SAFE(r1, &proxy->pending, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->pending, r1, next); + g_free(r1); + } + QTAILQ_FOREACH_SAFE(r1, &proxy->free, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->free, r1, next); + g_free(r1); + } + + /* + * Make sure the iothread isn't blocking anywhere + * with a ref to this proxy by waiting for a BH + * handler to run after the proxy fd handlers were + * deleted above. + */ + aio_bh_schedule_oneshot(proxy->ctx, vfio_user_cb, proxy); + qemu_cond_wait(&proxy->close_cv, &proxy->lock); + + /* we now hold the only ref to proxy */ + qemu_mutex_unlock(&proxy->lock); + qemu_cond_destroy(&proxy->close_cv); + qemu_mutex_destroy(&proxy->lock); + + QLIST_REMOVE(proxy, next); + if (QLIST_EMPTY(&vfio_user_sockets)) { + iothread_destroy(vfio_user_iothread); + vfio_user_iothread =3D NULL; + } + + g_free(proxy->sockname); + g_free(proxy); +} diff --git a/hw/vfio-user/meson.build b/hw/vfio-user/meson.build index 8749526cd0..cb958d0aa3 100644 --- a/hw/vfio-user/meson.build +++ b/hw/vfio-user/meson.build @@ -2,6 +2,7 @@ vfio_user_ss =3D ss.source_set() vfio_user_ss.add(files( 'container.c', 'pci.c', + 'proxy.c', )) =20 system_ss.add_all(when: 'CONFIG_VFIO_USER', if_true: vfio_user_ss) --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340155; cv=pass; d=zohomail.com; s=zohoarc; b=WxRP65GDAnpKFjm1rGYZ3My8wHGaO7jkPAKhVXeAuSo5WoCWLQE9hOQfnqlidGGunzoMZJ/TBC3WH39wSaxf9OelUFzDONzRILmD5zoN8nP4gsnEzvTcS+8MI5ARc+lIHJb7lV+U6WN6Q31CEbyHtMsee9r+2kzJ0mfBCr6ab64= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340155; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9Q5nBE24YdOu0GO4vEHibcU6YfrjdcI9DciSr5HUb+4=; b=PdxWCzyg/6JSwFASJtVDAzLg+Dgn2WexHKtNKCywS5eoY3q7lYLAM1hynuJNc0VildSveVBs8x4uXpCDYx1iFLs1QpjVrxpftv30++739WwDDATkGF5L5kwfUE8GmbsKmuEVtyALXtF20D+Cs2XuMk9KUD8DeSYinte0MG5Uwa0= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340155628163.74549561959952; Thu, 19 Jun 2025 06:35:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFNa-0004DE-3D; Thu, 19 Jun 2025 09:32:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNU-0004Be-5R for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:29 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNP-0008Ce-AU for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:27 -0400 Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55JBwK8n024158; Thu, 19 Jun 2025 06:32:21 -0700 Received: from dm1pr04cu001.outbound.protection.outlook.com (mail-centralusazon11020080.outbound.protection.outlook.com [52.101.61.80]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47c0vfjedr-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:21 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:06 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=9Q5nBE24YdOu0GO4vEHibcU6YfrjdcI9DciSr5HUb +4=; b=z8lv4uXkQTPHLmmW7+ijy7ch5/wJHMF2asz4VzLbdH9UgH1kU6rViGjfY FnaUK9+4J8PvWBXl2LMRDNZuoM48NzWNonOX4/af/w/oJhdLPkPiQHvAE5wGyJhp g2J1YVSsZCCZPnko0+yJoPK5h2tnLpgTK0IRTYr8bl5yhrqd1HuCOGz51/EnZzBV Dn40DFjc8u5yMubbt2odCYQ01hJKN9NBv1lyTCpg7oFwJp8gP/ZG8Yx/XITILVkM /qNfM8J5OlizFuCnSJGlz9EvtMclSDw7YHy7EHDAl/SbJesy/YvjCEez8222dNo1 ED5Bx1s2q1SisBXT5F/L7i18vnv9g== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=QcS1vPVV+Hmh1pU50zWhlCwOHLxv7QUf+hYHIdDmjCkEA6TN9xQf4K7RroalYE+T4ZHMKYCndHjHO5xSk/rgmjwX4XCFcobLRogGxxFhDXoN193OweOhULsNLbTWp+Ml2aSUp+7UohxtnLkSNtIfIQkYrKMHqYQxspXQKsBIjR0T0TN7RxLQ5eugpAmF07I8JzdbkKkk8uUnSnOEmNsKMvBkvPCsr3rakn8evI4S1OqIG7x4t0LH8VDR5UPFMacPcl7A9FbrbYaZG+j1lB1Rj+nY0JXBPgTVgnRY1E0n3U/1e8yXN8gYIi6fZHgzNTo10A0vDCy9MPNUbIRMSh4TPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9Q5nBE24YdOu0GO4vEHibcU6YfrjdcI9DciSr5HUb+4=; b=g0QY65JHWmYuexdGfDVqcN47sFzMSxFtE9YT/YVJ5LWAyxSwaFsedx9vyy58CSu525CSH4hwVGqMzU8VjayVdKX+jwmAIOfJdHB2Oj7vEm53YEXYgVRhTvKlhq9Q27DD+lGuZpdwrcRjxJP0jDSjNLyzacSR5dbUU5Wu7bZn/E6Xub5hQytvQyr1ItFV/2PF0bKPaCU5MSnEbds+awg4K2vyIZwC+nZh0lx1P6O0B6VBeAlu+embw5PyLo6TiPDzYVY0t6jdKLtIAJactsPgqNYEliliCD5twBThSELzs4g1PCiJsbBJp5ZyefOZM7nUpKhDr78Sf2I1d3MzIEW5XQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9Q5nBE24YdOu0GO4vEHibcU6YfrjdcI9DciSr5HUb+4=; b=eMXhTpLoKDItzb2CaqbUFv02b1xAWyOEfIp/uct1RX19PBoKKtOP4EtKtsj2eiVg/NccfnGSCKvU7r1vYs2YrAxsXnLy4erS4RVW7X8IC3rdipfbako58FBumzAv3Q7TlBEmDG4TO41AljrpCK9QDXQutk+InEXW4hnHwudQeJPrRXnwOhf8k/898glkaI9znQ5VNzYN5dA1sr9UcPwQW4kHznnKlLAkymJQBrxgZ2b/v8xQqtQXB1lymALEupXu4M5Y2pygHVj+HAkmnOP/GZWLauWkg1tfuHgf62JgnlZvXLtxDAsWiWVyLaYYOXId+1kxEhv6PuGbqPc58KqCmg== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 03/19] vfio-user: implement message receive infrastructure Date: Thu, 19 Jun 2025 06:31:37 -0700 Message-ID: <20250619133154.264786-4-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 745a96cd-5ee7-4db0-7921-08ddaf35af30 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?YWpSenJVWGRNWWx0bUNOMVF0VVkwWjhOOUJ3OHJ3ZzZsMUxmbjREVU5ycHVR?= =?utf-8?B?Ri9KZzhZOFh5MjdIN1RhSG9NMjIvcysrbE1ySDBQQUZKb29RMjEyeEVyNUk0?= =?utf-8?B?OTc3L1lxeGVCcS80enpBTUViaURjbzA0OTZVcGFHSUFkMThqTUJxOVhadFZF?= =?utf-8?B?VG5Za2RFSVhwNitKYmtoN292YVoxK0twYnF3N2Rwak9telJIbkZ5ZW5yVGdu?= =?utf-8?B?ZTBoeCt3RXVla1BZOHNnR3F1TGdBMmN4NHAyQ0RreEJEVW8rVTlEaGR1cEYv?= =?utf-8?B?Q3laNXdoU0dvaFBpMEw3Z1NKU1p6UktHSVBoVUZoK2xTOEZObkhnR3lHN2tl?= =?utf-8?B?Skw4QXJjNGc0Z1hWbnZpVTBBZ0pmUDliYmJpUkNUdXZabUt4WG9aM3VYVTBj?= =?utf-8?B?SVI4UXh0cVBWWTJJYmh1QW1wakhSd1BjcTBDQXZBdVRPRGk4T1VOT3oybFNY?= =?utf-8?B?TWlVVEZMVzVHSWFzZHlZenJ2OE5tdEZxS3RJeENuQ0U0QTMvL21ncDg5RUFq?= =?utf-8?B?N1YwK1J6akh1cEp5V1FscExoUnQwZjBrYUtyTStmT3hSYUN2b3VEaXBUS3pT?= =?utf-8?B?cG9HSGRVWDlUcitMejIva05WaVljMCtYWTIxbDFmdmhFQXl6Y3VBaktSWW4x?= =?utf-8?B?VXVVWjRFNGtrdS9FRHUxUFp2N0syd3dmVUQ3TVlENG1Fa0wxVkFjN1lSWWh1?= =?utf-8?B?VG1tYWk5R3BJbGxGVFU4d3pqcG96Q2ZvY3h3YnRwcmhvMkg5ckkzNGpwMzh0?= =?utf-8?B?eVo0WmhBdWFOdXBpZ0FYL2w5Qm1KeWEvMzJvalByQzlJcXFsUWFQQTc5cEtx?= =?utf-8?B?bnRaeXNvRmlPejMwVk50STdSNEloanRVcWZ0QmFENm84SlRkUjFqYjdGaVZ6?= =?utf-8?B?RVZBMlFpT0FROUtiZ2pFUzQzbVFWUE9WL2IrQUJ6THN3VnJ1UkxXQndZWGdV?= =?utf-8?B?REdHR0VMMmhIRERWbXkvelBMR1JselMvazlKN3NQelErQzhmaXg5WWlwU2w0?= =?utf-8?B?SGtPeW5GOC9ZREtxSklZenhuTFZZbUdCVUM4Wm5tUlhya2hCTmM1a2RLNkFK?= =?utf-8?B?SitoaE5tNUs5VGI2ay9sd3lEWE16S0RleFMrbk9Scy9GQzBiL2ZCWWh2eVRq?= =?utf-8?B?cEJTdk5rdjBJOFJqeHpta3NDbFl5akRxRXY0dklMWlcrLzVFcjFpcHhVWEJQ?= =?utf-8?B?QXpITlZqcTIrZ1BJWUpCMVFKc21XbHBuK2tGVzdRaFVSTm1XMFpPdXloZXJS?= =?utf-8?B?ZWtFL3lJb0tFLzY1eWhvaHpQNnYyOUZiQWxlb1JSRHpSNHFqaVpYdHdqZkhI?= =?utf-8?B?OGwrdHJwdXovRUc5SnpLWEN5R2t6azRFZ0dDVWFjcERTencyYlZlZVJWYzVm?= =?utf-8?B?QWhVWjJmUkE1Wmx3UkpvaTl3N3Q5ZEJzVUVoc0dFMldwVVlwV2FMY3JiRUJH?= =?utf-8?B?Z0diY2U3R3dTRW9kNWdPQ2JqUnQwQk1PWGxWdlpreGd5cXNpckJOZEkyNjJN?= =?utf-8?B?Y2FTVDZQQ0wxcVE0R0d1TGF1WGRibHlDRFV6ekhqdExQdWJ2ZTYzVnZoUWpN?= =?utf-8?B?M3lXZ3AwQlc0R0ZxRmxyaHBzdWtwRGNMYXhvMDRLb055YmFXeWh5czRlM3gv?= =?utf-8?B?aHlPdDBDZHh4WEhJa3FXbGp4RzZ0R1J0aldVbTR6VzhOYVZsSU1SV3p5ZFlH?= =?utf-8?B?b01qRlRYSmFkd2x0TUpTR3UwR3pKY25Rc1hZMDhjcDgzQ2xEcjdsak5wcUEz?= =?utf-8?B?S0tneWpLYzlCZ2pyeWhKTUlXTko2TnRWd0Y0TklhS3QwS0lla20vN3NIcUhu?= =?utf-8?B?TjdnRGo4SHYwOTFHc2VSMkVsZU1HZVVlZWlFNzFzL294UnBCRmd6cUc1U0t6?= =?utf-8?B?RjVxOFRkN1hSMXI3Y2M2T3htZ01OVEJiQWplR1lGWWZaUWc9PQ==?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?REx4YnNUbk9URk1obWlxZEE4L2FrVVRqWVQrZjJFeFNwcnA1TUw3RTZxdFF2?= =?utf-8?B?SGUzN1NlTmx2YzhIb2llVG1nV2VZc0lWZndKVjBlanBmNDVxSnRJRzhHbWx0?= =?utf-8?B?dURtWDEraUREZXYvY1VxUnpwaHZCZ3hHS203dm9QdUp5TXhMd2IyTEZ4cXpt?= =?utf-8?B?K21ISWRKY29DbUZKeHhnOWpXS2F4VHhNbkppMmx3aVpwU1hhNXE1ZzNEZEdo?= =?utf-8?B?alA1QWZsNEhIY0E4ZnVHa29CM1RldVJ1ZkFMdDhhU1dCdEtabEs5OUxPckRM?= =?utf-8?B?VmxKblc2QzFYTktsQ2U2ZnBDMHRURkhaUDIzSDlqcFVSYmZwMkNSOTZ2SElq?= =?utf-8?B?TWh1UFU1VWcyMWkwV0VGU2FCckVSZm9TQ3BPWFZIbDVWdjJKMFVjTFJyOWh0?= =?utf-8?B?MlR5UTFGeFhqYThrNExsRU43Z1kxZkMwL2pneU9zNVU1c0hYMldVTVVvdVk3?= =?utf-8?B?a2xjT3Y1NnhQWHJXb0RPUUFXTFJicVpJNmY3OGJFS0EvbWpaQzYvQnZGZ0JP?= =?utf-8?B?REN1dGVvbUhOVXFLSENxeHAwcDFtbFJSK1NKZU8xSGNmOVNsdVVxQVFqQXVO?= =?utf-8?B?c01hb0o4SXc1VncwNC9idXE3dng1ZXZVeStFSmxtMUsvdytTY3pCSHhXSjY5?= =?utf-8?B?QkEyQXdkSTdUbHdXTmlPRy9nbVJacktUanNmWXpJcU5PN3pTNE5aeGVXZjFU?= =?utf-8?B?cm1Zc3N3ZmVKNEdJTGg3d0VYd1FweDJoNHlGelNzM011L1VQdlJSbUNoRk5C?= =?utf-8?B?YzdKUDMrczN4VlE3TnFwU0FIdjI5aDlYd3pEK1hDdkhzd3EwT1ppWCsrRmdn?= =?utf-8?B?SmpycGRSS3BualJEbEpTYnhNVjhQNS9mYWVRNUNHMThkNHl3Nk5BK3ZGSk51?= =?utf-8?B?K01wSVphSFpuK2NlU0hJVHR2MXNEUXhKYnpqalFGUDVPd1ZJcmxFTVM3V3V1?= =?utf-8?B?UHlCa1RkbitncnU3eFRTMjdMKzRGaEtYNmpZOUN2Wm1uZVh4UklON3VvRlo5?= =?utf-8?B?N0xURjVWZ1JFdnZmZmNOelp1T1dBNlkrMFRjOUdDb3lIbjZuRkVsYkxEaFBE?= =?utf-8?B?eG5lU0lRMFdNcU8zbndWOERrOTh1VTNFS1JqaUZaZnZiWHhWY3YrNXEzOFNJ?= =?utf-8?B?aEthOVpOMkQrWmxpNnVrY055bzRpM3lnbVhTSEFKUER2bGVLYjZzSThjT0V6?= =?utf-8?B?Ung5M3YyejBBNnVYNkhkT2crL0hHN0JDQmxoemVmZ1g1WVg5Q09qRkJYT2Vu?= =?utf-8?B?UjBPVloxeUhRc24vTEZia29RSUNlQ3l1REEyNHFuUTNveGFsR3F3RXFZTlBP?= =?utf-8?B?TjdxVG04OE1aUllHNUt3aGxSTkpBWHNVczM1NVlnTTV5R3Y5ZWdkL3VvZS9T?= =?utf-8?B?ZG0yOUlMSU9yeVVrZXh5cDBMYzVQaEpSeHd1UWtGcmEvQ2pKZ29BeXhVRUxY?= =?utf-8?B?RlZXRGJ4WjBqNDRzNEowMzBTanRoRmdOdERGVzZSdUVocUJmenc5L3VmWk5Q?= =?utf-8?B?YVYyLzJYZFZZOFlabll6eGRDT2d2Q3B5VXMrSm1uMzJrU29IWkVHQWovemNh?= =?utf-8?B?ZkhId1kzMExFNG5YVFhKalk5dk83MXFTUWRSS0QwUzFhUXlwTUFsdDJXL09I?= =?utf-8?B?VGpaTmhsT2diUFRMUk5oZWdpQnppNTU4d1ozZURrNWwweEVFa0p3UFE1dURV?= =?utf-8?B?OExoRWdYSko3YXJET1NqRElPY04zRXRXS01pWFFWeWNTNUdSb1N5RDdxVTg5?= =?utf-8?B?d3Z1b3BUWVVZYVc5MkVoL3dHTVM3bkJFSUxVeEMwbkNOMmJTbk5LVXZ6MFRZ?= =?utf-8?B?ejVZUFBxUS80NWgwT2hseFJLd0d5Rzk0eFdiWU1MVy9LNlAybGxsRjY2YWJC?= =?utf-8?B?UUt6KzJVa2lWQVk1ZmpONTJNd3NEdEMyRGtvYW5yS2wzWjNMMmhNNjExRW55?= =?utf-8?B?RkNHbHRSMXVCeFQzTTFJQ0xYMG04eEdJTFMwcDJScTdwWGdKV282emozajht?= =?utf-8?B?bDlkR3pTTXVrQTNpRlpBbHJILys0ZHJjVnFMWUFpYmFVRzI2TzAzZStxaEIr?= =?utf-8?B?L3dlenZwMEpyODMwejNxTU9MQmtLNlRQVmNzL1dLNE16RVg4VHJ1SjRHZ3Yy?= =?utf-8?Q?umf/lzNV3cM5n7WbWi85J5Q6w?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 745a96cd-5ee7-4db0-7921-08ddaf35af30 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:06.7371 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: cR4i4KP4NpCR2EFNujl1O5lheIvMmMbzd6g62/e+h+YEF5NBuqsg7w62WswwzaWjZdoppmdnk5AlevxIkwm70w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-GUID: y-M5g7Z8NQ0ocgwF93S5v6x9ta9mZpOx X-Authority-Analysis: v=2.4 cv=Tc2WtQQh c=1 sm=1 tr=0 ts=68541165 cx=c_pps a=UMsKb9OrGFB5W2EpWsGVZg==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=vXCtUE0a-urccVGwgxAA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMyBTYWx0ZWRfXwqlbUVDVxnUt +2LbvIZ2yXXJvcJvqDjD7gmsC4/UGwZ3+5L+2MpS36E7xLs8tDUSoY5D08v4GMdDp5z9273U1IH NoudcLh7m7y80vnJdy3pE08It12qi5r0VXB032afCuT6osSe9WKu1Hdb0iuJ1QkBwjtlNmkLDe1 v1dWSu766/NZdCSz3yFr0S+uW1lnWdekBkCV+mAs5FPk0LpX9kJps7veYyoQ7LnsssKEAop9ol8 zdWjJjpWGcOOi49XNBN3u7qGagHM9k2CqGtnbtmjc7ypERDb+Gu9ZckzfvQFpe3WEA0AdZMsdiv KQm8nqtFgMAiCgqGKxIvvxeHWfnENRFvXRdNxVBLMIgQv5KcFzO45Y7+M0Mt38CqOl/fWg5cj2W OWbolGhdC4wC0OdOJEzjVjooHjPUwxQZw+7IL6UIBzrPkxuGJWPH/9zGcXdfg6BhDxBNMAcq X-Proofpoint-ORIG-GUID: y-M5g7Z8NQ0ocgwF93S5v6x9ta9mZpOx X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340157788116600 Add the basic implementation for receiving vfio-user messages from the control socket. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- meson.build | 1 + hw/vfio-user/protocol.h | 53 +++++ hw/vfio-user/proxy.h | 11 + hw/vfio-user/trace.h | 1 + hw/vfio-user/pci.c | 11 + hw/vfio-user/proxy.c | 409 ++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 6 + 7 files changed, 492 insertions(+) create mode 100644 hw/vfio-user/protocol.h create mode 100644 hw/vfio-user/trace.h create mode 100644 hw/vfio-user/trace-events diff --git a/meson.build b/meson.build index 34729c2a3d..3d2d8c97dc 100644 --- a/meson.build +++ b/meson.build @@ -3686,6 +3686,7 @@ if have_system 'hw/ufs', 'hw/usb', 'hw/vfio', + 'hw/vfio-user', 'hw/virtio', 'hw/vmapple', 'hw/watchdog', diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h new file mode 100644 index 0000000000..4ddfb5f222 --- /dev/null +++ b/hw/vfio-user/protocol.h @@ -0,0 +1,53 @@ +#ifndef VFIO_USER_PROTOCOL_H +#define VFIO_USER_PROTOCOL_H + +/* + * vfio protocol over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * Each message has a standard header that describes the command + * being sent, which is almost always a VFIO ioctl(). + * + * The header may be followed by command-specific data, such as the + * region and offset info for read and write commands. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +typedef struct { + uint16_t id; + uint16_t command; + uint32_t size; + uint32_t flags; + uint32_t error_reply; +} VFIOUserHdr; + +/* VFIOUserHdr commands */ +enum vfio_user_command { + VFIO_USER_VERSION =3D 1, + VFIO_USER_DMA_MAP =3D 2, + VFIO_USER_DMA_UNMAP =3D 3, + VFIO_USER_DEVICE_GET_INFO =3D 4, + VFIO_USER_DEVICE_GET_REGION_INFO =3D 5, + VFIO_USER_DEVICE_GET_REGION_IO_FDS =3D 6, + VFIO_USER_DEVICE_GET_IRQ_INFO =3D 7, + VFIO_USER_DEVICE_SET_IRQS =3D 8, + VFIO_USER_REGION_READ =3D 9, + VFIO_USER_REGION_WRITE =3D 10, + VFIO_USER_DMA_READ =3D 11, + VFIO_USER_DMA_WRITE =3D 12, + VFIO_USER_DEVICE_RESET =3D 13, + VFIO_USER_DIRTY_PAGES =3D 14, + VFIO_USER_MAX, +}; + +/* VFIOUserHdr flags */ +#define VFIO_USER_REQUEST 0x0 +#define VFIO_USER_REPLY 0x1 +#define VFIO_USER_TYPE 0xF + +#define VFIO_USER_NO_REPLY 0x10 +#define VFIO_USER_ERROR 0x20 + +#endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index a9bce82239..ff553cad9d 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -12,6 +12,9 @@ #include "io/channel.h" #include "io/channel-socket.h" =20 +#include "qemu/sockets.h" +#include "hw/vfio-user/protocol.h" + typedef struct { int send_fds; int recv_fds; @@ -28,6 +31,7 @@ enum msg_type { =20 typedef struct VFIOUserMsg { QTAILQ_ENTRY(VFIOUserMsg) next; + VFIOUserHdr *hdr; VFIOUserFDs *fds; uint32_t rsize; uint32_t id; @@ -67,13 +71,20 @@ typedef struct VFIOUserProxy { VFIOUserMsgQ incoming; VFIOUserMsgQ outgoing; VFIOUserMsg *last_nowait; + VFIOUserMsg *part_recv; + size_t recv_left; enum proxy_state state; } VFIOUserProxy; =20 /* VFIOProxy flags */ #define VFIO_PROXY_CLIENT 0x1 =20 +typedef struct VFIODevice VFIODevice; + VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp); void vfio_user_disconnect(VFIOUserProxy *proxy); +void vfio_user_set_handler(VFIODevice *vbasedev, + void (*handler)(void *opaque, VFIOUserMsg *msg), + void *reqarg); =20 #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/trace.h b/hw/vfio-user/trace.h new file mode 100644 index 0000000000..574b59aa89 --- /dev/null +++ b/hw/vfio-user/trace.h @@ -0,0 +1 @@ +#include "trace/trace-hw_vfio_user.h" diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 642421e791..bad2829f5c 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -22,6 +22,16 @@ struct VFIOUserPCIDevice { SocketAddress *socket; }; =20 +/* + * Incoming request message callback. + * + * Runs off main loop, so BQL held. + */ +static void vfio_user_pci_process_req(void *opaque, VFIOUserMsg *msg) +{ + +} + /* * Emulated devices don't use host hot reset */ @@ -80,6 +90,7 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error = **errp) return; } vbasedev->proxy =3D proxy; + vfio_user_set_handler(vbasedev, vfio_user_pci_process_req, vdev); =20 /* * vfio-user devices are effectively mdevs (don't use a host iommu). diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index bb436c9db9..31e08cbad3 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -11,15 +11,32 @@ =20 #include "hw/vfio/vfio-device.h" #include "hw/vfio-user/proxy.h" +#include "hw/vfio-user/trace.h" #include "qapi/error.h" #include "qemu/error-report.h" #include "qemu/lockable.h" +#include "qemu/main-loop.h" #include "system/iothread.h" =20 static IOThread *vfio_user_iothread; =20 static void vfio_user_shutdown(VFIOUserProxy *proxy); +static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, + VFIOUserFDs *fds); +static VFIOUserFDs *vfio_user_getfds(int numfds); +static void vfio_user_recycle(VFIOUserProxy *proxy, VFIOUserMsg *msg); =20 +static void vfio_user_recv(void *opaque); +static int vfio_user_recv_one(VFIOUserProxy *proxy); +static void vfio_user_cb(void *opaque); + +static void vfio_user_request(void *opaque); + +static inline void vfio_user_set_error(VFIOUserHdr *hdr, uint32_t err) +{ + hdr->flags |=3D VFIO_USER_ERROR; + hdr->error_reply =3D err; +} =20 /* * Functions called by main, CPU, or iothread threads @@ -32,10 +49,340 @@ static void vfio_user_shutdown(VFIOUserProxy *proxy) proxy->ctx, NULL, NULL); } =20 +static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, + VFIOUserFDs *fds) +{ + VFIOUserMsg *msg; + + msg =3D QTAILQ_FIRST(&proxy->free); + if (msg !=3D NULL) { + QTAILQ_REMOVE(&proxy->free, msg, next); + } else { + msg =3D g_malloc0(sizeof(*msg)); + qemu_cond_init(&msg->cv); + } + + msg->hdr =3D hdr; + msg->fds =3D fds; + return msg; +} + +/* + * Recycle a message list entry to the free list. + */ +static void vfio_user_recycle(VFIOUserProxy *proxy, VFIOUserMsg *msg) +{ + if (msg->type =3D=3D VFIO_MSG_NONE) { + error_printf("vfio_user_recycle - freeing free msg\n"); + return; + } + + /* free msg buffer if no one is waiting to consume the reply */ + if (msg->type =3D=3D VFIO_MSG_NOWAIT || msg->type =3D=3D VFIO_MSG_ASYN= C) { + g_free(msg->hdr); + if (msg->fds !=3D NULL) { + g_free(msg->fds); + } + } + + msg->type =3D VFIO_MSG_NONE; + msg->hdr =3D NULL; + msg->fds =3D NULL; + msg->complete =3D false; + QTAILQ_INSERT_HEAD(&proxy->free, msg, next); +} + +static VFIOUserFDs *vfio_user_getfds(int numfds) +{ + VFIOUserFDs *fds =3D g_malloc0(sizeof(*fds) + (numfds * sizeof(int))); + + fds->fds =3D (int *)((char *)fds + sizeof(*fds)); + + return fds; +} + /* * Functions only called by iothread */ =20 +/* + * Process a received message. + */ +static void vfio_user_process(VFIOUserProxy *proxy, VFIOUserMsg *msg, + bool isreply) +{ + + /* + * Replies signal a waiter, if none just check for errors + * and free the message buffer. + * + * Requests get queued for the BH. + */ + if (isreply) { + msg->complete =3D true; + if (msg->type =3D=3D VFIO_MSG_WAIT) { + qemu_cond_signal(&msg->cv); + } else { + if (msg->hdr->flags & VFIO_USER_ERROR) { + error_printf("vfio_user_process: error reply on async "); + error_printf("request command %x error %s\n", + msg->hdr->command, + strerror(msg->hdr->error_reply)); + } + /* youngest nowait msg has been ack'd */ + if (proxy->last_nowait =3D=3D msg) { + proxy->last_nowait =3D NULL; + } + vfio_user_recycle(proxy, msg); + } + } else { + QTAILQ_INSERT_TAIL(&proxy->incoming, msg, next); + qemu_bh_schedule(proxy->req_bh); + } +} + +/* + * Complete a partial message read + */ +static int vfio_user_complete(VFIOUserProxy *proxy, Error **errp) +{ + VFIOUserMsg *msg =3D proxy->part_recv; + size_t msgleft =3D proxy->recv_left; + bool isreply; + char *data; + int ret; + + data =3D (char *)msg->hdr + (msg->hdr->size - msgleft); + while (msgleft > 0) { + ret =3D qio_channel_read(proxy->ioc, data, msgleft, errp); + + /* error or would block */ + if (ret <=3D 0) { + /* try for rest on next iternation */ + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + proxy->recv_left =3D msgleft; + } + return ret; + } + trace_vfio_user_recv_read(msg->hdr->id, ret); + + msgleft -=3D ret; + data +=3D ret; + } + + /* + * Read complete message, process it. + */ + proxy->part_recv =3D NULL; + proxy->recv_left =3D 0; + isreply =3D (msg->hdr->flags & VFIO_USER_TYPE) =3D=3D VFIO_USER_REPLY; + vfio_user_process(proxy, msg, isreply); + + /* return positive value */ + return 1; +} + +static void vfio_user_recv(void *opaque) +{ + VFIOUserProxy *proxy =3D opaque; + + QEMU_LOCK_GUARD(&proxy->lock); + + if (proxy->state =3D=3D VFIO_PROXY_CONNECTED) { + while (vfio_user_recv_one(proxy) =3D=3D 0) { + ; + } + } +} + +/* + * Receive and process one incoming message. + * + * For replies, find matching outgoing request and wake any waiters. + * For requests, queue in incoming list and run request BH. + */ +static int vfio_user_recv_one(VFIOUserProxy *proxy) +{ + VFIOUserMsg *msg =3D NULL; + g_autofree int *fdp =3D NULL; + VFIOUserFDs *reqfds; + VFIOUserHdr hdr; + struct iovec iov =3D { + .iov_base =3D &hdr, + .iov_len =3D sizeof(hdr), + }; + bool isreply =3D false; + int i, ret; + size_t msgleft, numfds =3D 0; + char *data =3D NULL; + char *buf =3D NULL; + Error *local_err =3D NULL; + + /* + * Complete any partial reads + */ + if (proxy->part_recv !=3D NULL) { + ret =3D vfio_user_complete(proxy, &local_err); + + /* still not complete, try later */ + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + return ret; + } + + if (ret <=3D 0) { + goto fatal; + } + /* else fall into reading another msg */ + } + + /* + * Read header + */ + ret =3D qio_channel_readv_full(proxy->ioc, &iov, 1, &fdp, &numfds, 0, + &local_err); + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + return ret; + } + + /* read error or other side closed connection */ + if (ret <=3D 0) { + goto fatal; + } + + if (ret < sizeof(hdr)) { + error_setg(&local_err, "short read of header"); + goto fatal; + } + + /* + * Validate header + */ + if (hdr.size < sizeof(VFIOUserHdr)) { + error_setg(&local_err, "bad header size"); + goto fatal; + } + switch (hdr.flags & VFIO_USER_TYPE) { + case VFIO_USER_REQUEST: + isreply =3D false; + break; + case VFIO_USER_REPLY: + isreply =3D true; + break; + default: + error_setg(&local_err, "unknown message type"); + goto fatal; + } + trace_vfio_user_recv_hdr(proxy->sockname, hdr.id, hdr.command, hdr.siz= e, + hdr.flags); + + /* + * For replies, find the matching pending request. + * For requests, reap incoming FDs. + */ + if (isreply) { + QTAILQ_FOREACH(msg, &proxy->pending, next) { + if (hdr.id =3D=3D msg->id) { + break; + } + } + if (msg =3D=3D NULL) { + error_setg(&local_err, "unexpected reply"); + goto err; + } + QTAILQ_REMOVE(&proxy->pending, msg, next); + + /* + * Process any received FDs + */ + if (numfds !=3D 0) { + if (msg->fds =3D=3D NULL || msg->fds->recv_fds < numfds) { + error_setg(&local_err, "unexpected FDs"); + goto err; + } + msg->fds->recv_fds =3D numfds; + memcpy(msg->fds->fds, fdp, numfds * sizeof(int)); + } + } else { + if (numfds !=3D 0) { + reqfds =3D vfio_user_getfds(numfds); + memcpy(reqfds->fds, fdp, numfds * sizeof(int)); + } else { + reqfds =3D NULL; + } + } + + /* + * Put the whole message into a single buffer. + */ + if (isreply) { + if (hdr.size > msg->rsize) { + error_setg(&local_err, "reply larger than recv buffer"); + goto err; + } + *msg->hdr =3D hdr; + data =3D (char *)msg->hdr + sizeof(hdr); + } else { + buf =3D g_malloc0(hdr.size); + memcpy(buf, &hdr, sizeof(hdr)); + data =3D buf + sizeof(hdr); + msg =3D vfio_user_getmsg(proxy, (VFIOUserHdr *)buf, reqfds); + msg->type =3D VFIO_MSG_REQ; + } + + /* + * Read rest of message. + */ + msgleft =3D hdr.size - sizeof(hdr); + while (msgleft > 0) { + ret =3D qio_channel_read(proxy->ioc, data, msgleft, &local_err); + + /* prepare to complete read on next iternation */ + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + proxy->part_recv =3D msg; + proxy->recv_left =3D msgleft; + return ret; + } + + if (ret <=3D 0) { + goto fatal; + } + trace_vfio_user_recv_read(hdr.id, ret); + + msgleft -=3D ret; + data +=3D ret; + } + + vfio_user_process(proxy, msg, isreply); + return 0; + + /* + * fatal means the other side closed or we don't trust the stream + * err means this message is corrupt + */ +fatal: + vfio_user_shutdown(proxy); + proxy->state =3D VFIO_PROXY_ERROR; + + /* set error if server side closed */ + if (ret =3D=3D 0) { + error_setg(&local_err, "server closed socket"); + } + +err: + for (i =3D 0; i < numfds; i++) { + close(fdp[i]); + } + if (isreply && msg !=3D NULL) { + /* force an error to keep sending thread from hanging */ + vfio_user_set_error(msg->hdr, EINVAL); + msg->complete =3D true; + qemu_cond_signal(&msg->cv); + } + error_prepend(&local_err, "vfio_user_recv_one: "); + error_report_err(local_err); + return -1; +} + static void vfio_user_cb(void *opaque) { VFIOUserProxy *proxy =3D opaque; @@ -51,6 +398,53 @@ static void vfio_user_cb(void *opaque) * Functions called by main or CPU threads */ =20 +/* + * Process incoming requests. + * + * The bus-specific callback has the form: + * request(opaque, msg) + * where 'opaque' was specified in vfio_user_set_handler + * and 'msg' is the inbound message. + * + * The callback is responsible for disposing of the message buffer, + * usually by re-using it when calling vfio_send_reply or vfio_send_error, + * both of which free their message buffer when the reply is sent. + * + * If the callback uses a new buffer, it needs to free the old one. + */ +static void vfio_user_request(void *opaque) +{ + VFIOUserProxy *proxy =3D opaque; + VFIOUserMsgQ new, free; + VFIOUserMsg *msg, *m1; + + /* reap all incoming */ + QTAILQ_INIT(&new); + WITH_QEMU_LOCK_GUARD(&proxy->lock) { + QTAILQ_FOREACH_SAFE(msg, &proxy->incoming, next, m1) { + QTAILQ_REMOVE(&proxy->incoming, msg, next); + QTAILQ_INSERT_TAIL(&new, msg, next); + } + } + + /* process list */ + QTAILQ_INIT(&free); + QTAILQ_FOREACH_SAFE(msg, &new, next, m1) { + QTAILQ_REMOVE(&new, msg, next); + trace_vfio_user_recv_request(msg->hdr->command); + proxy->request(proxy->req_arg, msg); + QTAILQ_INSERT_HEAD(&free, msg, next); + } + + /* free list */ + WITH_QEMU_LOCK_GUARD(&proxy->lock) { + QTAILQ_FOREACH_SAFE(msg, &free, next, m1) { + vfio_user_recycle(proxy, msg); + } + } +} + + static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); =20 @@ -89,6 +483,7 @@ VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr= , Error **errp) } =20 proxy->ctx =3D iothread_get_aio_context(vfio_user_iothread); + proxy->req_bh =3D qemu_bh_new(vfio_user_request, proxy); =20 QTAILQ_INIT(&proxy->outgoing); QTAILQ_INIT(&proxy->incoming); @@ -99,6 +494,18 @@ VFIOUserProxy *vfio_user_connect_dev(SocketAddress *add= r, Error **errp) return proxy; } =20 +void vfio_user_set_handler(VFIODevice *vbasedev, + void (*handler)(void *opaque, VFIOUserMsg *msg), + void *req_arg) +{ + VFIOUserProxy *proxy =3D vbasedev->proxy; + + proxy->request =3D handler; + proxy->req_arg =3D req_arg; + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, + vfio_user_recv, NULL, NULL, proxy); +} + void vfio_user_disconnect(VFIOUserProxy *proxy) { VFIOUserMsg *r1, *r2; @@ -114,6 +521,8 @@ void vfio_user_disconnect(VFIOUserProxy *proxy) } object_unref(OBJECT(proxy->ioc)); proxy->ioc =3D NULL; + qemu_bh_delete(proxy->req_bh); + proxy->req_bh =3D NULL; =20 proxy->state =3D VFIO_PROXY_CLOSING; QTAILQ_FOREACH_SAFE(r1, &proxy->outgoing, next, r2) { diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events new file mode 100644 index 0000000000..89d6c11c4c --- /dev/null +++ b/hw/vfio-user/trace-events @@ -0,0 +1,6 @@ +# See docs/devel/tracing.rst for syntax documentation. + +# common.c +vfio_user_recv_hdr(const char *name, uint16_t id, uint16_t cmd, uint32_t s= ize, uint32_t flags) " (%s) id 0x%x cmd 0x%x size 0x%x flags 0x%x" +vfio_user_recv_read(uint16_t id, int read) " id 0x%x read 0x%x" +vfio_user_recv_request(uint16_t cmd) " command 0x%x" --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340156; cv=pass; d=zohomail.com; s=zohoarc; b=Iv+fkMTgW7eZqcwZlqI4RTNL2RShkHtSFDFJ92kJX1iSxp4KEs/uGuas46/x5gYC5MFpfvmJb+t6qK4J0ei+XGsK6kvwRY2sxmXsk/olv7BEqgkj3hrdkL38bcw/bTJmd+Es9Et51jpd+FRsInhq51r0Hx9Oq8xhbaG0wXwskXY= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340156; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=2bD4Msh/DgvNZqqoBsVjzDGkAWswT+7/lucg1p0zASI=; b=KJIQ4Fhn4qgt/LChmyKi4r5N+6eh6XH6wsczjoEJrAF/RVuJhKAQPnQmO2nEdCAUAK4Hl4MgyymQ0TPExIz0ctdcQpu+0KTT6N2OuYY/NxOLoazNpJy3S8LiwAZTUDiPUqVcy+STIfrSkEd8hjx1k05/JBFtwAX6DhY3H5k1pDg= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340156134984.9638334268925; Thu, 19 Jun 2025 06:35:56 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFNb-0004Dn-8j; Thu, 19 Jun 2025 09:32:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNX-0004CP-LD for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:31 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNR-0008Co-Dc for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:31 -0400 Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55JBwK8o024158; Thu, 19 Jun 2025 06:32:22 -0700 Received: from dm1pr04cu001.outbound.protection.outlook.com (mail-centralusazon11020080.outbound.protection.outlook.com [52.101.61.80]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47c0vfjedr-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:21 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:09 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=2bD4Msh/DgvNZqqoBsVjzDGkAWswT+7/lucg1p0zA SI=; b=bE4+x7cCfs4I5owB9EEuuVGj7IrhG6DAbvjkSHDlxWwtf6iffBEAw1kg3 ujGk1IEwV3pQSOYa9Vl7iOdBVRAf8yBmkWcRKdwJS17MTZlK6nyAvLryhMNnU0yq B2EP5gX/E9lEvFbZMzKxdGZvW5ccHgzdCHrhmxURNoim2wQUlMIw9IYeliBfRwtT 19TLjHpfgTR+sb59hgQ5fAWO6Q+RSJcPiLvYvIW6TeVTudaIPY2nBlT7mopvEe6e AMgzqK/fBZ9Eth/dnV7H8IfnYsNsTR0IWHbGaoRQef75pJlGf6XCCJGySPSH7Xt1 Yfzjtl0Y8P7v1cLYwU8XIZr3XTfkQ== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BHG86WYsfF0D3LAVynveBnDvIf6/lz3A5BjQHHgqLVKPZ3TTlB1lLiNhKlg1cEs2/wYIXUID0vcCvIIIypX8BL8PtmKIxuVlIb5sBDRYYkDbW+JyF0eLgEuftcyyKxwSkYIioh9KmWz5dGuvoNIuDZn8l8mDDDeOH7Q8/60lL0Oo/zbX74ORo+6FbJ5Ml1f2TysEx0B25GxHvyrZMxhXo3v/TuBCxctZqCBFfam9yq2sFAJIn10o1qPAMeNga1dwnSIUliPeSoEGaQ7DFLgXTY/cqLVFWy0XQT6GyYvsNIayLb80LE2NPpma3bNQjlka3zUcCSiKlYNAdmL+mdzFrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2bD4Msh/DgvNZqqoBsVjzDGkAWswT+7/lucg1p0zASI=; b=M2Hi4TXDixtRGROXh2tcrowwSXxiC2Ygo+QVKTLPwbrtbiiw2OUWPQrwyD/76LPrSV9rtKTdKlmT5tcKg0fB1X7aBSF+dm9dPbFxmHRPWpGdSBzxpAVvMxFZvUG8ZxJQrl1UyAmotvFFaOH9n+UKTRvYqmjlvc1faPKYsJO6tZedj0sstpgW5Noe7xFohFPU6VOyTTSzlwOxHhzEk8Pd4rbIXxiTQukLr5zMPwPK1YA0VgBlJ6IvmoUrKpSn6Bsz1vIXktN6gIXUnv2itAGSaiHrflKY1nYHolyrOsi/iJdkojFE5OEN1uF6+LCo6iym7vw48TRt4CYAxzHMgI1sLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2bD4Msh/DgvNZqqoBsVjzDGkAWswT+7/lucg1p0zASI=; b=x6OBl8h9+NjlyiR3Uev8we6I9br5kqOpjniM6dqOWC/Rg61NIR7LROlq+lml1H/ie5mhmSkAXzXsLvmPsEjYs60MbNKhPLhBeadlsdzPEM0DPlDJH+TDoo2UYTUniu16IFr0av214lCDtZdkv63teA94qr1bkAoyqvhNY7vFLGhOxnUWunVGu5uJ0nINhCcpMXEj6FVesW25hQfyaUDY5T05Y15+RH9YJ8YidPzQrpN5tlqeQuaRgatMvEhYubGnbEeSXA8fZlax5XGasrq+dPfgRmfqkqr8uIz14j+9y+T3ZksZPBMPcWtUoZyvTQHhxtI7hv/1c8DQWHJmM+2ROQ== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Jagannathan Raman , Elena Ufimtseva Subject: [PATCH v4 04/19] vfio-user: implement message send infrastructure Date: Thu, 19 Jun 2025 06:31:38 -0700 Message-ID: <20250619133154.264786-5-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 6cef2998-b04d-4413-1e63-08ddaf35b09c x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?7+n1wMskcG/pEfeLz/3qjpUl1kpt8KtsbTSZzZmwCVdMHI1An4f/ApNhS7zd?= =?us-ascii?Q?2ctDKfyMLqgd/iBmvgO7fgy822wHl0clmGlgXX9yHZcFJw3E+UlC1504jyZ9?= =?us-ascii?Q?DqpiEY+8ndmj9bCKw20LRZbbHNpWqY6uEcIcWuacBtyRoGOBa5p+HHVQ+pwQ?= =?us-ascii?Q?rCREP8E6kCSoEUrzUAUzsao/t8Cn30IVCjiXgu4/FpPN8zFmbuvnPEO+bPOB?= =?us-ascii?Q?Jj8eGeFJ+C1GIZcN+7B9/VfS2m6sxaZgYC8jVuEb1ToC3NvcREGroGwk/F96?= =?us-ascii?Q?hm3L/8BnCUaxDCiv25uDx5ldzAMG6nuNpdzrgnDMjdxfTZWkmTqPUob1Ce4z?= =?us-ascii?Q?x/Uyh79YVz0qWXlCdF58s9uDuR7pPxnETFuyQTJPJNXBBkFNzDySxNYwE8yL?= =?us-ascii?Q?j5gF5oMIRVU0DFY1sibvd+f+mWBl5YtzAfOXWFnNaLQXSR9xiPRoU+94tAAb?= =?us-ascii?Q?TXGH3MtB2cW9H8EOGeeNxX+Z0UlLhuuoJN3G90vqZJkniFUL2iDJwTuRiJM8?= =?us-ascii?Q?xPjVGUzvuNz0FhXDzZOd05O3rgcwuLVOxFjxRvXb7aNjDg36BpGjRx2+uLCa?= =?us-ascii?Q?3/qA0nUU0+hkJqITlOEN04XXWa/NWSmv9q3rQS/kXZUx4XfKaAOfgaqjr9mJ?= =?us-ascii?Q?rF/w8QczyPR6BUtPM6s903Le/mZ9juMOkXJDCa2ya829gUPHOEriMJAOK81n?= =?us-ascii?Q?xe9wl9EfmPXkO/fy95Ub2T52xfEj046jBBZF179Cyil1qk0BBKx+5P5lq3pZ?= =?us-ascii?Q?lzLG9Dxfja00uAXa08y1NvOQG+mjF08cmCjNEvN3HWA6HfSHcNZmjjDBwhkh?= =?us-ascii?Q?vewWWSpwXJGa5ecQ4fUeSA6zunYbr2HxpkQ4SFmxYwhQWhAGRKZLCX6N/bKQ?= =?us-ascii?Q?bHcm6luv2sPTgLcAQNElO9KduaQB+Vae7h7hYX7QazFMtrNNwoxKMgLQPkJI?= =?us-ascii?Q?JZ2j6BvF6aoI+1WqA6UiObXeTtGkUKrZG5VmAwzfA30F2rBkMKGw1gvHIC34?= =?us-ascii?Q?rw2CgAl6En46YDhJR70StoR1e76HHa5vpcqjkzbM9JfoxZ8E+ENZ08nTAal6?= =?us-ascii?Q?bPyMGs1Kzo+Qhi0RJBJBKvN5FRNP2C+a/DRhztCbEZCRcStwzWeObpa8Yx4a?= =?us-ascii?Q?fo7+uWxivjxyYqRG0qxc3JE3IqPSIPDm9mXnkDQMBioS6F1s68drWF4JP92I?= =?us-ascii?Q?AezjBAz8mPNizITKBv3fSpBmlqIYc5zdoLWYTUDuhh9zF352PhWVHRd+cnpx?= =?us-ascii?Q?IbXghneiMqwCbMLEVifI9vtslJKp5hgbg70jqFQSeN/7Pb8TrG1gFjyqe7S7?= =?us-ascii?Q?gLIRNlBqI4NLty1EyKY74IeqscToLtpBSw5OGeu6MljDFwBX4yBtP0hKUOG3?= =?us-ascii?Q?pIYz8AMX36glzWU8phvmrlpal78GwBQKD3VXFkqgH02iDE5vA0D+AUX4q96C?= =?us-ascii?Q?QNQ6lwfrk8k=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?+Yd+z9+4yhwxL/zjKFLlWXF4Z6zusqwhYyusPCe2sy9vuvKlr8PX9xwYoa9u?= =?us-ascii?Q?mZ50u/KgWEWoK6ewmFduqgDBOLL1hlS1cYopfe7tcxdNRYrJWZBmlU61UjGw?= =?us-ascii?Q?zFZTHRg1mdxwxIqEcQ4/CQE3w0ZpEnMwynTyhFn1b0uhJeQzkf5GZBdUABK9?= =?us-ascii?Q?UdTWITrOnU5/H3P42eZ0JIfa9JRMZsZ//OcdFUTg3szDTmggW/m5n7GYuRSN?= =?us-ascii?Q?FVNHzJnlmrZvOM8byMG9v8uwaAryMPLGBaSk8xar3fEA5ZYPw1sgttO5YlCW?= =?us-ascii?Q?7tGmCYHdfAFPsQLEloy7rN+ciTwWzS1QHOZWYByhsKuWrAX0Na5IaNDcn0IG?= =?us-ascii?Q?RssKwJ5RYiAWL77UAo5HA4uUbYFbJPbCWpRkgcfe/dEl/tVWq1QGwM2iAqVd?= =?us-ascii?Q?lOO3a6Zl2xkdz15jWV482iiv6IrHHHAlfAkWsi6PVcB5lkOsnlpa6ncbENJ6?= =?us-ascii?Q?/5BNcEx1PM0RvcxMShglkFC7bbe34/K6+2Si4ME2G0Nf6UurhWIhLUwiW6G0?= =?us-ascii?Q?aL0/Qf5n7Eixvy4Z9gmCC3dYLmw9w7BnizQ9/hariEmotNbnGW8N0bll+dei?= =?us-ascii?Q?UGzVcWjhUdsers4SXDU21mo/s6INl8/5GjcJRXHZfnT4VK6ZCj8ya9aTVAKH?= =?us-ascii?Q?RAggI9SlZxN31iRo2yPz9RREZfWJ2XuL+WoHNM81WQmR8r2UkJ0hocEYydY5?= =?us-ascii?Q?5kAdtzOmZMKStO/dbl/gBPM5Y4YEMJFt1NTLVkIxSvdnr1mVSNhZHk477oDD?= =?us-ascii?Q?Yb98e1OhLOGNLQNAu1OWIOMID6ibnLBYE91kEVEmqjFSDTkQdYevkChL6u+c?= =?us-ascii?Q?sSBdmd+T3uPNZFmaZ1GB52ICoG+r1oSVNEr571fhNeGhC09srztT+lTrWe9g?= =?us-ascii?Q?0eQ6+hKC7oVviPEtyR94/6+pfXAPVnqUd1cews/iYcINB5p577afI7hSkQsQ?= =?us-ascii?Q?3jek5c+Fde/iAo5R52MtiR/BZ3o5ZowY0ev3BBdMZNL8vZMdq84C2J1gcJVR?= =?us-ascii?Q?G+d790z0g69iBy8Cwf/9CB5H/bUhW4d7qZGwNTcB7tQP4QlAOphJT4JIKgBS?= =?us-ascii?Q?Zaodeb1NGvUlGf/fdyqkDf70+fvxva2bLoWdkdepRy31XcwbuaN38pHGhQHo?= =?us-ascii?Q?DX8GgL74IRIHBv8CRQqDDXX/QqaYWyOxmwfCMbcKri3SQIdx6ENPjJVPlBBN?= =?us-ascii?Q?jkLIk9AVWYEZ77ZgymgFwqEnRGkyXTKQMEis8/7YcJ85a56j4ea0z/7/XUBH?= =?us-ascii?Q?ZcBvpVDMPzCAp1grBjzVChhJpRIpsVntm6p2iPBivMxKcC1rOv2DvjuN/ugr?= =?us-ascii?Q?J6u+NpIgn4HZhl2vKIvam250s8Zhqpu3OTHIVd5pP9N47F4naK3B+cVccXOS?= =?us-ascii?Q?05ytGXNdQDFIOuzyAsc7vZrreurFMi+gTtBHD+ST/6h5P+x+e91D2zcUyBkG?= =?us-ascii?Q?uEkZEZoBxITzaB6jpH3vXOcCWRWK/PeB0vkwcAM7eo3s4HMxAJfvoYlHT4JD?= =?us-ascii?Q?TsvlCFIoHLE3K1KxVmKZNQ/NV1jRDMhEUMLW/wrFcBgDVnstFR6qH6fJ8Wvn?= =?us-ascii?Q?kra1zuJse/gFeiw2PIMe7QiDMIEqooKpwpdhWeGe?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6cef2998-b04d-4413-1e63-08ddaf35b09c X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:09.1784 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: LzTEJa8eYlMZy2XX0olq+hE6BsJLGG9LA7u4Gx3ZN4VyrFY8k8rV11atTe3NT9TJOAQGpsuH5MB6HOrM27w3sw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-GUID: i4yyOrdZ95iuZkfNNVyxCzlHxbo0OE0k X-Authority-Analysis: v=2.4 cv=Tc2WtQQh c=1 sm=1 tr=0 ts=68541166 cx=c_pps a=UMsKb9OrGFB5W2EpWsGVZg==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=UnscCpd6-9sbi9RN3AUA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMyBTYWx0ZWRfX6HgQpDsxMzWL 917Loid+k74HMhp53pTzAw8yEyG36YTtl2BhdiJDGk7iyvhhVxKQgdZncbF3DV7KMaJ+b2D2Gtf +xJyHfn/ZcTkK8XdLea9eEraJdNxWvQhDoP3ihJsNs+L0x0INwP6rxUSx2o8EO2RnDXfjT1pyOD NR7h3dB5kRYaJ5yCyYHR8q71lD6EDbBqTQF9Hhr1IufUN0LomSjT9CT/xeN1SsLkrmFI5OFYBbL g/hzgdWi4JnmV5+bd26rHqQ63qqfVND/6lxth0jT8KIUlJOmd27BCMPlYV4sK7JvCYxu7GOscAE fY4qhvGSoZBB5YXRMr4vo/wEPZFmyy+tsgzTiIBX36DhD0FYOROH5QkHq/gpMn9+rMfUxfg7bS3 leKy2tgaTXl5XFYoHpu6AkiCKM+IW3Cv1a8NK2rrfP7bNHEvBrOFyhDb2afyrrLFtmRj8JCg X-Proofpoint-ORIG-GUID: i4yyOrdZ95iuZkfNNVyxCzlHxbo0OE0k X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340157769116600 Content-Type: text/plain; charset="utf-8" Add plumbing for sending vfio-user messages on the control socket. Add initial version negotation on connection. Originally-by: John Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John Levon --- hw/vfio-user/protocol.h | 62 +++++ hw/vfio-user/proxy.h | 9 + hw/vfio-user/pci.c | 20 +- hw/vfio-user/proxy.c | 494 ++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 2 + 5 files changed, 585 insertions(+), 2 deletions(-) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 4ddfb5f222..2d52d0fb10 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -50,4 +50,66 @@ enum vfio_user_command { #define VFIO_USER_NO_REPLY 0x10 #define VFIO_USER_ERROR 0x20 =20 + +/* + * VFIO_USER_VERSION + */ +typedef struct { + VFIOUserHdr hdr; + uint16_t major; + uint16_t minor; + char capabilities[]; +} VFIOUserVersion; + +#define VFIO_USER_MAJOR_VER 0 +#define VFIO_USER_MINOR_VER 0 + +#define VFIO_USER_CAP "capabilities" + +/* "capabilities" members */ +#define VFIO_USER_CAP_MAX_FDS "max_msg_fds" +#define VFIO_USER_CAP_MAX_XFER "max_data_xfer_size" +#define VFIO_USER_CAP_PGSIZES "pgsizes" +#define VFIO_USER_CAP_MAP_MAX "max_dma_maps" +#define VFIO_USER_CAP_MIGR "migration" + +/* "migration" members */ +#define VFIO_USER_CAP_PGSIZE "pgsize" +#define VFIO_USER_CAP_MAX_BITMAP "max_bitmap_size" + +/* + * Max FDs mainly comes into play when a device supports multiple interrup= ts + * where each ones uses an eventfd to inject it into the guest. + * It is clamped by the the number of FDs the qio channel supports in a + * single message. + */ +#define VFIO_USER_DEF_MAX_FDS 8 +#define VFIO_USER_MAX_MAX_FDS 16 + +/* + * Max transfer limits the amount of data in region and DMA messages. + * Region R/W will be very small (limited by how much a single instruction + * can process) so just use a reasonable limit here. + */ +#define VFIO_USER_DEF_MAX_XFER (1024 * 1024) +#define VFIO_USER_MAX_MAX_XFER (64 * 1024 * 1024) + +/* + * Default pagesizes supported is 4k. + */ +#define VFIO_USER_DEF_PGSIZE 4096 + +/* + * Default max number of DMA mappings is stolen from the + * linux kernel "dma_entry_limit" + */ +#define VFIO_USER_DEF_MAP_MAX 65535 + +/* + * Default max bitmap size is also take from the linux kernel, + * where usage of signed ints limits the VA range to 2^31 bytes. + * Dividing that by the number of bits per byte yields 256MB + */ +#define VFIO_USER_DEF_MAX_BITMAP (256 * 1024 * 1024) + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index ff553cad9d..5bc890a0f5 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -37,6 +37,7 @@ typedef struct VFIOUserMsg { uint32_t id; QemuCond cv; bool complete; + bool pending; enum msg_type type; } VFIOUserMsg; =20 @@ -56,6 +57,12 @@ typedef struct VFIOUserProxy { struct QIOChannel *ioc; void (*request)(void *opaque, VFIOUserMsg *msg); void *req_arg; + uint64_t max_xfer_size; + uint64_t max_send_fds; + uint64_t max_dma; + uint64_t dma_pgsizes; + uint64_t max_bitmap; + uint64_t migr_pgsize; int flags; QemuCond close_cv; AioContext *ctx; @@ -78,6 +85,7 @@ typedef struct VFIOUserProxy { =20 /* VFIOProxy flags */ #define VFIO_PROXY_CLIENT 0x1 +#define VFIO_PROXY_FORCE_QUEUED 0x4 =20 typedef struct VFIODevice VFIODevice; =20 @@ -86,5 +94,6 @@ void vfio_user_disconnect(VFIOUserProxy *proxy); void vfio_user_set_handler(VFIODevice *vbasedev, void (*handler)(void *opaque, VFIOUserMsg *msg), void *reqarg); +bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp); =20 #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index bad2829f5c..61f525cf4a 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -20,6 +20,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserPCIDevice, VFIO_USER_P= CI) struct VFIOUserPCIDevice { VFIOPCIDevice device; SocketAddress *socket; + bool send_queued; /* all sends are queued */ }; =20 /* @@ -92,6 +93,16 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error= **errp) vbasedev->proxy =3D proxy; vfio_user_set_handler(vbasedev, vfio_user_pci_process_req, vdev); =20 + vbasedev->name =3D g_strdup_printf("vfio-user:%s", sock_name); + + if (udev->send_queued) { + proxy->flags |=3D VFIO_PROXY_FORCE_QUEUED; + } + + if (!vfio_user_validate_version(proxy, errp)) { + goto error; + } + /* * vfio-user devices are effectively mdevs (don't use a host iommu). */ @@ -101,9 +112,13 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Err= or **errp) if (!vfio_device_attach_by_iommu_type(TYPE_VFIO_IOMMU_USER, vbasedev->name, vbasedev, as, errp)) { - error_prepend(errp, VFIO_MSG_PREFIX, vbasedev->name); - return; + goto error; } + + return; + +error: + error_prepend(errp, VFIO_MSG_PREFIX, vdev->vbasedev.name); } =20 static void vfio_user_instance_init(Object *obj) @@ -153,6 +168,7 @@ static const Property vfio_user_pci_dev_properties[] = =3D { sub_vendor_id, PCI_ANY_ID), DEFINE_PROP_UINT32("x-pci-sub-device-id", VFIOPCIDevice, sub_device_id, PCI_ANY_ID), + DEFINE_PROP_BOOL("x-send-queued", VFIOUserPCIDevice, send_queued, fals= e), }; =20 static void vfio_user_pci_set_socket(Object *obj, Visitor *v, const char *= name, diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 31e08cbad3..0887d0aa1a 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -13,14 +13,19 @@ #include "hw/vfio-user/proxy.h" #include "hw/vfio-user/trace.h" #include "qapi/error.h" +#include "qobject/qdict.h" +#include "qobject/qjson.h" +#include "qobject/qnum.h" #include "qemu/error-report.h" #include "qemu/lockable.h" #include "qemu/main-loop.h" #include "system/iothread.h" =20 +static int wait_time =3D 5000; /* wait up to 5 sec for busy servers */ static IOThread *vfio_user_iothread; =20 static void vfio_user_shutdown(VFIOUserProxy *proxy); +static int vfio_user_send_qio(VFIOUserProxy *proxy, VFIOUserMsg *msg); static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, VFIOUserFDs *fds); static VFIOUserFDs *vfio_user_getfds(int numfds); @@ -28,9 +33,16 @@ static void vfio_user_recycle(VFIOUserProxy *proxy, VFIO= UserMsg *msg); =20 static void vfio_user_recv(void *opaque); static int vfio_user_recv_one(VFIOUserProxy *proxy); +static void vfio_user_send(void *opaque); +static int vfio_user_send_one(VFIOUserProxy *proxy); static void vfio_user_cb(void *opaque); =20 static void vfio_user_request(void *opaque); +static int vfio_user_send_queued(VFIOUserProxy *proxy, VFIOUserMsg *msg); +static void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize); +static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags); =20 static inline void vfio_user_set_error(VFIOUserHdr *hdr, uint32_t err) { @@ -49,6 +61,35 @@ static void vfio_user_shutdown(VFIOUserProxy *proxy) proxy->ctx, NULL, NULL); } =20 +static int vfio_user_send_qio(VFIOUserProxy *proxy, VFIOUserMsg *msg) +{ + VFIOUserFDs *fds =3D msg->fds; + struct iovec iov =3D { + .iov_base =3D msg->hdr, + .iov_len =3D msg->hdr->size, + }; + size_t numfds =3D 0; + int ret, *fdp =3D NULL; + Error *local_err =3D NULL; + + if (fds !=3D NULL && fds->send_fds !=3D 0) { + numfds =3D fds->send_fds; + fdp =3D fds->fds; + } + + ret =3D qio_channel_writev_full(proxy->ioc, &iov, 1, fdp, numfds, 0, + &local_err); + + if (ret =3D=3D -1) { + vfio_user_set_error(msg->hdr, EIO); + vfio_user_shutdown(proxy); + error_report_err(local_err); + } + trace_vfio_user_send_write(msg->hdr->id, ret); + + return ret; +} + static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, VFIOUserFDs *fds) { @@ -89,6 +130,7 @@ static void vfio_user_recycle(VFIOUserProxy *proxy, VFIO= UserMsg *msg) msg->hdr =3D NULL; msg->fds =3D NULL; msg->complete =3D false; + msg->pending =3D false; QTAILQ_INSERT_HEAD(&proxy->free, msg, next); } =20 @@ -383,6 +425,54 @@ err: return -1; } =20 +/* + * Send messages from outgoing queue when the socket buffer has space. + * If we deplete 'outgoing', remove ourselves from the poll list. + */ +static void vfio_user_send(void *opaque) +{ + VFIOUserProxy *proxy =3D opaque; + + QEMU_LOCK_GUARD(&proxy->lock); + + if (proxy->state =3D=3D VFIO_PROXY_CONNECTED) { + while (!QTAILQ_EMPTY(&proxy->outgoing)) { + if (vfio_user_send_one(proxy) < 0) { + return; + } + } + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, + vfio_user_recv, NULL, NULL, proxy); + } +} + +/* + * Send a single message. + * + * Sent async messages are freed, others are moved to pending queue. + */ +static int vfio_user_send_one(VFIOUserProxy *proxy) +{ + VFIOUserMsg *msg; + int ret; + + msg =3D QTAILQ_FIRST(&proxy->outgoing); + ret =3D vfio_user_send_qio(proxy, msg); + if (ret < 0) { + return ret; + } + + QTAILQ_REMOVE(&proxy->outgoing, msg, next); + if (msg->type =3D=3D VFIO_MSG_ASYNC) { + vfio_user_recycle(proxy, msg); + } else { + QTAILQ_INSERT_TAIL(&proxy->pending, msg, next); + msg->pending =3D true; + } + + return 0; +} + static void vfio_user_cb(void *opaque) { VFIOUserProxy *proxy =3D opaque; @@ -444,6 +534,119 @@ static void vfio_user_request(void *opaque) } } =20 +/* + * Messages are queued onto the proxy's outgoing list. + * + * It handles 3 types of messages: + * + * async messages - replies and posted writes + * + * There will be no reply from the server, so message + * buffers are freed after they're sent. + * + * nowait messages - map/unmap during address space transactions + * + * These are also sent async, but a reply is expected so that + * vfio_wait_reqs() can wait for the youngest nowait request. + * They transition from the outgoing list to the pending list + * when sent, and are freed when the reply is received. + * + * wait messages - all other requests + * + * The reply to these messages is waited for by their caller. + * They also transition from outgoing to pending when sent, but + * the message buffer is returned to the caller with the reply + * contents. The caller is responsible for freeing these messages. + * + * As an optimization, if the outgoing list and the socket send + * buffer are empty, the message is sent inline instead of being + * added to the outgoing list. The rest of the transitions are + * unchanged. + * + * returns 0 if the message was sent or queued + * returns -1 on send error + */ +static int vfio_user_send_queued(VFIOUserProxy *proxy, VFIOUserMsg *msg) +{ + int ret; + + /* + * Unsent outgoing msgs - add to tail + */ + if (!QTAILQ_EMPTY(&proxy->outgoing)) { + QTAILQ_INSERT_TAIL(&proxy->outgoing, msg, next); + return 0; + } + + /* + * Try inline - if blocked, queue it and kick send poller + */ + if (proxy->flags & VFIO_PROXY_FORCE_QUEUED) { + ret =3D QIO_CHANNEL_ERR_BLOCK; + } else { + ret =3D vfio_user_send_qio(proxy, msg); + } + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + QTAILQ_INSERT_HEAD(&proxy->outgoing, msg, next); + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, + vfio_user_recv, proxy->ctx, + vfio_user_send, proxy); + return 0; + } + if (ret =3D=3D -1) { + return ret; + } + + /* + * Sent - free async, add others to pending + */ + if (msg->type =3D=3D VFIO_MSG_ASYNC) { + vfio_user_recycle(proxy, msg); + } else { + QTAILQ_INSERT_TAIL(&proxy->pending, msg, next); + msg->pending =3D true; + } + + return 0; +} + +static void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize) +{ + VFIOUserMsg *msg; + int ret; + + if (hdr->flags & VFIO_USER_NO_REPLY) { + error_printf("vfio_user_send_wait on async message\n"); + vfio_user_set_error(hdr, EINVAL); + return; + } + + qemu_mutex_lock(&proxy->lock); + + msg =3D vfio_user_getmsg(proxy, hdr, fds); + msg->id =3D hdr->id; + msg->rsize =3D rsize ? rsize : hdr->size; + msg->type =3D VFIO_MSG_WAIT; + + ret =3D vfio_user_send_queued(proxy, msg); + + if (ret =3D=3D 0) { + while (!msg->complete) { + if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, wait_time)) { + VFIOUserMsgQ *list; + + list =3D msg->pending ? &proxy->pending : &proxy->outgoing; + QTAILQ_REMOVE(list, msg, next); + vfio_user_set_error(hdr, ETIMEDOUT); + break; + } + } + } + vfio_user_recycle(proxy, msg); + + qemu_mutex_unlock(&proxy->lock); +} =20 static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); @@ -472,6 +675,15 @@ VFIOUserProxy *vfio_user_connect_dev(SocketAddress *ad= dr, Error **errp) proxy =3D g_malloc0(sizeof(VFIOUserProxy)); proxy->sockname =3D g_strdup_printf("unix:%s", sockname); proxy->ioc =3D ioc; + + /* init defaults */ + proxy->max_xfer_size =3D VFIO_USER_DEF_MAX_XFER; + proxy->max_send_fds =3D VFIO_USER_DEF_MAX_FDS; + proxy->max_dma =3D VFIO_USER_DEF_MAP_MAX; + proxy->dma_pgsizes =3D VFIO_USER_DEF_PGSIZE; + proxy->max_bitmap =3D VFIO_USER_DEF_MAX_BITMAP; + proxy->migr_pgsize =3D VFIO_USER_DEF_PGSIZE; + proxy->flags =3D VFIO_PROXY_CLIENT; proxy->state =3D VFIO_PROXY_CONNECTED; =20 @@ -569,3 +781,285 @@ void vfio_user_disconnect(VFIOUserProxy *proxy) g_free(proxy->sockname); g_free(proxy); } + +static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags) +{ + static uint16_t next_id; + + hdr->id =3D qatomic_fetch_inc(&next_id); + hdr->command =3D cmd; + hdr->size =3D size; + hdr->flags =3D (flags & ~VFIO_USER_TYPE) | VFIO_USER_REQUEST; + hdr->error_reply =3D 0; +} + +struct cap_entry { + const char *name; + bool (*check)(VFIOUserProxy *proxy, QObject *qobj, Error **errp); +}; + +static bool caps_parse(VFIOUserProxy *proxy, QDict *qdict, + struct cap_entry caps[], Error **errp) +{ + QObject *qobj; + struct cap_entry *p; + + for (p =3D caps; p->name !=3D NULL; p++) { + qobj =3D qdict_get(qdict, p->name); + if (qobj !=3D NULL) { + if (!p->check(proxy, qobj, errp)) { + return false; + } + qdict_del(qdict, p->name); + } + } + + /* warning, for now */ + if (qdict_size(qdict) !=3D 0) { + warn_report("spurious capabilities"); + } + return true; +} + +static bool check_migr_pgsize(VFIOUserProxy *proxy, QObject *qobj, Error *= *errp) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t pgsize; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &pgsize)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_PGSIZE); + return false; + } + + /* must be larger than default */ + if (pgsize & (VFIO_USER_DEF_PGSIZE - 1)) { + error_setg(errp, "pgsize 0x%"PRIx64" too small", pgsize); + return false; + } + + proxy->migr_pgsize =3D pgsize; + return true; +} + +static bool check_bitmap(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t bitmap_size; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &bitmap_size)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_BITMAP); + return false; + } + + /* can only lower it */ + if (bitmap_size > VFIO_USER_DEF_MAX_BITMAP) { + error_setg(errp, "%s too large", VFIO_USER_CAP_MAX_BITMAP); + return false; + } + + proxy->max_bitmap =3D bitmap_size; + return true; +} + +static struct cap_entry caps_migr[] =3D { + { VFIO_USER_CAP_PGSIZE, check_migr_pgsize }, + { VFIO_USER_CAP_MAX_BITMAP, check_bitmap }, + { NULL } +}; + +static bool check_max_fds(VFIOUserProxy *proxy, QObject *qobj, Error **err= p) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t max_send_fds; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &max_send_fds) || + max_send_fds > VFIO_USER_MAX_MAX_FDS) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_FDS); + return false; + } + proxy->max_send_fds =3D max_send_fds; + return true; +} + +static bool check_max_xfer(VFIOUserProxy *proxy, QObject *qobj, Error **er= rp) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t max_xfer_size; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &max_xfer_size) || + max_xfer_size > VFIO_USER_MAX_MAX_XFER) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_XFER); + return false; + } + proxy->max_xfer_size =3D max_xfer_size; + return true; +} + +static bool check_pgsizes(VFIOUserProxy *proxy, QObject *qobj, Error **err= p) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t pgsizes; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &pgsizes)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_PGSIZES); + return false; + } + + /* must be larger than default */ + if (pgsizes & (VFIO_USER_DEF_PGSIZE - 1)) { + error_setg(errp, "pgsize 0x%"PRIx64" too small", pgsizes); + return false; + } + + proxy->dma_pgsizes =3D pgsizes; + return true; +} + +static bool check_max_dma(VFIOUserProxy *proxy, QObject *qobj, Error **err= p) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t max_dma; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &max_dma)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAP_MAX); + return false; + } + + /* can only lower it */ + if (max_dma > VFIO_USER_DEF_MAP_MAX) { + error_setg(errp, "%s too large", VFIO_USER_CAP_MAP_MAX); + return false; + } + + proxy->max_dma =3D max_dma; + return true; +} + +static bool check_migr(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QDict *qdict =3D qobject_to(QDict, qobj); + + if (qdict =3D=3D NULL) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_FDS); + return true; + } + return caps_parse(proxy, qdict, caps_migr, errp); +} + +static struct cap_entry caps_cap[] =3D { + { VFIO_USER_CAP_MAX_FDS, check_max_fds }, + { VFIO_USER_CAP_MAX_XFER, check_max_xfer }, + { VFIO_USER_CAP_PGSIZES, check_pgsizes }, + { VFIO_USER_CAP_MAP_MAX, check_max_dma }, + { VFIO_USER_CAP_MIGR, check_migr }, + { NULL } +}; + +static bool check_cap(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QDict *qdict =3D qobject_to(QDict, qobj); + + if (qdict =3D=3D NULL) { + error_setg(errp, "malformed %s", VFIO_USER_CAP); + return false; + } + return caps_parse(proxy, qdict, caps_cap, errp); +} + +static struct cap_entry ver_0_0[] =3D { + { VFIO_USER_CAP, check_cap }, + { NULL } +}; + +static bool caps_check(VFIOUserProxy *proxy, int minor, const char *caps, + Error **errp) +{ + QObject *qobj; + QDict *qdict; + bool ret; + + qobj =3D qobject_from_json(caps, NULL); + if (qobj =3D=3D NULL) { + error_setg(errp, "malformed capabilities %s", caps); + return false; + } + qdict =3D qobject_to(QDict, qobj); + if (qdict =3D=3D NULL) { + error_setg(errp, "capabilities %s not an object", caps); + qobject_unref(qobj); + return false; + } + ret =3D caps_parse(proxy, qdict, ver_0_0, errp); + + qobject_unref(qobj); + return ret; +} + +static GString *caps_json(void) +{ + QDict *dict =3D qdict_new(); + QDict *capdict =3D qdict_new(); + QDict *migdict =3D qdict_new(); + GString *str; + + qdict_put_int(migdict, VFIO_USER_CAP_PGSIZE, VFIO_USER_DEF_PGSIZE); + qdict_put_int(migdict, VFIO_USER_CAP_MAX_BITMAP, VFIO_USER_DEF_MAX_BIT= MAP); + qdict_put_obj(capdict, VFIO_USER_CAP_MIGR, QOBJECT(migdict)); + + qdict_put_int(capdict, VFIO_USER_CAP_MAX_FDS, VFIO_USER_MAX_MAX_FDS); + qdict_put_int(capdict, VFIO_USER_CAP_MAX_XFER, VFIO_USER_DEF_MAX_XFER); + qdict_put_int(capdict, VFIO_USER_CAP_PGSIZES, VFIO_USER_DEF_PGSIZE); + qdict_put_int(capdict, VFIO_USER_CAP_MAP_MAX, VFIO_USER_DEF_MAP_MAX); + + qdict_put_obj(dict, VFIO_USER_CAP, QOBJECT(capdict)); + + str =3D qobject_to_json(QOBJECT(dict)); + qobject_unref(dict); + return str; +} + +bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp) +{ + g_autofree VFIOUserVersion *msgp =3D NULL; + GString *caps; + char *reply; + int size, caplen; + + caps =3D caps_json(); + caplen =3D caps->len + 1; + size =3D sizeof(*msgp) + caplen; + msgp =3D g_malloc0(size); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_VERSION, size, 0); + msgp->major =3D VFIO_USER_MAJOR_VER; + msgp->minor =3D VFIO_USER_MINOR_VER; + memcpy(&msgp->capabilities, caps->str, caplen); + g_string_free(caps, true); + trace_vfio_user_version(msgp->major, msgp->minor, msgp->capabilities); + + vfio_user_send_wait(proxy, &msgp->hdr, NULL, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + error_setg_errno(errp, msgp->hdr.error_reply, "version reply"); + return false; + } + + if (msgp->major !=3D VFIO_USER_MAJOR_VER || + msgp->minor > VFIO_USER_MINOR_VER) { + error_setg(errp, "incompatible server version"); + return false; + } + + reply =3D msgp->capabilities; + if (reply[msgp->hdr.size - sizeof(*msgp) - 1] !=3D '\0') { + error_setg(errp, "corrupt version reply"); + return false; + } + + if (!caps_check(proxy, msgp->minor, reply, errp)) { + return false; + } + + trace_vfio_user_version(msgp->major, msgp->minor, msgp->capabilities); + return true; +} diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 89d6c11c4c..7a3645024f 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -4,3 +4,5 @@ vfio_user_recv_hdr(const char *name, uint16_t id, uint16_t cmd, uint32_t s= ize, uint32_t flags) " (%s) id 0x%x cmd 0x%x size 0x%x flags 0x%x" vfio_user_recv_read(uint16_t id, int read) " id 0x%x read 0x%x" vfio_user_recv_request(uint16_t cmd) " command 0x%x" +vfio_user_send_write(uint16_t id, int wrote) " id 0x%x wrote 0x%x" +vfio_user_version(uint16_t major, uint16_t minor, const char *caps) " majo= r %d minor %d caps: %s" --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340219; cv=pass; d=zohomail.com; s=zohoarc; b=IGjiRMTQNssM6UEpcSgMGTPEok9/LP0oaDuM7yjJciuAqo7RW3mq83RvsNSvKSKtWZ9c3lrRZxFYHx7pDSY7AYU9mpj0BAGeW8WNkai0hhRfxQt31UH8ZVVupMkPKgwlI6T1ilWX6dls6gcYSMypP29kFgXa/eN8fMx6hvRR9pk= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340219; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6RBYi8DRdfFPyblcpIU+rmlgrdPrUkTwUjNnhdmQB9Y=; b=L2UW9H/O323s/W18V1s5AKOjAqBx6PCKVV9Xk3wg9PRgj31oG6lLrmNRGxJpP0B9lhK42o8YX+htgD2Wl7aTu5BOyi/+3o3NdMMbdefljRZuaGzc1+5tfxPYDwm3dW59ZgX16/Y5+X087IIBXb5ALT48Py5BuQ18Wg4zwVcbc+k= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340219723381.80980720785965; Thu, 19 Jun 2025 06:36:59 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFNg-0004GJ-3i; Thu, 19 Jun 2025 09:32:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNe-0004FP-Cu for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:38 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNZ-0008EO-0K for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:38 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55J7OixG013270; Thu, 19 Jun 2025 06:32:30 -0700 Received: from byapr05cu005.outbound.protection.outlook.com (mail-westusazon11020091.outbound.protection.outlook.com [52.101.85.91]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 4798mykwpg-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:29 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:12 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:12 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=6RBYi8DRdfFPyblcpIU+rmlgrdPrUkTwUjNnhdmQB 9Y=; b=ZEiLGuCCMfedubP/tRTLykdVZ4v4c4bFXS5yw5VFSX9S+z9JyNMLdHkAZ 8huYaiPUPCPcs46tni8yXQb97zRFTjxaWZPGO0TMMfq+wsmpx2OiRrGlVXktDngi /sPzfMEm3eQjefYaAEhp/ED7vVwAMX4wrILihzXf9t9wYqgc1qblz5twh+EHaoTZ chsRxTbJNIW8hLAPGwW2UYL6FETraIIb8wT4bUiYDP6/k+WaLd/LZ+7lwbi9Npq2 ASdbGHTl4789tQhi/0Moes4lCDx351YneaGx0cEBZ0+ZzuI5kyDEYDQt8BcjpVmX sFug9gKVzKQTh7BMuqNQ971abtKOA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=uAzCmyZyCeGG14/Si1jni9NNL6Xx62e4vPKjW6h5tjT8Z5BiRwa3mBsq1D5Aaz5FwIyLzHP8Ka7vZOxTebmdJH/T17ElQ5QKfCsbdz6Fm3AT9EgpOhNEsQl5745OxtUt9qNAUe/b9Fh2ONQ1OMR0Wj3IMsAlNz9yzgQi15tGbk+6K1K0wD/Bxa4i+bvMUeYjyF7vpVqg5uzD38trmsktCsLmmVP9B/tP2WvHM5oXID8nYs0f9cRbIZDBW5NSClaPMTO4X9J03z3fFyzRN6FZ9ov/Owmd9Id8XCj+w5AhUuX512Dh+xi6JsTQrdDnUIhT/YI/Vu97C1rYg+f6VMycjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6RBYi8DRdfFPyblcpIU+rmlgrdPrUkTwUjNnhdmQB9Y=; b=tQp1hrPaexD0/nuvWy2E2s76nk8dgePAkR6k+pI9XvAl5bbRaCZkhVpWWbmP+jggwh8rSvl6zyw0wB4MSl3TC6Fey/C85MTCsteQKSLxoc4ltlz7ax8a3KnJDLD9wUofW1VOaqQyFMvKgT0yNSFlItIavWM0SKHsjalV3OvVu+LTXtWsg9pr/iBWkPPN0kxyjVVEgYyNKBd2F4sVdn9gQ+OBCYTIyCpa4Y2205+oc3xhw5me2gTUJgrpc8Q6WyxXYU2mBtVP/C1JWGeNmotRvPqKy2sAmDHGO0g45W0VnTxXHskeNBhxM9IHmKGNSt8KJ5IB+29ySPkNBnTTK3ECag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6RBYi8DRdfFPyblcpIU+rmlgrdPrUkTwUjNnhdmQB9Y=; b=Dw6biigi6awCJt0/tw+hLrzZsLfyiTBg2vFF3wPc80b6EeX7FPmPZo7L5T8vVTWt6br6w/zUiJaWcLJhfVfa2ylQv+rpmC5m/7l3WmytJzqZtS7UCL7j7esdq/x06N7aO9Wznp1tPb9MpkD0Demnnm0ezgQR+IT9jiK1s5qUGmNc+xINced3H0TdY3VCjByfWDFQ0PeCHHBrxKBZ0PMOScrcrcgEP87X/0FQyWRsLGN5DazU6hQlJHsesAURBNUuHZ4qwmlZZo5K7slxFiT0Zxij6oCJcW0iKYMU9Ujz4dn/VsvUxad5+g2BmtYXbxUyFoRkzZCDi3GOPWNa9ctGZw== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 05/19] vfio-user: implement VFIO_USER_DEVICE_GET_INFO Date: Thu, 19 Jun 2025 06:31:39 -0700 Message-ID: <20250619133154.264786-6-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 6e3a0cb5-2bc2-45c9-f051-08ddaf35b20d x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?QTUzKzJIbFp3clR6U2htVG56eW9SdHVSUzhxU2Y4QlA0SGFjRDVBMWVtZlNF?= =?utf-8?B?QWh0V202RytnUE92MkRMaGdxOUJ6aGxrRFpRcG1Wc0VOTnFUTTV4eG8xeE1k?= =?utf-8?B?MzZNOFB6d1FpQlZaK05hc2w0UEV4cnYrQkJTNWlzL1R4aDRYSGN0d3Fyamg5?= =?utf-8?B?K1FraWxNWmphOTMxb1pUdjU5a3h2V3ZvKzl2amxYSEFzMytEY3dyOHNvKzdC?= =?utf-8?B?ZW9Qb3dLWUdUdFhWSS9mdVpoeXFJRUEwdm91N1VJRlVKbDBJNzdoVUs2Sm43?= =?utf-8?B?d3paa2dhbDRNUllDN2QvS3B2NXpsY0dGc1NIOVprWE4wdGRpcWFBWldzTkNu?= =?utf-8?B?NTV4L0l6aVRValdsWGRxczJyenlOb09sMlUxeXFWbzZwcm1xMGxSb2ROUnJs?= =?utf-8?B?NkZUaUREZHN0SkxtanpVNk1Md09pbjRiQXkraitPQjY3eUxpSUs3RnhkOHdO?= =?utf-8?B?c2NuamlrRWJQWnBEby9XMDN0bGY1dS81Q0NwRDA3UFpQU3lnb2t6Ui9MYWd6?= =?utf-8?B?KzUwdERobUxmdjVLbTJKemtqQlZUMkg1ZmxCam00V1E0WExFdU1DQnIxblhh?= =?utf-8?B?cWZvMXNEbStkeURRMnVjeVZ6SzEreS9GNmFyZjA0bVc0bEF1VkhwTExDUitO?= =?utf-8?B?NlZ3bWZhekcwUHdpQlZ1ZlFIUWExZ0NPZ2ZNREl4eDBiOGZCdTNoVStJVnB6?= =?utf-8?B?YTd6MW5rZkxkTVVxOVY2THR6Umh6TWRPYXA1NGtteFFSelFIOVdRa01nRUh5?= =?utf-8?B?dlZyeHMwWWwva0hIOGc2RDllMEpBRzFLdjRhRTZ0Wm41UVU3c01TaU4zT2tV?= =?utf-8?B?QjVQdGFWS0FNZXFsakVrbUdSM2U1aEZlRGhsTHNhYkpESlZHM2JtWExEdVNs?= =?utf-8?B?TFhVd0szVyttaVB1QzVwNkx4TThjQXIyeUlqdmw5TWtsbDBuQjBaSU1TeGZO?= =?utf-8?B?cmdFNzhTaE5KZ1NrV1o3MFVqSHVvQU5ya3V5YmNhWUxmdFFDT083d2FTVWxC?= =?utf-8?B?eHVPcmRBbDVNRVkyY2N1UlZzYXFjemo1VnZrc3FDZWZ5RytIbDNYalhERTIw?= =?utf-8?B?YTkyWDA2NE5GSC9US1dmT0tKZE1WbEx5L2JwUWdZTllOdlhEVjJHZ1IremNp?= =?utf-8?B?Zi9TVzlFT3dMMEY1ZE01dDA2aDI5b0FoMXE1ckhVVmZnUW96SHJjc2ltNENK?= =?utf-8?B?NkZBTEgyRVFnVnlxVFZ3dUszNXFhOHZ0azQ3VHVpSldiaUZ6a2RyVVIyQUpx?= =?utf-8?B?aC9kMHcxWWFDZ3hrTHNrOGFiY0xTaEY4NVQwMVVoMVRmakt5d08wa2VvUEtU?= =?utf-8?B?ajNJcjhRdG5BWVBXUkxQZEcwRjhNYy85YS8vQ2RaVmNlWGw4TXk2cTdyN0lr?= =?utf-8?B?NEVLSmtCdThiak9tcWc4UFRlNmR2WmlEMGgyTUhnSU42RldwUlJmdVdPc253?= =?utf-8?B?bEh5VmxEbVovWDNSaUJHTjM5TmhQUmNneng1VXlYVHAxL204OXA1VnBUaHNt?= =?utf-8?B?OVhEejJhZEhLTnB0UjA5cXhqZFdhT1hjZ1duMjR0SmNZTzFzbUNRMERoenVX?= =?utf-8?B?elN6UUNCcXJuaVlGSkRCejhVS1kwdWxUalJOVW1tR2tMQU51TTZJM0R0Z2dI?= =?utf-8?B?OFMrcmxRemo4dlN4dHdrYkNPcU9SdzJoL3dVOXY2SWF1OGRCOU56dXVvOFE0?= =?utf-8?B?ZXhoWTN3Ui9IcGl6OXordkkvK0puTEFhQlJodm1tRWdhU3pHWmlwcnVCTTdT?= =?utf-8?B?NC8yejRnZXMrRXR5TXA2V3orK2V3Mnh2aUxGZmEzVUxJd0V1bVhuMjJPZVhz?= =?utf-8?B?WlJWT3VTRk1FNGpwcVYvdmVmZER0T2lXblZsZU1McEdqMlVJOXYwTUlqU3ZR?= =?utf-8?B?U2krU2R4akwrS0VzT2k5UnJ2bVhwNDhsdWg2anI2eThwMnNmcjVISnZaWEFp?= =?utf-8?Q?aDsQ66WTWIg=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?azlLTi80S0dHY0QzcVhWWHl5dGxzdE5ZSm5CY3A0b2pjUGpPQnJkbjNTaDBM?= =?utf-8?B?eEIwc05IdmJOQm5IdnJlUWdCN3NlVmVRNnZpV1RiNHF6UUVyREU5cloxU1la?= =?utf-8?B?ck8wdk0zei9pVG9vRHNZc2J4RlNyZFEvSExrYWlacWFSN1JwTW9FQXkyZzU1?= =?utf-8?B?b0EvMEpPaEoxQlJVbTE4akw4S0UrL0Q3eTVoYnA0SkVjWkQzemRXQW5TVWVR?= =?utf-8?B?N2dnTXlyWmlSL3VyOXI4eUNRRWczeHpHb21Mb2l4RXltejVKYitsOEdjM1Yw?= =?utf-8?B?b2g3dGV3dmM1ZmF4Yk0zaVlxOVc2ZzFrdWs3L2xTeFQrZEErUWh1M1hXSnc2?= =?utf-8?B?cXlTdGFacjNlQU1adkhqa2JGck5pYTRjV3YyZ2F3dk9neGVuSGVDTW9jWlFE?= =?utf-8?B?N2hOMlp6T2RsVnR2UW1ZaUhzQlRnQnpTbFIyVzlZZ0Vkd3Y2SDhuUnRyaWQx?= =?utf-8?B?YU1QVGxOWU9uYzJkcVBjdjN3OUovalhvcWJFMG9CS0ZPWi9ER2xKUGZ6MjlZ?= =?utf-8?B?UVpVckVPM1pJNi9sT0RNRURzWldFRGtLUndmWlhlK1d1NE5OQnZHU0REWEdl?= =?utf-8?B?Y0N3VzV3OHJxalhWOU55TGpSSXhGbmk3dkl2NGhMcVhSVHlPNEZCR28xRG50?= =?utf-8?B?MVZ4a3lpWmN4ek9tdW5TeEM1RUJBV25ybkhxcFdDUVc2cDlUbkxFNVE3MytD?= =?utf-8?B?SjRKcnlaS3ZhSjBmb2hpS2o0MTcyYk8vb1lVNnpFMnJmV0lPaDYyVngwMDVx?= =?utf-8?B?S3pUbUwxTk5NN0VLNzh3QmFkZGpOb1I0aDhJMTdxWEQ2bHBZcitGZ1QrS1JF?= =?utf-8?B?VmtiODhQcmJqQlVRN2p1d1FPMzR1SjhLczB4bytsRGsxdlJDZDNMZC8xaXlL?= =?utf-8?B?OXJMUjhIQ1BYMUNpRWJjcWY0NStiUDFjM3pORkFaMHNvc05JOGFmcnpoQk1a?= =?utf-8?B?SGVtdkVsRWtDbnRSUDk3MkVoNWVSTnpmNkk1elJ4YVlVKzZCaUIxQlRPSWg0?= =?utf-8?B?aUpIRGNjWlh1MmhQRXB0bVBqQlRmSVdxTDgwM1VZUFFJUndPU3hmT091REI3?= =?utf-8?B?RWFueW5PcUhlVEw5bGpMZHdXWlBEMjJpcmlveGVIN2VNUzJvOWxTQ2wzRDh6?= =?utf-8?B?L1lFdkl5aDMzOXptamFRWURxYkNteEN4eSsxRlY2cmlqRXZzYlZzbUVYT0dW?= =?utf-8?B?OEZBNnlHMlRXQm1vNko1OWxEa2pFbENJaFQ1VUdHUFQ4L2JoNGE1MmRXU1RD?= =?utf-8?B?NWZkaU1NdU5ValpxVzVNVkQzRTljeDN0ZGZPbUE0K29oR1Jucy93TTZ0cVJC?= =?utf-8?B?dzFMcFFzeENobGkwMm13eGpzN3BTdG9TRVNwR1htMW9mTDdnbHNYcGtrZHpY?= =?utf-8?B?d0w3bWJIdDRqbkRqRkorb2J0cmlvTWcwQU9KbXVYUEZJREpjaWxYMDF4K3Bq?= =?utf-8?B?NlUzd2tmSHhPL1l5c1JOeWsvZEJvUCtFQkNoV1FsVyt1QnR1ZVZ1bnB2Skww?= =?utf-8?B?cnp0RzVKTUVuU3M4a2FuWVh2bkFjTFRjZ09xczlNYkVuZU45MzJJSzVNY0pW?= =?utf-8?B?SzlIM1RnZ2wwajdNa3RUTmpheUJhbDZrVWVJeWw1c3RaWkxxSDRKQ1lTbTM1?= =?utf-8?B?TEdybmR2bm5MUmlnQ3BTMGJqcm5WSnBRUGdnSDBzcTFwZVVSQU9sZzY5OUls?= =?utf-8?B?c2R3MXM5cURMOUhGU1RCUTk3TDZzWmRINUEyTFhBUlFMUzIyNDZUOGlEYzgv?= =?utf-8?B?d1JPeVgyMnowOGFWZDRDRjIxdVh2d2ZUeUxya1RlK1JZZm5aQVJMZXJpSmN4?= =?utf-8?B?NlNiM0FhVlZDWCttaDZnVE1iaENQeEh0dDVpZHdkbURweEtmclpqSDBPZ2Vr?= =?utf-8?B?ZEhyQzc2dTZUQlV1ZVRVS2VoMldBVUd0Sk1HZStMYVV0SjRWUWI2Z3lud2Rm?= =?utf-8?B?TjZZd1BEK2Z3bXA2QURlZmtwRXJTTTFHYW9GNlZkV1liZWtoTTcrK1ViUXQr?= =?utf-8?B?QWxSR3R2emhxd200cUd1WUFZRGRwRGNwank4M2FqcGM2YUVYeG81TG4zWmU0?= =?utf-8?B?TEZrMXBDTjhJMEZPWGpJY0RLUm16TnFZMllBNUMwSkFVRG5WbVhiOHZlcFh6?= =?utf-8?Q?CRCMCOg/NVXsX65NidZKSJIhb?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6e3a0cb5-2bc2-45c9-f051-08ddaf35b20d X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:11.8441 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Zv2w+GTEYaRWX/d3n7srOEo+IbhvtEaV6JXq2o96L2SqJxDoPlvk0gIrh//tACC5/l+BO++a/o0bbC3Xh3IXmQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMiBTYWx0ZWRfX6b8hiI0cFXYU i5ZnkOvYRgndrgafKO3fSxucFdUZp41l/cGD8AcB3E8LljGSUMIJO920wWZUaKDLf5X3TDLoWSA WX2pQSSbGBTiBQCz1E3u2YzauID7fNZHiffbHQ+1/1/gt9AW3q9YaQ3TbQq3ObgTBfIyGCXmJpy AoRFx03KGtVSRkm8hhEBZu+xwcPJAkHzK+ZskPk6o/LpQP1kwgUH4JCHTERAuxiVjQJa1zOLKiR bC7qeSTxEVh5i0g5hFpyA6I5/4cRgPed57lrCeARMN6Mlp4m0Zqu6f/Z1i3ycBah06nn64uqHIp 1/DYiQA2RgNtfFXrf4WHZBT6dJ+5s6vKyHnFrFG0wis9fnYAodOPs9C+VTGYeGQzrBYt9TNDsMU fCRbrTJKsPKutC1BLfLSqZFVOxdtd8dOgyYXDh5l/pWkZ4HQX1VD+0ANoI701hmsJ/33Gw+J X-Proofpoint-ORIG-GUID: UgS-FsQGVyl3Zh9ZlGZi66h60F5TE699 X-Proofpoint-GUID: UgS-FsQGVyl3Zh9ZlGZi66h60F5TE699 X-Authority-Analysis: v=2.4 cv=LIxmQIW9 c=1 sm=1 tr=0 ts=6854116d cx=c_pps a=kkQK6hjpUACXm7ua2zl0Pg==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=9e9T-0rTUHAn1w8HAtsA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340222555116600 Add support for getting basic device information. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/device.h | 20 ++++++++++++++++ hw/vfio-user/protocol.h | 12 ++++++++++ hw/vfio-user/proxy.h | 7 ++++++ hw/vfio-user/container.c | 10 +++++++- hw/vfio-user/device.c | 48 +++++++++++++++++++++++++++++++++++++++ hw/vfio-user/proxy.c | 12 ++++------ hw/vfio-user/meson.build | 1 + hw/vfio-user/trace-events | 1 + 8 files changed, 102 insertions(+), 9 deletions(-) create mode 100644 hw/vfio-user/device.h create mode 100644 hw/vfio-user/device.c diff --git a/hw/vfio-user/device.h b/hw/vfio-user/device.h new file mode 100644 index 0000000000..d6f2b56066 --- /dev/null +++ b/hw/vfio-user/device.h @@ -0,0 +1,20 @@ +#ifndef VFIO_USER_DEVICE_H +#define VFIO_USER_DEVICE_H + +/* + * vfio protocol over a UNIX socket device handling. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "linux/vfio.h" + +#include "hw/vfio-user/proxy.h" + +int vfio_user_get_device_info(VFIOUserProxy *proxy, + struct vfio_device_info *info); + +#endif /* VFIO_USER_DEVICE_H */ diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 2d52d0fb10..e0bba68739 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -112,4 +112,16 @@ typedef struct { */ #define VFIO_USER_DEF_MAX_BITMAP (256 * 1024 * 1024) =20 +/* + * VFIO_USER_DEVICE_GET_INFO + * imported from struct vfio_device_info + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t num_regions; + uint32_t num_irqs; +} VFIOUserDeviceInfo; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 5bc890a0f5..bd5860e9b8 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -12,7 +12,9 @@ #include "io/channel.h" #include "io/channel-socket.h" =20 +#include "qemu/queue.h" #include "qemu/sockets.h" +#include "qemu/thread.h" #include "hw/vfio-user/protocol.h" =20 typedef struct { @@ -96,4 +98,9 @@ void vfio_user_set_handler(VFIODevice *vbasedev, void *reqarg); bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp); =20 +void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags); +void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize); + #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/container.c b/hw/vfio-user/container.c index 2367332177..664cdf044f 100644 --- a/hw/vfio-user/container.c +++ b/hw/vfio-user/container.c @@ -11,6 +11,7 @@ #include "qemu/osdep.h" =20 #include "hw/vfio-user/container.h" +#include "hw/vfio-user/device.h" #include "hw/vfio/vfio-cpr.h" #include "hw/vfio/vfio-device.h" #include "hw/vfio/vfio-listener.h" @@ -140,7 +141,14 @@ static void vfio_user_container_disconnect(VFIOUserCon= tainer *container) static bool vfio_user_device_get(VFIOUserContainer *container, VFIODevice *vbasedev, Error **errp) { - struct vfio_device_info info =3D { 0 }; + struct vfio_device_info info =3D { .argsz =3D sizeof(info) }; + int ret; + + ret =3D vfio_user_get_device_info(vbasedev->proxy, &info); + if (ret) { + error_setg_errno(errp, -ret, "get info failure"); + return ret; + } =20 vbasedev->fd =3D -1; =20 diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c new file mode 100644 index 0000000000..4998019b30 --- /dev/null +++ b/hw/vfio-user/device.c @@ -0,0 +1,48 @@ +/* + * vfio protocol over a UNIX socket device handling. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qemu/error-report.h" + +#include "hw/vfio-user/device.h" +#include "hw/vfio-user/trace.h" + +/* + * These are to defend against a malign server trying + * to force us to run out of memory. + */ +#define VFIO_USER_MAX_REGIONS 100 +#define VFIO_USER_MAX_IRQS 50 + +int vfio_user_get_device_info(VFIOUserProxy *proxy, + struct vfio_device_info *info) +{ + VFIOUserDeviceInfo msg; + uint32_t argsz =3D sizeof(msg) - sizeof(msg.hdr); + + memset(&msg, 0, sizeof(msg)); + vfio_user_request_msg(&msg.hdr, VFIO_USER_DEVICE_GET_INFO, sizeof(msg)= , 0); + msg.argsz =3D argsz; + + vfio_user_send_wait(proxy, &msg.hdr, NULL, 0); + if (msg.hdr.flags & VFIO_USER_ERROR) { + return -msg.hdr.error_reply; + } + trace_vfio_user_get_info(msg.num_regions, msg.num_irqs); + + memcpy(info, &msg.argsz, argsz); + + /* defend against a malicious server */ + if (info->num_regions > VFIO_USER_MAX_REGIONS || + info->num_irqs > VFIO_USER_MAX_IRQS) { + error_printf("%s: invalid reply\n", __func__); + return -EINVAL; + } + + return 0; +} diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 0887d0aa1a..c6b6628505 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -39,10 +39,6 @@ static void vfio_user_cb(void *opaque); =20 static void vfio_user_request(void *opaque); static int vfio_user_send_queued(VFIOUserProxy *proxy, VFIOUserMsg *msg); -static void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, - VFIOUserFDs *fds, int rsize); -static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, - uint32_t size, uint32_t flags); =20 static inline void vfio_user_set_error(VFIOUserHdr *hdr, uint32_t err) { @@ -610,8 +606,8 @@ static int vfio_user_send_queued(VFIOUserProxy *proxy, = VFIOUserMsg *msg) return 0; } =20 -static void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, - VFIOUserFDs *fds, int rsize) +void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize) { VFIOUserMsg *msg; int ret; @@ -782,8 +778,8 @@ void vfio_user_disconnect(VFIOUserProxy *proxy) g_free(proxy); } =20 -static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, - uint32_t size, uint32_t flags) +void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags) { static uint16_t next_id; =20 diff --git a/hw/vfio-user/meson.build b/hw/vfio-user/meson.build index cb958d0aa3..54af0da585 100644 --- a/hw/vfio-user/meson.build +++ b/hw/vfio-user/meson.build @@ -1,6 +1,7 @@ vfio_user_ss =3D ss.source_set() vfio_user_ss.add(files( 'container.c', + 'device.c', 'pci.c', 'proxy.c', )) diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 7a3645024f..6b06a3ed82 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -6,3 +6,4 @@ vfio_user_recv_read(uint16_t id, int read) " id 0x%x read 0= x%x" vfio_user_recv_request(uint16_t cmd) " command 0x%x" vfio_user_send_write(uint16_t id, int wrote) " id 0x%x wrote 0x%x" vfio_user_version(uint16_t major, uint16_t minor, const char *caps) " majo= r %d minor %d caps: %s" +vfio_user_get_info(uint32_t nregions, uint32_t nirqs) " #regions %d #irqs = %d" --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340170; cv=pass; d=zohomail.com; s=zohoarc; b=RLA/qvFMLwtOl2Fm6SfCL7aMQgtgDd9lg8hfWJoMN5wz82RPGuoss7XWE6Sr6Pn3yRNfFpjS72DFQPaATq1Mcc/iteuSy7izpuIm+p7OFKGtXPJeGZ+FHoripjPiZMvgJmSKDQ+1Etc0Um92KCdzN7B+8OoeziMOxEcp3vG/HMY= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340170; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Ove+EINyRJPK8DL5E3/dHKlBUNKPuLS4ob73rl4KR68=; b=O+vDSNgI4lx04MUdM6umBahtpRuGMJNMeoht/wlQIUiq6G3+rZS62p/B/7C55638SEYF2XSKdwFhQiYPNzjT0t2/nnxdmAbnhV13FIOvQEFGBUNYgrBatpJoN0HIx+svVI0EU0MA2sNEzY9CelqCPG62KexKTidCef1p0XMWVnw= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340170773931.9319919129603; Thu, 19 Jun 2025 06:36:10 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFNb-0004Dg-0u; Thu, 19 Jun 2025 09:32:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNY-0004Ca-5v for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:32 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNV-0008DQ-Lh for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:31 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55J8A05v013187; Thu, 19 Jun 2025 06:32:27 -0700 Received: from byapr05cu005.outbound.protection.outlook.com (mail-westusazon11020121.outbound.protection.outlook.com [52.101.85.121]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 4798mykwq7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:26 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:14 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:14 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=Ove+EINyRJPK8DL5E3/dHKlBUNKPuLS4ob73rl4KR 68=; b=Cx0ZRk+2hdd0Yfij+mJRoxEREGkxQWJUxHe4Q70BYXnUdojebrsOH24IN LWLIjmBfP/2MbMjP3xtBHtnqTQy8qomqRbZVPVV/JeBTGU2o+5Zb8T6xSOGArV69 CTskoxIuwGqE+57ZYFgzzVawA9aNKdNcWwJL0CqpvNWnPjEJEMS9NvS+0cT61jwV a1dckDHjriqzgkLwJh/lMwKZ3+WburS4c3EVDO03nIikAtsNiQZMs5L15TthTP7n QGMGw47ig03Ly1Nmqw6XYF35ghROrKUe2PdsJWYI9yncKHbHpKEkxRre5wT/635/ g4hn7HRTh3RrfVAohaTNoGp/HFYMw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Z7TkYc5Uon0se87V1xVTdARI2Qowv/FraEudUQAG/AW764tuHcq8loXLNuU9OlKD5xpVga1ydTbZoIONdudyao4djoeyRgUhoJ3IU9kp0mKFevNFvyZUZXtVM00drcZdAuRgm3jlkoMEJ31Cilc1DZl89XghOSnJ0j+qugHgPEoGONaPxRtlDUHmNle4H9ASaXBJNE6SwFjE6RiqmmdT6ZhdRPjIBPn8xxflFUvatf4rvL63Xh7xdHlgLN2YJWrfkaCo2RYLZz1fTZFp3n/MUUB/naSDe2wnwc9ZOkdSI6M6Jw0qskl7DwOOnBqQiMf3icv0su/THerkVYTLmEB45w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ove+EINyRJPK8DL5E3/dHKlBUNKPuLS4ob73rl4KR68=; b=gj7Xs60Lo74vYlSa/CcJ1xLirAmw7yk4kLTPNmHhJi+lioLeZYgGR1yqK7FFPdsOfOPAOMSN8JGuyjHRn5oBxR5bgmme6adFiir5E6JDwFWwBeB0wAE+Bl5aY+4kun2o7yJdMS+3zxou2GJ6gtUAN1mixHf2kdp4GkT0z+tYcWGOGnTYZAaVALf+Lx4iZeTqJwus8fpvcwA9j8+LUtUyZi3o8IpImZwIzDQ06UITMBvT91jt5POm2PpuMS65Rnln9lqWjoRIQKsZQWmjaQMY8Sa70NzhPy+38UkxSRkVoAMr3w6/iDKJefBX8O4LPGtq+6juZaAdZlm0AP1jSS8EiQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ove+EINyRJPK8DL5E3/dHKlBUNKPuLS4ob73rl4KR68=; b=AdHdRdyJSBQWhEvaRSriV8DIIqormD9rTGf5CQUeyq+SB0kzSfnUbwh9PWMP46zt0q/flLnW42k84gIdl73kP9ch96a47KR4WQvGC02mQc8c434Vgi7oorOF+uhNEv9ImhUcJlsqEdkaGRJUPDSEyP0leviUQDFElNDkGcPbIXl9rFGdXhBkr0hN/75bH+mdvNKeiwSKFCIS1FBCwVgK9EMB9OSvFmmw/S15OZiL/wMokM61FQdvpvbDzyDdok/Ldi3WDvLIBaRO/wCbbCuj0+Xj+0rwahEu1ZzL39EBwmcQvDqN2mh7qrIaZchxS5mfVnJAt6TjZ4aryl6z/jsF9A== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 06/19] vfio-user: implement VFIO_USER_DEVICE_GET_REGION_INFO Date: Thu, 19 Jun 2025 06:31:40 -0700 Message-ID: <20250619133154.264786-7-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 64e9abc9-83e7-4327-6b12-08ddaf35b3de x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?xabnLLMAHTWR4xYLjH21KhQwmkpnewbCRyjxoOWMvbJ6nVnA3sNhx5DGlGFc?= =?us-ascii?Q?EY9PC7dBuljQmmfDvYgteEQqDQbpMFj/AGn2Z0tudH3KsHMa8UxPPY/D1rKI?= =?us-ascii?Q?DcuskjJL4CPToV0dQU2ix42JJb5JMD2QSqgQG22Q73P3t4y0TjsbuCoV5prO?= =?us-ascii?Q?yEnF1bXHYOcbpjSTN/gufrMHRuckGXBvfW8XGDRipwoo+X0R5UVSV0p9uqDy?= =?us-ascii?Q?+iykXELWJ6uclku3NMxfyi0v/2Bma/pdR1t6nIyCZPiLyzfs65C/ZrNNKyVd?= =?us-ascii?Q?xBQkdzh6GWkr88OryZZ7Ca5HHw9IHDTj+9DhxYiC0p4YDFbBzOcUqmvS0PHQ?= =?us-ascii?Q?rTYq2eO7ZXHhWT0XX5PonK6lGX6TxPkhijf5bGaedKtv3Z6cHP/eR+ObTmgl?= =?us-ascii?Q?l37xwa4k6Rs73ZHvyoPhuEC00MEJyHY9vDYUKLl4oIgkCsTvDhWW8G4sool3?= =?us-ascii?Q?8mBEYu1TYqoz2h5M6rMrLJNqGTqFC9Owkd0/PkVOSHf4PIo9n1fcMzuHPlau?= =?us-ascii?Q?MjUgjuzMOtH3o2Gf7j8q5K0Ugv/myasZwtd+FgpNLR209mDyKYa3go0IS0bj?= =?us-ascii?Q?nVDlwT5AXsJ8urobenfl0YpSyTnLyWuhXCr6PPu+XVj1MkTQtozI2xOTTPT0?= =?us-ascii?Q?ow6ei6TH9RRAAITcsgDdrlYMSNG3aqs9QXmIYPYm42GVh5t/4xMYNORB6SJ7?= =?us-ascii?Q?SksCe5ubPkQyVAgzSxeeHoRxbT1ULdVKusCmF/WkGBH6xNWm4oo2fk1wFvj2?= =?us-ascii?Q?m1kzVDGleGEY8tKK3ryYb1AhilBrX6xq2DAjn1nHfwdspqwHQkntzilHnsrx?= =?us-ascii?Q?k3tEVOC8Gh5Fz2GNE1GTPcW+qe4lypSq26yjZAIlwVWwktQM9mbotuGEvNI9?= =?us-ascii?Q?st5jSvfhx3rA/Yty83wEWw3Dw27nwV9PxDL1eCpJ8NWNBeic3D8tdylNyMzL?= =?us-ascii?Q?8Um5HwdDLiE5efJxPRvYgBMBLaIGI2d8bvC6c5+Ohr0lt13OOaF1cC3PQLQt?= =?us-ascii?Q?nsODyWRGRR84cKhSYh6tk3ZG6QOrFF3TrX1pLHO/LbO2e6kIb1NAJP4Z/1Nv?= =?us-ascii?Q?J+TwtkhcKAWXkhmzx3leC+bT5y/eYlQZA31gQItPOgsL/ffgsRiDUJoWDRRn?= =?us-ascii?Q?kV8vOTqVMbrSgebGOaKU+U9vJFCwfYN1Ld9zPe8mzz08fMyX3QIkgbtAoiSA?= =?us-ascii?Q?JgMcsOdM7Ux1JAyshTBSHV4PHgs83uDbSgleyI6podk8KLfEfVxwdgVRcxhN?= =?us-ascii?Q?pMFjBddWh6xVUDdBEDDoltY07mWmgiQfs1RODBelXVs7ggu5qTJt4dJu/wSI?= =?us-ascii?Q?DpyvEfstvV070jgZaNPdAA0ezl2guRotumRuNEnJ74egg9EmjyHjl266U9AG?= =?us-ascii?Q?hJidTK72LDkIV2UVny/aVFMSXisyzz3OcUid+cOIBs+XAFWdAw=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?TufNtHKyw8HFFdE9cJ+PFr3l6aETt+LhaOVu/7lZXAi6NGsQCFiyQf3wGQgH?= =?us-ascii?Q?G9HmgVoPBRi4ml0BCM0uK6OHzr81ipPEjK0cpeQlGTVioHvIlsJsSU46F4JA?= =?us-ascii?Q?nTpG6Er9vUkwE/ipvWb46m0Dld4eGgbctdvzbEtj/tQsttcpSXJZduiufs7K?= =?us-ascii?Q?cCWTzYLc0xjwxorT8jvR6R6akiRzGEam9oiGEzPuuCwb5AXseQ3buKockI8b?= =?us-ascii?Q?usuGX2GMoLvspC8AfgVpQEG1BlpO6PHf4F92fDMilDyKP97k1m3b88PKmzOY?= =?us-ascii?Q?gVRZNv5Og65+WZRwNbe2t96s3HcKJeGzY7rt5+MJjXEC1uxFuhCUF3XI+xW+?= =?us-ascii?Q?Y/qI81uL8lX4e/j7z6baK7xfes2wkyY+3if7HDM+kJj3sc0ohQI1sQaS2SdV?= =?us-ascii?Q?K1eNVQkdZOVDyCATUDDS7qb2JmycALRa0HHpAJfI4qDesluLfADOWgq+dPx4?= =?us-ascii?Q?gEnH9S5heKQqsx30dAWkv1oFDFAqDlseD2aKHsf43AZz4HIahOQtOnuG5y/A?= =?us-ascii?Q?LOB4WJ9vjXy5B7k46LceAw+1rUmToywaJHaRtAgFdCO/4l352TE9iI/2lxhM?= =?us-ascii?Q?hQqcWTCBfvIKlnUsh85W0k/M1ReFMhEf9VfRr6ku9xK6298Lm63zbIZS1HTA?= =?us-ascii?Q?hMvLbKS4aLfU3tyLZTEl+xu2BdTJLP3xkAadoZhOBSmS7Tikrt7nS+kbaD68?= =?us-ascii?Q?Dv5Y77Z2ep3gVfILFZPe+uZe1qOq/HpCEsW5qj/Ei14Bk/hFN/yYTJnyxlS/?= =?us-ascii?Q?EtSWjVEXLLFoxjmPMZWY1teSI7pdgRO1nxqVAHLfugrimKnEBbagu0fliFBG?= =?us-ascii?Q?RJ8oNV0JV8CUvv40ZHvvf6BvUsD212ZKlzQVlzOej6EhIjUqREJ8rZ38BAM9?= =?us-ascii?Q?DdEXgdzw0h+jkNZKy15WlJniy8wCBH3ZDCryD4YZPeT6IKd66LzU39WexcCj?= =?us-ascii?Q?KRKo2UFmiKz9bm08s/Y0uodsUWD4ovwffvr/D+HAxYvMOYdoh6aDdlj6mZj6?= =?us-ascii?Q?pVcXNnMxxZZclrbeKf3csNmPHro64ViE4BmzeCnWjAk3Gm75A2Jspq+S6CEz?= =?us-ascii?Q?eul5BDEu0cm5SmrdmZPsibgaBuXaJqhr1cAlIHpz+CIDFhPU0oQ8YHX52SEB?= =?us-ascii?Q?IL/B4GqboKP2Kup8euW7uHidGd+rxWoY9IjLHy+VCg7bXwyzwX2hXyjHpsRt?= =?us-ascii?Q?y7lBK/A7wR6nohlfeGoUFgtfgyu919rqpNWVoTBnEsysdi6L63XOqUBTbfA7?= =?us-ascii?Q?eM5nVu+c1GF/aC6W544PyBJ1L1kx8/wpd+tEJ1MFGmaVRZg3HoY983u6p0qX?= =?us-ascii?Q?mBZACXatGw7m8I4GfL9QzoDryDDdumG8NoQjBKn7v24MOL8DpRJUUgpwVosl?= =?us-ascii?Q?ciIwPRGf3WFAX5q2atKkHm23+x4FqEudRwPQ+wwIKSdQNKcY2H9v4clUtajI?= =?us-ascii?Q?+I50y8Ej+LCUSFNYzOWB/JZXadY9n7TGOYQsdlD8dgLNo/2e+nAx/fm1RpK4?= =?us-ascii?Q?4cLn48p+K0xMS4yggQ8iEocBunC6Cq56wglyiecRMV9ynQTgpgE1NLxzkh7o?= =?us-ascii?Q?yPlazyHnNFfaR0DAhy2VDemBEQSpnCcM+sdNiXVL?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 64e9abc9-83e7-4327-6b12-08ddaf35b3de X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:14.5428 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ic/Ja+1IheD0dl6oSWT5xcgSqSUX0O10Xg5BI1bksvCAixEkYFGg80XMSZLOfZqhDMER+H/ah1fY8jdOlIEmWg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMiBTYWx0ZWRfX71qei9q4TThA 87yv6GZ+KTP1AuEVq13vJ3BGp3RYjZMPvUVv9rCeb2Hx8HGIGvYX6g8gMadQAqHMgi02DiqaNdl NTdCV3cPas7SsLzx9Jv/fqcq/SWwHp33I28koHNQhwCyxxxW/Axljj/Xqjv+KEP+nITO53ie4Mb 1PzR+DMMLy/zavJ4jdyZbde2jKv/Zxuz6HdAbphjSk7xZj9Lfzk3f4OltCTOP7Ufw+YdvAfnJxK E2C8XcdLTSLVnUOIwsvQNu010c32PJzR7GbH4IGNBaeuRmwAEF61tPuK/Pjb863bxQvgnHSf9uR VyP0IfgFdfTznEuKA0W0daKqFvMGKf9ZvciAO+Z83UuOfGWCeXzbdTyuLemD/Py9RbVxOQRwrQR 91j4NpIYrxLUGdcIn7SBrHqbNvGOq5d7vbvyeb0Y+7gyCnGeCICQwUYc1Yl+pR3MZ4w55WdA X-Proofpoint-ORIG-GUID: TdtxyHH0sX8Rzyh2uQ9V72yjW6y2yhnC X-Proofpoint-GUID: TdtxyHH0sX8Rzyh2uQ9V72yjW6y2yhnC X-Authority-Analysis: v=2.4 cv=LIxmQIW9 c=1 sm=1 tr=0 ts=6854116a cx=c_pps a=Yi5XoY1Pvk1/bV4Hj+lk7Q==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=u1fsugoMv0NfKDGiTLIA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340171883116600 Content-Type: text/plain; charset="utf-8" Add support for getting region info for vfio-user. As vfio-user has one fd per region, enable ->use_region_fds. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/device.h | 2 ++ hw/vfio-user/protocol.h | 14 ++++++++ hw/vfio-user/proxy.h | 1 + hw/vfio-user/device.c | 68 +++++++++++++++++++++++++++++++++++++++ hw/vfio-user/pci.c | 11 +++++++ hw/vfio-user/trace-events | 1 + 6 files changed, 97 insertions(+) diff --git a/hw/vfio-user/device.h b/hw/vfio-user/device.h index d6f2b56066..5fc226fd58 100644 --- a/hw/vfio-user/device.h +++ b/hw/vfio-user/device.h @@ -17,4 +17,6 @@ int vfio_user_get_device_info(VFIOUserProxy *proxy, struct vfio_device_info *info); =20 +extern VFIODeviceIOOps vfio_user_device_io_ops_sock; + #endif /* VFIO_USER_DEVICE_H */ diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index e0bba68739..db88f5fcb1 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -124,4 +124,18 @@ typedef struct { uint32_t num_irqs; } VFIOUserDeviceInfo; =20 +/* + * VFIO_USER_DEVICE_GET_REGION_INFO + * imported from struct vfio_region_info + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t index; + uint32_t cap_offset; + uint64_t size; + uint64_t offset; +} VFIOUserRegionInfo; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index bd5860e9b8..e5ac558a65 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -15,6 +15,7 @@ #include "qemu/queue.h" #include "qemu/sockets.h" #include "qemu/thread.h" +#include "hw/vfio/vfio-device.h" #include "hw/vfio-user/protocol.h" =20 typedef struct { diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index 4998019b30..3d39b9b98b 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -46,3 +46,71 @@ int vfio_user_get_device_info(VFIOUserProxy *proxy, =20 return 0; } + +static int vfio_user_get_region_info(VFIOUserProxy *proxy, + struct vfio_region_info *info, + VFIOUserFDs *fds) +{ + g_autofree VFIOUserRegionInfo *msgp =3D NULL; + uint32_t size; + + /* data returned can be larger than vfio_region_info */ + if (info->argsz < sizeof(*info)) { + error_printf("vfio_user_get_region_info argsz too small\n"); + return -E2BIG; + } + if (fds !=3D NULL && fds->send_fds !=3D 0) { + error_printf("vfio_user_get_region_info can't send FDs\n"); + return -EINVAL; + } + + size =3D info->argsz + sizeof(VFIOUserHdr); + msgp =3D g_malloc0(size); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DEVICE_GET_REGION_INFO, + sizeof(*msgp), 0); + msgp->argsz =3D info->argsz; + msgp->index =3D info->index; + + vfio_user_send_wait(proxy, &msgp->hdr, fds, size); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + trace_vfio_user_get_region_info(msgp->index, msgp->flags, msgp->size); + + memcpy(info, &msgp->argsz, info->argsz); + return 0; +} + + +static int vfio_user_device_io_get_region_info(VFIODevice *vbasedev, + struct vfio_region_info *in= fo, + int *fd) +{ + VFIOUserFDs fds =3D { 0, 1, fd}; + int ret; + + if (info->index > vbasedev->num_regions) { + return -EINVAL; + } + + ret =3D vfio_user_get_region_info(vbasedev->proxy, info, &fds); + if (ret) { + return ret; + } + + /* cap_offset in valid area */ + if ((info->flags & VFIO_REGION_INFO_FLAG_CAPS) && + (info->cap_offset < sizeof(*info) || info->cap_offset > info->args= z)) { + return -EINVAL; + } + + return 0; +} + +/* + * Socket-based io_ops + */ +VFIODeviceIOOps vfio_user_device_io_ops_sock =3D { + .get_region_info =3D vfio_user_device_io_get_region_info, +}; diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 61f525cf4a..d704e3d390 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -12,6 +12,7 @@ =20 #include "hw/qdev-properties.h" #include "hw/vfio/pci.h" +#include "hw/vfio-user/device.h" #include "hw/vfio-user/proxy.h" =20 #define TYPE_VFIO_USER_PCI "vfio-user-pci" @@ -103,11 +104,21 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Er= ror **errp) goto error; } =20 + /* + * Use socket-based device I/O instead of vfio kernel driver. + */ + vbasedev->io_ops =3D &vfio_user_device_io_ops_sock; + /* * vfio-user devices are effectively mdevs (don't use a host iommu). */ vbasedev->mdev =3D true; =20 + /* + * Enable per-region fds. + */ + vbasedev->use_region_fds =3D true; + as =3D pci_device_iommu_address_space(pdev); if (!vfio_device_attach_by_iommu_type(TYPE_VFIO_IOMMU_USER, vbasedev->name, vbasedev, diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 6b06a3ed82..1860430301 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -7,3 +7,4 @@ vfio_user_recv_request(uint16_t cmd) " command 0x%x" vfio_user_send_write(uint16_t id, int wrote) " id 0x%x wrote 0x%x" vfio_user_version(uint16_t major, uint16_t minor, const char *caps) " majo= r %d minor %d caps: %s" vfio_user_get_info(uint32_t nregions, uint32_t nirqs) " #regions %d #irqs = %d" +vfio_user_get_region_info(uint32_t index, uint32_t flags, uint64_t size) "= index %d flags 0x%x size 0x%"PRIx64 --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340032; cv=pass; d=zohomail.com; s=zohoarc; b=U16hSAqvaugJOHvH4bY9pbFJzQH/digGT18s5Tt/o9rSiZbRbzjzTiew6s3NbomoRMJJQg2gPSWYBRzN+0J+5xm5eOhonrlgnP0/OPb0kvmH61IqF45uWqQ9RMezcxCviLwnAn6RfZPZ/8lG+SvFYCdA/+fHOTAeZVUOlzVbEKo= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340032; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=vg7/nd9o+jHifUuxhL0cBIPbJzHOAyd4RzxP+M50tz4=; b=jc3P5GPLJcfD2dbroC+SRw+V3C0f5buLOGmcBRUGDkc5tCh6ZVB8BH7KEi9r6T7OR9NQub3tI0qsT9V8l3Hp6E9BTOoYw/M70eaTIxqRgh5bQ80N6bPpLflQIw5chvfOQJv3BIZ0NTYKyed8sDq4ZDFZMm9zGpmQQ4X38mEkX+4= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340027398368.3490083593973; Thu, 19 Jun 2025 06:33:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFNd-0004Et-Re; Thu, 19 Jun 2025 09:32:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNc-0004E8-CK for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:36 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNY-0008Dh-26 for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:35 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55J8A05w013187; Thu, 19 Jun 2025 06:32:28 -0700 Received: from byapr05cu005.outbound.protection.outlook.com (mail-westusazon11020121.outbound.protection.outlook.com [52.101.85.121]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 4798mykwq7-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:28 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:17 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:16 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=vg7/nd9o+jHifUuxhL0cBIPbJzHOAyd4RzxP+M50t z4=; b=bNwDovPzWKZzLboPv9iwhW5AoqJuM4H2v67oPNHhGB2rbVXTTGqc2pEQb Re7EAjTn4WlQ+tu9XCTaWH4/EcJwczvjJ1Zd5TFht/q1gerStXfwVDiSheGTMItW +v1YelykUgoBvJTHK3guNGr+8ZN34eQoMvg+T4obUEbK0r5sgHaJCogs7Ps95YCq oononmLMWBZH7xjF1aSzEHFbDmvfGLORi5gj8jvO+MYlqO+HbPXR2ZKTjZ3wsBhv gqBhfn6IbVkrJ8IY1sPHGsTjgRdhmgFSwJIGmfqcirubK4MKkscg6iaVawpqNWXn tOSZT4SK5h81GmVtKvQ6roEsMbU+A== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=g/ju+IIBpNqtuUhqKhCccS37QEycbYYHZHiIBCqukqFDVO4GnR4ZDmf/5+GKTKlKC/Drbz4739sNPybDgDBh3cpR6fYDdP3BZZ1VE5qmyXbqxFlwz1DBlGCQT7023eA5EZmd/69gPLLouKt+nuVxziOVrtQtytVSRgQEpP8fOfipaXn6bKt5ZqoMq19ysXwSmapW2eDpDwYrErrS5yGGiXxWQ26aTj7fTqacOgowWOhfxNYi1gBbj9s3rJ7otabGG5LqB0hjGWbQqSZ6RgUvOwn/5ajNA5kKgvpL17bdVbtWGFexO9cJdwtaYK+0fzez+dr7pGf5wWMlFMBkrzHKMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vg7/nd9o+jHifUuxhL0cBIPbJzHOAyd4RzxP+M50tz4=; b=RPgMLU2G2Xb+fvO7h56s0XRmVKnCrhTNB/31WK2ZaGe/i9OxSPsYwScDKkAeHqaYSVE+QNSjm0KF6R+LnKykaljkk+4And5CsJ+0pg7+Da53u6B4yXYSUkOSz2EnM7e3MAyd//M4MB49tYQKhvM3TD+S1jfolec3ju+nY1LAOz23GjRmc/o01o7PxlWzjbaEh0Ku0Gkv4YD6ONngN+2VZFc+N2/c741njj2KII3i5Wu6EM9Q1b7b95Eq3ASNCVTOHvNW53Q7VVhvLamnahyynenTaOETltnsqao85ZIouNQ/qnDW0RyLBaBcQ6oJQjOJAW4CmGNpcmJ1sIzW56cmqQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vg7/nd9o+jHifUuxhL0cBIPbJzHOAyd4RzxP+M50tz4=; b=XsriCabbni5B+kdIA9RXG1HpLGi4prcd98hb9giY3vECmbrbz6yupeDip+vsl6oCRS3mrKCNNq2SC/8MAjj/fFGDLZwl+A6iu4wp8A73hbeYvz1MKqKP63ceLY3BxYfsjxudtwHXV7Pj58ejN634NBf/4dm83IbQsrFejQpv2JP8i5cAFE6TEs1Y/U94LAzbaS/X4hbGHIYhzhwPGSPqxU2g+51fH4boz8wCq1WSkj6lRlC9XrGNR+b8m3lxD8DunGYKubNRuPDI27Waamx3VV3gKqF7XaBysRkrzPzsuyfgOADgiR6lkgeQVcqF1RH1A/pk7O9Tf/4u9hDHKHT0vg== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 07/19] vfio-user: implement VFIO_USER_REGION_READ/WRITE Date: Thu, 19 Jun 2025 06:31:41 -0700 Message-ID: <20250619133154.264786-8-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: b109b47f-7746-45f4-f53f-08ddaf35b54e x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?00t3UYdXEqpPOC486DlmTQ698ZvWapb/EOmwt2fzh/p0u9/O/+EbwpuT4x4y?= =?us-ascii?Q?3/cBHxTQN503rspl8Gyd4ZKTtqM52jFfgeR6koZqIcjKXoT5VpMWDzHLqduT?= =?us-ascii?Q?pTjUDHHqCxtQ6/oydBphncXHLmA+NlUPbKSB8q30gjuuKP934TjKkriiQRVO?= =?us-ascii?Q?b6qqVg7D2Oh0eMBOodN5szHP6FvypX4dI8FrppAdb+Zq2WMPUVlRb7hArSXN?= =?us-ascii?Q?3KLEvGW+TC+0Gr7BOv6E+7IKACrPXOvkhyjZRA5Y58d7zg/cjRZEM5/6UbA9?= =?us-ascii?Q?nFz2XVw+bQi5GjjE8TIX3bRyQFOHbAdM3AmnMxWmQai6oIYx+wHbB4w5n0Vn?= =?us-ascii?Q?7Jw2Rp7BT7MG5MtUlMG+E0vbcwSosv7WpupZ6x+pF/hngj0eAbOjQNHXIT1o?= =?us-ascii?Q?PU1YII3qlj13Xr/KutMWzc9bsAOBpbYM0AFkDZEC5A+ss/AmOxbo3RwPAFAc?= =?us-ascii?Q?CGvhRqpjYS7p43vGsrrRAsm84vGcKNjTOsxApLnZZZgErakINKovh1884g6F?= =?us-ascii?Q?Pb6j0bFj2CrSxrtOh0j5OZN3IQsgRqSPA54LtygUDK/u5gCbodO4bSbtTCrZ?= =?us-ascii?Q?YbS/SnTNL1xSJut9x3fU4gQZwR2/h6buC1G1pm8iyjCqXitGBHcoZFrvmzH/?= =?us-ascii?Q?8d7HIMA6FU1N0V6pXGnFy3PpERKdImQJOHazvicGCje8k3ydZKITOHJWhthZ?= =?us-ascii?Q?xrbgSOrZZQng+oGxPT1Tv0H14VJSCTUnNN3lW6m8D5yfzuinD+Hu0mE8BtA3?= =?us-ascii?Q?2BqhpKEvUopAk23cRSAg2p/yFXL2+PI8bwJMEqdnmFpoEnmM7SLzor0bxhYj?= =?us-ascii?Q?6txfdcZv1cRq75rzv+3n1TIEw6Wyf1+NP2yv4bzcOXS7z6TOfc93Gxa9QcNe?= =?us-ascii?Q?eRnrG9Aeemg3FFyO7LA2xQ60DKt6C1+DetRCwyYW/x6EVSnz1rrgDllyrz+m?= =?us-ascii?Q?w9LWaUBFaqAphs4UIw+02AiSfKOMTJavfaHDzETefeiJbKSbJwdkUkJ4AybS?= =?us-ascii?Q?4PRrm+X3G25N4SYg9Fv4+Gpjtk/u6aM01i3g8DZfB6ugNVH3Rv2cCxX01HZn?= =?us-ascii?Q?FHevp9IJge3ovRhYNR5mkZq4dKE1bcs6P6RZoZexxLcSlw2RnhEapjhFVnLe?= =?us-ascii?Q?Vt0w3LzLr9SAvbtPItOaBal7OMzjr5zex6fqR0rkmuDmCSHDwQa+ih+xmjRT?= =?us-ascii?Q?6qfWAjOG5nEgBuIj+WHrGeFlEa2BqGNX/zxqM+l0Y2YY8O3ZAkGlGYgnoB8/?= =?us-ascii?Q?NzsyoKwwGE/T/a6oo6HUR0o3JV3FK7RwZ9BieLe9otUeYEQMtiqQlTZ+U7qg?= =?us-ascii?Q?Wxu3ZhGhDSR9novpFAgF/PmF97AtVDz2Km0h/iUZr4NDZGV9CflVlaIg9OEE?= =?us-ascii?Q?xbY2PoUVoNxDoVDcPqXtYy/5WNUFSwaWRIDgzWjmVIabr3nmwoMUePzPmVP+?= =?us-ascii?Q?j7gbutXDfmo=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?6Qi7rcqwXtXXvG1L8DvQTHhJvf98zDV3JyA9g2bXe3QwJ3kwuhlsLqthofk/?= =?us-ascii?Q?E5kGOUdsdQHsAr2Lyyf9YICbm3q0OOtq7GSCfAVZ7OaDO+3MfP4s2Du3MCrR?= =?us-ascii?Q?1jiYJ9L6mXN4aDNDfYUfu10Bd4s4RTfJztHu7l/sKD6VwJe9I3eAG9Y2d68Z?= =?us-ascii?Q?YkCYUbG0Ka3e7xMPTJjwuuZO65LwCI2jYY0rQ8TT4Mp7BaRzOnaiu67e28oC?= =?us-ascii?Q?+VD4Jhn+VwPDQ6qMATplaBbdmkDUii0rYJ9Bty8pyY6eCkQF7H79BqsCJiqh?= =?us-ascii?Q?I6hkls9J4hWLrydhmUmCRoUQRE/y0wSEBOKJm8Lmb8wK2oO0WDTaZOAMFywl?= =?us-ascii?Q?pAgs6pBRsWxv9xtVLG7FfWDcqg8pHEbvxx7AfGARVSq5cyR2v5x2TnQmTBpZ?= =?us-ascii?Q?PLKGq92nVO+8okhe9/49EXnHFPdO1vT4b7mQHUY+AMPkKcoyCh6p2lrAdlKK?= =?us-ascii?Q?ZovBVgKuJbMGpPrwCmHqkj4lk6TXYNkX11UnSCLY4eJ7gCcq7b/yTjh7CQIg?= =?us-ascii?Q?bv2623ZVG+y7GboJXFYQxMm3amEcas3+TBkIpfi4sQcnwqO8mLK9SX/0yYVN?= =?us-ascii?Q?clUhmJ6SXVevGl6pvF4CtsnygAjeP6FWkBjc77+q96Kaj/z5j683cKt1H5zD?= =?us-ascii?Q?e64Qy8nSnYy3LdIYaNGpVXPIzeEfU0Yl2Hmu+R1mVPnJQv+rqVdAW5d4/fjg?= =?us-ascii?Q?/MJl1RC0AnDD+0uXBQhkzyqDfhtMlt/Q9Kue7CKX7b9a/xzMGrzoscwl58to?= =?us-ascii?Q?i8a7Rw462xKSbPBrh99Q2G4lAli0Vcgb9Zv3Oqfkezu2dk5gTMQZyrIFZhGj?= =?us-ascii?Q?OSjkvHoxbDDzMczCiDzrCYFOLrH/C5g/w5XvlPX5Qf1GR75MJoERZuvJ27rD?= =?us-ascii?Q?uc0/4k2vE0uyVBq/rqghAWPXAuoc6oynwnpo1WOZuYVs6opWtEJMYFRw4ZJ4?= =?us-ascii?Q?HVmkSdfmE/1kqlOM4BMc1XpTF3pQUJxbKrvjCWFnm/a/P+CtBShrOK+zFtzZ?= =?us-ascii?Q?Pe2QitGMYNy1zWGTCkkI/cZ811b+6SIbltaoIWMxbMBHotvwkft79lMjQWTU?= =?us-ascii?Q?cD07wW0LutU/NWnLB/fp9HFD2DOK/W72P6oIgJWmyH9GIenN1ilFRpd0+jc/?= =?us-ascii?Q?Zmxzo0NeZPiB4Nwm1bD7teDF16jkb1IdSZfsunUMcTeBp1L/dFg91CM3fs2q?= =?us-ascii?Q?rWjaYIoNDVGTeUTff43TvTs6PbkN8C/5RnUAFjyvaYEKr0n6hJP2lW/sfrK+?= =?us-ascii?Q?euGMvqnISUBEp/W19zqkQaMD3x/UVG6QHOfFPtB1yLJnwuLyj6/3x3a8iibO?= =?us-ascii?Q?qzpza9CFqIEHQMWkh9kOs83lenzifuzNbTQrPlVquLukdS6jQ37tmgBUnRGa?= =?us-ascii?Q?s/HtB7TGDucg2rSz0JBoOTimH3gZuKRJO+ZDCZQPPOjB0V1gIf7LFtLveZ8n?= =?us-ascii?Q?KVqJROdnWk5P/t5GjG1xF7NvTEbQV/reRccu8j5Gi83oMotIQf0u7TqwAHbA?= =?us-ascii?Q?miDM+4rmnibqWRzRIGxievz7HgBiPFbl1M/PTOy9pt2s07HuBoAP0+MUKSuw?= =?us-ascii?Q?xavmpm5Rz6s4dfwMrr8rZEUS1Y1B+5G2KDgTbZSu?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: b109b47f-7746-45f4-f53f-08ddaf35b54e X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:16.9198 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1z2KIDJ2G4w1Viex+Uv8lBBPmkpQANpCc+FgUJJJD/jRvbazflSw25gHIYWeYQfufy06Ql8Jonyvh8kDv3HTWg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMiBTYWx0ZWRfX00AZ/pV1NXVQ uzZKfncRchrY/Kx6u9oNTsjYOg4GB2wF0WVjNM20s96GZac/XSIcBriOQvQPo3Ai2wWVYvwvNuu 8+rpSpzfJqYxmq7ID+y5/htuuAWRAiKobaGl+SdGlGEzEVsl0Xb2SV7vZRPbObUXnteOlJaRMhc rsLL+guuh96Myu1VjLyxpLYxGmZrKQxLSkmP8XSxXfOFt6p2dPggDLz6iFIe2/5xkdsw7FPXhtC iUKaP87Roh99ikvgKoFkiGid/6aqQM4TWMVPzlLPTlMX2W8U6oP3f0MCW31kq15jlbERiNm3tmO cK+SQcCGpnGVmw7Hz8lx0kNu7iVVSWT8qy79KnxNvI7xc9ZJrs9OV9fwh+RCnUBHrLhtrWDnob/ st+Dj3fG9BCX3W/TLFPvijBv9IBeC7Vk+lrr5cr7ZECpkJNAPhQzMMi1HvaiV2G26+K3m/pn X-Proofpoint-ORIG-GUID: NkRo6uPjlvgbl1aakIwRRk9TAoE1jyUe X-Proofpoint-GUID: NkRo6uPjlvgbl1aakIwRRk9TAoE1jyUe X-Authority-Analysis: v=2.4 cv=LIxmQIW9 c=1 sm=1 tr=0 ts=6854116c cx=c_pps a=Yi5XoY1Pvk1/bV4Hj+lk7Q==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=lEmlVuBv_BsZ2DPd9E8A:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340033921116600 Content-Type: text/plain; charset="utf-8" Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/protocol.h | 12 +++++++ hw/vfio-user/device.c | 67 +++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 1 + 3 files changed, 80 insertions(+) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index db88f5fcb1..0cd32ad71a 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -138,4 +138,16 @@ typedef struct { uint64_t offset; } VFIOUserRegionInfo; =20 +/* + * VFIO_USER_REGION_READ + * VFIO_USER_REGION_WRITE + */ +typedef struct { + VFIOUserHdr hdr; + uint64_t offset; + uint32_t region; + uint32_t count; + char data[]; +} VFIOUserRegionRW; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index 3d39b9b98b..8ec5aba632 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -108,9 +108,76 @@ static int vfio_user_device_io_get_region_info(VFIODev= ice *vbasedev, return 0; } =20 +static int vfio_user_device_io_region_read(VFIODevice *vbasedev, uint8_t i= ndex, + off_t off, uint32_t count, + void *data) +{ + g_autofree VFIOUserRegionRW *msgp =3D NULL; + VFIOUserProxy *proxy =3D vbasedev->proxy; + int size =3D sizeof(*msgp) + count; + + if (count > proxy->max_xfer_size) { + return -EINVAL; + } + + msgp =3D g_malloc0(size); + vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_READ, sizeof(*msgp)= , 0); + msgp->offset =3D off; + msgp->region =3D index; + msgp->count =3D count; + trace_vfio_user_region_rw(msgp->region, msgp->offset, msgp->count); + + vfio_user_send_wait(proxy, &msgp->hdr, NULL, size); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } else if (msgp->count > count) { + return -E2BIG; + } else { + memcpy(data, &msgp->data, msgp->count); + } + + return msgp->count; +} + +static int vfio_user_device_io_region_write(VFIODevice *vbasedev, uint8_t = index, + off_t off, unsigned count, + void *data, bool post) +{ + VFIOUserRegionRW *msgp =3D NULL; + VFIOUserProxy *proxy =3D vbasedev->proxy; + int size =3D sizeof(*msgp) + count; + int ret; + + if (count > proxy->max_xfer_size) { + return -EINVAL; + } + + msgp =3D g_malloc0(size); + vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_WRITE, size, 0); + msgp->offset =3D off; + msgp->region =3D index; + msgp->count =3D count; + memcpy(&msgp->data, data, count); + trace_vfio_user_region_rw(msgp->region, msgp->offset, msgp->count); + + /* Ignore post: all writes are synchronous/non-posted. */ + vfio_user_send_wait(proxy, &msgp->hdr, NULL, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + ret =3D -msgp->hdr.error_reply; + } else { + ret =3D count; + } + + g_free(msgp); + return ret; +} + /* * Socket-based io_ops */ VFIODeviceIOOps vfio_user_device_io_ops_sock =3D { .get_region_info =3D vfio_user_device_io_get_region_info, + .region_read =3D vfio_user_device_io_region_read, + .region_write =3D vfio_user_device_io_region_write, + }; diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 1860430301..3f5aebe7ac 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -8,3 +8,4 @@ vfio_user_send_write(uint16_t id, int wrote) " id 0x%x wrot= e 0x%x" vfio_user_version(uint16_t major, uint16_t minor, const char *caps) " majo= r %d minor %d caps: %s" vfio_user_get_info(uint32_t nregions, uint32_t nirqs) " #regions %d #irqs = %d" vfio_user_get_region_info(uint32_t index, uint32_t flags, uint64_t size) "= index %d flags 0x%x size 0x%"PRIx64 +vfio_user_region_rw(uint32_t region, uint64_t off, uint32_t count) " regio= n %d offset 0x%"PRIx64" count %d" --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340320; cv=pass; d=zohomail.com; s=zohoarc; b=RR6njHYJx8XOcDRY+Q6/ncJLiXWyQuemntbVD988PEiQYjIIRGUWcQ79kvvHHn01E4TRP/Ebdmw5YYFZnxBJyD9RVa4B61cX06Q0VxALkLH3H9QOT1YhHehUbDhVsTQKENz+1t6Mr7O8eAyBLNmIGd39fSV1EwEqIFw4cZxDmJY= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340320; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=h3H3fxLIImpA3+uMFRcbHSmeNviC4QoeybtFHFgYgqI=; b=TkLeHkT5N4tmTeD6cNJXngy1r9OuGpUMEKBcpNJXUlyDJyIWvdJiNA3chrKRtKGt96HRKSPOAslqDKTStAXC9bAi1Wg6JEJ1IXufO6CXN6ZCw96UajQC9eI8Aoe+er5T8qf4uM/bsO39QuGPBvGg5gtrgxD2c9f1n+GptiKb1dk= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340320653535.4847904214544; Thu, 19 Jun 2025 06:38:40 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFNg-0004GI-1R; Thu, 19 Jun 2025 09:32:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNc-0004E5-5I for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:36 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNY-0008E1-C2 for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:35 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55J8A05x013187; Thu, 19 Jun 2025 06:32:29 -0700 Received: from byapr05cu005.outbound.protection.outlook.com (mail-westusazon11020121.outbound.protection.outlook.com [52.101.85.121]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 4798mykwq7-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:28 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:19 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:19 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=h3H3fxLIImpA3+uMFRcbHSmeNviC4QoeybtFHFgYg qI=; b=iVXSuQH7fTMG+uSo5TVUUnfDI7OUTEJ5k0tMrWzkpz9RB6BRtCWp+fcI6 Y0Kq3YC75Yl7/qvoxUPf/J/OKIxgUw/sIs9bEnZ0opTy2vuzHg0r22YqzMeCHmGg YxN5E9qcwiy/+Ub4idSdnd00aCb6ieUXlutKGJC/z4CNc0Yo5gL8cQY/hBFJ4cN9 qifwRMdK+iJBo+ntygfmT2d3beqH/nBQpPcwKnq+Dy6N+Hlye7dHJUEpsnJDBcvJ P2wE7Fe+gee9alkGN3wvJz8XAamlAVCC5cND4NVakXlXaQF8YbGpekInE8fI9ZXX LpbIJsZ3Cnp5ePs3RQFe7XdxgPA1w== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ZuE9xKUWY+IIdJ/fIL4nyPuMaGhyqHvNs4F5/QT7WFnkDd2ZTRRVN+MUqnNvNosJk+/T/fi+XiozToAcjwfHwFBxICvYPYkEwWwS4MSEZORc6E04phcR+lJGLWyKSOEhr6/5wemVA1w2H4PLbZB3fNKk4AVdfuyxzQmyeX5yaRQHrNw02+2N1xOw3wNKNvFX5ihstMoIC0oatbeKYO5yuDuty20KIxOqQgE+Hx8vbZVNzujMlHf+z8K9XpyPH3kcqzARsEs8+O6H+iir1pNt0kA27c5Ass59C0hd2uchjHDbOvAtvT2ZldmG3pBq7/ts/uAmZSosbIldKLFd4A9h/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=h3H3fxLIImpA3+uMFRcbHSmeNviC4QoeybtFHFgYgqI=; b=jaCPplYvLRmKU0MYrIugFEwknZG1yFDjaE6+LVTe1QOQ2yl759MdzMZdUgftu2X21ENIAnUaNO21qJ6IxVJjRuLCH9TE9WNOAlRhXlqYOKRZzp7NVecWauK8nzDMlf4loOiLdGFZUK/+lNnNSQ54msXxVQGOqbkXY1EIxO4AJkQ1m53pGl9HV8itwvf1VfEJiuuNyxikwPTnoxgYjVhiMTfpc0wKs1nA/YAHOpyXHOC0ed4RgU71bRn+M8clPFi4nEO4gNV18LEaeSiZSd/16/ag6lJJJMEgZSE1+CL4TLges/gvkcLDed++dcn9UuNbIroG9UDpzkV/JtCfqmJaMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h3H3fxLIImpA3+uMFRcbHSmeNviC4QoeybtFHFgYgqI=; b=T+eLd5EWfzQGjKZtJkbYNtjRF3BgNiGwV9HLo0PP1DJS1fV8lYM+rOPz7NjQgsI4BcHGq4dxmjNIt8E0vomhSho1xOtGfwaIk2FYBU5cF+MT5FmcKfHbZDdUIjwISNDqY0DMoMhIlBG5R1gWdeRTGW8/cZUc/zbuPc8Lht28fCTLslS0WrUXi//xNVaSdcT57/sAGAOZl3bz0gJME0dDVCiGMJWOvXF2ZqEzP+/GkGIW5GLgtRbMTB5oAiaOTOnz2yauCxQId9QdQZuMnZkkDGlT8DGxoCSDeuGvf1k1gbBO5TrNa2Q6I/SaVpBQSg2VWwqtH+UuD7rnyIzGm+XT5Q== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 08/19] vfio-user: set up PCI in vfio_user_pci_realize() Date: Thu, 19 Jun 2025 06:31:42 -0700 Message-ID: <20250619133154.264786-9-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 98f0b573-99e3-4af4-b81c-08ddaf35b6b1 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?DNFdlVQQJ5V4Sqoa0jp1JQ/HTV1Y+Y2ufcWE730bFrVVYl6oibvkduWh1pVV?= =?us-ascii?Q?fUq0oRUiBmhqfX52pON95N4OquvsT5xolF86xIRZvDfGP/r6itFUHL3f/MHD?= =?us-ascii?Q?JJEGE3ZpTofqb8KuXLBwlIWEBmVp15g9/jGaPxlFPu5inzGM9EyiZg4sICkj?= =?us-ascii?Q?BlOcOmyZGkRMSaPHKrtan8D0kriUtim7qtIfKhwlSlTizpfOw+8cAf/sbRA4?= =?us-ascii?Q?reKSjltMbNMMbS7zhMdNK7nogYAEJmwzMwhu/yKgDqYdurJx6m/P5S2b+RaB?= =?us-ascii?Q?Kxz/nrez1xDZL8TbzrjaTXstcKDbbRqt3AIcr8qXmYsTfeofbYE6j61mbw/K?= =?us-ascii?Q?pUeVu3VcbjmvoyTHkdY+A9czfF0MyzOTBbLkd3x+/NXFXKQloy7SBpjbRHDp?= =?us-ascii?Q?494xALvgkd6ZFxa0iQjxjnlCIeCbNBSMcgbFlO2O/td0e1tjqNUFFBYpMJSs?= =?us-ascii?Q?QeAbbZB1dU5g2yj84sN+Q89V6ynrLgFgeAxhaTsu5g2SHfFv3BK+hRVT/GeJ?= =?us-ascii?Q?2IPIVD9zCjLRtiutheP88YrtGaUq+07q0yyhN9cLm1XQJj2xAR5u+amXECXc?= =?us-ascii?Q?vGUSV4z5nhnUkpPGS4WaAh0GxA3mXKjFltCzR/61b8JYRQtaucJ6eS8Md8ZP?= =?us-ascii?Q?9GHLeyTtGF1L8F08BXE38Lup342G97MQPoJaFyK8UCr0TVHERt4DwKTQ9xr2?= =?us-ascii?Q?eoCvBx3UTAuktnlJs56slG+xm5A0978MocqX4ordT7IpLY7/Y46KUm0mn5Mc?= =?us-ascii?Q?lpgKYKaDUcbw4i1Jf19bBVw6pdJT8OhAMbOaoXadsd8vzKo4S8qC983TRXnN?= =?us-ascii?Q?mHo+gHrGClfOYgzwDO2PLtTFHezc8bqACFC9U62G5AsMede2ImCUKqD0IkqH?= =?us-ascii?Q?DX49pGs8fO4/3hH7+od/Y8T4jTYDgK/UcXgzTmK74jByzauPgDh3//rU4S1L?= =?us-ascii?Q?kGXwMipcRACXKxZkGAU4vzRyydxAaQwLMwmF9+DUDDH09cQqWAxTluonW+a3?= =?us-ascii?Q?CcTCtnNZfZer43PJB1p1SdZ7nMY3142skJLr47D6EPUwX1KsYGnwhyEQGoLR?= =?us-ascii?Q?MCDeWhxo6CjLhW68nP5a++0/py8RgQiaSPHt1dhJuxolzECPBtqwj6G5/FeO?= =?us-ascii?Q?L+pEYwGkua8v6gUObcz4b7NJq4BZqSeCP2SqyjEbuy/JVolt76GN/EHcxUpU?= =?us-ascii?Q?1HqIkL7DJhDZx2GuHxFlRSf9yTJEgf4mw2byMZFKMZK9ZxSBIYwCf+CU0m5g?= =?us-ascii?Q?A2FLs17kk3rg5NlE7SkiEjaZKYvuASoYxViZX7230YG3Qw86s8EVpPQHugoo?= =?us-ascii?Q?spMtkcMMBmhUgBgNZqS60mA9c0bGUbIuU0VkbDxF19js2bgAeH15sbhwjWXK?= =?us-ascii?Q?5DS6aQ4IOl0JxyB/khOcOycIJjspCvp8ziKAPldLofjB1g+CLCy0advM2kkN?= =?us-ascii?Q?YpQhTeUX0UQ=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?kXAJNPMG2uDL/HU1kUCRBKPq2Es7mhQrCRD/9ITdQaJpRf1cjcMffmppDWkC?= =?us-ascii?Q?zqwj+Y9TMs+eemxB0NZA7TxEMVGsUfts/EZ9ZivJDxuLp02IQR+NjUx5UIHp?= =?us-ascii?Q?N/R3yNWMSIIHtnmwhi29XgmwMayUOFhLf9gW3IbhmsXJxRoBrJ19GfPCPTVX?= =?us-ascii?Q?SmhVgU3kLdagJX8+LIJqtJLTQVsw5+Rmnm+XvLArFD6Hsu7MHtW3682YMKgX?= =?us-ascii?Q?4p/w6id31b+JHW5mLKvP1QdJz6j8iL5pJ2WyBa7YUSnJpj99Mrbj9FSAFLOV?= =?us-ascii?Q?P3w8BF74gIObVnV5r6BQwAmvbVtk5awJ4c310AV+rVh2lhSEvCG1mbygkIH+?= =?us-ascii?Q?dKbmaO+5+UO3nPZ/XB0c4sXOD/LPzb/wv53QhAeeAYpjEeSyUYjOihSCghct?= =?us-ascii?Q?kGMNlsq06vaXclW9GR0jaATJL45kaJ3gT7L9UGqTWPQOlKswpFWqrUlW2XmS?= =?us-ascii?Q?DiuPgEFJt3qlQlQk3P3jwQlTDLl+/zdJg0mztqGAy/9pGyy0/T+7/MaD7DT9?= =?us-ascii?Q?0mmEF3rqlKTxOQyGjFG3uhEEZAom5JPFDw5deoUTo355+SH5FCGe5G9uBb4z?= =?us-ascii?Q?dlERocBPbXhocn0Y0WJ0mHrvZtEG3tl4CPoWAraKTgywn+6vMuWPLozuQwQJ?= =?us-ascii?Q?dqREv5wuVvBhySiNiIpFPh2D5A24rZ+aqCo39nRy8+lCeSslXJVswFw5OMZV?= =?us-ascii?Q?gNQ5rMkWn+pfmvVmlbePDq5am2kxWIvAoAZ/2K68WyoQyCXdcZ9X/83BMf89?= =?us-ascii?Q?H56CntxLzpB6IdMCdXLDqDfNT7nrSmiFHo3UY6W8z8Gs3Tx0Hk/F3UGkbJCR?= =?us-ascii?Q?pSknKT3StaG6Vi2h5UHpGKS/PICv1R/SBzQH+OOnSZetKtezzD/4BjOU2Syt?= =?us-ascii?Q?qxl5hcKkHVWpcQmPuiWsaqE9uIrulDF4gSDMsWd/1qKldaasTgqAj0z9KtoI?= =?us-ascii?Q?BtMOxjmYLFz4TonrSxlZzXc2ov6pAtlLOF44esfMTB9sYkEPHLv3qlX1kgIf?= =?us-ascii?Q?pco6OPVF/QdQqOTxl5rZ7DbX6s9JD0sZLSAu8eR02fSBHIMbAWAE7K1oibHp?= =?us-ascii?Q?FWlu7D/WSdR0FmeRrLsAiMVYGIeN0KvZzUzczuiocREFG+vlpUF8Z3XxdM8x?= =?us-ascii?Q?hX1zeGnTIaod4pb9syr0CSbF7LB2feYBf4OS5taIKB6sR7i4tZw0thj5w5aq?= =?us-ascii?Q?rPCSFvaqz0J0JJvISU/GV3A3P/uKUVbi7kT/CKT4UZ3yu8FiyQ2LKrP4WX/X?= =?us-ascii?Q?S7Vr2QbLwBWZxkj8M6QKllB8f3SnzwWkCAFv8+S7MXRgA8W0/OqGwzsCD8Mm?= =?us-ascii?Q?6yqk2wYkOTjKGemLYYG01WaE+RdDmH712SOwz4w02QBjDcwaI6b4goVCSXd4?= =?us-ascii?Q?SWqxumEBm42VKEWl4QwF5lCd38nXjBO8yfyKL/fmZUdAnW1QuGvP//aeC2Gy?= =?us-ascii?Q?69mJBer2udM7EN2hbaUrIM9bI4Ch8gMnXFzdbFiDNhasSJJKlvvPoZhi9S69?= =?us-ascii?Q?sf4V9ltqkCyjm8jnAitupM+GYTfw7Mc7C1VI1BrNRTW2kocNswQw6nqBhEI2?= =?us-ascii?Q?B4Kev14Vsv4s4jMJifqVG+d1Yc9+jFxZnzWxhzcN?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 98f0b573-99e3-4af4-b81c-08ddaf35b6b1 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:19.3144 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yyqW5T2daY3+PqvNwjXggmEb4oK9Kz7FCazeWktHQdVGF9/THiJ3fejXYVgbfCV8LaHHrHxZWw2ZDyOBjo7w+Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMiBTYWx0ZWRfX/uc8wsaLazVN Im+rNFF/+qZYuZlxywWJR/yOgS2cm/rkSJVQzPQ+SV0Se53Y5SRQN/OROcVnbTzbL5XaIRb8ksC 7i9438bW2fWLMCDEqpXm60cfB1nlwL9tFzy1EX4U5vZ1hegBtgGUDq1RhOrsWVwwgn33bOisaRd 8HLNhhUEeAI9LxtnFrtECjgOqUkfBcSKX5zgHOTxp+kn2iMOLSc9icdUGCxiQ3JfIEaS6wxQbmt 3PViAOnPZXN9yShvGbMv6JO99ZimVnoIUXJb2o319VJsMEjcprzUaBxQUkyhplgmo8AZkK2G76Q vc9+lP8dEL1+AVht9LgmWxK4GOJcju7vJ+FPf3M5/s2K0uShfmDJuQJ6Zy8XCMtEk0IU2rLMtDo Q4BhkujDo6qdDrFqxUGMaIDIsZHKEnMmXDxmnyDo9pnDkdOcJKsp6AmSfTBflJm9ZD8LgDW8 X-Proofpoint-ORIG-GUID: CUonwYwkAeBRoKWIQTi-jdUZZD2P-SxN X-Proofpoint-GUID: CUonwYwkAeBRoKWIQTi-jdUZZD2P-SxN X-Authority-Analysis: v=2.4 cv=LIxmQIW9 c=1 sm=1 tr=0 ts=6854116d cx=c_pps a=Yi5XoY1Pvk1/bV4Hj+lk7Q==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=uN1uQ0dK97G9gywtELIA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340321599116600 Content-Type: text/plain; charset="utf-8" Re-use PCI setup functions from hw/vfio/pci.c to realize the vfio-user PCI device. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/pci.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index d704e3d390..b49f42b980 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -126,10 +126,39 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Er= ror **errp) goto error; } =20 + if (!vfio_pci_populate_device(vdev, errp)) { + goto error; + } + + if (!vfio_pci_config_setup(vdev, errp)) { + goto error; + } + + /* + * vfio_pci_config_setup will have registered the device's BARs + * and setup any MSIX BARs, so errors after it succeeds must + * use out_teardown + */ + + if (!vfio_pci_add_capabilities(vdev, errp)) { + goto out_teardown; + } + + if (!vfio_pci_interrupt_setup(vdev, errp)) { + goto out_teardown; + } + + vfio_pci_register_err_notifier(vdev); + vfio_pci_register_req_notifier(vdev); + return; =20 +out_teardown: + vfio_pci_teardown_msi(vdev); + vfio_pci_bars_exit(vdev); error: error_prepend(errp, VFIO_MSG_PREFIX, vdev->vbasedev.name); + vfio_pci_put_device(vdev); } =20 static void vfio_user_instance_init(Object *obj) --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340220; cv=pass; d=zohomail.com; s=zohoarc; b=amKI/bu6seaD/u8b5XANbzxjnyq/+oztmYIMpflkYUXu894hbc04F7rp35L7koSEOXWYLdLgrdvSGoUw4t6mGWxHtxlnzKCv9WzRMppkwflBY6r4PqflwbEW+UFeA1gk1ww6sQcG9cdHgkM+43b+/+uoGsf4gi2YbvGkxOQfMkE= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340220; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=4AQW+C846BaFh6JO8Nd5ChXiGKBzGo/8yVp2/T7vqZo=; b=dl909NP7SvbbQpdkYx/s++5Tsu9LAiwrM46WNiAs3RjgASuTSQi1tLrokYiDdqTvu9mYJ9/Lff+KIrTXQ/dj6pK5mz6ZHUR5Ve+9dVQPF9jDUx2V7kzH3xWDeRTePj9SRlHWB3bDY/OHoED9dpso/gTG6D10gu59LY4Ex6/sAIQ= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 175034022044762.753985449554875; Thu, 19 Jun 2025 06:37:00 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFNf-0004Fv-CE; Thu, 19 Jun 2025 09:32:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNc-0004EN-JD for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:36 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNY-0008EA-Ds for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:36 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55J8A060013187; Thu, 19 Jun 2025 06:32:29 -0700 Received: from byapr05cu005.outbound.protection.outlook.com (mail-westusazon11020121.outbound.protection.outlook.com [52.101.85.121]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 4798mykwq7-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:29 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:21 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:21 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=4AQW+C846BaFh6JO8Nd5ChXiGKBzGo/8yVp2/T7vq Zo=; b=qaxMPtX5yIwKdTVdMlsQaVhFK86IAryt6wHa27oJomp8dEGNN/5j2HDG6 UcbYANZwO0bTeY8ekhw/+bZgA2QmxZEAfEaO9JYNg43W1uqoUtZ8PmdxGsx+n4Pg 1esrkfus9p1lC0iMAJTCMPI4DYnIZosCM05bdHs9ie7aq/RdysS5mxPtL2gKocwU inb1HaB4tmBp5IFPaF/YYyqWiQl+thCCP5+uMwDxF3arvm6fPy/rdHz0EgxyW9tj 7Vzde/os4iOjAp9vtSuNqjSZ3nBIbC/Ffe01gAMkooVkmAcQ2tf3tooWQVZlBbrt fVxDxZDSGq2QEejDsulxRrX/X/Njg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=TOQrjuS/DYEmJrZ6OYd6W9tzwb4WKahg0FAS1tuQNJs0u6t1NJPftsBL/YVvATmZu/6CH3Yd52fE4PyODJydg0dEfkoLUovUYsWEw7zBTxjud4sDjHlrutQNF/RtP+zLddDB1dZqwaMhgQZaHrgD2RlkBmuTAiO3IFoe9fSvOTU1jNTvl1+pRYA9ApGXMNt0rENIRu+Egqnlqf4TSBz3tfLQoyqI/Zkj0S9UJuo0MdckwLlm3KxoG/wPv2aR/flo5v/aRUJ3S6lldU1qhamZiUXdSmxa55pCXM5jU17mReX23F+cOxvPD9tTFWt+JuK1S7mkvrg8HoVhvz7SSsSu0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4AQW+C846BaFh6JO8Nd5ChXiGKBzGo/8yVp2/T7vqZo=; b=XOGOdHUnNWA+J/9P0lkaiwTkVx8mfSI2P4AxRrqhQsXofItpTEEkmdxAcc4fNgUSqCIbdV3Rc+HqiTE3MR7yjgyitucTmc7FnFLL1haWcqLjeh26RXygpJt+G+hh6vOTit70uO3YuAwrGh7XYvQR8HN/x/GGQKXplzJT8KWEsWt49wyF7RW4D5XWHCROP077Iey5f+5LI7pz0MaYbsT477YHtMoEsr0cmwGpo9mx3XjJyMRnIGDWOGR2J2Qi20b5boBxnqVVZHQFLmMOpdQYjegE/sq2g3dzMGAAUwywQJUX42vnqOCXpvJZIK9wfXLLd4gtz1n58GVLBcOLG1Gyzg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4AQW+C846BaFh6JO8Nd5ChXiGKBzGo/8yVp2/T7vqZo=; b=xRJ2qbQMOKsDfCB2jtewnTwtaItUGj7Lym4O1UM2Fn6hXGPkyB3ENqW4pJHtQx/TMgV85CZ6R/EadexsDbE9q6Vya6Ga6YjUVVHORfJK/5SMUN4IHjWgZ9YXvTd1kigLFzhSFtsOdBBEvH29baED7/pS0lwI4EhQCWpDEKctxHr+3uGrEhafHvLyASgzlkCiQIydfqQaUF7LkC9lpb5zbE8gO1Ha4B5+4ipup47MdLHkHnrnmnmsfFxqtNkIFEbS5ewe2d1Z4RvY4BkyF2RHgMy2XzcRlpc5bJUE1OETTYIaLIdM3l5EbbNlk9dhDRKmAFRxphzpwrMCxxEbWQKZ/g== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 09/19] vfio-user: implement VFIO_USER_DEVICE_GET/SET_IRQ* Date: Thu, 19 Jun 2025 06:31:43 -0700 Message-ID: <20250619133154.264786-10-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 8edbee53-830f-4785-2f61-08ddaf35b819 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?+J0iMblJs7+wjYuqBIRFTPs9lMZtl8gbrG7iisqgk27s4AwVVASkysIzCqLe?= =?us-ascii?Q?mjHFyA1RhhQ/pqGG+PXjhzNQOsuIFMZtPnnBqfo/w7u3cHaNFgEd+tfj+YIp?= =?us-ascii?Q?o17ilyLZhiNC48r+dNvf38iviTICHefSlSGY+HIhui6y9kbQehtPtZ7rF6qi?= =?us-ascii?Q?7T2oYhFj4h/mTQ+YG8Cnc37DQAhDdC1GU48AF4IamSnczcK7r7T/dexqewBy?= =?us-ascii?Q?1rIwY+Ea37w8b65ScOmuhdejoZxaZi7xurwO8tj+dH1waj/BSiLjt6w7Eg1m?= =?us-ascii?Q?1mRw1m9eIyzHhFEDQ73IwwuKeqlrNtLBeVifDKkL2ajEzNy9Ti4zu6XdLiVb?= =?us-ascii?Q?cwHhp3ZjiSfjdPNk1iO5ylB0LA71vGO2/H380gGigkyX0bvSi+enhsNKSu1z?= =?us-ascii?Q?/ZuYQvRlPq32tf3Vh4xIHeW0gvi33ejP3r3D+tXfPl4bf3mxcKevGcSntqwg?= =?us-ascii?Q?9X7UofMeUcj8dowM9HaK1HaZgYgd35xb5EWRR6PPFKjubSksfor2b2WfBUPi?= =?us-ascii?Q?A0RMUnrdc1NYxTv1R0IL1zjf9s7YWQHOUO5NDOAknqYNgSmLpNYO8Lcr/kB2?= =?us-ascii?Q?ZpjKAkfuWXoXwDl7RzzekIJkTH63uraK4zQ8WDAfkfqsUHpSaruddggjR+XJ?= =?us-ascii?Q?CQDUQfNT8WfdChrDSjR13t8DBzQh3C8A1d/4KPacIh5N6SCLs/VtvMTRx/hv?= =?us-ascii?Q?GglgaBeHHejLJ7DqciTgoneKhmRoENPhdN+R3a2XFeO/t18rs7B4NZppS77q?= =?us-ascii?Q?pko+kyEHQvUkw8rnMvCpNITCW4qXiWo8Yb8EAmm9tVD/DvEqwlSWYj3vLuem?= =?us-ascii?Q?xesK2FhOcniWPL4nKsIqITS/oedsBBjmYusppKeiMo1FQpkC5QgR6+Z8lQOf?= =?us-ascii?Q?ykYswxxe1WEnl3H6TybOgRCpFPQG3zau6vRbUi21vUld5a9pUUla88+t4gHi?= =?us-ascii?Q?846nw8lYVo+xnJaeEr+8k/pokqSWrkxVxZd7FOsKR5bk5iIq8MHtdZ3SW/YI?= =?us-ascii?Q?PFcPIUo+O+Y9E5n6GUgf/59dHryCrEzCWqkxbz+s9rz+e7g3uz49h5AjDeOe?= =?us-ascii?Q?iVFBAoPT1J6JlBbCzTRDgV/R3TpBO9CgYVMfPkZihKYHeIHx1csQDj39cABH?= =?us-ascii?Q?C3rRXibPgIaNKdB/ZKniVhJ4B6jnypIjyimr7+bWtI+CjRiZc4RiVWvhndYf?= =?us-ascii?Q?VaiIMLDmbOlj0TRTdTUpKCOZu5yd0pSRUe/kTr2hYufnnmJtoEcMFmGdargA?= =?us-ascii?Q?oeSbBuBiaJT6n1bIOuSHzdWu2SurmLN7SWdQwmKcKjuaBfq/Vo5HDDjSncuO?= =?us-ascii?Q?qC9qw0uC0bqbF5kcvvd5uyEjZVppr/xuODtZEH3fYtynHVOTNIQAFs5Ir5ub?= =?us-ascii?Q?97LqFsEDfa9Bdiw2P2z84EJ/ePVkzm5uv17LjBIuyPozFFXzMg=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?LZjTWx1vVQY/sJucakdQipQi4lTxBUM4e7Wu6s6OfeTGpiaeRJEX0pQl3nNQ?= =?us-ascii?Q?3VnLeLit8JtJf8RiPTb/MBtL9wxqJNIwDMm9jM1UeDAmy9gemku5IFSzfilz?= =?us-ascii?Q?/cZ2Zt6N1R1IrClZV4Ljq/q6HWC8z19vm4luhOCJMRowikdVIASbk3MDYS/d?= =?us-ascii?Q?RRrf8Jl+DlgzWYdoJuMr9HB0zxolBjTmjzurN3kFD07ssKSWpFiQJ2PXIuYK?= =?us-ascii?Q?cp9ydKrPkrbVR70sD8fMyc/SfVSn3Q+z2ATqslNfwTPVQ7W3k2X0hOVa4+Y6?= =?us-ascii?Q?DFWigvfhMPIArVJj0epn5yvIXezgRj7JUgWTeJ/sTz+30uM/k45pbmpDWT/+?= =?us-ascii?Q?21bw+AYC1Timcl48q1bbNaGeBb4ncwbTfIxNkFhshzWkjawYtenuSuJwM4vO?= =?us-ascii?Q?jjhdhPXWvC9htZqPb91mvH2t+w6m68c4tVbg5DhhdiLR6GHkZozSz2B43cAd?= =?us-ascii?Q?dtpeo6dyxwYTqmayhGI5JFZPsMfbllDlvYFLSGklsuL+QqmHcezOiO9OL1y2?= =?us-ascii?Q?3DzCgUEylq7/veecnX49wuxiMr2aJ6Al5NMMHR5ttWlxPDH3y7WZVightCkW?= =?us-ascii?Q?P/h7nSmDvSXYtc2OcsqbL70Nd/uy9k2l2e6WGmdkYYg+21E6ItvMzQHFwQ/F?= =?us-ascii?Q?L1K0Q7anti6CAwnJPfCwrN1qZP/2EQZJF3Y7r0qNhZ/YR9YRTtthfDFvo0O1?= =?us-ascii?Q?bl0kHC93K2nyoh/W3AH9pGvz3MZzIbDONl+nfE6Cu2iZF6ruIM+zC6+qieOc?= =?us-ascii?Q?5z1AK1fkTMwFaziScz56YMFdUmzfM2OjefRrn+gdzJF/WELLmnML6ZkitDWc?= =?us-ascii?Q?7o5UGtEKJokvLl7yaA2gJyvKpAtlw0QTjf3/Y5GoJx/xIO4XPv7jU9bAJUiV?= =?us-ascii?Q?pi60uADnwBugAObMlcxcCVwGdQxiMRE/DdxxS3+5nTZwnRbD6u2iiNvyyzM8?= =?us-ascii?Q?eKBL68KotN1fkKdQEVHtz4YGuBxD9MPlaRx8qhO49IOc+pgc2t99+v53OjSB?= =?us-ascii?Q?F9wld0w75pT//0uuwrojKND1JS6t4KNeVPhfe+c+tD0dPpKc83EfWAfAEOSG?= =?us-ascii?Q?Wnl72PjBfrVZX8nazDBkPQQJ7yp7A//61tzJCLLJPhBE3zKTpWQnIp8SLlwg?= =?us-ascii?Q?fddckyW+H1mSdpIoB2vqqB0P5S4kRosyQ0oWTcEr3I6oifpF3qjEAEwgfd5j?= =?us-ascii?Q?vNrjjakSPZx6zlbPJSJwS4OA0aCcVCikcx4ulDWO4QAYBbHTCaCd5xrvql4A?= =?us-ascii?Q?1qIb9DoJk3+MCJ0cO1C909XctNs0hLFFpNbtew+EQR/BrZDvL6zCqJXuQ1NE?= =?us-ascii?Q?PDgl/iu6P/t3NwmzzNeCnSO1j/jCmFRGBmINCdeZhidfmELW7UWJ0mBGh53i?= =?us-ascii?Q?dIHKPrzbrtgctdHftxveh8Cpko14+pBI09wDUVQviWVPaTxBukT3w+vaEtfc?= =?us-ascii?Q?Lp9e9E3QI/I08b7Tn/kouYAkNqCX+xKa9YnzkuN/hXcmlwdj8kbTl868AVXf?= =?us-ascii?Q?7R0sE+WpuvTZ/h2layzsfo3quAkb0NFdZy0qY602lkAQOa/gWC2xR5z/Z5QX?= =?us-ascii?Q?8TuUxYNynZYFAQcJqI3qrRaGW3ZZNGgU1zJsGvlO?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8edbee53-830f-4785-2f61-08ddaf35b819 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:21.6574 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ygSVBqnAP+jUXht74H9RoKqo+ODoXMfJQn65hQpJ9w5VA4Yp9EWHw9CFGWhSZCkK1k/mWM5g3h47QBWiT+wZPQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMiBTYWx0ZWRfX9DzDdi5g2EYl b33tdWc4USNGSFU9UmIkl3cX+uUh1XFDcR1t6H0qjPYibAoXBTU4Zea/o+Gg/NMMsgLGFl/hWxW 2HJJozIBH1d8pdWLu6UvQmr0djUm0fd79b3ClL6gwuDRyEyXm/TgnhrR/GT/8XgR/9vt554Zbky 9Li1x7kb+sEoj8ryo78rtONsEmAFsQJJsTqrVsWNwfLps1CeKgerhkLXIEPUBuikbGCHUzAtCbB u3aqpjcaXbvo+Z/0p1ucg+NRBx9tCa5w+wS7DcLUDNc1MO471iXqwuQ11z+k9PtBZo3DdEeH3Yv BApEa7kEAH6SRq+sExN0/YWv8/luJTLDLc6oUKr1bzxL1VdTKL+hOn9mDGR8zMCwGE9ARGmHUM7 kGxdAjB1xqWmU5HBL+8YqwtHqXjGfklfRJW3iNkVcNGc0OV/ZF+VwJk15yli5FKYCSSJHw2S X-Proofpoint-ORIG-GUID: eMalymKn7My15u6YCjJx2GErlUOH88N5 X-Proofpoint-GUID: eMalymKn7My15u6YCjJx2GErlUOH88N5 X-Authority-Analysis: v=2.4 cv=LIxmQIW9 c=1 sm=1 tr=0 ts=6854116d cx=c_pps a=Yi5XoY1Pvk1/bV4Hj+lk7Q==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=03k_OfxykpieI8sgW_IA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340222501116600 Content-Type: text/plain; charset="utf-8" IRQ setup uses the same semantics as the traditional vfio path, but we need to share the corresponding file descriptors with the server as necessary. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/protocol.h | 25 ++++++++ hw/vfio-user/device.c | 121 ++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 2 + 3 files changed, 148 insertions(+) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 0cd32ad71a..48144b2c33 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -138,6 +138,31 @@ typedef struct { uint64_t offset; } VFIOUserRegionInfo; =20 +/* + * VFIO_USER_DEVICE_GET_IRQ_INFO + * imported from struct vfio_irq_info + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t index; + uint32_t count; +} VFIOUserIRQInfo; + +/* + * VFIO_USER_DEVICE_SET_IRQS + * imported from struct vfio_irq_set + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t index; + uint32_t start; + uint32_t count; +} VFIOUserIRQSet; + /* * VFIO_USER_REGION_READ * VFIO_USER_REGION_WRITE diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index 8ec5aba632..0d7ed9c5ba 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -108,6 +108,125 @@ static int vfio_user_device_io_get_region_info(VFIODe= vice *vbasedev, return 0; } =20 +static int vfio_user_device_io_get_irq_info(VFIODevice *vbasedev, + struct vfio_irq_info *info) +{ + VFIOUserProxy *proxy =3D vbasedev->proxy; + VFIOUserIRQInfo msg; + + memset(&msg, 0, sizeof(msg)); + vfio_user_request_msg(&msg.hdr, VFIO_USER_DEVICE_GET_IRQ_INFO, + sizeof(msg), 0); + msg.argsz =3D info->argsz; + msg.index =3D info->index; + + vfio_user_send_wait(proxy, &msg.hdr, NULL, 0); + if (msg.hdr.flags & VFIO_USER_ERROR) { + return -msg.hdr.error_reply; + } + trace_vfio_user_get_irq_info(msg.index, msg.flags, msg.count); + + memcpy(info, &msg.argsz, sizeof(*info)); + return 0; +} + +static int irq_howmany(int *fdp, uint32_t cur, uint32_t max) +{ + int n =3D 0; + + if (fdp[cur] !=3D -1) { + do { + n++; + } while (n < max && fdp[cur + n] !=3D -1); + } else { + do { + n++; + } while (n < max && fdp[cur + n] =3D=3D -1); + } + + return n; +} + +static int vfio_user_device_io_set_irqs(VFIODevice *vbasedev, + struct vfio_irq_set *irq) +{ + VFIOUserProxy *proxy =3D vbasedev->proxy; + g_autofree VFIOUserIRQSet *msgp =3D NULL; + uint32_t size, nfds, send_fds, sent_fds, max; + + if (irq->argsz < sizeof(*irq)) { + error_printf("vfio_user_set_irqs argsz too small\n"); + return -EINVAL; + } + + /* + * Handle simple case + */ + if ((irq->flags & VFIO_IRQ_SET_DATA_EVENTFD) =3D=3D 0) { + size =3D sizeof(VFIOUserHdr) + irq->argsz; + msgp =3D g_malloc0(size); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DEVICE_SET_IRQS, size,= 0); + msgp->argsz =3D irq->argsz; + msgp->flags =3D irq->flags; + msgp->index =3D irq->index; + msgp->start =3D irq->start; + msgp->count =3D irq->count; + trace_vfio_user_set_irqs(msgp->index, msgp->start, msgp->count, + msgp->flags); + + vfio_user_send_wait(proxy, &msgp->hdr, NULL, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + + return 0; + } + + /* + * Calculate the number of FDs to send + * and adjust argsz + */ + nfds =3D (irq->argsz - sizeof(*irq)) / sizeof(int); + irq->argsz =3D sizeof(*irq); + msgp =3D g_malloc0(sizeof(*msgp)); + /* + * Send in chunks if over max_send_fds + */ + for (sent_fds =3D 0; nfds > sent_fds; sent_fds +=3D send_fds) { + VFIOUserFDs *arg_fds, loop_fds; + + /* must send all valid FDs or all invalid FDs in single msg */ + max =3D nfds - sent_fds; + if (max > proxy->max_send_fds) { + max =3D proxy->max_send_fds; + } + send_fds =3D irq_howmany((int *)irq->data, sent_fds, max); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DEVICE_SET_IRQS, + sizeof(*msgp), 0); + msgp->argsz =3D irq->argsz; + msgp->flags =3D irq->flags; + msgp->index =3D irq->index; + msgp->start =3D irq->start + sent_fds; + msgp->count =3D send_fds; + trace_vfio_user_set_irqs(msgp->index, msgp->start, msgp->count, + msgp->flags); + + loop_fds.send_fds =3D send_fds; + loop_fds.recv_fds =3D 0; + loop_fds.fds =3D (int *)irq->data + sent_fds; + arg_fds =3D loop_fds.fds[0] !=3D -1 ? &loop_fds : NULL; + + vfio_user_send_wait(proxy, &msgp->hdr, arg_fds, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + } + + return 0; +} + static int vfio_user_device_io_region_read(VFIODevice *vbasedev, uint8_t i= ndex, off_t off, uint32_t count, void *data) @@ -177,6 +296,8 @@ static int vfio_user_device_io_region_write(VFIODevice = *vbasedev, uint8_t index, */ VFIODeviceIOOps vfio_user_device_io_ops_sock =3D { .get_region_info =3D vfio_user_device_io_get_region_info, + .get_irq_info =3D vfio_user_device_io_get_irq_info, + .set_irqs =3D vfio_user_device_io_set_irqs, .region_read =3D vfio_user_device_io_region_read, .region_write =3D vfio_user_device_io_region_write, =20 diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 3f5aebe7ac..053f5932eb 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -9,3 +9,5 @@ vfio_user_version(uint16_t major, uint16_t minor, const cha= r *caps) " major %d m vfio_user_get_info(uint32_t nregions, uint32_t nirqs) " #regions %d #irqs = %d" vfio_user_get_region_info(uint32_t index, uint32_t flags, uint64_t size) "= index %d flags 0x%x size 0x%"PRIx64 vfio_user_region_rw(uint32_t region, uint64_t off, uint32_t count) " regio= n %d offset 0x%"PRIx64" count %d" +vfio_user_get_irq_info(uint32_t index, uint32_t flags, uint32_t count) " i= ndex %d flags 0x%x count %d" +vfio_user_set_irqs(uint32_t index, uint32_t start, uint32_t count, uint32_= t flags) " index %d start %d count %d flags 0x%x" --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340320; cv=pass; d=zohomail.com; s=zohoarc; b=QwTQFTQJ1rIRt3wcPQhZtiZIDEA/PH25dsCnPsNsXhofZTQqyumOLydX8AxYiqtFMwWzPltRy0PIzWxSFqZPpe/EBXGd3CiYCTyizhnr9MpT5LibFJvPXiLtO20ZavuBhDBNWaVrrgJeBkKEVAKvexLQi5UVbgcmgVOMjz8CgKc= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340320; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=YxMHKKbdXd8YZJD0Iy1oIU3OITBXEu/+2TxCD9ow/fM=; b=UQyJTGn2+3pVcJK4IZEMPgqBlmTUBs/rbYgLdAc6dXLzen6p60nbVRs9XQsGwajRinrgmk9e4+lHXbkoBlzgP0S3zqn74WbtuJfnxmSjO6UfZXSBUd0cR+7mXR8+qXxomew3281a0C6rx7ZDjM/jM722OKLMb9U1eUAbVT+Fybw= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340320518309.6355056884331; Thu, 19 Jun 2025 06:38:40 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFNf-0004GH-N7; Thu, 19 Jun 2025 09:32:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNc-0004E7-Cj for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:36 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNZ-0008EP-0K for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:35 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55J8A061013187; Thu, 19 Jun 2025 06:32:30 -0700 Received: from byapr05cu005.outbound.protection.outlook.com (mail-westusazon11020121.outbound.protection.outlook.com [52.101.85.121]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 4798mykwq7-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:30 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:24 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=YxMHKKbdXd8YZJD0Iy1oIU3OITBXEu/+2TxCD9ow/ fM=; b=nqQwToU8SM9qgVJgKyHwpPXqf4Qv6iSpLiJyJiZ1f1NNnT2emTsQhdFC8 Xe4ozsAHiFN00sHOvY7q3W73gkTjMkRAigAbz9jo86nsPCgZ0LqML0unpe5mhWZd uKhM/oik+YrUxNbSjMASaoQWB5CqeGjXAjez6X6MVL142D7Ta3x3C4ENlzU/fIdd RKvl/nogS1Q8quvjBCU04QZ10jC2S9oB/QHVBzkacY0p16zPcNNaUeltOgsK0LjW vpm6jMq3ERYsRRYVScUPjkIR1bNUZQDcTAkujWXj7HTWROD1naoeSuP+O9lJeh4J RKVLFSga1zWFHV2KlCebi8C4Omh8g== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GZAZ32P4CSumJLUtxmosSm36VAB6RuV/RvJkWzRZCdhcASCc4Jv2U0fzfkHD2XtOhMv/8gzQiM8jDEzNaqP7QcGlL8gKZEBswFSNWULxJHwwWDAw6sExKyFbCRw4YNJFEtGCGEkboN9Jls8IF7cRt7uzqFMVTThqJ38LQm0gUQx6cXrY8sZE+d9/najCqUt4oQYh3QVTGdU27XnYaUJ0l97yQkRjxlx3qVQxn/qlYO/c7J/j1vWo5WCUEllK3WXDB1KugXYN76kRo9XPnrbv62utrOoNTIlPMm35R6GV+ThWuL7p7pc6TdHErPaYHjUBGFvoHc3IS6nkRb90+4qJdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YxMHKKbdXd8YZJD0Iy1oIU3OITBXEu/+2TxCD9ow/fM=; b=VxojwAHMuh0pMMqbtG7x8TFTBsCF0Xt4x1K7ZPQ4Mx5Ldl9Ybce7VAb6KPMZaBW4RcVCzHpSz/jHNT/A9o/7EMkTgNBnMqNc1agowcOuO1RGV8OwXU4HnTocjoXQIbHidhyfWyW2a2hbHzGX2/qR9uGgYrZaJO86iIr3njm3X15nq4/WrPeRo2h6P/7/PZLImx4Cm62CPYYzS0yyMDf9H8dDQQhyyVlrz2pJFNhI8Kr2lxi2KgcfDJnrvW31npA3V2H0sYEkqctSzFBxJg1efXxMqXuGsPuuF1eml3mQHD+4Mk8gkdjslAO36136k7UJDL3AZqjWRPu1ad/2mCb8Xg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YxMHKKbdXd8YZJD0Iy1oIU3OITBXEu/+2TxCD9ow/fM=; b=qCkBNK/XB2nGfZlcOrp4qsHaH3GnrJQ0Dx9nV858gEzkvNGmfcp/D2HOzyIL7YpEa1QAiEodIKd+KdF8WdLnBvcOb8R5sEd2anOEyOFuG1UvWvVDfW62IqRt96/ZBVctWYgoM0zFcsRuyuXBMIIT01llGM2yOzSebWidcf7ty6oBX5E/jkv6rHo+yPPY2WfBukr4gTD9pdQAJUM8ArBHzDek3AW5vFVZ56AhqeOroDoapGsGiPtbhzh08SkQ/6eW3NTABz7ALzgafazyJD5+ZEm9aowyB1SZs2ZuanT/EW8l9UtcRD2kqXWO4pp9affA1NKOoY5Q1R1I6X9FqnSqmw== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 10/19] vfio-user: forward MSI-X PBA BAR accesses to server Date: Thu, 19 Jun 2025 06:31:44 -0700 Message-ID: <20250619133154.264786-11-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 539f24a0-847c-4407-d446-08ddaf35b97a x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?A8vyjQE6adoyOTchixbTEmGa3d4E2jKXsKyHLATkarb0MzixaowAAEQOJi6o?= =?us-ascii?Q?/DRnIWjO3D5lcSjq9/yUH75KuBbH6bp9zM+MJhsCohK7tiBZ8UisqCuRMc9E?= =?us-ascii?Q?8obb7/aGGMvZxgXbNKlJx0hmJWDdmobay2F4HTARiivo+BRGmpwzvz4KT8rx?= =?us-ascii?Q?wOjsPc+cpMoGe2xDhVza0BwdmRRR9aNhknpFMOrVzGMywalxbsh3G9CEhA7n?= =?us-ascii?Q?D+nDFYDmtU8XEJhPHzaOglZUBq/S/RgqfTutlHKw6pFNYQCtwhbTUXfr+Dqt?= =?us-ascii?Q?GdDy3RVtWzJezWDjJkBpNltvFwwANu/afYy8VrTyjZWDfaXXDupPiZ0DZw5D?= =?us-ascii?Q?iyC3d+p+AuFkUjG9Lz2twHdhKD1DJ5uwUY7UClyeuUK+Flmc0xJeiUAyrZnP?= =?us-ascii?Q?HwbKFFX9RPIEZgWe464nmefsLAZVZuxd3k62vKQsl90PQj9kJf+dQMdcpNsk?= =?us-ascii?Q?xWMwpWoKO+vfnvEXjhnmNb9kZjl17mv77noLwuPMPShDQJ746DSabvt15fpd?= =?us-ascii?Q?B2NWilfl5CJaBm6tfE6fxOdRcZ47rNdd3elF/+gwPHJwWDOXOkB164/UVYKI?= =?us-ascii?Q?yrL286NIKsISct5TsZAfScF5u3Gvebp4rQ0RvOSqW5M7rq2zQNK34V1OYzKt?= =?us-ascii?Q?zitLKOdh0ejlFoGTMpPd8RZig1iAxUXF37Stl0L6H5sAqrjKF2InKIIQlnMn?= =?us-ascii?Q?ghe/Zmz69G/m3e0Ik421v3QfTVv8RF6ciIYm7qQUpcEbfCTmOe2GJEAXw7nl?= =?us-ascii?Q?PFx37HoHjgE4WneN47d/cTf5v0ZvOQitKJyfnqRcjO1/M6A4fcKDKKlMNksf?= =?us-ascii?Q?JnxMxHvaEtIKThrdGJM1chCv2ykOR2dL67zeUugk4Ml2Z4PToJAl3BLcwTjq?= =?us-ascii?Q?bs02EC2h2YnErpA0IMjhZeM4Q8ezD+6xbJ4omDXTd1YpXsQ+st+o6svLXmY4?= =?us-ascii?Q?G1sT0EAzKRl2POVamEq4uMYVIpCMtAUHv/IaU2G3zLU1o+B+oW+vqkV9+c7l?= =?us-ascii?Q?CM4TSoab3SVOZD9upNSOnNg9m3R1JqUkQM78MgqcngeXfp7EgLVrixa/MCmm?= =?us-ascii?Q?00xb2UT5gwmy1b63lOiLUYmBO4eWmDe2SdDJ8/0VdmPGS0n1+DAOPvYdH1be?= =?us-ascii?Q?Sa3OfcMPqWrJ/zjBfHz/AzhNXb6q/DRzFsmaJGvR88p39t+PuoCDyzLzOq3f?= =?us-ascii?Q?O8sVGhFnXQJlbYG7fDKHHCJSlDnvcOQZtzVm/DCtimWPAwvQn/+OX0iKNJoX?= =?us-ascii?Q?NC84MFRwDt9Bbwx60H17d5mbnlsF4UP9wudTEdoDdCepkB7dhVX5N2bCU+pr?= =?us-ascii?Q?dY9KliSn9MvA+J9+enpqZL4kVsuGDjfEZCGWJ/kAPNjaFziyNBtoFxwi54hc?= =?us-ascii?Q?BczkwnCwaGC4/T4tYDW9UREda0/ljRxPPuGgzAkiltkLtM3wQ/JvX/uexjOa?= =?us-ascii?Q?T1kLX8vxJaM=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?VkBTAZBAP4c9TGLSaD/PyhsAmuiDK5rTSOIFzwWRg+k02QajzeBUx+38VcnR?= =?us-ascii?Q?jhH31QUhY8QDok+xwpeMZsf9gXF0rN+g13UXYxayxdhYxzHmDbRuJdGRZb4m?= =?us-ascii?Q?gr0FGZmRDol3cSkuJDTJebYjfb2VjolIW/aV+PJffOrA5Dqk7+HgzrfJOx9t?= =?us-ascii?Q?766KQqWe4z8/nAAsyS5u4wNdlWGQsf/4JlB6f30rngF5gMpsDMB+oJr5Ky4s?= =?us-ascii?Q?wyAyrLxNQrcrNN39hTaxhFh7mVaJa4kZbUf+1J755N1n9ayA9/Lgoorrh9Qk?= =?us-ascii?Q?tUd9vmSLNuszhvi9S8DLrQuCYBmmaziWHpjWHz7wmbb3Weq3A96BZYqYINy8?= =?us-ascii?Q?0cp+07K4tycrJR08G9R5BAvQKRiwrsiei6eaUmCbdiL6q6OAhdFEGIfpC4mD?= =?us-ascii?Q?U0Vosa7/XVEczsi9DjE5g1D+1124OIqk3uqvADRLzdmCL8UExOF3AQIQ3bBb?= =?us-ascii?Q?FJdTXbDICxiAamWlWmpzAmAX+6IOUKS07cLDOR815oRJ0Kj8v8gmc/jjpmC6?= =?us-ascii?Q?YquRTIfg9k3OZHi2mEfjKvJ05aO6MeKZ+YR60KEtXkIR0G0hAonZt4e1fUep?= =?us-ascii?Q?fauJQiis1bx27oKUwdgMQh1XUy7XqLipc0zJTwrwTRM3ikNncDiktjo0TmrU?= =?us-ascii?Q?kIRQnjOTaZXB4u3pTK8rqjWcC+9pBRot9ov03BxgNdEHQNceaCUWqZMa2FK/?= =?us-ascii?Q?IJp07Fe9y9ucnVYQ8B1Du4flFjOd4FXPWtvGBZAz3T0L6U57BAz9UXeNgaOq?= =?us-ascii?Q?luKKT/k8r5QEp/VHfr3h8bfO5aQCjmS9Xve+CB7vy82ONRDiA082OZySFkCB?= =?us-ascii?Q?oGbq6qOHt5DQ86+mdb4If7ultqVCQzg6Piy1k/ByjB9rg+PA/80MIjwH7pkO?= =?us-ascii?Q?KaeCMma2TNP9b+iABu7aThyYTaYZq3LxlgBvaTo7PcQrkuI9qZ+SvMovp8vY?= =?us-ascii?Q?eD+Te26zUIlY7o7vcJCHLCv3Qyjsrs8baqUIghFkSa2br4uxGCpHBcLrVXfo?= =?us-ascii?Q?rY7nQczawkHM49obY26P6UXlEyuVsZ8jwTTKKkGTZXYR5ZtBV8HYSbMWekqj?= =?us-ascii?Q?fx7zwrCrADjm4ZnY5lgdt64sTk5fZmMu3sIzfRmlhg3THc1RD+AypenQ3NZF?= =?us-ascii?Q?X9r/JN4PII5d/rNDXp1U2GAFqsyRo9I+4wOL7GwTIfIKc8McLsqowTa7fY5u?= =?us-ascii?Q?IzeWTuPOJMbWzKTSh0xO8W08ie3xfZcXa7qOpaQLHuUT58plmyvuFwYvsl0a?= =?us-ascii?Q?MsUb9QrgZUOW6S9DVH/wXshaQrs+O5v7lDGYxYf6Kv9k5pG11T2UNaZZPtRf?= =?us-ascii?Q?W8dOoRpy16+7LgD0WsEnMzE4SrBM/wcpLjvKJGDMs7f82qbhYIOLCnneae9L?= =?us-ascii?Q?ZDFpms0IR6rg9fkw1DLqLP8KDEKV3uhLzLXCEV98+oblsLdv7lIylhQ2HANv?= =?us-ascii?Q?tZpopICt6M8ljx7+hZ9ZDGKiTnOJr/4mqThoxvi421FLM2cFsl+4UFM+2M2U?= =?us-ascii?Q?Zlf2Je4+ZJZ8vEpS0a1uuzSL9V2Z/QPDzVbHvdiJePSBtA/SXcgtCGGsFG5j?= =?us-ascii?Q?iEFd6WVpuQ1YCAocvG3Oh0+StBKaoO7JS/BKQK31?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 539f24a0-847c-4407-d446-08ddaf35b97a X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:23.9339 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: NV2bDDJ5X8Ez515DV283rzrPWVmwELFe7dTPVNzEmmsCZB/D9MMYT4Uxh1mwxrOltVzD3+e0HSEnRltLAtnNyw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMiBTYWx0ZWRfX/8xcAzC06sm4 62KYQqO3oazz52ybxJJPW4GYSfjKrt8kMt1ZZtZM1Eo4jnZY2q7FDD8Xd6ScyQ4yy32xoC2Txee S09HB3nc1+178qhzzMlxWT/3t7kLjO2YJSfVHP8ELy+zEXsSgoNb6rMzhR/6I43r9EkFLsb4rHL qiEfePFv5NFYFrZ8KocC4E49c4pNQqXbWQ76ugyDv9cg+iGUt6BPWbZhNJ1rFVQND4Clbep0CRK 8wHATJIbWCX2of8MwyH7ORZ0l8bPWI7pa3meZPbEDvy10KfIjKGbRYFPq/38o10TJ9gdugzyphf /2x7NgY39IgdrL7dTZdbIHAHSFGyuIufJTRpKuxLISeIO2tZuIP9zSg6T+dKA1iH9e2waewYQHA IrInYBGkWTHjs9YIShH9XiYry/Pns6uk6l4vIa3S82wLXE3gJ30/HqHgP7xDupnJcsfT4qjn X-Proofpoint-ORIG-GUID: jvwc54tvSjP1BzAQLBgJ3Mu16WTePz-v X-Proofpoint-GUID: jvwc54tvSjP1BzAQLBgJ3Mu16WTePz-v X-Authority-Analysis: v=2.4 cv=LIxmQIW9 c=1 sm=1 tr=0 ts=6854116e cx=c_pps a=Yi5XoY1Pvk1/bV4Hj+lk7Q==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=P96-NN1LHiVJMZ2xip4A:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340321743116600 Content-Type: text/plain; charset="utf-8" For vfio-user, the server holds the pending IRQ state; set up an I/O region for the MSI-X PBA so we can ask the server for this state on a PBA read. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio/pci.h | 1 + hw/vfio-user/pci.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index d3dc2274a9..5ba7330b27 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -116,6 +116,7 @@ typedef struct VFIOMSIXInfo { uint32_t pba_offset; unsigned long *pending; bool noresize; + MemoryRegion *pba_region; } VFIOMSIXInfo; =20 /* diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index b49f42b980..c0f00f15b1 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -24,6 +24,62 @@ struct VFIOUserPCIDevice { bool send_queued; /* all sends are queued */ }; =20 +/* + * The server maintains the device's pending interrupts, + * via its MSIX table and PBA, so we treat these accesses + * like PCI config space and forward them. + */ +static uint64_t vfio_user_pba_read(void *opaque, hwaddr addr, + unsigned size) +{ + VFIOPCIDevice *vdev =3D opaque; + VFIORegion *region =3D &vdev->bars[vdev->msix->pba_bar].region; + uint64_t data; + + /* server copy is what matters */ + data =3D vfio_region_read(region, addr + vdev->msix->pba_offset, size); + return data; +} + +static void vfio_user_pba_write(void *opaque, hwaddr addr, + uint64_t data, unsigned size) +{ + /* dropped */ +} + +static const MemoryRegionOps vfio_user_pba_ops =3D { + .read =3D vfio_user_pba_read, + .write =3D vfio_user_pba_write, + .endianness =3D DEVICE_LITTLE_ENDIAN, +}; + +static void vfio_user_msix_setup(VFIOPCIDevice *vdev) +{ + MemoryRegion *vfio_reg, *msix_reg, *pba_reg; + + pba_reg =3D g_new0(MemoryRegion, 1); + vdev->msix->pba_region =3D pba_reg; + + vfio_reg =3D vdev->bars[vdev->msix->pba_bar].mr; + msix_reg =3D &vdev->pdev.msix_pba_mmio; + memory_region_init_io(pba_reg, OBJECT(vdev), &vfio_user_pba_ops, vdev, + "VFIO MSIX PBA", int128_get64(msix_reg->size)); + memory_region_add_subregion_overlap(vfio_reg, vdev->msix->pba_offset, + pba_reg, 1); +} + +static void vfio_user_msix_teardown(VFIOPCIDevice *vdev) +{ + MemoryRegion *mr, *sub; + + mr =3D vdev->bars[vdev->msix->pba_bar].mr; + sub =3D vdev->msix->pba_region; + memory_region_del_subregion(mr, sub); + + g_free(vdev->msix->pba_region); + vdev->msix->pba_region =3D NULL; +} + /* * Incoming request message callback. * @@ -144,6 +200,10 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Err= or **errp) goto out_teardown; } =20 + if (vdev->msix !=3D NULL) { + vfio_user_msix_setup(vdev); + } + if (!vfio_pci_interrupt_setup(vdev, errp)) { goto out_teardown; } @@ -192,6 +252,10 @@ static void vfio_user_instance_finalize(Object *obj) VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); VFIODevice *vbasedev =3D &vdev->vbasedev; =20 + if (vdev->msix !=3D NULL) { + vfio_user_msix_teardown(vdev); + } + vfio_pci_put_device(vdev); =20 if (vbasedev->proxy !=3D NULL) { --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340210; cv=pass; d=zohomail.com; s=zohoarc; b=B6Pb558wtYYh1QotxIs+BiaeDYM7zlmZSqKYKChdPoJSK7S+/bpBVPXa32tGuHikL/Ab5cKaiNJ0QHt7Lmep5ojUw3OM3KMwqdqtQ88bGtOybthoqAIlq66kq+59YgDag0/l06hR5qmqT9+gpr7BrllWenj9FaEOoXFzYAE7334= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340210; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=OAko/TheJzo+jExbY7mQx1qoRlsB8FX27i+Yx++LBMQ=; b=YkNbyLrv/5TIYhdhp1Yl8lBD64MzLARWtaKjFsLqvLl9Ekzvyi3t/AULjO5pWPy3c1AVP9Y/rPT9fvtORg6SP9LN0caVgh0YzFihYYuh7uNX0KdUtWXjjirw+Lbw3m7SclmX7XZx+T8Rgp3ga1d4RCik+TeHGT/Ju9qPSTVNq38= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 175034021037816.42298858439176; Thu, 19 Jun 2025 06:36:50 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFNh-0004HM-PS; Thu, 19 Jun 2025 09:32:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNe-0004F0-2w for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:38 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNZ-0008Ea-6W for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:37 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55J8A062013187; Thu, 19 Jun 2025 06:32:31 -0700 Received: from byapr05cu005.outbound.protection.outlook.com (mail-westusazon11020121.outbound.protection.outlook.com [52.101.85.121]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 4798mykwq7-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:30 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:26 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:26 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=OAko/TheJzo+jExbY7mQx1qoRlsB8FX27i+Yx++LB MQ=; b=sZGoVnWe3Rjq4vcadU0eaGxdd0FMVucO8fosXGfVCAc+ZT/crYZtfZhRO 3LBbSbxkLguR/cXXJsIi2bajV4czI4iB/8ugbw4rXwiLPJcRJ6IKJLxtVETgOwFp L8/+WkPZJXscojV8tdj+Z0eZhKYLxtwvBJEntqYWXF6nvuHdmmbR/UVviFqkpkNm XcH5LbtlrJABzf/YrsDYCyNrrz37mk457MLg6hG8w14kk0EIMJ5/DnN3Rd8v/29t i5vmMtcN1Sto1B6xrmPTEzdcsYqmRRDeDdug4ruZRVUycEALYWc5mutG1wu7px1w E4D8wSa6tW4G4vfCvQs3ZF3Yxyv5w== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=AQ4vaLbPEEcYxKGU9HzNJZz/F2UcXOW4FFXZ80htZzlmpfavLlHRLbm94SBFEW4aTYesQnYoY5vr2cTXQ/WSM1iXzUS60G7rWnrXMDw/XrwDKOSlQ7/HctjGwS+cWbCA7v33IysJ+9qfoytABCU59XFRsBCEJThcZNtix5t9Swip4OKWEmw5ElfyySA6htOKt0JlJjhQzzgvgzRdEy1ZmX1rKIMDhYdSOVtFilR4Ht6GmkFGo7tHj8UDG6H6cJ7gjtOTAKwfrTrG/OANV9lXezmkwNY+DAU4zHEHpfa3ZwOUQNPTeRRY8h/aTxw8JpcvFeXN44shRhuxMPPY6Tp+jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OAko/TheJzo+jExbY7mQx1qoRlsB8FX27i+Yx++LBMQ=; b=k+4+ybRO0zp1f1fjXSGw/MVPSXrddrFl5LeG/H6e5rvmHGWnIFMB/WKPG8GoLNH/KQQtbyHlNOFWOKHl0xeJwWX/YyS+kt5sR50U/+pl5LAhOPbz3ah+CtebT6vbRp8jtKJH7IaShi27lXwtjg/zL2z7LJvoia0TSdf5N59BWu6tawraTY++F6NV+/obU7iXdFf+sxMFGs9ahIC2u1DygICz2uzv5Swyfozla1WNGkqgsxPLV9VYe7bEoMYLB6mrkNzMnZanTV6a3w3jtU4KbU1YJExrksXOrrQBG2uV+ek2gKAlbQmWz161oNbM8qlNNa/s2a/J/X80pSygvBNnyA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OAko/TheJzo+jExbY7mQx1qoRlsB8FX27i+Yx++LBMQ=; b=nc3A7ySfASC0JTKcAF/an4ij3n6RGDgzYTQXISs+7wQ1QSLjj3grZQ2tm3IQkCLXOS/RDCbHKrpQxW4t9LhjNS02uzg11aWVroLJ5gap4rtzzzPNFszYnl11lWBhzd0xSj5yUYzWSTs5RZMRiy7E7nG1jPjKIVbWXPtpd5d69NUVJcWhLpBvtmtIGkszyHfLz/3XpgbreIUi1/BEz0x8I2lmIkudJe2fGyhSGpUKNdZUHzb15BTfjncSTafJjKz9y14HHdJqD1Z05tmeCUDifV8c/7hrUVwwBH/m/toSYrxcg1q43k8V8pXwiFOfk9ty7+V7RmwycPqpoA7sV4jnzw== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 11/19] vfio-user: set up container access to the proxy Date: Thu, 19 Jun 2025 06:31:45 -0700 Message-ID: <20250619133154.264786-12-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 60eb8384-cd29-4cbe-7af4-08ddaf35bad5 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?YgJyDDTrYcRTTIRC3qktHs5T4xZa75k1SjzuxyDx8FiLxPe+lgwZbOohL6Wk?= =?us-ascii?Q?PeJWCJ7ZjltChydKONP1JZRN7i2RkXjqvkSo7OUHiKVhmhUkQvAzTQPmqPwP?= =?us-ascii?Q?h/piQfnfzkt8QgqH2JBKjy2XbTq/lzHsTzBhs3fcqcmVJ3TwM7h230suh/9A?= =?us-ascii?Q?aGcrCKVauhY9yPe6Rm3Ccj6s5qfaUNuaeQf3WZmrwuNQ0QH5yfDwJUxflhYo?= =?us-ascii?Q?moldQuXumZoinIfuWj3Qs6X9SR7uJFFwJW8DJPx+lhcthwJpyQ8vV3Kt/RJ9?= =?us-ascii?Q?Ksomk1JnVeM8DpBVA4WvCPQ2eS7NyLKKCM0LIh/m3uPToDoAnbWn9OQ+7qyI?= =?us-ascii?Q?txnxmBgKX5x1S8lumj2wZKXB4v7moCG8nwKAH2RpibiGSkuP92WIizzvFvgw?= =?us-ascii?Q?jT90d5VGClYtb229jPcwVMJVRYaZO4uIvZt1rTg8vHTFT0pLgmA0h7Zz/Ukh?= =?us-ascii?Q?nPhW1/iHaSdM3K1HK9iMd7z9Xnfu4UiofEKdrPBQS/RthkOzC3tEXhtaD8+C?= =?us-ascii?Q?BqTxT6nnJMJqHthXy/HHnaWItWbofVYMwHeRt1/l+3mqBZd1b8GDKgUy4XGZ?= =?us-ascii?Q?OhLDK0yR8+AAXWNWErNa+5vzQUCa0vM8OVeoqH3eE4OCqpZZEpgStOItJiE7?= =?us-ascii?Q?JrMFUDa+farIYkOhX4I5oWYiwAAXQctfkWPwy7tXJXRjmQ5f7mZpckBNqC9d?= =?us-ascii?Q?5tTEt+4tqsnEJjyf/IsdwnBfpjAW+Obj21x1oPiX9f0uhL83MqwQ09UqYfdb?= =?us-ascii?Q?HUO1vgiHgE7rfTVJt7KIEjlmJ+2bb6N4+HTuJsKyh1ddZv28oc5SAiyETDm/?= =?us-ascii?Q?SvQfhlFY/8MeVCEe2kM5lzT8p5FxnqqGZkYVvPDGupgbQm2jMAJtTqSXZ19n?= =?us-ascii?Q?kF0ev9A1LqaZ3pz4QN9vaFqJYxhXCXi0Wabt4JydG8t40dUKSP2SNquXZz/9?= =?us-ascii?Q?WEAe0XuquWbA+zi6aO1IY5smbgxc0cDRGLugj5uwGt2fUk/WdrNIsJzDs8hr?= =?us-ascii?Q?3gNMmHitBBpudH4Y09qk2X2wlxBIDxF05HERc1W2FzDnYvSfq+lJPUF+Mb/m?= =?us-ascii?Q?xPi4UnzQLCc82HwqxxpmahRcHaiW8XM2C27r8pVn7Np4Z8mKTTXHzkdbZ3kq?= =?us-ascii?Q?WPIu/IoAnfu0Yu6IvID4C0RzEPaaFqegdCISV9HNrULV4C43L5miM3NLw9B0?= =?us-ascii?Q?Ys7AZU67/+b53uNSQO+NfYRMF6erskyjqbEHlx8T4JdAVuPHESf/+S3luaDZ?= =?us-ascii?Q?L5thPTpgS6zR2jVRV05ldC8jnd1wOmvYNXb6r5aXZV1OSvF6qHtVcU0SBgtI?= =?us-ascii?Q?HndvWBNLEaSBUTwUPP4e070QQ3DYoNaX6FL3Vfb3XzLTzHr/yVawJzvnc7vk?= =?us-ascii?Q?rntnnVuRfQu0mc2lI647z7duu8OXOk8ceHHKZ2mCMJ4RZVeOs+kPBdSVqK+X?= =?us-ascii?Q?eqWM8iCdZgc=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?L4CTOr6xBmDjxGLFsBByCK8faDYlIYPrD5PRnxBtcXmeSrDhCSfda5vg5okb?= =?us-ascii?Q?T1lwRxykfB9AjELcRVOYNsNiJCLWTKWNoDnuFthen0+oVQRWk9KV5I5Dniic?= =?us-ascii?Q?Ll/3TPGvBTK4n0A7DVpk5zBCBUDAU9vSFAK9YuvPoPxUBDzhKQzm9CiPuRUj?= =?us-ascii?Q?XC5YaDSxDNm7NLEe6LpUcT8AT/GMKKs+UXMiA+Vjl5oOMEkxFku8Rxxx9DmM?= =?us-ascii?Q?OCN78pv3UMHKaey7zcmBYbP1eIgFA0Q+Vxcbt7+m7NOavDddmIRrF/SqTAv5?= =?us-ascii?Q?M2t/62UWu047npUH6AkXCbwGpknUdj3lUM1xrs82NDihMMxYInLJTd+acRAY?= =?us-ascii?Q?1br2KvaCFima/EO7nFUM3sYhFADRnZ2tnRcjKZUCFLAkYPXJhRE0mXnKLRFD?= =?us-ascii?Q?REYfO3q+twT23w2SXGn1D74Sm3MG7V5bcvF0mWN3RlufWhFjzO3/AcDYEdzy?= =?us-ascii?Q?hSDutRANpJuG6Tuq/DaV1OYzatOgpAfJnmMDFq35ooBd5QuUZlfPwKfaUsnb?= =?us-ascii?Q?Ow48nLtZRliWJkC6XvAikCxTjVq1M0IWekSZhgg3kVBoxVGNT+GsIrKF++ou?= =?us-ascii?Q?p32OZGC83sqV3LDLRydLoAYSEWoRnBPZ4zrHvpOkrosgmNhfh2Fl0YKOKZ2L?= =?us-ascii?Q?El5QhRwxVpRhz93nS0/pnMBTFSRwNF8ubvtTC0vRj2fm1Ily1yF9E5BrgaSn?= =?us-ascii?Q?+gDNkZ79QUvKHpmglwDVKAiVi29WsQZh633Hks5NqXT1h3Nd18xD+WC31ZU1?= =?us-ascii?Q?0iJBPW5bA44g+VoLw8MxO2rQCmFzbRXxsKXmMTZ2NFQ8w8B/7H5SaXn0CowP?= =?us-ascii?Q?vrJQao5IKlh57YX7FbnlTOWXrhO5C3jEtlzrwqTq+ZhvBM8/s+6dw353afO3?= =?us-ascii?Q?6n8QeBR70R5nYDsj+geft8dV/pw6wKU5yMvdGmCbagLiquw2eyeaviuGDHqK?= =?us-ascii?Q?+YdmKayddmNgAiz84tyYVmN6Ur3iFmM/1DOUWdKQOzRLZAtn9xaSo9FFf9TI?= =?us-ascii?Q?hFW3L2flGnRYcvgsqsjLC89Mfc7hAAvQfmhB/QrrGlsu9TL2DiPyTR9N7kNk?= =?us-ascii?Q?C2GngXK/MtpDt3m5ELLI9DZzw2bSY3+edclmVHH/UyUikPqwVgnfZeKi6rLl?= =?us-ascii?Q?c8m4dp7IieJJJ+c3q9sPxFzcHp5aieoUTs3nOopv9bFwZNXVxBDMOs+YeHLh?= =?us-ascii?Q?C8Jh7qWxEz1JH4ZdcRx143jDkJIwcHxEOqqqPhuu/KG47glFYLjNwugrltTY?= =?us-ascii?Q?vSVWMpgr9fFNwz+3zOR7UjyRQfI8I28CZ/jbv5LxHjMt1Ki2j73RJk4eUdHk?= =?us-ascii?Q?sLZ1lDfpyWb29Ve+Vkq6sE9IlGbfrl0JC0xgGp0cp3ijF8zXlMuNiRb4HDFA?= =?us-ascii?Q?5HfOxxCx6GYixJusVuvPDTXl2HQZ7ogJmIn3O/78pujETdIxwyX6CUkvoVwp?= =?us-ascii?Q?3faiuUeyqTgSn7m+r1URecdjczgrcTsYT1gaLvHR8sYrHvND5q/kCZzFeZRp?= =?us-ascii?Q?dKYEo43kWoLP9q5tZ4WABXnE6sSSu38g/imC56a/XLPdTbqE4Xx6ou5r5Hn7?= =?us-ascii?Q?PZ+NQryY7+tbxogZBi5OdaYO1B4D+veZ6pLvQ45f?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 60eb8384-cd29-4cbe-7af4-08ddaf35bad5 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:26.2353 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QjLDyXK34r/gpOeIH/cjZMVp3QL2jhxhqITAfnBefzVKI+v2Q29qmuQSE3aAGCrGvViTIuVQwnuikA+JkF/L4Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMiBTYWx0ZWRfX6+DBhrQ8FH33 +LjkPez1KFy6+nv1MU9+VtAOyEnkzUs/xHubGfPl2g/Ora45fZqMghUyGt9YISwMGmr/gSey38k Udp3JbLPHYNEgYgUe6vHPYSa7xZYEDLDiLLdlXfZEkQF5sSe3TnrSmPQXQAStCeXphaCqaFTjkH cKptEpXJtCFNNZyGzUys+2knSWAtNSnLgo+KAAI9abPVRnFiDjjT7gSci5W3LjMWUd0kg3xgNyn GpBmNbfp2WYgZY8u9vpa7pCLHFLNQ0Ak3uQCTvOcQaz4n567MKxWzgBcWDuQwSM+8woF+fHbPkj QsbxY0QWp3LRh/W93p1weGcyBa4JC4cdV0g8w/To6OGZqcLDqINmLBf/P7vyxDXPD/25V22CQ6m Vaj5acu6JOoqjLWU6D3AU75Lps0I5pPOR7qL+CvL0flSsH5w/li/s4t1XBjKBLMQdn5YmdYM X-Proofpoint-ORIG-GUID: zDg-rph7kgY659cf1JE-NrII6GqdavAv X-Proofpoint-GUID: zDg-rph7kgY659cf1JE-NrII6GqdavAv X-Authority-Analysis: v=2.4 cv=LIxmQIW9 c=1 sm=1 tr=0 ts=6854116e cx=c_pps a=Yi5XoY1Pvk1/bV4Hj+lk7Q==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=7VGmajfop4gqLxerqI0A:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340212034116600 Content-Type: text/plain; charset="utf-8" The user container will shortly need access to the underlying vfio-user proxy; set this up. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/container.h | 2 ++ hw/vfio-user/container.c | 43 +++++++++++++++++++++++++++++++--------- 2 files changed, 36 insertions(+), 9 deletions(-) diff --git a/hw/vfio-user/container.h b/hw/vfio-user/container.h index e4a46d2c1b..2bb1fa1343 100644 --- a/hw/vfio-user/container.h +++ b/hw/vfio-user/container.h @@ -10,10 +10,12 @@ #include "qemu/osdep.h" =20 #include "hw/vfio/vfio-container-base.h" +#include "hw/vfio-user/proxy.h" =20 /* MMU container sub-class for vfio-user. */ typedef struct VFIOUserContainer { VFIOContainerBase bcontainer; + VFIOUserProxy *proxy; } VFIOUserContainer; =20 OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserContainer, VFIO_IOMMU_USER); diff --git a/hw/vfio-user/container.c b/hw/vfio-user/container.c index 664cdf044f..f7c285ec2d 100644 --- a/hw/vfio-user/container.c +++ b/hw/vfio-user/container.c @@ -49,15 +49,28 @@ static int vfio_user_query_dirty_bitmap(const VFIOConta= inerBase *bcontainer, =20 static bool vfio_user_setup(VFIOContainerBase *bcontainer, Error **errp) { - error_setg_errno(errp, ENOTSUP, "Not supported"); - return -ENOTSUP; + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + + assert(container->proxy->dma_pgsizes !=3D 0); + bcontainer->pgsizes =3D container->proxy->dma_pgsizes; + bcontainer->dma_max_mappings =3D container->proxy->max_dma; + + /* No live migration support yet. */ + bcontainer->dirty_pages_supported =3D false; + bcontainer->max_dirty_bitmap_size =3D container->proxy->max_bitmap; + bcontainer->dirty_pgsizes =3D container->proxy->migr_pgsize; + + return true; } =20 -static VFIOUserContainer *vfio_user_create_container(Error **errp) +static VFIOUserContainer *vfio_user_create_container(VFIODevice *vbasedev, + Error **errp) { VFIOUserContainer *container; =20 container =3D VFIO_IOMMU_USER(object_new(TYPE_VFIO_IOMMU_USER)); + container->proxy =3D vbasedev->proxy; return container; } =20 @@ -65,16 +78,18 @@ static VFIOUserContainer *vfio_user_create_container(Er= ror **errp) * Try to mirror vfio_container_connect() as much as possible. */ static VFIOUserContainer * -vfio_user_container_connect(AddressSpace *as, Error **errp) +vfio_user_container_connect(AddressSpace *as, VFIODevice *vbasedev, + Error **errp) { VFIOContainerBase *bcontainer; VFIOUserContainer *container; VFIOAddressSpace *space; VFIOIOMMUClass *vioc; + int ret; =20 space =3D vfio_address_space_get(as); =20 - container =3D vfio_user_create_container(errp); + container =3D vfio_user_create_container(vbasedev, errp); if (!container) { goto put_space_exit; } @@ -85,11 +100,17 @@ vfio_user_container_connect(AddressSpace *as, Error **= errp) goto free_container_exit; } =20 + ret =3D ram_block_uncoordinated_discard_disable(true); + if (ret) { + error_setg_errno(errp, -ret, "Cannot set discarding of RAM broken"= ); + goto unregister_container_exit; + } + vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); assert(vioc->setup); =20 if (!vioc->setup(bcontainer, errp)) { - goto unregister_container_exit; + goto enable_discards_exit; } =20 vfio_address_space_insert(space, bcontainer); @@ -108,6 +129,9 @@ listener_release_exit: vioc->release(bcontainer); } =20 +enable_discards_exit: + ram_block_uncoordinated_discard_disable(false); + unregister_container_exit: vfio_cpr_unregister_container(bcontainer); =20 @@ -124,14 +148,15 @@ static void vfio_user_container_disconnect(VFIOUserCo= ntainer *container) { VFIOContainerBase *bcontainer =3D &container->bcontainer; VFIOIOMMUClass *vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); + VFIOAddressSpace *space =3D bcontainer->space; + + ram_block_uncoordinated_discard_disable(false); =20 vfio_listener_unregister(bcontainer); if (vioc->release) { vioc->release(bcontainer); } =20 - VFIOAddressSpace *space =3D bcontainer->space; - vfio_cpr_unregister_container(bcontainer); object_unref(container); =20 @@ -165,7 +190,7 @@ static bool vfio_user_device_attach(const char *name, V= FIODevice *vbasedev, { VFIOUserContainer *container; =20 - container =3D vfio_user_container_connect(as, errp); + container =3D vfio_user_container_connect(as, vbasedev, errp); if (container =3D=3D NULL) { error_prepend(errp, "failed to connect proxy"); return false; --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340002; cv=pass; d=zohomail.com; s=zohoarc; b=jW+nKze57IG8ox9wfoRmpZS342T5nmiQraJM5cRS4/vBsLj1yv9IrXzoI66wFDhvkMvHJuT+zD6bPqck3GUPTqrGQO7Nv6qDKZgWxeLgP66b/pAjfTOaCNrbLOgqvyjbSqKR63AZiRNyQLJ1h+XMJhcUd5kzPvyxW487UF7MxCc= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340002; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=uCzQmp+PBwY0Hsi3DZvIjqZzIdHgwj6eSlaUGkIYqmw=; b=kujoSK/Qv13qECFffIGb0p0bFJhlHOyZjucbTmgq/B47+qzfcLLZ3ATd1R/C8x3gqVnTXjTZDvfHZYLK8BhHJZfHcj8E4nA4QN1ifav4HdcwjWOMdnsVn0vGWsVnV3a8FuzZhMtBNkyUuGR2I5vbAFl8kYQEz647aOuoaMPLwVA= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340002136142.2281559877506; Thu, 19 Jun 2025 06:33:22 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFNg-0004GN-Dg; Thu, 19 Jun 2025 09:32:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNe-0004Ex-09 for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:38 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNZ-0008EV-5k for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:37 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55J7OixH013270; Thu, 19 Jun 2025 06:32:31 -0700 Received: from byapr05cu005.outbound.protection.outlook.com (mail-westusazon11020091.outbound.protection.outlook.com [52.101.85.91]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 4798mykwpg-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:30 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:28 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:28 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=uCzQmp+PBwY0Hsi3DZvIjqZzIdHgwj6eSlaUGkIYq mw=; b=QN3tYymxq1ql0mYIL+/BMPu/AVMtsUxlOC0Mexodxt91SX+hswxPAVYga /nkSSdKiDsyl0IJx6Rfpj5H930/tQEsYdm3ansOCtAdzXRVSNiyiyx4GhNl8+nEH ULUVEVcLuRafjpxkConDq9fuzte8oonKsiCFLekXG/awtznjw+c71bgBzH0+RpW1 gu2iIoHDLu6HY6Gqg23pME6WGjILaO6RhjIrL9DzEzpJn0PH6kZuhfdWei2IIR/j 63ZhihEbJUvgaulEgueZN8hFQSbl8MwjwW8r4KA84e3v0jUin0A7be6HnM9dDqNo SEPipjNBME1e/QfYoueAUr+DdhTfg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=goSv09wDJxy2w7WwVg+cxdid21aPnejs6u7RHYEgMV4VZHRFcaTgmLvHvsGL+XOho6LlvJ6YHBtvMcb8Dwi2gTAEN4PB5Vwuvq4H7vaKKMd8e2ZYgqhcECGLKL4V5SepT8mbYU2Pj6BW+fmdvUqbV6acV8GE6iYMdljhq6aX1PGN7jiJuZyntEUigBIoeiBbaBtIE0EMwA0+aBoXKN2o+lccLVIe+LfnXoAsqXn9QiA3oaQhb3OcIBZJMcuMsR/lGLKFM+PquA0CLeUD2jyg0kir/yyPv7++Jd835moz9bh65T/IhAHcO5YumC9haDrAHgHRYshuPVF2/SpA2//jDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uCzQmp+PBwY0Hsi3DZvIjqZzIdHgwj6eSlaUGkIYqmw=; b=QRdHcllydxv8OMo+1nRN67RysS1bilFa8FaC9yN7nRbBlQ5FQjZLx/cnySE0qB44ov6YNMi8wXY5kfezvwYcPOHq+kdHe3OEXihDDMZ/zCuZNWjx30HCCQTgTWIga+3R4+TW+f15ZqFNqHFI7M8jQrHlco0BwP11cMmkraxy0dWogUdS2CQIxrX+SjcBJO+hjcQ4n0Cv2D2OJjDUHb5KZjijzM+6Q1KJ0CPvS2j4i6IxPHqBMArB1xSSPU63sbq4sQvvZwJAK4go5y2Tdcmc1ZGaPZm3fHIDDFF45K+Toe4su2HuxNwkdPqymgwTAEWiwR5iYaqHU6Qs2AHmn4U1Xg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uCzQmp+PBwY0Hsi3DZvIjqZzIdHgwj6eSlaUGkIYqmw=; b=K7OvhBQvs7w8HjHD5FNE9j/ROrG+HZEx3IAIX5Y3lFDiqEi0nicoyBOudiQ0hS+0+h8IPoJ20RYqmoBIfWdpnS3TM0QojfZ7ayhA+MQ6KSBKjkGBIhHZ7QVABlWLe51UgM/iS2Qlvk9bDtAJh+OFopeXyRpPbPz3rjXjNNvxZo5b34K4VwKqe4SmmOKd1WzXzr7G0bzeHiJnMTkx7TmZgslriNbyXR9g395p21Qp+eHFCZV7LjbdTgbIeKbWfW4HmcGJa2lim0M5TIz/8vxl2lTNfvxCAmVuXQ5xVmezJvg1kEMDhjzSA0dvRR6qj/5q/h0hrKbovqWWvYFhspTjwA== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 12/19] vfio-user: implement VFIO_USER_DEVICE_RESET Date: Thu, 19 Jun 2025 06:31:46 -0700 Message-ID: <20250619133154.264786-13-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: b85614c3-e8fa-49e4-3883-08ddaf35bc37 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?9FfZhFOTxTwquYW3Mc8zKd1h5NEvUoFRNtG4qGWygOZkVLPxd5O4u5e5cx5p?= =?us-ascii?Q?J0xvFGpv+TBIFvr6oixDaHyV//6X8f3wk558LSoFcvYFhC/WUUsVHa+Ai5s0?= =?us-ascii?Q?IxaPP4yNNcWPzOTLojpF0cx3bHzepIu9GvZwlGD9WMcv5V6cH/MBRsJTZ8ro?= =?us-ascii?Q?9os7oaQEyJCmcc63xQJmjX0aEeTKvYPvTHwd+Q28PNsYRIamxel6u78JTX26?= =?us-ascii?Q?jq9rPKxzHDXi/8MW71cw70IcEPPhcyHy4QOTUehgb/te3z2xK/XCmwiBcLLJ?= =?us-ascii?Q?9c25zeYe28Qz8BsHHB8IxF2AelmOpvA9iFwpDHkC/vL0kjevVjVuAXsXm7CG?= =?us-ascii?Q?Mcl5dmZLTqTqOyYmWMr8m6YbrGR5Hy56ZgZi8qZdY8MEG3dfDZdBbWE9JRqQ?= =?us-ascii?Q?ASq765qTR9jSIqAPbDbDCGldLHYFcRrpDMsQ0d5BD2Jfh/hdtvnP+KdRTn9l?= =?us-ascii?Q?tlcWv1thpL0o+6egjKU7CrU8iDWvq4fy+mJgSXL/PIdbXn1smgB/N6itLwuM?= =?us-ascii?Q?w0D5c67f3zDlZzZAuExCqopC9eqVAooFyYdStxTgc8k1j4Xj7sQTzu6lFD4X?= =?us-ascii?Q?HD0iVKfIMEwZpIDui6pS2wmfSrulMG/uVM9YguCyYQt48kIKnEJNvRkMYfHH?= =?us-ascii?Q?e6i7FV4gIGlfCTXg6HOHsCzoMMK9qtb89URFyfxHRZsP9epWG2KzK4U1SCxU?= =?us-ascii?Q?o+GJIvJC+UTxhfQDeDCjADNidPS9WA7En2WaxUGWXOTBAYHjeAjunzgzt09d?= =?us-ascii?Q?B8kJLAFE+jprSy0rDbTNjzl639x3aiw+R0FQ/CoxFoZ4Dlokt4JaI8gqy5kQ?= =?us-ascii?Q?CfZMdHBipe9rYjWnU1Ec02ikZUmOBXfxA/GGzFwoV+li+ogC1anhB0q8Ui69?= =?us-ascii?Q?yu2rednYhRZO2828uFqOPH/AQdMXt2fXCaIGoGXWl9SQpS0p4Y4lR66HBZxR?= =?us-ascii?Q?LjSqC7EkfCGpL5b2c9M8GQ7tpANsGOisgWGyW3GPfKib7eN2h/sGOmXQogyE?= =?us-ascii?Q?Dpz3h278jvZ6zD08IG3IZN+uOBoUvtrKj+EL45mfBHKawA1H+KdvdkuWX3Ff?= =?us-ascii?Q?WtbsQaPz50oUnwDYV/UjNWf9jvnKYUm4Jih7Mn7rwVIkQcW5rEfWAMxw9+Bc?= =?us-ascii?Q?brcdLJ/rMgCFohU1GqXmeF7pLvF7lnT2tRDqFwA+QDwiJgWRKu/NDtKdgYHS?= =?us-ascii?Q?6RVlw1mCeNIIvpIMNZriMGpnu1vBu8rY4MgMBRZ8ldFiCaNlJ0qHwOgNNI84?= =?us-ascii?Q?0NqSkxgO2Ja7ipi0pa2MvqU9hi18xg71mLaeF7enX3WWLbvh6LcSI4AIwS5Z?= =?us-ascii?Q?oKOOB79NYPANgYnUEemG6vO/qe9WnCk/oXp4fKLI7Pt0UzKCtRt99g3x0wjM?= =?us-ascii?Q?oqKxp9XX4cmWiJLV8gaV1DW2LT+q1oFMkSgm39ne32SZUKi2rdxkR9BbKA8w?= =?us-ascii?Q?EwCnnPLxZjk=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?k/UV9fpy6u/vDbiT706DO/2kA60Osvndpig1oAqs4aME68VpoDc4E1bFceP7?= =?us-ascii?Q?iSHbxV/sm0mSD/DuXkCEwZofvtz4HCeDqhy9+HXRHD8lxXwZCytoQ3vYT76z?= =?us-ascii?Q?4kH01YzS893WEP9Z03JYjfohxkvZ1FCuavfZ8AboQoSSTHALv239vpwSkoX6?= =?us-ascii?Q?K/bIRMuBVFX7zFA+XSN8iLwNHkHU4cBlAu1zNmmDhq3Bb9E7cKC1DmjdZHD4?= =?us-ascii?Q?MpVHPyEK1ENtZLs6rFsbN2+QkR3qvdAsAM1WQ37dOd3nujg6uv2jzVz37C3w?= =?us-ascii?Q?smO+J4uq916HjvuqYyypTU2HpRseYMivrfSIblYpxUrmQ/ryPkTBXqe7W8/a?= =?us-ascii?Q?YfwDBEj/DB7Y4Uac+hkj/lMPcoABnJJRhDbutxeXcC98hpQIfva8ZXPuWjYt?= =?us-ascii?Q?h2OO7/ud7Clewk1a6VPz2YA/Uq8p0IC7cf+gdCD7O67x3Szr7qspxWNeE0kk?= =?us-ascii?Q?uhcqSuaomG0V+NqqDo42Dt5dJU2uziF3UPnSmQvsUeygd/hBbF5699xG2e23?= =?us-ascii?Q?d5d8NQg51kSuDf/xFCGCzWQwEUBTQ1UmDxpkIRFimg2NnWRb5gJPdhdB2I8z?= =?us-ascii?Q?iLmpzUZwRHsFML1AdJPOt+uZf4a6G+si/+wnO/jvdl26Wh7KdAmPhbD6RA/N?= =?us-ascii?Q?WrL87FIhWUDT8t+lSyqpqg3t8tjW+NpwB4DK8TjYC1cJBl/SaMn4x8XFcUur?= =?us-ascii?Q?S+q05G1fCO4JKiYyejD/4BXm9ih5i70K8XC7VvvjPklbohIW5K8GDA7VtfYO?= =?us-ascii?Q?j7fcTd91QGxNd+3VRRdoCHAfM9wEBLdAd0uoPY6iQwrVn7/UPhGNp5/GMAfu?= =?us-ascii?Q?uohtRLbeoZ6DhXbevLs2WpdFDFKf/FReNX3VVI8qKaXr6nx58aB6Rjlqv0Do?= =?us-ascii?Q?U04sYMNYfMK4y18jzqLHuITfNQ0KZhjA53qA34cjd+5P+oWghgBnMcv6Wa2r?= =?us-ascii?Q?IX1VMCo4Ivx/VCDOftXX3gGnnXrpV1R0zVvuCKDLiKoYjDSTKgL+XSm78EMZ?= =?us-ascii?Q?x7YtMV+6mldLtgteUQBVbWkrQ2owv7hxi0bs8Ae1wAzwxdZi54oaBdoN7xCG?= =?us-ascii?Q?+3eLQhzlnYmMO5+R+6IUqrUG+ZQCfYaEmpogxRWNs3u4hrd8MCR2dwait5jo?= =?us-ascii?Q?kX3SnygWTWwel/4pExIfnc1W3MHbK3UKKzQF+NMH393SDBPbFE3t7L5EEO+z?= =?us-ascii?Q?Cv2k0SDXwpPK9cixTJKZpBXGKtTjVb5AW36DySzOPjUupOy5bDEcH1TsmCBD?= =?us-ascii?Q?YWN8s/CfJ0X+M7tSOV3gPJ74uTXgydv5YtcfeOIEhv+l6wJ18/QUfw+KazPk?= =?us-ascii?Q?2bjymgdjVkJ0xUh4gRc/0/DfiofyC0PJnUXM8vH/9gnu0OgnSqqMlA5OQhyP?= =?us-ascii?Q?Y3DsinQ8wO3UEiXShbZ6vDytn9D9eyuLSoMIZFPIOxh1TDIqAwiRdXJZKHSk?= =?us-ascii?Q?j6bSBdAL4ZrxVfphthEBVbkaLz7elJAXuNuDrtZgwcKEJlUCA6DUOGjU5y8F?= =?us-ascii?Q?sI/PWhflOIs63br7cffRUvMrmYfxnm/cRx/IYzfOS89Q6YXbfciWkQENEumA?= =?us-ascii?Q?OvbRsx4hL/0HQoTmM87lPLQojXevtgbK7/5qR++6?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: b85614c3-e8fa-49e4-3883-08ddaf35bc37 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:28.5667 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xWmFrdt8Un6yACfL9n7AtOy6S10NBJ+S/JPfRAw1hx5aUUAsHD60VrOSVC4HwRnfSSozBPjO1IfFx8h0A+V8cA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMiBTYWx0ZWRfX+jFdHnBnGzFX /ppoJGbV0pMASEHP+Ng3bqLMC1dQxVFCEVT7mXtUsXXtbzL2vvWMXA+5v7lLZd8NXJB5qNBXJiG IcdEUPT51jK1meEO+YVTLPJnUc6ZW4ztSyCEfC4FE1HA1g9rcs0hiDQCAxRohjNf++pjEVUTq1H tKTMwZY5FO8xjtq5Hv/wfqisxAr98Mpf8VuCbRY9HQa7QQTLg0aSEaYgAvWwg8wp0paS1Bw5hSu QUxnlyELrN6y4LoTYtduvAzZ+plnLcuD52bHYdkWdbepdmfqsgBB+egpQfnHNvVDRdL6tvEVQp8 IEZ8WimyYe8GOVfK9XhszUEWaZoU26Kn21xYpUkmnb9BEkIiIuvjyJrAdpeCuWxu4w0FDvRUZBN tDxCF76bZToN+lyNOS2IEj7BZ8lWzeMjtbgVPdKVYySsQD927ye6g3ENJGfxHZLuXufHLeGe X-Proofpoint-ORIG-GUID: 1EVRZTGF2NZu4FJhROHfBwVKxmkQ0157 X-Proofpoint-GUID: 1EVRZTGF2NZu4FJhROHfBwVKxmkQ0157 X-Authority-Analysis: v=2.4 cv=LIxmQIW9 c=1 sm=1 tr=0 ts=6854116e cx=c_pps a=kkQK6hjpUACXm7ua2zl0Pg==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=70NfNXkHNs9el5T2m-kA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340003493116600 Content-Type: text/plain; charset="utf-8" Hook this call up to the legacy reset handler for vfio-user-pci. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/device.h | 2 ++ hw/vfio-user/device.c | 12 ++++++++++++ hw/vfio-user/pci.c | 15 +++++++++++++++ 3 files changed, 29 insertions(+) diff --git a/hw/vfio-user/device.h b/hw/vfio-user/device.h index 5fc226fd58..86fbc33650 100644 --- a/hw/vfio-user/device.h +++ b/hw/vfio-user/device.h @@ -17,6 +17,8 @@ int vfio_user_get_device_info(VFIOUserProxy *proxy, struct vfio_device_info *info); =20 +void vfio_user_device_reset(VFIOUserProxy *proxy); + extern VFIODeviceIOOps vfio_user_device_io_ops_sock; =20 #endif /* VFIO_USER_DEVICE_H */ diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index 0d7ed9c5ba..5e67cd56e3 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -47,6 +47,18 @@ int vfio_user_get_device_info(VFIOUserProxy *proxy, return 0; } =20 +void vfio_user_device_reset(VFIOUserProxy *proxy) +{ + VFIOUserHdr msg; + + vfio_user_request_msg(&msg, VFIO_USER_DEVICE_RESET, sizeof(msg), 0); + + vfio_user_send_wait(proxy, &msg, NULL, 0); + if (msg.flags & VFIO_USER_ERROR) { + error_printf("reset reply error %d\n", msg.error_reply); + } +} + static int vfio_user_get_region_info(VFIOUserProxy *proxy, struct vfio_region_info *info, VFIOUserFDs *fds) diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index c0f00f15b1..49d12763ab 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -263,6 +263,20 @@ static void vfio_user_instance_finalize(Object *obj) } } =20 +static void vfio_user_pci_reset(DeviceState *dev) +{ + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(dev); + VFIODevice *vbasedev =3D &vdev->vbasedev; + + vfio_pci_pre_reset(vdev); + + if (vbasedev->reset_works) { + vfio_user_device_reset(vbasedev->proxy); + } + + vfio_pci_post_reset(vdev); +} + static const Property vfio_user_pci_dev_properties[] =3D { DEFINE_PROP_UINT32("x-pci-vendor-id", VFIOPCIDevice, vendor_id, PCI_ANY_ID), @@ -310,6 +324,7 @@ static void vfio_user_pci_dev_class_init(ObjectClass *k= lass, const void *data) DeviceClass *dc =3D DEVICE_CLASS(klass); PCIDeviceClass *pdc =3D PCI_DEVICE_CLASS(klass); =20 + device_class_set_legacy_reset(dc, vfio_user_pci_reset); device_class_set_props(dc, vfio_user_pci_dev_properties); =20 object_class_property_add(klass, "socket", "SocketAddress", NULL, --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340155; cv=pass; d=zohomail.com; s=zohoarc; b=gvt31vdtJsS9UCdYxZTZL+8oGsta4XEDN4AVXoPmPh+joXbXLXwx6/cuQQHCX5wLwXDuDxmbShwcgXQZY84MdMw4b1J/0wfxFiO57xbTBwLhQGDVDWxd5JV+AWL1seAxoSOW7/msmHPXq+/rXVmsaAQIzA397dKMvGHtTrLGX0w= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340155; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6311tszg3B3Xgw32Umsi2InitNKIbeJrTmEavkjkrtI=; b=M15a+v+MWMpvd/j1bhN+Uh54RLlVYiIwx3jZ31fc852hS3rlOszQRFLsF8fV61K8O48nhXd/hVOw3EYXEHUXE3jieYz/YW6RY7PNNqrL0JTKSdXS8hUOCkuuB5J0oA4rIQ89cfyPazpv5rgNkpXK8h7k2OM/3rdXzBszOoLBx3s= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17503401556074.984622447761922; Thu, 19 Jun 2025 06:35:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFO8-0004lJ-T3; Thu, 19 Jun 2025 09:33:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNq-0004Sd-UN for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:51 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNm-0008GX-Hq for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:50 -0400 Received: from pps.filterd (m0127843.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55JAvH22013023; Thu, 19 Jun 2025 06:32:41 -0700 Received: from sj2pr03cu002.outbound.protection.outlook.com (mail-westusazon11023091.outbound.protection.outlook.com [52.101.44.91]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47976j3rag-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:41 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:31 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:31 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=6311tszg3B3Xgw32Umsi2InitNKIbeJrTmEavkjkr tI=; b=s7wFWXN7Thm+svKBHxws0SxtU9oXzUVvlIfucwJo1bIoZVoJwRN1xlA3g HCli0sIUIRDZ/csMSgNlIr3gIXg/Z4dv81lR6OEb59yXL6017czciX6T2rI11Feq wiXrEBwQ0nkYn+SXik+LzFElMMyBCEMBj724QYf48HwBxIBzIJJwspZ2U92mJ9rS t58UVIAKhqISdxot5BTT4kQ8Kt4yx35w9Gz5az3Jmhnh26PacyA2TNiO+n6FxK8m XTPJfFTRDGOdpilsEv/pVC+CxkBZxCH+oz/NS6xTP7Daf0AHLrctnueyhvD+71u6 8iJOyMuIDOuVYV3BS6cIUthINcpTQ== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rgjTK9mE24alg/OHl25xfl4OREIfn+Ji6kzIPnRXB06ylgvLGCgrO0ZdQlqflbaVnejipE+A1qxw0BZYyZtvxf7PlsLgWWXmUgfVKRz3bl5t+AVfLRlO8LNNqlI0b8jT9NDvS9JppAiujGYPLyelRv0frmxiF2v9K8YADMLkgAqQsV2tNXqmqJa3KfEytv7NhdskY14l4JzWRn5FQgRhlNO7LFznc+wzhPjStWl7FjmYYyYrVz+xNXSChUBDdTi6UOZqchCE1xuNpho4Zn0J1BUH96OzinRXuzDBjMAOBAtvwkznRUGy7l2t+NYQrv0xCVZK+lzvI9/bZMp1X+bGkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6311tszg3B3Xgw32Umsi2InitNKIbeJrTmEavkjkrtI=; b=Nyr3Mmbi01cbvE9jTvAs5vWw0HO+i/G/I8ZVWdaitrfFi46+eCcX0GqRBap6kYcuHkAy5cpT/YaMoyU0jUAIP+XnNVVx/ISqR/QmSJYJzqXaNVa+40nER+TQLJFETypzAPeHNL/m07Iks+FGGkkE0KA83sCFbHKS5En/OnDSVo66FOQkFXAtQjNXWI3boFeggmUjxb/NFKdwiGxB6pPHIbIQnQos+f2iXziPqq7sAJ9TxK23abDWr/BWp02506D5Ii6203II11jrwH8Qo0k3SMd0NiPYL+TQqFjHcvXyEDdf8S3bAJ42n5VxhvLwxknc++kJ+79Ulo2r7vGFnrVr0Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6311tszg3B3Xgw32Umsi2InitNKIbeJrTmEavkjkrtI=; b=TQPQjIMawCyqraNbA37rMPy4YV/VgF6JanLqNJe3hpOcQ4YKEHSei0bisdDMg+ofZhEO5K2IwYLOF2DXrvou7cOcnQk2Scj9O1ULiJjvtdQOL91jL4Pl2rSu/WyYzpv3W88P/OSQo9nWF0DGTdAny/ibI1owoFTv7we3X995NOVnWXf4eaVCQjjYrp1pvmlDppXMfrT+fD6imm/1uqaU6LjUnc41F1v4Zz5KEDcMeUVAd7BURRrUx/oaMa9pdYo/NufqV8nw/rQXdS+I1/qJDGeZGrH2FE/qfeU7uYGGqYcCaq23sOA02CC6H5ZXgMvLnqFYWVrMSCQjqVWi2VAPaA== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , John Johnson , Jagannathan Raman , Elena Ufimtseva Subject: [PATCH v4 13/19] vfio-user: implement VFIO_USER_DMA_MAP/UNMAP Date: Thu, 19 Jun 2025 06:31:47 -0700 Message-ID: <20250619133154.264786-14-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 6cd8a697-9efc-4f23-e367-08ddaf35bdb3 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?2HZ66Ku1qsiAGTxD0i4EHC3YTLxGPZnVDh/+PvzaAw+o9XqX0szk4keljwfn?= =?us-ascii?Q?tXvHVH4WnFQZ9nuoQS3hXuNi/YkoQDksKqnuxEkb1237laSw32ZTTpzE4pFK?= =?us-ascii?Q?EMtUa9R4AzqG7Knh/ZUzDSR4vgLpsutXk0M6/Z60Z/h/Obgui5fJGpXTc2Ay?= =?us-ascii?Q?4M08znCraYMyZdEQMDMzW46ICPHXbVcVBVsysADz4wHIxaueU7bEsH2zyJQa?= =?us-ascii?Q?r6EctisFBHlRQpMf6nY/1cEQ0uqEAMpM1UbYNFgx1Jgt4G/nQlsJ4SrdMBgW?= =?us-ascii?Q?k8r0WZw4rz3T241NyCDMP/4wIHJCxsnJ07Y7xaJ/sVP2QD02sZ7z9J21xvVE?= =?us-ascii?Q?FKv2G/V1Hcla39/7xgMnJq6RGHwr0Fkfwj/LPW76XEERQlABB54NUs+iIJNC?= =?us-ascii?Q?D0hfADsuSVjL40HLpne0RuMTiqA4QejOUnjlgsTtfre/NluhIaV8azeHvZY0?= =?us-ascii?Q?5BIKtiH6vHIwceh4QxWqAkIf64+ZcFOrw/Lt9Nosip2wRNYJ2pGC7fcdfMNo?= =?us-ascii?Q?8jTrId2w2FKvEUox39uYDCfcPYqkXOcYHhucuj9e//GeVj6r1jpDlNwq7IAq?= =?us-ascii?Q?AF/N4FpUxkIJosGITfwV8FFalXbhZBaBkB1pr0Wu4nkJqYnlcl9c4DS25n74?= =?us-ascii?Q?ix5C15Un7jp1JIQElaUIMRGw1PiiqK5h2RlPI480J3AnWlAfOajPqpwJlfG/?= =?us-ascii?Q?lXu99QCRD11yE78MyjrO0/DbRDmjJY4OAKOpCVWccbGgbdrlobgtJKeW9Cv9?= =?us-ascii?Q?zrvIVHKnAs5MvzKHQC1G8bpeQGoan9qYKvNE6ecIaQf27Ry6i+kZiOIkU6wA?= =?us-ascii?Q?DA+O7tC8OwQ9G88zst7/uBh1NM3U2czqEsOXFtquXeh+UjHWguy0bS2qTqsv?= =?us-ascii?Q?1xtL6wLGATwA3eW5/G/VSLF9rMA11mJmi37LLW8tkeFNnkwA7wdQD2JImq3x?= =?us-ascii?Q?g8/4+bEy2CxGwUyAFy8DafWwhVWCJG7kcm40gLNXxtOQ8I+Z7kiOIz314CKg?= =?us-ascii?Q?hK8r0qSgLkEVK41WNQ1zzG9lZ0Rum4FBbIXTrbWHlT2VYVxFG/bRscSDD66A?= =?us-ascii?Q?koqBTt5YBqNveST8RcymvD4gqlqkCrZfZtDPOJnWElWRQOUtJoA2cvFp9fSc?= =?us-ascii?Q?Few6R4cc9oRAWuPow0DLCbEFr2zr4TPwt9+BTzPjOqoHJDRhdI7wQB67IRuw?= =?us-ascii?Q?Fbmr5JV9l1nhtiPdHS3uZ9toDJNy/3Ii1SXBObrvUZoibnoSJaKtOn3LERYc?= =?us-ascii?Q?zXqnb0V2qM0lZzQz/qI7u5cImAoCXDw3tnn6hhLthykbKDzGkkm86+S8v1H3?= =?us-ascii?Q?I5B10y21FP1iyvs84HDIaQYTVLnb4o+3VUMUHUSA1T94zrNz+oFtJ5D5ArrS?= =?us-ascii?Q?bqi0SIZMO56xsyWaPD7KQDIpQk4tjvnxiZMgl/J97Q3ge0hQax5WlDf5Vp1o?= =?us-ascii?Q?Xzmm/SSdQgw=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?6cfM4El49kJQH+Uftw5zUa1sokNu+qMORRdwJgI63IvZ0O1CpA3alRethhvP?= =?us-ascii?Q?diIDJ8kagzMlWnm9mDt+PYlajoVGdRdBs7nPH0Yjmk/+7KzvK/R372YKvV1o?= =?us-ascii?Q?mXBqblqxfJNGIKh8ZjnCMZJpoak+SmShO0GSTOEArJnFWIR/sp/pIfldL15f?= =?us-ascii?Q?3KwdePiIerJkqPnTGKdbP2qE50b92SOfFHJbgJgvYqWEov7gwLUMeY4lEub5?= =?us-ascii?Q?3kKdl5eGc44aUIxstisSjYdS2A7Wn8x9vnYJkL2EGW4n+E8lprHmuKUh5HOi?= =?us-ascii?Q?VJ4I+n/QZLi/Tux9VrKCC0hPKEolHwmEuWV4gKZUQ9R025o0jrU/OaPqYzw2?= =?us-ascii?Q?+Rd0FkwEGAVl6SbHnySE/fD8XwgX39jUBygo+5EzSplq8zZwTOOvp6cOBY4b?= =?us-ascii?Q?atOmLPyJARdncnKZOWLh5bceac+DvMPj+oTQ6XhwqTm3IYIqL+cur1g7eJJR?= =?us-ascii?Q?C6sE3Q8Nd/X/pqDFdKXUmqHnGfgCWdpZEZ0NQueSDzW4tZQ7nLXOLWH4DCp3?= =?us-ascii?Q?fE/IcG6vJY1320q6Htj1ROEtTfMIsFGz6yGEYvvYYnPNqDXr5q3iHmsupsvu?= =?us-ascii?Q?5KF+ytjbEppJ88jlYa7QlEzt6XDTtxH87XQ7k3qjtzbS8gWGhJKNvlYM7oJf?= =?us-ascii?Q?MouW2k4OzJHAFAzh14ZkAqemcg8YoeVH+hkrubLymgSsZKZOj8HaqXvp+DjM?= =?us-ascii?Q?CPWYYdDM5/CiHH5hDtHU4VEqDjKPBfluaedLveERLnBOck0YyKhJ6AlZ+Lag?= =?us-ascii?Q?7nqlcCiNLTlN7HDrG733PO83ugf14I2qqD3P0vKZRlLziuzhR9lvs7NaXeUn?= =?us-ascii?Q?h2UlWlLJPHWmk0gK58Ut7oaZpFSDHWFhpdhbybaRfc+si21f5Uzip6Yia8Q/?= =?us-ascii?Q?fa853UPvL4m3Hegg/dViRjI9SuHvpApdRoOh7sZdmK7NUHYGUbxZBEGTvkSJ?= =?us-ascii?Q?30vpO+UkdKL9mOTVqo43sWhpOXPOGjL5jmI2PsbHVr/cuxzD31XrMsgfNQ8g?= =?us-ascii?Q?ziDmrZqk8InqszfRPids7x26Y1MfG9YcWIAcnct49zgEM8ogGl3J7qIsqHi4?= =?us-ascii?Q?eowLteI/GMJ+DTGtg8j0mtbXS/fX7JV1zVr97eAzJxfycez6ozI9jz/SHPI0?= =?us-ascii?Q?bRe4DypJMdjPd0F5w57xtEisNgPiUwuPCq23hPKKS9x06nY9U5nXV0l2PV0z?= =?us-ascii?Q?UT4QD0rmFDJ2wuxcRXpN2jI/cafDam4qVIfusJY3eurw2GPpd2qE05GRZGbX?= =?us-ascii?Q?sCgSGJgkikTDTqc24YUm6mMi/VancrVno8TpupgVpVUX+RB+GH9Wiw5UVRSJ?= =?us-ascii?Q?UzrIGYGojXLUeoNep/84pFjdPz1zM/vPHjFgknhArPA2m4tapcPE/fLpL8re?= =?us-ascii?Q?psGv+fFX2C9KcQgJdYBG4JsJYVklx3vdqNLJlMQr0NAY4NS7XCiStukT9c/C?= =?us-ascii?Q?+pAXQvJ1lYgLDFKns1gmxtdq4db3fj/H6MfjXCCZNbzCBJ2vUQUNK9q8+csl?= =?us-ascii?Q?uQFMrSHsbyMxu3GqUr9Rs8VRXxeMBfG+N6ysv88uLpIIqSqiqNG8fYZukooB?= =?us-ascii?Q?V6LztHb+h+K17Jzwj2KPV7it/jbWP16Na/yeHUVv?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6cd8a697-9efc-4f23-e367-08ddaf35bdb3 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:31.0910 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0xWYn7+CqJ7wNkxitx5y9gQBk+qTo2zn1k4trGBlA583bDieUy1GcwiXxQnW3Ox1sjS8sMGdKjnILjXGxm2/sg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMyBTYWx0ZWRfX6/gZbH2FUjTv Wty4y9DZg7M30fg3yUGHyzfZ2BJ7T6ImumjHMa36UTF3PC4uSJBa+3CibBKE4A7oA6oI12ob7TI 2DbD1SoDfTzjUk1oMlzWup49YlWFFo0JRDTUowxy3TVqXzEKR5UKXEpifJrgbGzphvVENGnelyr o8LYtTGFJntwkUQ1J15rhnjfpY9GAL4TpIkaV3DGzfdtXxcIsXbl//wvAuO2fonSivIOO6w3YH+ PWZhH3F97WnNaFLQuNDV1mJN4nw9HWA8ymBXCHhjgmV6kw43ZZW17hd+Db/uhua2c0GTK36HK+f OTBoK4xyOAPgVILKClIHht2UAIVL2T7sAQEzVTLgKQpQQnJUHqxO8fsBhvmQFSI3xLs0nS/+7Uy WpiI8oD31gJood3rFwEvc2KO3MAt1sA4VHTgT4sVc49zeE2nJuhKt56X35uPFOcNKiG1SusH X-Authority-Analysis: v=2.4 cv=RcqQC0tv c=1 sm=1 tr=0 ts=68541179 cx=c_pps a=b9bpObgcEOTp51h0EtjbnQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=RQZ_2NmkAAAA:8 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=1ZqixiJJ9Y4JtCwCYkQA:9 a=46pEW5UW3zrkaSsnLxuo:22 X-Proofpoint-ORIG-GUID: noPGGX5BtC_ph3nHya6cpVFeI-3AAl2r X-Proofpoint-GUID: noPGGX5BtC_ph3nHya6cpVFeI-3AAl2r X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340157781116600 Content-Type: text/plain; charset="utf-8" From: John Levon When the vfio-user container gets mapping updates, share them with the vfio-user by sending a message; this can include the region fd, allowing the server to directly mmap() the region as needed. For performance, we only wait for the message responses when we're doing with a series of updates via the listener_commit() callback. Originally-by: John Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John Levon --- hw/vfio-user/protocol.h | 32 +++++++++++ hw/vfio-user/proxy.h | 6 +++ hw/vfio-user/container.c | 108 +++++++++++++++++++++++++++++++++++++- hw/vfio-user/proxy.c | 77 ++++++++++++++++++++++++++- hw/vfio-user/trace-events | 4 ++ 5 files changed, 223 insertions(+), 4 deletions(-) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 48144b2c33..524f3d633a 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -112,6 +112,31 @@ typedef struct { */ #define VFIO_USER_DEF_MAX_BITMAP (256 * 1024 * 1024) =20 +/* + * VFIO_USER_DMA_MAP + * imported from struct vfio_iommu_type1_dma_map + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint64_t offset; /* FD offset */ + uint64_t iova; + uint64_t size; +} VFIOUserDMAMap; + +/* + * VFIO_USER_DMA_UNMAP + * imported from struct vfio_iommu_type1_dma_unmap + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint64_t iova; + uint64_t size; +} VFIOUserDMAUnmap; + /* * VFIO_USER_DEVICE_GET_INFO * imported from struct vfio_device_info @@ -175,4 +200,11 @@ typedef struct { char data[]; } VFIOUserRegionRW; =20 +/*imported from struct vfio_bitmap */ +typedef struct { + uint64_t pgsize; + uint64_t size; + char data[]; +} VFIOUserBitmap; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index e5ac558a65..a9a36e4110 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -70,6 +70,7 @@ typedef struct VFIOUserProxy { QemuCond close_cv; AioContext *ctx; QEMUBH *req_bh; + bool async_ops; =20 /* * above only changed when BQL is held @@ -99,9 +100,14 @@ void vfio_user_set_handler(VFIODevice *vbasedev, void *reqarg); bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp); =20 +VFIOUserFDs *vfio_user_getfds(int numfds); + void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, uint32_t size, uint32_t flags); +void vfio_user_wait_reqs(VFIOUserProxy *proxy); void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, VFIOUserFDs *fds, int rsize); +void vfio_user_send_nowait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize); =20 #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/container.c b/hw/vfio-user/container.c index f7c285ec2d..a9cc4b197e 100644 --- a/hw/vfio-user/container.c +++ b/hw/vfio-user/container.c @@ -12,23 +12,125 @@ =20 #include "hw/vfio-user/container.h" #include "hw/vfio-user/device.h" +#include "hw/vfio-user/trace.h" #include "hw/vfio/vfio-cpr.h" #include "hw/vfio/vfio-device.h" #include "hw/vfio/vfio-listener.h" #include "qapi/error.h" =20 +/* + * When DMA space is the physical address space, the region add/del listen= ers + * will fire during memory update transactions. These depend on BQL being= held, + * so do any resulting map/demap ops async while keeping BQL. + */ +static void vfio_user_listener_begin(VFIOContainerBase *bcontainer) +{ + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + + container->proxy->async_ops =3D true; +} + +static void vfio_user_listener_commit(VFIOContainerBase *bcontainer) +{ + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + + /* wait here for any async requests sent during the transaction */ + container->proxy->async_ops =3D false; + vfio_user_wait_reqs(container->proxy); +} + static int vfio_user_dma_unmap(const VFIOContainerBase *bcontainer, hwaddr iova, ram_addr_t size, IOMMUTLBEntry *iotlb, bool unmap_all) { - return -ENOTSUP; + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + + VFIOUserDMAUnmap *msgp =3D g_malloc(sizeof(*msgp)); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DMA_UNMAP, sizeof(*msgp), = 0); + msgp->argsz =3D sizeof(struct vfio_iommu_type1_dma_unmap); + msgp->flags =3D unmap_all ? VFIO_DMA_UNMAP_FLAG_ALL : 0; + msgp->iova =3D iova; + msgp->size =3D size; + trace_vfio_user_dma_unmap(msgp->iova, msgp->size, msgp->flags, + container->proxy->async_ops); + + if (container->proxy->async_ops) { + vfio_user_send_nowait(container->proxy, &msgp->hdr, NULL, 0); + return 0; + } + + vfio_user_send_wait(container->proxy, &msgp->hdr, NULL, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + + g_free(msgp); + return 0; } =20 static int vfio_user_dma_map(const VFIOContainerBase *bcontainer, hwaddr i= ova, ram_addr_t size, void *vaddr, bool readonly, MemoryRegion *mrp) { - return -ENOTSUP; + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + + VFIOUserProxy *proxy =3D container->proxy; + int fd =3D memory_region_get_fd(mrp); + int ret; + + VFIOUserFDs *fds =3D NULL; + VFIOUserDMAMap *msgp =3D g_malloc0(sizeof(*msgp)); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DMA_MAP, sizeof(*msgp), 0); + msgp->argsz =3D sizeof(struct vfio_iommu_type1_dma_map); + msgp->flags =3D VFIO_DMA_MAP_FLAG_READ; + msgp->offset =3D 0; + msgp->iova =3D iova; + msgp->size =3D size; + + /* + * vaddr enters as a QEMU process address; make it either a file offset + * for mapped areas or leave as 0. + */ + if (fd !=3D -1) { + msgp->offset =3D qemu_ram_block_host_offset(mrp->ram_block, vaddr); + } + + if (!readonly) { + msgp->flags |=3D VFIO_DMA_MAP_FLAG_WRITE; + } + + trace_vfio_user_dma_map(msgp->iova, msgp->size, msgp->offset, msgp->fl= ags, + container->proxy->async_ops); + + /* + * The async_ops case sends without blocking. They're later waited for= in + * vfio_send_wait_reqs. + */ + if (container->proxy->async_ops) { + /* can't use auto variable since we don't block */ + if (fd !=3D -1) { + fds =3D vfio_user_getfds(1); + fds->send_fds =3D 1; + fds->fds[0] =3D fd; + } + vfio_user_send_nowait(proxy, &msgp->hdr, fds, 0); + ret =3D 0; + } else { + VFIOUserFDs local_fds =3D { 1, 0, &fd }; + + fds =3D fd !=3D -1 ? &local_fds : NULL; + vfio_user_send_wait(proxy, &msgp->hdr, fds, 0); + ret =3D (msgp->hdr.flags & VFIO_USER_ERROR) ? -msgp->hdr.error_rep= ly : 0; + g_free(msgp); + } + + return ret; } =20 static int @@ -220,6 +322,8 @@ static void vfio_iommu_user_class_init(ObjectClass *kla= ss, const void *data) VFIOIOMMUClass *vioc =3D VFIO_IOMMU_CLASS(klass); =20 vioc->setup =3D vfio_user_setup; + vioc->listener_begin =3D vfio_user_listener_begin, + vioc->listener_commit =3D vfio_user_listener_commit, vioc->dma_map =3D vfio_user_dma_map; vioc->dma_unmap =3D vfio_user_dma_unmap; vioc->attach_device =3D vfio_user_device_attach; diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index c6b6628505..315f5fd350 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -28,7 +28,6 @@ static void vfio_user_shutdown(VFIOUserProxy *proxy); static int vfio_user_send_qio(VFIOUserProxy *proxy, VFIOUserMsg *msg); static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, VFIOUserFDs *fds); -static VFIOUserFDs *vfio_user_getfds(int numfds); static void vfio_user_recycle(VFIOUserProxy *proxy, VFIOUserMsg *msg); =20 static void vfio_user_recv(void *opaque); @@ -130,7 +129,7 @@ static void vfio_user_recycle(VFIOUserProxy *proxy, VFI= OUserMsg *msg) QTAILQ_INSERT_HEAD(&proxy->free, msg, next); } =20 -static VFIOUserFDs *vfio_user_getfds(int numfds) +VFIOUserFDs *vfio_user_getfds(int numfds) { VFIOUserFDs *fds =3D g_malloc0(sizeof(*fds) + (numfds * sizeof(int))); =20 @@ -606,6 +605,36 @@ static int vfio_user_send_queued(VFIOUserProxy *proxy,= VFIOUserMsg *msg) return 0; } =20 +/* + * nowait send - vfio_wait_reqs() can wait for it later + */ +void vfio_user_send_nowait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize) +{ + VFIOUserMsg *msg; + int ret; + + if (hdr->flags & VFIO_USER_NO_REPLY) { + error_printf("vfio_user_send_nowait on async message\n"); + return; + } + + QEMU_LOCK_GUARD(&proxy->lock); + + msg =3D vfio_user_getmsg(proxy, hdr, fds); + msg->id =3D hdr->id; + msg->rsize =3D rsize ? rsize : hdr->size; + msg->type =3D VFIO_MSG_NOWAIT; + + ret =3D vfio_user_send_queued(proxy, msg); + if (ret < 0) { + vfio_user_recycle(proxy, msg); + return; + } + + proxy->last_nowait =3D msg; +} + void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, VFIOUserFDs *fds, int rsize) { @@ -644,6 +673,50 @@ void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUse= rHdr *hdr, qemu_mutex_unlock(&proxy->lock); } =20 +void vfio_user_wait_reqs(VFIOUserProxy *proxy) +{ + VFIOUserMsg *msg; + + /* + * Any DMA map/unmap requests sent in the middle + * of a memory region transaction were sent nowait. + * Wait for them here. + */ + qemu_mutex_lock(&proxy->lock); + if (proxy->last_nowait !=3D NULL) { + /* + * Change type to WAIT to wait for reply + */ + msg =3D proxy->last_nowait; + msg->type =3D VFIO_MSG_WAIT; + proxy->last_nowait =3D NULL; + while (!msg->complete) { + if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, wait_time)) { + VFIOUserMsgQ *list; + + list =3D msg->pending ? &proxy->pending : &proxy->outgoing; + QTAILQ_REMOVE(list, msg, next); + error_printf("vfio_wait_reqs - timed out\n"); + break; + } + } + + if (msg->hdr->flags & VFIO_USER_ERROR) { + error_printf("vfio_user_wait_reqs - error reply on async "); + error_printf("request: command %x error %s\n", msg->hdr->comma= nd, + strerror(msg->hdr->error_reply)); + } + + /* + * Change type back to NOWAIT to free + */ + msg->type =3D VFIO_MSG_NOWAIT; + vfio_user_recycle(proxy, msg); + } + + qemu_mutex_unlock(&proxy->lock); +} + static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); =20 diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 053f5932eb..7ef98813b3 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -11,3 +11,7 @@ vfio_user_get_region_info(uint32_t index, uint32_t flags,= uint64_t size) " index vfio_user_region_rw(uint32_t region, uint64_t off, uint32_t count) " regio= n %d offset 0x%"PRIx64" count %d" vfio_user_get_irq_info(uint32_t index, uint32_t flags, uint32_t count) " i= ndex %d flags 0x%x count %d" vfio_user_set_irqs(uint32_t index, uint32_t start, uint32_t count, uint32_= t flags) " index %d start %d count %d flags 0x%x" + +# container.c +vfio_user_dma_map(uint64_t iova, uint64_t size, uint64_t off, uint32_t fla= gs, bool async_ops) " iova 0x%"PRIx64" size 0x%"PRIx64" off 0x%"PRIx64" fla= gs 0x%x async_ops %d" +vfio_user_dma_unmap(uint64_t iova, uint64_t size, uint32_t flags, bool as= ync_ops) " iova 0x%"PRIx64" size 0x%"PRIx64" flags 0x%x async_ops %d" --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340320; cv=pass; d=zohomail.com; s=zohoarc; b=Ykb5xR+6Gy7LCU0DioBBTxWqTC8Bhur+v3RA/WvIeLH7Dtianumn+/c5sjlPJQx9PEqddrFoXlW2FZRjv45G3kHa9MXOBmlxroji1i3AqUIaXjFUwy+2gG/Pi9On712BgQqSaQl7TXQppYtfrxi3o5P0Hlm8A2Qk26jMl1rBHIY= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340320; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Z9o5YubL1i4HRBF8nKK3/9MxxJxTJXZT/EZtnIjvNQM=; b=bdbJjm9HHYF/BBvNX2M9RT+e2BlAaCyrzyE+myOOSCGg/Dj9iySRPOL6PpKCD3yCelm1c+S/ul9MefFcXCO+DMoKnnfHXSEi+xShCc1cM8DS+O4pWmOwi0HkkVgQNq1XlfqvK7UBBdTOJQ2p9hRWuoJnCNT+g0WF4SO0bFzCik0= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340320753572.398229800941; Thu, 19 Jun 2025 06:38:40 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFNu-0004WW-Es; Thu, 19 Jun 2025 09:32:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNo-0004P9-UG for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:49 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNl-0008GS-Sq for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:48 -0400 Received: from pps.filterd (m0127843.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55JAvH23013023; Thu, 19 Jun 2025 06:32:42 -0700 Received: from sj2pr03cu002.outbound.protection.outlook.com (mail-westusazon11023091.outbound.protection.outlook.com [52.101.44.91]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47976j3rag-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:42 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:33 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:33 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=Z9o5YubL1i4HRBF8nKK3/9MxxJxTJXZT/EZtnIjvN QM=; b=VdGJcI/eoMRx4EfFbVP/xx0eZVqW2z0cpg60D196dCW24nMV6QG5pm8CI PdjXvgmj1gBZneuxPsX1ppxx1bIMG9KG/x7HPZ/hycvWOefiDPFqtIymIbXI5vAW VzfM0IAz+e8FSwoLVFkRguf5rs7gTKPsfiRxSvdRSQYakmW7Q1NF8oBks7+bMlGP Te2rDwNyIGAKY3ZUFmVKE+cYEbWrPIKPemrdU1k8/nMOQVIR/KakgZ6HKgl7FP3U HXqaVz55Wa5e+bcNK61FU6ha07VGeygzt6ftwcAUY6cBX1mwTAYST18l6NjnAFhp Ux32iYCJ5pzW7qi5fpEK9Yb5vIYGA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jlEa5wvYS5NbGmtUQK073rXjdOiQSJcQwDAbIf015GnL5YZ+rfd4AcE13hGqwwq9sFLOIV2mm+roil7pn1/7asrHntpW0k4VGrcGn6CJK/vXs7GUJ26a6JEfQY+6FSXYin6r+Rm96NW6QDjjuKZIfZAa0oLEAWVD5RD2sjlpvZ8kyO+37eEK+IR77SO0HrVN9N9TzfTho83U9hCeAqXPuvP1HWE9VHfF5uz3wSxxCTvNEpJCIRMVcytila3s4JKhmSGirVpTz40naaEaSEKKh2HHdz+koFWpiZRpF4H8WV0WH1gqaHAWckVWtnFc6TbKfVs6WAmXlcaCGhUJFJFEgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Z9o5YubL1i4HRBF8nKK3/9MxxJxTJXZT/EZtnIjvNQM=; b=rQtRK1p3I92bFnFNpeGDPtAOPuQTDBm1omgozUmZ/FKKDYdAuTckdTDsb0L4VpBDlDJk6uGkpcLhZ6GPA4qOn9APTZawkCwiyxzrAy6IDSm86psOt+JBEpfxM6R/bmoyIr072Sz6PVVl/TrdrVTCaF/A7WlVNXkRIskRhdy7mAuu6PAKHaL8iKK4JuB8SnqFSamR/GTwCtV7iAEtaAr4eb7nbfHh90TQnXs6cgBNKKQZAmhODnoYsgFkui0WzxWV0xKMk/p06YHu3K5Des9d8JMb+xp3aTYthRDOQ5DsvRzeJlpBNmcqfPeQlnwFtQe0hF8SZywKSL1Xbe1WD+pY2g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z9o5YubL1i4HRBF8nKK3/9MxxJxTJXZT/EZtnIjvNQM=; b=GDFfL9PV7QPYnWq5UxzHXF+0XDasSDR0oxNndSjTYWjNQvcybk8mVa8J22A8pBde/6ECrSOPMor4Lq5uLUoTKm/3h5NwzqlbVxSmxyekmzMMCNW60fZUWE075A3TUMVLofMbkEv+7EGIzmZAAhgtLNxNIqOTb/jYGSTtecR5UQQAbTZdJsSMkgbOJFODaz+9ho5esLRlMRJK1gF9MIJpjm21zzaTuomBAkYjN7DvMAuYL67erlATXHPb3pVAixkXSdNcXpxud1r4QUZZAYiR1nDCO2YNYsEXCeo2tUawzn0Vkxvhy+8zAg8lz8p4Mr18/wLe5EXWHG2kIWSOJCze1Q== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 14/19] vfio-user: implement VFIO_USER_DMA_READ/WRITE Date: Thu, 19 Jun 2025 06:31:48 -0700 Message-ID: <20250619133154.264786-15-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 8ec00104-f45f-4353-15b9-08ddaf35bf1f x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?5l0sllaBlCA3kd3QmtlhqkQWDtDa+Qwsbxid2PSi3g8vUMKkE+2NVuIH6/fz?= =?us-ascii?Q?JzrBgx/2XIl/ukRy8yrBqfaeXOuCJAo1XYqXD6IE1M2SQNHD6t19QSOdIUxM?= =?us-ascii?Q?7jvROMf/CRD5TcKZ/Ueys2lZxgpJtNcbuP6idfuHCPKfTNXCGfV1d5aRgqJT?= =?us-ascii?Q?GwmGlbsdZaj0+++Vtms1caskuHNnlyZNkRvA8ueZHapkM7br9KMgyByQ7Pr/?= =?us-ascii?Q?3GkZ6AHByjzKuo3mExi0BSiZyZswGrvt5G6jJftmN7JcS4AH8njcm7ZlyMsP?= =?us-ascii?Q?IiFUx4zsLtr75FeWyQAcvdU19uaHeIFEATPUkhbY4Q+9W0iHq0k4/UNhkrWr?= =?us-ascii?Q?EvjtL2TAL6TPRjgd3/ZpbfXM+R4JCtR2Bc+Dc22FupSwGFJYjEH91so0/KFa?= =?us-ascii?Q?wlbXHAOD/Mke+BUU22m56khVWbxdDPJivwotMxgitza/teKl1Db/ETRAxGbI?= =?us-ascii?Q?3Bct2lZ2D8GV4xMJAqIQDL/smAyAXuHkg5WO5U2J+FOn+kwbWwVssbModPFr?= =?us-ascii?Q?BKhd/sEYRA3+0Zu/s00G6HoKdn1trxx17WYegOETt8HpDgNHkOKOPYIc4hBr?= =?us-ascii?Q?ruhaQJA9bjZBQjMfo2WtWTZ0BgOkF0iGeH23nZZVGOgJUHc3I0B1+/B0f7Tu?= =?us-ascii?Q?HOIdYGQUWZgE3OwG3oPxwQMkTiX44Os/C29Ql42y1RgiGq1InGm/q7ncuLog?= =?us-ascii?Q?NY2MqbfdYqV/RlaqRjjc5zuXyJbqrNw+LpLbZEyFMe7zQeVMd/zDOcTS92kl?= =?us-ascii?Q?wq8FAC98TxnVSK1dvybBDUSEwHxkbVFgv1Yi+exhgm3Kz1eLLk+omU8yHCDA?= =?us-ascii?Q?1Hm32QWPvRAG5dWqERZ79+5OKT5Xe1wVGJby5K4ETQb7EgVqcni9dEZ3HEtO?= =?us-ascii?Q?RxJaPsyxlFmTVJXJvf0kmRyvBcjKErerK0d9PY4qdPq+3DhiquqBniyOACRB?= =?us-ascii?Q?pPzx/6mckI6Uu+qyXzm3lWs0lGA1Yxau0azHpfzEYfStAhbe1bqAqiw5Ba/8?= =?us-ascii?Q?B0jnV6Tl5foKox3kRVsAHUheLSavHNE2Bh31+dx9FSLmWjEDWpXN5xGSAYbm?= =?us-ascii?Q?Gh3ZUBzoPfK5fvpi4KYpEBIQ9YEfbeTvkhrAuoR4QMAELkzPNctAbBosozYN?= =?us-ascii?Q?3ZSqK14Tq0LM3AMx91gx3nxH/LiQRHm6Lx/ykrz5jJEGDHoN1RFRvL6jVH0W?= =?us-ascii?Q?Ps0zlkQEusqGUraZ3io8ewdUsKyVtD+t1oMzbiveaE4kymHdKDTd0frNGTC1?= =?us-ascii?Q?wg50lgO/tirE36hNDjLQncF8MLzS5a1/uFKwkqy1rpiV6n0g55HZ8NCbd4nW?= =?us-ascii?Q?dRgE05a3ZZ+UTFkh3aH4PycBKbrlKTVM3e9ZIGDgnBccW8CWu1JmP4JBmA+/?= =?us-ascii?Q?6ABrfPLK4zR5PfP1CCCXCzwtQQAV2T5sS/OKN9P2vqCqbMtve1hD1N0R4KGm?= =?us-ascii?Q?D8W1Ozi5V+U=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?jpC+tSDMAnTL/CGy+4JIMjo7nI0ey+7NnsI21HwYcFx4d10ariID/9qwgZTH?= =?us-ascii?Q?zKWgyqjrpIQVumitdnVAPMXk/CkunapYUOqzTmonnrGH54n/yb39KXNuhZrZ?= =?us-ascii?Q?+f98ax6sL9cwRAWhGvTYCFec5ncqQCkhD5c2aMhVYymS4Wu2STZ/9uu9sNKp?= =?us-ascii?Q?dlSaniWLb3NLu092MenZbwCkVM9RkJEgFf2eR0fdzhf8XgHNBrcsqynFqLFb?= =?us-ascii?Q?qkeNvy3gq/vhrV9QGJ0QNYR1m1tvCDQjQcWmW3fjXiNzmytXO1UPzT+LvyFs?= =?us-ascii?Q?UO8nWiEePYNGFTLM4+KfdKx44wnaVX407sv51TVK0jHGZnoVD1HvIDa6hsaI?= =?us-ascii?Q?Bk48zIIrDDcSvrzdjaoELIuJFID4klmuZ+m8voThKroB4th44+Ws9M89HnHi?= =?us-ascii?Q?UPsUdEm+w1um+eh7iM4Zc+QgHuvgSG41MwythrrNxBt6z8LHcjZwIBIB6nr0?= =?us-ascii?Q?6ibKHzFnnuoYxocll7pjYxcyclmfmB0at0XNhwqmft8rgnS5kN+MjqW6X83G?= =?us-ascii?Q?aGriWGdEUMNF0rENIQmdzmayiM1e7qHK/UYE30tVZcz1rpebSkY0hmQwNaRC?= =?us-ascii?Q?oY2Ekl/Ybhqx1I+Y830qoP4rsxl0r9dz3xubQBYm6CUdXr+gSFx/xBEbbrXX?= =?us-ascii?Q?USCl8o/bnWarUm3VdbMi98uWg0ZB1+XN1oFTYV4ceB1+zFeGmgfcxQ94h9oR?= =?us-ascii?Q?VzI+nfgM23EhlMVMPpXWbn9+tBGTet/9KSPdSsAai1cX7XZiEhNp9yRKkoDv?= =?us-ascii?Q?kplWFOwjQar5vg5u4DAe40XaNqtxi6nKBPEqTB4Erc1YAmJpvw9pvV8IXOsi?= =?us-ascii?Q?k+bpZFOu10j1cosV1imkF4sS8Sak0cMsr0t6hutf/dG4S0XKPk8PE4V7VgjJ?= =?us-ascii?Q?ueSZ7qpkIv7WYSMaVQteEL8YN/OnHly0bVRjEBn7ZxT6LQBJKkYQK16gLGIg?= =?us-ascii?Q?2yTY0iUjbW+mWTyFCwcGvalMmW8XH5zM+GaFl53sKLy5ocMSOJnmvuqlfTnH?= =?us-ascii?Q?NnLXfkG4uSK/3mffPwaPpXlel5OpGpzqfml+zcP035lRXoavDDvVv4FETg0k?= =?us-ascii?Q?YovCh/12FzVfPHGYSsp59QGBZC/vVFYE4MNB9KVbV+td7NsyaQ/Kc6g1ksEq?= =?us-ascii?Q?Tvy2wL6p23AbmRrtR3Or7ZcFcp87MFP9uDZbzH7Re89NGGO9NkyTC87THkWz?= =?us-ascii?Q?8mJZu8FbdjYLE4ja3KRCzcQa/CAYMBzW9mVfB0YRHQGWHXGroZ8HbkLI6J6I?= =?us-ascii?Q?H0l9d460gy03Z3UEvCGPjnPl36+oqFYUcqL/AoNg/FTYW9JvwPua670ifCY0?= =?us-ascii?Q?R+DWmTICS7OMdxDNWSAd5pTbpZHaLCQMqlTXWzqnrYRcJPMe68VPv1u0bxVD?= =?us-ascii?Q?xR8fVaBMgPzscaZAxGGdGycfAMuedYqlgrNK4ygfHxz/L/d00vIk738Yqvvt?= =?us-ascii?Q?8jBZkifugJn5LdaqaGmyoX8zqQsPmgpUlw39N3qHk5F5RHhkXla4yIt4ux60?= =?us-ascii?Q?5gSZ5JXlHr0Lfhz97E8DtZM9muMjxLKdkK8DZwqlJtdVR7Xf83JjYYvHRcxZ?= =?us-ascii?Q?zFQSfJ44oi5bbwCL83d5lWbSFaeNEj3OitKeEb/l?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8ec00104-f45f-4353-15b9-08ddaf35bf1f X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:33.4398 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lfwN7cf0OqjbHZDqmD7QMyHY2xLZtDVFixExdgUhTZH1BvzXaZVp/Fdfe1AeQ1/lxYYGTjOfoiGaGr1nf88xTA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMyBTYWx0ZWRfX6KtQwpf8KPqI C51IxbRZSw6JP6m85Dbk5s/VFSHmbB6POJbgnaDfC34Z4VuKulwtfoEmuKNFhNxMkWSuy8g+HuD mOYY7eTdf/Lvgjgkb4ygaNUqzsb1bNPNZRSPiwE3UWyln0qi89fmwnoAmWkeZPac+tibIEEM6Md qwgftUR8+k0HvBHRoEXQIb9DIkPOJAYaDFU4PEGX18JfmaOBexNSpTp4igzSLJIjIMGv67fKnuY cIw0pK6WIjbGNstPY5S6MOhBmN51ythpjZdfhtycUxD7CimAhGf4MTCsYm5EJSDSObCwzXuN0sb FfMQa+evIuP6ynBKWsYNXDrMZoCO0ydX2O1LC06qiVNaqLFQSFrfhAwV+UCs/LYMBXhOV1/30aA d4WR2Qi/X40Nu5qBdP8iu8RT6PLO9XYSYcyz1cwSRySN/t5iCLD9Vgae7sp061nmwbVi1zf3 X-Authority-Analysis: v=2.4 cv=RcqQC0tv c=1 sm=1 tr=0 ts=6854117a cx=c_pps a=b9bpObgcEOTp51h0EtjbnQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=vgBDHfCE-WenxT7XOAUA:9 X-Proofpoint-ORIG-GUID: J7DoYLeYUycfWMV2Cq1eMrtUHo5VM9DD X-Proofpoint-GUID: J7DoYLeYUycfWMV2Cq1eMrtUHo5VM9DD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340321874116600 Content-Type: text/plain; charset="utf-8" Unlike most other messages, this is a server->client message, for when a server wants to do "DMA"; this is slow, so normally the server has memory directly mapped instead. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/protocol.h | 13 ++++- hw/vfio-user/proxy.h | 3 ++ hw/vfio-user/pci.c | 111 ++++++++++++++++++++++++++++++++++++++++ hw/vfio-user/proxy.c | 84 ++++++++++++++++++++++++++++++ 4 files changed, 210 insertions(+), 1 deletion(-) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 524f3d633a..3e9d8e576b 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -200,7 +200,18 @@ typedef struct { char data[]; } VFIOUserRegionRW; =20 -/*imported from struct vfio_bitmap */ +/* + * VFIO_USER_DMA_READ + * VFIO_USER_DMA_WRITE + */ +typedef struct { + VFIOUserHdr hdr; + uint64_t offset; + uint32_t count; + char data[]; +} VFIOUserDMARW; + +/* imported from struct vfio_bitmap */ typedef struct { uint64_t pgsize; uint64_t size; diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index a9a36e4110..2712a55463 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -101,6 +101,7 @@ void vfio_user_set_handler(VFIODevice *vbasedev, bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp); =20 VFIOUserFDs *vfio_user_getfds(int numfds); +void vfio_user_putfds(VFIOUserMsg *msg); =20 void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, uint32_t size, uint32_t flags); @@ -109,5 +110,7 @@ void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUser= Hdr *hdr, VFIOUserFDs *fds, int rsize); void vfio_user_send_nowait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, VFIOUserFDs *fds, int rsize); +void vfio_user_send_reply(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int size= ); +void vfio_user_send_error(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int erro= r); =20 #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 49d12763ab..040660d197 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -9,6 +9,7 @@ #include #include "qemu/osdep.h" #include "qapi-visit-sockets.h" +#include "qemu/error-report.h" =20 #include "hw/qdev-properties.h" #include "hw/vfio/pci.h" @@ -80,6 +81,95 @@ static void vfio_user_msix_teardown(VFIOPCIDevice *vdev) vdev->msix->pba_region =3D NULL; } =20 +static void vfio_user_dma_read(VFIOPCIDevice *vdev, VFIOUserDMARW *msg) +{ + PCIDevice *pdev =3D &vdev->pdev; + VFIOUserProxy *proxy =3D vdev->vbasedev.proxy; + VFIOUserDMARW *res; + MemTxResult r; + size_t size; + + if (msg->hdr.size < sizeof(*msg)) { + vfio_user_send_error(proxy, &msg->hdr, EINVAL); + return; + } + if (msg->count > proxy->max_xfer_size) { + vfio_user_send_error(proxy, &msg->hdr, E2BIG); + return; + } + + /* switch to our own message buffer */ + size =3D msg->count + sizeof(VFIOUserDMARW); + res =3D g_malloc0(size); + memcpy(res, msg, sizeof(*res)); + g_free(msg); + + r =3D pci_dma_read(pdev, res->offset, &res->data, res->count); + + switch (r) { + case MEMTX_OK: + if (res->hdr.flags & VFIO_USER_NO_REPLY) { + g_free(res); + return; + } + vfio_user_send_reply(proxy, &res->hdr, size); + break; + case MEMTX_ERROR: + vfio_user_send_error(proxy, &res->hdr, EFAULT); + break; + case MEMTX_DECODE_ERROR: + vfio_user_send_error(proxy, &res->hdr, ENODEV); + break; + case MEMTX_ACCESS_ERROR: + vfio_user_send_error(proxy, &res->hdr, EPERM); + break; + default: + error_printf("vfio_user_dma_read unknown error %d\n", r); + vfio_user_send_error(vdev->vbasedev.proxy, &res->hdr, EINVAL); + } +} + +static void vfio_user_dma_write(VFIOPCIDevice *vdev, VFIOUserDMARW *msg) +{ + PCIDevice *pdev =3D &vdev->pdev; + VFIOUserProxy *proxy =3D vdev->vbasedev.proxy; + MemTxResult r; + + if (msg->hdr.size < sizeof(*msg)) { + vfio_user_send_error(proxy, &msg->hdr, EINVAL); + return; + } + /* make sure transfer count isn't larger than the message data */ + if (msg->count > msg->hdr.size - sizeof(*msg)) { + vfio_user_send_error(proxy, &msg->hdr, E2BIG); + return; + } + + r =3D pci_dma_write(pdev, msg->offset, &msg->data, msg->count); + + switch (r) { + case MEMTX_OK: + if ((msg->hdr.flags & VFIO_USER_NO_REPLY) =3D=3D 0) { + vfio_user_send_reply(proxy, &msg->hdr, sizeof(msg->hdr)); + } else { + g_free(msg); + } + break; + case MEMTX_ERROR: + vfio_user_send_error(proxy, &msg->hdr, EFAULT); + break; + case MEMTX_DECODE_ERROR: + vfio_user_send_error(proxy, &msg->hdr, ENODEV); + break; + case MEMTX_ACCESS_ERROR: + vfio_user_send_error(proxy, &msg->hdr, EPERM); + break; + default: + error_printf("vfio_user_dma_write unknown error %d\n", r); + vfio_user_send_error(vdev->vbasedev.proxy, &msg->hdr, EINVAL); + } +} + /* * Incoming request message callback. * @@ -87,7 +177,28 @@ static void vfio_user_msix_teardown(VFIOPCIDevice *vdev) */ static void vfio_user_pci_process_req(void *opaque, VFIOUserMsg *msg) { + VFIOPCIDevice *vdev =3D opaque; + VFIOUserHdr *hdr =3D msg->hdr; + + /* no incoming PCI requests pass FDs */ + if (msg->fds !=3D NULL) { + vfio_user_send_error(vdev->vbasedev.proxy, hdr, EINVAL); + vfio_user_putfds(msg); + return; + } =20 + switch (hdr->command) { + case VFIO_USER_DMA_READ: + vfio_user_dma_read(vdev, (VFIOUserDMARW *)hdr); + break; + case VFIO_USER_DMA_WRITE: + vfio_user_dma_write(vdev, (VFIOUserDMARW *)hdr); + break; + default: + error_printf("vfio_user_pci_process_req unknown cmd %d\n", + hdr->command); + vfio_user_send_error(vdev->vbasedev.proxy, hdr, ENOSYS); + } } =20 /* diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 315f5fd350..198217565b 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -359,6 +359,10 @@ static int vfio_user_recv_one(VFIOUserProxy *proxy) *msg->hdr =3D hdr; data =3D (char *)msg->hdr + sizeof(hdr); } else { + if (hdr.size > proxy->max_xfer_size + sizeof(VFIOUserDMARW)) { + error_setg(&local_err, "vfio_user_recv request larger than max= "); + goto err; + } buf =3D g_malloc0(hdr.size); memcpy(buf, &hdr, sizeof(hdr)); data =3D buf + sizeof(hdr); @@ -673,6 +677,33 @@ void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUse= rHdr *hdr, qemu_mutex_unlock(&proxy->lock); } =20 +/* + * async send - msg can be queued, but will be freed when sent + */ +static void vfio_user_send_async(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds) +{ + VFIOUserMsg *msg; + int ret; + + if (!(hdr->flags & (VFIO_USER_NO_REPLY | VFIO_USER_REPLY))) { + error_printf("vfio_user_send_async on sync message\n"); + return; + } + + QEMU_LOCK_GUARD(&proxy->lock); + + msg =3D vfio_user_getmsg(proxy, hdr, fds); + msg->id =3D hdr->id; + msg->rsize =3D 0; + msg->type =3D VFIO_MSG_ASYNC; + + ret =3D vfio_user_send_queued(proxy, msg); + if (ret < 0) { + vfio_user_recycle(proxy, msg); + } +} + void vfio_user_wait_reqs(VFIOUserProxy *proxy) { VFIOUserMsg *msg; @@ -717,6 +748,59 @@ void vfio_user_wait_reqs(VFIOUserProxy *proxy) qemu_mutex_unlock(&proxy->lock); } =20 +/* + * Reply to an incoming request. + */ +void vfio_user_send_reply(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int size) +{ + + if (size < sizeof(VFIOUserHdr)) { + error_printf("vfio_user_send_reply - size too small\n"); + g_free(hdr); + return; + } + + /* + * convert header to associated reply + */ + hdr->flags =3D VFIO_USER_REPLY; + hdr->size =3D size; + + vfio_user_send_async(proxy, hdr, NULL); +} + +/* + * Send an error reply to an incoming request. + */ +void vfio_user_send_error(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int erro= r) +{ + + /* + * convert header to associated reply + */ + hdr->flags =3D VFIO_USER_REPLY; + hdr->flags |=3D VFIO_USER_ERROR; + hdr->error_reply =3D error; + hdr->size =3D sizeof(*hdr); + + vfio_user_send_async(proxy, hdr, NULL); +} + +/* + * Close FDs erroneously received in an incoming request. + */ +void vfio_user_putfds(VFIOUserMsg *msg) +{ + VFIOUserFDs *fds =3D msg->fds; + int i; + + for (i =3D 0; i < fds->recv_fds; i++) { + close(fds->fds[i]); + } + g_free(fds); + msg->fds =3D NULL; +} + static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); =20 --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340044; cv=pass; d=zohomail.com; s=zohoarc; b=GaNR5UaudG3j3n5LGmuTDjN0rYmp0hrOCXfH25F5GG0KZU5K3vpAcjecCOuWp41e12w2A7BUfX3RN+/1jmsC8NmUtn2eWftZBojejBpGJHti15HMcBUI21OWJYcE8mHydFhA0N5tu9XBD0XhLOkLYLzZ3rZJ3yZ0DhRJostojK8= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340044; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=27jtgJpP8H4CRv82Q0qbKLjrICwoiKBrJDaYNaJgJUs=; b=IR5jYyRLsIKFq5CUlnF36OL3BYx1tYhkktCO49cfmBYWGB2sjAhlzSCMgPdodetPMTWkJWUpbh5oLvqnnNLgAE5ji/+SvZ/Od3BVX8V9iWu1F0jvppfdDIL2TCD6dD3AOGKpJKA2CTQiItoVb+8wNXIyl5jtOQFpFl3TwITK1Wo= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340044511647.5308324687304; Thu, 19 Jun 2025 06:34:04 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFOZ-00053c-Rj; Thu, 19 Jun 2025 09:33:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNq-0004PT-1F for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:52 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNl-0008GT-Su for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:49 -0400 Received: from pps.filterd (m0127843.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55JAvH24013023; Thu, 19 Jun 2025 06:32:43 -0700 Received: from sj2pr03cu002.outbound.protection.outlook.com (mail-westusazon11023091.outbound.protection.outlook.com [52.101.44.91]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47976j3rag-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:43 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:35 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:35 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=27jtgJpP8H4CRv82Q0qbKLjrICwoiKBrJDaYNaJgJ Us=; b=K7yVakyIeYN+6EcyLPZuRm8YG184EBf/TZSgWHIXTe4CwmHMwbQLB6pJE Y8jM0IqusMyC8vrzKKsgFgCexvaFJwA8b6DDVvK2EJOn2PPpKxKtoE6A/QLdrGCq bdAqaVrj6uNCbJhEy30NC9oxq98TlVjTYiL9sanEDfhlLw61Np1mTr4lfg/7Cplo mqtSD2Gb95I7IZhTmdsKnIIAPbUZusOS7wVGDwnM/NNvdwytZ2mwfKxL2+mF+Wyc 1Lyj/YuK2E/0SFrVozWQmXBgHNLV7HTK4NryMoTGgcjbhszG4afNK4AoPbRPEmQR lJV1L/G7y6oPeOvdme4xdJPThwydg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=SdEOZmMdawCDaHHjiexYzE4tKT/3OhyDtDBYKiMX7PxgGtgJiXBaVj/jYzMOOLvmtLDVaWZZTdkNeglGiVGWXTTae/97qE3Py87gVKWZHE9KBTdm//0d2KjtGcGi3hUE3P1iZ096lJEA3ZA6OqcQlhZqP87Jch8oeR1cLYRPWnAKgZ4LzMSQ0U/WMrgPOkAvfN7RpyrqDMSFCsWr2amnWZdyAsCsHnQnzlrePMLx3730l2Q+NS7ojX1rRKdHTUSz0iWtj8xxTpZx8VNjaw76F0mpeyrViMAJ6e6L8AP6sWxEoRbiDPZJQYc3Ql2UVnMkxxQP7yI6tbLqdTaLiO2Fpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=27jtgJpP8H4CRv82Q0qbKLjrICwoiKBrJDaYNaJgJUs=; b=SjLTqsO6aBJNFUnN8XzEq8M4eCM/HCpHCDVBSrpnGglWGTV3yFV9nfKqkIc3GFIyO+yrxSGV8Fm5dqexXyl9SzqFfrJrl19dMWOXt/a/PZosKhEEi2fYb3/+ddlgBt+ba/MWITxT9gIKSfRxVZO8kPeCRhcUL/4QLg0I1Hj4bWWWn91Z+TzKFeknFH3lRmRCGNsMfDkIz76JmmI7RTwFTvp+ERh+CsdZN8sxl1RS5hS7EFaoNX+VZEFowzVOXffHHYQyzdF6s2P5HXtpzSHQT/N6Lqf3cGyy17mC8fb5bPjyoP1HvgRy14AeVc5vlW/WpIlbsQ5qQ2U0mP5E0XLH5w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=27jtgJpP8H4CRv82Q0qbKLjrICwoiKBrJDaYNaJgJUs=; b=vVv0XimqnCxQqmK3CQCO8hUR87md9UsaUKT2N8eH+XrX11R8p32cSskWBswXuzKDHXJWZZzz7kPi/hslMIoQyTVCNEssYq6+a0j8iJq+q7xK3UtHUssePDmfl9qknE3/lXgu8DYuawUv8/xunjyaJh6s/vbMsbkDOe51UCsu9VHfykU+PZhJrg0U4HBlRfsgcHCi5CFdLsCopzNtSuma7ma9wvIv00MVV+nBWZvV8kfaKFXOoLo1e11eohE6EKzkSVNVWL1a9rZRyJot7WT1wyXjyeT8CDgy7ygt3ysUT1PcoyBCzBmKV7MbatSbd/9xZxNFft5y/T7fLvuVMpBO2A== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 15/19] vfio-user: add 'x-msg-timeout' option Date: Thu, 19 Jun 2025 06:31:49 -0700 Message-ID: <20250619133154.264786-16-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 7a86b4f2-6efc-4887-cf7f-08ddaf35c087 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?j7ML2KhWasmGS5808mxFAuDZxxbSWxisKCn57kkhAHhXRVp3mA2P0okXN4wV?= =?us-ascii?Q?5D55TlGepDRsGygI76TrGTgiqZ6V6JGwW740/NzGgG1scyOr5K3tN7VN9mZ3?= =?us-ascii?Q?dnVFNlIKrcx0G423zdIItWukvjcv3wd5wux6KeEpEJsA+Y5UY9V0mA7qG2Yl?= =?us-ascii?Q?c4xKarQljA54VVQZ5dttVLmEg4gPO19Y0cA1+4FE4Iol80WKBIfiR6hB0/CB?= =?us-ascii?Q?vu70k9ewBsJa6lfjElgVBnVlxPagMSRfxzMSWzVyaVWCvtY8CyDsBLo9n+9b?= =?us-ascii?Q?WgKECTMqyhZwv1EErr1ujcP5f411lOH9SH0s/s4jldxoogHuMMbqVK1zxX1w?= =?us-ascii?Q?Vjdwysm0HFODL3P0RnItfYEijKcmx5yDCcshYIjZt1n4F+trW3N8CcXUCrQg?= =?us-ascii?Q?8Qa4i+PPx88Zv4HDL10nDCNy4FX3uXn+A6w4rWJQL50MA2iWAV0bmBdrrC1N?= =?us-ascii?Q?RfqG+x9GlUa6TA6tki8LzzNKkfjAYY86t/J8tjxzimUH6fhVeNdJPYL0jusc?= =?us-ascii?Q?hnFBO/3a7XDSZkmT7p1Mrosf1HIHSMP6I8kbQ3vlqwFX6vb912srt/NegvgM?= =?us-ascii?Q?NyeO3/Oql9rW54PJDScZvApA9ZoI7pqO/axA9YMRrJJ8lv1taVE0Iq9XtPy1?= =?us-ascii?Q?JyZ2WA6Yr8oOELZWVfqC2Zo1noVN5gFeKFze8DutFuM1zX0aEbQEBByigHNq?= =?us-ascii?Q?IuAogcFNljuJscpBXBwzRtH/tEDYqCFRkApduS5dx3+bIuPqPhqPQP5CfYeC?= =?us-ascii?Q?3XA4b6kyPQzTq/jvDGAvaR23UsgHDpg5brD7nziexlgeOKH11ocKhrbIMlUu?= =?us-ascii?Q?bAusVYKJaxe1NFh0rG1Ij8RzT4l8keUCkPtS9zGV+GL7rrpktT1LEyjAsFn+?= =?us-ascii?Q?Ny9wur8S+dN9WDhOETJw5T/2HPoUSxmzW1cPZQ/sqHOuEFaUd14I5+B/Gx81?= =?us-ascii?Q?bkPuHzWa+hB7b03N6zxkxSbxQTl1hUOZTzYVu0gjFk1FsFyFD2pgKBwwje4e?= =?us-ascii?Q?bO8XjmjBRJ05s6fHQW7uWnsmFYAp8/6cK3J/By7uD80FBgeBqGe6KXBY3E57?= =?us-ascii?Q?fBuRH8NZG1ZX0eSTwJowDPiq6CzPgCqu/zIcFk8UgMgkrptD7XaSzTEG72CW?= =?us-ascii?Q?zrwLBW4WIPHQhRm+GoG1q2F1UwpuRzqZwoi5MqW1xx518G89a0wDoZ4lmhqo?= =?us-ascii?Q?35GGHHFGC+cmMsCXXQFoNwr+0utpWvVSS/98PIu9NBr47x7pgc2w3o2MWhmw?= =?us-ascii?Q?NBe+5UWKwcLyPGqOe8WP6YeZ5NSnoCITiOnqLs0SFtXue8gEHlpNL+xOlKk1?= =?us-ascii?Q?grQFvJ27H0HwaLxyEdFimXKlHWk7G6Xh2uwVRoY58p+9EA9LauMKrQbgpW/b?= =?us-ascii?Q?lIv//7Fyp6nP/gHICl5DbU5dQPL0qr6TJ5xw37jIwGTyhYuqkg=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?AhUMT47+5VqlucGRAcCnLNbsbxxPieq4Z6gToWKXysR8yX8pWeOqqS4NHTy+?= =?us-ascii?Q?vGdV89TMFOuNWKR/I9TRWruTrqqaizGc/Uxq3YEpaWb/Z4bz8hWbET3Isdvd?= =?us-ascii?Q?GLDGdamZqkU0wxqMbNO6ahYRQ3Q+zVYQsTWutI+0aOh0ahsCLyFRiqbk20pO?= =?us-ascii?Q?Cl/pSTsoebqyBMDvUviMr6UdP/UCB3imO0VbqFHazMFIZTax3tqzFsxkD0Y6?= =?us-ascii?Q?mxYcyLfgEzTqLaRSbmG7m+LzeXEBsxR1JxgG7b+63aqdrqlO+x5G8oSNU1nV?= =?us-ascii?Q?fF3x43aVJMDVYcqNV85Ps4dgjeWP3hJrSkbS+Zv/Llaw/5bCVs6F7FYi4gWv?= =?us-ascii?Q?Pb9Dz0jYn1MvjE9HLMfXytP+WbAHsI5tO4BYvspyxKSSUJaj0rnURHA4st6o?= =?us-ascii?Q?89eXgZm6ZIxuH1zBmbCEV3uq3ltM9csKRPoDGN9/QYF87oDWLqCeJ4Sn3Q/y?= =?us-ascii?Q?le6YuDGe+xY3En1/th9nlXhDbmxq7Mqxg3c2h+T5kjVMaLAQiUN5klSsi4+I?= =?us-ascii?Q?b0jCvF/LBEU0Cp/UCf55mgcgsdXzc1wMoFPHikKPNEMxYDvHw1ivNlfIVwcg?= =?us-ascii?Q?/1A5EOhMiGcU+a3ie27HGFaFTdae4ZPXYDGJ2nDhV43ztZY5qwVmZcvNhkAd?= =?us-ascii?Q?usKCwDC7tvPR/36RrIPu+0axyGHCVXa90aVl2Vp6tUmCBL+izvasqVnYdPIj?= =?us-ascii?Q?0+qqEAN4R6kyZCVK2usAIXhkm4oAzlP2xgaZU2pDnS8HeMQrhWhn/bBDKIHj?= =?us-ascii?Q?o54Q+451m6og5btz2OXaIVi4zOCjfJ6/Jvl2sJtAOh4Uqg33MrNrOB/2G1af?= =?us-ascii?Q?n3PP0Js279d/YSy5EzxrVriXVrsXFYHUU0dYIeuTFtnvp6XwkWLETwZy2gEW?= =?us-ascii?Q?meBGmxPj/rbVuLpH2nwYn18ESqJ/v53M2n7pjHPAcqoM6633XxCLWD0ND76E?= =?us-ascii?Q?mtfAY8U6G6b6/wtbj1XmIAmVxIpkVByEJUItlUB+atewvFHRGbGs1b0lMy3V?= =?us-ascii?Q?1JTHMvGRbbH9bWEdBGdGih4aMRVzuccDzFrdpE7NTXwBipeBLsdYcazMLFwV?= =?us-ascii?Q?2Gkyda/52cOX4WLiSI1WGeAclQ1sb8HTIAvTY/FfiBjDYdUpLhNOIuQOCt7U?= =?us-ascii?Q?25yIzvRh6ffdtQ7ELv3e1Fwn4U7Hvvp8XbiCBoPR5fuKWQg2SxpEFgtDNJGb?= =?us-ascii?Q?q4DTIKzpdASlTRkgILtwsOcXQANgYRbigkahM99MOi8ejbOKofp7Qw6FDIAK?= =?us-ascii?Q?D3CHJ7+HfXJpuvNUcVNS3BsOx6A0vUjogNm+CAM4nL9LaXmgior6LHz+xFEM?= =?us-ascii?Q?IaCkvMz95Clavh7vw5moS0tqMI7G2Gcc4XrsLtjJdjtYC11uZm3Pxwa65BSs?= =?us-ascii?Q?RTUZoWjUkExu59l7Q9Lh0qtZ3avZVmadUQifeiYXJO60xj/WacIw5jUSaPQc?= =?us-ascii?Q?SGnvcTsBl65pKYSDGUoK9QyRhS05AySAWEwc8MFPDMZoXSCc8UUrobZXzQ6H?= =?us-ascii?Q?oLl4nlRkLDZOYFG1dJO3DW21CkpWR0PyVj9AXaPqqr8acFTwv+6ygajvBYQN?= =?us-ascii?Q?4v7HbRYH+sBZXwxuUc9IeNylwHsQvV2gzgcBceLk?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7a86b4f2-6efc-4887-cf7f-08ddaf35c087 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:35.7945 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1BAWhCZjO3cPBYkW3FJsdASECmT3fYCFzs0wkBgVJP+7s1j9wGkHzRYgPefhtn5j+KYunVR/cQsepD/XjHkY7w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMyBTYWx0ZWRfXwt9/mB121hXQ NIN/oRJVZVQuqPtmRXyq1Y926tewPWabsvw9AnR3pYiFkV9BQDsts0FQRL5N+RJAkDzhgzSQ2nK 53xUbgWSgh4HoXirj9dXP67L+7XZCALjhcDkLWR5orula+WcIVatpladwn4U/w5hiwWGui2R1Ma aMwq/pNRPOOGNq5tFAZQzWPsv/dqBxJgN9Uh/WNtwdePEE/JJUNHNUaDS12AIi6SX/8ZLNWE5Cu yXn/agog460ICVfPKybD6NOpz7nrGtjQH5yDYql/OKTnS/ZrO0VlTbzstaGncI+2KVYnSx4qA3y D7MXsuIdes9xbiiNA6py27XYQX8RbnMPGfW6tTvo4541nd+916mcWei4oEjjHD9DcyN3ZcH/V2X kA91uJbOcbbqUV597sP9hrRFXvSnW4q5Vovs21coZ+3uGTzNGUHQHjmz0dIWZsWS+dlP5DKA X-Authority-Analysis: v=2.4 cv=RcqQC0tv c=1 sm=1 tr=0 ts=6854117b cx=c_pps a=b9bpObgcEOTp51h0EtjbnQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=Ql6ANm6PklPDYutWWPYA:9 X-Proofpoint-ORIG-GUID: XHkatDk-2MIW6O8brOCl4jEs-EPmh8U8 X-Proofpoint-GUID: XHkatDk-2MIW6O8brOCl4jEs-EPmh8U8 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340046172116600 Content-Type: text/plain; charset="utf-8" By default, the vfio-user subsystem will wait 5 seconds for a message reply from the server. Add an option to allow this to be configurable. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/proxy.h | 1 + hw/vfio-user/pci.c | 5 +++++ hw/vfio-user/proxy.c | 7 ++++--- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 2712a55463..127defaefb 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -67,6 +67,7 @@ typedef struct VFIOUserProxy { uint64_t max_bitmap; uint64_t migr_pgsize; int flags; + uint32_t wait_time; QemuCond close_cv; AioContext *ctx; QEMUBH *req_bh; diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 040660d197..f260bea490 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -23,6 +23,7 @@ struct VFIOUserPCIDevice { VFIOPCIDevice device; SocketAddress *socket; bool send_queued; /* all sends are queued */ + uint32_t wait_time; /* timeout for message replies */ }; =20 /* @@ -267,6 +268,9 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Erro= r **errp) proxy->flags |=3D VFIO_PROXY_FORCE_QUEUED; } =20 + /* user specified or 5 sec default */ + proxy->wait_time =3D udev->wait_time; + if (!vfio_user_validate_version(proxy, errp)) { goto error; } @@ -398,6 +402,7 @@ static const Property vfio_user_pci_dev_properties[] = =3D { DEFINE_PROP_UINT32("x-pci-sub-device-id", VFIOPCIDevice, sub_device_id, PCI_ANY_ID), DEFINE_PROP_BOOL("x-send-queued", VFIOUserPCIDevice, send_queued, fals= e), + DEFINE_PROP_UINT32("x-msg-timeout", VFIOUserPCIDevice, wait_time, 5000= ), }; =20 static void vfio_user_pci_set_socket(Object *obj, Visitor *v, const char *= name, diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 198217565b..33c3532021 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -21,7 +21,6 @@ #include "qemu/main-loop.h" #include "system/iothread.h" =20 -static int wait_time =3D 5000; /* wait up to 5 sec for busy servers */ static IOThread *vfio_user_iothread; =20 static void vfio_user_shutdown(VFIOUserProxy *proxy); @@ -662,7 +661,8 @@ void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUser= Hdr *hdr, =20 if (ret =3D=3D 0) { while (!msg->complete) { - if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, wait_time)) { + if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, + proxy->wait_time)) { VFIOUserMsgQ *list; =20 list =3D msg->pending ? &proxy->pending : &proxy->outgoing; @@ -722,7 +722,8 @@ void vfio_user_wait_reqs(VFIOUserProxy *proxy) msg->type =3D VFIO_MSG_WAIT; proxy->last_nowait =3D NULL; while (!msg->complete) { - if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, wait_time)) { + if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, + proxy->wait_time)) { VFIOUserMsgQ *list; =20 list =3D msg->pending ? &proxy->pending : &proxy->outgoing; --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340155; cv=pass; d=zohomail.com; s=zohoarc; b=T+k2pagYPzlUDSGXZLWyzzz81iLONxALDeGZvWgtZAgq1UGLdmSK10XuAtkMLblUA/IqIPmqqbxT9O2dq3RRan3Zt3giX9B5KJY3x26VCHyusCHblzMnlPky4m79fFigCaYsTfx/WilbBweIWUVGwzEOo9IWVlq4svjVhfTBDY8= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340155; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=QxcXZhSRAeQ7QaN/9leAb3bkGCeXKMOPqXIAf2vlfCE=; b=XsqT/yPrIeOxxmdTIm5AzUqq5mh9sG/dtCG6b7pyvJ9LB6q1VSMebur6JkDA8ovFckn4/J9G5whGIeUfeSyaKUXMSMvgwm/tdGyR5NDvFTQshyd+0GVKDNg/ta+zYpnjFMorQkZNb04Vh96/mNgLLcdOnV2QXaYYNTMd6FXhETY= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340155450952.5067563540566; Thu, 19 Jun 2025 06:35:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFO7-0004fQ-9f; Thu, 19 Jun 2025 09:33:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNq-0004PV-AI for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:51 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNm-0008GW-8U for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:49 -0400 Received: from pps.filterd (m0127843.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55JAvH25013023; Thu, 19 Jun 2025 06:32:44 -0700 Received: from sj2pr03cu002.outbound.protection.outlook.com (mail-westusazon11023091.outbound.protection.outlook.com [52.101.44.91]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47976j3rag-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:43 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:38 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:38 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=QxcXZhSRAeQ7QaN/9leAb3bkGCeXKMOPqXIAf2vlf CE=; b=EJs4u0AUw6/+ntFOTuRY20gh1JDkLwFADqrG5D8ZJ52fE0/ydmtpp7LS2 EtHiZIYFiLAuMrmu5rIAjxQ4Q6B1QAT1EsJzx+MHSh9EpYEyhp0BMQkzmqmSQ/O7 qZuiqj73qgY36PGZYpAIBaafU0ki+LmNZBTIFWU9nyEtM0j5p1IWrWcEUsbv0IN7 nIxFtUmJYZcXmniXNrC1LPwYtPSJNfIhrbRLQbCwY3D/8TbnyGZSeIi98Q1F8kvI LuOu/KAEb3Xiy//kE9xH8AhRIu7qQPnXfubHMjccnt4ZjhIPz2SEvkV4ttGHaYka 5sHAYXwCm38Ot5eg53uLBZnh5MiFA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kSj5JHUmeZNy7QTRAwWorTu4CIuiVv1EkkAimlObIWeZW9cn5eSKQjsIaBbdQP+ILSUAKFuAA+lwgTGr7pkOWmLGsknntqNt++gRwkc5Is7QJf6E9P8hTwTCl9+296qxW9Fy1CaIWiAfIPwX4NX31SxOWjBm/SGHX4jwwK3PXyXCtrqThNmA8xi0YRtmbUBEA+3sjSLcvVdLq8nvVscHFdZSDuLarHxd569VVAe3sTqi0KY6WjjeeRE1l64JsJNveFwQwB7VUtkgOcYP1lWWu13HBZP/R3n+syzSu0KF9kqEMHRPBSpJlGBbxHpnVqjUZGvn4llGz++SDxrbNV3xYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QxcXZhSRAeQ7QaN/9leAb3bkGCeXKMOPqXIAf2vlfCE=; b=BmPkKWEXb7h72xe6dce6S0omu5WCtvrIEBxxmDO1oz0m6g6cmq8YonvFoX7ybjIr5AN6UWOAuJFo8bBLhWETCZgZVmeHrM1ftNiMGlYwgw4/rHc/Hh4BgS1+vL4P/Q2QqIwqrxl0CRpvYVDgfXwFYyXRChUxEUr7NMOkB+5ZhaHFxEDbFNHOgwx/+ev+fX+iSqOG+aPMNZytZcEzCLDmDS23OJ1gt2+OjZ6G5f68BD+NZ/7GJ5ClBtumhle+ZnF1VCoH5kjqvXmsFSOKNa7IZtpr2jC88nPfg4DWx4G2s5qIg73tidS+sJWtMMBZbvVprqhiQyBhvaccw8umupbc1A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QxcXZhSRAeQ7QaN/9leAb3bkGCeXKMOPqXIAf2vlfCE=; b=j8o/d8Dtzj5m4iAhu3z1AxnCRs0tH5yHxvr3/Oj7pcKu2Xn2RB6Wc3hybrm01ybD/G7M3mRfOovZ2VKyDizy22mxTXsW7oWfazl5Oppy6mFQS9yDMY3MuixkndsOriiWbc4/FvNRETMuJXssz/kYcEHsN7UCxtE6ba5h/E4OpsYQl5uqvRB3yxERlisDlaP99BepAzl+2h+fYLwfKWOEtwj2pkDapmFnSN+UG5NijCzi/xWL74lO9EpAtDWTckmVvh7vINiyuEI9H481eabrSYzosSHRFAZM82PEUCbb08tct7/wnjCFppX43iGhcRb52F9a9yK0HjY4MuFBIlvJ+A== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH v4 16/19] vfio-user: support posted writes Date: Thu, 19 Jun 2025 06:31:50 -0700 Message-ID: <20250619133154.264786-17-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: 9a6704c8-e9d1-440e-311f-08ddaf35c1bf x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?e17mp9OzvRT5N2blCEO5NuJQYfyIBRfzkuIG7dITh5lG+ukae7vZqNNmAdFZ?= =?us-ascii?Q?8+bDrvAg/nKeAQ45k+zleyMlLOcmf9e7evsCkRLhFJ4CBnJcTeDBuObU7hME?= =?us-ascii?Q?X3FEZk/GA+JVf/QiMeHlTqZIjMrlSwqRvqWGRj2CatkY42bpq1PCPjW4zyfL?= =?us-ascii?Q?x7EwY8eWT/HlpSj1dnb5OKspljfiZ+gXcaduTHCiBCyFpMh0fSYzd/zi6I6O?= =?us-ascii?Q?QuL755Z5XOn42yASGJWijFG57PWn/JLD2rKQBbZUMX/okwzFcSzdQM4kYKcG?= =?us-ascii?Q?Ago1YTVrIL+qNQS/MGAgiLXBCBdxDRLlaonnyjUiml9MckCFuCfKG0iCHDa6?= =?us-ascii?Q?RiVrU8lcYV+Zc7iGsNKz32QRFuNpkHJFBQZAA7TNFkS5WiPAKfNFX1Fda123?= =?us-ascii?Q?ZXgdL0xGeKWeEjTtVdIxrP/uyiHvzjh4Vtcw4i20ZkclREq32ORWszxHHpdJ?= =?us-ascii?Q?Q8PyUrBG2jlAopQ02JMCNbmvNHjL0BSj1u6tS7zmLgzLot7xv3xyBRppOT7F?= =?us-ascii?Q?ocU3yiiQZR/PYhO4HQITd1gno2mF7bQ9yjxg+Zd7yierJ9IG+gJTuDjLDmR0?= =?us-ascii?Q?/z2V86h35qyaEOQPzTY+YlQAcxnE1Ki/Og5ZLQEmHrN/4GKAhOYYONkWyL2/?= =?us-ascii?Q?rsMaU/MWfRhbibheUL57msfPVx3Aciiu+okY0znKAR96VQA52YrIan15t3up?= =?us-ascii?Q?U4vObYYhdluJGS2tRVtD0h5oVbfKcOkfNP3Br45h2dCB6ZxIh8qF/uWE3D3r?= =?us-ascii?Q?9g1354pBJNWSsUQ+E2akJofaIW4thyk/DmQqhfH7pdir7/xld4V4HGJ9zY+2?= =?us-ascii?Q?QLHR8QhdL72FDqjVc4q2ANupugCY89UWwCA9jvHOrneTFWRE6Q2iWW3Y55je?= =?us-ascii?Q?pvUWboByGOwNoh9t6xug7+1kEgejzLRr+qvV+XDv/ycgvA6wNS2BMjQElcV0?= =?us-ascii?Q?2VZnFek0zQl1JqaoUDAPFRb6h4W5e3QFxpqb1sabyo61aW9T8YnEKIPLd04q?= =?us-ascii?Q?KIJfEb4MCFdoqg78t2r3WYDfM7z/vpBGHx9KG8biPL4k6pktPCiuVZ+KV3Xk?= =?us-ascii?Q?zVd7QWsiCg0iaZoAYaaq/G8REcSXMHo56YTE5fXhEHr3MjN5Viodl3A+Ugqy?= =?us-ascii?Q?e6VVG8E3iqqxcy7gsrtSI8BYG5ZmZKUk22EuYG98zjpSNGiYs+R+Zx5I7+tO?= =?us-ascii?Q?LDCGpcfugxUN92lOdt5XlXq7SMIsXJwu8f5fmJiZDgc7dAG/gsPSBL5V7mcL?= =?us-ascii?Q?VVpu2DiKwYmw+2Y6zEIC3OL1jFgv6jZX6kMH+HnxFSUaA4nxE2M778Fjegzw?= =?us-ascii?Q?wQAE0R/LZ2wtf2bgJKP4irdyw2JwJwCpm4EZ6ph9KE+/aKFv9y9kuTLqB2bD?= =?us-ascii?Q?yQqI2aIZaP8/cIG4nrdFQ3MNX5s5Yq9cauG2+lNiMqxAhjB1OopLtbqAqjLq?= =?us-ascii?Q?JPoTzzczliU=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?UkjC07OHhvBDbC+bIQp/3cD/CPh2XWlkYBhXdghDA/TCJK6dZMwlOINUdMks?= =?us-ascii?Q?KH17CyX3+wzskDkIcMy9VKCAGqm8tM7hokXCL8i4jR/37o7bVK0dOMxT90O5?= =?us-ascii?Q?uWytVlMJc6k34V8JPIlZEbb5QwtZruozxOtKCbl7OUdfpZBX0fwNCZepCeZQ?= =?us-ascii?Q?xl2xCdUXNZPYItPKq8PeQNyLz3O0gNpeEIexZGXB9JCTyGUlis6xpaFBGzgy?= =?us-ascii?Q?QZg1CyVSVgM4tU2gM8g/L3vjz9GJ8qY3biwr5aiKVLJXGwVyFvINFlDYrKUS?= =?us-ascii?Q?uj7Yr4zCixEnGLCDsUxZh6m0PZkQCENeGtu3zqDNDW+BfwEjTIf60W2wZysk?= =?us-ascii?Q?eBV7cN+y4UDL0l1vgY0OEEbhHMK+3hniV+SEjYh9YiXgz7iVK4d5X+nxDVwG?= =?us-ascii?Q?VvZXqj1GE0pPxf8UKdOUGUwL8vMXr9L2u8y618t238MCZjIfCKS59zYR6API?= =?us-ascii?Q?FlVYhSWfzRx6MtHiptA3eHS8e4lPUQ/C0AIAWb9KveJxsj5wleim2AmPE6Tz?= =?us-ascii?Q?8Ze6Vbrhm1LE1OQF4qyDUOOI39Ww2asGh+8p7+5xfzH/Y3+SI/wnGbPH7IEs?= =?us-ascii?Q?R3r0y++k8TzDMLtA54oyCW8XnvKBIH1MYjP2JhIMKIevuv3IW3rgiaQNjH0D?= =?us-ascii?Q?pl46LcjAKscESSCB1a6W/Mra1P6W7ELzEKCyjzNGdlhisP1lnhKPuJb6ziRH?= =?us-ascii?Q?/e4mWtjuLiQfxvlS2tEx7AKPYDt6M5Zo6wLzjOeStOUYsPKxN78godrxX5yP?= =?us-ascii?Q?TOtvptQBq+lkcvyZRrUc/WgX7E1h6OLDy0M439EITzgnahRU8smVp6yoyFyR?= =?us-ascii?Q?/yo4fyT5ZEgjKaueOL+qCCwdCwKLxKeq3lWNVmcizxbjXw67V+dXy97Bsw9K?= =?us-ascii?Q?v1e8K7syI6DG2/gcWcFh7PUv34Y8WJNb/oIhSAjLOOTtZ5muEcz4CK6dKu6D?= =?us-ascii?Q?CtZ4xTYxAoDpv+8T4Ok/gL6qKgP33YXTmcvl/FrqLwOXdWcKEyJVf5ph1pUr?= =?us-ascii?Q?RzTjBjzbUkUYwn1ffeCkIB0h4vy971lTuDirLvSKopWF++pvub9GUPaqS2Ve?= =?us-ascii?Q?gua+OUUi2atWWb+YyujSKGsbTLo8PMvpiQWA3VI6lLCMP6HBAcWpbAghw3iN?= =?us-ascii?Q?YmJwaOzLQNfm2JAGVcsUKlojiymraABEfklTEkZj8hzx2zJiv+9WkaFNFZVb?= =?us-ascii?Q?kE3QetwiVI79rosqI10bkXjeSU+rcqy5bNEG7WtDnXzYFrqTh0TcQoMgoEZC?= =?us-ascii?Q?nEpI/05/DGMoDr/ODsz/Dw7+KWlqoTSOfs7DfU+x3BdTgOlxLYhwoTaUDi/J?= =?us-ascii?Q?mjXH30j/7vRQKNiZwve/iH92GWn7+Xov7Wsel1FJtPSzRTeymWWfbHhwuYmC?= =?us-ascii?Q?VfMayHPcdWajkZlyuhaI5F1hv6TX3EsKj2uZlmmmbdXZTeJnSzomCX20fKbK?= =?us-ascii?Q?zrz69pmV6cihwSvw9evE4BYwQtnoRejUXeXgcufa4pZhwIxK9VqenqW2QJ5J?= =?us-ascii?Q?m+JiOg8BJkfEdxTmdTIlwI1Kqkk7HHh3JNqOQwY4pB4VnX2Naa6WRQ7inKY2?= =?us-ascii?Q?aBuO37G/dr4iKLRHsCJZCllSwXI33irbQgjlcXRT?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9a6704c8-e9d1-440e-311f-08ddaf35c1bf X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:37.9802 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +4yEqa2/9WBJX0G0jKixFGv2XMLi5fpnB0Q4pd+asTaPRaHIkELyVcTyXLFWcseqQ2xkHYBLO89QuP+vZz1jBA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMyBTYWx0ZWRfX6Mum987ymqCB 08fvBIfp5MSFKgzxD0V36NtByXvSnZjgbQleUkjRR44xoAolWAIVYMmfnULk+JOAzV+yS+zAg2O TFnr2dVBpkGTWlkwjyzIGMWBznyGViwHo5RUH0htMwPdlEc0JqV2xUF+r91/kQ+xJkIWiBV1ZQF OHw3VY9d+Uu/NqyyqaUWvG2GPlEyl249hnlh/RcIv6Pr/8bxBUTwb0ujF/QoLpD0oflH8Kaao4S LaGf4Iln/3u1+/OYO+UvAHflGaHYma0lRv2LWQ8AD8mRaVxaJYUcx5L0AVj6xx8IWNJp8GuheqW lJEUe4vrzTNUXOhYR7o+nwLQMaBaNnMsX5fYpLXEvb7uohU5U+bXnDptxEpKaEb+1ezYLfZ6Rgy dPzcNWmNbK2TP1OVhKHgKqT3d687yzd4UU20JVozPIopjfsAn/x0ZThvs2sdBs1FkYkYxDqt X-Authority-Analysis: v=2.4 cv=RcqQC0tv c=1 sm=1 tr=0 ts=6854117b cx=c_pps a=b9bpObgcEOTp51h0EtjbnQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=64Cc0HZtAAAA:8 a=hLZCMK2IaNhFlHKNMJgA:9 X-Proofpoint-ORIG-GUID: XW4aMGBnZhdBRocmKlUOnMXBnEi3MPAd X-Proofpoint-GUID: XW4aMGBnZhdBRocmKlUOnMXBnEi3MPAd X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340157729116600 Content-Type: text/plain; charset="utf-8" Support an asynchronous send of a vfio-user socket message (no wait for a reply) when the write is posted. This is only safe when no regions are mappable by the VM. Add an option to explicitly disable this as well. Signed-off-by: John Levon --- hw/vfio-user/proxy.h | 5 +++++ hw/vfio-user/device.c | 37 ++++++++++++++++++++++++++++++++++--- hw/vfio-user/pci.c | 6 ++++++ hw/vfio-user/proxy.c | 12 ++++++++++-- 4 files changed, 55 insertions(+), 5 deletions(-) diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 127defaefb..28233f0d33 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -91,6 +91,7 @@ typedef struct VFIOUserProxy { /* VFIOProxy flags */ #define VFIO_PROXY_CLIENT 0x1 #define VFIO_PROXY_FORCE_QUEUED 0x4 +#define VFIO_PROXY_NO_POST 0x8 =20 typedef struct VFIODevice VFIODevice; =20 @@ -104,6 +105,8 @@ bool vfio_user_validate_version(VFIOUserProxy *proxy, E= rror **errp); VFIOUserFDs *vfio_user_getfds(int numfds); void vfio_user_putfds(VFIOUserMsg *msg); =20 +void vfio_user_disable_posted_writes(VFIOUserProxy *proxy); + void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, uint32_t size, uint32_t flags); void vfio_user_wait_reqs(VFIOUserProxy *proxy); @@ -111,6 +114,8 @@ void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUser= Hdr *hdr, VFIOUserFDs *fds, int rsize); void vfio_user_send_nowait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, VFIOUserFDs *fds, int rsize); +void vfio_user_send_async(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds); void vfio_user_send_reply(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int size= ); void vfio_user_send_error(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int erro= r); =20 diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index 5e67cd56e3..e787fd542d 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -91,10 +91,21 @@ static int vfio_user_get_region_info(VFIOUserProxy *pro= xy, trace_vfio_user_get_region_info(msgp->index, msgp->flags, msgp->size); =20 memcpy(info, &msgp->argsz, info->argsz); + + /* + * If at least one region is directly mapped into the VM, then we can = no + * longer rely on the sequential nature of vfio-user request handling = to + * ensure that posted writes are completed before a subsequent read. I= n this + * case, disable posted write support. This is a per-device property, = not + * per-region. + */ + if (info->flags & VFIO_REGION_INFO_FLAG_MMAP) { + vfio_user_disable_posted_writes(proxy); + } + return 0; } =20 - static int vfio_user_device_io_get_region_info(VFIODevice *vbasedev, struct vfio_region_info *in= fo, int *fd) @@ -270,6 +281,12 @@ static int vfio_user_device_io_region_read(VFIODevice = *vbasedev, uint8_t index, return msgp->count; } =20 +/* + * If this is a posted write, and VFIO_PROXY_NO_POST is not set, then we a= re OK + * to send the write to the socket without waiting for the server's reply: + * a subsequent read (of any region) will not pass the posted write, as all + * messages are handled sequentially. + */ static int vfio_user_device_io_region_write(VFIODevice *vbasedev, uint8_t = index, off_t off, unsigned count, void *data, bool post) @@ -277,21 +294,35 @@ static int vfio_user_device_io_region_write(VFIODevic= e *vbasedev, uint8_t index, VFIOUserRegionRW *msgp =3D NULL; VFIOUserProxy *proxy =3D vbasedev->proxy; int size =3D sizeof(*msgp) + count; + int flags =3D 0; int ret; =20 if (count > proxy->max_xfer_size) { return -EINVAL; } =20 + if (proxy->flags & VFIO_PROXY_NO_POST) { + post =3D false; + } + + if (post) { + flags |=3D VFIO_USER_NO_REPLY; + } + msgp =3D g_malloc0(size); - vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_WRITE, size, 0); + vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_WRITE, size, flags); msgp->offset =3D off; msgp->region =3D index; msgp->count =3D count; memcpy(&msgp->data, data, count); trace_vfio_user_region_rw(msgp->region, msgp->offset, msgp->count); =20 - /* Ignore post: all writes are synchronous/non-posted. */ + /* async send will free msg after it's sent */ + if (post) { + vfio_user_send_async(proxy, &msgp->hdr, NULL); + return count; + } + vfio_user_send_wait(proxy, &msgp->hdr, NULL, 0); if (msgp->hdr.flags & VFIO_USER_ERROR) { ret =3D -msgp->hdr.error_reply; diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index f260bea490..be71c77729 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -24,6 +24,7 @@ struct VFIOUserPCIDevice { SocketAddress *socket; bool send_queued; /* all sends are queued */ uint32_t wait_time; /* timeout for message replies */ + bool no_post; /* all region writes are sync */ }; =20 /* @@ -268,6 +269,10 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Err= or **errp) proxy->flags |=3D VFIO_PROXY_FORCE_QUEUED; } =20 + if (udev->no_post) { + proxy->flags |=3D VFIO_PROXY_NO_POST; + } + /* user specified or 5 sec default */ proxy->wait_time =3D udev->wait_time; =20 @@ -403,6 +408,7 @@ static const Property vfio_user_pci_dev_properties[] = =3D { sub_device_id, PCI_ANY_ID), DEFINE_PROP_BOOL("x-send-queued", VFIOUserPCIDevice, send_queued, fals= e), DEFINE_PROP_UINT32("x-msg-timeout", VFIOUserPCIDevice, wait_time, 5000= ), + DEFINE_PROP_BOOL("x-no-posted-writes", VFIOUserPCIDevice, no_post, fal= se), }; =20 static void vfio_user_pci_set_socket(Object *obj, Visitor *v, const char *= name, diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 33c3532021..0436994647 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -680,8 +680,8 @@ void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUser= Hdr *hdr, /* * async send - msg can be queued, but will be freed when sent */ -static void vfio_user_send_async(VFIOUserProxy *proxy, VFIOUserHdr *hdr, - VFIOUserFDs *fds) +void vfio_user_send_async(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds) { VFIOUserMsg *msg; int ret; @@ -802,6 +802,14 @@ void vfio_user_putfds(VFIOUserMsg *msg) msg->fds =3D NULL; } =20 +void +vfio_user_disable_posted_writes(VFIOUserProxy *proxy) +{ + WITH_QEMU_LOCK_GUARD(&proxy->lock) { + proxy->flags |=3D VFIO_PROXY_NO_POST; + } +} + static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); =20 --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340155; cv=pass; d=zohomail.com; s=zohoarc; b=EnFSBo6jkp6amk1bwLuyhPBsIZ5GiJjdK0xqm3P3/EA3wuOY45BCe+KsTf4Su5vT0ZJd/Q0zKQLARM1jX1EzPVxZpGcCKSCdEgPJpBpqwmpycMKtGjOncvkbl/2OSgClITP2SNJaX4rs8A67r1bSoYn9WzUjs5vrx11SN+ESOyQ= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340155; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=O7D1DH71ZL0cecuRVvta2JFUkHHepebG/jc/avoOLhs=; b=bt3S+5IK4KTJgeTSpfVWt0PA6i/6w3Gb8qagwxe3vSSNk1BpWyI5jVKOh6DEPIU6V2/91kazEOdVNnkMv9fpqHPJ3qWXIqCBCxEUNhJPDVmIHldfwpBn+sVK0zPJ+SZ6wVVcncIz/GcTqXLnSl6KZ0o7SfzROGtA8BC0VWjrGzI= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340155406891.5194558512284; Thu, 19 Jun 2025 06:35:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFOT-00053Q-7F; Thu, 19 Jun 2025 09:33:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNq-0004Sc-Us for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:51 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNm-0008Gi-NN for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:50 -0400 Received: from pps.filterd (m0127843.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55JAvH26013023; Thu, 19 Jun 2025 06:32:44 -0700 Received: from sj2pr03cu002.outbound.protection.outlook.com (mail-westusazon11023091.outbound.protection.outlook.com [52.101.44.91]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47976j3rag-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:44 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB9383.namprd02.prod.outlook.com (2603:10b6:510:289::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Thu, 19 Jun 2025 13:32:40 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:40 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=O7D1DH71ZL0cecuRVvta2JFUkHHepebG/jc/avoOL hs=; b=g5d0qPF+LHlxilL751NxmH0PdGKKw3ZjT+WUDmSFctUjwJrI/NFhoUVoX lILZNiJz6fUuA2d6SRacyti2AvRaSnkeT+gUe/S4qMxbKeQQLeJNQ6JNIVjDccDG Ney2rGDG6ye+IMwCE2gu7IkJIpIm2XVsEfK301ohYEAHMC15s2RAPZTvrb99nyLr j6H3vglHY1b7DCP1QK/325gQjW81UIXa9BgIn0+7hcOcjhqmvYmfONiufEAdkriQ gnN8ntb3JQezIT1lOeQWEWXNrIOnEyVW2b3MA2aaiqssVVkbsEgt7fRt07LmtYDD yp+611aUyDXK/0g6L+ER+EjVkI2FQ== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kYCA7dngeRif79uMWpMJCa7Dbh06trSjG7TUfCG73B5+2/ATVfYbFEGO2nFObOKOzgj+we6IELT6EQbuRmph/Ig9CmpyuVGk250dZmM6yID4tKiTyjMl0I1RIa56BuARMQSaPf8iVjIKGf8H4Zafzb4k3SQt+dQXyyuw6071+fqgCmh1K07kI9khzoL6sP1drVhwysFAgtuvZ4NOVrHy2Qjvxbms4+fqw5WRGbj/ih9Bk6ZqATb2XMdHe/hbDggHCPYu0Ci3pAWreNOjqTwLffE89nKroiIo0lr3wToccvHpyNGnOh0zxz5XTeo2Pbbg80WFwq6hU0V38zRGWBGk+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=O7D1DH71ZL0cecuRVvta2JFUkHHepebG/jc/avoOLhs=; b=N6GRRs4zOaCpCxZi4Ehlvwc0wSaKy176CQu4ekRMh3MQ4REijBzgGUaRJUNVC7YwNf+sqmWBWZEh+L4vTtzPRBWgQ++j4CSuYOyo86uQwKWFH/qako+eCTyVmi9D+qzWnNOxYYTYoD3G6uua7fQXGcyOK6ChTqOngA8x2oVGhHR2iSV6hyYFmNNjcviL9rH+FEipAgj0NzyZvKYFoMah7EvKZe+H1CaEbsAPgZzjveI3aAqeG/KGNrVATRM/Kxfxda1mot8uykks/d6ui3V7UXVsmbMYJSlQsrEiFCCAyqXkAwAdSO+txxk0miu5PeuhBu3ahGWF/Y5moB9KNFcE4A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O7D1DH71ZL0cecuRVvta2JFUkHHepebG/jc/avoOLhs=; b=lHJ2zjg5K2DM9WzFoAXNeVSMRmq1zexUNfKgHvydq8QgtnYvBjlReuklfaEB/SKGLAneCsrCXZ8mTQ/A+jaLxCtiEXp+1/WvIbRl6JRSPCxUsDkijdi+uRY4hgKnSGQOTbiZ+ledne/+W/j16r7+fvVk+gRJwAGOKoR4fcdOrMaIaxK8UlSk/8QVdWLqNrxZD4am+SAPy5HwY+thaBdGFsz0z9m1oggrZ5hcY6VyzxQZLvqgPbOB89GuZ5KxgCwXU9T2WqHy8LF+OfHLPB1RpuZwuKVdg6KYOmf/xNjgVfbt6v/4gy1HAVPGsLoX64MgSYFbPvKEakRy0bK/JhGyxw== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v4 17/19] vfio-user: add coalesced posted writes Date: Thu, 19 Jun 2025 06:31:51 -0700 Message-ID: <20250619133154.264786-18-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB9383:EE_ X-MS-Office365-Filtering-Correlation-Id: e50cfe9b-95e8-45d5-a098-08ddaf35c339 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?NdBhwUD4Q77muquk9oySbl7I9QDjz232ymE5uojwklmm//X4H/lyC5mfaueP?= =?us-ascii?Q?67maoP2W+2JONJwU3W1CwkeL/+7O/ACTcTlMtrWwji4KpotH+0bawJztcsj5?= =?us-ascii?Q?vzpNu1GP/SSnjArLzOCgVW2oQYmaB+uli5bOmk3I9HUrunmgotklMFsiTWBh?= =?us-ascii?Q?8eWr9Z/wiMETs+vG8e85i3Jfw1XzfmzEdDCdz3B6/9i8XkQF1CroSBEw2v14?= =?us-ascii?Q?AEKTECIRvX2V6GDIvc6IH4ygAYJQkFYDiJCyPA14vK/w+m6oaIlWsSfYK3bY?= =?us-ascii?Q?PlfRQr6v1GUu5gRNZ8clCXknVy0AyOfvig8RG9d5h/pH0pVMjMkaFrOYqtAr?= =?us-ascii?Q?5AIrl2gb09wxA0GlOAjrsEMEKX+b90VOyKe0vkrDMQaefHyIur2v5zNxrZwc?= =?us-ascii?Q?zk9tZorBmnenRBzL4PTdDLwCGXsjgfTjbVpcFnG3aoOQR5GxTLHPZ2v/g+6K?= =?us-ascii?Q?nhLY+/XCv32z2fHzlA9KDeZKY8uA7gyNyBaHXCkIZXiQdLfTk9YimMmgEtZo?= =?us-ascii?Q?bZS1oEaX4uMPjg6KKUzB+laXUzWMwXGpjt0ce2+OidD+4BeTjiIdse1UuMZT?= =?us-ascii?Q?e/FNZFOTAGhhbFfct7mbB28Ck+9rNjdyG6igqHlrR9bF6+IPK+RSP7jpxpfj?= =?us-ascii?Q?wDZtfh2HdjV24MCg/mUlFwuf/bXGt/hn09dmDkQ2QJoxpZZtGTTPKjWP+I22?= =?us-ascii?Q?1jutAOqcRn0gmvsUwflZPi+M0f7X8qwIRRRJdM/8c+K2K1O5Oitq8NEq9BKI?= =?us-ascii?Q?yqXNtupB7pWUSm9r1IgnTq57Xci1s2EOP1rRGUwJ0+S7O06WIAq/nDFO0yPt?= =?us-ascii?Q?yODc9rBR41oS5z12x93aBhJNlBamSozUuaPzIHe/LGbl6fzwalP+Ys+Y2bAq?= =?us-ascii?Q?s3UBkf0C8tdZNRdKxJLgPIEjFIXXdfA5l7E40yvCDx6rp33dghTljCXSabWl?= =?us-ascii?Q?8WeDzEV+vCcoCypl6XX66nCjFdsrGG4IPHkiUnicqlhGI6t5GZadpi/lTfL3?= =?us-ascii?Q?vgeWUfstSnyyt2FthtC+YoxzbYRue1Vm8AhS3cyiU0JfKqO71NLoks1hU6sG?= =?us-ascii?Q?VbA5yzZBUny6Id754S8/E8jrYi8cRm7jH/a8iBqkMFp3JoUvXaXgo5OI/urT?= =?us-ascii?Q?m1b50eLZv3lAvP3cJc5obgr2tGrv+0fIHuTST3I2SON9SqDhjgVu2TSCSMdL?= =?us-ascii?Q?M1uiWeVZJee48+3udR9IuWsy4bvPyAx6rv6E0Umezcf7v3khv7jrYwYOKG31?= =?us-ascii?Q?858do47qyqSCY1VFuOeCCgd8H+KZOmhCZnuCYGI9S4yGHHU96PRp7FT3QVYp?= =?us-ascii?Q?7QpyUlVTtOiLO9DVDUklZGAhWCPGhQy9zo+A2HbVBBNTBTQRYIDm65eVZucL?= =?us-ascii?Q?pbBqIHpyYRg+82qy5vdHTIC+EmTJAt4RXnxjGqdLEUPFZ61O2TKNP9e+iS2Q?= =?us-ascii?Q?FJakKrWy+gE=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?6Hr8wwNbxHd/wcbd2jSSL0NTOiVhL01QK7KRVkK2wO1HLA/SDyAD4zP/v0AX?= =?us-ascii?Q?tTBnwYM621U6lgb4UruwwKkUb2dPtubAjz3KJtUhjrKQQHkoRYIxAtuGK6Lj?= =?us-ascii?Q?wPS2msPnZYW+pthZcrWVirDiWyrJqi+MEjFfWYJ2vrW9DVHv4/IiLboqVi67?= =?us-ascii?Q?c9UD30kChnjuzoP1F/x4Alb5o+grXQnH2QrEK03H0XQdM1lnoInhs5aSs2gi?= =?us-ascii?Q?ElLrIuLD7dNyYAYBexDY/UJqGLoKQ/+L7jADhWdH8vMHyF/UzN5GQr1YDV1+?= =?us-ascii?Q?wbOWlIV4yS3f981/jueLWGL+y/p9FNRmNAO9yRHbT1pmZWLFaaPVm1PJzyg0?= =?us-ascii?Q?/P+T/4JW/9x2b3M4qDTjtd9T0wj7WEMhIBUoSUch3RQB2o7p4LjVVu7rg2+4?= =?us-ascii?Q?Iir+6/8vjfl+dnAhD9wR8nxBiomGQJWp1tm4zNFh56/s9S7AdYhUjC/bjXuw?= =?us-ascii?Q?mfa5HMWSZqTERaSzQ9aDC9xmOLvpx5MZJ9pQoom7f0lw1do/P0m2QCQ+Tu//?= =?us-ascii?Q?xSZi9my/frKreMSHA28PHhJAQCJPJbLKV2+yN8f2bHsyJeLJ9BdhCAowUqq4?= =?us-ascii?Q?Fwahcl1Winxty/9t1e5QjbhRVcH2Cp4vh1+11A7JK4luofgP6CewDPxvuHje?= =?us-ascii?Q?cO1E7mvXiDrtlyMqJFuFQAFFfrX0b2UFrz+M8HjpOJz54SulLuDOzI+DFGFJ?= =?us-ascii?Q?5+mBBPqlsf0dR0iG8eFg+50ZJ9VOhmhMgQp8Dn8ZQsFzfp5kD2pZv8c9Oddl?= =?us-ascii?Q?cvFwoj1kfrhja7HKWOXq2Q7UsXswVL1ToNGutf00gFDtubKUmR0JzJ+DBAtf?= =?us-ascii?Q?OCMorn8b/qNre4l2NGz8MluVSwtcdxb5HcnkITef/CeMwSAeLpPVHWxleSQh?= =?us-ascii?Q?Ra2jFhC6Asp/6MRu2ScQijuRR+m3HO0FA7XAeEOXsF0Sey0bL3ylLucAsuT4?= =?us-ascii?Q?kfDnZScLI31tlqq2CwkQOppiKzLEXer07ssfCN+xsh2LjpPmoPBDAXdPQSvB?= =?us-ascii?Q?OOeXUY6SLtJIEkdEjdIeL8G+FFs7R+mhDnkC0xhkmKIN+nxARkQR3LDvDFPa?= =?us-ascii?Q?ru+Zuqf9kJkGgvhdO9Ur1qs1xvLTy6Ckx8EtogL6Q/t6w/XQ2npwEF90qYVM?= =?us-ascii?Q?JhmF1gjwMe/6qk38IQv1hWZwuqWEJVoxBBieCu6L418y6xywjx7VguSZ1F1r?= =?us-ascii?Q?m/G7dM7a0YjLizcqm4oBw6KWXJGAwD06LUdKLyZY0opjr/RO5WF8r72zUSiz?= =?us-ascii?Q?AxoO1XDHbZFUPEm/LaYYPqAS87ZdfGZdrcfKayWBqbhLl1dKDCCoCbu+kipq?= =?us-ascii?Q?rnUsckvd/wpEa4rd7po+qE1+1We7G2TuXBsOwmZEDvzT9efmkooVp2AOEeDn?= =?us-ascii?Q?eWBNXUKpaQjm1ke0c0z7LXwjhc6CKqj9V+hnFOPCCyxLdfRiJB7MYBMo7tpp?= =?us-ascii?Q?J0kqTVlLYA9MdxR0lMYzBU+mKegHi9NEYimI8NYgxVm5YbbB7it3w59XsoSC?= =?us-ascii?Q?HeACYFzu36YRjmAMvE0sH1rlsBunamldaEuhJdKjEpnG/ilFmoIv5cBWhLTX?= =?us-ascii?Q?410ErlHhBVYxFUkQo2U53MW8onwYN8CzGe84EIUO?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: e50cfe9b-95e8-45d5-a098-08ddaf35c339 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:40.3189 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XzVid0GEaCBAWyPIwAZeKXQqbzqN+YQD1z5tK0SmHdDv89uRkdLL81rrVfumiD8g02eaZx9HIKqNvU9amjXaLg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB9383 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMyBTYWx0ZWRfX1I5qwmifbtna uMdMNN0GocgWkzr6WpTg8Mild3y83e9C0XNnDXqikcxp65UlkAJJ6Hhp/rF4Cg3o0zzFYZGKU24 IsbsUTMxuTk+yg53LnyiqcCx3OqAs7LhtIw8smmisYw6HBQv1/bYegKSPDyh37pwXGGQwQe8ocu s2n6vdFKxoGTIqpxaUu5Qp+TAJ8MjfnSr9Aj5+J5Rey0ENU1DO5JuKkg2claHr+Ed0SaPEAMJ8q jo3hltOXgM+DwlKZ5QRhjWQ1XoIS0HY4zPRKR6S8+ZnmAPCCu/8EyLchMmHbVaa+DwpQgEloGim Fnf7Z6vIBeeY11whG3GeJ3djB53tI5w+unYowEdgy+0ednlR1Yfls44N5KuObfa40f1geu1YYqf 0b81lWQOw+2Xr12Qm3igmSysTDNsitvyerdbM5TUFFCMEzK7GTBAUnJ7tSqGyER6qNTEi4V6 X-Authority-Analysis: v=2.4 cv=RcqQC0tv c=1 sm=1 tr=0 ts=6854117c cx=c_pps a=b9bpObgcEOTp51h0EtjbnQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=pwxC0n7aooic8zlvp-IA:9 X-Proofpoint-ORIG-GUID: 5ty87x5o4K8Qe5wr9Aq52B6CkLd0sOEO X-Proofpoint-GUID: 5ty87x5o4K8Qe5wr9Aq52B6CkLd0sOEO X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340157815116600 Content-Type: text/plain; charset="utf-8" Add new message to send multiple writes to server in a single message. Prevents the outgoing queue from overflowing when a long latency operation is followed by a series of posted writes. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/protocol.h | 21 ++++++++++ hw/vfio-user/proxy.h | 12 ++++++ hw/vfio-user/device.c | 40 +++++++++++++++++++ hw/vfio-user/proxy.c | 84 +++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 1 + 5 files changed, 158 insertions(+) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 3e9d8e576b..3249a4a6b6 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -39,6 +39,7 @@ enum vfio_user_command { VFIO_USER_DMA_WRITE =3D 12, VFIO_USER_DEVICE_RESET =3D 13, VFIO_USER_DIRTY_PAGES =3D 14, + VFIO_USER_REGION_WRITE_MULTI =3D 15, VFIO_USER_MAX, }; =20 @@ -72,6 +73,7 @@ typedef struct { #define VFIO_USER_CAP_PGSIZES "pgsizes" #define VFIO_USER_CAP_MAP_MAX "max_dma_maps" #define VFIO_USER_CAP_MIGR "migration" +#define VFIO_USER_CAP_MULTI "write_multiple" =20 /* "migration" members */ #define VFIO_USER_CAP_PGSIZE "pgsize" @@ -218,4 +220,23 @@ typedef struct { char data[]; } VFIOUserBitmap; =20 +/* + * VFIO_USER_REGION_WRITE_MULTI + */ +#define VFIO_USER_MULTI_DATA 8 +#define VFIO_USER_MULTI_MAX 200 + +typedef struct { + uint64_t offset; + uint32_t region; + uint32_t count; + char data[VFIO_USER_MULTI_DATA]; +} VFIOUserWROne; + +typedef struct { + VFIOUserHdr hdr; + uint64_t wr_cnt; + VFIOUserWROne wrs[VFIO_USER_MULTI_MAX]; +} VFIOUserWRMulti; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 28233f0d33..0486097781 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -85,6 +85,8 @@ typedef struct VFIOUserProxy { VFIOUserMsg *last_nowait; VFIOUserMsg *part_recv; size_t recv_left; + VFIOUserWRMulti *wr_multi; + int num_outgoing; enum proxy_state state; } VFIOUserProxy; =20 @@ -92,6 +94,11 @@ typedef struct VFIOUserProxy { #define VFIO_PROXY_CLIENT 0x1 #define VFIO_PROXY_FORCE_QUEUED 0x4 #define VFIO_PROXY_NO_POST 0x8 +#define VFIO_PROXY_USE_MULTI 0x16 + +/* coalescing high and low water marks for VFIOProxy num_outgoing */ +#define VFIO_USER_OUT_HIGH 1024 +#define VFIO_USER_OUT_LOW 128 =20 typedef struct VFIODevice VFIODevice; =20 @@ -119,4 +126,9 @@ void vfio_user_send_async(VFIOUserProxy *proxy, VFIOUse= rHdr *hdr, void vfio_user_send_reply(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int size= ); void vfio_user_send_error(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int erro= r); =20 +void vfio_user_flush_multi(VFIOUserProxy *proxy); +void vfio_user_create_multi(VFIOUserProxy *proxy); +void vfio_user_add_multi(VFIOUserProxy *proxy, uint8_t index, + off_t offset, uint32_t count, void *data); + #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index e787fd542d..e2664760cb 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -8,6 +8,8 @@ =20 #include "qemu/osdep.h" #include "qemu/error-report.h" +#include "qemu/lockable.h" +#include "qemu/thread.h" =20 #include "hw/vfio-user/device.h" #include "hw/vfio-user/trace.h" @@ -294,6 +296,7 @@ static int vfio_user_device_io_region_write(VFIODevice = *vbasedev, uint8_t index, VFIOUserRegionRW *msgp =3D NULL; VFIOUserProxy *proxy =3D vbasedev->proxy; int size =3D sizeof(*msgp) + count; + bool can_multi; int flags =3D 0; int ret; =20 @@ -309,6 +312,43 @@ static int vfio_user_device_io_region_write(VFIODevice= *vbasedev, uint8_t index, flags |=3D VFIO_USER_NO_REPLY; } =20 + /* write eligible to be in a WRITE_MULTI msg ? */ + can_multi =3D (proxy->flags & VFIO_PROXY_USE_MULTI) && post && + count <=3D VFIO_USER_MULTI_DATA; + + /* + * This should be a rare case, so first check without the lock, + * if we're wrong, vfio_send_queued() will flush any posted writes + * we missed here + */ + if (proxy->wr_multi !=3D NULL || + (proxy->num_outgoing > VFIO_USER_OUT_HIGH && can_multi)) { + + /* + * re-check with lock + * + * if already building a WRITE_MULTI msg, + * add this one if possible else flush pending before + * sending the current one + * + * else if outgoing queue is over the highwater, + * start a new WRITE_MULTI message + */ + WITH_QEMU_LOCK_GUARD(&proxy->lock) { + if (proxy->wr_multi !=3D NULL) { + if (can_multi) { + vfio_user_add_multi(proxy, index, off, count, data); + return count; + } + vfio_user_flush_multi(proxy); + } else if (proxy->num_outgoing > VFIO_USER_OUT_HIGH && can_mul= ti) { + vfio_user_create_multi(proxy); + vfio_user_add_multi(proxy, index, off, count, data); + return count; + } + } + } + msgp =3D g_malloc0(size); vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_WRITE, size, flags); msgp->offset =3D off; diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 0436994647..c9bcf39f4c 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -13,12 +13,14 @@ #include "hw/vfio-user/proxy.h" #include "hw/vfio-user/trace.h" #include "qapi/error.h" +#include "qobject/qbool.h" #include "qobject/qdict.h" #include "qobject/qjson.h" #include "qobject/qnum.h" #include "qemu/error-report.h" #include "qemu/lockable.h" #include "qemu/main-loop.h" +#include "qemu/thread.h" #include "system/iothread.h" =20 static IOThread *vfio_user_iothread; @@ -441,6 +443,11 @@ static void vfio_user_send(void *opaque) } qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, vfio_user_recv, NULL, NULL, proxy); + + /* queue empty - send any pending multi write msgs */ + if (proxy->wr_multi !=3D NULL) { + vfio_user_flush_multi(proxy); + } } } =20 @@ -461,6 +468,7 @@ static int vfio_user_send_one(VFIOUserProxy *proxy) } =20 QTAILQ_REMOVE(&proxy->outgoing, msg, next); + proxy->num_outgoing--; if (msg->type =3D=3D VFIO_MSG_ASYNC) { vfio_user_recycle(proxy, msg); } else { @@ -568,11 +576,18 @@ static int vfio_user_send_queued(VFIOUserProxy *proxy= , VFIOUserMsg *msg) { int ret; =20 + /* older coalesced writes go first */ + if (proxy->wr_multi !=3D NULL && + ((msg->hdr->flags & VFIO_USER_TYPE) =3D=3D VFIO_USER_REQUEST)) { + vfio_user_flush_multi(proxy); + } + /* * Unsent outgoing msgs - add to tail */ if (!QTAILQ_EMPTY(&proxy->outgoing)) { QTAILQ_INSERT_TAIL(&proxy->outgoing, msg, next); + proxy->num_outgoing++; return 0; } =20 @@ -586,6 +601,7 @@ static int vfio_user_send_queued(VFIOUserProxy *proxy, = VFIOUserMsg *msg) } if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { QTAILQ_INSERT_HEAD(&proxy->outgoing, msg, next); + proxy->num_outgoing =3D 1; qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, vfio_user_recv, proxy->ctx, vfio_user_send, proxy); @@ -1109,12 +1125,27 @@ static bool check_migr(VFIOUserProxy *proxy, QObjec= t *qobj, Error **errp) return caps_parse(proxy, qdict, caps_migr, errp); } =20 +static bool check_multi(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QBool *qb =3D qobject_to(QBool, qobj); + + if (qb =3D=3D NULL) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MULTI); + return false; + } + if (qbool_get_bool(qb)) { + proxy->flags |=3D VFIO_PROXY_USE_MULTI; + } + return true; +} + static struct cap_entry caps_cap[] =3D { { VFIO_USER_CAP_MAX_FDS, check_max_fds }, { VFIO_USER_CAP_MAX_XFER, check_max_xfer }, { VFIO_USER_CAP_PGSIZES, check_pgsizes }, { VFIO_USER_CAP_MAP_MAX, check_max_dma }, { VFIO_USER_CAP_MIGR, check_migr }, + { VFIO_USER_CAP_MULTI, check_multi }, { NULL } }; =20 @@ -1173,6 +1204,7 @@ static GString *caps_json(void) qdict_put_int(capdict, VFIO_USER_CAP_MAX_XFER, VFIO_USER_DEF_MAX_XFER); qdict_put_int(capdict, VFIO_USER_CAP_PGSIZES, VFIO_USER_DEF_PGSIZE); qdict_put_int(capdict, VFIO_USER_CAP_MAP_MAX, VFIO_USER_DEF_MAP_MAX); + qdict_put_bool(capdict, VFIO_USER_CAP_MULTI, true); =20 qdict_put_obj(dict, VFIO_USER_CAP, QOBJECT(capdict)); =20 @@ -1225,3 +1257,55 @@ bool vfio_user_validate_version(VFIOUserProxy *proxy= , Error **errp) trace_vfio_user_version(msgp->major, msgp->minor, msgp->capabilities); return true; } + +void vfio_user_flush_multi(VFIOUserProxy *proxy) +{ + VFIOUserMsg *msg; + VFIOUserWRMulti *wm =3D proxy->wr_multi; + int ret; + + proxy->wr_multi =3D NULL; + + /* adjust size for actual # of writes */ + wm->hdr.size -=3D (VFIO_USER_MULTI_MAX - wm->wr_cnt) * sizeof(VFIOUser= WROne); + + msg =3D vfio_user_getmsg(proxy, &wm->hdr, NULL); + msg->id =3D wm->hdr.id; + msg->rsize =3D 0; + msg->type =3D VFIO_MSG_ASYNC; + trace_vfio_user_wrmulti("flush", wm->wr_cnt); + + ret =3D vfio_user_send_queued(proxy, msg); + if (ret < 0) { + vfio_user_recycle(proxy, msg); + } +} + +void vfio_user_create_multi(VFIOUserProxy *proxy) +{ + VFIOUserWRMulti *wm; + + wm =3D g_malloc0(sizeof(*wm)); + vfio_user_request_msg(&wm->hdr, VFIO_USER_REGION_WRITE_MULTI, + sizeof(*wm), VFIO_USER_NO_REPLY); + proxy->wr_multi =3D wm; +} + +void vfio_user_add_multi(VFIOUserProxy *proxy, uint8_t index, + off_t offset, uint32_t count, void *data) +{ + VFIOUserWRMulti *wm =3D proxy->wr_multi; + VFIOUserWROne *w1 =3D &wm->wrs[wm->wr_cnt]; + + w1->offset =3D offset; + w1->region =3D index; + w1->count =3D count; + memcpy(&w1->data, data, count); + + wm->wr_cnt++; + trace_vfio_user_wrmulti("add", wm->wr_cnt); + if (wm->wr_cnt =3D=3D VFIO_USER_MULTI_MAX || + proxy->num_outgoing < VFIO_USER_OUT_LOW) { + vfio_user_flush_multi(proxy); + } +} diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 7ef98813b3..64fac9137f 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -11,6 +11,7 @@ vfio_user_get_region_info(uint32_t index, uint32_t flags,= uint64_t size) " index vfio_user_region_rw(uint32_t region, uint64_t off, uint32_t count) " regio= n %d offset 0x%"PRIx64" count %d" vfio_user_get_irq_info(uint32_t index, uint32_t flags, uint32_t count) " i= ndex %d flags 0x%x count %d" vfio_user_set_irqs(uint32_t index, uint32_t start, uint32_t count, uint32_= t flags) " index %d start %d count %d flags 0x%x" +vfio_user_wrmulti(const char *s, uint64_t wr_cnt) " %s count 0x%"PRIx64 =20 # container.c vfio_user_dma_map(uint64_t iova, uint64_t size, uint64_t off, uint32_t fla= gs, bool async_ops) " iova 0x%"PRIx64" size 0x%"PRIx64" off 0x%"PRIx64" fla= gs 0x%x async_ops %d" --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340169; cv=pass; d=zohomail.com; s=zohoarc; b=cqCmo1lFBtLmBS9ySD+MIu0GSw8FRBs2m0++qKq1oJSMrs6qIvfllIAFGCjm5h/k7LZgBbtaruhfrcccmosaUAKYx7CrRqxew8dZKhXTp89N66zYcuBdfl+a8byxS/bzesN68FAZ0pg1+/MAVPLeIrgVXZKx0toeQ3dkrDSRKdE= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340169; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=hMgoJgjOg7ssfmFRY5ufTDdstNzwMj4vOLAsc1Sx8QY=; b=Psk0zT7lq9AIdkW4A4gDv8RGrnwwv6IzKueMiZ2EYb6XDyZldywFS/c0uAz4MOKsaoJMT1B/NbBqpxVXxiPfNqeT2B51Q0+88IdHIVwgh8jHi/WsUoI2buoiebIqWvvkWc/iDeEw756FVXDfDV1Cp4bEuE9Ihrx9ykvlRrdUenk= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17503401694951002.0673211782479; Thu, 19 Jun 2025 06:36:09 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFOl-0005UP-Kf; Thu, 19 Jun 2025 09:33:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNq-0004Se-Us for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:51 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNn-0008H2-VA for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:50 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55J3EV5u001584; Thu, 19 Jun 2025 06:32:46 -0700 Received: from sj2pr03cu001.outbound.protection.outlook.com (mail-westusazon11022092.outbound.protection.outlook.com [52.101.43.92]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 4796mbkv13-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:46 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by CO1PR02MB8458.namprd02.prod.outlook.com (2603:10b6:303:156::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.19; Thu, 19 Jun 2025 13:32:43 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:42 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=hMgoJgjOg7ssfmFRY5ufTDdstNzwMj4vOLAsc1Sx8 QY=; b=EE0FvaOJRGWmscjEWVrBUN3C7JdCtlK1XIxyRshGZ86wiBP3EfsUeYTYE xe3zD+LcY/nvjkplPWZ6XBxbHnHFaCbhUAh0auf4f+tO72zCT3bwnNC3Se76VAQf 7b11n9q/WNGneE2bJpzpng5T1EQSy98EifCw1LzdHH3cTb7LJO+Bk5W8U5MTij/c gZfusJO6lUALdlCSLemYU9/RkQq30OFkFngJp+w6DvzSULhVPPxN8NkAnK4DhUQK 89/Uy5NSPyndbCKdWFUIygFRP2oC7mh2rk1m6pAjjclKK5KfJ7zjk+3Qin8d7IvH R5nd/jT8KjLYYOHRo2f76c4Ym5gzA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=mabBmBEud2xFOVQ9n6OB8IkXq2ZnbHT1p6bU9RgpiXB2o8RkzIcbdsZHG7DLYgdK7xQaAbKpKRJctCdYgDuoZZWdkiiqtp7pxXS3Dudj/EsDNLvAZNH8dz0pQpCF2cZtG9e2AiB8/hyRBFa7gku8jUigOVPcCS+TSjJS/VLCha4ImhDdr1bOCQO0AB/N8GhC9oJ3/54NbEtHlncyZedeINCMBszNOdn+6xR5YuX4YMRRsBgRYEZRFgt8oe/ifiJp4KVC9+qtvzofmfvCg2/E2iq5419HUMLgo6J4muiKGYh9g7mC/EUniT0dE4HjM4XTVybMhwcUStq6PQ6pCcK3FQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hMgoJgjOg7ssfmFRY5ufTDdstNzwMj4vOLAsc1Sx8QY=; b=sGq4WOVGchB0jOpnDZ6wWqFwLFVnVCw4994iGsQ40P9tYuZE55FX761rz3OKyAd4Su/5CH5/RS5vEt/+OCqW6yYn3xPmkwiMsy2vklqKbgfeQCVfChNZbDDSiKYYUxuP+zPQv3wFBtOScXAYMaIJet7qMOryfB5B8cDH/Uwn0s9HrDSyYuMX8WdZAHj+vaun1b4bxkZnQXu8ihzsoUr6elwFEgQC+QH0YZbFCzP6wQRnhyqG5tDIa2x+kHoYS5BR5LdbGlLj0/4B2qhy+moPE++JTxuoS0IV+XvH3hFsffBIO6Q+suuIxuBF9h0zoPadcSkRkjFg0mFzGVsWmuuM9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hMgoJgjOg7ssfmFRY5ufTDdstNzwMj4vOLAsc1Sx8QY=; b=eFGoOEyXPWVzfz29O3lWry0bNWovCogNRsptfkRDH4vXEwdCWPejleZMgwlzE9Auv0AsxP4qsNQARsgM30lyjpifY6zkSs9mXphNruC/zVF+s96lQ1FouR5RMmMlqMKHJAWgZNPrxOgZXtZcysebAjn8Z4n+k7I+7N9zoRBUb842PS8Hkx3uWtkjAsuXb/8KpAdsfEV/j2ESdxKdqTutey2chCaGGQaSgVDvnLp+kHwv/8bqhJdfKkYH2JcNHxw5121gLUWuyCmOZsxEftwJH96Rdm9vcrRldXnxydfUNnwI4tI4KIOHjMWIdazrp7PyLHMf6SWfS4p/NO+e9ihkUw== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH v4 18/19] docs: add vfio-user documentation Date: Thu, 19 Jun 2025 06:31:52 -0700 Message-ID: <20250619133154.264786-19-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|CO1PR02MB8458:EE_ X-MS-Office365-Filtering-Correlation-Id: 39d7767a-9ef7-410f-da1a-08ddaf35c479 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?TtGV3R1D85RVdr36npYxGE1MvlvxQkssUiCIJal5QeR1PeSQEhtgnmuTJMvf?= =?us-ascii?Q?JvKuBldRDQ7udoO6tVpoR2RHlZFCJVcbHrWDeeeE0YO4LPOKdSbQW1I/p/rm?= =?us-ascii?Q?fynOw00QxLwY4BAyoK0bjmRRK/AS4xv027GkxqM/w1k3yLuU7Tk8AU8StOv/?= =?us-ascii?Q?l7CFPfADAtkVRuSuvKPQiMJSPYYd/S5sqXmm/O50ADhryYqyGjEC/XPenfBG?= =?us-ascii?Q?b6ixZuQwazsvy4TfsP2crhMQ/qD5EvLAa0IK23x9K9ICBQLrgpCXbscc78hP?= =?us-ascii?Q?7DJ7AWKMApGrolJvAgE4bjIvqpbICz+vD8PMvaO/AzmhYZ5t3ZhSDdKoyNJn?= =?us-ascii?Q?u+X3IXOo3cY0/alZRpuK5vY9KOasmoLAKKWXU/WQW1nV37p5+clxMXa6F+aH?= =?us-ascii?Q?IEobeVt4mKmYECbGciewskFEKETFmidOlQZkCWETCuWzqKrtursAlolL3osJ?= =?us-ascii?Q?8EjSDUO0yesQtsmC/JO7L85qeDMn5jli5fIJ2s6KYEH3bl9GZgSiu289a2Fy?= =?us-ascii?Q?GoPSWW2bT4TqAHv08jWpfK3RPqcAWn33APoDYCziIJHNy3I8WrF3tlptic3k?= =?us-ascii?Q?UUDENFlvKZz9GUDRn7dvfbhVJWRkiRDsbhBFer7WaDtszlblw3u30xbDRGN6?= =?us-ascii?Q?Q1vn6588ses9f3PHk8lYjLLvcrEow5UDJ+kK9aaDFesUhzdCTAcbm/hZI4U5?= =?us-ascii?Q?c11+AouBL6asjx+7Nkp3wxGt2MeXVfrbQO6nI/iBV+SeeQxuB20D+fbVuOVz?= =?us-ascii?Q?QsVGuAJfT143eopOK+3GCKnLGklDTCmVPJnYBHj2UXx2/FKjnvp+bG0SuJ4c?= =?us-ascii?Q?rCTvHmaDzqlW+J8gTb1g5ONz33h1kPfa7cToNjcD7x2Fr682HbddGIGjZyOD?= =?us-ascii?Q?MVkFyR2nI5doSGA5l38fx16JmCC73nTRPKEK7+QOPJUon5+JCT7BrOjpNgMe?= =?us-ascii?Q?C8SdperW5QEJEHKd7GEDb9wd1AceqA9K2nT7SdnsVN56eY1RvVoJ0403TX4X?= =?us-ascii?Q?+a8abss9UvC5zSgO5UBF/HYA1CUNpiwEm3syizrMeaZuY8dOCy+w0au5mAgo?= =?us-ascii?Q?VgkQ1UnaD7lRNSt5/2NHqeHHmxa2wPOyunLR9uLEWYPRNDk/HI4UgrnnW37k?= =?us-ascii?Q?ZXra6+U/tkxgD4hVBjbwYIAC5Qp7TKK/AsFdvxI9Cm1D7lmLfmC4MhRcKTvp?= =?us-ascii?Q?dTv285Tlmi9xOm4tpnv+dd5EQ8hQ+lR6PyGC+HFdsijIlvu9++/+EJbUaXAk?= =?us-ascii?Q?1+iocscH8/GAMgH8T+ATMskjabeTgJE+apYuAD/sop8+DJaufljW8wciDC+1?= =?us-ascii?Q?3VzlS3P3FgMkPuRakKn1r8FP+NWq/Xsvhnulpo3pTCqNPbzzHLJG2VX8vF8W?= =?us-ascii?Q?gmEae7JJ2M+EC6iacFDvxtTzLaXoT8iHC2gJvKBJlDtSQfsXqti9VHIeu1gy?= =?us-ascii?Q?Lr8aLmCaGdg=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?3aG2KN/r58lNyw6rXYH3x2uB8OAJ48PZwx7Yr0Tc1BaROVt11RGXf6rvs7LY?= =?us-ascii?Q?IUHzQb+HaC4gJV2dRKlFo98g0ANedMSL78NKqMQYQVusw9RnSSOtu//fy3VG?= =?us-ascii?Q?TvXgajZ674w7dmDWPuZRccfw5L5SPcIMbmhqnVePl19PsTfRnnWmW+Zbv19Q?= =?us-ascii?Q?uuqnZ7gEVrdaBsJ925l4G/8Tp1P0596YfiegJJQ/yTrHmtH3Mps5oWJI8rW4?= =?us-ascii?Q?EUXagzdrxtxxhHqjiCH36u7Q17Euye+RI0OeB6ELxl6UluE3gDrUPoljNkPb?= =?us-ascii?Q?j59Ad1T4g8Lv9VG/luyev5KI0Vd1itGcaKS9dSfXQx2Tw8cAYeM5R05R6+j5?= =?us-ascii?Q?7lBdcl3BrVBuzfWP+JEN6T9m+6e6+koqJaJjKOTA5LjzrIGrQPVVVXw1qTO9?= =?us-ascii?Q?+kAt2lbt2o889d5dxNH2izxW8Y5iwUl8yNaT/LGcGMIPXDlwAjfW8DKeRsmb?= =?us-ascii?Q?eFHR5Uu89eEdnTWgcfbvouh7+HZt/2l6MQdZ8+eTwSqyupM/tTScogo6/Ijw?= =?us-ascii?Q?2PnNhqIVxlv/4+OkIzliomlx2RURAVnBRX32RFIBtPEOhIp6tV+tAZMyCf3t?= =?us-ascii?Q?iekpLMBAPskIpnu4SFcmv2xnjhvFfZ2TjJf38ZPML4RmoAHTsS1tk86nArQH?= =?us-ascii?Q?EwUVp0eT65bJ4CZXuxW7p68oZ17iu1F4jXfwC2VI2Qkggf6d5W3LywDOo+In?= =?us-ascii?Q?HhCABr0ciOf7jenbSNeT6BTQihd5cExwDv5Ii40AytjnJB3vd1u2fR1xn3sp?= =?us-ascii?Q?9NiLkNvDnc4S5z7oeOGkqLr7f2NWU3SR+1RI+8dECggw0JaB33+euSR/4/A2?= =?us-ascii?Q?OoKs0d9H0/J6FQxR9lvIXtqHdoP2EPs6JTxPPV3VNVz3YUEcuy27u7CsdNFL?= =?us-ascii?Q?kd/kOxTbmWnFeMQYOroEV2848SGCnaEdCTgt5jSajGTpaMLTVRYc/OZtsMvb?= =?us-ascii?Q?/SRpinh7qb8jon01VkOI7HKnI3EXQ+5i6QoShkY4pb3LU8VcSUPFf2mnBQJe?= =?us-ascii?Q?0VgmBVL1k9CRn0s1mleBO7+Xdbg3jl576p5ZuA0ISo5c2o430/watcUCkvm3?= =?us-ascii?Q?6K0YJKHFHBJQr2vHGGtbRccqIMSDpIHDY1SWXFVaZ80/wJWo9wnkPA3lX+Ke?= =?us-ascii?Q?+Bd4FPeL0lysZT9ZOmValO9Lv3SurbvUunxSoXxF7XQMMnPKhpJktNa3oUTY?= =?us-ascii?Q?+0RVq+Qr1LBGqM7xtulnOR3VMMPnuchnE3H+JmUJorlEvmh2f6n6U9g8PuEr?= =?us-ascii?Q?AVvWoTfXbgjA5GJL+FGXN/K8+AB/ag7BqoVHRt9cd3FnG2H/Lr3S2KQedZhd?= =?us-ascii?Q?lW6opr4YQetLyZwAeBQHRA8Zi9WfmWmGe4j3V8SoMYOyseZ8sU+ly8R01mZB?= =?us-ascii?Q?fmd9wzwE+3A/gli+nC2xf7vGCKstMjpRRu1vmACPoZ09Ieb1LSllfAp6NXM5?= =?us-ascii?Q?zt2Im8swQ0h5G4H10ODs7i9YA+9Kr6dSjCBE8Z2QD8aVkLxMFpNTyC8/f30s?= =?us-ascii?Q?FpsO+sCnNhDQiHbw+0F+lVrlIIo+asgr5rdYO5ev51cLE5Wqq1I1YptcJer2?= =?us-ascii?Q?4QhjNPWRZVwT7AMGTqkw+2NBzVrVhKZq/7S50WJv?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 39d7767a-9ef7-410f-da1a-08ddaf35c479 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:42.5205 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lFHYrixLeP3kYbXP81BCkfTDfhVhBaPJC6FXIjzTOzrcgBmfdoKNVp2KPtTP1QurgdCEF6RnpH7jfEnHqispAw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR02MB8458 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMyBTYWx0ZWRfX3zhSHi9X32L/ n5wtnffwyIuKv0zCA3oMn+ohz0uFWDWYYnBUNS6kyxYFv4TCAYFpTjg69hPhRWxxoPCMmSGi0An usaYwIoEnVcuwokulbfoQDVcRlaeEJ7hG71mJrmhAiTVQYkq9ruq881MLDb6rHcUsfjG8dDaPgj 5HWv6xPQ1aFvbNHoJGLu4nFuxRI86n+/EO8gvY/wMKDVe6quigYm7xN1EQbcpsdoaWi4K2axTnh RsaHn+UOOGZTOYgbCDvBnU1eC7xCD8lwYlbw3Etme2e5ZH3DjZOugz6cwojlv5dkzEjTrAqvFMm SqQzDDyUsCWfGjc6tqPbFwZ0zMbjIgLqp7pC9cB63Hlf5iwnWSaE2JSxPTQwM2Rq7/jszgQVdBN bJ4Ujph/AJjk+LxV/BpzCbYJXSnMGzJgAK8QelaxjG/kZLKBhnt+O5jvAY2fI/m0xwowF5Zi X-Proofpoint-GUID: W8b5LNX2zWirr8XFNYl_OzZXr18VN4PR X-Proofpoint-ORIG-GUID: W8b5LNX2zWirr8XFNYl_OzZXr18VN4PR X-Authority-Analysis: v=2.4 cv=cJXgskeN c=1 sm=1 tr=0 ts=6854117e cx=c_pps a=VT9wVgPwsxARPCPuKYV0eQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=NEAV23lmAAAA:8 a=64Cc0HZtAAAA:8 a=LOSiUy74ef7l4vre4JwA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340171770116600 Content-Type: text/plain; charset="utf-8" Add some basic documentation on vfio-user usage. Signed-off-by: John Levon --- docs/system/device-emulation.rst | 1 + docs/system/devices/vfio-user.rst | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+) create mode 100644 docs/system/devices/vfio-user.rst diff --git a/docs/system/device-emulation.rst b/docs/system/device-emulatio= n.rst index a1b0d7997e..911381643f 100644 --- a/docs/system/device-emulation.rst +++ b/docs/system/device-emulation.rst @@ -85,6 +85,7 @@ Emulated Devices devices/can.rst devices/ccid.rst devices/cxl.rst + devices/vfio-user.rst devices/ivshmem.rst devices/ivshmem-flat.rst devices/keyboard.rst diff --git a/docs/system/devices/vfio-user.rst b/docs/system/devices/vfio-u= ser.rst new file mode 100644 index 0000000000..5d8965d4ef --- /dev/null +++ b/docs/system/devices/vfio-user.rst @@ -0,0 +1,24 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D +vfio-user +=3D=3D=3D=3D=3D=3D=3D=3D=3D + +QEMU includes a ``vfio-user`` client. The ``vfio-user`` specification allo= ws for +implementing (PCI) devices in userspace outside of QEMU; it is similar to +``vhost-user`` in this respect (see :doc:`vhost-user`), but can emulate ar= bitrary +PCI devices, not just ``virtio``. Whereas ``vfio`` is handled by the host +kernel, ``vfio-user``, while similar in implementation, is handled entirel= y in +userspace. + +For example, SPDK includes a virtual PCI NVMe controller implementation; by +setting up a ``vfio-user`` UNIX socket between QEMU and SPDK, a VM can sen= d NVMe +I/O to the SPDK process. + +Presuming a suitable ``vfio-user`` server has opened a socket at +``/tmp/vfio-user.sock``, a device can be configured with for example: + +.. code-block:: console + +-device '{"driver": "vfio-user-pci","socket": {"path": "/tmp/vfio-user.soc= k", "type": "unix"}}' + +See `libvfio-user `_ for further +information. --=20 2.43.0 From nobody Sat Nov 15 14:51:45 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750340211; cv=pass; d=zohomail.com; s=zohoarc; b=iPU+9xvusU1gBhhU10BsMyDHfmpxVUVYOPBOYV8EDofsUSjudglNTrzbTqhKxshE/xlcejJtaCwuWTq2Vys32s07+P487FyOvssQc016Sht1JF6oelGz6OjjQPrUmzg96AnpbqHCADNdiWcSCjc9rmqvHhnjjuMvX+MxGGWjN4U= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750340211; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=mmwPN/KCaVj4cLTF35Ep7kkprwzzMG2rUaK6n2cv4N0=; b=BQQcPghJf7GXBAeiwcIXO8mUD3Jv7WRZhQ6y9KnKsDLokU/DGErm++6BOS3Qm+Q9vn8yE8gVq8IpVttHBOwEd76d2BWQCYImIeinuT1MFoiv8hVHLHNq9TdZdg1FeKfUFE1C6WFlL77OKgmL1A+Oz7poamUKNkbvET+rKIljC7c= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750340211725357.6393047522579; Thu, 19 Jun 2025 06:36:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uSFO7-0004fm-Dd; Thu, 19 Jun 2025 09:33:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNv-0004Xw-M5 for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:58 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSFNp-0008HE-OX for qemu-devel@nongnu.org; Thu, 19 Jun 2025 09:32:54 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55J3EV5w001584; Thu, 19 Jun 2025 06:32:47 -0700 Received: from sj2pr03cu001.outbound.protection.outlook.com (mail-westusazon11022092.outbound.protection.outlook.com [52.101.43.92]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 4796mbkv13-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Jun 2025 06:32:46 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by CO1PR02MB8458.namprd02.prod.outlook.com (2603:10b6:303:156::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.19; Thu, 19 Jun 2025 13:32:44 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8857.016; Thu, 19 Jun 2025 13:32:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=mmwPN/KCaVj4cLTF35Ep7kkprwzzMG2rUaK6n2cv4 N0=; b=CC6yV3qI6v2xGshNuhK8/ywQy/ojqd+p3R8zdlzFnydNsLDD1Jcm6OLed 8Ahwx5do2qQfMqbFKIYnhvThlPP1O+2O7FAl+4tsE7/WN18GuxLEbpyMEdii/9x9 l/guRoTMkcOAFWqetU3ZmOhpEjHfC8l5CPMxJfdO2XjTE3pyFmtSas+eBUCp+iOq EFk0ymvaERH+EEToYjJiq8JrhFtiuzZ0s4E46HO7aL41K7uI+MWAxKsnku69puqA /JR5eXzdBccCTYZzBLQhWV1l2al3iCcogmYta/aquhVNg9mjbsnGyBV+5wcSU/ZB nu/osOrsh1gUS2cBRUdkRDwujbD/A== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=uyruGBedFQWGjknNFDnTeVEyZnWXHcQ4Pm867wzD9DOtre0j8cpM6eKVIQiUM30EghLIzofpsWTx/oo7DyrsKvDOj9VBGAvm/hddYmg8plnbPKsj/a9DNe4HD6SiBF4A0LDoZj1dk5ZNFj2q6uWvLEfLr/b4luQq9Lz3+SfAsIc5SpRuN+ds6ZTocf4+M9HuF3UH+/mwpRATAucjpzl+VBm8JhkeDEjRollDs5p6NKo2n+0nCjE488PN3wBt2qBA6fkoLtKLnqsUrsMtYGllVh/PoFsAD6FBInEv6vRutdioUaYf+64/X4MM42Qu+2gpLqyASAlPmqvedByQ6amDSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mmwPN/KCaVj4cLTF35Ep7kkprwzzMG2rUaK6n2cv4N0=; b=MHvMcYUnAFzKN4RBQvY4EqF6icbo3yka7mTVa1YU68lJOGd+ctqvIEF8y2XwYHyh0urrRSHWeXPHRFMyog9K196965cFSQGpXGmyHvocKUmg9cVZZVFQjSgP5h7P7SD56WopSjkIhO94eU5rW//CVm/9Q/r6dsyv+/LXg2E5BZlPMRMsWS9LJKclN2XtfWQVvViT3Cm1f3e4b/4NX7cG+fH/DVkzABkX0ZlIrMJjpqozgT4zzPHRzNqBLPg/iUcTG45+LsKocmoKYddW3cDtpNJaaiG+SEIDgfVWlq/jSaJH7WSCxKOnyF2x6mpz5AgZldSF3mO634KNLm8VpNfxjA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mmwPN/KCaVj4cLTF35Ep7kkprwzzMG2rUaK6n2cv4N0=; b=CK1MmLHuFgHDvD6z94XW06VMpUJu/e1/0LcZD007E3WX3zjTprJB+I+iUUt+qcpMsj8+255L4TCG9xJ/rQRNWYSlov2LnPbTxPcN/7UGNrUC1tpcKBBcoeOHtWIvVHOaO1i0EqtArh8SXiPCxydsulUKzDrIA0+b11TQ1mM71KNiwzUtRDOyn1tWJNdrWUdEErbEkfHrF0HDpVfCEDz6Pp0MinMwfGTmnX87zUbuIeNBiFi2v+a7GvMQN8cij+raKfGhquOgMS1bOna5ViCCQ7OXJEG0KVGLW5bBFoT11dbFWxFfRDa1clTHuhmEPNYtMZMwJuABok7u6wesXNTP3w== From: John Levon To: qemu-devel@nongnu.org Cc: Thanos Makatos , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Levon , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH v4 19/19] vfio-user: introduce vfio-user protocol specification Date: Thu, 19 Jun 2025 06:31:53 -0700 Message-ID: <20250619133154.264786-20-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619133154.264786-1-john.levon@nutanix.com> References: <20250619133154.264786-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO3P265CA0025.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:387::6) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|CO1PR02MB8458:EE_ X-MS-Office365-Filtering-Correlation-Id: 3718f32e-204c-42a5-e654-08ddaf35c5c3 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?x4d9ie5IGOct03Seroxu654NNh2LY6vIfG3oujTd2vBbaeov4jkJcUZ3kCBY?= =?us-ascii?Q?ikXyWJ9VRocstNiHSb6uT7D9Vj8mVlc5gnI9O9beR/O3BgT/eWKpUruyLx2Q?= =?us-ascii?Q?m26DovoAPrwss63Lbo3aViyjXG44S1EcBRnYISq4uZQ68cx8d+CDn45GKg12?= =?us-ascii?Q?DRhXCUjsA8D2OBzUuFipjseQseAPIcRocO52ny47xyqV7A6CWnnyM3mBGAk5?= =?us-ascii?Q?imQCb9lnkLX8sMuzl4wgWL5s2moGC3TZ0sIEvQBNXM9u1NRFcWzFlJzmLbG+?= =?us-ascii?Q?8kDorPzzxlYk2DLQ47hcTp/pLH2WrQZWIDA8awja7OgUSg1JLKShHfhIOQFD?= =?us-ascii?Q?50drRAlea/hw/GTVMfDvEMqFjEaawNgAQnggR2c36F8vvMBEM2QNdBaIYOK6?= =?us-ascii?Q?x4WCYvtUjtuYlGiRNsIAk3tKWrnxcBHPoVhuKpc/RZz/xmN1rRuttz2fcSJn?= =?us-ascii?Q?ILrCz5rV0ooL4Bmh8CwsSuxC9tnpQekGe1vHpN/A4rIkxSq42BNiKybqcRLh?= =?us-ascii?Q?wnfDdtwMDmXeuhZiRfixulL5siEhT4RMiN1xmxuqZEWQdA79/6+/RO5Okd9I?= =?us-ascii?Q?5tBznkOGp3MryPGCptmL8WKirn+IVJCD2M9gr9qeFfL3K8lEPBKA5qiaPSHo?= =?us-ascii?Q?1RCpD+Vqv1+Pnq94rsKCh3IlA5Tm6iDzioPTYBKHiWfm8vyMAGmw/MdeCuBB?= =?us-ascii?Q?vZJlESnnEmdAZGPYugBcJ58qxAh9N1iHRmXWPsARigG/nlfHHR+pobsp0WiV?= =?us-ascii?Q?xpDWn4Bx5eNyXavjPCTCaJFX5H+axz+QOgbN8Hzs+32kGphScnW22ZTxWKUt?= =?us-ascii?Q?z/J2mPJZWfGnHwBlN5vrvq3Tx8o7qQMRZXJ14i4KSiWwO5u4V83GiMhAlkJa?= =?us-ascii?Q?RTJ5zD9lvRp5xV/A3L9ztVSLIWPXwVnecgRX9SIoWaO5gc1AaqDheUVY23lu?= =?us-ascii?Q?vwlZ8L+SDcJKjftbt6pLuxp4mim3/0NNH2Y3G92dsXwAT81hDg+VjUKx7ANF?= =?us-ascii?Q?MFMAJgcGQyscTWBnRZ71f54K8KtxKePHOKGAGuqe4j9ovbw2c7w2x5dMRWlU?= =?us-ascii?Q?vxVXAhTc+86kB7CBJn6oD1BNFgFNW7aYjJNxmZM9DSVUSn3NwmPcPwesW7B3?= =?us-ascii?Q?BVEdT6pPRwnZ3Xj8nBkFoxD/jSNo1BK3w2drE89T2O+Ozqg9wJeBXF2iSg9u?= =?us-ascii?Q?Gsr3NSnOrnWBLqT365OhEgcdS16283FOSmNn6Tz6BSgsfsIXK8TcjXyW64Py?= =?us-ascii?Q?4y9Hpw0VgzzerbJ7SM3DsV7NO78rjlXK3QuXJJbUoKV7Ro1geeWX+8Jygy2/?= =?us-ascii?Q?3hRU9x7FqcFHUSjY+rVEtzjrYwXyQ633wzxNS8cwMvUPhUaCONe6wdgDn6oC?= =?us-ascii?Q?SDRHItBVpEtn127Pv8Dq/whFXCGYWFNrXSBRZX07xa8FxdBkfHPvlyD9pEd3?= =?us-ascii?Q?izmOoFgoYk4=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?gswIskSxTFs5Xy9CJS3OtiT6H/78S5W7Ka11+xpd1b8UU3Gy63NHbfHX8qUG?= =?us-ascii?Q?39K7h60lzYcDmvUzWpNMJqteQ12GBcPRLkf4oNdE7PRbSq4/XzLfWzel4b5o?= =?us-ascii?Q?lkBi+Wg5A6MwXn4Hi6yxmQUo9bp3o/rlvOo6ueZHAQA5jY9ZGqtBMmYIy2ur?= =?us-ascii?Q?PbtT0AVApue2DRJs9UYVe1SSLhHQ2RJiEk9BPA+Gh2ZbKfQarV7MXFIL+Elf?= =?us-ascii?Q?9CkBzShKlVIu9EGrIQm3AYtD9hyeMnFh4wUtwkrGlU+yQqX1Z1tHSxsYrJU/?= =?us-ascii?Q?tJ9TjRPI/PtFWlBwDLPpuhEsJnGZL5UaNItSdDNvPUXElg+PDu11e5NMbvbz?= =?us-ascii?Q?hvTRmVQER3ncYSp++YILPKSkeuxB9OqJ6BBefew0Y1mACWbnMx0TttZhExPc?= =?us-ascii?Q?C1l2Ra5MO6/Hh349SfNNq188JV7c7yNkDJM1jhah0smj7LJPRSRfAkuoGVxA?= =?us-ascii?Q?oTbU+N3ozjW9tbe2Lrc09heDvg17POUOU4/L/PeVu+N+KUQrJUGHFH7Chc91?= =?us-ascii?Q?FUzi4RTISkdiKIdSfEtgLvoG0g/sJ8BezXy/kgVvZwYcwNJIgva/Ypvtiajs?= =?us-ascii?Q?6H5xHxTQzEIg44c7pB11un2FNKtaf3yzqqv1u8pxG3cx3p830vocpbdlNxu0?= =?us-ascii?Q?GGyr5f76znNfWrk88Qt818x70aQw0R+94yXGCz10Tbc3eTYKRE6chsqq4PMl?= =?us-ascii?Q?NUoFDIKhoDAm5YHDyRHWa0sZdfJZCMYTu163cWBGTbqZYX0OQWlcoBXmuUGX?= =?us-ascii?Q?dDR9LboUitIWF745iZqV8jnVSgy0bCKxxsCZxQBo4C679Sxl391pVo4TZ7jq?= =?us-ascii?Q?cqskr1Dxju2KreR71tjVThherNxdD7rn/9U91zs96XfdfJkrOkFllZR3mCNT?= =?us-ascii?Q?EqtozWOvt4cjn22Wd4vtus0fR3Wg9/1ycuiHwGSLqg8Nfm8djn49qMH9XWu2?= =?us-ascii?Q?Ayf0d2+fOv8dPFwpT4vHTk40BHpiCJUBxt7d2y42SD1y0l+FXymE5cwbzBP9?= =?us-ascii?Q?WaaSO5+RbFWzF5/qK+CAChRBkAqkrxWks9D8BFM03/c+WUyY0NDBN1FoF1pA?= =?us-ascii?Q?7PUb5jeKnlCBjXdIGM8N5Y2z8+JvKTond6Rb1P7ErguKT9efFKL2jiKzjEHf?= =?us-ascii?Q?hIUSJW+zGBijT4+UpUyKYe0exjpZzgXN173yPsY0WPxi6A6EucZtDNQgzC2g?= =?us-ascii?Q?K7DNTb9we/il8ZMlr2L127eWvx6KRTgDAQYZdql6VQbcYIvEi+wd1cCjB8uL?= =?us-ascii?Q?VOi4rK0qFomlD5iCMs12jP1E022AxHH5W69//r++xlZ2fSnN/m1psq+VQbKD?= =?us-ascii?Q?mWa+O13R7fR80fc6S0JZTiTtn3o5O0ZEnDZFlVG4UEMrEZ4eoTh9I5BfRW/q?= =?us-ascii?Q?fqIecL+Oxyj0fwB2zW91hvRycMO3F2tG3lskAa4yzYRDdyKtJaF5h54bRjir?= =?us-ascii?Q?mtH3wla8U5/IFynvQH7JTWVPDZ/nL8PwBWCZrwEb5t/sWq2r/vZQG6Zn20qs?= =?us-ascii?Q?VduM+kE4UgOT+m6f0tSt6liwXqDYymBxRJSqPrNgqvO0mKJpjogjwhRJn4ni?= =?us-ascii?Q?XpHcURZR/pvtdXJTaKN1WlKczTRSNOgdoj62sDRP?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3718f32e-204c-42a5-e654-08ddaf35c5c3 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2025 13:32:44.7499 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IxrGPs2Pij7Z4r5cSN8N9v0Gn4Xh6XR1RhaW0OZKR4LnFNmWuDYx0TRByu4Laz5j9lcIcJqMm35tKCWHWriHaw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR02MB8458 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjE5MDExMyBTYWx0ZWRfX+37XdySHKiXT sqo1EuJZo0+RVMewYfMu1/Fs8aHAudw8UhFutdn47SiqEKwz/w3GSAHbQBaSmNkYcyQ0wGPLTFR 4kP4b+CyCp+qiQwOXXmX3801F5dMSds0J/Jdg7XcrCtOmkZFHhu1gGZWIEXC0gpyTIe+O7u3A/1 sZqsGQBBQAz0khrCicGVwMYAQTfUyU9CFpxBhtUTZm2FNXeD9C2cU64Qs5G8KbVGRAAD62GQftA 5hG25CzRMvugIiYuAmyyQQltF5erkV14UlpdYtPB/pAH7oteKxmm1Ormesk/ItwD09dHR/laaoD Cv+FMl2EY8azeylQkS6q6nBk3zxHkmSlaggtqqYJ4c5xFdtYTwfXsEAUdkq2dzDvJgfeCOb6wCv nbEwYS2lQ/Bh/u7MEIbbPUuE+5T9K7C0VVsoihP8Efnb/ugF4tnrao41LnrQqBU8FBVvN0ft X-Proofpoint-GUID: _Bv-N09FG53xXNMyczrc9FiwmPvpCHxT X-Proofpoint-ORIG-GUID: _Bv-N09FG53xXNMyczrc9FiwmPvpCHxT X-Authority-Analysis: v=2.4 cv=cJXgskeN c=1 sm=1 tr=0 ts=6854117f cx=c_pps a=VT9wVgPwsxARPCPuKYV0eQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=VwQbUJbxAAAA:8 a=pGLkceISAAAA:8 a=64Cc0HZtAAAA:8 a=h9SPtozqHe0Wh_k42joA:9 a=x5O-bmgOEKQMiMGR:21 a=N0yriQgkpQ4A:10 a=HplO-upLQ7EA:10 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-06-19_05,2025-06-18_03,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -46 X-Spam_score: -4.7 X-Spam_bar: ---- X-Spam_report: (-4.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.897, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, WEIRD_QUOTING=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750340212460116600 Content-Type: text/plain; charset="utf-8" From: Thanos Makatos This patch introduces the vfio-user protocol specification (formerly known as VFIO-over-socket), which is designed to allow devices to be emulated outside QEMU, in a separate process. vfio-user reuses the existing VFIO defines, structs and concepts. It has been earlier discussed as an RFC in: "RFC: use VFIO over a UNIX domain socket to implement device offloading" Signed-off-by: Thanos Makatos Signed-off-by: John Levon --- MAINTAINERS | 3 +- docs/interop/index.rst | 1 + docs/interop/vfio-user.rst | 1518 ++++++++++++++++++++++++++++++++++++ 3 files changed, 1521 insertions(+), 1 deletion(-) create mode 100644 docs/interop/vfio-user.rst diff --git a/MAINTAINERS b/MAINTAINERS index 9bc1005b07..b63410cbde 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -4243,7 +4243,6 @@ F: hw/remote/proxy-memory-listener.c F: include/hw/remote/proxy-memory-listener.h F: hw/remote/iohub.c F: include/hw/remote/iohub.h -F: subprojects/libvfio-user F: hw/remote/vfio-user-obj.c F: include/hw/remote/vfio-user-obj.h F: hw/remote/iommu.c @@ -4254,6 +4253,8 @@ VFIO-USER: M: John Levon M: Thanos Makatos S: Supported +F: docs/interop/vfio-user.rst +F: docs/system/devices/vfio-user.rst F: hw/vfio-user/* F: include/hw/vfio-user/* F: subprojects/libvfio-user diff --git a/docs/interop/index.rst b/docs/interop/index.rst index 4b951ae416..c5829fb223 100644 --- a/docs/interop/index.rst +++ b/docs/interop/index.rst @@ -24,6 +24,7 @@ are useful for making QEMU interoperate with other softwa= re. qemu-ga-ref qemu-qmp-ref qemu-storage-daemon-qmp-ref + vfio-user vhost-user vhost-user-gpu vhost-vdpa diff --git a/docs/interop/vfio-user.rst b/docs/interop/vfio-user.rst new file mode 100644 index 0000000000..50be6cdc0c --- /dev/null +++ b/docs/interop/vfio-user.rst @@ -0,0 +1,1518 @@ +.. include:: +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D +vfio-user Protocol Specification +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D + +.. contents:: Table of Contents + +Introduction +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +vfio-user is a protocol that allows a device to be emulated in a separate +process outside of a Virtual Machine Monitor (VMM). vfio-user devices cons= ist +of a generic VFIO device type, living inside the VMM, which we call the cl= ient, +and the core device implementation, living outside the VMM, which we call = the +server. + +The vfio-user specification is partly based on the +`Linux VFIO ioctl interface `_. + +VFIO is a mature and stable API, backed by an extensively used framework. = The +existing VFIO client implementation in QEMU (``qemu/hw/vfio/``) can be lar= gely +re-used, though there is nothing in this specification that requires that +particular implementation. None of the VFIO kernel modules are required for +supporting the protocol, on either the client or server side. Some source +definitions in VFIO are re-used for vfio-user. + +The main idea is to allow a virtual device to function in a separate proce= ss in +the same host over a UNIX domain socket. A UNIX domain socket (``AF_UNIX``= ) is +chosen because file descriptors can be trivially sent over it, which in tu= rn +allows: + +* Sharing of client memory for DMA with the server. +* Sharing of server memory with the client for fast MMIO. +* Efficient sharing of eventfd's for triggering interrupts. + +Other socket types could be used which allow the server to run in a separa= te +guest in the same host (``AF_VSOCK``) or remotely (``AF_INET``). Theoretic= ally +the underlying transport does not necessarily have to be a socket, however= we do +not examine such alternatives. In this protocol version we focus on using = a UNIX +domain socket and introduce basic support for the other two types of socke= ts +without considering performance implications. + +While passing of file descriptors is desirable for performance reasons, su= pport +is not necessary for either the client or the server in order to implement= the +protocol. There is always an in-band, message-passing fall back mechanism. + +Overview +=3D=3D=3D=3D=3D=3D=3D=3D + +VFIO is a framework that allows a physical device to be securely passed th= rough +to a user space process; the device-specific kernel driver does not drive = the +device at all. Typically, the user space process is a VMM and the device = is +passed through to it in order to achieve high performance. VFIO provides a= n API +and the required functionality in the kernel. QEMU has adopted VFIO to all= ow a +guest to directly access physical devices, instead of emulating them in +software. + +vfio-user reuses the core VFIO concepts defined in its API, but implements= them +as messages to be sent over a socket. It does not change the kernel-based = VFIO +in any way, in fact none of the VFIO kernel modules need to be loaded to u= se +vfio-user. It is also possible for the client to concurrently use the curr= ent +kernel-based VFIO for one device, and vfio-user for another device. + +VFIO Device Model +----------------- + +A device under VFIO presents a standard interface to the user process. Man= y of +the VFIO operations in the existing interface use the ``ioctl()`` system c= all, and +references to the existing interface are called the ``ioctl()`` implementa= tion in +this document. + +The following sections describe the set of messages that implement the vfi= o-user +interface over a socket. In many cases, the messages are analogous to data +structures used in the ``ioctl()`` implementation. Messages derived from t= he +``ioctl()`` will have a name derived from the ``ioctl()`` command name. E= .g., the +``VFIO_DEVICE_GET_INFO`` ``ioctl()`` command becomes a +``VFIO_USER_DEVICE_GET_INFO`` message. The purpose of this reuse is to sh= are as +much code as feasible with the ``ioctl()`` implementation``. + +Connection Initiation +^^^^^^^^^^^^^^^^^^^^^ + +After the client connects to the server, the initial client message is +``VFIO_USER_VERSION`` to propose a protocol version and set of capabilitie= s to +apply to the session. The server replies with a compatible version and set= of +capabilities it supports, or closes the connection if it cannot support the +advertised version. + +Device Information +^^^^^^^^^^^^^^^^^^ + +The client uses a ``VFIO_USER_DEVICE_GET_INFO`` message to query the serve= r for +information about the device. This information includes: + +* The device type and whether it supports reset (``VFIO_DEVICE_FLAGS_``), +* the number of device regions, and +* the device presents to the client the number of interrupt types the devi= ce + supports. + +Region Information +^^^^^^^^^^^^^^^^^^ + +The client uses ``VFIO_USER_DEVICE_GET_REGION_INFO`` messages to query the +server for information about the device's regions. This information descri= bes: + +* Read and write permissions, whether it can be memory mapped, and whether= it + supports additional capabilities (``VFIO_REGION_INFO_CAP_``). +* Region index, size, and offset. + +When a device region can be mapped by the client, the server provides a fi= le +descriptor which the client can ``mmap()``. The server is responsible for +polling for client updates to memory mapped regions. + +Region Capabilities +""""""""""""""""""" + +Some regions have additional capabilities that cannot be described adequat= ely +by the region info data structure. These capabilities are returned in the +region info reply in a list similar to PCI capabilities in a PCI device's +configuration space. + +Sparse Regions +"""""""""""""" +A region can be memory-mappable in whole or in part. When only a subset of= a +region can be mapped by the client, a ``VFIO_REGION_INFO_CAP_SPARSE_MMAP`` +capability is included in the region info reply. This capability describes +which portions can be mapped by the client. + +.. Note:: + For example, in a virtual NVMe controller, sparse regions can be used so + that accesses to the NVMe registers (found in the beginning of BAR0) are + trapped (an infrequent event), while allowing direct access to the door= bells + (an extremely frequent event as every I/O submission requires a write to + BAR0), found in the next page after the NVMe registers in BAR0. + +Device-Specific Regions +""""""""""""""""""""""" + +A device can define regions additional to the standard ones (e.g. PCI inde= xes +0-8). This is achieved by including a ``VFIO_REGION_INFO_CAP_TYPE`` capabi= lity +in the region info reply of a device-specific region. Such regions are ref= lected +in ``struct vfio_user_device_info.num_regions``. Thus, for PCI devices this +value can be equal to, or higher than, ``VFIO_PCI_NUM_REGIONS``. + +Region I/O via file descriptors +------------------------------- + +For unmapped regions, region I/O from the client is done via +``VFIO_USER_REGION_READ/WRITE``. As an optimization, ioeventfds or ioregi= onfds +may be configured for sub-regions of some regions. A client may request +information on these sub-regions via ``VFIO_USER_DEVICE_GET_REGION_IO_FDS`= `; by +configuring the returned file descriptors as ioeventfds or ioregionfds, the +server can be directly notified of I/O (for example, by KVM) without takin= g a +trip through the client. + +Interrupts +^^^^^^^^^^ + +The client uses ``VFIO_USER_DEVICE_GET_IRQ_INFO`` messages to query the se= rver +for the device's interrupt types. The interrupt types are specific to the = bus +the device is attached to, and the client is expected to know the capabili= ties +of each interrupt type. The server can signal an interrupt by directly inj= ecting +interrupts into the guest via an event file descriptor. The client configu= res +how the server signals an interrupt with ``VFIO_USER_SET_IRQS`` messages. + +Device Read and Write +^^^^^^^^^^^^^^^^^^^^^ + +When the guest executes load or store operations to an unmapped device reg= ion, +the client forwards these operations to the server with +``VFIO_USER_REGION_READ`` or ``VFIO_USER_REGION_WRITE`` messages. The serv= er +will reply with data from the device on read operations or an acknowledgem= ent on +write operations. See `Read and Write Operations`_. + +Client memory access +-------------------- + +The client uses ``VFIO_USER_DMA_MAP`` and ``VFIO_USER_DMA_UNMAP`` messages= to +inform the server of the valid DMA ranges that the server can access on be= half +of a device (typically, VM guest memory). DMA memory may be accessed by the +server via ``VFIO_USER_DMA_READ`` and ``VFIO_USER_DMA_WRITE`` messages ove= r the +socket. In this case, the "DMA" part of the naming is a misnomer. + +Actual direct memory access of client memory from the server is possible i= f the +client provides file descriptors the server can ``mmap()``. Note that ``mm= ap()`` +privileges cannot be revoked by the client, therefore file descriptors sho= uld +only be exported in environments where the client trusts the server not to +corrupt guest memory. + +See `Read and Write Operations`_. + +Client/server interactions +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D + +Socket +------ + +A server can serve: + +1) one or more clients, and/or +2) one or more virtual devices, belonging to one or more clients. + +The current protocol specification requires a dedicated socket per +client/server connection. It is a server-side implementation detail whethe= r a +single server handles multiple virtual devices from the same or multiple +clients. The location of the socket is implementation-specific. Multiplexi= ng +clients, devices, and servers over the same socket is not supported in this +version of the protocol. + +Authentication +-------------- + +For ``AF_UNIX``, we rely on OS mandatory access controls on the socket fil= es, +therefore it is up to the management layer to set up the socket as require= d. +Socket types that span guests or hosts will require a proper authentication +mechanism. Defining that mechanism is deferred to a future version of the +protocol. + +Command Concurrency +------------------- + +A client may pipeline multiple commands without waiting for previous comma= nd +replies. The server will process commands in the order they are received.= A +consequence of this is if a client issues a command with the *No_reply* bi= t, +then subsequently issues a command without *No_reply*, the older command w= ill +have been processed before the reply to the younger command is sent by the +server. The client must be aware of the device's capability to process +concurrent commands if pipelining is used. For example, pipelining allows +multiple client threads to concurrently access device regions; the client = must +ensure these accesses obey device semantics. + +An example is a frame buffer device, where the device may allow concurrent +access to different areas of video memory, but may have indeterminate beha= vior +if concurrent accesses are performed to command or status registers. + +Note that unrelated messages sent from the server to the client can appear= in +between a client to server request/reply and vice versa. + +Implementers should be prepared for certain commands to exhibit potentially +unbounded latencies. For example, ``VFIO_USER_DEVICE_RESET`` may take an +arbitrarily long time to complete; clients should take care not to block +unnecessarily. + +Socket Disconnection Behavior +----------------------------- +The server and the client can disconnect from each other, either intention= ally +or unexpectedly. Both the client and the server need to know how to handle= such +events. + +Server Disconnection +^^^^^^^^^^^^^^^^^^^^ +A server disconnecting from the client may indicate that: + +1) A virtual device has been restarted, either intentionally (e.g. because= of a + device update) or unintentionally (e.g. because of a crash). +2) A virtual device has been shut down with no intention to be restarted. + +It is impossible for the client to know whether or not a failure is +intermittent or innocuous and should be retried, therefore the client shou= ld +reset the VFIO device when it detects the socket has been disconnected. +Error recovery will be driven by the guest's device error handling +behavior. + +Client Disconnection +^^^^^^^^^^^^^^^^^^^^ +The client disconnecting from the server primarily means that the client +has exited. Currently, this means that the guest is shut down so the devic= e is +no longer needed therefore the server can automatically exit. However, the= re +can be cases where a client disconnection should not result in a server ex= it: + +1) A single server serving multiple clients. +2) A multi-process QEMU upgrading itself step by step, which is not yet + implemented. + +Therefore in order for the protocol to be forward compatible, the server s= hould +respond to a client disconnection as follows: + + - all client memory regions are unmapped and cleaned up (including closin= g any + passed file descriptors) + - all IRQ file descriptors passed from the old client are closed + - the device state should otherwise be retained + +The expectation is that when a client reconnects, it will re-establish IRQ= and +client memory mappings. + +If anything happens to the client (such as qemu really did exit), the cont= rol +stack will know about it and can clean up resources accordingly. + +Security Considerations +----------------------- + +Speaking generally, vfio-user clients should not trust servers, and vice v= ersa. +Standard tools and mechanisms should be used on both sides to validate inp= ut and +prevent against denial of service scenarios, buffer overflow, etc. + +Request Retry and Response Timeout +---------------------------------- +A failed command is a command that has been successfully sent and has been +responded to with an error code. Failure to send the command in the first = place +(e.g. because the socket is disconnected) is a different type of error exa= mined +earlier in the disconnect section. + +.. Note:: + QEMU's VFIO retries certain operations if they fail. While this makes s= ense + for real HW, we don't know for sure whether it makes sense for virtual + devices. + +Defining a retry and timeout scheme is deferred to a future version of the +protocol. + +Message sizes +------------- + +Some requests have an ``argsz`` field. In a request, it defines the maximum +expected reply payload size, which should be at least the size of the fixed +reply payload headers defined here. The *request* payload size is defined = by the +usual ``msg_size`` field in the header, not the ``argsz`` field. + +In a reply, the server sets ``argsz`` field to the size needed for a full +payload size. This may be less than the requested maximum size. This may be +larger than the requested maximum size: in that case, the full payload is = not +included in the reply, but the ``argsz`` field in the reply indicates the = needed +size, allowing a client to allocate a larger buffer for holding the reply = before +trying again. + +In addition, during negotiation (see `Version`_), the client and server m= ay +each specify a ``max_data_xfer_size`` value; this defines the maximum data= that +may be read or written via one of the ``VFIO_USER_DMA/REGION_READ/WRITE`` +messages; see `Read and Write Operations`_. + +Protocol Specification +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +To distinguish from the base VFIO symbols, all vfio-user symbols are prefi= xed +with ``vfio_user`` or ``VFIO_USER``. In this revision, all data is in the +endianness of the host system, although this may be relaxed in future +revisions in cases where the client and server run on different hosts +with different endianness. + +Unless otherwise specified, all sizes should be presumed to be in bytes. + +.. _Commands: + +Commands +-------- +The following table lists the VFIO message command IDs, and whether the +message command is sent from the client or the server. + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Name Command Request Direction +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +``VFIO_USER_VERSION`` 1 client -> server +``VFIO_USER_DMA_MAP`` 2 client -> server +``VFIO_USER_DMA_UNMAP`` 3 client -> server +``VFIO_USER_DEVICE_GET_INFO`` 4 client -> server +``VFIO_USER_DEVICE_GET_REGION_INFO`` 5 client -> server +``VFIO_USER_DEVICE_GET_REGION_IO_FDS`` 6 client -> server +``VFIO_USER_DEVICE_GET_IRQ_INFO`` 7 client -> server +``VFIO_USER_DEVICE_SET_IRQS`` 8 client -> server +``VFIO_USER_REGION_READ`` 9 client -> server +``VFIO_USER_REGION_WRITE`` 10 client -> server +``VFIO_USER_DMA_READ`` 11 server -> client +``VFIO_USER_DMA_WRITE`` 12 server -> client +``VFIO_USER_DEVICE_RESET`` 13 client -> server +``VFIO_USER_REGION_WRITE_MULTI`` 15 client -> server +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Header +------ + +All messages, both command messages and reply messages, are preceded by a +16-byte header that contains basic information about the message. The head= er is +followed by message-specific data described in the sections below. + ++----------------+--------+-------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| Message ID | 0 | 2 | ++----------------+--------+-------------+ +| Command | 2 | 2 | ++----------------+--------+-------------+ +| Message size | 4 | 4 | ++----------------+--------+-------------+ +| Flags | 8 | 4 | ++----------------+--------+-------------+ +| | +-----+------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0-3 | Type | | +| | +-----+------------+ | +| | | 4 | No_reply | | +| | +-----+------------+ | +| | | 5 | Error | | +| | +-----+------------+ | ++----------------+--------+-------------+ +| Error | 12 | 4 | ++----------------+--------+-------------+ +| | 16 | variable | ++----------------+--------+-------------+ + +* *Message ID* identifies the message, and is echoed in the command's reply + message. Message IDs belong entirely to the sender, can be re-used (even + concurrently) and the receiver must not make any assumptions about their + uniqueness. +* *Command* specifies the command to be executed, listed in Commands_. It = is + also set in the reply header. +* *Message size* contains the size of the entire message, including the he= ader. +* *Flags* contains attributes of the message: + + * The *Type* bits indicate the message type. + + * *Command* (value 0x0) indicates a command message. + * *Reply* (value 0x1) indicates a reply message acknowledging a previ= ous + command with the same message ID. + * *No_reply* in a command message indicates that no reply is needed for = this + command. This is commonly used when multiple commands are sent, and o= nly + the last needs acknowledgement. + * *Error* in a reply message indicates the command being acknowledged had + an error. In this case, the *Error* field will be valid. + +* *Error* in a reply message is an optional UNIX errno value. It may be ze= ro + even if the Error bit is set in Flags. It is reserved in a command messa= ge. + +Each command message in Commands_ must be replied to with a reply message, +unless the message sets the *No_Reply* bit. The reply consists of the hea= der +with the *Reply* bit set, plus any additional data. + +If an error occurs, the reply message must only include the reply header. + +As the header is standard in both requests and replies, it is not included= in +the command-specific specifications below; each message definition should = be +appended to the standard header, and the offsets are given from the end of= the +standard header. + +``VFIO_USER_VERSION`` +--------------------- + +.. _Version: + +This is the initial message sent by the client after the socket connection= is +established; the same format is used for the server's reply. + +Upon establishing a connection, the client must send a ``VFIO_USER_VERSION= `` +message proposing a protocol version and a set of capabilities. The server +compares these with the versions and capabilities it supports and sends a +``VFIO_USER_VERSION`` reply according to the following rules. + +* The major version in the reply must be the same as proposed. If the clie= nt + does not support the proposed major, it closes the connection. +* The minor version in the reply must be equal to or less than the minor + version proposed. +* The capability list must be a subset of those proposed. If the server + requires a capability the client did not include, it closes the connecti= on. + +The protocol major version will only change when incompatible protocol cha= nges +are made, such as changing the message format. The minor version may change +when compatible changes are made, such as adding new messages or capabilit= ies, +Both the client and server must support all minor versions less than the +maximum minor version it supports. E.g., an implementation that supports +version 1.3 must also support 1.0 through 1.2. + +When making a change to this specification, the protocol version number mu= st +be included in the form "added in version X.Y" + +Request +^^^^^^^ + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D =3D=3D=3D= =3D +Name Offset Size +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D =3D=3D=3D= =3D +version major 0 2 +version minor 2 2 +version data 4 variable (including terminating NUL). Optional. +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D =3D=3D=3D= =3D + +The version data is an optional UTF-8 encoded JSON byte array with the fol= lowing +format: + ++--------------+--------+-----------------------------------+ +| Name | Type | Description | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| capabilities | object | Contains common capabilities that | +| | | the sender supports. Optional. | ++--------------+--------+-----------------------------------+ + +Capabilities: + ++--------------------+---------+------------------------------------------= ------+ +| Name | Type | Description = | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| max_msg_fds | number | Maximum number of file descriptors that c= an be | +| | | received by the sender in one message. = | +| | | Optional. If not specified then the recei= ver | +| | | must assume a value of ``1``. = | ++--------------------+---------+------------------------------------------= ------+ +| max_data_xfer_size | number | Maximum ``count`` for data transfer messa= ges; | +| | | see `Read and Write Operations`_. Optiona= l, | +| | | with a default value of 1048576 bytes. = | ++--------------------+---------+------------------------------------------= ------+ +| pgsizes | number | Page sizes supported in DMA map operation= s | +| | | or'ed together. Optional, with a default = value | +| | | of supporting only 4k pages. = | ++--------------------+---------+------------------------------------------= ------+ +| max_dma_maps | number | Maximum number DMA map windows that can b= e | +| | | valid simultaneously. Optional, with a = | +| | | value of 65535 (64k-1). = | ++--------------------+---------+------------------------------------------= ------+ +| migration | object | Migration capability parameters. If missi= ng | +| | | then migration is not supported by the se= nder. | ++--------------------+---------+------------------------------------------= ------+ +| write_multiple | boolean | ``VFIO_USER_REGION_WRITE_MULTI`` messages= | +| | | are supported if the value is ``true``. = | ++--------------------+---------+------------------------------------------= ------+ + +The migration capability contains the following name/value pairs: + ++-----------------+--------+----------------------------------------------= ----+ +| Name | Type | Description = | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+ +| pgsize | number | Page size of dirty pages bitmap. The smallest= | +| | | between the client and the server is used. = | ++-----------------+--------+----------------------------------------------= ----+ +| max_bitmap_size | number | Maximum bitmap size in ``VFIO_USER_DIRTY_PAGE= S`` | +| | | and ``VFIO_DMA_UNMAP`` messages. Optional, = | +| | | with a default value of 256MB. = | ++-----------------+--------+----------------------------------------------= ----+ + +Reply +^^^^^ + +The same message format is used in the server's reply with the semantics +described above. + +``VFIO_USER_DMA_MAP`` +--------------------- + +This command message is sent by the client to the server to inform it of t= he +memory regions the server can access. It must be sent before the server can +perform any DMA to the client. It is normally sent directly after the vers= ion +handshake is completed, but may also occur when memory is added to the cli= ent, +or if the client uses a vIOMMU. + +Request +^^^^^^^ + +The request payload for this message is a structure of the following forma= t: + ++-------------+--------+-------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+-------------+ +| flags | 4 | 4 | ++-------------+--------+-------------+ +| | +-----+------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | readable | | +| | +-----+------------+ | +| | | 1 | writeable | | +| | +-----+------------+ | ++-------------+--------+-------------+ +| offset | 8 | 8 | ++-------------+--------+-------------+ +| address | 16 | 8 | ++-------------+--------+-------------+ +| size | 24 | 8 | ++-------------+--------+-------------+ + +* *argsz* is the size of the above structure. Note there is no reply paylo= ad, + so this field differs from other message types. +* *flags* contains the following region attributes: + + * *readable* indicates that the region can be read from. + + * *writeable* indicates that the region can be written to. + +* *offset* is the file offset of the region with respect to the associated= file + descriptor, or zero if the region is not mappable +* *address* is the base DMA address of the region. +* *size* is the size of the region. + +This structure is 32 bytes in size, so the message size is 16 + 32 bytes. + +If the DMA region being added can be directly mapped by the server, a file +descriptor must be sent as part of the message meta-data. The region can be +mapped via the mmap() system call. On ``AF_UNIX`` sockets, the file descri= ptor +must be passed as ``SCM_RIGHTS`` type ancillary data. Otherwise, if the D= MA +region cannot be directly mapped by the server, no file descriptor must be= sent +as part of the message meta-data and the DMA region can be accessed by the +server using ``VFIO_USER_DMA_READ`` and ``VFIO_USER_DMA_WRITE`` messages, +explained in `Read and Write Operations`_. A command to map over an existi= ng +region must be failed by the server with ``EEXIST`` set in error field in = the +reply. + +Reply +^^^^^ + +There is no payload in the reply message. + +``VFIO_USER_DMA_UNMAP`` +----------------------- + +This command message is sent by the client to the server to inform it that= a +DMA region, previously made available via a ``VFIO_USER_DMA_MAP`` command +message, is no longer available for DMA. It typically occurs when memory is +subtracted from the client or if the client uses a vIOMMU. The DMA region = is +described by the following structure: + +Request +^^^^^^^ + +The request payload for this message is a structure of the following forma= t: + ++--------------+--------+------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++--------------+--------+------------------------+ +| flags | 4 | 4 | ++--------------+--------+------------------------+ +| address | 8 | 8 | ++--------------+--------+------------------------+ +| size | 16 | 8 | ++--------------+--------+------------------------+ + +* *argsz* is the maximum size of the reply payload. +* *flags* is unused in this version. +* *address* is the base DMA address of the DMA region. +* *size* is the size of the DMA region. + +The address and size of the DMA region being unmapped must match exactly a +previous mapping. + +Reply +^^^^^ + +Upon receiving a ``VFIO_USER_DMA_UNMAP`` command, if the file descriptor is +mapped then the server must release all references to that DMA region befo= re +replying, which potentially includes in-flight DMA transactions. + +The server responds with the original DMA entry in the request. + + +``VFIO_USER_DEVICE_GET_INFO`` +----------------------------- + +This command message is sent by the client to the server to query for basic +information about the device. + +Request +^^^^^^^ + ++-------------+--------+--------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+--------------------------+ +| flags | 4 | 4 | ++-------------+--------+--------------------------+ +| | +-----+-------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_DEVICE_FLAGS_RESET | | +| | +-----+-------------------------+ | +| | | 1 | VFIO_DEVICE_FLAGS_PCI | | +| | +-----+-------------------------+ | ++-------------+--------+--------------------------+ +| num_regions | 8 | 4 | ++-------------+--------+--------------------------+ +| num_irqs | 12 | 4 | ++-------------+--------+--------------------------+ + +* *argsz* is the maximum size of the reply payload +* all other fields must be zero. + +Reply +^^^^^ + ++-------------+--------+--------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+--------------------------+ +| flags | 4 | 4 | ++-------------+--------+--------------------------+ +| | +-----+-------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_DEVICE_FLAGS_RESET | | +| | +-----+-------------------------+ | +| | | 1 | VFIO_DEVICE_FLAGS_PCI | | +| | +-----+-------------------------+ | ++-------------+--------+--------------------------+ +| num_regions | 8 | 4 | ++-------------+--------+--------------------------+ +| num_irqs | 12 | 4 | ++-------------+--------+--------------------------+ + +* *argsz* is the size required for the full reply payload (16 bytes today) +* *flags* contains the following device attributes. + + * ``VFIO_DEVICE_FLAGS_RESET`` indicates that the device supports the + ``VFIO_USER_DEVICE_RESET`` message. + * ``VFIO_DEVICE_FLAGS_PCI`` indicates that the device is a PCI device. + +* *num_regions* is the number of memory regions that the device exposes. +* *num_irqs* is the number of distinct interrupt types that the device sup= ports. + +This version of the protocol only supports PCI devices. Additional devices= may +be supported in future versions. + +``VFIO_USER_DEVICE_GET_REGION_INFO`` +------------------------------------ + +This command message is sent by the client to the server to query for +information about device regions. The VFIO region info structure is define= d in +```` (``struct vfio_region_info``). + +Request +^^^^^^^ + ++------------+--------+------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+ +| argsz | 0 | 4 | ++------------+--------+------------------------------+ +| flags | 4 | 4 | ++------------+--------+------------------------------+ +| index | 8 | 4 | ++------------+--------+------------------------------+ +| cap_offset | 12 | 4 | ++------------+--------+------------------------------+ +| size | 16 | 8 | ++------------+--------+------------------------------+ +| offset | 24 | 8 | ++------------+--------+------------------------------+ + +* *argsz* the maximum size of the reply payload +* *index* is the index of memory region being queried, it is the only field + that is required to be set in the command message. +* all other fields must be zero. + +Reply +^^^^^ + ++------------+--------+------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+ +| argsz | 0 | 4 | ++------------+--------+------------------------------+ +| flags | 4 | 4 | ++------------+--------+------------------------------+ +| | +-----+-----------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_REGION_INFO_FLAG_READ | | +| | +-----+-----------------------------+ | +| | | 1 | VFIO_REGION_INFO_FLAG_WRITE | | +| | +-----+-----------------------------+ | +| | | 2 | VFIO_REGION_INFO_FLAG_MMAP | | +| | +-----+-----------------------------+ | +| | | 3 | VFIO_REGION_INFO_FLAG_CAPS | | +| | +-----+-----------------------------+ | ++------------+--------+------------------------------+ ++------------+--------+------------------------------+ +| index | 8 | 4 | ++------------+--------+------------------------------+ +| cap_offset | 12 | 4 | ++------------+--------+------------------------------+ +| size | 16 | 8 | ++------------+--------+------------------------------+ +| offset | 24 | 8 | ++------------+--------+------------------------------+ + +* *argsz* is the size required for the full reply payload (region info str= ucture + plus the size of any region capabilities) +* *flags* are attributes of the region: + + * ``VFIO_REGION_INFO_FLAG_READ`` allows client read access to the region. + * ``VFIO_REGION_INFO_FLAG_WRITE`` allows client write access to the regi= on. + * ``VFIO_REGION_INFO_FLAG_MMAP`` specifies the client can mmap() the reg= ion. + When this flag is set, the reply will include a file descriptor in its + meta-data. On ``AF_UNIX`` sockets, the file descriptors will be passed= as + ``SCM_RIGHTS`` type ancillary data. + * ``VFIO_REGION_INFO_FLAG_CAPS`` indicates additional capabilities found= in the + reply. + +* *index* is the index of memory region being queried, it is the only field + that is required to be set in the command message. +* *cap_offset* describes where additional region capabilities can be found. + cap_offset is relative to the beginning of the VFIO region info structur= e. + The data structure it points is a VFIO cap header defined in + ````. +* *size* is the size of the region. +* *offset* is the offset that should be given to the mmap() system call for + regions with the MMAP attribute. It is also used as the base offset when + mapping a VFIO sparse mmap area, described below. + +VFIO region capabilities +"""""""""""""""""""""""" + +The VFIO region information can also include a capabilities list. This lis= t is +similar to a PCI capability list - each entry has a common header that +identifies a capability and where the next capability in the list can be f= ound. +The VFIO capability header format is defined in ```` (``stru= ct +vfio_info_cap_header``). + +VFIO cap header format +"""""""""""""""""""""" + ++---------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D+ +| id | 0 | 2 | ++---------+--------+------+ +| version | 2 | 2 | ++---------+--------+------+ +| next | 4 | 4 | ++---------+--------+------+ + +* *id* is the capability identity. +* *version* is a capability-specific version number. +* *next* specifies the offset of the next capability in the capability lis= t. It + is relative to the beginning of the VFIO region info structure. + +VFIO sparse mmap cap header +""""""""""""""""""""""""""" + ++------------------+----------------------------------+ +| Name | Value | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| id | VFIO_REGION_INFO_CAP_SPARSE_MMAP | ++------------------+----------------------------------+ +| version | 0x1 | ++------------------+----------------------------------+ +| next | | ++------------------+----------------------------------+ +| sparse mmap info | VFIO region info sparse mmap | ++------------------+----------------------------------+ + +This capability is defined when only a subrange of the region supports +direct access by the client via mmap(). The VFIO sparse mmap area is defin= ed in +```` (``struct vfio_region_sparse_mmap_area`` and ``struct +vfio_region_info_cap_sparse_mmap``). + +VFIO region info cap sparse mmap +"""""""""""""""""""""""""""""""" + ++----------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D= =3D+ +| nr_areas | 0 | 4 | ++----------+--------+------+ +| reserved | 4 | 4 | ++----------+--------+------+ +| offset | 8 | 8 | ++----------+--------+------+ +| size | 16 | 8 | ++----------+--------+------+ +| ... | | | ++----------+--------+------+ + +* *nr_areas* is the number of sparse mmap areas in the region. +* *offset* and size describe a single area that can be mapped by the clien= t. + There will be *nr_areas* pairs of offset and size. The offset will be ad= ded to + the base offset given in the ``VFIO_USER_DEVICE_GET_REGION_INFO`` to for= m the + offset argument of the subsequent mmap() call. + +The VFIO sparse mmap area is defined in ```` (``struct +vfio_region_info_cap_sparse_mmap``). + + +``VFIO_USER_DEVICE_GET_REGION_IO_FDS`` +-------------------------------------- + +Clients can access regions via ``VFIO_USER_REGION_READ/WRITE`` or, if prov= ided, by +``mmap()`` of a file descriptor provided by the server. + +``VFIO_USER_DEVICE_GET_REGION_IO_FDS`` provides an alternative access mech= anism via +file descriptors. This is an optional feature intended for performance +improvements where an underlying sub-system (such as KVM) supports communi= cation +across such file descriptors to the vfio-user server, without needing to +round-trip through the client. + +The server returns an array of sub-regions for the requested region. Each +sub-region describes a span (offset and size) of a region, along with the +requested file descriptor notification mechanism to use. Each sub-region = in the +response message may choose to use a different method, as defined below. = The +two mechanisms supported in this specification are ioeventfds and ioregion= fds. + +The server in addition returns a file descriptor in the ancillary data; cl= ients +are expected to configure each sub-region's file descriptor with the reque= sted +notification method. For example, a client could configure KVM with the +requested ioeventfd via a ``KVM_IOEVENTFD`` ``ioctl()``. + +Request +^^^^^^^ + ++-------------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+------+ +| flags | 4 | 4 | ++-------------+--------+------+ +| index | 8 | 4 | ++-------------+--------+------+ +| count | 12 | 4 | ++-------------+--------+------+ + +* *argsz* the maximum size of the reply payload +* *index* is the index of memory region being queried +* all other fields must be zero + +The client must set ``flags`` to zero and specify the region being queried= in +the ``index``. + +Reply +^^^^^ + ++-------------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+------+ +| flags | 4 | 4 | ++-------------+--------+------+ +| index | 8 | 4 | ++-------------+--------+------+ +| count | 12 | 4 | ++-------------+--------+------+ +| sub-regions | 16 | ... | ++-------------+--------+------+ + +* *argsz* is the size of the region IO FD info structure plus the + total size of the sub-region array. Thus, each array entry "i" is at off= set + i * ((argsz - 32) / count). Note that currently this is 40 bytes for bot= h IO + FD types, but this is not to be relied on. As elsewhere, this indicates = the + full reply payload size needed. +* *flags* must be zero +* *index* is the index of memory region being queried +* *count* is the number of sub-regions in the array +* *sub-regions* is the array of Sub-Region IO FD info structures + +The reply message will additionally include at least one file descriptor i= n the +ancillary data. Note that more than one sub-region may share the same file +descriptor. + +Note that it is the client's responsibility to verify the requested values= (for +example, that the requested offset does not exceed the region's bounds). + +Each sub-region given in the response has one of two possible structures, +depending whether *type* is ``VFIO_USER_IO_FD_TYPE_IOEVENTFD`` or +``VFIO_USER_IO_FD_TYPE_IOREGIONFD``: + +Sub-Region IO FD info format (ioeventfd) +"""""""""""""""""""""""""""""""""""""""" + ++-----------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D+ +| offset | 0 | 8 | ++-----------+--------+------+ +| size | 8 | 8 | ++-----------+--------+------+ +| fd_index | 16 | 4 | ++-----------+--------+------+ +| type | 20 | 4 | ++-----------+--------+------+ +| flags | 24 | 4 | ++-----------+--------+------+ +| padding | 28 | 4 | ++-----------+--------+------+ +| datamatch | 32 | 8 | ++-----------+--------+------+ + +* *offset* is the offset of the start of the sub-region within the region + requested ("physical address offset" for the region) +* *size* is the length of the sub-region. This may be zero if the access s= ize is + not relevant, which may allow for optimizations +* *fd_index* is the index in the ancillary data of the FD to use for ioeve= ntfd + notification; it may be shared. +* *type* is ``VFIO_USER_IO_FD_TYPE_IOEVENTFD`` +* *flags* is any of: + + * ``KVM_IOEVENTFD_FLAG_DATAMATCH`` + * ``KVM_IOEVENTFD_FLAG_PIO`` + * ``KVM_IOEVENTFD_FLAG_VIRTIO_CCW_NOTIFY`` (FIXME: makes sense?) + +* *datamatch* is the datamatch value if needed + +See https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt, *4.59 +KVM_IOEVENTFD* for further context on the ioeventfd-specific fields. + +Sub-Region IO FD info format (ioregionfd) +""""""""""""""""""""""""""""""""""""""""" + ++-----------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D+ +| offset | 0 | 8 | ++-----------+--------+------+ +| size | 8 | 8 | ++-----------+--------+------+ +| fd_index | 16 | 4 | ++-----------+--------+------+ +| type | 20 | 4 | ++-----------+--------+------+ +| flags | 24 | 4 | ++-----------+--------+------+ +| padding | 28 | 4 | ++-----------+--------+------+ +| user_data | 32 | 8 | ++-----------+--------+------+ + +* *offset* is the offset of the start of the sub-region within the region + requested ("physical address offset" for the region) +* *size* is the length of the sub-region. This may be zero if the access s= ize is + not relevant, which may allow for optimizations; ``KVM_IOREGION_POSTED_W= RITES`` + must be set in *flags* in this case +* *fd_index* is the index in the ancillary data of the FD to use for ioreg= ionfd + messages; it may be shared +* *type* is ``VFIO_USER_IO_FD_TYPE_IOREGIONFD`` +* *flags* is any of: + + * ``KVM_IOREGION_PIO`` + * ``KVM_IOREGION_POSTED_WRITES`` + +* *user_data* is an opaque value passed back to the server via a message o= n the + file descriptor + +For further information on the ioregionfd-specific fields, see: +https://lore.kernel.org/kvm/cover.1613828726.git.eafanasova@gmail.com/ + +(FIXME: update with final API docs.) + +``VFIO_USER_DEVICE_GET_IRQ_INFO`` +--------------------------------- + +This command message is sent by the client to the server to query for +information about device interrupt types. The VFIO IRQ info structure is +defined in ```` (``struct vfio_irq_info``). + +Request +^^^^^^^ + ++-------+--------+---------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------+--------+---------------------------+ +| flags | 4 | 4 | ++-------+--------+---------------------------+ +| | +-----+--------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_IRQ_INFO_EVENTFD | | +| | +-----+--------------------------+ | +| | | 1 | VFIO_IRQ_INFO_MASKABLE | | +| | +-----+--------------------------+ | +| | | 2 | VFIO_IRQ_INFO_AUTOMASKED | | +| | +-----+--------------------------+ | +| | | 3 | VFIO_IRQ_INFO_NORESIZE | | +| | +-----+--------------------------+ | ++-------+--------+---------------------------+ +| index | 8 | 4 | ++-------+--------+---------------------------+ +| count | 12 | 4 | ++-------+--------+---------------------------+ + +* *argsz* is the maximum size of the reply payload (16 bytes today) +* index is the index of IRQ type being queried (e.g. ``VFIO_PCI_MSIX_IRQ_I= NDEX``) +* all other fields must be zero + +Reply +^^^^^ + ++-------+--------+---------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------+--------+---------------------------+ +| flags | 4 | 4 | ++-------+--------+---------------------------+ +| | +-----+--------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_IRQ_INFO_EVENTFD | | +| | +-----+--------------------------+ | +| | | 1 | VFIO_IRQ_INFO_MASKABLE | | +| | +-----+--------------------------+ | +| | | 2 | VFIO_IRQ_INFO_AUTOMASKED | | +| | +-----+--------------------------+ | +| | | 3 | VFIO_IRQ_INFO_NORESIZE | | +| | +-----+--------------------------+ | ++-------+--------+---------------------------+ +| index | 8 | 4 | ++-------+--------+---------------------------+ +| count | 12 | 4 | ++-------+--------+---------------------------+ + +* *argsz* is the size required for the full reply payload (16 bytes today) +* *flags* defines IRQ attributes: + + * ``VFIO_IRQ_INFO_EVENTFD`` indicates the IRQ type can support server ev= entfd + signalling. + * ``VFIO_IRQ_INFO_MASKABLE`` indicates that the IRQ type supports the ``= MASK`` + and ``UNMASK`` actions in a ``VFIO_USER_DEVICE_SET_IRQS`` message. + * ``VFIO_IRQ_INFO_AUTOMASKED`` indicates the IRQ type masks itself after= being + triggered, and the client must send an ``UNMASK`` action to receive new + interrupts. + * ``VFIO_IRQ_INFO_NORESIZE`` indicates ``VFIO_USER_SET_IRQS`` operations= setup + interrupts as a set, and new sub-indexes cannot be enabled without dis= abling + the entire type. +* index is the index of IRQ type being queried +* count describes the number of interrupts of the queried type. + +``VFIO_USER_DEVICE_SET_IRQS`` +----------------------------- + +This command message is sent by the client to the server to set actions for +device interrupt types. The VFIO IRQ set structure is defined in +```` (``struct vfio_irq_set``). + +Request +^^^^^^^ + ++-------+--------+------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------+--------+------------------------------+ +| flags | 4 | 4 | ++-------+--------+------------------------------+ +| | +-----+-----------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_IRQ_SET_DATA_NONE | | +| | +-----+-----------------------------+ | +| | | 1 | VFIO_IRQ_SET_DATA_BOOL | | +| | +-----+-----------------------------+ | +| | | 2 | VFIO_IRQ_SET_DATA_EVENTFD | | +| | +-----+-----------------------------+ | +| | | 3 | VFIO_IRQ_SET_ACTION_MASK | | +| | +-----+-----------------------------+ | +| | | 4 | VFIO_IRQ_SET_ACTION_UNMASK | | +| | +-----+-----------------------------+ | +| | | 5 | VFIO_IRQ_SET_ACTION_TRIGGER | | +| | +-----+-----------------------------+ | ++-------+--------+------------------------------+ +| index | 8 | 4 | ++-------+--------+------------------------------+ +| start | 12 | 4 | ++-------+--------+------------------------------+ +| count | 16 | 4 | ++-------+--------+------------------------------+ +| data | 20 | variable | ++-------+--------+------------------------------+ + +* *argsz* is the size of the VFIO IRQ set request payload, including any *= data* + field. Note there is no reply payload, so this field differs from other + message types. +* *flags* defines the action performed on the interrupt range. The ``DATA`` + flags describe the data field sent in the message; the ``ACTION`` flags + describe the action to be performed. The flags are mutually exclusive for + both sets. + + * ``VFIO_IRQ_SET_DATA_NONE`` indicates there is no data field in the com= mand. + The action is performed unconditionally. + * ``VFIO_IRQ_SET_DATA_BOOL`` indicates the data field is an array of boo= lean + bytes. The action is performed if the corresponding boolean is true. + * ``VFIO_IRQ_SET_DATA_EVENTFD`` indicates an array of event file descrip= tors + was sent in the message meta-data. These descriptors will be signalled= when + the action defined by the action flags occurs. In ``AF_UNIX`` sockets,= the + descriptors are sent as ``SCM_RIGHTS`` type ancillary data. + If no file descriptors are provided, this de-assigns the specified + previously configured interrupts. + * ``VFIO_IRQ_SET_ACTION_MASK`` indicates a masking event. It can be used= with + ``VFIO_IRQ_SET_DATA_BOOL`` or ``VFIO_IRQ_SET_DATA_NONE`` to mask an in= terrupt, + or with ``VFIO_IRQ_SET_DATA_EVENTFD`` to generate an event when the gu= est masks + the interrupt. + * ``VFIO_IRQ_SET_ACTION_UNMASK`` indicates an unmasking event. It can be= used + with ``VFIO_IRQ_SET_DATA_BOOL`` or ``VFIO_IRQ_SET_DATA_NONE`` to unmas= k an + interrupt, or with ``VFIO_IRQ_SET_DATA_EVENTFD`` to generate an event = when the + guest unmasks the interrupt. + * ``VFIO_IRQ_SET_ACTION_TRIGGER`` indicates a triggering event. It can b= e used + with ``VFIO_IRQ_SET_DATA_BOOL`` or ``VFIO_IRQ_SET_DATA_NONE`` to trigg= er an + interrupt, or with ``VFIO_IRQ_SET_DATA_EVENTFD`` to generate an event = when the + server triggers the interrupt. + +* *index* is the index of IRQ type being setup. +* *start* is the start of the sub-index being set. +* *count* describes the number of sub-indexes being set. As a special case= , a + count (and start) of 0, with data flags of ``VFIO_IRQ_SET_DATA_NONE`` di= sables + all interrupts of the index. +* *data* is an optional field included when the + ``VFIO_IRQ_SET_DATA_BOOL`` flag is present. It contains an array of bool= eans + that specify whether the action is to be performed on the corresponding + index. It's used when the action is only performed on a subset of the ra= nge + specified. + +Not all interrupt types support every combination of data and action flags. +The client must know the capabilities of the device and IRQ index before it +sends a ``VFIO_USER_DEVICE_SET_IRQ`` message. + +In typical operation, a specific IRQ may operate as follows: + +1. The client sends a ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_EVENTFD|VFIO_IRQ_SET_ACTION_TRIGGER)`` alo= ng + with an eventfd. This associates the IRQ with a particular eventfd on t= he + server side. + +#. The client may send a ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_EVENTFD|VFIO_IRQ_SET_ACTION_MASK/UNMASK)``= along + with another eventfd. This associates the given eventfd with the + mask/unmask state on the server side. + +#. The server may trigger the IRQ by writing 1 to the eventfd. + +#. The server may mask/unmask an IRQ which will write 1 to the correspondi= ng + mask/unmask eventfd, if there is one. + +5. A client may trigger a device IRQ itself, by sending a + ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_NONE/BOOL|VFIO_IRQ_SET_ACTION_TRIGGER)``. + +6. A client may mask or unmask the IRQ, by sending a + ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_NONE/BOOL|VFIO_IRQ_SET_ACTION_MASK/UNMASK)= ``. + +Reply +^^^^^ + +There is no payload in the reply. + +.. _Read and Write Operations: + +Note that all of these operations must be supported by the client and/or s= erver, +even if the corresponding memory or device region has been shared as mappa= ble. + +The ``count`` field must not exceed the value of ``max_data_xfer_size`` of= the +peer, for both reads and writes. + +``VFIO_USER_REGION_READ`` +------------------------- + +If a device region is not mappable, it's not directly accessible by the cl= ient +via ``mmap()`` of the underlying file descriptor. In this case, a client c= an +read from a device region with this message. + +Request +^^^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ + +* *offset* into the region being accessed. +* *region* is the index of the region being accessed. +* *count* is the size of the data to be transferred. + +Reply +^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ +| data | 16 | variable | ++--------+--------+----------+ + +* *offset* into the region accessed. +* *region* is the index of the region accessed. +* *count* is the size of the data transferred. +* *data* is the data that was read from the device region. + +``VFIO_USER_REGION_WRITE`` +-------------------------- + +If a device region is not mappable, it's not directly accessible by the cl= ient +via mmap() of the underlying fd. In this case, a client can write to a dev= ice +region with this message. + +Request +^^^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ +| data | 16 | variable | ++--------+--------+----------+ + +* *offset* into the region being accessed. +* *region* is the index of the region being accessed. +* *count* is the size of the data to be transferred. +* *data* is the data to write + +Reply +^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ + +* *offset* into the region accessed. +* *region* is the index of the region accessed. +* *count* is the size of the data transferred. + +``VFIO_USER_DMA_READ`` +----------------------- + +If the client has not shared mappable memory, the server can use this mess= age to +read from guest memory. + +Request +^^^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 8 | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. This address = must have + been previously exported to the server with a ``VFIO_USER_DMA_MAP`` mess= age. +* *count* is the size of the data to be transferred. + +Reply +^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 8 | ++---------+--------+----------+ +| data | 16 | variable | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. +* *count* is the size of the data transferred. +* *data* is the data read. + +``VFIO_USER_DMA_WRITE`` +----------------------- + +If the client has not shared mappable memory, the server can use this mess= age to +write to guest memory. + +Request +^^^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 8 | ++---------+--------+----------+ +| data | 16 | variable | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. This address = must have + been previously exported to the server with a ``VFIO_USER_DMA_MAP`` mess= age. +* *count* is the size of the data to be transferred. +* *data* is the data to write + +Reply +^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 4 | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. +* *count* is the size of the data transferred. + +``VFIO_USER_DEVICE_RESET`` +-------------------------- + +This command message is sent from the client to the server to reset the de= vice. +Neither the request or reply have a payload. + +``VFIO_USER_REGION_WRITE_MULTI`` +-------------------------------- + +This message can be used to coalesce multiple device write operations +into a single messgage. It is only used as an optimization when the +outgoing message queue is relatively full. + +Request +^^^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| wr_cnt | 0 | 8 | ++---------+--------+----------+ +| wrs | 8 | variable | ++---------+--------+----------+ + +* *wr_cnt* is the number of device writes coalesced in the message +* *wrs* is an array of device writes defined below + +Single Device Write Format +"""""""""""""""""""""""""" + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ +| data | 16 | 8 | ++--------+--------+----------+ + +* *offset* into the region being accessed. +* *region* is the index of the region being accessed. +* *count* is the size of the data to be transferred. This format can + only describe writes of 8 bytes or less. +* *data* is the data to write. + +Reply +^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| wr_cnt | 0 | 8 | ++---------+--------+----------+ + +* *wr_cnt* is the number of device writes completed. + + +Appendices +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Unused VFIO ``ioctl()`` commands +-------------------------------- + +The following VFIO commands do not have an equivalent vfio-user command: + +* ``VFIO_GET_API_VERSION`` +* ``VFIO_CHECK_EXTENSION`` +* ``VFIO_SET_IOMMU`` +* ``VFIO_GROUP_GET_STATUS`` +* ``VFIO_GROUP_SET_CONTAINER`` +* ``VFIO_GROUP_UNSET_CONTAINER`` +* ``VFIO_GROUP_GET_DEVICE_FD`` +* ``VFIO_IOMMU_GET_INFO`` + +However, once support for live migration for VFIO devices is finalized some +of the above commands may have to be handled by the client in their +corresponding vfio-user form. This will be addressed in a future protocol +version. + +VFIO groups and containers +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The current VFIO implementation includes group and container idioms that +describe how a device relates to the host IOMMU. In the vfio-user +implementation, the IOMMU is implemented in SW by the client, and is not +visible to the server. The simplest idea would be that the client put each +device into its own group and container. + +Backend Program Conventions +--------------------------- + +vfio-user backend program conventions are based on the vhost-user ones. + +* The backend program must not daemonize itself. +* No assumptions must be made as to what access the backend program has on= the + system. +* File descriptors 0, 1 and 2 must exist, must have regular + stdin/stdout/stderr semantics, and can be redirected. +* The backend program must honor the SIGTERM signal. +* The backend program must accept the following commands line options: + + * ``--socket-path=3DPATH``: path to UNIX domain socket, + * ``--fd=3DFDNUM``: file descriptor for UNIX domain socket, incompatible= with + ``--socket-path`` +* The backend program must be accompanied with a JSON file stored under + ``/usr/share/vfio-user``. + +TODO add schema similar to docs/interop/vhost-user.json. --=20 2.43.0