From nobody Sat Nov 15 14:11:13 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750879876; cv=pass; d=zohomail.com; s=zohoarc; b=javucG9tO9xTotiYxFbWOn5UYAiz1DBVKlUasf5PWFHhnKnBTJkr6k3sUX+3yNQ1fEhOB/ZwIB2hOLdYerRHC7EI8m+I3a5W6iJjAvEYIB51sl0CXEO73OVOQfUJj8b2gaYsz7+hb1xQusq4Py96WzIoSM7bqfBV8TEmeDFtoHc= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750879876; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=+JW0z2M0XDMLfwMf2cvg4TTzvrHungvz7UY+9GcaC1k=; b=NNKX9MtKcaErwejYjiUMPBccX9hpFUUmSojy8vA3jOXESlMBio5oMzObRCQcmmv9//cdeqf0jJcPzHAytLMHSM4UHAYHOToX+162tKBuiAR+SUE55ATe1GDUO5ete0QduA21sbnYHqjQRNtp3EGDYgdAJavQFPExG96p8zye2eM= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750879876422947.514104143229; Wed, 25 Jun 2025 12:31:16 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpF-0007ys-OM; Wed, 25 Jun 2025 15:30:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpE-0007yP-9E for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:28 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpB-0003GI-5t for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:27 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PDQbP3009293; Wed, 25 Jun 2025 12:30:22 -0700 Received: from nam04-mw2-obe.outbound.protection.outlook.com (mail-mw2nam04on2099.outbound.protection.outlook.com [40.107.101.99]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47dt9carm1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:22 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:20 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=+JW0z2M0XDMLfwMf2cvg4TTzvrHungvz7UY+9GcaC 1k=; b=LIvigfV4aGnmFl6sEtnnNIhEh+GoR0OiiyYC3hyxE5DUilXLYCy1Dp2dF 7yooQzrHjIhV6vKdqVciFrWw+BN5BEB5XSwf+XMHlYN1rbUR43m69XBqvJGI1yZ5 pULVW1PQl2exGjWZU1/md9piggCWfBF+ZmBpUuuEB6yd8fa0p2aZjrPV4OmNPcmZ TpgJXbKBYa6VfZTKYX1S2Sp47bbmTEtOKMW+nO6yomI+9bvp56LONmJ2swkMOEMq 7XfLMZn8ONlkOS1BtRnY7Kl8OidiRs4k7l7NBDgDREo7SANbesAMQBbmSv/pvYVj D15SsQvxxUWApXuVKIu1BxnCzRKrQ== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=fWobJ/tcpmU7ItsHvpf/lSdNY4mM701aS0IwnJoiH62WoeCxqivi1FnGx8hFTzFTgjgDZV2G50XLh24y27JkrdZz9uc/AHIjnPWvxOEX1odhue6lZGnPY2fH5+RdZL9rV9sPtqU1tgU4El6ECwyTrVbwYgMPDvn9kOyWP0wijv7hm6pkww1ZSAiC/unbq1xiygGMXAmp3sz3SkuPrhcjMgG0vrExxMPqc0rJyIZFySRVjoU3EWFPCUHC+4o3NOotiVrg8UsAig+fahMI8HNJyQEORwA9SYTfZTxIW+IRayFRpY2D2IZ+iGfJi4NkyKtbrpaR+8a0Ainfl5ovXoHMaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+JW0z2M0XDMLfwMf2cvg4TTzvrHungvz7UY+9GcaC1k=; b=cW7mc8/+qKd4gY3Zx00fPPay57Wmh88SjT7WAYg6mbqUlx0m2oWW0eSiXhwYA6WJskcA55s3A6lEcBc1croIaEENJNabdNfNI/iDc9KJZY0XyubbdD1DUrOUCCXmz9xVHufM1gykMe1g82l7I83Zrfy6EjhJLimKJdACX5O0MaBs/JbOfZCsq7+4P6bGMieQYuL6wZL3U12BH/d5mICe2d850AZMk6JoOOZKBdZWwPare9AYS68QR3JlqhR/eLUAVme8zOyQwZph5MDsT1HTbBVUm8993kBo8yAg9o6xWGLvDdXR4oZ4HTkhdB+XsxGwxZcPYVuS+1NDFYvx76yejQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+JW0z2M0XDMLfwMf2cvg4TTzvrHungvz7UY+9GcaC1k=; b=wab729luCQqkB9H3zXXdlbyVf92YWEUPJBCg7MX+CmtDyc2EPuI29jx5yL7D2zaLRgHJQ/Wk/tcVAQSx8TQSV2lPbQRjTl6cU9UvTFH/8ZiM30doEGxQSJ832cWLfK8gET4cZgfMw5aHXKgwj2Vm/lnG+idi9FrUtbrCs1I80kZaQbNGGyf89f8moauPPce3cY9wbBg8NM/aF57fPrfm2AUGL5SvEQkqCvalHfHlC+Fd0nf70UB332JVp7JgAaIFRmkUx9HfBDy9qUIEzQFEJ/iMG5GVimNTzlsN6FEf0coc4ge36fFWQcQaaB2UEzNCjnA5O7icZ8DEht63/Z+zww== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 01/19] vfio-user: add vfio-user class and container Date: Wed, 25 Jun 2025 20:29:53 +0100 Message-ID: <20250625193012.2316242-2-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: 5086dc76-b947-4dcd-a08d-08ddb41eb89c x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?B?ZFBjVzE0N1hrbUFhdlpXMkhLWDgrSHBNQXdxR0xRdEFLTWIvaFJoSWVEUUxW?= =?utf-8?B?NDQvSW9EN2trUGJiTVdIRENYMGFlVnJSZnJ4WnhYUVZWODJBcFF5LzFxd2tQ?= =?utf-8?B?OFdtRkE1SEdLUGVMWC9oZnovaExEUWQwOVg5Y2lwL0liVFo5YmMxZnQ2eE9L?= =?utf-8?B?cXgyYkRJRFFDZ1gvWk1DN1ZxNk9WaUcrSzFsNVI0TURDc1RWRlVMM2VzeEcy?= =?utf-8?B?anlGbElQVFJicUFTQUlhNzgxM1RLVWc2a3pES05WbFcvY0RkZkVtUk1yR0pK?= =?utf-8?B?N044QVlBek0yYXIxa0swT2gzajAwRGdCVkNkaUdVSWpXT0NvRzczZGRyWW5w?= =?utf-8?B?OTI3a0Z0UFdDTWZQOVd0c1ZwTzJZVVordkNLVHVGb0tkaVpzNEJ5a3Rndm1V?= =?utf-8?B?YUZMa29Ram41blNHOWpFMTNyRmpQODllRjg4MEZKVk45SlYrNlk4SDdsYTFV?= =?utf-8?B?OHkwZVVQZHJHdFd4Qks4SjBrYTNYS3VaWCszaGdtTnJFZTFQTHUxOVRWQzNJ?= =?utf-8?B?bzNwa1pUSkJKOENWQXdJRFlGRnpMSVZCRjc0N0RaYzJSQ3FFdi84QzBSQ0c5?= =?utf-8?B?a0IxUDY1MzdNUmViV3cwa3BCbTBFeGVQQ1Y5N3c0RzZxYVhqaXBmZFhWRTRr?= =?utf-8?B?aXZ0d2ZDUWltNG9tOWtZR3A5SFBSaGZxSjVBL050WjZHZFpTM0tFTWgxYlBV?= =?utf-8?B?WkxTM2RidTgyZlZBZm4wMFArbkIxSFI0bE1uN29YbDdvN3I4ZTdJNmxBeVp0?= =?utf-8?B?enZDU1FMb0ZXTlFjSGFmd3BDbnV1cnRLbXQzWUY3R1RtY3p2V3hMSnNRcnJ6?= =?utf-8?B?cWFBanhQaDdDbksvMmZNZDIxWFl1VWVIcTJCNkZjbGlaeWhsbHAwWWV2clNv?= =?utf-8?B?VGtKcXFnMTh0d2FhUkFqR0hMbnpkRmFXYlc3RkRhQVdRNzE0VW1SZ3h4NVQ4?= =?utf-8?B?c3BCMUxpalFzNGxWWWx2bktHaDd3WTdEZDJqeHNYQ1FnMjlqcFNybTZQUndp?= =?utf-8?B?SVkwTlJFNHB1aEpVOENyc1d3VDRIcjNWZitxTUdaS0FETTV4b2VHdklTRm4x?= =?utf-8?B?cEkxRFRIcllkVFZYcFA2bU9laklVRU42dktGZkFaaUxVS2dsK3p2a09Sd3Fu?= =?utf-8?B?UWV1MlRuWXNNZkIzTHZVREhwUXR0WC9MWXVpZjJNbnFLZ3F1VkEvVHBQRC9D?= =?utf-8?B?VERaYTFDSEVDeWNjNXNEWUpEVWpYc0FKcERPUXBDZFJXREdzTDhPaGZhdlZT?= =?utf-8?B?VDJxWG8xcXJ3TVoxN3EwekxDRDFvQzVHaEJJV3lhOVdyQWhDWVozYnhWKzhk?= =?utf-8?B?a0h6ekU0bmFiK1IzNHhyODE1Tnl5YkRTQmxZUnBkei9ldDgyTnUzUXJIMEVJ?= =?utf-8?B?M29mQjB4ZkxhQ0ExNm5kUlJhQ3dLUFU5VnNEV05KOFhkV2lqdDJ2NHNXdVhJ?= =?utf-8?B?cHViWWQ0RnpGYkxHenVRRmhNaStlZ3pobHhTL2pkczhSYTZkM3VzYTFuY3RO?= =?utf-8?B?eUt4S0VnSUtKNCttTEtBa2VoaHNUQjlmZlhrd2Rsa3FMZVVNM1dYOE5oYnZP?= =?utf-8?B?VCt4QUxBMSs0akNsKzdxeklURndYbnpkUzJoTmxJSGdRQ0RQU3Uza2FwZFVF?= =?utf-8?B?dkt5TWRUaUlVZUdRajFvSFBPVy9uQ25YMDhQci9qN2h0d2JZcEFRRGlmUUdh?= =?utf-8?B?RUE5ajc4dDZ6ZUtZOUhudTFpeHhYN1kwdkp1K0szLzRPaWZia0ZId1FGRFpE?= =?utf-8?B?eDVWcllmbmsxcFUrUkZ6eGxOek00aHNvKzdUNDlGMEdVWFlYRmxnNDVDUnhh?= =?utf-8?B?K3RGZDBUVFN0V25QRFAxUXNOSllidmg3cUs1Y0VieFNybDRjN1REZ29URVo0?= =?utf-8?B?ZEpzUkFiSGJWNjR5SGI4N1NDVUtoRU9ueXFiWVljWG9EdFRGM25VWXdFcGQ2?= =?utf-8?Q?nZjDPYFMAv8=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cWtiSXhZSFFpL0toMzZrb2MvZFVpOE05aGtHZzByZUtzSzNENDVQbUVrMHJn?= =?utf-8?B?MzFVYVF3K2tMSk0xSkhhZ09xVXQ4a0Z0WnVwdVVhRVZjKy84bWtYcWVrZjNS?= =?utf-8?B?cjNPMldnUk1wd1pMWmc1d0FFMG01Zm9Nekxtc2JyNit3d0kveFlGdkZuRmZE?= =?utf-8?B?dUs3dk9JS2cvU2Z4eHNidlVFVndqWmVXb09GWi9FbjhCczV1VmJiSEEwQ0xq?= =?utf-8?B?T3k3WVAzcmh1WXRaLy9OaWRYSnZxMEEzQTNaM1ZNTGFRZkQ1UkhHb0dKYUEr?= =?utf-8?B?anJId1J6b3RZR2UyZlc5Nk0zQm1kQXg1dlZ5YmREVTBuQisrU1JvSjBkcGFK?= =?utf-8?B?M09NV2pvY2lzbVhSUllLdGt5Z09IL3ZQMVpPdTJBVWdZQWJ6RHZVcmpQWUZ3?= =?utf-8?B?R0hhQUc3eUhWZU1rclhKeVJzQXgyZEIyVjFWT0lSYU5KNjRxeU1sQmhXc3J3?= =?utf-8?B?emV4azk1b2J4TVFiTXZjWVZZOFZxNU8vQzFLOXc1M1BnalpmOCtHSVFmMVhO?= =?utf-8?B?WmFvcUZSZTY1RUpRN1RyRTFvVWN3dFM5NGhwVGpjVDBraGN5T3VhY3R6SnV5?= =?utf-8?B?VWVJVDBLeFp0Y29hajRNR0VYdzFUMU1wMmJkUjNEb1V2QiswTkdZN0ZNSmpO?= =?utf-8?B?S3hFSWRsMGZENTRtaEdUTExLVmpYb1lLSXFqclBXVXR5TjBacVNCYzRGT0VW?= =?utf-8?B?djdtMXlRK2tmSm9QR3EvaXZDUTBCZEx2QlA2SWtOYlgvWHFlQlFlbGhVQnB5?= =?utf-8?B?Ry90ZENSV3hFWTkyTmRmSEFndzNnQ00rZEZZcnpyUDFrd1E3YjdKKzk0eGls?= =?utf-8?B?UUZET2hYTDRndTUyWTY2SjZXSlR5eVFzU1d4Y0JNOHRtYStzSmx2dVJOdmVr?= =?utf-8?B?ZzdTamZ0aitWOTB3UVFUbGdTYmM3ZFpZQ1RQR0tZOEpwQUdGQSt3NklKSEt6?= =?utf-8?B?SE5Zd0x1QXZJZWgyWFBlVnJWdlZjeEsycFNMbmdlNXczcGhpWE5lbkp2Z3R3?= =?utf-8?B?d3dPcTVXMFlZYXhTZlU5OVRkSjduKzYwNWJTS2p1UTJocWtQWWhaTFJZMEZv?= =?utf-8?B?dzdnYmhWbVlMWVNaVFozM0ZTUVVKeHpJc1RhL1VpOFdKNU9QcDREdWxOOUVV?= =?utf-8?B?T29xWk81RFJRMWUrT0UzenRKeG1JMmRBT3d6UDdlVUJLbUpaVUcrdEs1UHZ1?= =?utf-8?B?NDJoeHFuaXpEekRNMDNFd3VZR216S1VpS0hvOUMzVnJ6TnlJMGt2Sy9ORk9S?= =?utf-8?B?YWg3V24zZEdKVEl5UGo2LzFxU3N6VHh3emlRYktyUWZycWpucm9aOGNmVkJ5?= =?utf-8?B?UDhlY0lTV1JHRXFrQlRlVThsdkFwcjZ1dDc5dEdJT2c2em1CQkNWYWFqQzVM?= =?utf-8?B?cnNhV1ZHU2ExT2t6SGRLSFNreTNYOFo0b1FpOERHOW50VStDSzBNM3RTOUdx?= =?utf-8?B?RnNyMHJDZWVtZGVxMTZETkFONElrUkpCaWJ3U05iWFhFNkl2Y2crS0ZucUdW?= =?utf-8?B?cXZ5cUJuQ0hZQjlvTEQxUXFMN2lKM2pBMzJpdmJrRFkvRFcwY1JaTkFGTDh4?= =?utf-8?B?QUJkRGtxN01SbGlBVXpaK2J4K3ZSTnBzdDFoQU5raGpabmVwNzJDWjBmVzhu?= =?utf-8?B?d3Uxak5EYk82a3B4bFZVMGlsY2JnWmZObnRHK1dhemhRNVh0ZGxkTUxoSTAy?= =?utf-8?B?T3g0anJkTTFVd2ljS2VieVJMUEVZa2p0NnpjUk9yRWQ5TzFWU1RseFExcy9H?= =?utf-8?B?MVozb2hUU0czbDg5Q21yNnJXbUN5cC9GN1lpeXcvekJDN2pTbmRJbUgxa2Jz?= =?utf-8?B?dW0vSTZLVXpScWllKzY5UGF2d0QvbHFrTWxFeStGeFNNcHlGY0VQR0pwUDdl?= =?utf-8?B?ZnNhU3RjNElnNi91bEt5SGxEN1JZZHQrY3haVzliNzk0dUNZTzNuNFZQdGZj?= =?utf-8?B?MVpqcnpzN2Y2a0EwNUxJaXlqSytFZUVmRDJIYTVQYU1PZHlyS3FrSGFTZlln?= =?utf-8?B?dHR6dVd3OWR5ZjFkNDc4R0I4RE94NldGUW40WS84dERqbUxNOXRRdXhWOTdN?= =?utf-8?B?aXVBTmMrVkZCY2YvZW9jUEt6aDdUREp0MVROcFNZZDN4cXlvRE1JU3RMV2Jn?= =?utf-8?Q?3O0fM9p7vxTIHbakgorHP2PzF?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5086dc76-b947-4dcd-a08d-08ddb41eb89c X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:20.0059 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zyVvgKh7btCBwYlDzphvLDMMwSjZRsMxzO9lEsPsb6p0ENQ5csi5lHkjV9gIWGpBZDk+QvsyLM4Ais28jGJTFQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX+9OdsSPj8fYb TUuZNh/peuFFb13JfgGnAUA//7VeuujWt7UJiy7noGKy9aL0OLb2ntpckIqE0RLREA8bcvalagV iETXvW0KPTgVwlQarMtc6OYrt2HFTSX2B9NKCg4SCUVJzil5EtAdyeITJe0SctoBCZFetZZY7HR ZoMMqnBF7fRCH0A/pJvi7C+f6YEjZ7UGk3YK6rPRRedXi5gowZUDHFQ00qxQq9O9TrwpMPRf1Mf Rpl0mggz0lfC6DFY1b7aPNc2DCFSxmyAQJ7kMXoOKv9qOJ8ED1gEVORKTfkjl4tkbPQj6Cm4w1P JH2eBgSeiRX2Ui8clF+fSDOFwaQ9PTlNIBvIwTDqNHnS//WtDPhKXiz6ViCoI7TMjdKjg1uEulH AW3i5VbsYuUS3LNoHAOBvKj2OFUQHRossDZALll3fWMlonx2V4A3n7Q7I0QxwkaoEYdd/a4H X-Proofpoint-ORIG-GUID: d_0A21v9OHk8nKCAGaV1aiY8OD1gsFzC X-Authority-Analysis: v=2.4 cv=MeVsu4/f c=1 sm=1 tr=0 ts=685c4e4e cx=c_pps a=BDhOPDsQvggip5BazaeBNQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=20KFwNOVAAAA:8 a=z4glEzOvAAAA:8 a=u7rQ6wk7R_5OE3_9H9YA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 a=92dS5hN0c3Q7EetK7xW5:22 X-Proofpoint-GUID: d_0A21v9OHk8nKCAGaV1aiY8OD1gsFzC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750879880407116600 Introduce basic plumbing for vfio-user with CONFIG_VFIO_USER. We introduce VFIOUserContainer in hw/vfio-user/container.c, which is a container type for the "IOMMU" type "vfio-iommu-user", and share some common container code from hw/vfio/container.c. Add hw/vfio-user/pci.c for instantiating VFIOUserPCIDevice objects, sharing some common code from hw/vfio/pci.c. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- MAINTAINERS | 8 + hw/vfio-user/container.h | 21 +++ include/hw/vfio/vfio-container-base.h | 1 + hw/vfio-user/container.c | 208 ++++++++++++++++++++++++++ hw/vfio-user/pci.c | 185 +++++++++++++++++++++++ hw/Kconfig | 1 + hw/meson.build | 1 + hw/vfio-user/Kconfig | 7 + hw/vfio-user/meson.build | 9 ++ 9 files changed, 441 insertions(+) create mode 100644 hw/vfio-user/container.h create mode 100644 hw/vfio-user/container.c create mode 100644 hw/vfio-user/pci.c create mode 100644 hw/vfio-user/Kconfig create mode 100644 hw/vfio-user/meson.build diff --git a/MAINTAINERS b/MAINTAINERS index 27f4fe3f25..2369391004 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -4253,6 +4253,14 @@ F: hw/remote/iommu.c F: include/hw/remote/iommu.h F: tests/functional/test_multiprocess.py =20 +VFIO-USER: +M: John Levon +M: Thanos Makatos +S: Supported +F: hw/vfio-user/* +F: include/hw/vfio-user/* +F: subprojects/libvfio-user + EBPF: M: Jason Wang R: Andrew Melnychenko diff --git a/hw/vfio-user/container.h b/hw/vfio-user/container.h new file mode 100644 index 0000000000..e4a46d2c1b --- /dev/null +++ b/hw/vfio-user/container.h @@ -0,0 +1,21 @@ +/* + * vfio-user specific definitions. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef HW_VFIO_USER_CONTAINER_H +#define HW_VFIO_USER_CONTAINER_H + +#include "qemu/osdep.h" + +#include "hw/vfio/vfio-container-base.h" + +/* MMU container sub-class for vfio-user. */ +typedef struct VFIOUserContainer { + VFIOContainerBase bcontainer; +} VFIOUserContainer; + +OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserContainer, VFIO_IOMMU_USER); + +#endif /* HW_VFIO_USER_CONTAINER_H */ diff --git a/include/hw/vfio/vfio-container-base.h b/include/hw/vfio/vfio-c= ontainer-base.h index f0232654ee..3cd86ec59e 100644 --- a/include/hw/vfio/vfio-container-base.h +++ b/include/hw/vfio/vfio-container-base.h @@ -109,6 +109,7 @@ vfio_container_get_page_size_mask(const VFIOContainerBa= se *bcontainer) #define TYPE_VFIO_IOMMU_LEGACY TYPE_VFIO_IOMMU "-legacy" #define TYPE_VFIO_IOMMU_SPAPR TYPE_VFIO_IOMMU "-spapr" #define TYPE_VFIO_IOMMU_IOMMUFD TYPE_VFIO_IOMMU "-iommufd" +#define TYPE_VFIO_IOMMU_USER TYPE_VFIO_IOMMU "-user" =20 OBJECT_DECLARE_TYPE(VFIOContainerBase, VFIOIOMMUClass, VFIO_IOMMU) =20 diff --git a/hw/vfio-user/container.c b/hw/vfio-user/container.c new file mode 100644 index 0000000000..2367332177 --- /dev/null +++ b/hw/vfio-user/container.c @@ -0,0 +1,208 @@ +/* + * Container for vfio-user IOMMU type: rather than communicating with the = kernel + * vfio driver, we communicate over a socket to a server using the vfio-us= er + * protocol. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include +#include +#include "qemu/osdep.h" + +#include "hw/vfio-user/container.h" +#include "hw/vfio/vfio-cpr.h" +#include "hw/vfio/vfio-device.h" +#include "hw/vfio/vfio-listener.h" +#include "qapi/error.h" + +static int vfio_user_dma_unmap(const VFIOContainerBase *bcontainer, + hwaddr iova, ram_addr_t size, + IOMMUTLBEntry *iotlb, bool unmap_all) +{ + return -ENOTSUP; +} + +static int vfio_user_dma_map(const VFIOContainerBase *bcontainer, hwaddr i= ova, + ram_addr_t size, void *vaddr, bool readonly, + MemoryRegion *mrp) +{ + return -ENOTSUP; +} + +static int +vfio_user_set_dirty_page_tracking(const VFIOContainerBase *bcontainer, + bool start, Error **errp) +{ + error_setg_errno(errp, ENOTSUP, "Not supported"); + return -ENOTSUP; +} + +static int vfio_user_query_dirty_bitmap(const VFIOContainerBase *bcontaine= r, + VFIOBitmap *vbmap, hwaddr iova, + hwaddr size, Error **errp) +{ + error_setg_errno(errp, ENOTSUP, "Not supported"); + return -ENOTSUP; +} + +static bool vfio_user_setup(VFIOContainerBase *bcontainer, Error **errp) +{ + error_setg_errno(errp, ENOTSUP, "Not supported"); + return -ENOTSUP; +} + +static VFIOUserContainer *vfio_user_create_container(Error **errp) +{ + VFIOUserContainer *container; + + container =3D VFIO_IOMMU_USER(object_new(TYPE_VFIO_IOMMU_USER)); + return container; +} + +/* + * Try to mirror vfio_container_connect() as much as possible. + */ +static VFIOUserContainer * +vfio_user_container_connect(AddressSpace *as, Error **errp) +{ + VFIOContainerBase *bcontainer; + VFIOUserContainer *container; + VFIOAddressSpace *space; + VFIOIOMMUClass *vioc; + + space =3D vfio_address_space_get(as); + + container =3D vfio_user_create_container(errp); + if (!container) { + goto put_space_exit; + } + + bcontainer =3D &container->bcontainer; + + if (!vfio_cpr_register_container(bcontainer, errp)) { + goto free_container_exit; + } + + vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); + assert(vioc->setup); + + if (!vioc->setup(bcontainer, errp)) { + goto unregister_container_exit; + } + + vfio_address_space_insert(space, bcontainer); + + if (!vfio_listener_register(bcontainer, errp)) { + goto listener_release_exit; + } + + bcontainer->initialized =3D true; + + return container; + +listener_release_exit: + vfio_listener_unregister(bcontainer); + if (vioc->release) { + vioc->release(bcontainer); + } + +unregister_container_exit: + vfio_cpr_unregister_container(bcontainer); + +free_container_exit: + object_unref(container); + +put_space_exit: + vfio_address_space_put(space); + + return NULL; +} + +static void vfio_user_container_disconnect(VFIOUserContainer *container) +{ + VFIOContainerBase *bcontainer =3D &container->bcontainer; + VFIOIOMMUClass *vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); + + vfio_listener_unregister(bcontainer); + if (vioc->release) { + vioc->release(bcontainer); + } + + VFIOAddressSpace *space =3D bcontainer->space; + + vfio_cpr_unregister_container(bcontainer); + object_unref(container); + + vfio_address_space_put(space); +} + +static bool vfio_user_device_get(VFIOUserContainer *container, + VFIODevice *vbasedev, Error **errp) +{ + struct vfio_device_info info =3D { 0 }; + + vbasedev->fd =3D -1; + + vfio_device_prepare(vbasedev, &container->bcontainer, &info); + + return true; +} + +/* + * vfio_user_device_attach: attach a device to a new container. + */ +static bool vfio_user_device_attach(const char *name, VFIODevice *vbasedev, + AddressSpace *as, Error **errp) +{ + VFIOUserContainer *container; + + container =3D vfio_user_container_connect(as, errp); + if (container =3D=3D NULL) { + error_prepend(errp, "failed to connect proxy"); + return false; + } + + return vfio_user_device_get(container, vbasedev, errp); +} + +static void vfio_user_device_detach(VFIODevice *vbasedev) +{ + VFIOUserContainer *container =3D container_of(vbasedev->bcontainer, + VFIOUserContainer, bcontai= ner); + + vfio_device_unprepare(vbasedev); + + vfio_user_container_disconnect(container); +} + +static int vfio_user_pci_hot_reset(VFIODevice *vbasedev, bool single) +{ + /* ->needs_reset is always false for vfio-user. */ + return 0; +} + +static void vfio_iommu_user_class_init(ObjectClass *klass, const void *dat= a) +{ + VFIOIOMMUClass *vioc =3D VFIO_IOMMU_CLASS(klass); + + vioc->setup =3D vfio_user_setup; + vioc->dma_map =3D vfio_user_dma_map; + vioc->dma_unmap =3D vfio_user_dma_unmap; + vioc->attach_device =3D vfio_user_device_attach; + vioc->detach_device =3D vfio_user_device_detach; + vioc->set_dirty_page_tracking =3D vfio_user_set_dirty_page_tracking; + vioc->query_dirty_bitmap =3D vfio_user_query_dirty_bitmap; + vioc->pci_hot_reset =3D vfio_user_pci_hot_reset; +}; + +static const TypeInfo types[] =3D { + { + .name =3D TYPE_VFIO_IOMMU_USER, + .parent =3D TYPE_VFIO_IOMMU, + .instance_size =3D sizeof(VFIOUserContainer), + .class_init =3D vfio_iommu_user_class_init, + }, +}; + +DEFINE_TYPES(types) diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c new file mode 100644 index 0000000000..86d7055747 --- /dev/null +++ b/hw/vfio-user/pci.c @@ -0,0 +1,185 @@ +/* + * vfio PCI device over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include +#include "qemu/osdep.h" +#include "qapi-visit-sockets.h" + +#include "hw/qdev-properties.h" +#include "hw/vfio/pci.h" + +#define TYPE_VFIO_USER_PCI "vfio-user-pci" +OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserPCIDevice, VFIO_USER_PCI) + +struct VFIOUserPCIDevice { + VFIOPCIDevice device; + SocketAddress *socket; +}; + +/* + * Emulated devices don't use host hot reset + */ +static void vfio_user_compute_needs_reset(VFIODevice *vbasedev) +{ + vbasedev->needs_reset =3D false; +} + +static Object *vfio_user_pci_get_object(VFIODevice *vbasedev) +{ + VFIOUserPCIDevice *vdev =3D container_of(vbasedev, VFIOUserPCIDevice, + device.vbasedev); + + return OBJECT(vdev); +} + +static VFIODeviceOps vfio_user_pci_ops =3D { + .vfio_compute_needs_reset =3D vfio_user_compute_needs_reset, + .vfio_eoi =3D vfio_pci_intx_eoi, + .vfio_get_object =3D vfio_user_pci_get_object, + /* No live migration support yet. */ + .vfio_save_config =3D NULL, + .vfio_load_config =3D NULL, +}; + +static void vfio_user_pci_realize(PCIDevice *pdev, Error **errp) +{ + ERRP_GUARD(); + VFIOUserPCIDevice *udev =3D VFIO_USER_PCI(pdev); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); + VFIODevice *vbasedev =3D &vdev->vbasedev; + const char *sock_name; + AddressSpace *as; + + if (!udev->socket) { + error_setg(errp, "No socket specified"); + error_append_hint(errp, "e.g. -device '{" + "\"driver\":\"vfio-user-pci\", " + "\"socket\": {\"path\": \"/tmp/vfio-user.sock\", " + "\"type\": \"unix\"}'" + "}'\n"); + return; + } + + sock_name =3D udev->socket->u.q_unix.path; + + vbasedev->name =3D g_strdup_printf("vfio-user:%s", sock_name); + + /* + * vfio-user devices are effectively mdevs (don't use a host iommu). + */ + vbasedev->mdev =3D true; + + as =3D pci_device_iommu_address_space(pdev); + if (!vfio_device_attach_by_iommu_type(TYPE_VFIO_IOMMU_USER, + vbasedev->name, vbasedev, + as, errp)) { + error_prepend(errp, VFIO_MSG_PREFIX, vbasedev->name); + return; + } +} + +static void vfio_user_instance_init(Object *obj) +{ + PCIDevice *pci_dev =3D PCI_DEVICE(obj); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); + VFIODevice *vbasedev =3D &vdev->vbasedev; + + device_add_bootindex_property(obj, &vdev->bootindex, + "bootindex", NULL, + &pci_dev->qdev); + vdev->host.domain =3D ~0U; + vdev->host.bus =3D ~0U; + vdev->host.slot =3D ~0U; + vdev->host.function =3D ~0U; + + vfio_device_init(vbasedev, VFIO_DEVICE_TYPE_PCI, &vfio_user_pci_ops, + DEVICE(vdev), false); + + vdev->nv_gpudirect_clique =3D 0xFF; + + /* + * QEMU_PCI_CAP_EXPRESS initialization does not depend on QEMU command + * line, therefore, no need to wait to realize like other devices. + */ + pci_dev->cap_present |=3D QEMU_PCI_CAP_EXPRESS; +} + +static void vfio_user_instance_finalize(Object *obj) +{ + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); + + vfio_pci_put_device(vdev); +} + +static const Property vfio_user_pci_dev_properties[] =3D { + DEFINE_PROP_UINT32("x-pci-vendor-id", VFIOPCIDevice, + vendor_id, PCI_ANY_ID), + DEFINE_PROP_UINT32("x-pci-device-id", VFIOPCIDevice, + device_id, PCI_ANY_ID), + DEFINE_PROP_UINT32("x-pci-sub-vendor-id", VFIOPCIDevice, + sub_vendor_id, PCI_ANY_ID), + DEFINE_PROP_UINT32("x-pci-sub-device-id", VFIOPCIDevice, + sub_device_id, PCI_ANY_ID), +}; + +static void vfio_user_pci_set_socket(Object *obj, Visitor *v, const char *= name, + void *opaque, Error **errp) +{ + VFIOUserPCIDevice *udev =3D VFIO_USER_PCI(obj); + bool success; + + qapi_free_SocketAddress(udev->socket); + + udev->socket =3D NULL; + + success =3D visit_type_SocketAddress(v, name, &udev->socket, errp); + + if (!success) { + return; + } + + if (udev->socket->type !=3D SOCKET_ADDRESS_TYPE_UNIX) { + error_setg(errp, "Unsupported socket type %s", + SocketAddressType_str(udev->socket->type)); + qapi_free_SocketAddress(udev->socket); + udev->socket =3D NULL; + return; + } +} + +static void vfio_user_pci_dev_class_init(ObjectClass *klass, const void *d= ata) +{ + DeviceClass *dc =3D DEVICE_CLASS(klass); + PCIDeviceClass *pdc =3D PCI_DEVICE_CLASS(klass); + + device_class_set_props(dc, vfio_user_pci_dev_properties); + + object_class_property_add(klass, "socket", "SocketAddress", NULL, + vfio_user_pci_set_socket, NULL, NULL); + object_class_property_set_description(klass, "socket", + "SocketAddress (UNIX sockets onl= y)"); + + dc->desc =3D "VFIO over socket PCI device assignment"; + pdc->realize =3D vfio_user_pci_realize; +} + +static const TypeInfo vfio_user_pci_dev_info =3D { + .name =3D TYPE_VFIO_USER_PCI, + .parent =3D TYPE_VFIO_PCI_BASE, + .instance_size =3D sizeof(VFIOUserPCIDevice), + .class_init =3D vfio_user_pci_dev_class_init, + .instance_init =3D vfio_user_instance_init, + .instance_finalize =3D vfio_user_instance_finalize, +}; + +static void register_vfio_user_dev_type(void) +{ + type_register_static(&vfio_user_pci_dev_info); +} + + type_init(register_vfio_user_dev_type) diff --git a/hw/Kconfig b/hw/Kconfig index 9a86a6a28a..9e6c789ae7 100644 --- a/hw/Kconfig +++ b/hw/Kconfig @@ -42,6 +42,7 @@ source ufs/Kconfig source usb/Kconfig source virtio/Kconfig source vfio/Kconfig +source vfio-user/Kconfig source vmapple/Kconfig source xen/Kconfig source watchdog/Kconfig diff --git a/hw/meson.build b/hw/meson.build index b91f761fe0..791ce21ab4 100644 --- a/hw/meson.build +++ b/hw/meson.build @@ -39,6 +39,7 @@ subdir('uefi') subdir('ufs') subdir('usb') subdir('vfio') +subdir('vfio-user') subdir('virtio') subdir('vmapple') subdir('watchdog') diff --git a/hw/vfio-user/Kconfig b/hw/vfio-user/Kconfig new file mode 100644 index 0000000000..24bdf7af90 --- /dev/null +++ b/hw/vfio-user/Kconfig @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: GPL-2.0-or-later + +config VFIO_USER + bool + default y + depends on VFIO_PCI + diff --git a/hw/vfio-user/meson.build b/hw/vfio-user/meson.build new file mode 100644 index 0000000000..b82c558252 --- /dev/null +++ b/hw/vfio-user/meson.build @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0-or-later + +vfio_user_ss =3D ss.source_set() +vfio_user_ss.add(files( + 'container.c', + 'pci.c', +)) + +system_ss.add_all(when: 'CONFIG_VFIO_USER', if_true: vfio_user_ss) --=20 2.43.0 From nobody Sat Nov 15 14:11:13 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750879969; cv=pass; d=zohomail.com; s=zohoarc; b=GfpBvmB2CPjmIBiUbV1Tk+VkktMscymvzSmCfw92yu20Gur+i5ncqvteVhDNMffj1WYl0C0DeWp7aqQaROyBW1wY/z1y4QXRM3RmRnhpHTzu3gxNzjqaUxVGrn1P4Is3Bm7RSU44/FhNhK1cO9HrKMnsRexxP0CBKhjdKlFeYSE= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750879969; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=N4EobuCCtWRuZK8PlH1YWzQe+7zofV6zf3JMiBNLN9Y=; b=fBkvjPTeH2LHuxfJnMvtsRxYzae5Am8Y1Ej5dwb59Ftfmg5jIHpETYxOj62ajFVvJoAgKnk66z5aGjx0AnlIfgjY2ksFiYDyX/UCoUuC6alg0A8EGggui1n30TU63fPCnw3kHSzL/ADDYL4YuYmrBuZ981E1Wsosa3EpND9ivqs= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750879969593830.5839301384145; Wed, 25 Jun 2025 12:32:49 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpH-0007zO-Ga; Wed, 25 Jun 2025 15:30:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpG-0007zG-G3 for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:30 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpC-0003Gn-RA for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:30 -0400 Received: from pps.filterd (m0127842.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PHZeal021791; Wed, 25 Jun 2025 12:30:25 -0700 Received: from nam04-mw2-obe.outbound.protection.outlook.com (mail-mw2nam04on2099.outbound.protection.outlook.com [40.107.101.99]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47geghhf7u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:24 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:22 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:22 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=N4EobuCCtWRuZK8PlH1YWzQe+7zofV6zf3JMiBNLN 9Y=; b=s1DJYa2DDWCdPdg451S8fDU8CRq/PIvT8mFZBoD7HQVLTGW7ThA0f2wx6 evRyxOrieCG9qvxLZAWB12TLOnGd+8IN4sp180lS+ClPlYSWAU9TPw5EPoS1gYmb Cvqrzc+QcvDAh0/k+ehKydXktJVyRKtVMxHVo9v9+3hIbtoNnKkvVvInXi//69TW 6h/dpWnhu7u3+kWnVMdQNVmW7nE1r/2eBpgVpC0nwp0+p6zVnUrTxaBKw4BLnc1J X8/zRD+G2L2RD+IjRCruV7QZq7eFNSXGQOnaWlZOVgPt0TXrcIdAnE3Ze3pdYtMw f5WzUxAZ1fHCSBBizFrGiQ0XxPBKA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=bpkCWuUILwzDac6FiCBJt85NICz3YlKbBGKuRPRNQmL0BpY+TXVcb5M4j7oyIG3YD38KNJCu5gioXwQFy3Z01F5rOCfHuqafJU49AEGGlP1vdEKBLov5xAcYFEXL87mEyY2wHay61A8nFIQF6YYGT3928+VkLnekU+RK+3RlyQi/nzpndwIsN+McEyQETkM6BdsagPFd0+QyFKa04ifYY4zyVScfzmXB1qKp22/4RIJZ1QOS6bIqxYkXPDiwl0uccJ+z8Buz0iuLRNiYpbScHU6NWfqpVs2vNjoTCsamrYfJmO9PzcoCo3ZMy3MqLxaiU2o9wqkXXtPQva6VwkeYPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=N4EobuCCtWRuZK8PlH1YWzQe+7zofV6zf3JMiBNLN9Y=; b=CfEbh96WOo1jXyGKmBnLxuQ5Tp4IikQrJz5sDzmauyKSzV0vFCx3vrX/GyrlLSmk89HgvvEKXz8asCY8HilQ4kTCvCrVZfI6y3nAYeWySSBk/AOa/vtCzhoBZmOhgbvwTdVkB7up2Jl46jyYGGsCexfod81g2wx5RRI4GdFh34mqA4P0A4l+c3vJAe29lFPMz43iUily8v8BkU988bjlQ/ovlRUosKHEwQcMQ+IPFZUu5mLuupfGlHGjPJoK2E7KHXYrK6gaa/cqbhE+oIZSRywUUQDWp1Nkz1/UzbFNUwIDXk37L6QDDWgKHiUeUlHRWRTyDoOxDrbwEL9CD+G1xw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N4EobuCCtWRuZK8PlH1YWzQe+7zofV6zf3JMiBNLN9Y=; b=Nq01H6/gos/2TE0lxGO1b7iYYZj1il1/FLNDB+oISAc9/Lbl3MZUXEL8gxLIQNg1v5FC6MH4q/sbB6lEruyz8r8ybvMyGRzON0pWvQWkiYvVze60Xb2zkGTPzXvWfXjf9Tsu/9osbpW1E4PPDtnuj9I4fo7QWcgFPq/7iHY3Db2ZSM/NLxYWgedOa9kHhWIrh7D4VATOXKEyrxOuXX/21n7Kc93/W8uxSkXj6Tk0sT8f4AdczEVjuqoI3GlZzsom58N/OAL0YM0F1s8K2un9FwdatMwh8kf8BBDTw7rVKCJrXFED4pK4BpzWEB8rWL7j71utbsqvUCsrTC49zf9YmA== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 02/19] vfio-user: connect vfio proxy to remote server Date: Wed, 25 Jun 2025 20:29:54 +0100 Message-ID: <20250625193012.2316242-3-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: bc576bfe-e256-40b4-4859-08ddb41eba4e x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?B?MEpodlRpbDJpdWFNNlRqc0dMcUZjMTAvdGtrWlREQmI1YW9LUTMxS1pWZVdR?= =?utf-8?B?VlJXbFlrR2lvWC9YaGRsdURyYnhNSzBDVG1WOGNnVjVLQkxRZ1dQZGhvYWxr?= =?utf-8?B?M3lsWE56aTBFekg5MTZwN0NhdGRHd2hFUW9nMW1iM0dWTExOZE4zMlFGbS9O?= =?utf-8?B?TktZS1crUElLRWZVTlZ1MDA4MDRmeXlkRGZpdG1waXQvaDd3TFJTS0swUU40?= =?utf-8?B?alQyaHlRME5iL3BCcXZiV0MraUJtbWFNa3psLzVRaCtXOWQ2aFlWdzhYK2JP?= =?utf-8?B?WU42QzdCbUVhcG5YbGk5SmpxVEhQNmpHWVhTbnA1aHFkU2pYN2x0c1IzTXRX?= =?utf-8?B?WWJhTThYc0xSa0ZJMVZVZFBvSERjVHNYK1B0RjFSZ0EwaXFNTWtxbHdKQmow?= =?utf-8?B?bUk0VVRFcEIzd2wyaWliTE1EeXNZNVhTZ3VKTHEzOTlSSThpSjgvZDBESHYr?= =?utf-8?B?SmRlQUw1Ty90ZXJ0THBJVkxMQU1zNlk1cm4yV0U3TWt1bDhDRUl4M3NWRVRT?= =?utf-8?B?UDZjR0RMYVBrU21MWlhVYjB6NDUxT1hzZGJacnc1c0J0QVI3SHRwRnlxb3lG?= =?utf-8?B?c213MHlGSHA0cGRia2t1TDZsWmxYeC9heFFGMWpkcGdQWk56cEpHUHdBemFC?= =?utf-8?B?YUJBd0dCeXRKTnRsM0JlS3ZWQnk1dGVYcS80Y1VmTVR1ODF3d2hrN2pRcjhu?= =?utf-8?B?a2tKam1DSWk4YVhJNTMzZFBrMHB3S2ZEWEpsNWpjSG5sRTBZQ21EbmFGV21Y?= =?utf-8?B?aVZwMmt3MEtFMHZGaUpFQjJ6MkwvMktTbmg5SlJraTFLRE9rcGpJRFovZUV2?= =?utf-8?B?OENXVmMzRzQ0c2FwYVpvYVhqbjdjdXNxLytYWE1YTXhFZ3VrMXBQTDNGVmtp?= =?utf-8?B?VzVpQWJsZmxBQngwWkkyL1ZmSWZtRkpsYmNpTXY1aWFzbFZaM1g4Nzh2Qjll?= =?utf-8?B?RUNKOERteUlWTFhzMURPeURya3pMMXNZb0loUURkNEd4NHBUZEVIYTBCMFEr?= =?utf-8?B?K2lYUWxXZVhhcEc2bXRCd1FqUmVVelEzVHRvNllMcUtVNDQ0bmwrcTlCSlFU?= =?utf-8?B?Rkwxa3U0RjBlWTdGaUxFNElaWkhnQmdBb2M4SjZaSGtUK2tEY2tBNFNQbm9H?= =?utf-8?B?NjRiU1JFZ1hiSkg3VVQ0dE5rS0xKZUJ2Sk8wUE5pVTVHVWV1bGNCa1lJS3Ey?= =?utf-8?B?Sk1VbXNldWJDczUzZE95TGQrRkdrVFYwSlVBOWxWb1RVVXBnMmhyWEJ6aUh5?= =?utf-8?B?WTRuSnhLSFJzQVhGVVBVSlRpY1hKYUVObEVPZGJsc3gxTzlTZlEyb0R6NmE5?= =?utf-8?B?U3kxdjBnRitjUWpmZmQzTmdWM0F0SzdyVFRQdjh5T0NTR09ZV3BZMENmaUxO?= =?utf-8?B?V21Pa0NQbEpNdGdFM0l4NGF5MGM4b1FmTlRSWDA5Zmxqd20wRUNPU0Vhd3Rk?= =?utf-8?B?ZkhGSDI5dHdabFBzcnF3cG4vVnJUYlNMVUFvRUdiWjZpYy83cFZ3amlHWXRz?= =?utf-8?B?RWhuUDZsSFRGOWw3YVBKWWNBTTVwRWRxSGRMMkZ3REJ3UXBHc2xHMzBoVSt3?= =?utf-8?B?YTZEK09LTjlRZlE4Z2JVQ3crNGJKUEVlNjRRVVlKK0R5NTYzNGtvc1ZBUk5o?= =?utf-8?B?UHhic3J1WmcwWHRMZHhPUGxxS3kvOHhlbVU3cFdVNithZUJYNFlOWVZXZHZT?= =?utf-8?B?MDhPaTQzeHpiNy9ta3h6OUxSY3pOeDJUYkJzcnQzY1F6RUVUSGpxN1NvTlc1?= =?utf-8?B?QWNVNFF2dWRFd0hrUnoxTTErZUJmM2lpL2toZXBwdk9xcUljL3lXQ2RmUGZk?= =?utf-8?B?U0pGbUU3YlZqUVppdUU5Wmdxb1NuZDlrR1RUcmRKdXE0TFRLN2dTdlNNRVhn?= =?utf-8?B?Uy9yTXNWU3JHS0tnYk54elc5SEttQjUxUDFuajNXS1lkUjNQajF6SC9hRjZX?= =?utf-8?Q?UCfQO/L4lp0=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?K2w3dkJkMTUrOHkyeFczdTgvSGNrZXU2bnR0N05uRGl1TjRvSStjN2tMbzRq?= =?utf-8?B?cFJocVF3VjA3blFMeXhKeHFiNWRWKzhxYTFTTTZXK3hqWXdVSVpsTjN2d3Yw?= =?utf-8?B?bEQ0RHp0N1hSZDE0eUJ3cU0rcnRtMmx4cEYwZUVURVJVYkxKWStUOVFJRHEr?= =?utf-8?B?bTNpQU1DQ0pEUHQvekJWbGtvbldTMU9HZXlSb3I2ZkEwdytnMTdSbm5lRmVE?= =?utf-8?B?aVZrWk1NeGkxTHcyVnJkeEYrYVlqMThwOEhwRWJ6VTFmSWVRclF5cFEzcmNW?= =?utf-8?B?MVRlWGNsYTN2THo1VDJzbFBzTllUUjJueEREeGxmM3puNTNZVmtKbXFsbjhV?= =?utf-8?B?cW1mRlRqb0orOXZmTkxKNlBITElwKzdjeXNNK0VMR0FUdENGMTdaSnU5cjFx?= =?utf-8?B?Z3RpdkpOMDZ1Ly9TS3lzbHlkSFVRQVAyOU5RRnE3SHg0Y3diNGdWQVJCNDZN?= =?utf-8?B?U3g4enBLU1E3R0tDMnBZYTQydTNkK0h1R3VKakg4bzNkcFJ4Z3I3RkdqYWdV?= =?utf-8?B?a2hmelp4UXB0LzFiT21EUWZMU3FUOFl0Rktoa28rUHZCakxBZE5MbHYrVjh3?= =?utf-8?B?ZmhjY0RualpLYmZpeXEzUm9iM25TQlRoVXVZdUNxMDdpM0YzUmVrZ0pzcEM2?= =?utf-8?B?WFlCcnloOEZHYWV2MitiZ2NCMTRnRVYvQ243eXNaSGx4RlNYVUQxYW1PY2VX?= =?utf-8?B?NXRTc1ZjcTBWTmlkWEVFWlYvYVQ3blNnNnhXb29yeFhDV0tLTXVqZnhyUkxV?= =?utf-8?B?bm5hMVlDQ2NtbnV3YTZwNnZOblVHNHJzdExNdlZoUnpWQ2JJOE41KzRzTWVY?= =?utf-8?B?QU54TG5iL1V4bnhpSzdva3ZtbFY4TmRwcjhXUVlpeGQzSFk5NVZ0elo3OHVa?= =?utf-8?B?OS9aeG5GU2RHZWZpbjgydHZhdzJ0RXVIN1RnMWg2QlA4QThJR0ZwRldPdkxk?= =?utf-8?B?cVlmdVVBK2RPQ1BnMFpVQ0IxTzY2V1owSWJhQi9tcDRSOHhEc2lTZHJLYWty?= =?utf-8?B?S1ppeUpBMk1yd3lOMkRXclg5dnpzQUZqNlBTNjlRbTRaSndQRDEzMGVLZ29U?= =?utf-8?B?eHJxbENUcmN4R2RQMkhJWW1qMFlvc2RpSGN2bTFZTEIxdnhVcEx3REQrV2JD?= =?utf-8?B?RDN0cTBLUmNCYk1MWGVHREdSMStTYmxRL2crT2hUUmU4SytWSnNueldHMDhq?= =?utf-8?B?QzY4b3NCcEY5VEh6YlVsZTFQRHZFays3MUZ6L1VMb2lzb3NFU0VRalJOeURn?= =?utf-8?B?a0Rkbk5IOU40aU1SYjhmdE9JRHRhNTRnZVNXMElDK3pOWC9oUFNmUTlNRGc4?= =?utf-8?B?NDBVTlVvNHI2a2I3c1gyOXdXSHJlM3g5b2lIcGw0K3o5YyttTW1TNjZ3bjZY?= =?utf-8?B?UFZnT3ZycjRYeGF1ZjZQUDBtcVZDU2R6MDhSVWJuL2JseWpHOU56YWt4Q3pw?= =?utf-8?B?VWkvRWQ4RVRHTS9Kd21pNFZFbWlWUzNFcE8veVVTUmpEdlpBVkwzOE1nNEtF?= =?utf-8?B?cVVTbk5PSTJNb0FheFhNUWpsYUFSakhQVCtrNUkyUSsxdHVXSlAwdC9xZjFY?= =?utf-8?B?TTV0OHJuZ0ExRkJySUxBQ1dnWWk0L2QxdEJUblpzdGlhMEZNak5IV0RqVUU2?= =?utf-8?B?L3NIdm5jTkdoRm9lUGRWSStKcFZoc1locDg3VW4wclozSW9Ud25vdS9Lc1hn?= =?utf-8?B?N081RFdHVDduc2FON1hwMmJmYmY1OVZFNHZzaXkvSG5GVmNiZ3FmZ1B2ZXp4?= =?utf-8?B?aTVYMHpvOGFqQ3FINVYzeHVrc0hiYkNuNzBIeDlueDlDSXpia2VWekhIOEpz?= =?utf-8?B?TVB6UlFYVXNnbXRzTG9YRVdEK1JJMzk0VHhpVnYzSUx6VU1OUDNhOVd5Ri9w?= =?utf-8?B?a2MzTWdPU3lvWEZZL0g0QnVHTTVyakEzV1VDQi9yemJ2cjNMNzdubnpNK3Ry?= =?utf-8?B?Z2Vwc0JTSi9wNzlpVGxtaVhIOXdlWUoyWXdXVnZzbThIV3lMdElGandWU1RE?= =?utf-8?B?YkJ6ekF5RnNvMHA0Rnk1ZGExenNMeUM3dUFLVmZvcVpyeDd0R0FhS3hMVitx?= =?utf-8?B?bEVKb3VTTS9KT242SnlWa2N6Vkg3bFVZeG0zZlNFYkR0TlZudHgzWnV0M3FW?= =?utf-8?Q?6j0yqLlMenP5beNPQfiOc4/vK?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: bc576bfe-e256-40b4-4859-08ddb41eba4e X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:22.7908 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jiIcBFd4Us7n7Nyj1sj8/X+gK3b8a0bDeMhTh3gnaqY5UfqAt5hamY4xLEWdC52RB0i+yGNX1nqyqwuHeEMLDQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Authority-Analysis: v=2.4 cv=fPo53Yae c=1 sm=1 tr=0 ts=685c4e50 cx=c_pps a=BDhOPDsQvggip5BazaeBNQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=Fsi43M_n0QUer9rQu0sA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: NCX869KavWWMU1PznvFB8fPtf2b7zfQg X-Proofpoint-GUID: NCX869KavWWMU1PznvFB8fPtf2b7zfQg X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfXyr/hSlz19Bm/ TALKAsTUaxhAC+LaOmo2hh0dS/F91KAfXQt2gYuEOQuadwzWr3/XLVqB7lDFqaWqsdQ92+HMf0n wjqDeYfB1O5B3XwZzscfGOLypclWJ6ueQ4uSYLTeIdjCgI0A8zg1wRbQjTNA9MDvt7Zl4TuBNXU HX3ku1pj7KJ7UIREDBre12NJ0Ipfqh4ptw9peS8s6Gmjs1ba5RW9K19V5UuF0DwttxgCMHSXSUB 3PWuYHVy2c68x6KXXHZXZtm2mSFERRrGLbgnQMaBhX+l+0Z8J0Gsbql9kQx0TYbRBNx23EyzpfN 7rNA2wWQ80c8uZGTT6q7yitGFmc9STfsr+UDzd2XFDof6B7AJ82314k7YmYDlev6fNDTP89uMCB 1/pe3+IRE7wKbSx6hvW6/NqUZDSfaey0tTJTjI6mv3F4gZgAn+KVupNJModcvx4b1B48WX96 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750879971334116600 Introduce the vfio-user "proxy": this is the client code responsible for sending and receiving vfio-user messages across the control socket. The new files hw/vfio-user/proxy.[ch] contain some basic plumbing for managing the proxy; initialize the proxy during realization of the VFIOUserPCIDevice instance. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/proxy.h | 79 +++++++++++++++++ include/hw/vfio/vfio-device.h | 2 + hw/vfio-user/pci.c | 22 +++++ hw/vfio-user/proxy.c | 162 ++++++++++++++++++++++++++++++++++ hw/vfio-user/meson.build | 1 + 5 files changed, 266 insertions(+) create mode 100644 hw/vfio-user/proxy.h create mode 100644 hw/vfio-user/proxy.c diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h new file mode 100644 index 0000000000..a9bce82239 --- /dev/null +++ b/hw/vfio-user/proxy.h @@ -0,0 +1,79 @@ +#ifndef VFIO_USER_PROXY_H +#define VFIO_USER_PROXY_H + +/* + * vfio protocol over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "io/channel.h" +#include "io/channel-socket.h" + +typedef struct { + int send_fds; + int recv_fds; + int *fds; +} VFIOUserFDs; + +enum msg_type { + VFIO_MSG_NONE, + VFIO_MSG_ASYNC, + VFIO_MSG_WAIT, + VFIO_MSG_NOWAIT, + VFIO_MSG_REQ, +}; + +typedef struct VFIOUserMsg { + QTAILQ_ENTRY(VFIOUserMsg) next; + VFIOUserFDs *fds; + uint32_t rsize; + uint32_t id; + QemuCond cv; + bool complete; + enum msg_type type; +} VFIOUserMsg; + + +enum proxy_state { + VFIO_PROXY_CONNECTED =3D 1, + VFIO_PROXY_ERROR =3D 2, + VFIO_PROXY_CLOSING =3D 3, + VFIO_PROXY_CLOSED =3D 4, +}; + +typedef QTAILQ_HEAD(VFIOUserMsgQ, VFIOUserMsg) VFIOUserMsgQ; + +typedef struct VFIOUserProxy { + QLIST_ENTRY(VFIOUserProxy) next; + char *sockname; + struct QIOChannel *ioc; + void (*request)(void *opaque, VFIOUserMsg *msg); + void *req_arg; + int flags; + QemuCond close_cv; + AioContext *ctx; + QEMUBH *req_bh; + + /* + * above only changed when BQL is held + * below are protected by per-proxy lock + */ + QemuMutex lock; + VFIOUserMsgQ free; + VFIOUserMsgQ pending; + VFIOUserMsgQ incoming; + VFIOUserMsgQ outgoing; + VFIOUserMsg *last_nowait; + enum proxy_state state; +} VFIOUserProxy; + +/* VFIOProxy flags */ +#define VFIO_PROXY_CLIENT 0x1 + +VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp); +void vfio_user_disconnect(VFIOUserProxy *proxy); + +#endif /* VFIO_USER_PROXY_H */ diff --git a/include/hw/vfio/vfio-device.h b/include/hw/vfio/vfio-device.h index d45e5a68a2..7c0669de0d 100644 --- a/include/hw/vfio/vfio-device.h +++ b/include/hw/vfio/vfio-device.h @@ -47,6 +47,7 @@ typedef struct VFIOMigration VFIOMigration; =20 typedef struct IOMMUFDBackend IOMMUFDBackend; typedef struct VFIOIOASHwpt VFIOIOASHwpt; +typedef struct VFIOUserProxy VFIOUserProxy; =20 typedef struct VFIODevice { QLIST_ENTRY(VFIODevice) next; @@ -88,6 +89,7 @@ typedef struct VFIODevice { struct vfio_region_info **reginfo; int *region_fds; VFIODeviceCPR cpr; + VFIOUserProxy *proxy; } VFIODevice; =20 struct VFIODeviceOps { diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 86d7055747..642421e791 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -12,6 +12,7 @@ =20 #include "hw/qdev-properties.h" #include "hw/vfio/pci.h" +#include "hw/vfio-user/proxy.h" =20 #define TYPE_VFIO_USER_PCI "vfio-user-pci" OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserPCIDevice, VFIO_USER_PCI) @@ -54,6 +55,8 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error = **errp) VFIODevice *vbasedev =3D &vdev->vbasedev; const char *sock_name; AddressSpace *as; + SocketAddress addr; + VFIOUserProxy *proxy; =20 if (!udev->socket) { error_setg(errp, "No socket specified"); @@ -69,6 +72,15 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error= **errp) =20 vbasedev->name =3D g_strdup_printf("vfio-user:%s", sock_name); =20 + memset(&addr, 0, sizeof(addr)); + addr.type =3D SOCKET_ADDRESS_TYPE_UNIX; + addr.u.q_unix.path =3D (char *)sock_name; + proxy =3D vfio_user_connect_dev(&addr, errp); + if (!proxy) { + return; + } + vbasedev->proxy =3D proxy; + /* * vfio-user devices are effectively mdevs (don't use a host iommu). */ @@ -112,8 +124,13 @@ static void vfio_user_instance_init(Object *obj) static void vfio_user_instance_finalize(Object *obj) { VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); + VFIODevice *vbasedev =3D &vdev->vbasedev; =20 vfio_pci_put_device(vdev); + + if (vbasedev->proxy !=3D NULL) { + vfio_user_disconnect(vbasedev->proxy); + } } =20 static const Property vfio_user_pci_dev_properties[] =3D { @@ -133,6 +150,11 @@ static void vfio_user_pci_set_socket(Object *obj, Visi= tor *v, const char *name, VFIOUserPCIDevice *udev =3D VFIO_USER_PCI(obj); bool success; =20 + if (udev->device.vbasedev.proxy) { + error_setg(errp, "Proxy is connected"); + return; + } + qapi_free_SocketAddress(udev->socket); =20 udev->socket =3D NULL; diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c new file mode 100644 index 0000000000..bb436c9db9 --- /dev/null +++ b/hw/vfio-user/proxy.c @@ -0,0 +1,162 @@ +/* + * vfio protocol over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include + +#include "hw/vfio/vfio-device.h" +#include "hw/vfio-user/proxy.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "qemu/lockable.h" +#include "system/iothread.h" + +static IOThread *vfio_user_iothread; + +static void vfio_user_shutdown(VFIOUserProxy *proxy); + + +/* + * Functions called by main, CPU, or iothread threads + */ + +static void vfio_user_shutdown(VFIOUserProxy *proxy) +{ + qio_channel_shutdown(proxy->ioc, QIO_CHANNEL_SHUTDOWN_READ, NULL); + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, NULL, + proxy->ctx, NULL, NULL); +} + +/* + * Functions only called by iothread + */ + +static void vfio_user_cb(void *opaque) +{ + VFIOUserProxy *proxy =3D opaque; + + QEMU_LOCK_GUARD(&proxy->lock); + + proxy->state =3D VFIO_PROXY_CLOSED; + qemu_cond_signal(&proxy->close_cv); +} + + +/* + * Functions called by main or CPU threads + */ + +static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D + QLIST_HEAD_INITIALIZER(vfio_user_sockets); + +VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp) +{ + VFIOUserProxy *proxy; + QIOChannelSocket *sioc; + QIOChannel *ioc; + char *sockname; + + if (addr->type !=3D SOCKET_ADDRESS_TYPE_UNIX) { + error_setg(errp, "vfio_user_connect - bad address family"); + return NULL; + } + sockname =3D addr->u.q_unix.path; + + sioc =3D qio_channel_socket_new(); + ioc =3D QIO_CHANNEL(sioc); + if (qio_channel_socket_connect_sync(sioc, addr, errp)) { + object_unref(OBJECT(ioc)); + return NULL; + } + qio_channel_set_blocking(ioc, false, NULL); + + proxy =3D g_malloc0(sizeof(VFIOUserProxy)); + proxy->sockname =3D g_strdup_printf("unix:%s", sockname); + proxy->ioc =3D ioc; + proxy->flags =3D VFIO_PROXY_CLIENT; + proxy->state =3D VFIO_PROXY_CONNECTED; + + qemu_mutex_init(&proxy->lock); + qemu_cond_init(&proxy->close_cv); + + if (vfio_user_iothread =3D=3D NULL) { + vfio_user_iothread =3D iothread_create("VFIO user", errp); + } + + proxy->ctx =3D iothread_get_aio_context(vfio_user_iothread); + + QTAILQ_INIT(&proxy->outgoing); + QTAILQ_INIT(&proxy->incoming); + QTAILQ_INIT(&proxy->free); + QTAILQ_INIT(&proxy->pending); + QLIST_INSERT_HEAD(&vfio_user_sockets, proxy, next); + + return proxy; +} + +void vfio_user_disconnect(VFIOUserProxy *proxy) +{ + VFIOUserMsg *r1, *r2; + + qemu_mutex_lock(&proxy->lock); + + /* our side is quitting */ + if (proxy->state =3D=3D VFIO_PROXY_CONNECTED) { + vfio_user_shutdown(proxy); + if (!QTAILQ_EMPTY(&proxy->pending)) { + error_printf("vfio_user_disconnect: outstanding requests\n"); + } + } + object_unref(OBJECT(proxy->ioc)); + proxy->ioc =3D NULL; + + proxy->state =3D VFIO_PROXY_CLOSING; + QTAILQ_FOREACH_SAFE(r1, &proxy->outgoing, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->outgoing, r1, next); + g_free(r1); + } + QTAILQ_FOREACH_SAFE(r1, &proxy->incoming, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->incoming, r1, next); + g_free(r1); + } + QTAILQ_FOREACH_SAFE(r1, &proxy->pending, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->pending, r1, next); + g_free(r1); + } + QTAILQ_FOREACH_SAFE(r1, &proxy->free, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->free, r1, next); + g_free(r1); + } + + /* + * Make sure the iothread isn't blocking anywhere + * with a ref to this proxy by waiting for a BH + * handler to run after the proxy fd handlers were + * deleted above. + */ + aio_bh_schedule_oneshot(proxy->ctx, vfio_user_cb, proxy); + qemu_cond_wait(&proxy->close_cv, &proxy->lock); + + /* we now hold the only ref to proxy */ + qemu_mutex_unlock(&proxy->lock); + qemu_cond_destroy(&proxy->close_cv); + qemu_mutex_destroy(&proxy->lock); + + QLIST_REMOVE(proxy, next); + if (QLIST_EMPTY(&vfio_user_sockets)) { + iothread_destroy(vfio_user_iothread); + vfio_user_iothread =3D NULL; + } + + g_free(proxy->sockname); + g_free(proxy); +} diff --git a/hw/vfio-user/meson.build b/hw/vfio-user/meson.build index b82c558252..9e85a8ea51 100644 --- a/hw/vfio-user/meson.build +++ b/hw/vfio-user/meson.build @@ -4,6 +4,7 @@ vfio_user_ss =3D ss.source_set() vfio_user_ss.add(files( 'container.c', 'pci.c', + 'proxy.c', )) =20 system_ss.add_all(when: 'CONFIG_VFIO_USER', if_true: vfio_user_ss) --=20 2.43.0 From nobody Sat Nov 15 14:11:13 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750879960; cv=pass; d=zohomail.com; s=zohoarc; b=h3z3CbDX+H5W0YHB6hjv6dTJNC48MKamSt43xV2rUXBU8qVoSNsImcHPq+k1AtdOXVcIHZBCTUWWpGvh+CLBd98tQLma16Pt6BY+1ft3TRsKXhNpv2lUmF0wsuBHF3So9RRzF2Gs6oY1TS05rcaiyynAlI0IMx9LchMKWwJyxck= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750879960; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=3VtVkOj29BrAPqtXIVFu08O9WpDLdbR+Xej4jNDewwI=; b=aOgUqdcB0vcKaoctFXoMAVdSq4hyE2DMCdoKhiIGvQ5LCAyFcEwe82HSQrkGxlxdbOKMshHtULMhBLjqtu+fUdoZ98nslA5SBraSUEhpiXRWJU/JGMvq5s5J+UCxpcPRnn5gaJEdKUvTGzHyldavRy/MrS9ABvUg+sR4dyEyDuM= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750879960225485.76229042398245; Wed, 25 Jun 2025 12:32:40 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpL-00080K-57; Wed, 25 Jun 2025 15:30:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpJ-0007zj-5Y for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:33 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpF-0003Nd-2I for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:32 -0400 Received: from pps.filterd (m0127842.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PHZeam021791; Wed, 25 Jun 2025 12:30:27 -0700 Received: from nam04-mw2-obe.outbound.protection.outlook.com (mail-mw2nam04on2116.outbound.protection.outlook.com [40.107.101.116]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47geghhf81-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:27 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:25 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:25 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=3VtVkOj29BrAPqtXIVFu08O9WpDLdbR+Xej4jNDew wI=; b=BPwKQyW2EBPLVIRSMeo7TZBleO9w674KHEIZLcTPPEgLNrmiqJOmcQohU eoObcVDEnJOktas4obpIMc3P56ikVD+yK1NC1OC2syhet6bKVy2PvgmS50qwRWQi iWT0oqZng7B6dPXAFc3mCbAR/R1x93fvwVgbdOIeQgtNesZO0PTgQ8Rwf1aAMP/P Hi5cIzsdpis+0c42rtwezMcPasDNLz8LRVLs+R5AR5hWOD2r9P9h51H0rkKjWgW1 3/CmoWG+fNBlZaxrRHjRn1RaP+Xv9K3UmxSpeaX8jiDhE+c+xlLqjKfcpfB/gNDX bM43ubrdeBRkW6tbP+w7dC8EwI4kw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=nSRy2YNcRlT99JJ2UFWoroXVlOC+pCJr1msjXxxLXCr9q9P3R3r2IgJ6sBOGeLZ37/8oSN4b9p5nk0JbD5nndTL2N2pNkaOD9McnDxqjJO4nPl2rn2khXPHqW7Qhuq6mhIk/8sby4iI3DlQPOrbhUWCgWoYBkDXWaVW5ObUoG5TRPfQBDcKI9QCqT3AGkdg0yfKkz3hXalECSEFG5lAP9i0RS0VPprXYbucoEzrDw8JrsYVEjgsdDOwNnR/1BnkE+Do/xC0wztXYu+sPHnBBpgSMIIESG9/zfaLdHAKj/Ryckmfm1YaLSuZM0icI8Dk7lsdrnQfyNvjxWnaqxcmQyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3VtVkOj29BrAPqtXIVFu08O9WpDLdbR+Xej4jNDewwI=; b=KRgTvCWsZc99Bx05Mp7uh3Z9ehKMhPd4gWo+6+5UQjpTM5ri2eQuHkoNFVfPPlt9ZGnMwt4gyNm8mgUglXSemWa1ZEeK/Y4VWaBySH+qO6fd20+mzVqrobX2Tio3r4vyvGbi6rCtm2orukimaWhhRqkqR0IfbazhYeG2L1XA+PoWzKwmCpH19latACZZpoAcCvI4qBOQyQw7xrJDcWKeZcjvwjTr2AeWz/drigvRprx19npdbmrhrtXtvvftL+qphfQxcFg1Khp7nHotrzVywjv8HxtlWqhsMHO7WSfLNtRZ+ZrUBFMLOYNnYYS/jAFW9z/OOB4B+tqWw6s83fx8hA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3VtVkOj29BrAPqtXIVFu08O9WpDLdbR+Xej4jNDewwI=; b=s1BIfsJgirOU/mR/XAf1DwDxBnnzvmRYE/L0iXzcUxENDBgv9FYDbofoI5zcE/lW0uCz4d0BbJxZymQB/AkKy4MXirstFis9cy06sFAa9uiXzFG0yjzfOZrWRFJlIndLJcqBMCL3g73viaVCoUXquvcb/tUSTNXwDUNIGuyGc4FOWH4P8QoqHMXnZS4jRynotUMm6/0SKoe/oQBWl26rXQ7uLB/wKM9T9wkxWh1LT6tIXJukHXat6L7p4lD/6fqJSP0pOIUHHNAmsc14qjbSmscysQohCo6VweKk2bis0NR7ib89QlJAuXwV0AQlxgzgTjn99Xs1z/a4KQBKd25GkQ== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 03/19] vfio-user: implement message receive infrastructure Date: Wed, 25 Jun 2025 20:29:55 +0100 Message-ID: <20250625193012.2316242-4-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: c48ba999-3f96-41cb-4919-08ddb41ebbf3 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?B?dFdnVG50Rmp2TGRSR2xNQnA1MGtOYnBSWVRrVFFJeDM5QWdXcC9YU29aeE83?= =?utf-8?B?b1NwdTVSbjh0bks1YlFQSXoxUHZNYVlmVTVRV01iOE1udkNYL0dLMWtydHJq?= =?utf-8?B?RWp4dFRNR1R3SGhHRU13VVJVMHQ3OEJ5V1E2WS9rUytGZ2lULzNMY21acE0v?= =?utf-8?B?bklEM3JjSXVtL3VYRW9KYWxwdks2Y1BYSDhscVZGSnJna1RKSzN1ekJuVEkr?= =?utf-8?B?MXVuWHp4MW5KNVZNL0JQT0hjMUI0bFkzVk1qalJCaEFTbFVaVmNGRHVhMUhF?= =?utf-8?B?RXNUM3p0d3ZZa2luRTRsTmRaYnpYZUcybEV5L25xaUNpVm80aFZEWTFwT0pq?= =?utf-8?B?alFUVFF1RUZCdEQydHdXN0dyU0xlb0wycjdZM2ZaYXpBZmtJNVJLSFNBZmJh?= =?utf-8?B?U3o5aW43UGk1YTJEY3E2aXJ0Qk42Q2VSVXJqdjV3L3d4Zmo2ZndrbW91S2tP?= =?utf-8?B?QlE2a05wSzJkWkNaMUorbDFlTUFhYy8xY3ptZldhU2M3SnMzcE54b1ptZEY4?= =?utf-8?B?TE9tUHZoQlhVOVcrSkh0bTRXdFA3emxZamJhY3J6eHVrUlRTUjJES2kyR0Ew?= =?utf-8?B?b0Ziajk4aTZtR1VXcjAvMWsvWFlyU0JUSGI0OUVuL1o0NU9VUmpsNkFldWZQ?= =?utf-8?B?cFJIbmlpNmVPbFhVamVzZEZZYTI0YnpJVVAwbWU3a045Wjg3NXhNMzFvL0Y5?= =?utf-8?B?NkprczA5WE5qWGZTMnVHZWhDazljMGUrODRiSGhnaGFPcGNzOVRDcDN4RWNu?= =?utf-8?B?UVE2RWhQSisvQ0Z2ZUY3b2xCVzdMcDRBQUNFdkVoUThQNnp2NHV5bGNtVjNz?= =?utf-8?B?MlZuWnlPSGg1OG1FaG5aeWZLMjFuSVlPL1ZNZmFUOXFSYVFKVkNyRjJIZExZ?= =?utf-8?B?TkRnL2RtcHg0dVhqdTdEMDRFdDBGSUo0cElQdGViUlpVdThvaW1VYXR5azVF?= =?utf-8?B?QlZKNlJsc0I2QUp1QzllRnhacC9rQ3hmMDhOckZnZ2d5RWp3YSs2dENGaHU2?= =?utf-8?B?RXdXOVNNWTAwVDd3R0hzbEZKQk8xSTdvdit3bXpjTUNaRndUdVFVQnkwcVRW?= =?utf-8?B?YzlwREV6WVBWQy8rUEl2TkJJS0xjNGdHSzdkN0h0T2t1WFhsVW1LdFVuQk9B?= =?utf-8?B?amhzaXhTdUcwVitEcGZiWVAwM0xjRUUyS3B5blNkNFdBbFBxUjhPSVBRNjFG?= =?utf-8?B?NEY1NWhZRzFqTzdCdzBNVEZjamNreUdIejZEc3EzOC81WDF2RU8vWHBOaHFu?= =?utf-8?B?MHY5SjlRMzRMKytBbHREeEw2LyttMDE5T1dxOWVSdDhQT1lMV3NPRkIrRXNj?= =?utf-8?B?MXo5RFgxY3JLQkU5WFd6Ty9CSXRNcnZVR0UxTURqUjZ1eU5SOWJJeUViNVdk?= =?utf-8?B?ckMrSG02VlVHVGh3dlBJOStpSjZPbFlNZ1Q5ZVo2YkdTbUg5ak9vVkROc1hU?= =?utf-8?B?UFJqWHJRazB0aUM0N2pPNG5yWU5MWnNyWCtpSnBnOVdHa1NZVXpwcDh6bkVq?= =?utf-8?B?dE1DWkN6MEh3TlkyNWtUTDFhSllEbVhQQk1ZQ2RMaHdVK1NhY2xEMDRRS0xr?= =?utf-8?B?dUhXM1lzMDZUTmpLR3RNc2o2OVZJYjVJU2FZYm5xRGNjU0txdFl1SVdOaW5C?= =?utf-8?B?ZEo2N1VIK3QzT2pMWEZ3RUpmR2RVQ3VsMU0xZERObk5CSHNJUEFCSzBiTHVL?= =?utf-8?B?Sm92SEhSVm9nSmtwZHdwRFlkeXRyVGFwN0NNeEh5TUtUWlJCNGk5elpSR1pj?= =?utf-8?B?b3ljU3JTeEd0eTdNYk1JdjlDdFJEMTVFdmxvc21sZG1vV3dseFRCVmpLWDNZ?= =?utf-8?B?Z0FJZEVSaFlldkJSWlFBNU9ZeWtBUXJMMVBGMERVbE1EbjY1WktXazVPa054?= =?utf-8?B?UFRLbGJEa2w0TldNWTFCR0pRcG1DdEhrY29ISFBtOUZ3VXc9PQ==?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RGVocVdBaTFxTmgrMmk5WDcvMDk1cmdPSm1aYW5DZWZzSVFsbjFLeXVmNllO?= =?utf-8?B?Q010VEFTbWx1b2pBM0ZKNWFrMVlaYjVmbnRlQWdDdEdRMlAzUWVIY0lWbXI0?= =?utf-8?B?eEFuVWdUbFFibnBhaU90eEQwbWR4UFhCN0tNdzhaRC8wYlo0bW9jcytHSTQ5?= =?utf-8?B?UC83QVBxR0R1bkQ0Y004Wk1UZTNMbDJTc3RzYnJKSkVLcW1xQnJpOXhBSzI2?= =?utf-8?B?c09FVVdFNE5USER3ZHJlTXJITEhvVloyck96RElxUm1MT0l4YTlOVk9wdXAy?= =?utf-8?B?bUlHbnFzdmdZY0FHM3BqNlFNeFY1c01PaFd2USs5ZzMwY3R1cVJVN0VRT0Nm?= =?utf-8?B?YUk3M3g1YVBzNi82VEZJVGN3Y1NzMk5meXQ3MWFGd2NaaDdRRy9tMjY0Qlc3?= =?utf-8?B?UWJJb3EzZWJYRkJvTnArMmp4MzRTODVuWlVGTkNXaEtwbTVNcllkSG13Z1M4?= =?utf-8?B?eTdEeDNtcU83U3ErOG1zcU1Ec3Jmc1MxVm1mdUFuaDRUU1J1NStIWTFVKzJT?= =?utf-8?B?T1VYdkJEY1dZUUFrQnhVSUpzN0ZGRjhjZnc2cHFnbkhTekUxdWxEbGxNSXZk?= =?utf-8?B?ZHFpd2RWL1gzMnBsRkNwcmZSMm1PTmRPMy9XOEt2eGVsTWx1cGdGQyt4cG5H?= =?utf-8?B?K0k2ODIxaDZPcjBEYW9PWG5qcmZTSExSVFZ4ek9pdzRneDRQYTJhbzJCWFpF?= =?utf-8?B?K2hoZHA0OWtvYWJmbDVOUjUvNnh2SThVMkNuRUlDWDlPd1k5K2wrTEJ3ZXRV?= =?utf-8?B?N1hVazUvTEdhd1kzSXhIZ2prdjZNZTd3UDZQNU00SnMrbFFmR2xvcjNndU1L?= =?utf-8?B?YnMwKzhwNEJLRnNJd0NUODlyUHdOb0VrZEdzQjVmMWVFYkV6R2g2RERqb3Ez?= =?utf-8?B?eHhjeTYyaEI0eDM0Y2g1RUxlNjR1bXMzbUwvWHJNTWx0enFBYWR2eDNhTVQ1?= =?utf-8?B?SmQvSXdHRElZS1lMOE1OckluTEd5aC96QXZOTUJUZG9GaHlqeERDOTc4M3VE?= =?utf-8?B?UVVxcjUxNEtwaTQwT1p4TDZQTkJYMnBLcmlXQTVhekJBdDZSZTUxM253UFVZ?= =?utf-8?B?K25GU2Z5SFNmR00wSi9oNHBUQW8wRWgwdmRxdU1laWJ6SXVheUhHZ3VpRVp6?= =?utf-8?B?M0lGdmRpWkZnYU8zZTUxMEI3eWZjTXJFSGkvZHlQRWZnMmJ2VmVpNVIzY3FE?= =?utf-8?B?bXdlR3doYWRJNDFRL0FjbVFwLytJa1l5cDNzSEZqOFlPcTdPSkJVUzRsbnA5?= =?utf-8?B?ckxQYitsL0czM1hTS1hRWkEvRGxHVkxWR2xJYXBZUk1oT1JFUXhzZk5VWTNH?= =?utf-8?B?UmN5QkFidllWSHZFaVJTQzhLb2hDbGFMTVZHTWdubmpKUEZ4WTNVdlQ4TFVw?= =?utf-8?B?K2YwNTJFZnk3M2FuSDZOVjZHYjlUdEkwY01wWjZCR1VoOEl2SnM3RFlqYTdE?= =?utf-8?B?Umk2Nm1aQWx3a2VvRWx4UlJrT1daUUFDWUt0L1drTlhhOWxoWTFkU1VqOE5i?= =?utf-8?B?Z3lCS1JWVkJSM0FGVnNRN0JKM3dZUEJPQ2oraUZpek9rMDlRR3AvMVhQVSt5?= =?utf-8?B?ZmRzMkFIcmFBY0xyZXl6Tmc4cWtZNUtHRW40QjRBcVNUbVF4NCtzM1pZWlNB?= =?utf-8?B?c0NDRTRrOVZnYTI1Zy9UY0ZQbGZrM3pFb0RiaU04MThvSEdxd3BiU3ViajYz?= =?utf-8?B?WVhjYkhITktkRlVxdE4vSzFZZEpzZGw2cmdPcEc0RnlhZEsxc2NGelYxeXp3?= =?utf-8?B?bnRyZ2ROeXBkVkFLMTBRQUpRUmRmdG1TUSt6SWxzMkZJSkgwMWJDQ3ZXWlBU?= =?utf-8?B?Y2FqZk04NVoxMm1BSWRIOXRSdUZkcGtsbURlZHJzNFBFOVZ4ekRERmxRclVl?= =?utf-8?B?KzI5TERpWW1ER1VTM09vVzd1M2YwcUhKR08yeWFkTHBRKzkzZUFSNXNpNlB4?= =?utf-8?B?QnBqT3NxbkpSZGxVWU0zbE1KSi9ZV2s5eDZOUWdBRFcwZnRQUHcva0s2dDU4?= =?utf-8?B?UjRzY09Sb01xeU1DcnFESnR2SWpjS1h6YTE5VGMySkxxeWZPNERXa1czMjJ4?= =?utf-8?B?aEVFT2lRQXNRc0w4VDBaZy9sUzBwNVJmWkE1YTVNbDZXRy9XaXB0WnhLZ2dV?= =?utf-8?Q?4VnVQJhG5YwVuY3liSqM//Y65?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: c48ba999-3f96-41cb-4919-08ddb41ebbf3 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:25.5047 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: h8fQONLugRwqjKXGjISb59biW/6ENmCINQcMKpeK02EaGqMDdJokFgXWM0u6qr8PmA/fYZBi1KOHP/m+9hisvg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Authority-Analysis: v=2.4 cv=fPo53Yae c=1 sm=1 tr=0 ts=685c4e53 cx=c_pps a=5QP2e45hYM+ySLztt10wLA==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=vXCtUE0a-urccVGwgxAA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: hKfbrgoDFGcNo9hNAd1RJdG_HFGQDYX2 X-Proofpoint-GUID: hKfbrgoDFGcNo9hNAd1RJdG_HFGQDYX2 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfXzPiSSi1p9WQD bb4a/pWbvjWmy9Rs9GtP50PonMd1s970oVTIveV5SCrJlDYqZS2BorpATE+fhNvFGENrlxP1Mfv RnGI6y1Nas9AzPIkcNZk1kFS8LqUoetcAFA8nhA2ipdMF7knkNHa25UnKxxVIJNEFMxlLDjaml6 8+3yKT7SsN8ghg6ImAEPAuuVdnCmfn9MHOfR9ETh9NXWXD1cKfAD1bcDNecamyeBP3js44UxKLB MDRKOtBfBO0tkPekkk2+tZbO2dkKJE8aiovQhNJEoZFwGvT0Qcxpm0rHHHcfHfBNNZ4C0paXWRW 0XOwmkmq7/L1l1BbZq3CFORCcnF4JIvjx5Rqwdq3G+qgFZQHHQo2YQSW07Jbk5BSz7oQS7PQy9S 1vL56v6duKEFLzTaQ1xgQIYtkJvTXynAAnQIuFg5gx7nMYWoiPi6ZKVQkHAEjP7ZzkjgJ5ix X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750879961186116600 Add the basic implementation for receiving vfio-user messages from the control socket. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- meson.build | 1 + hw/vfio-user/protocol.h | 53 +++++ hw/vfio-user/proxy.h | 11 + hw/vfio-user/trace.h | 4 + hw/vfio-user/pci.c | 11 + hw/vfio-user/proxy.c | 411 ++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 8 + 7 files changed, 499 insertions(+) create mode 100644 hw/vfio-user/protocol.h create mode 100644 hw/vfio-user/trace.h create mode 100644 hw/vfio-user/trace-events diff --git a/meson.build b/meson.build index 4676908dbb..dbc97bfdf7 100644 --- a/meson.build +++ b/meson.build @@ -3683,6 +3683,7 @@ if have_system 'hw/ufs', 'hw/usb', 'hw/vfio', + 'hw/vfio-user', 'hw/virtio', 'hw/vmapple', 'hw/watchdog', diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h new file mode 100644 index 0000000000..4ddfb5f222 --- /dev/null +++ b/hw/vfio-user/protocol.h @@ -0,0 +1,53 @@ +#ifndef VFIO_USER_PROTOCOL_H +#define VFIO_USER_PROTOCOL_H + +/* + * vfio protocol over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * Each message has a standard header that describes the command + * being sent, which is almost always a VFIO ioctl(). + * + * The header may be followed by command-specific data, such as the + * region and offset info for read and write commands. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +typedef struct { + uint16_t id; + uint16_t command; + uint32_t size; + uint32_t flags; + uint32_t error_reply; +} VFIOUserHdr; + +/* VFIOUserHdr commands */ +enum vfio_user_command { + VFIO_USER_VERSION =3D 1, + VFIO_USER_DMA_MAP =3D 2, + VFIO_USER_DMA_UNMAP =3D 3, + VFIO_USER_DEVICE_GET_INFO =3D 4, + VFIO_USER_DEVICE_GET_REGION_INFO =3D 5, + VFIO_USER_DEVICE_GET_REGION_IO_FDS =3D 6, + VFIO_USER_DEVICE_GET_IRQ_INFO =3D 7, + VFIO_USER_DEVICE_SET_IRQS =3D 8, + VFIO_USER_REGION_READ =3D 9, + VFIO_USER_REGION_WRITE =3D 10, + VFIO_USER_DMA_READ =3D 11, + VFIO_USER_DMA_WRITE =3D 12, + VFIO_USER_DEVICE_RESET =3D 13, + VFIO_USER_DIRTY_PAGES =3D 14, + VFIO_USER_MAX, +}; + +/* VFIOUserHdr flags */ +#define VFIO_USER_REQUEST 0x0 +#define VFIO_USER_REPLY 0x1 +#define VFIO_USER_TYPE 0xF + +#define VFIO_USER_NO_REPLY 0x10 +#define VFIO_USER_ERROR 0x20 + +#endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index a9bce82239..ff553cad9d 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -12,6 +12,9 @@ #include "io/channel.h" #include "io/channel-socket.h" =20 +#include "qemu/sockets.h" +#include "hw/vfio-user/protocol.h" + typedef struct { int send_fds; int recv_fds; @@ -28,6 +31,7 @@ enum msg_type { =20 typedef struct VFIOUserMsg { QTAILQ_ENTRY(VFIOUserMsg) next; + VFIOUserHdr *hdr; VFIOUserFDs *fds; uint32_t rsize; uint32_t id; @@ -67,13 +71,20 @@ typedef struct VFIOUserProxy { VFIOUserMsgQ incoming; VFIOUserMsgQ outgoing; VFIOUserMsg *last_nowait; + VFIOUserMsg *part_recv; + size_t recv_left; enum proxy_state state; } VFIOUserProxy; =20 /* VFIOProxy flags */ #define VFIO_PROXY_CLIENT 0x1 =20 +typedef struct VFIODevice VFIODevice; + VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp); void vfio_user_disconnect(VFIOUserProxy *proxy); +void vfio_user_set_handler(VFIODevice *vbasedev, + void (*handler)(void *opaque, VFIOUserMsg *msg), + void *reqarg); =20 #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/trace.h b/hw/vfio-user/trace.h new file mode 100644 index 0000000000..9cf02d9506 --- /dev/null +++ b/hw/vfio-user/trace.h @@ -0,0 +1,4 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + */ +#include "trace/trace-hw_vfio_user.h" diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 642421e791..bad2829f5c 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -22,6 +22,16 @@ struct VFIOUserPCIDevice { SocketAddress *socket; }; =20 +/* + * Incoming request message callback. + * + * Runs off main loop, so BQL held. + */ +static void vfio_user_pci_process_req(void *opaque, VFIOUserMsg *msg) +{ + +} + /* * Emulated devices don't use host hot reset */ @@ -80,6 +90,7 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error = **errp) return; } vbasedev->proxy =3D proxy; + vfio_user_set_handler(vbasedev, vfio_user_pci_process_req, vdev); =20 /* * vfio-user devices are effectively mdevs (don't use a host iommu). diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index bb436c9db9..349ea2b27c 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -11,15 +11,31 @@ =20 #include "hw/vfio/vfio-device.h" #include "hw/vfio-user/proxy.h" +#include "hw/vfio-user/trace.h" #include "qapi/error.h" #include "qemu/error-report.h" #include "qemu/lockable.h" +#include "qemu/main-loop.h" #include "system/iothread.h" =20 static IOThread *vfio_user_iothread; =20 static void vfio_user_shutdown(VFIOUserProxy *proxy); +static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, + VFIOUserFDs *fds); +static VFIOUserFDs *vfio_user_getfds(int numfds); +static void vfio_user_recycle(VFIOUserProxy *proxy, VFIOUserMsg *msg); =20 +static void vfio_user_recv(void *opaque); +static void vfio_user_cb(void *opaque); + +static void vfio_user_request(void *opaque); + +static inline void vfio_user_set_error(VFIOUserHdr *hdr, uint32_t err) +{ + hdr->flags |=3D VFIO_USER_ERROR; + hdr->error_reply =3D err; +} =20 /* * Functions called by main, CPU, or iothread threads @@ -32,10 +48,343 @@ static void vfio_user_shutdown(VFIOUserProxy *proxy) proxy->ctx, NULL, NULL); } =20 +static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, + VFIOUserFDs *fds) +{ + VFIOUserMsg *msg; + + msg =3D QTAILQ_FIRST(&proxy->free); + if (msg !=3D NULL) { + QTAILQ_REMOVE(&proxy->free, msg, next); + } else { + msg =3D g_malloc0(sizeof(*msg)); + qemu_cond_init(&msg->cv); + } + + msg->hdr =3D hdr; + msg->fds =3D fds; + return msg; +} + +/* + * Recycle a message list entry to the free list. + */ +static void vfio_user_recycle(VFIOUserProxy *proxy, VFIOUserMsg *msg) +{ + if (msg->type =3D=3D VFIO_MSG_NONE) { + error_printf("vfio_user_recycle - freeing free msg\n"); + return; + } + + /* free msg buffer if no one is waiting to consume the reply */ + if (msg->type =3D=3D VFIO_MSG_NOWAIT || msg->type =3D=3D VFIO_MSG_ASYN= C) { + g_free(msg->hdr); + if (msg->fds !=3D NULL) { + g_free(msg->fds); + } + } + + msg->type =3D VFIO_MSG_NONE; + msg->hdr =3D NULL; + msg->fds =3D NULL; + msg->complete =3D false; + QTAILQ_INSERT_HEAD(&proxy->free, msg, next); +} + +static VFIOUserFDs *vfio_user_getfds(int numfds) +{ + VFIOUserFDs *fds =3D g_malloc0(sizeof(*fds) + (numfds * sizeof(int))); + + fds->fds =3D (int *)((char *)fds + sizeof(*fds)); + + return fds; +} + /* * Functions only called by iothread */ =20 +/* + * Process a received message. + */ +static void vfio_user_process(VFIOUserProxy *proxy, VFIOUserMsg *msg, + bool isreply) +{ + + /* + * Replies signal a waiter, if none just check for errors + * and free the message buffer. + * + * Requests get queued for the BH. + */ + if (isreply) { + msg->complete =3D true; + if (msg->type =3D=3D VFIO_MSG_WAIT) { + qemu_cond_signal(&msg->cv); + } else { + if (msg->hdr->flags & VFIO_USER_ERROR) { + error_printf("vfio_user_process: error reply on async "); + error_printf("request command %x error %s\n", + msg->hdr->command, + strerror(msg->hdr->error_reply)); + } + /* youngest nowait msg has been ack'd */ + if (proxy->last_nowait =3D=3D msg) { + proxy->last_nowait =3D NULL; + } + vfio_user_recycle(proxy, msg); + } + } else { + QTAILQ_INSERT_TAIL(&proxy->incoming, msg, next); + qemu_bh_schedule(proxy->req_bh); + } +} + +/* + * Complete a partial message read + */ +static int vfio_user_complete(VFIOUserProxy *proxy, Error **errp) +{ + VFIOUserMsg *msg =3D proxy->part_recv; + size_t msgleft =3D proxy->recv_left; + bool isreply; + char *data; + int ret; + + data =3D (char *)msg->hdr + (msg->hdr->size - msgleft); + while (msgleft > 0) { + ret =3D qio_channel_read(proxy->ioc, data, msgleft, errp); + + /* error or would block */ + if (ret <=3D 0) { + /* try for rest on next iternation */ + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + proxy->recv_left =3D msgleft; + } + return ret; + } + trace_vfio_user_recv_read(msg->hdr->id, ret); + + msgleft -=3D ret; + data +=3D ret; + } + + /* + * Read complete message, process it. + */ + proxy->part_recv =3D NULL; + proxy->recv_left =3D 0; + isreply =3D (msg->hdr->flags & VFIO_USER_TYPE) =3D=3D VFIO_USER_REPLY; + vfio_user_process(proxy, msg, isreply); + + /* return positive value */ + return 1; +} + +/* + * Receive and process one incoming message. + * + * For replies, find matching outgoing request and wake any waiters. + * For requests, queue in incoming list and run request BH. + */ +static int vfio_user_recv_one(VFIOUserProxy *proxy, Error **errp) +{ + VFIOUserMsg *msg =3D NULL; + g_autofree int *fdp =3D NULL; + VFIOUserFDs *reqfds; + VFIOUserHdr hdr; + struct iovec iov =3D { + .iov_base =3D &hdr, + .iov_len =3D sizeof(hdr), + }; + bool isreply =3D false; + int i, ret; + size_t msgleft, numfds =3D 0; + char *data =3D NULL; + char *buf =3D NULL; + + /* + * Complete any partial reads + */ + if (proxy->part_recv !=3D NULL) { + ret =3D vfio_user_complete(proxy, errp); + + /* still not complete, try later */ + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + return ret; + } + + if (ret <=3D 0) { + goto fatal; + } + /* else fall into reading another msg */ + } + + /* + * Read header + */ + ret =3D qio_channel_readv_full(proxy->ioc, &iov, 1, &fdp, &numfds, 0, + errp); + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + return ret; + } + + /* read error or other side closed connection */ + if (ret <=3D 0) { + goto fatal; + } + + if (ret < sizeof(hdr)) { + error_setg(errp, "short read of header"); + goto fatal; + } + + /* + * Validate header + */ + if (hdr.size < sizeof(VFIOUserHdr)) { + error_setg(errp, "bad header size"); + goto fatal; + } + switch (hdr.flags & VFIO_USER_TYPE) { + case VFIO_USER_REQUEST: + isreply =3D false; + break; + case VFIO_USER_REPLY: + isreply =3D true; + break; + default: + error_setg(errp, "unknown message type"); + goto fatal; + } + trace_vfio_user_recv_hdr(proxy->sockname, hdr.id, hdr.command, hdr.siz= e, + hdr.flags); + + /* + * For replies, find the matching pending request. + * For requests, reap incoming FDs. + */ + if (isreply) { + QTAILQ_FOREACH(msg, &proxy->pending, next) { + if (hdr.id =3D=3D msg->id) { + break; + } + } + if (msg =3D=3D NULL) { + error_setg(errp, "unexpected reply"); + goto err; + } + QTAILQ_REMOVE(&proxy->pending, msg, next); + + /* + * Process any received FDs + */ + if (numfds !=3D 0) { + if (msg->fds =3D=3D NULL || msg->fds->recv_fds < numfds) { + error_setg(errp, "unexpected FDs"); + goto err; + } + msg->fds->recv_fds =3D numfds; + memcpy(msg->fds->fds, fdp, numfds * sizeof(int)); + } + } else { + if (numfds !=3D 0) { + reqfds =3D vfio_user_getfds(numfds); + memcpy(reqfds->fds, fdp, numfds * sizeof(int)); + } else { + reqfds =3D NULL; + } + } + + /* + * Put the whole message into a single buffer. + */ + if (isreply) { + if (hdr.size > msg->rsize) { + error_setg(errp, "reply larger than recv buffer"); + goto err; + } + *msg->hdr =3D hdr; + data =3D (char *)msg->hdr + sizeof(hdr); + } else { + buf =3D g_malloc0(hdr.size); + memcpy(buf, &hdr, sizeof(hdr)); + data =3D buf + sizeof(hdr); + msg =3D vfio_user_getmsg(proxy, (VFIOUserHdr *)buf, reqfds); + msg->type =3D VFIO_MSG_REQ; + } + + /* + * Read rest of message. + */ + msgleft =3D hdr.size - sizeof(hdr); + while (msgleft > 0) { + ret =3D qio_channel_read(proxy->ioc, data, msgleft, errp); + + /* prepare to complete read on next iternation */ + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + proxy->part_recv =3D msg; + proxy->recv_left =3D msgleft; + return ret; + } + + if (ret <=3D 0) { + goto fatal; + } + trace_vfio_user_recv_read(hdr.id, ret); + + msgleft -=3D ret; + data +=3D ret; + } + + vfio_user_process(proxy, msg, isreply); + return 0; + + /* + * fatal means the other side closed or we don't trust the stream + * err means this message is corrupt + */ +fatal: + vfio_user_shutdown(proxy); + proxy->state =3D VFIO_PROXY_ERROR; + + /* set error if server side closed */ + if (ret =3D=3D 0) { + error_setg(errp, "server closed socket"); + } + +err: + for (i =3D 0; i < numfds; i++) { + close(fdp[i]); + } + if (isreply && msg !=3D NULL) { + /* force an error to keep sending thread from hanging */ + vfio_user_set_error(msg->hdr, EINVAL); + msg->complete =3D true; + qemu_cond_signal(&msg->cv); + } + return -1; +} + +static void vfio_user_recv(void *opaque) +{ + VFIOUserProxy *proxy =3D opaque; + + QEMU_LOCK_GUARD(&proxy->lock); + + if (proxy->state =3D=3D VFIO_PROXY_CONNECTED) { + Error *local_err =3D NULL; + + while (vfio_user_recv_one(proxy, &local_err) =3D=3D 0) { + ; + } + + if (local_err !=3D NULL) { + error_report_err(local_err); + } + } +} + static void vfio_user_cb(void *opaque) { VFIOUserProxy *proxy =3D opaque; @@ -51,6 +400,53 @@ static void vfio_user_cb(void *opaque) * Functions called by main or CPU threads */ =20 +/* + * Process incoming requests. + * + * The bus-specific callback has the form: + * request(opaque, msg) + * where 'opaque' was specified in vfio_user_set_handler + * and 'msg' is the inbound message. + * + * The callback is responsible for disposing of the message buffer, + * usually by re-using it when calling vfio_send_reply or vfio_send_error, + * both of which free their message buffer when the reply is sent. + * + * If the callback uses a new buffer, it needs to free the old one. + */ +static void vfio_user_request(void *opaque) +{ + VFIOUserProxy *proxy =3D opaque; + VFIOUserMsgQ new, free; + VFIOUserMsg *msg, *m1; + + /* reap all incoming */ + QTAILQ_INIT(&new); + WITH_QEMU_LOCK_GUARD(&proxy->lock) { + QTAILQ_FOREACH_SAFE(msg, &proxy->incoming, next, m1) { + QTAILQ_REMOVE(&proxy->incoming, msg, next); + QTAILQ_INSERT_TAIL(&new, msg, next); + } + } + + /* process list */ + QTAILQ_INIT(&free); + QTAILQ_FOREACH_SAFE(msg, &new, next, m1) { + QTAILQ_REMOVE(&new, msg, next); + trace_vfio_user_recv_request(msg->hdr->command); + proxy->request(proxy->req_arg, msg); + QTAILQ_INSERT_HEAD(&free, msg, next); + } + + /* free list */ + WITH_QEMU_LOCK_GUARD(&proxy->lock) { + QTAILQ_FOREACH_SAFE(msg, &free, next, m1) { + vfio_user_recycle(proxy, msg); + } + } +} + + static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); =20 @@ -89,6 +485,7 @@ VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr= , Error **errp) } =20 proxy->ctx =3D iothread_get_aio_context(vfio_user_iothread); + proxy->req_bh =3D qemu_bh_new(vfio_user_request, proxy); =20 QTAILQ_INIT(&proxy->outgoing); QTAILQ_INIT(&proxy->incoming); @@ -99,6 +496,18 @@ VFIOUserProxy *vfio_user_connect_dev(SocketAddress *add= r, Error **errp) return proxy; } =20 +void vfio_user_set_handler(VFIODevice *vbasedev, + void (*handler)(void *opaque, VFIOUserMsg *msg), + void *req_arg) +{ + VFIOUserProxy *proxy =3D vbasedev->proxy; + + proxy->request =3D handler; + proxy->req_arg =3D req_arg; + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, + vfio_user_recv, NULL, NULL, proxy); +} + void vfio_user_disconnect(VFIOUserProxy *proxy) { VFIOUserMsg *r1, *r2; @@ -114,6 +523,8 @@ void vfio_user_disconnect(VFIOUserProxy *proxy) } object_unref(OBJECT(proxy->ioc)); proxy->ioc =3D NULL; + qemu_bh_delete(proxy->req_bh); + proxy->req_bh =3D NULL; =20 proxy->state =3D VFIO_PROXY_CLOSING; QTAILQ_FOREACH_SAFE(r1, &proxy->outgoing, next, r2) { diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events new file mode 100644 index 0000000000..ddeb9f4b2f --- /dev/null +++ b/hw/vfio-user/trace-events @@ -0,0 +1,8 @@ +# See docs/devel/tracing.rst for syntax documentation. +# +# SPDX-License-Identifier: GPL-2.0-or-later + +# common.c +vfio_user_recv_hdr(const char *name, uint16_t id, uint16_t cmd, uint32_t s= ize, uint32_t flags) " (%s) id 0x%x cmd 0x%x size 0x%x flags 0x%x" +vfio_user_recv_read(uint16_t id, int read) " id 0x%x read 0x%x" +vfio_user_recv_request(uint16_t cmd) " command 0x%x" --=20 2.43.0 From nobody Sat Nov 15 14:11:13 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750879991; cv=pass; d=zohomail.com; s=zohoarc; b=Xt92cQv3825xIUtlS1q+SQAVhjQfDvIf6h45K77CYlLbTkmb2bX1fEqd4CYnsywPFYm73KADFf5XiEbvVg4enSabhJbTIDuIwxN/S89XyWSRmoc/8/M3j9ux0i6GdCNx8xEu1j9m+gHclnSu7xiJinLMDusm+2sHG6DYcmJhJDw= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750879991; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=QsoMfZATuoMJ/ARP2DHdVNTu1gZQzjF59jCorR5dIfA=; b=WRiOJxIT+Aje8iTU+8ASbK4EtTNenUvZvwFBqCaUFSVBw4xXWcwa+pAYtxJ8VH5l9xl1rqmG/4/1Bk71rKWqfN7cblMRRyVKewZyFFbaWrCj+0PLlImjvT3LAN4oTut9VA//F5ZYtygAyYho4BKKGsN3rzkVuY7kV4XtpEBSONg= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750879991501780.8056970567121; Wed, 25 Jun 2025 12:33:11 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpd-00082A-7m; Wed, 25 Jun 2025 15:30:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpL-00080S-US for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:35 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpI-0003Uj-Vl for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:35 -0400 Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PDssmc025796; Wed, 25 Jun 2025 12:30:31 -0700 Received: from nam04-dm6-obe.outbound.protection.outlook.com (mail-dm6nam04on2116.outbound.protection.outlook.com [40.107.102.116]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47gjdtrsa3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:30 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:28 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:28 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=QsoMfZATuoMJ/ARP2DHdVNTu1gZQzjF59jCorR5dI fA=; b=xDP0Yx3HTeURJiKzqs7vjrPx4SVhIVbeOouy7xnQM1/oFf1b0HU/961w2 itlRNcXHabPqiuaheI/6pLXAgT17jFz+zMgaJTbrA/2gOQKn7vlUXAuYeI7ldE7G xKP7jYJWZsP2X/Bz71QuSMCQ46JfKYd9cNVrHryprQIUx95/KQ07sHimqEH2YX6E czFglvdWKCtwlucY4LaQ4ehnoNDp42FCLfUqTciYE/L5/FChsGgU9BHCGfaSfzpc s/yBKw2SPkgEbIjehLnK5Ip5phm1H/Z0lc88EJI1EceQHtzrUbylj9USmn8TcPx2 wl72+3fldT/N30zkGYD7coDV2Jf1w== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gSaCeuzds2INWzdM2xV744Dt+gzhnz7yBGw20sh02Z6YTkoRxB8WWHCifA4PHC9tmqgVKwt9AIMiT2ac49USEQyj3Cn7yKADjiCcBtGqiUMB8YqSZin/9vsUgTfc26NKAT2RUq7aTP5q5qE8RKltGDX1akAXwnP28tNCbgMbnQvbr2TPr7ng0gyYjgZZtbYdmac6xCNRM3npec3BvYr52J7zhSSs/vslxKucrhOKCfl6AIktuVn2Xj1A6IdJSMEV6aNF638cMDRJySRot+jrhj3kuwPPfIqScrLONyScbrDvRy6zdaZZ3vCwdEqIBpR7zgRsi9NFG8Nx3QDp5p8pkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QsoMfZATuoMJ/ARP2DHdVNTu1gZQzjF59jCorR5dIfA=; b=GqPN0RArikmwzb+psoE0+eE1M6VvNlU6/Egfg5hmlrXjJdMH7v6+swJVKQbHbOfjGsn9uAV1k8M8Kn82mLe9eD9G2h2kziT2RwUtxiaffaLZzlnqCa64C6pD0qIHlgXcnEJU+9wdVFEelgAAt9vZT0IxjYKx1og4eIaeWJ8BOrnjowRiH0mX2ivpz+npy1DRFmPZRXp5EbUX72l+IyZrBx5RaVU63Jz8w2GEF3iuCRPBgYXZ0kfeFd/dEMFiEjFDOzcyJp0M3Yidb161iYuFKjugnpu10PSPqleAmAHSpoy/HmOr6U/KLpvwIWyKRSr7na/nJHUe9ybLrs4O2I3MqQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QsoMfZATuoMJ/ARP2DHdVNTu1gZQzjF59jCorR5dIfA=; b=S/6lhdAJQthhfvAGR/olymCSWYOTLLLwa62v05AXIg5miFAmhPlow8gQGi5iEJB+QH+QD0+2d5gk5Yp6h3/s0C+9SS11RUx73+ZLK0tgz9JlikY1+lec5guIXeKOfLcVFdH1Z9MA0Hj94KbB98JW0HDYIBK863USoxXspWyybirVQdhmKoqMmcPdbZBv4t5dirae7x6svN0ckyQNrgRHfXyZaGmNNMRF0S2mppri9ymi705xbsuSUZy8HZTNpZ+kMgFkyoKFkRIFg4pMH9wOwEelGwsqEcWTCWcZ18wfcxE0QzuXN5t/kNGvRQzzDyjNgtctQkMqq+EdY9lCM8ZZ9Q== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Jagannathan Raman , Elena Ufimtseva Subject: [PATCH v5 04/19] vfio-user: implement message send infrastructure Date: Wed, 25 Jun 2025 20:29:56 +0100 Message-ID: <20250625193012.2316242-5-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: 63685f7d-b1eb-4017-d193-08ddb41ebd65 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?i6CsnbJpJj8i4QVi1bZsKKobdk5s0PFVgTy4LTJQ37o+k67voYu3xrqBiuPE?= =?us-ascii?Q?jUfmoWhMiq4DVcNVA4Kqu6gSiTrOS0+6lWv0mWIOROt/PPZcq0txgeUgvV/a?= =?us-ascii?Q?nDeuMzONZRnkJBuQ4TcUwXe8Bz1zqOFmif0Viv4gPt3wfVKN9Fk6L607wIFK?= =?us-ascii?Q?Q9Na6HCUxQi33gmuOgKk/fb2YgAX1cEj4OXM++Y2rnUcnzi5pqsWRW3wz+BD?= =?us-ascii?Q?eGGQ8OC6723gO0NE9QIieRt5RMYVILjU4empH6pZJY84wlwO4uZOBXLv3Smn?= =?us-ascii?Q?RtpC0H2TpKZQzjrOsWM3fDE9zgCEADziCPdRAa5WXwiA9nnvmIdBB3vWvDSK?= =?us-ascii?Q?ahFbc/hGxuuFliH1vO4fW50B8Leyc0ZB+bi0t7H4VosHApniZzHR2UsX/zGT?= =?us-ascii?Q?BYl0lHdvPFoKnu20zBKCWRaVgwGXYmfTquKmXU3s3FmUI3JfeiOGXrJZAqD9?= =?us-ascii?Q?goIA7Ah31cCcd5FhfkOfdS2UP109AJDK1THRgc0FxGyne+lpTX57zrBB628N?= =?us-ascii?Q?Um+cmaJ5/4JA5zW72Y2D5dvKWidR5pn6Wt9TioPMUMuf87DXwuPDmTKbRIf9?= =?us-ascii?Q?0OP+TYRj/Q4ZG3sOljK+ryV5/vlr7FBXQ7DbVEo6gAIRUnDO/4b+h2YYhxtN?= =?us-ascii?Q?r25hhDfEZNY16IzQH2rO6S7rm87ZMthFT3eeM3MYIv2Nn4CuxXQg63B++OoP?= =?us-ascii?Q?zTjMk8XQxoE1sUPoUqUWadWBtILg8PTjwKXvpfy432JI0+fIw6M+5NJSWSYd?= =?us-ascii?Q?ZTKoiv/KekMjiLGGKIcIrsdUFT95GU1dttSGNCV/7ormPYAtF9bSGgNl1OgY?= =?us-ascii?Q?KxC3RXhJzPZL+Pr13HD/nUtlqwFOUkAmPsy7JjaFTm9itIKJMo5KgxOSbTOP?= =?us-ascii?Q?Tfsh8/R6Q4RjaCL6tdFo8uHOdc1eTkO+O/6rnIy/4EbQwYtliSzz0+8vBpHk?= =?us-ascii?Q?+xE+UAKqgUqgpTfTuiwD7U1jr5ltBYAfixKUcBTr/2U/3HPJeR/ivxIvY1as?= =?us-ascii?Q?HrGvjQi0TPI901uMXartkACYUlbp9eNUBi4fgFN/jJOxYBb8r5gF8dgk+0u9?= =?us-ascii?Q?SsURa+STa6uYnLmXZqx6DRrESBSknp0JQID+VZ29jezAGE0MXRJ1wl4Juchl?= =?us-ascii?Q?KAjOEMGQAugWUGKFwCBi+fyzpP31P6nAxL2b9BRLVKkA3V9ApSV/0/xNiUJa?= =?us-ascii?Q?LunC2B0BQCXGDkLplWo4lYcKwixhAev/5uxSAlOZ7LPWZI2ywUgamek8Fydv?= =?us-ascii?Q?hyiD37QUMS38rLTkyI/RETGCogbEAgvgTzk7HzQEoDeyTH1mMLeaJc89TLG1?= =?us-ascii?Q?UhMJ/1rQTWN9B5Pq2Lq/+kfzxVhIOZ7eL15Yro5WbHueRwrQJzEAhGT8WGe3?= =?us-ascii?Q?q1EDOsKJnvZiifJHNXlv1Jv98nZnhd1ySB0HRek+uq4oKI/9U9/lauXMKJlo?= =?us-ascii?Q?kFH/6bfn4Do=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?nPhx6O4FjProwiv/H6FiAa0ANe4v6TYCrOp5b4FYL5OhmGCQwo6rr9IhIbhs?= =?us-ascii?Q?zUcyEAc8f+iGS0qMzPYVkzmiWjS+IRzCPhdaOEH6QUeo4z8BdhZjy/83mjQE?= =?us-ascii?Q?dbxERgFj6s/Axnl5soN99uyw1Yj+3tr0eyd4cmkMghJis2WAPfG1wMgas2t8?= =?us-ascii?Q?3//COYVRC6C7o4wsaHx67gS4e58s1SzaarbP+tpwwldZqldUC8u3jhcDaEDh?= =?us-ascii?Q?8SM8q3Rg37HBp12ivuRw+BfUEvuYy0OnWv38o6SIJL+HXe36krh1itPQLfkw?= =?us-ascii?Q?HYIo1omJVPUCYLz1Mj90Kqqkn3Byia4FYOE25BKAO44hls1P0DiXvCX7QAHQ?= =?us-ascii?Q?k/cae/k0St7qsaHR5TW6XboeM+ad4+JlfE6LLKKmt5fcug4dayWxFD3D5Jaz?= =?us-ascii?Q?HJRaT6tXS7xeYQDHV3t+ncHD2olmnxyzAe455ezCezM0uMyhhX2CGVtUNxiK?= =?us-ascii?Q?f4IlFgiHVmDkrzvSbgG4G3GalumZillPovgWBBMcnl1YO39D4ftiYOXWK15y?= =?us-ascii?Q?Kwcmwa8GgXdTkkaUCwzCaF1kAS/03+d3+pCe7KPTE4hDXFCg74e60ztvSfKd?= =?us-ascii?Q?ud1AGUbAZN6615gpRwZ1nDP7r+FxVe7xokxPquDbIoF0cQkivUydTCra24iN?= =?us-ascii?Q?FBBWSeciwUE4gTeux9Bj8Uqh0EcfoBGT1+qJCKqX+qHs8aN+GmSkKG6LMETV?= =?us-ascii?Q?y6/uS686QIYhcNAMnadtqkfX2KNhWdqmzI2YQ+YePQSbkvmys5LVo6f8odVu?= =?us-ascii?Q?HII/a2+MIVErX8vXRrTbDwA3y1aVcfTDliiD/0BEAM2mas4BHq1Y2Ym3w0VQ?= =?us-ascii?Q?jUj3zPcwZmRwf/RWFDVGlB+eAYX+/lhwFbhFCl9T6gjOSH9ktr9YzF97giBG?= =?us-ascii?Q?pV6CRob5COzYOBQfI7T08fvxiH302/RhF3Lt1ySG9AGE5MvZ0Y7RWtM25gix?= =?us-ascii?Q?ahH1JzmDtEU0aUo9lBOdf2aFF4ZNMsNQvDmVs5DRxCkk8NwWgfUzSncNJG2I?= =?us-ascii?Q?XNADDUk8GQ++zTdVgFf7DqjA3esz3n5d+Ri3lVLMq1R0b4R4jAmmG5UQp15Y?= =?us-ascii?Q?+NZoVYtkO0jQ18vD4luDKErcDzyHyF7L/kZjmQ/FamoCq6GDXIJI88TWo/ZH?= =?us-ascii?Q?/rhmata1qatABnzHwYuWH7sgQMw5ClAJyRSsXBeKWdFKOYALbWNfSLRimw2f?= =?us-ascii?Q?f7M6qi6dWF3M+D8S91ogeHDpkcjqIy+GTsrwS0RUa5KjV4r4pHR3FunHCFpL?= =?us-ascii?Q?KOqnqwJUb5E5Lox3xfV/g/XpL9Ve5ELZ4/RBKBb1I1UrXsruQewdfqjGKH52?= =?us-ascii?Q?wWzinfxIl5goiQbO8satYzrolifnss++IwYZIcEVTQ32EY8wJIjju6Gs+/YK?= =?us-ascii?Q?Ar2li55cNQHFwcV8MqRWTVNkZ4BjK5gYD37hxfZje+iZJbX/lV7MODV8ovnR?= =?us-ascii?Q?NvfAKIhvUL8cNqr5VWRsyGZK4mKpbfvVT21s9lg6gW4JFM5dT3QwsdDSMM2y?= =?us-ascii?Q?XK/6dvKMYJ4V130Kpk8hUsIsQlMG5KkDVsD38egzzd+vtPIJ2JMDKwNaOQKm?= =?us-ascii?Q?zcDGwNnBqBg7aEk56XsLaaF6QOI3mgCa/Ige3+dt?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 63685f7d-b1eb-4017-d193-08ddb41ebd65 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:28.0251 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Pa9QoYn+7IC6Vrp7O0sge1QkuCJ613fqbcX12/AHdUWjzW3VjW1uW1Y7piOxBF+ddBn+sv0VB+1HbwwSmeRdYw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Proofpoint-ORIG-GUID: PcRkH1LjiW7OKkOGUF-NblZoHzRg7kSr X-Authority-Analysis: v=2.4 cv=Fcs3xI+6 c=1 sm=1 tr=0 ts=685c4e57 cx=c_pps a=/2wCtPJy3NvKizEtkrRzQQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=UnscCpd6-9sbi9RN3AUA:9 X-Proofpoint-GUID: PcRkH1LjiW7OKkOGUF-NblZoHzRg7kSr X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX4TRZhh2fdu9+ 1VwqJ8N1MnFDwI58zkX8H1MySSyXHRTvI5emzfSYKOv41dmaApxGiN1NTFMaHGIRENJ4dLNBqaY pWAsvMCGZurypccrx7YuElC3rlGZXB+4AaCNF83/JAtGSYfSv+cmrlSy+JIF6UNKkrZn/DIERIO oyLsLYHtz0mDWlrIQLZR3YnlskafgyPiSCtuSZWyKMGzZVLu29ik7eSs3NgisUTggixmGQ1Idue ej8aMkN5G05rIeG3bV1zf7U3PaE1TpIJ9VJ3/p6L71CPDv4ot0MZo8bXYyd6YJLGSDb02J940nZ M/jrkYqGX6nyKsnXhR2i4zu9se/PyWKqKM+Mqm17hP7KZvbX5JXAgSuAN0aC49AoMqhX5wb6Cuc 0UmTeEaktBRGW9+rxO2x73NsR9hw9mggb3tT7RnGc2WoR5XegNt1JxZNyVhUaOwsKcf5GNo3 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750879993582116600 Content-Type: text/plain; charset="utf-8" Add plumbing for sending vfio-user messages on the control socket. Add initial version negotation on connection. Originally-by: John Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/protocol.h | 62 +++++ hw/vfio-user/proxy.h | 9 + hw/vfio-user/pci.c | 20 +- hw/vfio-user/proxy.c | 515 ++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 2 + 5 files changed, 606 insertions(+), 2 deletions(-) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 4ddfb5f222..2d52d0fb10 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -50,4 +50,66 @@ enum vfio_user_command { #define VFIO_USER_NO_REPLY 0x10 #define VFIO_USER_ERROR 0x20 =20 + +/* + * VFIO_USER_VERSION + */ +typedef struct { + VFIOUserHdr hdr; + uint16_t major; + uint16_t minor; + char capabilities[]; +} VFIOUserVersion; + +#define VFIO_USER_MAJOR_VER 0 +#define VFIO_USER_MINOR_VER 0 + +#define VFIO_USER_CAP "capabilities" + +/* "capabilities" members */ +#define VFIO_USER_CAP_MAX_FDS "max_msg_fds" +#define VFIO_USER_CAP_MAX_XFER "max_data_xfer_size" +#define VFIO_USER_CAP_PGSIZES "pgsizes" +#define VFIO_USER_CAP_MAP_MAX "max_dma_maps" +#define VFIO_USER_CAP_MIGR "migration" + +/* "migration" members */ +#define VFIO_USER_CAP_PGSIZE "pgsize" +#define VFIO_USER_CAP_MAX_BITMAP "max_bitmap_size" + +/* + * Max FDs mainly comes into play when a device supports multiple interrup= ts + * where each ones uses an eventfd to inject it into the guest. + * It is clamped by the the number of FDs the qio channel supports in a + * single message. + */ +#define VFIO_USER_DEF_MAX_FDS 8 +#define VFIO_USER_MAX_MAX_FDS 16 + +/* + * Max transfer limits the amount of data in region and DMA messages. + * Region R/W will be very small (limited by how much a single instruction + * can process) so just use a reasonable limit here. + */ +#define VFIO_USER_DEF_MAX_XFER (1024 * 1024) +#define VFIO_USER_MAX_MAX_XFER (64 * 1024 * 1024) + +/* + * Default pagesizes supported is 4k. + */ +#define VFIO_USER_DEF_PGSIZE 4096 + +/* + * Default max number of DMA mappings is stolen from the + * linux kernel "dma_entry_limit" + */ +#define VFIO_USER_DEF_MAP_MAX 65535 + +/* + * Default max bitmap size is also take from the linux kernel, + * where usage of signed ints limits the VA range to 2^31 bytes. + * Dividing that by the number of bits per byte yields 256MB + */ +#define VFIO_USER_DEF_MAX_BITMAP (256 * 1024 * 1024) + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index ff553cad9d..5bc890a0f5 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -37,6 +37,7 @@ typedef struct VFIOUserMsg { uint32_t id; QemuCond cv; bool complete; + bool pending; enum msg_type type; } VFIOUserMsg; =20 @@ -56,6 +57,12 @@ typedef struct VFIOUserProxy { struct QIOChannel *ioc; void (*request)(void *opaque, VFIOUserMsg *msg); void *req_arg; + uint64_t max_xfer_size; + uint64_t max_send_fds; + uint64_t max_dma; + uint64_t dma_pgsizes; + uint64_t max_bitmap; + uint64_t migr_pgsize; int flags; QemuCond close_cv; AioContext *ctx; @@ -78,6 +85,7 @@ typedef struct VFIOUserProxy { =20 /* VFIOProxy flags */ #define VFIO_PROXY_CLIENT 0x1 +#define VFIO_PROXY_FORCE_QUEUED 0x4 =20 typedef struct VFIODevice VFIODevice; =20 @@ -86,5 +94,6 @@ void vfio_user_disconnect(VFIOUserProxy *proxy); void vfio_user_set_handler(VFIODevice *vbasedev, void (*handler)(void *opaque, VFIOUserMsg *msg), void *reqarg); +bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp); =20 #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index bad2829f5c..61f525cf4a 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -20,6 +20,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserPCIDevice, VFIO_USER_P= CI) struct VFIOUserPCIDevice { VFIOPCIDevice device; SocketAddress *socket; + bool send_queued; /* all sends are queued */ }; =20 /* @@ -92,6 +93,16 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error= **errp) vbasedev->proxy =3D proxy; vfio_user_set_handler(vbasedev, vfio_user_pci_process_req, vdev); =20 + vbasedev->name =3D g_strdup_printf("vfio-user:%s", sock_name); + + if (udev->send_queued) { + proxy->flags |=3D VFIO_PROXY_FORCE_QUEUED; + } + + if (!vfio_user_validate_version(proxy, errp)) { + goto error; + } + /* * vfio-user devices are effectively mdevs (don't use a host iommu). */ @@ -101,9 +112,13 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Err= or **errp) if (!vfio_device_attach_by_iommu_type(TYPE_VFIO_IOMMU_USER, vbasedev->name, vbasedev, as, errp)) { - error_prepend(errp, VFIO_MSG_PREFIX, vbasedev->name); - return; + goto error; } + + return; + +error: + error_prepend(errp, VFIO_MSG_PREFIX, vdev->vbasedev.name); } =20 static void vfio_user_instance_init(Object *obj) @@ -153,6 +168,7 @@ static const Property vfio_user_pci_dev_properties[] = =3D { sub_vendor_id, PCI_ANY_ID), DEFINE_PROP_UINT32("x-pci-sub-device-id", VFIOPCIDevice, sub_device_id, PCI_ANY_ID), + DEFINE_PROP_BOOL("x-send-queued", VFIOUserPCIDevice, send_queued, fals= e), }; =20 static void vfio_user_pci_set_socket(Object *obj, Visitor *v, const char *= name, diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 349ea2b27c..874142e9e5 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -13,11 +13,15 @@ #include "hw/vfio-user/proxy.h" #include "hw/vfio-user/trace.h" #include "qapi/error.h" +#include "qobject/qdict.h" +#include "qobject/qjson.h" +#include "qobject/qnum.h" #include "qemu/error-report.h" #include "qemu/lockable.h" #include "qemu/main-loop.h" #include "system/iothread.h" =20 +static int wait_time =3D 5000; /* wait up to 5 sec for busy servers */ static IOThread *vfio_user_iothread; =20 static void vfio_user_shutdown(VFIOUserProxy *proxy); @@ -27,9 +31,12 @@ static VFIOUserFDs *vfio_user_getfds(int numfds); static void vfio_user_recycle(VFIOUserProxy *proxy, VFIOUserMsg *msg); =20 static void vfio_user_recv(void *opaque); +static void vfio_user_send(void *opaque); static void vfio_user_cb(void *opaque); =20 static void vfio_user_request(void *opaque); +static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags); =20 static inline void vfio_user_set_error(VFIOUserHdr *hdr, uint32_t err) { @@ -48,6 +55,41 @@ static void vfio_user_shutdown(VFIOUserProxy *proxy) proxy->ctx, NULL, NULL); } =20 +/* + * Same return values as qio_channel_writev_full(): + * + * QIO_CHANNEL_ERR_BLOCK: *errp not set + * -1: *errp will be populated + * otherwise: bytes written + */ +static ssize_t vfio_user_send_qio(VFIOUserProxy *proxy, VFIOUserMsg *msg, + Error **errp) +{ + VFIOUserFDs *fds =3D msg->fds; + struct iovec iov =3D { + .iov_base =3D msg->hdr, + .iov_len =3D msg->hdr->size, + }; + size_t numfds =3D 0; + int *fdp =3D NULL; + ssize_t ret; + + if (fds !=3D NULL && fds->send_fds !=3D 0) { + numfds =3D fds->send_fds; + fdp =3D fds->fds; + } + + ret =3D qio_channel_writev_full(proxy->ioc, &iov, 1, fdp, numfds, 0, e= rrp); + + if (ret =3D=3D -1) { + vfio_user_set_error(msg->hdr, EIO); + vfio_user_shutdown(proxy); + } + trace_vfio_user_send_write(msg->hdr->id, ret); + + return ret; +} + static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, VFIOUserFDs *fds) { @@ -88,6 +130,7 @@ static void vfio_user_recycle(VFIOUserProxy *proxy, VFIO= UserMsg *msg) msg->hdr =3D NULL; msg->fds =3D NULL; msg->complete =3D false; + msg->pending =3D false; QTAILQ_INSERT_HEAD(&proxy->free, msg, next); } =20 @@ -385,6 +428,62 @@ static void vfio_user_recv(void *opaque) } } =20 +/* + * Send a single message, same return semantics as vfio_user_send_qio(). + * + * Sent async messages are freed, others are moved to pending queue. + */ +static ssize_t vfio_user_send_one(VFIOUserProxy *proxy, Error **errp) +{ + VFIOUserMsg *msg; + ssize_t ret; + + msg =3D QTAILQ_FIRST(&proxy->outgoing); + ret =3D vfio_user_send_qio(proxy, msg, errp); + if (ret < 0) { + return ret; + } + + QTAILQ_REMOVE(&proxy->outgoing, msg, next); + if (msg->type =3D=3D VFIO_MSG_ASYNC) { + vfio_user_recycle(proxy, msg); + } else { + QTAILQ_INSERT_TAIL(&proxy->pending, msg, next); + msg->pending =3D true; + } + + return ret; +} + +/* + * Send messages from outgoing queue when the socket buffer has space. + * If we deplete 'outgoing', remove ourselves from the poll list. + */ +static void vfio_user_send(void *opaque) +{ + VFIOUserProxy *proxy =3D opaque; + + QEMU_LOCK_GUARD(&proxy->lock); + + if (proxy->state =3D=3D VFIO_PROXY_CONNECTED) { + while (!QTAILQ_EMPTY(&proxy->outgoing)) { + Error *local_err =3D NULL; + int ret; + + ret =3D vfio_user_send_one(proxy, &local_err); + + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + return; + } else if (ret =3D=3D -1) { + error_report_err(local_err); + return; + } + } + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, + vfio_user_recv, NULL, NULL, proxy); + } +} + static void vfio_user_cb(void *opaque) { VFIOUserProxy *proxy =3D opaque; @@ -446,6 +545,128 @@ static void vfio_user_request(void *opaque) } } =20 +/* + * Messages are queued onto the proxy's outgoing list. + * + * It handles 3 types of messages: + * + * async messages - replies and posted writes + * + * There will be no reply from the server, so message + * buffers are freed after they're sent. + * + * nowait messages - map/unmap during address space transactions + * + * These are also sent async, but a reply is expected so that + * vfio_wait_reqs() can wait for the youngest nowait request. + * They transition from the outgoing list to the pending list + * when sent, and are freed when the reply is received. + * + * wait messages - all other requests + * + * The reply to these messages is waited for by their caller. + * They also transition from outgoing to pending when sent, but + * the message buffer is returned to the caller with the reply + * contents. The caller is responsible for freeing these messages. + * + * As an optimization, if the outgoing list and the socket send + * buffer are empty, the message is sent inline instead of being + * added to the outgoing list. The rest of the transitions are + * unchanged. + */ +static bool vfio_user_send_queued(VFIOUserProxy *proxy, VFIOUserMsg *msg, + Error **errp) +{ + int ret; + + /* + * Unsent outgoing msgs - add to tail + */ + if (!QTAILQ_EMPTY(&proxy->outgoing)) { + QTAILQ_INSERT_TAIL(&proxy->outgoing, msg, next); + return true; + } + + /* + * Try inline - if blocked, queue it and kick send poller + */ + if (proxy->flags & VFIO_PROXY_FORCE_QUEUED) { + ret =3D QIO_CHANNEL_ERR_BLOCK; + } else { + ret =3D vfio_user_send_qio(proxy, msg, errp); + } + + if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { + QTAILQ_INSERT_HEAD(&proxy->outgoing, msg, next); + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, + vfio_user_recv, proxy->ctx, + vfio_user_send, proxy); + return true; + } + if (ret =3D=3D -1) { + return false; + } + + /* + * Sent - free async, add others to pending + */ + if (msg->type =3D=3D VFIO_MSG_ASYNC) { + vfio_user_recycle(proxy, msg); + } else { + QTAILQ_INSERT_TAIL(&proxy->pending, msg, next); + msg->pending =3D true; + } + + return true; +} + +/* + * Returns false if we did not successfully receive a reply message, in wh= ich + * case @errp will be populated. + * + * In either case, the caller must free @hdr and @fds if needed. + */ +static bool vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize, Error **errp) +{ + VFIOUserMsg *msg; + bool ok =3D false; + + if (hdr->flags & VFIO_USER_NO_REPLY) { + error_setg_errno(errp, EINVAL, "%s on NO_REPLY message", __func__); + return false; + } + + qemu_mutex_lock(&proxy->lock); + + msg =3D vfio_user_getmsg(proxy, hdr, fds); + msg->id =3D hdr->id; + msg->rsize =3D rsize ? rsize : hdr->size; + msg->type =3D VFIO_MSG_WAIT; + + ok =3D vfio_user_send_queued(proxy, msg, errp); + + if (ok) { + while (!msg->complete) { + if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, wait_time)) { + VFIOUserMsgQ *list; + + list =3D msg->pending ? &proxy->pending : &proxy->outgoing; + QTAILQ_REMOVE(list, msg, next); + error_setg_errno(errp, ETIMEDOUT, + "timed out waiting for reply"); + ok =3D false; + break; + } + } + } + + vfio_user_recycle(proxy, msg); + + qemu_mutex_unlock(&proxy->lock); + + return ok; +} =20 static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); @@ -474,6 +695,15 @@ VFIOUserProxy *vfio_user_connect_dev(SocketAddress *ad= dr, Error **errp) proxy =3D g_malloc0(sizeof(VFIOUserProxy)); proxy->sockname =3D g_strdup_printf("unix:%s", sockname); proxy->ioc =3D ioc; + + /* init defaults */ + proxy->max_xfer_size =3D VFIO_USER_DEF_MAX_XFER; + proxy->max_send_fds =3D VFIO_USER_DEF_MAX_FDS; + proxy->max_dma =3D VFIO_USER_DEF_MAP_MAX; + proxy->dma_pgsizes =3D VFIO_USER_DEF_PGSIZE; + proxy->max_bitmap =3D VFIO_USER_DEF_MAX_BITMAP; + proxy->migr_pgsize =3D VFIO_USER_DEF_PGSIZE; + proxy->flags =3D VFIO_PROXY_CLIENT; proxy->state =3D VFIO_PROXY_CONNECTED; =20 @@ -571,3 +801,288 @@ void vfio_user_disconnect(VFIOUserProxy *proxy) g_free(proxy->sockname); g_free(proxy); } + +static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags) +{ + static uint16_t next_id; + + hdr->id =3D qatomic_fetch_inc(&next_id); + hdr->command =3D cmd; + hdr->size =3D size; + hdr->flags =3D (flags & ~VFIO_USER_TYPE) | VFIO_USER_REQUEST; + hdr->error_reply =3D 0; +} + +struct cap_entry { + const char *name; + bool (*check)(VFIOUserProxy *proxy, QObject *qobj, Error **errp); +}; + +static bool caps_parse(VFIOUserProxy *proxy, QDict *qdict, + struct cap_entry caps[], Error **errp) +{ + QObject *qobj; + struct cap_entry *p; + + for (p =3D caps; p->name !=3D NULL; p++) { + qobj =3D qdict_get(qdict, p->name); + if (qobj !=3D NULL) { + if (!p->check(proxy, qobj, errp)) { + return false; + } + qdict_del(qdict, p->name); + } + } + + /* warning, for now */ + if (qdict_size(qdict) !=3D 0) { + warn_report("spurious capabilities"); + } + return true; +} + +static bool check_migr_pgsize(VFIOUserProxy *proxy, QObject *qobj, Error *= *errp) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t pgsize; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &pgsize)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_PGSIZE); + return false; + } + + /* must be larger than default */ + if (pgsize & (VFIO_USER_DEF_PGSIZE - 1)) { + error_setg(errp, "pgsize 0x%"PRIx64" too small", pgsize); + return false; + } + + proxy->migr_pgsize =3D pgsize; + return true; +} + +static bool check_bitmap(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t bitmap_size; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &bitmap_size)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_BITMAP); + return false; + } + + /* can only lower it */ + if (bitmap_size > VFIO_USER_DEF_MAX_BITMAP) { + error_setg(errp, "%s too large", VFIO_USER_CAP_MAX_BITMAP); + return false; + } + + proxy->max_bitmap =3D bitmap_size; + return true; +} + +static struct cap_entry caps_migr[] =3D { + { VFIO_USER_CAP_PGSIZE, check_migr_pgsize }, + { VFIO_USER_CAP_MAX_BITMAP, check_bitmap }, + { NULL } +}; + +static bool check_max_fds(VFIOUserProxy *proxy, QObject *qobj, Error **err= p) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t max_send_fds; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &max_send_fds) || + max_send_fds > VFIO_USER_MAX_MAX_FDS) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_FDS); + return false; + } + proxy->max_send_fds =3D max_send_fds; + return true; +} + +static bool check_max_xfer(VFIOUserProxy *proxy, QObject *qobj, Error **er= rp) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t max_xfer_size; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &max_xfer_size) || + max_xfer_size > VFIO_USER_MAX_MAX_XFER) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_XFER); + return false; + } + proxy->max_xfer_size =3D max_xfer_size; + return true; +} + +static bool check_pgsizes(VFIOUserProxy *proxy, QObject *qobj, Error **err= p) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t pgsizes; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &pgsizes)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_PGSIZES); + return false; + } + + /* must be larger than default */ + if (pgsizes & (VFIO_USER_DEF_PGSIZE - 1)) { + error_setg(errp, "pgsize 0x%"PRIx64" too small", pgsizes); + return false; + } + + proxy->dma_pgsizes =3D pgsizes; + return true; +} + +static bool check_max_dma(VFIOUserProxy *proxy, QObject *qobj, Error **err= p) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t max_dma; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &max_dma)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAP_MAX); + return false; + } + + /* can only lower it */ + if (max_dma > VFIO_USER_DEF_MAP_MAX) { + error_setg(errp, "%s too large", VFIO_USER_CAP_MAP_MAX); + return false; + } + + proxy->max_dma =3D max_dma; + return true; +} + +static bool check_migr(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QDict *qdict =3D qobject_to(QDict, qobj); + + if (qdict =3D=3D NULL) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_FDS); + return true; + } + return caps_parse(proxy, qdict, caps_migr, errp); +} + +static struct cap_entry caps_cap[] =3D { + { VFIO_USER_CAP_MAX_FDS, check_max_fds }, + { VFIO_USER_CAP_MAX_XFER, check_max_xfer }, + { VFIO_USER_CAP_PGSIZES, check_pgsizes }, + { VFIO_USER_CAP_MAP_MAX, check_max_dma }, + { VFIO_USER_CAP_MIGR, check_migr }, + { NULL } +}; + +static bool check_cap(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QDict *qdict =3D qobject_to(QDict, qobj); + + if (qdict =3D=3D NULL) { + error_setg(errp, "malformed %s", VFIO_USER_CAP); + return false; + } + return caps_parse(proxy, qdict, caps_cap, errp); +} + +static struct cap_entry ver_0_0[] =3D { + { VFIO_USER_CAP, check_cap }, + { NULL } +}; + +static bool caps_check(VFIOUserProxy *proxy, int minor, const char *caps, + Error **errp) +{ + QObject *qobj; + QDict *qdict; + bool ret; + + qobj =3D qobject_from_json(caps, NULL); + if (qobj =3D=3D NULL) { + error_setg(errp, "malformed capabilities %s", caps); + return false; + } + qdict =3D qobject_to(QDict, qobj); + if (qdict =3D=3D NULL) { + error_setg(errp, "capabilities %s not an object", caps); + qobject_unref(qobj); + return false; + } + ret =3D caps_parse(proxy, qdict, ver_0_0, errp); + + qobject_unref(qobj); + return ret; +} + +static GString *caps_json(void) +{ + QDict *dict =3D qdict_new(); + QDict *capdict =3D qdict_new(); + QDict *migdict =3D qdict_new(); + GString *str; + + qdict_put_int(migdict, VFIO_USER_CAP_PGSIZE, VFIO_USER_DEF_PGSIZE); + qdict_put_int(migdict, VFIO_USER_CAP_MAX_BITMAP, VFIO_USER_DEF_MAX_BIT= MAP); + qdict_put_obj(capdict, VFIO_USER_CAP_MIGR, QOBJECT(migdict)); + + qdict_put_int(capdict, VFIO_USER_CAP_MAX_FDS, VFIO_USER_MAX_MAX_FDS); + qdict_put_int(capdict, VFIO_USER_CAP_MAX_XFER, VFIO_USER_DEF_MAX_XFER); + qdict_put_int(capdict, VFIO_USER_CAP_PGSIZES, VFIO_USER_DEF_PGSIZE); + qdict_put_int(capdict, VFIO_USER_CAP_MAP_MAX, VFIO_USER_DEF_MAP_MAX); + + qdict_put_obj(dict, VFIO_USER_CAP, QOBJECT(capdict)); + + str =3D qobject_to_json(QOBJECT(dict)); + qobject_unref(dict); + return str; +} + +bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp) +{ + g_autofree VFIOUserVersion *msgp =3D NULL; + GString *caps; + char *reply; + int size, caplen; + + caps =3D caps_json(); + caplen =3D caps->len + 1; + size =3D sizeof(*msgp) + caplen; + msgp =3D g_malloc0(size); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_VERSION, size, 0); + msgp->major =3D VFIO_USER_MAJOR_VER; + msgp->minor =3D VFIO_USER_MINOR_VER; + memcpy(&msgp->capabilities, caps->str, caplen); + g_string_free(caps, true); + trace_vfio_user_version(msgp->major, msgp->minor, msgp->capabilities); + + if (!vfio_user_send_wait(proxy, &msgp->hdr, NULL, 0, errp)) { + return false; + } + + if (msgp->hdr.flags & VFIO_USER_ERROR) { + error_setg_errno(errp, msgp->hdr.error_reply, "version reply"); + return false; + } + + if (msgp->major !=3D VFIO_USER_MAJOR_VER || + msgp->minor > VFIO_USER_MINOR_VER) { + error_setg(errp, "incompatible server version"); + return false; + } + + reply =3D msgp->capabilities; + if (reply[msgp->hdr.size - sizeof(*msgp) - 1] !=3D '\0') { + error_setg(errp, "corrupt version reply"); + return false; + } + + if (!caps_check(proxy, msgp->minor, reply, errp)) { + return false; + } + + trace_vfio_user_version(msgp->major, msgp->minor, msgp->capabilities); + return true; +} diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index ddeb9f4b2f..a965c7b1f2 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -6,3 +6,5 @@ vfio_user_recv_hdr(const char *name, uint16_t id, uint16_t cmd, uint32_t s= ize, uint32_t flags) " (%s) id 0x%x cmd 0x%x size 0x%x flags 0x%x" vfio_user_recv_read(uint16_t id, int read) " id 0x%x read 0x%x" vfio_user_recv_request(uint16_t cmd) " command 0x%x" +vfio_user_send_write(uint16_t id, int wrote) " id 0x%x wrote 0x%x" +vfio_user_version(uint16_t major, uint16_t minor, const char *caps) " majo= r %d minor %d caps: %s" --=20 2.43.0 From nobody Sat Nov 15 14:11:13 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750880101; cv=pass; d=zohomail.com; s=zohoarc; b=myEpCNytIVGBvh/RMnT58LaIu4oLhudaRU/OH944poeCytTbywcQTvWFolISbDoY4vPbfe9hiJqQM4+Uxo72tyIbfMTA5XEGWyrWAgE6pd8YG+NVHdSFri4GR1n+ut7rR2YcXFjwAzXDu5Fvu4beqir4SowzbK7zgg78vTfGrIg= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750880101; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=55dXXLexKx7Msp6EBd1yEeV9hv07K06mca0WBr2LHf4=; b=hRdNOR9GO7zA7qMJBIi4mDYHHcoszucaUHuJ2ivQ9r3wyFqTKXkI9Gb72E1isCLel9L5DgDH7zfPzFhCJlYMd0L0p6Z8MnDHZUBBwYPccm43hZOgL+hmnZgF3F/XajT+UDdr351xI1Y0eXEA/cqLepSR/TBYNDKC1i1J3dVZuDQ= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750880101240727.8390854079088; Wed, 25 Jun 2025 12:35:01 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVph-00083H-2v; Wed, 25 Jun 2025 15:30:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpM-00080b-AI for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:36 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpK-0003VK-14 for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:36 -0400 Received: from pps.filterd (m0127842.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PH0WY9021654; Wed, 25 Jun 2025 12:30:32 -0700 Received: from nam04-dm6-obe.outbound.protection.outlook.com (mail-dm6nam04on2112.outbound.protection.outlook.com [40.107.102.112]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47geghhf88-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:32 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:30 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:30 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=55dXXLexKx7Msp6EBd1yEeV9hv07K06mca0WBr2LH f4=; b=mqjk+Lvh23CgSCgQnbe9FpGMUTjKkIJkFIHXGoo04xgTcgr04L4t9OzV4 SB/lo1RbNTiLPMBsySqyO3Pn+TX9HZjHzkk1HKVVkxzuzVNcYsiJo/LQtSFpNZqd 9ImtxWO5Ssu86zJVrw68o4NXpj6CaSL8d41vMt8yUvgFhZtUNRizlS+aKvnTNfw0 H4xeZ3/h0Q9gJYtChj+ibkeK0axlhg4WOEXimY7ovmeycdpKaq+kFBSLjgQmsbbI gMAAdR3XtIsn0eaDNMONMYqy383lIaAPZ5peSgTrzTqzuOulU0jkwMx+fsZkzAPd 7ro0lZcubmFOXCIMYLHXej82JcMNA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ThL9iRnc9o9mclDbdBU6NWfVn+I3cgmcFJBWa2on0wVrAFOFWMq5iMWqXlbVxOXfU3ad5B7COh02HwN0QyzCJgtzt4ynhP1VywgG0nqdWC94nQtQk8qAeCkKNZNx2X8sdISV7Gxc5cbMBDVJc+02Iia1l9MTu2lCv15AfQK7MEPBLiCwNcV71tuC6kYpgMLP4Fb75sfkRmIA7XW6sBWuIdczXgqRy7Y1lFwVU7NyxckF/1MwqZxeAgt3nZ4OOxKiC+QX3tz9tVpl3UkXE+M5Rdoph5bgMBU6z1B4Qj849z4eIfxvM/LSkRtnUNKKiUkbLmtc4xYfSt5CoWrsF0GqKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=55dXXLexKx7Msp6EBd1yEeV9hv07K06mca0WBr2LHf4=; b=maKzDB1VlAMaER1xM9N38AoD52FRle13Efr8G8RE3saZYYXIk+PfYjUW3DkLnuMonHCynLcPdxw3n9rFXrG8iX7ciY74jX9ueGqeaWHeLOQ82J+06Rdu+MmQgw/komhfDQ3cL7Ly3B5uynVydl5214sOt8uoUIc/oK5aNcescOl5U562aaPu/AJl1ty25ShgS7wj3wkpWeS+MdYE76XbMKp56TDoNfhM1flc6I7o1NR0mbsaC+nRVyUpYL2BMrFjRDs/qCGlMRD+fZ83BEjTS6FOZHps3UvGX/NbrlAkaCocBs3ynTN2P7IZ7+mHn+z04sGzI3nFkA2vnP8v0Bmp/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=55dXXLexKx7Msp6EBd1yEeV9hv07K06mca0WBr2LHf4=; b=wHERLLjV/5ddXedZDqi9vbg+MV5lvl3iHVI1uNhn3eCX1qkjkJv5cM9tbT/7fpiWaj/4uu63aALcUAztswMvleM2wCs9ucKpOEmvTboqDHVpd9SVC7wIfUlD9g2JT7armOt5JwOOOyBMDQF7GbT5cucyBJ7Mmnr+Agft69RsgeIsiJmxFTdbKXiez6egPDIlBOXXobUn7VqrCuxnA6uE4IQv6aPXzA/bLTSVKVWq0OWWHemKB7DNeu/qzn8X4WYNRm5L75Kaq828xK6l4K/UW1tbnltt8fzYOgFFjqD0MBIKAZrtT3qmmOIaoUQ1ljCvbOGDyaEHpPDNbQxQLewSEA== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 05/19] vfio-user: implement VFIO_USER_DEVICE_GET_INFO Date: Wed, 25 Jun 2025 20:29:57 +0100 Message-ID: <20250625193012.2316242-6-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: e04ce7e6-f495-47aa-5582-08ddb41ebef8 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?B?T2lqVWdVSUsweERYSUhOaVpNQ09KRHFUN1VpVVNGYnE4YlRtNjdMOXQyVi9P?= =?utf-8?B?ZHFHdHZMOHozdDl6emFDckU4b0grYWxRLzJ3NXpBZ3JhMThja2p2bHNKSTI0?= =?utf-8?B?Ujc5cmJPTDBCN3RMWWkxOUhPMHBSSTRlV0J4NXlpOGZCaHA4bDNURWRpaWVX?= =?utf-8?B?V2Q1ZCtTNjIzWHc1OWJBc3B6T2txLzZvbHJCbHp6OThnTldFTXlpRGRENnhH?= =?utf-8?B?d3BRWDhVendKN216QkRCdTJaR2R3VXpSUWJsUjFoOFFMVzFCRzlUQnU4M20r?= =?utf-8?B?L2JZb2tDNy9XRkdkVWR0Sjg3NGg2dVZvd0tPU1dRN3NsQUs4NnhHWjgvWWFu?= =?utf-8?B?TXV0Ty81TkNJZnNHNGlwTFQ0NHQydG4xRjJRM05PQkh2UWgyRCtZWm5CcFk5?= =?utf-8?B?bGxwek9abjNWdlc1M25YY2wvZXl1d1R5ckE4M3NUUWQyOUp5VWdqN3VrSlFx?= =?utf-8?B?K212UG5rVDczeGcrSFQzd0hUZW95UndQK0pVVmZUcVRkZURZbytJSHYzTUZX?= =?utf-8?B?Y3pnYWdJR0tjK3RVeENsaDJmUzRZQ0lYWnBJeTVMNXc1bGNaL3UxU0NWaWMr?= =?utf-8?B?c0gxbW1mMWFNMHZPN2Y3U2NFcW5kSGUyOStmZkhoOHpTUjVXL1NLV3R3d3o4?= =?utf-8?B?UFdNcnVENFRuVjl4dGJYZ1B0dTlIUXJRQnVnSDdVSEd4bE9TNlB3K2xjci9i?= =?utf-8?B?UG1reHRLQnRhR2ZNdDNhbmtCazlURXMxT1dqN0xjVWV2ZlROUCt3ZlhXaHk5?= =?utf-8?B?NmZPcXF6ZTAvM0piUTlLb3ZWRHdEcUVpZ2tvSFpPbnZmQVRva2lOajRaS01O?= =?utf-8?B?QnZEdzh4K2NaU241V3BzZUltUDlCNm01K1NEa05odnZxZi9VOHMyRG1qWHlz?= =?utf-8?B?dHNFcVFtMFl4cXpaQ3JlWlpHUjlZNzNrU1RIZ3hJWEJlelJXdXp6VEVrOHhD?= =?utf-8?B?TUJkVWdHNnF0ZU5NN2Zhd0ZUbG96WjdZejNkQnZJR1dwVmxTcnc5TC9VcTdx?= =?utf-8?B?a0Y1SkxIUm4xVnZwMi9FWVFGU3dVRGJWUm1VMWJ2TWk5NE1hSitUWGRrWWhU?= =?utf-8?B?ZzRsNTBwSEZKM21FNnVwSkVhTzAwa2ZtK0lVUmlwUXB2aG9YZi9sRnBpYzBM?= =?utf-8?B?WmNWSmNOdkpQTEM3Z2VhOUJpVytxUjN6ZmZ5bHR5UkJaYk9WdFlDSGtJYjRC?= =?utf-8?B?WkQ0b3h6TWNRNWVGNGo3NUZvMUlOZzVkdlI1UURrUStiZjNKWW1DTDA5V203?= =?utf-8?B?bms5QVFjY0ZhbWFuVUpmSHY0aXZsY2Q0RFFrcm1oQ1dCZGJFSFlyUEo0Yk5Z?= =?utf-8?B?ZTZkZnR5QW5sdDNPRXNVbjBRYmNXa0ZIWGxjMDNDdW40SE5yb2ZNeXdLbVJY?= =?utf-8?B?NWNRTS9lNE1TTkhLWFVuNllORE1yaW13UzBjcEVZL21KVlpRRlJsMWMzWFIz?= =?utf-8?B?VVkxWnRRYzFVaXcvejV2eFVBNGlyK1pXNkdYTExCMHJVVWNyaEp1OCt6VHVo?= =?utf-8?B?Y2p4ZVAwcktibXAxNFcvd0JFNWRIQnRPYTJDTkZhbGdsd2pPL0k1ZmRKRDkv?= =?utf-8?B?WHZidWxnV25ZTm51U1U1enBYeXl4bk1DZkF1eWxHSS9MczJjc2V0d1MwM2g0?= =?utf-8?B?Ylg5VkZmeWZGc0NDMHFpUjZxRWZmNklxSFZLOTUzUGRXNXZ5UDRlblBZT3dQ?= =?utf-8?B?ZTk1bEdrTzljdGNUUG12YnFiZzgwLzErWTVaWnFBQk5xQ2xwaDVTaXNSZUQx?= =?utf-8?B?aENPc1ozYkF5TjZFU21VMjVHTG9laE9KRGhaNG15bm1iVmVxRTdaQ2tNcmFj?= =?utf-8?B?dyswU0JkR3dVOVFTOWd2VDdmZWZsa20rSmh1ZUVuNjhEWExvVEZWL3FvR0du?= =?utf-8?B?c1pSdW4yd3Yra0lPVjNDNVVZbzFCYmE3NWNoL0tBL2hSVlUrd1l5Y0lwc2xx?= =?utf-8?Q?Jlo5XqxsEqs=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TktMNzQyUDhhUStuZUc0QXhKWkNnRFVoTDBOV2srdzFiQ29KNUJZWmhlVUxq?= =?utf-8?B?cmRrWDMvU1NWUngvWFFRK3NDUXU0R1gvSStyZWJFKys3NVZCYWEwWE01eDRV?= =?utf-8?B?REZnT3gvdmhxVWJaY1hFcFVtaW42a3gvV0NrYzRqWkFlSHRCT05RWVA4WGlN?= =?utf-8?B?b21jNEpLSU14dzNkMjdnaDZraWwwcnpwdHJsYUI3NUt1YTVJeU9JWVFHV0FE?= =?utf-8?B?eHNGRWJlUk81Mk1UbHBSZkVvYXg3Sk5yVEVHMVNmbmFsUnM1TGlFdTlic3lJ?= =?utf-8?B?ak5Vb1dJLy93eXRQUE9uRkIzbjVHL2RQSjF4dTNTbUJyc29EUGFHdzZwNEZh?= =?utf-8?B?NWYzRVB3Y3JqUiszQ3U2QlR5VWppY0xKT3A5eGx3bTYzUVNlRnhHbDJQa1Vz?= =?utf-8?B?Q2FXTzM5OXpRck11NGt0bjZvV2RZSTFDRXRrRmNxTUYvTHAwU21CaDM1U2w0?= =?utf-8?B?Qjd4TzRscENrV0h6eUJMOGFqUWhNN29rem80WmV6L0JtN0hSUDYyT2JKSkFx?= =?utf-8?B?TGMxdzVoK0xOWWlodkRNT1pxTkhocks3a080QU9OdE5CWGNaWEpPcWpEcjV5?= =?utf-8?B?K2R1OFRQajlkWTNUdzdHK3VRZ0RXUlIvY0FyNHVEeDAxR0ZNUHZqSnljNzd3?= =?utf-8?B?MEp2TEJUSkE3bkpNSXZuWHZuMWZud0FUbjZsbE9keHVrRGRFOURZVlhrUVJE?= =?utf-8?B?cUVseDZZb0JIYndPaFBqdldxaDVHUGpMUmFPRnAyTWhNeHJkTmxUaTlGWlZ4?= =?utf-8?B?V0t3a0F0TEpIOVpBTmpON2YzemNjeXdMdjZUSjVvbnEwc1FmY09RYS9VOUt3?= =?utf-8?B?dXNpb0NUK1ZtVGxLSjZiSmlEUGl3Um05eG8vVDF0SE1XUERkV0JsQ1ZvZ1o0?= =?utf-8?B?cUtKNElDMytKTFdXQUR3dFlUTmJONWt3NHZvWWFzcGk5eXdVRnpEeGM0Sjdw?= =?utf-8?B?K29Pa0h2MW9tZG85bERmNEMvbGdlcGE4dHRvT1lBQk5Wc21XNjdpRkFqR05k?= =?utf-8?B?TG5GcUdyMUdmMzVUa2NRTUdSSFk3bUJmVzI2eWVwbnZyc3BBS3llcjIyM21W?= =?utf-8?B?VDJzY1lHeWw4aW9FK1M0bEkvUm1WcVcvV3VranhydE12SWVDU2xlNnZONkNL?= =?utf-8?B?U0ZOcWxGMTJoWWNwZHkzQ0IvellWVEptOHhQekphVlVqVC93dnNoeEVwMjNL?= =?utf-8?B?bTNNNkRzcTg0QWJTb09qOWVuZlhnc0xTQlBQM0M4RGlCWUZnbE10ajFNMjRO?= =?utf-8?B?cXoyeW04TTBHdzFHWEIyYzk1WWFmZm5CdWF2Unp1S1doOWJEOGk3UnpFL3Fp?= =?utf-8?B?Rm5hcUhuWndIVXFwVEJVWU00WHROaDFVbTJzYjI2eHlvQ0MzWCtocWtGUGpY?= =?utf-8?B?UVhRRENGWWdHM3JhSGNvbFZNaHRqZmYxUGhMUjN0Z3Uxdk0ydExVTmJudmdk?= =?utf-8?B?VloxWWZRbzJ4WHRoRWJPR2JsZ0Q5QU01WldmUERYWC9wbkNsd2tkbG14R1Rh?= =?utf-8?B?WVNKSXJKd1JMVHJoclNwdFFjaDVWWDc0Mk9Fc2dsQjF0MXkxeHpUWUNGTWlm?= =?utf-8?B?STlCdy9FMnZSYjhlWUFEdTFJQ2tGMHJRa3VxYThJQmdWUWdtVzEwcERZaXA5?= =?utf-8?B?TVFMWkVjOFJJMXJLYVdPWWl1ZS9uUS9QNGcxSzJ0RjRGYjIycThCUFNHOFE5?= =?utf-8?B?MVZJdkVnMzdTUEZ5TmF4UEJJL2VxWmRwQzM0YmwvQkxSK2hiVkp1SHJGMWJq?= =?utf-8?B?aktKOHQ3a1JEZXRJaytwUHA1aGgyUkgyU2pReW5WMXVpaGJXTllQQjQ3eGdp?= =?utf-8?B?NDJUUVIvRThjcGU2MFZHbis5V3c3UzBqM2xnVytQSzNOZENtamY0SFhNM3A4?= =?utf-8?B?OVVpOHJqaUtXNnI1alplR09aczZ4aE5RVzRKN3FqaXFsVldIOWk2eWNaVmZ6?= =?utf-8?B?bXM0a3lnSUZSUHV1bklKNXRHc1RtSHhVejZkUzRDaGR0UmlMcGFEempyTENP?= =?utf-8?B?YW9BMnZZQzd0Z0lUb00xY0ZneXpRNEx3eUhhRWd2R0hzUmNXQk9UejdLbHE2?= =?utf-8?B?OTRxdVhNYVBtRUMvTDdVd01yMFp2Sld2cEpvMnBGb0RtOERxdGJwR1c3YXdE?= =?utf-8?Q?gVQGWr+mWROhHN55flbpd11DI?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: e04ce7e6-f495-47aa-5582-08ddb41ebef8 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:30.5548 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: f4wuI3N5YSCVDOJoNugyTFDsXOaSSz0Hr5SxE4HWv+THIHtJW1t7W0E3otruhknpHqIV7fkV6+5f3Du27N0vBQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Authority-Analysis: v=2.4 cv=fPo53Yae c=1 sm=1 tr=0 ts=685c4e58 cx=c_pps a=d88lBm4WbpKxCRKYgJm9OQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=CKY9KQTx7O4RNs4pBzQA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: 5lmy8PYUoPFp2h79xXj2BhCbKN3aJ4IG X-Proofpoint-GUID: 5lmy8PYUoPFp2h79xXj2BhCbKN3aJ4IG X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfXx0LVoOxb3e9w +dvvBo2ySC7cpk6TOIy0HX7Pgp4m/naEexQTOxT7N8YpCBKQWXJOtT7JIu4Kl0dFjnqqmwf2ZWL cqW4XIKGWx6w/mP1oCHohvu3CHNInhvMlO0cBV85j+uT24ysykd6xvsKGncoVRGfA0W1+rdKJbN Ro4/5z15s0aH1dCXi+0Ggr0CQyYgA1FwMS+D5d+pbQBbpQlkQub9fGs3V30TTRgwG010s10UfgM zlzaLiYJo/Kx+Y3cgpf1YUkZeucVbuS47YMu9o4QhvWCZZZ1LUCsuZmj2UaoS3UwrNiyPcBWdSj WLPMD+3z6bjlzBoTbdjRKm130SmYziaaj6onZ//YwWdRZkraRBwPPb4/7BOPmpR3qcpnNwJB7vc UFudkiP8Pf8pYTkWoN/S/2uOqSsB2ocv9hqMucZzNkahDGrQ3xoIvy/UKftUfZq2EhCCh/v2 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750880103045116600 Add support for getting basic device information. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/device.h | 20 ++++++++++++++ hw/vfio-user/protocol.h | 12 +++++++++ hw/vfio-user/proxy.h | 7 +++++ hw/vfio-user/container.c | 8 +++++- hw/vfio-user/device.c | 55 +++++++++++++++++++++++++++++++++++++++ hw/vfio-user/proxy.c | 10 +++---- hw/vfio-user/meson.build | 1 + hw/vfio-user/trace-events | 1 + 8 files changed, 107 insertions(+), 7 deletions(-) create mode 100644 hw/vfio-user/device.h create mode 100644 hw/vfio-user/device.c diff --git a/hw/vfio-user/device.h b/hw/vfio-user/device.h new file mode 100644 index 0000000000..ef3f71ee69 --- /dev/null +++ b/hw/vfio-user/device.h @@ -0,0 +1,20 @@ +#ifndef VFIO_USER_DEVICE_H +#define VFIO_USER_DEVICE_H + +/* + * vfio protocol over a UNIX socket device handling. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "linux/vfio.h" + +#include "hw/vfio-user/proxy.h" + +bool vfio_user_get_device_info(VFIOUserProxy *proxy, + struct vfio_device_info *info, Error **errp= ); + +#endif /* VFIO_USER_DEVICE_H */ diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 2d52d0fb10..e0bba68739 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -112,4 +112,16 @@ typedef struct { */ #define VFIO_USER_DEF_MAX_BITMAP (256 * 1024 * 1024) =20 +/* + * VFIO_USER_DEVICE_GET_INFO + * imported from struct vfio_device_info + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t num_regions; + uint32_t num_irqs; +} VFIOUserDeviceInfo; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 5bc890a0f5..837b02a8c4 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -12,7 +12,9 @@ #include "io/channel.h" #include "io/channel-socket.h" =20 +#include "qemu/queue.h" #include "qemu/sockets.h" +#include "qemu/thread.h" #include "hw/vfio-user/protocol.h" =20 typedef struct { @@ -96,4 +98,9 @@ void vfio_user_set_handler(VFIODevice *vbasedev, void *reqarg); bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp); =20 +void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags); +bool vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize, Error **errp); + #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/container.c b/hw/vfio-user/container.c index 2367332177..f5bfd54316 100644 --- a/hw/vfio-user/container.c +++ b/hw/vfio-user/container.c @@ -11,6 +11,7 @@ #include "qemu/osdep.h" =20 #include "hw/vfio-user/container.h" +#include "hw/vfio-user/device.h" #include "hw/vfio/vfio-cpr.h" #include "hw/vfio/vfio-device.h" #include "hw/vfio/vfio-listener.h" @@ -140,7 +141,12 @@ static void vfio_user_container_disconnect(VFIOUserCon= tainer *container) static bool vfio_user_device_get(VFIOUserContainer *container, VFIODevice *vbasedev, Error **errp) { - struct vfio_device_info info =3D { 0 }; + struct vfio_device_info info =3D { .argsz =3D sizeof(info) }; + + + if (!vfio_user_get_device_info(vbasedev->proxy, &info, errp)) { + return false; + } =20 vbasedev->fd =3D -1; =20 diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c new file mode 100644 index 0000000000..4212fefd44 --- /dev/null +++ b/hw/vfio-user/device.c @@ -0,0 +1,55 @@ +/* + * vfio protocol over a UNIX socket device handling. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/error-report.h" + +#include "hw/vfio-user/device.h" +#include "hw/vfio-user/trace.h" + +/* + * These are to defend against a malign server trying + * to force us to run out of memory. + */ +#define VFIO_USER_MAX_REGIONS 100 +#define VFIO_USER_MAX_IRQS 50 + +bool vfio_user_get_device_info(VFIOUserProxy *proxy, + struct vfio_device_info *info, Error **errp) +{ + VFIOUserDeviceInfo msg; + uint32_t argsz =3D sizeof(msg) - sizeof(msg.hdr); + + memset(&msg, 0, sizeof(msg)); + vfio_user_request_msg(&msg.hdr, VFIO_USER_DEVICE_GET_INFO, sizeof(msg)= , 0); + msg.argsz =3D argsz; + + if (!vfio_user_send_wait(proxy, &msg.hdr, NULL, 0, errp)) { + return false; + } + + if (msg.hdr.flags & VFIO_USER_ERROR) { + error_setg_errno(errp, -msg.hdr.error_reply, + "VFIO_USER_DEVICE_GET_INFO failed"); + return false; + } + + trace_vfio_user_get_info(msg.num_regions, msg.num_irqs); + + memcpy(info, &msg.argsz, argsz); + + /* defend against a malicious server */ + if (info->num_regions > VFIO_USER_MAX_REGIONS || + info->num_irqs > VFIO_USER_MAX_IRQS) { + error_setg_errno(errp, EINVAL, "invalid reply"); + return false; + } + + return true; +} diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 874142e9e5..aed7b22e2a 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -35,8 +35,6 @@ static void vfio_user_send(void *opaque); static void vfio_user_cb(void *opaque); =20 static void vfio_user_request(void *opaque); -static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, - uint32_t size, uint32_t flags); =20 static inline void vfio_user_set_error(VFIOUserHdr *hdr, uint32_t err) { @@ -626,8 +624,8 @@ static bool vfio_user_send_queued(VFIOUserProxy *proxy,= VFIOUserMsg *msg, * * In either case, the caller must free @hdr and @fds if needed. */ -static bool vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, - VFIOUserFDs *fds, int rsize, Error **errp) +bool vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize, Error **errp) { VFIOUserMsg *msg; bool ok =3D false; @@ -802,8 +800,8 @@ void vfio_user_disconnect(VFIOUserProxy *proxy) g_free(proxy); } =20 -static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, - uint32_t size, uint32_t flags) +void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags) { static uint16_t next_id; =20 diff --git a/hw/vfio-user/meson.build b/hw/vfio-user/meson.build index 9e85a8ea51..2ed0ae5b1d 100644 --- a/hw/vfio-user/meson.build +++ b/hw/vfio-user/meson.build @@ -3,6 +3,7 @@ vfio_user_ss =3D ss.source_set() vfio_user_ss.add(files( 'container.c', + 'device.c', 'pci.c', 'proxy.c', )) diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index a965c7b1f2..b7312d6d59 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -8,3 +8,4 @@ vfio_user_recv_read(uint16_t id, int read) " id 0x%x read 0= x%x" vfio_user_recv_request(uint16_t cmd) " command 0x%x" vfio_user_send_write(uint16_t id, int wrote) " id 0x%x wrote 0x%x" vfio_user_version(uint16_t major, uint16_t minor, const char *caps) " majo= r %d minor %d caps: %s" +vfio_user_get_info(uint32_t nregions, uint32_t nirqs) " #regions %d #irqs = %d" --=20 2.43.0 From nobody Sat Nov 15 14:11:13 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750879894; cv=pass; d=zohomail.com; s=zohoarc; b=FcECS1yDwopS9bvSP2u3iNo2guwqtQWV3QUg9ktz0iCW1RG9+S+1Hbj/MtKrjH+4F5gEei/pYXCbzj1imq8iEHWu0I5LE+KIJeVZTRaCYC40Vj7elv8FTnYRt9EOt2KFVNexo5UJWGmsHTkOYNBtdaoC/5K2rGzNsti8+UunjXU= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750879894; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ipwxrd7dvyRiZ3WRkuBCWgYI6rKFvryFDkV4qjtHC6o=; b=aLW7Xc6UZAnxZFg6992x2PWui7HG+CMm3ZxLudeEi3y42zL/zuShRIkeG2nFeu1SwvIahepzN7htdb/d7Rw0bF8BEMHQSTcSKB8AZnxOtL9OAsBjuLtpzs5trdUP7xvQyrSIAbNn05zqiiQrn5nxTWS+SaxVdfR825t9B+vOwA0= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 175087989456179.33730173280082; Wed, 25 Jun 2025 12:31:34 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpe-00082T-5T; Wed, 25 Jun 2025 15:30:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpP-000817-IP for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:49 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpN-0003b6-AC for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:39 -0400 Received: from pps.filterd (m0127843.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PJKsJV020637; Wed, 25 Jun 2025 12:30:36 -0700 Received: from nam04-dm6-obe.outbound.protection.outlook.com (mail-dm6nam04on2136.outbound.protection.outlook.com [40.107.102.136]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47dtuhaqnk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:35 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:33 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:33 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=ipwxrd7dvyRiZ3WRkuBCWgYI6rKFvryFDkV4qjtHC 6o=; b=qjd3A1cQNKQig2vqD5cFep6kdsFMYVXzrOGM7g/oOM6FvzrJwwvxFym72 tzAru642KInx4XoRLZq1nIAjPWXRzFvble1kFt4nmudTvsvabGeFso78neyQkeDt 0CucLm6aiod7CXe7xsfYrVn8iHHAJPUrgovCoQgsRREzhvR0Hb9TFiyMlm6wUnn/ GMMSdUXNEUkQrGDNcXLg5gvaMsqtfOOJOAd0bAXZPOrZ6WWCaE6DWzzHbHfchmEt QlMX4GlyAodA1Qme7T1bxw5rS5sgfT1jBVzTfe7JkUcXIaJ4vqL2XNq2I4gqUfA7 BHIOcKiUPaH323zM9buKxRpXsUM6g== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CMTpM9CVI+ZLMOQYvRNIyreHHQHRn14CiLleS+q1leMX/iQKa/IWSCQzho96xhanNuFaEqT6o2UlHhAr5eT5DPYMKErkKTK4klxCh+rb6SGugWVIr0x6RurhjkhghaIjOZen7wlVFf0ByJIcLrkCz12Xu7vJIXE76vL5uZP2Alp6prELv9JHwoxAcu6jR3R8DMP0gdFGRVsYrjwedYWGY4pWYlYe7GNtAzuF/PMRBwuLtm1gykd8v1LPKfupbTGNsB048x5hKKFAnd3PGxbvrutp8VHKCU8B5VXxDDcC/97aOGe4mSkTVjlyAYBbQClW6qARxgrPFjlweyk4Fz5Gfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ipwxrd7dvyRiZ3WRkuBCWgYI6rKFvryFDkV4qjtHC6o=; b=G9dnSHFYBcHuKTqGBK8gM1ancvzhB/Fcf0LuqsCLJoN5daC92iUeeqHY12dKVbm1T4tTTsdgSut7+sKmQ6hpYWeXqj2jJHpzIZ0/i+9AUHlBV99PRKV8pQ/dNiSSSVrjibrLYHC7uXx7bL9cKe3SYnPT7pfbjhuRN/YjNzNtBQyYu6ffs71hMZzb1vHQ691AvQ9fJQTeFUNWkFUR8SQmvluf9ArwpdaZcR26tSi6WNMn76LPvGTGUTXixd8K8RXEOM/vdkWSTtnEPevmmGWDWvaRPw32YeBmlCjoU9Sq1Z+SnGh2stFe686/Rkn6IOO+s5Y2jCCCTHpkZM1G9L9WRQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ipwxrd7dvyRiZ3WRkuBCWgYI6rKFvryFDkV4qjtHC6o=; b=EcIoqUoDynOtbUvYYPyBTWc+kNKPgH2XccFNSzojzCqTVL9LZHhy+0Xo+6DRUqqWV6Qw6nnBYULwmlffTzRxZ8wTRSl+fLRcPXn0v5lThdziTm8Kd51II6D3PlXvX7ncFNen8OY/Px8a8hjXK8+6gDicopVIhCzpvM+EgwlV0F1yf8urbGjlfurE9QUVMG1CYdTNIBzUIzj+v/AA9/rOmnBOlqtoca38OayBOKTdBjVt4yaPYUIklGJHG9OUxwHT8AJ49khmGiHBacjkSID8n8S2J38oNw5jzTq03G/C3aLaHFgzSyQyYenV+U9sOjgoxOGK0/eksT4ah0VKiJ4x+w== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 06/19] vfio-user: implement VFIO_USER_DEVICE_GET_REGION_INFO Date: Wed, 25 Jun 2025 20:29:58 +0100 Message-ID: <20250625193012.2316242-7-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: c288a4cd-5f7c-403d-a1e1-08ddb41ec0d6 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?omYpKWD23utHdA8s+l9aDaTTlgnEexk4To2SNAjHLO883z/0aPmjpSesv8G0?= =?us-ascii?Q?sKmkHfyK8S8OXf0nxvq9tJs0kPS63fVt1q3t1+YX5wrw8bruKkSUQ7o/IjuX?= =?us-ascii?Q?RmPpJpY856MJ0Dl3GcvRu7SeklBaHinUQxPzoooXiERxpTqbmD7cGtSGA26o?= =?us-ascii?Q?UHkK9/D1cDWhSet2+U/N95pvP33hOkbx7h0KJylw/53Az52EdXfpidPJyYCP?= =?us-ascii?Q?KSa504jwkPY94OhZSJWGDK8gh9U+1QYH17Tg1J9cp5XID9rXf4ZNutpZWv/e?= =?us-ascii?Q?EttRFzTprn/zoAe5LLDDwDteS0Km07kN/P9iNb+fbtGH3iy5SQxpTf6dquck?= =?us-ascii?Q?+2VOi1KCkYLCBTqe3UGjKklUmjKU2DPq7qgCqU2lc6POzgKleWqf7JjTr7ST?= =?us-ascii?Q?KdNARqmFjZ9zfrQX9j/+6q+MXMzE8SexAOTDxsRcdWjXPropSnMXzyj5YOK+?= =?us-ascii?Q?Er56MsUpxOOsoUwVZ/bVpkX2DR6Lz1siunZksZH7suYu7a4abG/bd7r3f7Uu?= =?us-ascii?Q?Q6QsazDgY9+3JQx03Rpajob9bTqt2lGyayjcF6RvhHqbsQmR8xnkw7UReC+1?= =?us-ascii?Q?o4759AYhakN8fBjHqbRZi4rSc0rEdeyESfIirahJoz3yenZbzG0vNy34s5hu?= =?us-ascii?Q?Bl+xswReXw5SfuSwbUqZveh2XnxfCaKWiCua2KLB+MDOvecCXeRvz24gW+uP?= =?us-ascii?Q?AyD4gJ7zMhHPnNHxa839bg8eL1ifFR+xYSv13rJplkXMNWgzck7NxqUWfEGo?= =?us-ascii?Q?eJeQOk5LWMGMy9v6i7QVfBACA++o2JbMYOB2vqA/tocDq0f+/HFdil0B0MR7?= =?us-ascii?Q?9gHZz/X4f4HoU5vcGV2fvYb3wWJa4tE+QkII1rVEjWS9F6WfwxVfLIi7sMyB?= =?us-ascii?Q?P79/y2ft20bwrLyt4ayB4wS57tB//SVgJK4balkHhwpUzGRG70dJDedxedoK?= =?us-ascii?Q?qgnMeUnmc6/fA4LTLd1+vkJ7sdCRGsZnbKiuD8Nk8eoQ4dJryJmZzvj9/zxp?= =?us-ascii?Q?VZvG61XVfUxouR8C3sGyAAHMjCRkFqLDeIbp7is+GxoyB/j7dQXb7jTFdhEw?= =?us-ascii?Q?cgLVgshHP6hsTJvYUer0Vf1HWfvthqrxu+YbeIoZt3BvP2QDg1r2o8qfcNrK?= =?us-ascii?Q?Im6xNKfvc8Ap4cTH5VcS/uuKniOCcNEFT/u39vYiwBXc4CdZibvHDuWKwx8r?= =?us-ascii?Q?Aam/Xr3DJtgsnDdNmbtra0F1xGVZSVzQwtFMP/XNoAONCBjpVKaCDpZQb0Gj?= =?us-ascii?Q?WSS72TsjjVUNVnTzfF0VwFyjFS16LavjmhlXoKlnz+LICsTYvt09ioYve3Tb?= =?us-ascii?Q?jvR31lIkP8F1zAdRW3P7WWGJJN04qC58O1SemgG7SW0v/AtOGyWH7qaub3i1?= =?us-ascii?Q?b6h4XqPWACFENXTB/GplrCkJpgJ9l993bIh6obkGXRxF6oau9g=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?EpubiQDYjTNEDOIQ34VXNl1G9Egq99gaJc78lWTskyEGZawG4XvGf21/2rYi?= =?us-ascii?Q?3y/7M07xu9jDnIPXYMEnrOSB0KRZqvxtvPFM30c4ZRRQaVLA47jaNnpCpfLo?= =?us-ascii?Q?7dSnWAvE0nrrLH5bmCT/pd/mzcA8gfpVaLdnNCvog1g/fcS/3AWTqSAfnb5K?= =?us-ascii?Q?R4lSbPhhizbWSfifoWwKm4ncgyUdWzL4jVvmjI+4UfUfN9IxGv0C/f2sBKJy?= =?us-ascii?Q?PXXX5CiwCQebL1XEqcMOxKChl+yyMeDeXvyAdP5kepIp33z7gGSLAuiP37wr?= =?us-ascii?Q?CWJLW2JqzX3lR+Ji3whDewwNuQg1wPzwOj2ItEWcSZDHJJo5t2CU/domVKzB?= =?us-ascii?Q?yqhHzcD/AiB2Dw2L+m08YtHVqXkWtnihlKpB1ZUx3DZ9RGfOSUBEI3NlgWAw?= =?us-ascii?Q?4yuEhp1Zu5PSbHsgaOM2/MGChxD8ETWwfpceXKcLunXMDUxxJlSdOXve8Ezb?= =?us-ascii?Q?CBt4wZx9HV0LcFZ3s5JvgKwxqaeriX2GsFXcCFZb+Pn0De2HIVJrHMKAyUkV?= =?us-ascii?Q?/SSQbMeDjhftZiCcwUjhPMewsHzTOW1j6v6G4w42gMiIi5tn94qUFPpqeitj?= =?us-ascii?Q?8JzajW2cvxPMoQRBZ8VLfBKGm5t9hqxCm5xmE3kFxf6J3l5ZHyzpQ9KnVf33?= =?us-ascii?Q?q62GPEFC8KSBC+xHEnOWtJqSHVEvWs0UW3o56iaDndSzFu4q5jX6lxs2naAR?= =?us-ascii?Q?oK3pdxkjbaxJXXknZ3FzYOzrijTE42JS3M9I7JZrztdDt9seJkDy4+qbA30q?= =?us-ascii?Q?Bnuy6PHkUKbJpLwF7a9vTe97eBvXzPaTI9caFbwf3rV01B/kTgjVaf0C6xvW?= =?us-ascii?Q?Dgv/YP9zF7gNHSWy4BDpoLYQfVf/3e6Xt0Vi/P+x2NRRU7IsX2W/lvgfSoxc?= =?us-ascii?Q?1UQ/0xfX+dH0TOMqYtRExQLWi4d4DmPu0I1i6QZofXKRU+Vyjc1XquozAHKF?= =?us-ascii?Q?xZUBHSomEaczhRszl+jpYVlPybNkVeIplZJ5MhTR3t6QNl7DzgUXzTjuVpZ6?= =?us-ascii?Q?xpMofy1kIIKSGzkbdsWRbOPaQOaQdlxojQ2X10gSikJIkloQDu+gbC4LQ5U1?= =?us-ascii?Q?MfLiwI1zpqZdAYLi28UHMshgLDz8/Nmi5nw1GtuL8+lYzxui+ayRiIhc+tHC?= =?us-ascii?Q?9mqOw1Cpm0CVHXS714ZbyU0T2Pb0biEe6PYOw/R1u4tbWJ8izHtkUhuYxajF?= =?us-ascii?Q?rsYTJ46pGOm6CNFBiV5Wvk9eXX1w10DFf1BciLebQRMOFCGFFrlUDlLOR5P8?= =?us-ascii?Q?kp0KWHDWee9iT+KWqRI83h/EA0RBuwbLVHrhFwRJ/WZ8uxVIdEP3TPkFY/3x?= =?us-ascii?Q?vSYyGryqvJKd/jHCD2hTmLU9mY608MaxnZgr3fIAPlfVV78m2AkI/6BIr3+y?= =?us-ascii?Q?YhGxDGw6AvYhvnGtqjbHowvj/ORWg/RglwAMuNwzUHEMNk6XDLFPJwmHALen?= =?us-ascii?Q?67uT4/UvnDqZKC31YeTEvmYcT1/RYylAw+I9jQ/4DDLrY1b4xj4gyg3HFRC4?= =?us-ascii?Q?z388ljzWr9x9mOc+HIyZuZzosA+aPTKZW85yE85/P+01m3JRmbgScGH1zwsc?= =?us-ascii?Q?Cr712C7HNH6hlEhoq5d7JVU4+OuOyPcV3GPEZ+yg?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: c288a4cd-5f7c-403d-a1e1-08ddb41ec0d6 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:33.8243 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: A/LFNHhdERoCccu0DSaruz3+ejJXd7ItDwY1LGnqfIEL0lJjZBYnVVl1eGFHSjHWwkPaVEEEiWkSXW9wpQSZnw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Authority-Analysis: v=2.4 cv=UK3dHDfy c=1 sm=1 tr=0 ts=685c4e5b cx=c_pps a=52DuV3gV4edYeGGkgF1W5Q==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=u1fsugoMv0NfKDGiTLIA:9 X-Proofpoint-GUID: uTKFmLj1ZFri8TyOFeRrAst5n0L6GJhq X-Proofpoint-ORIG-GUID: uTKFmLj1ZFri8TyOFeRrAst5n0L6GJhq X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX7JBtuUh/0X86 mFkmHMpYjEceuz2VMWZI/w3iNHyiD6sZPp8x1TxkQ8MGn/3RLqw6DLjonKgynOuP+Sog8zBcO9T f470fyxAEDHtnb69LcxfGsOT+e/33jPXuVQ7tdjvl0/y7030PGPLkMe5uQlfSfqKcnk7PMhUWQ9 n+eO0Dqh2RwKB4nyxtZC/+Tvc1N6U06pz/P+MGDxno5YhCrwG7La0vBzJYWqBsTsYng9wMoa9D6 YuCvALvXgXm/R+OtuZD+0JXxsMDOnvAxmxYUrQAROu8OXh9OBCA2NjOxsK7leYl4WuwyH3riARm dvXvD6CzGJl47bSy83/9sczhcUmySqanpwMhQIzEJYcx8748OEd3/mM+DlSOcWDKJOoSNzjPJjs YIJnf2I7g3oYUYUIAMQitEIWnFdgP1VnK/femGD+9H757fNcVe504UymC5LO10Pph5oWOSAf X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750879896371116600 Content-Type: text/plain; charset="utf-8" Add support for getting region info for vfio-user. As vfio-user has one fd per region, enable ->use_region_fds. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/device.h | 2 ++ hw/vfio-user/protocol.h | 14 ++++++++ hw/vfio-user/proxy.h | 1 + hw/vfio-user/device.c | 74 +++++++++++++++++++++++++++++++++++++++ hw/vfio-user/pci.c | 11 ++++++ hw/vfio-user/trace-events | 1 + 6 files changed, 103 insertions(+) diff --git a/hw/vfio-user/device.h b/hw/vfio-user/device.h index ef3f71ee69..619c0f3140 100644 --- a/hw/vfio-user/device.h +++ b/hw/vfio-user/device.h @@ -17,4 +17,6 @@ bool vfio_user_get_device_info(VFIOUserProxy *proxy, struct vfio_device_info *info, Error **errp= ); =20 +extern VFIODeviceIOOps vfio_user_device_io_ops_sock; + #endif /* VFIO_USER_DEVICE_H */ diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index e0bba68739..db88f5fcb1 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -124,4 +124,18 @@ typedef struct { uint32_t num_irqs; } VFIOUserDeviceInfo; =20 +/* + * VFIO_USER_DEVICE_GET_REGION_INFO + * imported from struct vfio_region_info + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t index; + uint32_t cap_offset; + uint64_t size; + uint64_t offset; +} VFIOUserRegionInfo; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 837b02a8c4..ba1c33aba8 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -15,6 +15,7 @@ #include "qemu/queue.h" #include "qemu/sockets.h" #include "qemu/thread.h" +#include "hw/vfio/vfio-device.h" #include "hw/vfio-user/protocol.h" =20 typedef struct { diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index 4212fefd44..d90232a08f 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -53,3 +53,77 @@ bool vfio_user_get_device_info(VFIOUserProxy *proxy, =20 return true; } + +static int vfio_user_get_region_info(VFIOUserProxy *proxy, + struct vfio_region_info *info, + VFIOUserFDs *fds) +{ + g_autofree VFIOUserRegionInfo *msgp =3D NULL; + Error *local_err =3D NULL; + uint32_t size; + + /* data returned can be larger than vfio_region_info */ + if (info->argsz < sizeof(*info)) { + error_printf("vfio_user_get_region_info argsz too small\n"); + return -E2BIG; + } + if (fds !=3D NULL && fds->send_fds !=3D 0) { + error_printf("vfio_user_get_region_info can't send FDs\n"); + return -EINVAL; + } + + size =3D info->argsz + sizeof(VFIOUserHdr); + msgp =3D g_malloc0(size); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DEVICE_GET_REGION_INFO, + sizeof(*msgp), 0); + msgp->argsz =3D info->argsz; + msgp->index =3D info->index; + + if (!vfio_user_send_wait(proxy, &msgp->hdr, fds, size, &local_err)) { + error_prepend(&local_err, "%s: ", __func__); + error_report_err(local_err); + return -EFAULT; + } + + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + trace_vfio_user_get_region_info(msgp->index, msgp->flags, msgp->size); + + memcpy(info, &msgp->argsz, info->argsz); + return 0; +} + + +static int vfio_user_device_io_get_region_info(VFIODevice *vbasedev, + struct vfio_region_info *in= fo, + int *fd) +{ + VFIOUserFDs fds =3D { 0, 1, fd}; + int ret; + + if (info->index > vbasedev->num_regions) { + return -EINVAL; + } + + ret =3D vfio_user_get_region_info(vbasedev->proxy, info, &fds); + if (ret) { + return ret; + } + + /* cap_offset in valid area */ + if ((info->flags & VFIO_REGION_INFO_FLAG_CAPS) && + (info->cap_offset < sizeof(*info) || info->cap_offset > info->args= z)) { + return -EINVAL; + } + + return 0; +} + +/* + * Socket-based io_ops + */ +VFIODeviceIOOps vfio_user_device_io_ops_sock =3D { + .get_region_info =3D vfio_user_device_io_get_region_info, +}; diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 61f525cf4a..d704e3d390 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -12,6 +12,7 @@ =20 #include "hw/qdev-properties.h" #include "hw/vfio/pci.h" +#include "hw/vfio-user/device.h" #include "hw/vfio-user/proxy.h" =20 #define TYPE_VFIO_USER_PCI "vfio-user-pci" @@ -103,11 +104,21 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Er= ror **errp) goto error; } =20 + /* + * Use socket-based device I/O instead of vfio kernel driver. + */ + vbasedev->io_ops =3D &vfio_user_device_io_ops_sock; + /* * vfio-user devices are effectively mdevs (don't use a host iommu). */ vbasedev->mdev =3D true; =20 + /* + * Enable per-region fds. + */ + vbasedev->use_region_fds =3D true; + as =3D pci_device_iommu_address_space(pdev); if (!vfio_device_attach_by_iommu_type(TYPE_VFIO_IOMMU_USER, vbasedev->name, vbasedev, diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index b7312d6d59..ef3f14c74d 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -9,3 +9,4 @@ vfio_user_recv_request(uint16_t cmd) " command 0x%x" vfio_user_send_write(uint16_t id, int wrote) " id 0x%x wrote 0x%x" vfio_user_version(uint16_t major, uint16_t minor, const char *caps) " majo= r %d minor %d caps: %s" vfio_user_get_info(uint32_t nregions, uint32_t nirqs) " #regions %d #irqs = %d" +vfio_user_get_region_info(uint32_t index, uint32_t flags, uint64_t size) "= index %d flags 0x%x size 0x%"PRIx64 --=20 2.43.0 From nobody Sat Nov 15 14:11:14 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750879876; cv=pass; d=zohomail.com; s=zohoarc; b=FCUyZDkw3dy9YbnjyC5xCwjpU6Um3KJitSbOtBMwLLHNTU91EGdSV9rUlvH29uxuqQbJp27Z6e65RoYg0ycbL3Ud1ac/z7erk3mObT3TJ/EF4LfveIsOzVQEBOUzAfTFVQ6tFswP4tDFS8DtQhdbjoJKtlzI3/OZq7BVedwb0gY= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750879876; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=W6lVipkzK8AxHpK+9s5/3JtXLdB+95iXdXcHsT/Nre8=; b=ay0r2aDfFY2Qz283tLMQONC0wP7YMPDSoa535K8uZMbhwqjyb7x23d8diMXP1A8PyeXcUbkmmwFGdw1ycETYDHa0Z0SSsh9rZfj5T3kDelH3Yu1PX0ECz6OpXmF5HUKpZxdGzLANV/+0AiXuWxnWlb1ZgV9Qj2KwMi5SYXl9B88= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750879876535161.5874668122458; Wed, 25 Jun 2025 12:31:16 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpe-00082U-5y; Wed, 25 Jun 2025 15:30:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpV-00081T-N9 for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:51 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpT-0003fZ-Go for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:45 -0400 Received: from pps.filterd (m0127842.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PI2Men021450; Wed, 25 Jun 2025 12:30:38 -0700 Received: from nam04-dm6-obe.outbound.protection.outlook.com (mail-dm6nam04on2109.outbound.protection.outlook.com [40.107.102.109]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47geghhf8e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:38 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:36 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:36 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=W6lVipkzK8AxHpK+9s5/3JtXLdB+95iXdXcHsT/Nr e8=; b=OOFZhcNyJUpPyqD/n1AWMukLqJ18/nBkhRhGz/4jgi/IxRVdYUzsdioCu OqtSXM5K0ZmtRx0t4bnUqiASbtXSigJGQ6pQiPic9TVtlPic0sZs0KhmhQ7rMcgT qQfqscqBdVUny61EV5FLWJ39CQYk6WBn1G2FS1onbG/h4Q5GzjWLaM2n1fIiErW+ bUErOH+hoGb/UcAg+NZN0dKMdriHczNV+h7lqffq3nnucMLC6YHojyG/tDd0ZRvI TIPVK5or0FGYWpqTB+6w61GZ2oOCbGIBXI2Li5p40zYzts5A+9n8J/sL2v1M9ZoB iOASdhYg4FP4sqd6d/FVaWLJe7Anw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=lQf+7Wed0UNtIHdmH/jOCZVYEWr4ubZfA45wuvG0OUWt/ZDP7C7WwCgXgHR93xExlR7ShjWnzhMq89V4vsUcN08CVF4tvCFRbGo4a4W3MY0unWiHn5COtT5eme/it6aV2AZ5zykpjY5RoDxG3roSUTbQKKcrs54QknUu/JMwng/zN0+92Gxw1dnc7p70fJs+N5QxTrJHlu79sR8qshvL6pKyM2jU5JoIhA+HKWBC2y/zMSZszBWEakb+lrFoKgNySDVtD3P0IcFhBRncnaZO9CuCoKSiNFU4nC876+pZGCgpd+h3WISVt83EkzgjndDptwAt71HXTXrbtfyfXG83MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=W6lVipkzK8AxHpK+9s5/3JtXLdB+95iXdXcHsT/Nre8=; b=uE6kSEWSdvPLOv/3yvJg+tlG9i2J43ntwDAn2yjFIw0oNi2RU22vfPblrh9l/i+kVfGumnQq4Mk+Fc5PJzoPZwTN/oobk3w417HaYNwl45Qn7Es/EACaCCYWPE2CFYI1T36A9XMPJd6QESxU56a2DpZ74FLjUQQ9HcMDdU9JRxWwpqX/SXTffKDTiLP225boFiNXjyQOTQAQsSIhc3/ixu14ZdnhbhWJTM4qyBfWOKNKL0+H5e/g9F7iazDNXtChpSmXHWD+67CaF0fM9bKOjkc4Vm5AV21vPRxRdG6dXSaF2Zk0EXiBXq6XyOUCtFc26OK0/hnVJfUuywTy8VLkpQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W6lVipkzK8AxHpK+9s5/3JtXLdB+95iXdXcHsT/Nre8=; b=waPddUZ/LYwQ7gc/R6hxSvqbaFt3wN6Pjftv3ukCms7/gwJFYYcRc4DBuJ9ur8mIClMySCBAlVoq2EZc/GtAtfxuwduOLhyKG+4ABEHeibZg/dUzr4XGOhokjY/LndUdDY/z/tWsmj5x5YNog4e6QEhucCBNnRtU3EQ9WNkJq1AT8tnfEjON6Dz1rZSvvZZiM1AzuopF2+NLGheTMzID31qp3GNjlBGv9eKn+dG7ao7Hp3rgjNRdwZnVRoDw5Bjs3YLe3g3nmgji3Dd38TcHpFl4B1Kw/yIJCRj1zZtz8exmOt98Kn/+m5fEGIjm0VmTtGwxdtFdcT6RI2Y9opwrhg== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 07/19] vfio-user: implement VFIO_USER_REGION_READ/WRITE Date: Wed, 25 Jun 2025 20:29:59 +0100 Message-ID: <20250625193012.2316242-8-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: 55a39818-b561-416b-bdfd-08ddb41ec27e x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?MqLCQawTBys3QehlcVVt0lLMiC9NGWxzBwOH8CnhgpeLhWvZyEH150VGNy9r?= =?us-ascii?Q?Gzw/9LcL7INPMT+deCcRtmuMozDg6KTIZtm8i/cOmPPTRn+AGPXJeGkIiAsH?= =?us-ascii?Q?FiPV6jp/+/qcMbyGvmdNftYKiRxkI5oZ1nCkVitTnk01KOB7uqIUywbZUReT?= =?us-ascii?Q?I0q65LJv82qsxQ9MsFK3Mwgq+C7LzMgsJopV1FBdCJP/c2N1p2bUYxdDZJDZ?= =?us-ascii?Q?4MF0fPnbSLbmEjKLsNRQooiyFPnOmlCGH5pibIEkZ/GPXjTfP2q60aMrzFdF?= =?us-ascii?Q?Xewdzv4uchryjC9s9ZxAcL0HWOOkFUcj1YPvikFm7bP7HasCB+mEgeMydN6B?= =?us-ascii?Q?1gs7XkkvfFVxyIYKG0dCkhpeuJMqBISbUVbbKtNdzmKuij1+9Z4zsF44BQfw?= =?us-ascii?Q?cJncITTVe2KDcDHH3ac4DBPmmseNKjl9paPdVCSbjCWEFNLKLc7Klh1218CI?= =?us-ascii?Q?B83MwLxqCNznmWX5QsW8sLvkntgLkhTCz1Fo8rg311aCtGWvVnaVSSzIAIaf?= =?us-ascii?Q?vYyyayZ0KPrHCqI9ji+erbjrLSx59igESUNvPtXCzdqpLiEV05B63K3LEacL?= =?us-ascii?Q?tTTckZ6YlB4de1FTpt0yUUO4TdaObFHox+KXYG3HYRKMgGSnkzf3ftIyDE8A?= =?us-ascii?Q?8vXoJYEhBO8x0LF4gr9Ll+Cqc2jNv7iIcS4406bramrzFOT8IP9sEUlSfPjd?= =?us-ascii?Q?sxW4B254Emw49ha01gQGK7fkf0vWHTZYLSSFU+ayqCqd2VmMY1BvisrvKK+C?= =?us-ascii?Q?Mt5hLRn1PsvBqXBXHR3ZDPOYkk9RZAJ8QLtc92+YoT5xWHf9AwKbanD10U/L?= =?us-ascii?Q?S2HQmRDyJe55fuifL77bJMcgbUEWn7D9g14TiY1gE3GwLPaFHmnSWLJGWopj?= =?us-ascii?Q?YnpDzGqmzrumP9wzz4syJlMpQghIzhzcaJzutCv1FEWe4e79/bktDSdbXyKP?= =?us-ascii?Q?lw3eBKRdmXh5+0md+COApGMTwwZb9Ke32zPfopAWmfevN2QmIshskJRGJ8V7?= =?us-ascii?Q?Dj0t0hwRewAYI/BS54dsPGrmDGVyFHKDDaeNYhmkwiPUJ6tKMa4q94bX45Ai?= =?us-ascii?Q?2O3BbMPY5Wy02V9UdWjJ91pS7lwS+Q90prheOV5SydYQFb0yDYmlAWw8CeoU?= =?us-ascii?Q?voninbsg97v47STs84It3RB0e+Z9dVL7PjEe51KyL5i67N0KXbpGfZWm12G1?= =?us-ascii?Q?586EsCnUc+DVaCO9Oq2hTXhC1mHTyyYJs8L2cyhA96NkIPmdCFVsY86B4CQY?= =?us-ascii?Q?mfJbFpFTf9inqQ1PQmeS7o28l/UGbt0ylhCfrYhfI1UJtmsf1kJwuGChBBj3?= =?us-ascii?Q?SUTzfP911kNNDbezeFWs501cn1EHCTmF3cpObObmpGL2w5PGPF+5mECRM02R?= =?us-ascii?Q?2sWANwu8WybQoPcLP19801SzLuvoi1JaHSG+3EAoJNxIlUq+kvJFFZEjknSd?= =?us-ascii?Q?6zQr7dDivu0=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?2a5ozbPUFxp88eN2KqBThpZxvwzdMM7N2rRp9zHpWdaYuraQBmzdcKJnKyBE?= =?us-ascii?Q?OChecFlbIGhsV50ffRYVk5YZ+l4efvZc7VbhZdm/5p/yvv+NniqTzjXLt1Fv?= =?us-ascii?Q?Vs/hUHjxrMDVq5dYGgUkgZBoB6BHntKswiQ0krVgtBu18NIQs2SiCSclPhb4?= =?us-ascii?Q?f2DX5c1hzx92QXFFII8hI3aDCxQb1pscSGxQDVXjoB9UrRYLztBtVA6Q6kA4?= =?us-ascii?Q?D0mSgDX+lrMYpa0bul0XSMXs7GA06oi8PB1DscK/DAKBIUFjI28N8JRnScH+?= =?us-ascii?Q?Gd3FAQmtFynrPbE/meNjlZOKofHVBw7kIIsai6Ts9Oc+vLZfKQjEzj9WEYL0?= =?us-ascii?Q?jVp7baE3R0dPxZrX1tEHDrR+ACgFGArvzTdLdBrzSjSqbSUW0lS9Izj1hmaf?= =?us-ascii?Q?KLNVbz1TCPwLadnBEYSSGg32F1JsWHmQonsyUtQGp4475NbF/UyGAjudapp0?= =?us-ascii?Q?MRKNdsHLFfp/Zy4CgTdO7Z+4B4bOq46hgoxOYXHQfKeT1nVpAfl7fhjYGq83?= =?us-ascii?Q?Q0obLJ4rj+zTYlgt0INGZB3Opf0PNlNZXL/+Sa/2v7Lsx/MM7fFFOV5raiAD?= =?us-ascii?Q?IIwi+WLnqMQS6u84p7Oor+80xSQeVMK4G6f5ecR6AgvEagAI9fyTDbthnhTW?= =?us-ascii?Q?kHqnMTQLAl0j8vk74mczJhLQS7nJnxXZw9KIy89LayyueQPT6g/y1YfqHQ3j?= =?us-ascii?Q?CDIY/4hc06djq2YEG3IttJYUPt4JdXBjmoHlA+qB4sQVuPmXUE83L0b0sM/B?= =?us-ascii?Q?wzSlOGOXlMNb3eATHjmkzaGdmJSKilqBcH+bqgJRwATd0mzXg2vfKsDBzTz/?= =?us-ascii?Q?9vEekepG48g6hUEwSZod7c/yjCbk1bZTzUZXfds7IGoNs1lB9JsBvKXbiiCp?= =?us-ascii?Q?sexzjLOd/qASIMwrTYEX36KNZhTqEO0gRYpDng1ItJrNdb2X/SvQh6aN9zkN?= =?us-ascii?Q?88+zmcvzF4BA7ZyRlzeEMMw3UYcJp7FRwlGdh3xQRGyoyxVA7dSAXIV6ri1w?= =?us-ascii?Q?RtXCvN+0TM3wL/LKj43QUR7lewXYDUKtlMOIKG9xk7ApmovoaUGcGtCRAy5w?= =?us-ascii?Q?WR1GC5QAvT70UDRYItu8r7tDjU1gmc+oEW2Jb+2ArsClgj4rMEltY0oLM/Xv?= =?us-ascii?Q?6iE8gxbiiHQ0IFfjI2OIjI+cnbKsP8wS47GGulOKBjVqqXr+aMY2ihFzrUOL?= =?us-ascii?Q?eQMVXpmk6uS4n4MlPs1qaE6WASYFeFnqCD/2eOYOGIN2EUgU7DBU1ZqmjmfZ?= =?us-ascii?Q?tZOXG2vVn63FZHHISVti1fP5ySCiGo3X54SMmq+tWxMcL0UcM+dgUCE4vOwp?= =?us-ascii?Q?6uHWchmp9meaDKU/rGCN2s1Rxb3MdRgOJMXAQhiFLWyYdznF/fr/QVE0LcwI?= =?us-ascii?Q?KRMDlMYypfIe6sGH8KwwSBTnt0zpvno9LvvtlT+y4bsvxfENwiIbmxfChBu7?= =?us-ascii?Q?mku43SDhtRE5y7i5PSu3BQAdUCKLg+2EsjpD+xPSbGpgGrNw+KyZDgZiUAid?= =?us-ascii?Q?Ui3+dfeqp6co0UixvBnVsLiaOI1GzxoAD7oO6VyDTLln5wdilKhswZm8I9sI?= =?us-ascii?Q?/ul4eS0/r8n184JRHoIAIy2pxtgNAzpXTVH47uJC?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 55a39818-b561-416b-bdfd-08ddb41ec27e X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:36.6610 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xiLzG5RChjAFTND3MhJ+KTxzRgsn0JNr3deBfKck/HnvR5IJWj55MdYsQuhS6SVYyv6PSCZJ/EEsAS79wQ7g8g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Authority-Analysis: v=2.4 cv=fPo53Yae c=1 sm=1 tr=0 ts=685c4e5e cx=c_pps a=bisBo8TJYI3cbxihtjb+Jg==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=8FWstLx5Ke4EXc4DKsQA:9 X-Proofpoint-ORIG-GUID: cIgbvXX7kWwxNl6iW3zkXJ7pvyxhgfii X-Proofpoint-GUID: cIgbvXX7kWwxNl6iW3zkXJ7pvyxhgfii X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX5tasLUH9S/iK 7Ti1OwJAWzwII+EUXlqErBg0izn4UhAZ011VnHRNWjxeQcs2J9XAoImjQeMgKXn701wLPyXIXh+ zVjpsUF4ZDeI3yZfOLdgYR4P5cq7JhDgTWZd0p06jhtCCNO9EbhlohgmRi2Vm1LL+xpWODVLg0Y qQ4pUhNYVSn5L+Gz0cx68tP6SYxNeEkZ4PN/ef6koPj0JeNv+kujcgin0Uik1pHllnPlZHbIr2k 5nRD9gaylVv6KjaG8VFFGT1b0Vweny4Awpwcb4AmW8OCE+fRensSfY3kYMqrFjOm2tXAhhQ7gMr 4GXyorqwtBRupoCKnDeJgD1rOX0uG6Y5JktolvvA5D+34NNixagOw5Hxc0OFc8OdZi7DKKen68i 5UgQRu9YULDxgmb5CRQX36Q8A1+syC+CV8olFway54uf4bbLWo8JchFFGT25cZkjpW8nr94X X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750879880201116600 Content-Type: text/plain; charset="utf-8" Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/protocol.h | 12 ++++++ hw/vfio-user/device.c | 79 +++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 1 + 3 files changed, 92 insertions(+) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index db88f5fcb1..0cd32ad71a 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -138,4 +138,16 @@ typedef struct { uint64_t offset; } VFIOUserRegionInfo; =20 +/* + * VFIO_USER_REGION_READ + * VFIO_USER_REGION_WRITE + */ +typedef struct { + VFIOUserHdr hdr; + uint64_t offset; + uint32_t region; + uint32_t count; + char data[]; +} VFIOUserRegionRW; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index d90232a08f..4ef6a2d850 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -121,9 +121,88 @@ static int vfio_user_device_io_get_region_info(VFIODev= ice *vbasedev, return 0; } =20 +static int vfio_user_device_io_region_read(VFIODevice *vbasedev, uint8_t i= ndex, + off_t off, uint32_t count, + void *data) +{ + g_autofree VFIOUserRegionRW *msgp =3D NULL; + VFIOUserProxy *proxy =3D vbasedev->proxy; + int size =3D sizeof(*msgp) + count; + Error *local_err =3D NULL; + + if (count > proxy->max_xfer_size) { + return -EINVAL; + } + + msgp =3D g_malloc0(size); + vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_READ, sizeof(*msgp)= , 0); + msgp->offset =3D off; + msgp->region =3D index; + msgp->count =3D count; + trace_vfio_user_region_rw(msgp->region, msgp->offset, msgp->count); + + if (!vfio_user_send_wait(proxy, &msgp->hdr, NULL, size, &local_err)) { + error_prepend(&local_err, "%s: ", __func__); + error_report_err(local_err); + return -EFAULT; + } + + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } else if (msgp->count > count) { + return -E2BIG; + } else { + memcpy(data, &msgp->data, msgp->count); + } + + return msgp->count; +} + +static int vfio_user_device_io_region_write(VFIODevice *vbasedev, uint8_t = index, + off_t off, unsigned count, + void *data, bool post) +{ + g_autofree VFIOUserRegionRW *msgp =3D NULL; + VFIOUserProxy *proxy =3D vbasedev->proxy; + int size =3D sizeof(*msgp) + count; + Error *local_err =3D NULL; + int ret; + + if (count > proxy->max_xfer_size) { + return -EINVAL; + } + + msgp =3D g_malloc0(size); + vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_WRITE, size, 0); + msgp->offset =3D off; + msgp->region =3D index; + msgp->count =3D count; + memcpy(&msgp->data, data, count); + trace_vfio_user_region_rw(msgp->region, msgp->offset, msgp->count); + + /* Ignore post: all writes are synchronous/non-posted. */ + + if (!vfio_user_send_wait(proxy, &msgp->hdr, NULL, 0, &local_err)) { + error_prepend(&local_err, "%s: ", __func__); + error_report_err(local_err); + return -EFAULT; + } + + if (msgp->hdr.flags & VFIO_USER_ERROR) { + ret =3D -msgp->hdr.error_reply; + } else { + ret =3D count; + } + + return ret; +} + /* * Socket-based io_ops */ VFIODeviceIOOps vfio_user_device_io_ops_sock =3D { .get_region_info =3D vfio_user_device_io_get_region_info, + .region_read =3D vfio_user_device_io_region_read, + .region_write =3D vfio_user_device_io_region_write, + }; diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index ef3f14c74d..733313cd1f 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -10,3 +10,4 @@ vfio_user_send_write(uint16_t id, int wrote) " id 0x%x wr= ote 0x%x" vfio_user_version(uint16_t major, uint16_t minor, const char *caps) " majo= r %d minor %d caps: %s" vfio_user_get_info(uint32_t nregions, uint32_t nirqs) " #regions %d #irqs = %d" vfio_user_get_region_info(uint32_t index, uint32_t flags, uint64_t size) "= index %d flags 0x%x size 0x%"PRIx64 +vfio_user_region_rw(uint32_t region, uint64_t off, uint32_t count) " regio= n %d offset 0x%"PRIx64" count %d" --=20 2.43.0 From nobody Sat Nov 15 14:11:14 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750879915; cv=pass; d=zohomail.com; s=zohoarc; b=YbTZzE3ajiMhNbEXDK6230XtCwM2uDVFe1FHE+8at++FZuFt+ZlFq6+8Mk6scS3M9n6b3TL7j+e7QYbRir1Y822hS45iQshQatd2vsvTy3bCObjZjOx2a0OD5ICzcyS+z3Qayz74F9MVcf6bi+OSiN0uuSuPH0+Z4n8DLyLyekY= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750879915; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=h3H3fxLIImpA3+uMFRcbHSmeNviC4QoeybtFHFgYgqI=; b=C8bsX5ICM7QmMpIu4HqPCi9GcqgSYXlKHKDPpCfjF8UDxQwfqTocbgnV4sBjDKuPJMP+Ss88fHnOErnPULIPrzMTru700V6Hg9DQStqOcLQCe6OFlqnyi+i+E9SoeIg/peMqQUOTHDQhtomIum9x/OSuVEL1yd3cyqW6HcKjlDc= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750879915159193.90606041081583; Wed, 25 Jun 2025 12:31:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpk-00084G-0T; Wed, 25 Jun 2025 15:31:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpY-00081Z-SX for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:51 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpV-0003kn-IG for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:47 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PEoMMb018767; Wed, 25 Jun 2025 12:30:42 -0700 Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04on2124.outbound.protection.outlook.com [40.107.100.124]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 47dva0hhty-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:41 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:39 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:39 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=h3H3fxLIImpA3+uMFRcbHSmeNviC4QoeybtFHFgYg qI=; b=xTs7e7MOrWHUaLhqIY/ReFOnSncEcTOcyvD6jGs9J6K6ijHYu7P6niXR/ j2jEuSuMzq3dXmd9fHZh9jf6S1P6q97u4xdbOw6J7RFfdihs94r2ZMbpI5b/T2Kz yWd3tIA66+9pHOqKF3OVkZZvzwrSCdiP0k3b8TSG9Sd+89X+rWOfigdnsSYLIays oiyzqxSDj3de3A6hDVJtVdV/AoOB3tNm4agcFiHH4OxksjU3MXERzjdREG9BcWRN zFBgs1luXe+/9ElB5hq6EWcaXK0oo1D6kVZVnEz0VOHQTxCK3wpA3/tATuhTmbjZ ly2Fw3AQoWOak9GkhjEqGAJ8tyltA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BO+XTAh92NPA+acYK34I8KaHTSIoFx+Kba7hGjG921INJ6iXm56YGpe1WvfFAwnCvH2gWCz7zt+Mv9u3QOqAwGB+sFDNBcAkQ7Tc8GNrcvs7Hdmmz6LHojroCb4iNf8S0adALL1GCEBP64GuBamrZr+AhV6J89GyteuIMyBfROUMkX8QWvSv/Cz2QGETaykpbiw1cNs2f3KHJ/c84/GXqijB/zxdgP9st5GwPnKpsx7TpvALf66KCnBR28LNnDMlJ6BMgFliyFeoX0ethJwAHPNwgJaH8W8ttW7CvwugNjlSLdsG3BJs90142yeKe/9ashnKhAK1BgwGN0nCnMpInQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=h3H3fxLIImpA3+uMFRcbHSmeNviC4QoeybtFHFgYgqI=; b=IwSe+Y7N68xDtF7OFVllVsFUVFeea30ksfIpCUv4YyUcM/eGl0KhHxTXgxnuZj9QHgh9tfqHCzPiUPrPEXdBto7xR5JT1NwgJX/VTGLPznecjPwsqGktu3m9E5PyuH1fUyxlgG0iLvSUfNXoP3c/LTMz2cIVp95ZuCOB+41FaW45312K1APvDv5gpF+YaVY88WHACYQalFFDrxyjJLYbozuU5r+kAK65gZ9Kf5uycF6tvJhWf1IrP60yBlQuFb3TKc2xiV+kr+NxxgqroLL9qEBIufl0i6npQ1H01GUX5Xk73cOVlPVvVWQSyOBAAZhaxI6VSvgrzYVN3/4+CRfMMw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h3H3fxLIImpA3+uMFRcbHSmeNviC4QoeybtFHFgYgqI=; b=YTXxOAhKz71LBQH34yNdxBfUZKyfLdQDOhrVCDa6/AJC7NcuaOBVEIm2XMuveJOgn1UVrCo0J25yyCb0hFLTG3yQxnZnMIvJUsu43tT5pxHN4NNQXwR1eZ8KLtWzM+TG/51OltUrAqPHrHCUH6sdHybm4A+/9QK+CbC2rc4ekC//W/LpUoW14frynqVLeHG6NAEY9dcDCjsWKIqo1fRQqqsT64HdgkizBKF9aIcMK4ZOD2bh/VGEkqrYM0ATzvqNq9Uuj4R1dGwiSq84/o8gwqqLFpFYTkM7W6gh2U5+uVkF/euHzMvkALe4F9zV8B03KCdGKw+N79VC1DDOQMYiug== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 08/19] vfio-user: set up PCI in vfio_user_pci_realize() Date: Wed, 25 Jun 2025 20:30:00 +0100 Message-ID: <20250625193012.2316242-9-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: 3ded493e-54bb-4121-572b-08ddb41ec452 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?OHUggPP6jKOuKt3bZ2eFa9UX1pFkQbu3+4rTLM4dMS8iOVYsssIC5lRckuGO?= =?us-ascii?Q?r8JoTFaSi3C2H/2gWW0BpIZBOWJw4LLe32dPrej7aDx8R2YlUQOMiZzJ0a2O?= =?us-ascii?Q?Xa/nsCTk73ZOK1d+foYczYmSEJIHB5Jo8bHtD7xwlNMYyOfK8dkj2OrCzTYA?= =?us-ascii?Q?Nv9Zpse7FxsVtGuPrcHySMXEFqh3sMzNo67GGKEfEoqk+3gq4tASqq8Elrfp?= =?us-ascii?Q?+2xalmtTpKzTDtphR0vm7XbE5h977scZFQwNVL0m9BxQ4gyApH/YqTWhDtV1?= =?us-ascii?Q?WWrHirJ2zge6r81uJhP5GOYnNT0Sq3t9VBxO27GOoGiWFNGCT6CMTRaKCkB7?= =?us-ascii?Q?++cwNCkbvYIe2GGSIILxSirJzM4srBd4kFAXd3dIY9BiCpN01qI+u4Nbd8u6?= =?us-ascii?Q?rdov4asnUAzHaGoowaIY8frbVQqpyi5nRTXwtmGvF1+m3uWBxOsz5KZYxt8N?= =?us-ascii?Q?US9hjIKjxR3laWDEMpBhC4mhZRq0zlN7q8lKNrEOlpptOxgGcPDR4knuZNIt?= =?us-ascii?Q?FMyVgR0jvq1bgDA0upQ8s+LeAyTR/lqHWicd72iic5iUlEdbKlSgptj8Xxma?= =?us-ascii?Q?lvs6Dfa0JxvqrWaCR/lJBtQ3zT+VXfZiy/uuCAvRqUj+UFYMV9zDrop7vbfK?= =?us-ascii?Q?Fe6FIuHzDUMmaC8Wuj9i05KyL8RIp78ahy7bnIT2hnqQk2z8+mJd+Kredaaa?= =?us-ascii?Q?fwMdvklIaumsXG+8nz4pxx317xJZyeCvFEBQ87XWWNfiWgPhw2qt2K7qSOFD?= =?us-ascii?Q?b6l/JNZfJxHe+oTzR+TYXf904re1ZT0gv0n/ZXT2ixd5HvO5mDazTT9DTnV2?= =?us-ascii?Q?9qJFeanL1gr5bRy4BA9walWio1RiOrCf8arJ+gpc6AZHp78V5OgFDC7yR+dR?= =?us-ascii?Q?uOF3WGEuYE3BcyXSmEbhPp0vBgC4dhUa3JEQzR9kNcCFUIZj8p7B1Alivqa4?= =?us-ascii?Q?3M9gf8owcW5Qt4WmQ1VS+PgmRn8tFkNGAPuA/UM1gyL6b5tH3LTICO8v+YNy?= =?us-ascii?Q?yao52HnB/Y9qIppAm1iHGlpwsJebFQRU4WnE1RfnBq8oZgOaZ5G9Cs35Hsw9?= =?us-ascii?Q?dLOIOziURxT4QfIrvcWQFmqqgaITtvY8dTmj3zSEuwixl6R4IBVp1EUnlUYW?= =?us-ascii?Q?02pQqHgkdcK6X4gO0DVQw2IXvE/79LfbE6heS1xI1PxuMeOVEdB5OdJcDkkO?= =?us-ascii?Q?A+PgnJKSD9PBkMelKwiUC+JoiMFqMbKvu4Tyfs0DNK4ezKBqzBp69Of48dVf?= =?us-ascii?Q?pF+0vIgJ5IWBSb0rRGmiWuJwFCkw9ruJ2SE1JgCPs3LgdXG0GMh9pt4bFJNh?= =?us-ascii?Q?Clk50sVtHmEexjQmDDnP591zu5Vfi3WMCeXBjeDZPqHDj1cusOLlaa0Y0WgK?= =?us-ascii?Q?Z7pOEQJs+EiFOiuWqvv7n+99/wYkEqDIhuzZ5ajXxXOvdlx1rGozniVhs6VH?= =?us-ascii?Q?qPcg3pMEhSY=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?w0oXjO3A8lqHA+xuTqpbZgeVrumOUkiDNB+7jRajPhvtVUhGy0pTpe3HB62L?= =?us-ascii?Q?y91Q8SV2bGNuSsw275Frh/+thKaZlC3NJ4DyxZ6RjvYTurhfipuv8Vstyk5e?= =?us-ascii?Q?eBBx8zt5BurfxP8D7fN8hH/eiwOymUBSdwrOum6ft041pHP0tz4rVRqmh8z6?= =?us-ascii?Q?C9SAkk/FA2X5vdoQ54kDW7Icl1UQ/WioLKkLnUdsDQmHSwJPojMQrF07+2WD?= =?us-ascii?Q?iFesATxjnrdDtSZ73dig6sPrriCctRDRwqUIvyEpx9pF30hP1/rIQPpO2+yb?= =?us-ascii?Q?EMzTtO8kzGE5sNh3f+M+SjVkOUaCmaki87tagUl8U1C5Ysw/synFqdMSDQKp?= =?us-ascii?Q?mmP531sXzPnxbQJdmT/M95+kFAkj7wSAazXZ9dc6KgfNq0KUUUJuiZPy6hPb?= =?us-ascii?Q?CkWp6yzq8DTIsKJbjJhgfOCtAxJ+NiXcLsri/jb3paQLEyqDUjtmrzT4KvqV?= =?us-ascii?Q?8IOQeLYLkRvqt3/pP3A5sjnLft7McdAOXsbEdXDrXT75bTDysKkAD8HkrD6F?= =?us-ascii?Q?+ACNJG+/yp6Ew+ZXql8F+1htzB5xh9EfckDqPnR+LYX5a9r28sO7anJHcg5X?= =?us-ascii?Q?P+3DHm2sW+k8szzyRpjASMe1ePLKHiragO3aYCsrJKaTJ6CyXas3xvlzdFJG?= =?us-ascii?Q?mdQvqUi73O4QX1gpv64Xf3aewDoR3jjBl1MvMkhJ3031e4P+2lyZkw5jy5Jx?= =?us-ascii?Q?k8v0QGM6kPzPOrwgRfdZMBjZbIbYTCp6B6q6vv1RwCW1FZcLAoT0MaKcfE/5?= =?us-ascii?Q?rFn6knbO57C8nfqge6dgfvxUH0IJ589uzrwv9EeKpxYGRYY5hshK/zlUdxy6?= =?us-ascii?Q?XgKeAd5qawPvCW/5r0PvPidp2gYk1f1fePxcntKDnqk1jBaohss8C0ZjNwpm?= =?us-ascii?Q?nlbAV4UXU0xM5bOGJs17rI/u5wZ4BezstrS1a+llQgS0hfWAe76npuUEEw3C?= =?us-ascii?Q?AjE14bF1oYPCVBD31jSLfFzNWo1FqW4IkHLoGz3keeI9fUZ4C33G8xg2wuQa?= =?us-ascii?Q?ER8RekM0q5VElbzzwF7LtCrleGGil/T+yrQLnfZ7NjlgVMUfG/h/cyzmQt6B?= =?us-ascii?Q?ro4mGc9YLANK2/65qO7Nh34B0Y33xVnbeV9Kz9s6SyRFjgV+OhcXEZlkG3aN?= =?us-ascii?Q?CsdhOHfFga08dVj9vPwtBMLNF4g+AC/pACGJ0j43LbzdnQuIWuDVTBACt6t1?= =?us-ascii?Q?51k/WebQ0xeRFdU+4si82ME/vU3/HPOQbINqtOYFuox+OHYF4Pq1amTVyMlo?= =?us-ascii?Q?O+BYUs5b546nVcSoVdIZfp2mhbXUMlW2RD4AVTQYwMWOF5dD14RnM3cud2Zx?= =?us-ascii?Q?NGu4mGMYgn2FE2Lr517xVujUimPFcR8DVkYJZN/wFH0dniCRqeqWBPiM9XFL?= =?us-ascii?Q?/WwmVXZTEHLLe0pbKp0ToDnqTbsbw1zbEcLY+G7Z81go19hKEuZ+5Tw4tiEm?= =?us-ascii?Q?syAhCaA0kBxXSrbHwMeH4bDKQrzOteRJC1AFhpQRO5H2N4DO3HMavbsC3tAV?= =?us-ascii?Q?GB/OxeOZw+kvB5us+wTTQHYly2JxhqzHEjqKr2KT4zitrmZ8oPB9fmfC9X8X?= =?us-ascii?Q?mgzUFgl0LWidKumfQS/noJ4Nof3k+zxgyQSpr0/J?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3ded493e-54bb-4121-572b-08ddb41ec452 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:39.5061 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IQLgiozq/cm+Q5QxfxUS+IkBTFZbZWduYmGDqdMb1sqL3wByZIA6+biFWYLDqLrS9jpsjUX6GFb/lR0Is0ze9w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX7uFldzq7Uuiq Jwa9Cv8S5tcOK2LAgupeBGlnz42+dcBy7M7y3jEugoQSqZCDnKTqytwkyEO2wFI73UrY7XM+As5 Ja4ERuLNtu9mRCdlvOcAQGMTCOZc/mQU17s8fCbHxiAS8Dfp5r7/gNcRS5onhP1Cyk5gjGsroQX Lar5X379j0dKkakPV5FMZw3nF5b+9Nbne4GKYsFcUHgSdVqcyuZG2OU0dIYw1CMQwt58z7i7JGo Vvtxpu9TyWlaxrn9fMLt/JnpFxtTfafqh1YPwmJdSGeHh/NvBNqgxiSlr+2BT5xPUio+nohLK0Q wQqBKcxgu95YdMCOTOt/krVunvf/IxLvGwimfVLkM+b74NkgOT2yy3HwpKlqODHUBeVlybqZBet Evpaac00xiJC4hrcaJjw18EYNAfWyrFpEyovaMWcz8+jgv+WpphY2YsyrLCfIUnhLgGSTQSM X-Proofpoint-GUID: Qqr34IU7pjD4diV3ltgKn_mq4wsGCQ7d X-Authority-Analysis: v=2.4 cv=Qohe3Uyd c=1 sm=1 tr=0 ts=685c4e62 cx=c_pps a=U37v7hwCbtcZYjemgGMOTw==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=uN1uQ0dK97G9gywtELIA:9 X-Proofpoint-ORIG-GUID: Qqr34IU7pjD4diV3ltgKn_mq4wsGCQ7d X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750879916374116600 Content-Type: text/plain; charset="utf-8" Re-use PCI setup functions from hw/vfio/pci.c to realize the vfio-user PCI device. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/pci.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index d704e3d390..b49f42b980 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -126,10 +126,39 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Er= ror **errp) goto error; } =20 + if (!vfio_pci_populate_device(vdev, errp)) { + goto error; + } + + if (!vfio_pci_config_setup(vdev, errp)) { + goto error; + } + + /* + * vfio_pci_config_setup will have registered the device's BARs + * and setup any MSIX BARs, so errors after it succeeds must + * use out_teardown + */ + + if (!vfio_pci_add_capabilities(vdev, errp)) { + goto out_teardown; + } + + if (!vfio_pci_interrupt_setup(vdev, errp)) { + goto out_teardown; + } + + vfio_pci_register_err_notifier(vdev); + vfio_pci_register_req_notifier(vdev); + return; =20 +out_teardown: + vfio_pci_teardown_msi(vdev); + vfio_pci_bars_exit(vdev); error: error_prepend(errp, VFIO_MSG_PREFIX, vdev->vbasedev.name); + vfio_pci_put_device(vdev); } =20 static void vfio_user_instance_init(Object *obj) --=20 2.43.0 From nobody Sat Nov 15 14:11:14 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750880057; cv=pass; d=zohomail.com; s=zohoarc; b=cnX7TJwEJatId/NgyuJlXeVlFYYFOfX7kb7TuQJAoqjjfAsQf/CKNyOz+xN4nOU2MqstBXk4voqcJINDwR7HyHvHat3y7libytKU63aLGhFLKRLNc6/6Fy6QUmjxJjnOWqA8U3KeXpAFoHYrtJ9bAZtsEQ+J3R1WFW4xYRhOeoQ= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750880057; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=T9B3Y3U/nH5NPvxifW0ZsA8iX403dZXpoBuCN9ptObk=; b=Lz/fVE6SiXwHW8Af26FPDhb+rNGi5Brzlm+lZXNMFKzjw3Z70RLCjI791VT8DA9akfcHJfnJ7FJBGoQ6QbwWVJFf7/iXnt8gJgnK0z59IPXaYKb5dG9UA1q8igsRC28tKpwdTyv0dLITAg2JH2WZuuC8WBHegEXvc02eOJOGx/E= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750880057107231.6932766258559; Wed, 25 Jun 2025 12:34:17 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpm-00085C-LD; Wed, 25 Jun 2025 15:31:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpb-000827-Gz for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:51 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpX-0003nS-DF for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:50 -0400 Received: from pps.filterd (m0127839.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PEo27c001328; Wed, 25 Jun 2025 12:30:44 -0700 Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04on2117.outbound.protection.outlook.com [40.107.100.117]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 47duth1k7r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:44 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:42 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:42 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=T9B3Y3U/nH5NPvxifW0ZsA8iX403dZXpoBuCN9ptO bk=; b=Rm9MAbpdeMR6qc2sdzYg4J9hWk+t7+9Kn08jTbK7HU0+XlIqNXY+DfGTm 9NoZ1hyUS+5Tc+dr01K2nI/LwtHnUvw5u4AABIxaXm5qgsuDEX1LXWW4x3pTCqOm JqekS9s3sajhF84bs//9C2jFMTBbyxNfV+MpGjtMSU2Uwex71ye7ZTTmhMQoMqZf mO95HZbaODQ8TKvwj7EN/q89vy0K1gXC5D/HP5PJ/YtjglalH2Ts11GRj4b0vkbt CbAsRrCo/Bs33hd9/lW7sbI5hKwqkrQfoY/+dn43pgYdhJbJUVHxt6lydo6OqRwK E9bptVFbqtcb/JT/Ih+CSdV+FIjSA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=S3IHtTPo/jr8D86WsJsaEBzmaGqhtbe/9LM9hY4d329Ri1T4bCMudsWcI9o9ru1Mu7vyLik8CMu88iq+5oMDEaskqVY+H8kfwObwgf9mnLPhssxP97ZQc9zlbgpbKZaMGB//EDThmqCmMFU24N+pxZ0ZJWLjR1Ismab1FNL450EZmnoYtoYVN8HrCHz+BFmrTgA6rvR++gX0jZYLD1c8TO5kEuJlVv6UoMPAsUhplEoL6TB810Bl4Z2Fv/KNGvbulj81sUTt771bk79QufftGeOm45mZp50qg4yyY7VCgAusPlNXf5LaxhsiJVqsp+j7mKiMiAdR1U8Lr6vufdZObQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=T9B3Y3U/nH5NPvxifW0ZsA8iX403dZXpoBuCN9ptObk=; b=Uhbnh1UnxwoH9Tqe/paHty17uCNTafzUHuKpze/EUILb34ofFVfMfngPko+F/9iQcpXja1TrjD9g6LrApMUiFp0qESDQ3Lg+8EVmlrMz+3J4HIqFdgr6n0YhRAugBizgfk/7pr/8EeTaPrFrmSOrp/Et7Mgz5+AkRPyiGt1DwGfSawaLNrSUlwc2LWvN7s61AU9BA75Npj1Br0zIM1FuXv2Rbh210NHCkGL4v2Qt3lgRSngytW4p+vE2lVl5zi8kERFV8OwSJZKyA//DhdWL1LqN/leoNFyyG6XVV957uooyrdnnS6iZiXiI549o9bOKmbH05HGBj6l8AB04WuRjZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T9B3Y3U/nH5NPvxifW0ZsA8iX403dZXpoBuCN9ptObk=; b=da2gN5QYgiEQxMgYIu2aR/ieWEdgXgBPrS0jtCJ3bWGbitWvNithOPijTiVSAm7zejSiYrjjkt/5E+O+vFGMqGmv7AhsfKOdgeVdSxPls6+8rkYdCecxpGvFWjSrHECoUXcc7+N+0CUpbQoJoHwqA9tjVTOOgmhdtzOnsC4/IKelfsi+7jf5CaKxB7nzfOV0vhJ0UbUhyn//wYWeThfFIiK6Bk6yIAYP+IiK6s7BQcpeb6zITYTjq8mGXQCO27SXuhJYMxW5O6KwK3o7a4V/O66rxU1xwWqcoBFN6kWbgB/sXc41ULEdvLdyWDzrz2uS6Mdh/zHzTzns6au1qq1Yzg== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 09/19] vfio-user: implement VFIO_USER_DEVICE_GET/SET_IRQ* Date: Wed, 25 Jun 2025 20:30:01 +0100 Message-ID: <20250625193012.2316242-10-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: c75e182e-98c1-4100-abc0-08ddb41ec5c8 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?CbS3bF4UCQLnD1o6rfaZT2B9ARhigTgMxSEUj2WyFu9ZU43K/W2LLqy97Inj?= =?us-ascii?Q?8uQ+FxW/pYjWGtmdDIaCR/qWT30Fa9h6V/oVPaZ0kIMSf7idTSdj3zjDhKMG?= =?us-ascii?Q?e9aQiCkTxgQfxkIiLQIGi7Eu0m/TK2IooD4LJTB71JpQ8NWt+2CLQYEtAUXg?= =?us-ascii?Q?ex76SKPewWmZ6AFteDEZF2Jgm5hN5jCABZtFdZdBcg5Gt5trKYjuD01rAgdy?= =?us-ascii?Q?R9OoPuUYn4YPB67Gfl3M7F2qJda3LGcXwz18dKMiBguSm0zokL7cZSvu13Gq?= =?us-ascii?Q?j0cAg5N9FWO7K9pJMtRFPRtjdnx9TTNZPXKgFaZ9lfQjWyladfvDj9aJhW4z?= =?us-ascii?Q?tIhTxspmbI2l+9nEpf0yExzqHyXxXbPeEwRC1/Ho6oMurt8wmLS5sL7C5QZj?= =?us-ascii?Q?abKEydVYFR76Q9+zwPOyBgCgHG2Yt/NUFpOvym4M0jKiffwtIx/6yhxRRdys?= =?us-ascii?Q?wJTnyTu98Di5J4P5CvYDHG3G3p7MqeqYAyWffgRfUTR+1i/gOX0xv1Qc3MaP?= =?us-ascii?Q?btZZwFlzFXBq6cP7Gu2qiAiq4N63fkbDX5j8j+vCPnwhjV+udA1SgPPuUAPu?= =?us-ascii?Q?N8Y+uCOrpsY/UKSIWphXjGyWcQ/pg0Hc4bK3RVCRco+rjGiroj5wjp7ZHdkI?= =?us-ascii?Q?agQ7yuTvmT7nM/KZZGRSV+zAhBoue4V5U8m49khjOO/fVX9ZLn4px7qjKLbi?= =?us-ascii?Q?xQkseAYEXUakQAuCg54VoGuEd7u0K6flLsQaZBAtMrdaOrwdfRpSNe+tAoEV?= =?us-ascii?Q?ZWVrh3QrJqs7SVjxnUSm//8FUbIseQvS7wbTnxcVF3LkK7O4Sob/Yvv6U/DA?= =?us-ascii?Q?t23/EAYqZkqdRR/4qqHOIONuKmwiwXyyJnjLK/zGOj7HUlJqdhIsZX6zo11d?= =?us-ascii?Q?1AYp3xNJpyIJflEo8oDF59X2I6Hl6fvmVaq3W7Oyfe2UbC8rkdB3OSlLGWAa?= =?us-ascii?Q?IaOjQ88fHFgsw3Wo4h2adkZZKcu+9jjsLV4YfhCW5JvwJkV90NeQnm/SO3c8?= =?us-ascii?Q?pT0H2mAxtPLaU3P+CjuW9ue1YALh48Pe/1aCY53YakaD4rqCiRHAywp9+G3B?= =?us-ascii?Q?kGxHsBhB/j/dSifI3Ofj7WIuTPtVyLlINXL3zJEJPm0FE+ML9HGTVdeUSxgd?= =?us-ascii?Q?1javc7KjXq3EHlTQm9rQT7EaFANFYFeVDlUDZ9syeu/FskWpWNcUx2Pycasv?= =?us-ascii?Q?KvI7kqW/1qogKKtE0D3HFnhPjX9Y2dYRzauGqwG5NoYUPsMpVK1DkhV+37ME?= =?us-ascii?Q?6hS9+IQawTwJqKhCJeT2IBJ7rMPlod0U2/WT0Oyl1QA9U3cdy8QGRomNwC25?= =?us-ascii?Q?MaPmc6R5fWeAv1cvXTbjYFU0WRxsGJXfc5dZYIZwH9Ez9+VxockwL7gi3057?= =?us-ascii?Q?INkk0u/O07/nRYvkIuwgRHaRPPLzY8XqDcO7Fmas74W8978dlQ=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?dBAAJxoMTT48jnbwTJgAewBPVEuyT3stFDMzsjQKJCdmA1cutOdIB4tajVTR?= =?us-ascii?Q?pIA74agVUBj+5OoPlYzsp6fcmEIgCe+Cae6cGc4/Uy5me9x22uZm/rKX6KVs?= =?us-ascii?Q?iIMB1faBwBXfPlLp6+u/LxWzhcLz3E32mSEr4AA5Tl0CHxPtPEu7zOawmUL/?= =?us-ascii?Q?G1fquC7gQqbYtv5ELisR9lfXpc9vYp9n6Sy4VYDkMvFxTgYefRfWOolR0liZ?= =?us-ascii?Q?9Baxdd87u9HOC4sTAeWU0Z+6YzamE4xh6D3RR/rnmbyDLALmx+hn2WNcj0Lb?= =?us-ascii?Q?AFpzrtmMnHa18pCxx0BSXCa5/uHF7LCNe4p1eFrz8LF5N6CZvck91Q6Fi4mT?= =?us-ascii?Q?vfTDbDQMJxtZ5bg5C/5PMbG1OnkaFQyXyf6QD8E5b5lskr6enRLJLOSbbatk?= =?us-ascii?Q?LEBGvpcMLkzmUaXnoHPm/qKh9dlyndWF+NvIwnKoo8+QM/2pEaCMDVhloObc?= =?us-ascii?Q?ZiIC2TSlXXq7fati+s7z/iKoHELE7HN7B8EpdO+kU5kSmySqP8x+KDrY7f69?= =?us-ascii?Q?wx0pAX0eCG3ogN2ixRZnkeBqQnXS3+RmOHsNd0elqjONNTFdXxOguO8t8oPZ?= =?us-ascii?Q?XDlXWi+IdJLfhAnDEKoDTlpMqCApTKtLpLB9nCJ0XnsrgqPEx4Jk/HP30xe8?= =?us-ascii?Q?khLknm85pOrOt87N0AF0CfVTsh4skeU+abdEQSC78l/du1ddHahpNWCgyKq1?= =?us-ascii?Q?LxrWacYaEK5xl913qsu6O61meN2jVq/BiMGv4hPIgi1mauOnflmfzUt8S7Dm?= =?us-ascii?Q?M5+KeL3rXLXoxS/0ESnsmdnkUfOt3Ge3uCD6QIdPYv1sC71huHTwtmTJewQ5?= =?us-ascii?Q?8fhm94fHnDTFYf4vbrdXs8lfJFbAhUi4HKbXoOoNJFuwBOWdvCgfSxUkO3ln?= =?us-ascii?Q?QNNuPgGd/5JQy1nF3bVoRkEE1sXfswQBoyva8ixUPSAv7KddDe9ApfS8FKXW?= =?us-ascii?Q?KKkJVIbbGbdYRokIj3eCegcTzijlFMmt05ucQVFqkYeZBvnjBQBR1moy0hwf?= =?us-ascii?Q?an2v16EQZgNdGHECmrRo0gAVnSc3hTYU0/5aiC6b/8gnzF3B7v7klgl5g0Qr?= =?us-ascii?Q?4jgcbKQGnbyTiZ7g+NhYgwYqumzzxdfuWnVJ1iXqSN3F8s4KVN3BeO1gOAaH?= =?us-ascii?Q?2/J4iGiFXxMVlKlngOs6ZXHQeM9bg+n+coFeB442guePETsBNzcQUMMqKDSo?= =?us-ascii?Q?sb9gfVSSoK2j34JjgqkrJuoWcAnoH6U+7NKLYHQ6bCexAHsKUV+XsfB6An/M?= =?us-ascii?Q?95dMrPN+CMCD2s6obfErfj7H1jF6FehrEVcVP6I87SfaWyM4Z578KSF8ft2Y?= =?us-ascii?Q?TmOXQUb/K5JXP47r6t5+I7DivqP/5oW+PuZ9+oWtMJmTzVOMdHe6SukuTA+x?= =?us-ascii?Q?HFRWZ8q63ZhK/GgElrF1zk8zw9uyklEToZ2wpmBMfCsJxs5o4FaBNKvT6dMt?= =?us-ascii?Q?bi40bs5pu5LureLO0QcVkEs8rZUQPKRAX+nkyqy60KsTp02xZUpHNzDP+Dlf?= =?us-ascii?Q?Y3k9OsqKBo4DNbEFiscwC2/7O1MJ9cTFokLtHBq/UDMicheZt5JPpuymySUB?= =?us-ascii?Q?NdfjvXSGkpVVRVjOsDLOoaB6LF/iuQQVaSWGdWEx?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: c75e182e-98c1-4100-abc0-08ddb41ec5c8 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:41.9753 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jaAPFPc908xWnrVerEpxpZ4tV7Cmr6Wk5ga5JumkTGUgAe2//ly3qd61jcipF/0KK56fS7b0gfnoY5mC48pFIw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX0nZ1saCyLXQW dshlLzqx0moHB0p4FKpz3VMumEhfCayCYbe8DVUpuIrrX8NolmhDMFktEnJGw/LgtT84Tpsb679 0qJiedEgLjtD2fU4jWTG5B44hN8bVXuTWGoObo7gVmavtOEAvsbl1vhx6M4lIdGYDfBL669RdUs 2zGPJA5+OXZujP2UygOn9lnVGNQLiNRgakVpktriSc9ufIWkCqoS18scmlqf8XRgWU70TgKLzZO OLkkJLBfYjwjrwC5al+YgQQqlfP1rNrSOyydnCsTgCwLmYslahvk8FjEH+qU0QnEQRetW415odX 4dDr+A89nrDQACt7+b9CZ5rOzfBdGXLGrcJrye39YOKIuLo1nExuZxJEGXQJ8St6q7sVNqm9WtK /wH44zPNOwoCaLIZGQ/J9ww/PJaQ2LNz3zEScOuE+IzKigK7WddkNLwaa7qnTWH+hUKn0pBM X-Authority-Analysis: v=2.4 cv=WNp/XmsR c=1 sm=1 tr=0 ts=685c4e64 cx=c_pps a=35YEAAF7hVpMx76/Bwrsrg==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=mEHtUxI8w2dYJqbWasEA:9 X-Proofpoint-ORIG-GUID: BbrtnHlPZh0-0TTYMp_d9KXB1cP8mqb9 X-Proofpoint-GUID: BbrtnHlPZh0-0TTYMp_d9KXB1cP8mqb9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750880058446116600 Content-Type: text/plain; charset="utf-8" IRQ setup uses the same semantics as the traditional vfio path, but we need to share the corresponding file descriptors with the server as necessary. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/protocol.h | 25 +++++++ hw/vfio-user/device.c | 138 ++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 2 + 3 files changed, 165 insertions(+) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 0cd32ad71a..48144b2c33 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -138,6 +138,31 @@ typedef struct { uint64_t offset; } VFIOUserRegionInfo; =20 +/* + * VFIO_USER_DEVICE_GET_IRQ_INFO + * imported from struct vfio_irq_info + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t index; + uint32_t count; +} VFIOUserIRQInfo; + +/* + * VFIO_USER_DEVICE_SET_IRQS + * imported from struct vfio_irq_set + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t index; + uint32_t start; + uint32_t count; +} VFIOUserIRQSet; + /* * VFIO_USER_REGION_READ * VFIO_USER_REGION_WRITE diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index 4ef6a2d850..f01b3925c5 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -121,6 +121,142 @@ static int vfio_user_device_io_get_region_info(VFIODe= vice *vbasedev, return 0; } =20 +static int vfio_user_device_io_get_irq_info(VFIODevice *vbasedev, + struct vfio_irq_info *info) +{ + VFIOUserProxy *proxy =3D vbasedev->proxy; + Error *local_err =3D NULL; + VFIOUserIRQInfo msg; + + memset(&msg, 0, sizeof(msg)); + vfio_user_request_msg(&msg.hdr, VFIO_USER_DEVICE_GET_IRQ_INFO, + sizeof(msg), 0); + msg.argsz =3D info->argsz; + msg.index =3D info->index; + + if (!vfio_user_send_wait(proxy, &msg.hdr, NULL, 0, &local_err)) { + error_prepend(&local_err, "%s: ", __func__); + error_report_err(local_err); + return -EFAULT; + } + + if (msg.hdr.flags & VFIO_USER_ERROR) { + return -msg.hdr.error_reply; + } + trace_vfio_user_get_irq_info(msg.index, msg.flags, msg.count); + + memcpy(info, &msg.argsz, sizeof(*info)); + return 0; +} + +static int irq_howmany(int *fdp, uint32_t cur, uint32_t max) +{ + int n =3D 0; + + if (fdp[cur] !=3D -1) { + do { + n++; + } while (n < max && fdp[cur + n] !=3D -1); + } else { + do { + n++; + } while (n < max && fdp[cur + n] =3D=3D -1); + } + + return n; +} + +static int vfio_user_device_io_set_irqs(VFIODevice *vbasedev, + struct vfio_irq_set *irq) +{ + VFIOUserProxy *proxy =3D vbasedev->proxy; + g_autofree VFIOUserIRQSet *msgp =3D NULL; + uint32_t size, nfds, send_fds, sent_fds, max; + Error *local_err =3D NULL; + + if (irq->argsz < sizeof(*irq)) { + error_printf("vfio_user_set_irqs argsz too small\n"); + return -EINVAL; + } + + /* + * Handle simple case + */ + if ((irq->flags & VFIO_IRQ_SET_DATA_EVENTFD) =3D=3D 0) { + size =3D sizeof(VFIOUserHdr) + irq->argsz; + msgp =3D g_malloc0(size); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DEVICE_SET_IRQS, size,= 0); + msgp->argsz =3D irq->argsz; + msgp->flags =3D irq->flags; + msgp->index =3D irq->index; + msgp->start =3D irq->start; + msgp->count =3D irq->count; + trace_vfio_user_set_irqs(msgp->index, msgp->start, msgp->count, + msgp->flags); + + if (!vfio_user_send_wait(proxy, &msgp->hdr, NULL, 0, &local_err)) { + error_prepend(&local_err, "%s: ", __func__); + error_report_err(local_err); + return -EFAULT; + } + + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + + return 0; + } + + /* + * Calculate the number of FDs to send + * and adjust argsz + */ + nfds =3D (irq->argsz - sizeof(*irq)) / sizeof(int); + irq->argsz =3D sizeof(*irq); + msgp =3D g_malloc0(sizeof(*msgp)); + /* + * Send in chunks if over max_send_fds + */ + for (sent_fds =3D 0; nfds > sent_fds; sent_fds +=3D send_fds) { + VFIOUserFDs *arg_fds, loop_fds; + + /* must send all valid FDs or all invalid FDs in single msg */ + max =3D nfds - sent_fds; + if (max > proxy->max_send_fds) { + max =3D proxy->max_send_fds; + } + send_fds =3D irq_howmany((int *)irq->data, sent_fds, max); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DEVICE_SET_IRQS, + sizeof(*msgp), 0); + msgp->argsz =3D irq->argsz; + msgp->flags =3D irq->flags; + msgp->index =3D irq->index; + msgp->start =3D irq->start + sent_fds; + msgp->count =3D send_fds; + trace_vfio_user_set_irqs(msgp->index, msgp->start, msgp->count, + msgp->flags); + + loop_fds.send_fds =3D send_fds; + loop_fds.recv_fds =3D 0; + loop_fds.fds =3D (int *)irq->data + sent_fds; + arg_fds =3D loop_fds.fds[0] !=3D -1 ? &loop_fds : NULL; + + if (!vfio_user_send_wait(proxy, &msgp->hdr, arg_fds, 0, &local_err= )) { + error_prepend(&local_err, "%s: ", __func__); + error_report_err(local_err); + return -EFAULT; + } + + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + } + + return 0; +} + static int vfio_user_device_io_region_read(VFIODevice *vbasedev, uint8_t i= ndex, off_t off, uint32_t count, void *data) @@ -202,6 +338,8 @@ static int vfio_user_device_io_region_write(VFIODevice = *vbasedev, uint8_t index, */ VFIODeviceIOOps vfio_user_device_io_ops_sock =3D { .get_region_info =3D vfio_user_device_io_get_region_info, + .get_irq_info =3D vfio_user_device_io_get_irq_info, + .set_irqs =3D vfio_user_device_io_set_irqs, .region_read =3D vfio_user_device_io_region_read, .region_write =3D vfio_user_device_io_region_write, =20 diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 733313cd1f..aa8f3c3d4d 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -11,3 +11,5 @@ vfio_user_version(uint16_t major, uint16_t minor, const c= har *caps) " major %d m vfio_user_get_info(uint32_t nregions, uint32_t nirqs) " #regions %d #irqs = %d" vfio_user_get_region_info(uint32_t index, uint32_t flags, uint64_t size) "= index %d flags 0x%x size 0x%"PRIx64 vfio_user_region_rw(uint32_t region, uint64_t off, uint32_t count) " regio= n %d offset 0x%"PRIx64" count %d" +vfio_user_get_irq_info(uint32_t index, uint32_t flags, uint32_t count) " i= ndex %d flags 0x%x count %d" +vfio_user_set_irqs(uint32_t index, uint32_t start, uint32_t count, uint32_= t flags) " index %d start %d count %d flags 0x%x" --=20 2.43.0 From nobody Sat Nov 15 14:11:14 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750880007; cv=pass; d=zohomail.com; s=zohoarc; b=Og3O2J9OAh3y7e2uNIjGwq+aZLHyZaozVOJogVXIPIKqV4jhuIBDZeaEcT8IPARgIsDnv24L5c51kQyTsgwvPhgxzgAdn+HlzGq5whnz3AXNAuwJIDIb9mWelriRJm1uOyf1KOi08+9F4mWK5DmqlT44hpE1Dlt4ZS8gU8zFATc= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750880007; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=YxMHKKbdXd8YZJD0Iy1oIU3OITBXEu/+2TxCD9ow/fM=; b=bsb7VuK5AgN3zzCpnQaw+JRLjiQLbT73zQPnUlKb9FMZ6nIdCPusShF44i3gBu08OK1FHXTCADD9VkFdQOxzQmmjJG3y+OmGEB9RvPh3YGyMEB4kKT4jkYFD1GpiJ+FnzRO3rXNTOdBabOX0tndzBuIA5u9jSYA4suBq9jHqvxk= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750880007248583.9639750013732; Wed, 25 Jun 2025 12:33:27 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpn-00085R-TZ; Wed, 25 Jun 2025 15:31:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpd-00082K-Au for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:53 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpZ-0003rB-NT for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:52 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PAkUhQ006957; Wed, 25 Jun 2025 12:30:47 -0700 Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04on2120.outbound.protection.outlook.com [40.107.100.120]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 47dva0hhu4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:46 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:44 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=YxMHKKbdXd8YZJD0Iy1oIU3OITBXEu/+2TxCD9ow/ fM=; b=nGZJkVzjINpoaNL/pnlJwAqYc+Xln5wT4e+UpyqLtoPOKha+XPqKyquRE plfhx5qieJ6EDbmDSps7Wi6kvpuc/y0Lo1Z4X9/TNBnwjaWinO2vhozWnOgzkPLE ANPoc/aeGHURQ1A9FlOEKG8RnjVjgx2IT6AbNOSJ7WPaAW/fSJriPn5FXNWt27O6 Wtwpbde94oJHLKmENBGjvwvp7bM+Ux9IC0F+qfqqTwL1iWIpm1HvQlbpBKdP7z6W naJ0uhy2KBzi3hyxhzv4y+Cdx3z184uAQZ0ubbCb79MyTjt5o6s9Wpfp+QM5m5Hz b+tzJkb41PTm0LRemxfyshdWlL4aw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=hD0v6xQIKi2r14CDxOeeBssUAVLsHEuEJB0/RbemAQIcrRIXFT2QBDwr1jqS5ZvstUOLmg/0/3X3eyni/2SrX4GmLcqQrlJWN+PxDEMPRrFg+SRBZLmp2JuKpUYHsVSYEbc+oGoB7QVN5RcUAFz0aV1St26w3AYe8KIiBPNyqfJt58x6ss8j1rGnY+VoXwz+YVys60Nh6YbaQ2mO/LZGAw2uhxo6AwuoeKZGPMiuwDvGRmHSJF/XUA318EewjIR0n3tm0QC3d8O3SVqtbYZPsw3FDhG8NIjMSnRyg6RJ7KFXSpGPG8+wG6/S3H5Eu82+iUTdombAWAY1zCZe1lQMOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YxMHKKbdXd8YZJD0Iy1oIU3OITBXEu/+2TxCD9ow/fM=; b=GG1zbmL+3pFMwcZhM32n+HN5pa8UxtVJhGC3wZrb7+M+UaIQAlvcAKovScZ2ffl9GTICRO+YMd++kHP8DGhUTJ1G2+xWw9SvzRObOAhpkHO2sucMa09l1Hykzrp5yb+/8vfJbgcZOameDtss2K2tDVCZc3/zgu/GqciSFCz75oo9lG0mZ91ezwrJxIQp2AW+EMKiTGP8jvHI9yuUoAH+oHs0EHhsA+rI+RWKRMI2Il9PSVM9/gpASiT/2Xq2LxPciyjq9yhv/WEP5lKZhVEx0O/k74WSQ2I+ljqXqtHtYB0yzMxLJxvhabVy46ZbzAG37cfWMRm/UDNQtKNNRHZ30g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YxMHKKbdXd8YZJD0Iy1oIU3OITBXEu/+2TxCD9ow/fM=; b=onHIJK6PVTh5gPUWZCPP9T20RZeNhNE0HxmXCXf6bJo3BTe0sn31h2L4bgy1dGV98idZ6t7C1JRd0jbA3vgU2Vjg6FiqtFMlugyrxHA7/YhDsMZ8HpEfNgE9sk6oWxUHTpdUpd02WzUt5JDWI4nJzlF1p4ygGTLsUyRXB5o3ccDZHdf1mnv663rbC6lnRxVWh+xdWor+7/P+QRDhQMZmibK9+BQBX7GdpzrFDOkHybtIJ9CX0PTeYbOI/wFfCkDsRt4+mLO73H3iVYfHKxYDG3XmdVZQkCr2/HUy6UA6aLgGzYl+STr80qRblhbqXexPFyabOyGGucLi6CIWSVN/HA== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 10/19] vfio-user: forward MSI-X PBA BAR accesses to server Date: Wed, 25 Jun 2025 20:30:02 +0100 Message-ID: <20250625193012.2316242-11-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: 17a05c8b-e86f-476c-eaf7-08ddb41ec748 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?+d29s85mU2N2/w3NauQe30qfA5FzTiDdtCXniBJp8+QRm9vI4Pb13TQ8iNoJ?= =?us-ascii?Q?2ezKzGcMY9AC2TghGTQaE+0sv404Hb45nJDbEeDibcPiMX/BPQ93kZFCwvHP?= =?us-ascii?Q?OHIQeyCSBTJU7thiZrU7F3NXEsjSF/9q3Y9UPrnBEV2b0vx/X6njPYCUlhQw?= =?us-ascii?Q?U9Y+6V2KzE+fvA9pXe8M/1zSk5Vs1aZ/v69m9aYwoKd5OBhjKXFTxy44+iay?= =?us-ascii?Q?Ezkii2hKCtiw7nG14rKDqkeHtqlhkQs9SIyIAgo9UQuT56s+GG/S51PTnkFQ?= =?us-ascii?Q?WaDzl3yVRkXThiUiImX9KwYSuFDX7j54X+I8VyRECzAohkjiIiBex4amBeir?= =?us-ascii?Q?Ez92k2bv479O+i/oE3n6dONgyk8CcUZErMmVfppdxCW3X0Fc97EzBNPrQu7F?= =?us-ascii?Q?TmDOFOgnv1kwn4J2I/ikmJIYoVd+yWQeDg2V8zPJbctJZAYjDPFVugEgqltM?= =?us-ascii?Q?JmJn5ydEOD4SvtcKh4uUTrHcZz+wWLlw4Rdn73RVNJeE2b1ulMqydqttvtns?= =?us-ascii?Q?lyP7MUDYYpYtkKBl4wwKGlKc24PiVm/1y26Z2jMIg6H1xvc83wogKPoz4lW0?= =?us-ascii?Q?6s5YmQZ1yPOIFWDTej/UNlMiiy/JioNsayYtSyeaYiQigcMuohEnKTFbcXz5?= =?us-ascii?Q?XEA5aY+VZiYarLYYaJWCGvCVdBxGg8tG7PJ0TyrSIBOANIU94hZXBYdtYr0z?= =?us-ascii?Q?FUf4VMwu76tviXo1Ad+t6A31dfMTDM7onucOWZ136qc+bGG0AsHKJNIse1UM?= =?us-ascii?Q?AMyCWasG/wZ2jJE5tkuoE8xjxreJ1ho8u4wZQsLugFTkOhWJcd8xgQ9PxXcW?= =?us-ascii?Q?q0UwQiH5j5OL6xPgPHTxVQpFPlJqd7vys+kI4owjFGHw97s72gbIwmmNG7fO?= =?us-ascii?Q?YzL8cITxVJCjlbXmD8xYx7H/ylIlEVoXHua8YCumkCdMkT7eEPEbFa13C4i7?= =?us-ascii?Q?/RzZD+FLt3dPLtBnnjv0hAdGZLOX63sVAwIiQQMeOV+APGqhDRgKMDsQSv1o?= =?us-ascii?Q?5GKRfYxyp66ti4enw49n84rrGoYuC80o2Xtrh7yWrBmP7XkNPDP+6eb1Xv5U?= =?us-ascii?Q?X5qMCPh6qTqre0kC2knGwo6IXP4L5QImFd96rD/czAHTQMgM8/eQ2NUl7RTQ?= =?us-ascii?Q?eGEDhftluvGxNz8t+T7/ZlZOYhFVBN0Eg76+j7HEJRxJURz5/p6tRioMQyP/?= =?us-ascii?Q?sjDn4zYJEAC1f7bg+nmmalhSYkEWgD3lv4OvlziMpthULwNv1Ol9aSUTKmN6?= =?us-ascii?Q?oI85W9hLua2KAPd2pN4pbtKMNFfLApWCXAVEXzA4B5myBa4UjyzmuBwAd6OJ?= =?us-ascii?Q?JZT9ZxArtX/b2BYUkkJWGGNVKQDJuLa9cODP34Y6XKcJ1OsH5M4uelezWD+t?= =?us-ascii?Q?udNWOeEb5LcHGiCp8sa1ZIWkXWKoWGwawIAVVMpYlS0Isbpold6uqer6w5QW?= =?us-ascii?Q?vAemgKmVO9g=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?0YpjTUbtpGTQ/xtAfEoEI8J2jpnm0a2u6m37vEPVD1eE/wo2BdXozpM6+rnx?= =?us-ascii?Q?yrToWORWlVmhNYGs4+jn3LfR6M1Gffcm45c/jC4HCIzQFJZ9tJknGnYHEjcM?= =?us-ascii?Q?5ED45Cfh1ptTlz0VNAoCNDfRQiNrAYUBypuoCR4bViqMomqL6QC0uEfZU1UL?= =?us-ascii?Q?fo+8Hf8URNwtLUPkdWos1zbvfpjZOlb/EDDwojtlnurXb/utOBYvdf5aX6P3?= =?us-ascii?Q?2E0ZqVO0RbtJBt9ygw+cyKcLEZRxYjiALsWVYh44C1g3tMtVl7lG+kELuGDb?= =?us-ascii?Q?QA7k1FzjLwSmqR4F68ej/2o3+U8YeCCrB3T6TYbxL6Wsj0nh2fMxTIutjP/1?= =?us-ascii?Q?xGDKpbbeGm8XTwj7yh8ryc5KTujK1GJUwPJtbZ9Xp8tCDjg11v9uKI/0+v+Y?= =?us-ascii?Q?Rep5QH1c4jk3J9UQPMA64hwc/tDLxSViMP1f8i3fVDwGI2oRf9zZXfbXkJJV?= =?us-ascii?Q?B5p2BDkp8ZjcWW/Q9m+2u5n+/58gY+D1U6LdcjFn77WUbMG6aZ5SBhkoYmeE?= =?us-ascii?Q?YaosIOPFtoprvCOQr4UmkZvpk2A2MuRsELAzlGphbUI/dDo5EpT20Bs1YHuG?= =?us-ascii?Q?whGobbLDIjgtG2nLq2XFTU3MVSELIBrNjs/H0czNN35EToFCQKZdBzlR52ro?= =?us-ascii?Q?FsYtIcclG8qtnb9W4da63tR/bFqUZRf3ABSM3RdGoubMCbDe78rcHiQYLyx8?= =?us-ascii?Q?ym2iqrMI3rr00imMX5KV6WvHiFyVKbOd5swb/FVKS/oGBvEfKRwiYzbHy6TY?= =?us-ascii?Q?ZSrQYgeTcy1f7mPzP2fdy3kOhYgfzK0eMarwsPI5Oz3SIY/2usXT4pkOYEkR?= =?us-ascii?Q?VlURdR8rkfwf7tMdEENiQ41vANr/HxdGuDHlw4GE+9T1my/+Xq75BkUX7PHA?= =?us-ascii?Q?X5SqHsGxTW/c0Lrh8PQCsMNhoNrol65SJiwsEeNifAPK2QBhXO4voJNyqSxN?= =?us-ascii?Q?rlGIr6k1ZHSmSPZjStVsOhPWBvTCeXmOQRjJpYKKNtjJdUTB4F+5t6YX+pdk?= =?us-ascii?Q?GI9Gp2mXMPTA1851nJMC6ZQ+ZYoVzj4Cjq644yV28WKBC3bWtIf8IqNd0rUE?= =?us-ascii?Q?7V55y8n6nPddL3i6407aJ0JXWTbiRNP+wFoZSmI7kvDqRK/oXxOQ0/YSFw6p?= =?us-ascii?Q?PcP49Vwx/c9Af4Wp44PHNKi9BnLlS3psdNnDa4CyZ1mdCNUXjMgkekCPqRHR?= =?us-ascii?Q?bFBR8lS/7NwdLjk1c2uJht1ASZJv5ucTlV9E+IyeYi29sjc7TDSzXMCSUSBQ?= =?us-ascii?Q?IJWa90Ixduz6PyiPC005iAQk7VTljr/vSzt06MTrM5jqxKCtc4zJZXu56PNY?= =?us-ascii?Q?lS1/aVR5X1NAg3b64nzEm5A4wYyhFKtELIeODJTp9Xm6YvjyMEfdaOuePyyI?= =?us-ascii?Q?8++Czt1BGE7MZWa6cGW0+Be5eFgdiLt6Ap2Gb7cGjKa23T+JKYPOm/4awgJz?= =?us-ascii?Q?zO1R0qVErv65t2NAsX91YGUYcs6MzVC1j+lx812IdH7mFCtwCwVs7A5ACfZr?= =?us-ascii?Q?LJbNf8w+dfnSGu1zLWr40QPYByMe6zk9+oQxTpmNKCNlnCUWEGR4wrw+9FQa?= =?us-ascii?Q?lUUXdr9rOWP5Wxxy7tZ42wd9Vft/XGR6gPo1M0E4?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 17a05c8b-e86f-476c-eaf7-08ddb41ec748 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:44.5322 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: m8UiAGfDoQbIC+GGh4DxPukMjnjtaUSV3OcW9VxxH/YuwwsbFAuGp938LcQNZjldeEUhkOVnlKMTJFaWdDJ5LA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX8XI1IdF2Haf/ dJfMc0NQvwEAq15M7LdZBxRyojX7iBrhlx5bhDSRtU6v1Wbbjo1PCnPUoEgFmpDJYfc4xlCCOXf fE1JW86EQ0uKiQVUCNTj2OEtZ2D5FXR47pJWEB1xWigo8ieTH9pVJGaxMWWuwDR/N5UgYzzy7Xj iyASEJHOid5RvNIXRt3rrz3u91TsTYEgNAHfSZEynf0COVK2oEFAnBlOwpyDJiAMcldG4H/QaJ0 7CurMGJu0XGGfCMtEALUuYRtPdPLRzAntqcnod/hBgZYtfDxdt8BK8e7mOB2Y1pIYluTeIiA7Nl w8EasPACZ0Joov1+JqNA4D+bp6hBM+oM/aomEFBvffPUVaPPKDLzY1gP5oDxTpnr6NJBWp/pDaq MLuYFtQMbCyO3NtSgNGbO0cy1rKOIZwFeBF0n9EVbatWA7b/URtmfjntEIvpCx7LiauhqKD+ X-Proofpoint-GUID: w0zcnjbSueyfCAX1_cVor1apbEXy3C-f X-Authority-Analysis: v=2.4 cv=Qohe3Uyd c=1 sm=1 tr=0 ts=685c4e66 cx=c_pps a=Jz6HH7vbUFBwcxwfI/vS/g==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=P96-NN1LHiVJMZ2xip4A:9 X-Proofpoint-ORIG-GUID: w0zcnjbSueyfCAX1_cVor1apbEXy3C-f X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750880009595116600 Content-Type: text/plain; charset="utf-8" For vfio-user, the server holds the pending IRQ state; set up an I/O region for the MSI-X PBA so we can ask the server for this state on a PBA read. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/pci.h | 1 + hw/vfio-user/pci.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index d3dc2274a9..5ba7330b27 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -116,6 +116,7 @@ typedef struct VFIOMSIXInfo { uint32_t pba_offset; unsigned long *pending; bool noresize; + MemoryRegion *pba_region; } VFIOMSIXInfo; =20 /* diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index b49f42b980..c0f00f15b1 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -24,6 +24,62 @@ struct VFIOUserPCIDevice { bool send_queued; /* all sends are queued */ }; =20 +/* + * The server maintains the device's pending interrupts, + * via its MSIX table and PBA, so we treat these accesses + * like PCI config space and forward them. + */ +static uint64_t vfio_user_pba_read(void *opaque, hwaddr addr, + unsigned size) +{ + VFIOPCIDevice *vdev =3D opaque; + VFIORegion *region =3D &vdev->bars[vdev->msix->pba_bar].region; + uint64_t data; + + /* server copy is what matters */ + data =3D vfio_region_read(region, addr + vdev->msix->pba_offset, size); + return data; +} + +static void vfio_user_pba_write(void *opaque, hwaddr addr, + uint64_t data, unsigned size) +{ + /* dropped */ +} + +static const MemoryRegionOps vfio_user_pba_ops =3D { + .read =3D vfio_user_pba_read, + .write =3D vfio_user_pba_write, + .endianness =3D DEVICE_LITTLE_ENDIAN, +}; + +static void vfio_user_msix_setup(VFIOPCIDevice *vdev) +{ + MemoryRegion *vfio_reg, *msix_reg, *pba_reg; + + pba_reg =3D g_new0(MemoryRegion, 1); + vdev->msix->pba_region =3D pba_reg; + + vfio_reg =3D vdev->bars[vdev->msix->pba_bar].mr; + msix_reg =3D &vdev->pdev.msix_pba_mmio; + memory_region_init_io(pba_reg, OBJECT(vdev), &vfio_user_pba_ops, vdev, + "VFIO MSIX PBA", int128_get64(msix_reg->size)); + memory_region_add_subregion_overlap(vfio_reg, vdev->msix->pba_offset, + pba_reg, 1); +} + +static void vfio_user_msix_teardown(VFIOPCIDevice *vdev) +{ + MemoryRegion *mr, *sub; + + mr =3D vdev->bars[vdev->msix->pba_bar].mr; + sub =3D vdev->msix->pba_region; + memory_region_del_subregion(mr, sub); + + g_free(vdev->msix->pba_region); + vdev->msix->pba_region =3D NULL; +} + /* * Incoming request message callback. * @@ -144,6 +200,10 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Err= or **errp) goto out_teardown; } =20 + if (vdev->msix !=3D NULL) { + vfio_user_msix_setup(vdev); + } + if (!vfio_pci_interrupt_setup(vdev, errp)) { goto out_teardown; } @@ -192,6 +252,10 @@ static void vfio_user_instance_finalize(Object *obj) VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); VFIODevice *vbasedev =3D &vdev->vbasedev; =20 + if (vdev->msix !=3D NULL) { + vfio_user_msix_teardown(vdev); + } + vfio_pci_put_device(vdev); =20 if (vbasedev->proxy !=3D NULL) { --=20 2.43.0 From nobody Sat Nov 15 14:11:14 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750880095; cv=pass; d=zohomail.com; s=zohoarc; b=T0zVsWJw6OAIs8FS3GEiaxjjlQSIO9sPOy0+Wrpi0lYHxVqM7q+DsXq76IaTPkDcK9b5Hce36154f5dxqeBle7XHpQwDM+abwdsV0kHjMz4v2QrLu7zqV62sIZKTvn9A6E4t5ED9y/hdQcu+kEUOyXItWZP6gevUD5KatSXHkY8= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750880095; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=kczgc2ZtSuPIBJchr98XRymjdmJ6lrkKDvrR3yOtVPY=; b=E+KlGAQkkF8m+w9LnfAnwmVeJ1xiPDQmUoTIAMiVagRGtWFiJtDV33vqlmhBmLtknycOe2j4ryF9sK3yErqqf9Ey+Q/qIBhzl0bL69oNBHJd1GZLpX+4euG1CJlKv8hqZhft4JLzFGBnblVDlLOE3hYxqTyDvPi8zri36OZrCyY= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750880095426567.1087661006381; Wed, 25 Jun 2025 12:34:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpm-00085B-L1; Wed, 25 Jun 2025 15:31:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpf-000838-7v for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:55 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpd-0003uo-36 for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:54 -0400 Received: from pps.filterd (m0127837.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PEr5Z9008669; Wed, 25 Jun 2025 12:30:50 -0700 Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04on2105.outbound.protection.outlook.com [40.107.100.105]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 47g7uuj2e6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:49 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:47 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=kczgc2ZtSuPIBJchr98XRymjdmJ6lrkKDvrR3yOtV PY=; b=T1YSGNN/vFKvQRNllSJPquZb87zsnoJEa+FkaUlK+zuMnw0dep6XPIGZS ZYAAMxmk5zFv7MFDDpjJowN5GWGkRnP09Mq8mWEe1Kf6eTaObrPWcn66UJs1jeNp 8oj94pouLlnt26gXW5CbpV9QVU/yZOZgSMClLWFUWcS23ydnab1R3RD9SptTXKOo m2bRhgw2eR3y5nqo4yb2eLC2NxxEKkLZuN4sGLsdL1+Pb9BGIC6LYI+rImv9BJeg RvTydhCkDrwPiHxTI1k/BCyFxprBu2cmgfc+DCvaQ/LOcJJbfo0OesydHe5gMO9k Pjuj6buwD53vb6rERfduKIPWW3FtQ== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Sk4Z2op4G1UGZqAExoPXME2YDiLAf8fdluX/77M1arosF4Czxja4/iBe6YV1txWmdzjUsMwKdrPLJOUbnayofx6UXLiu8acX8eCO5Ua2yFeNcGlzp3eghxuiTeN83yfdEFkg6a4BTeDqi50oMBSt+MdMvZl8/X9HHnq80zRQ1LH73Kgdr8TdP79cIQ64qXgxzjLMaru/zMhHpb5yPGiIp9o1Z5TMGC8RctPRfQ6d8l6cnnKu4i1ZMMeXLfPknhhMfO/e/ejmdA6fU06scGQIFmVcNyvyjzF9dzAoodRWYD/+TQQRG5ZC8x8jAFzcTEqhBousY+Lr0ZTx7zQdufS2Xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kczgc2ZtSuPIBJchr98XRymjdmJ6lrkKDvrR3yOtVPY=; b=GwTFjht+A4fuT/av1fVRENxUvPz6mtxS6lGV9Op8SbFsILJB0cDO3Md7SP+KugO7pmYDZ6om2aGymzW9E+guH73PDnoOR/ZDxgnPT9w+sZaKYhZas99f4FSM81Vdc9Z6mGeEokCQ929QOzFNDeLzEY0pW+0xJEfzysf0Ntdx6pQ15XapEOBBWEJCH3vZrVBULUZxKhFyOTViaNCY6YhehONba2O1sjpfv8M64j8JFH6ikxcKZgR0ahPJxWgE3oexkE3204KrgOorRaM1u6FbWI5ioO8JR8/hupVx3is3/xhhGI75sBbzEa9XFkAJ+OqFwV4E70Xzf+pSoTb436BgdA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kczgc2ZtSuPIBJchr98XRymjdmJ6lrkKDvrR3yOtVPY=; b=CgTrPI04K8i6HcKim/7HcOl9jne7krWXuJZ+AsgxlSk8mh4q/5riPRfAQEZTpbMEaxt4zPKuhiHdWT3nZvNc/z6WW3bbI9N/5+tYtehyRm0QInu+nSFUz4YTyRlG7sqkC+FnqV4gKfFvHYbg0ydsPeOMwVgpnlbSqkT530p9d4Fd0Byfns/DkmuCZV+5ksC7T7bddP8l9xuxr6cbAUbKNHE04TjZj8h0IRUgzWC1W0QEWvj+vDTbFqBWKnZjwlZHPKs/peS3+I6pqeR5jaK72pQ2Liz90dna2IzI7WFPKJSHYO/e32J0FYE5EfZmkaX9Jaq0N/1NRP/o1BimowwYYA== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 11/19] vfio-user: set up container access to the proxy Date: Wed, 25 Jun 2025 20:30:03 +0100 Message-ID: <20250625193012.2316242-12-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: 2cebe217-4a01-4087-9dbb-08ddb41ec8e6 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Zmz3/YNW2Kd2HntTmt/6PEzph7MCgDB861tLqnW4LkhKZ16pMLdwIHz7KYOX?= =?us-ascii?Q?39mTtEqH3sQxU39QhPQpBGpyuDc4d3zVIu64helLpISP9l3SRr2FP3CP9cL9?= =?us-ascii?Q?FKYurz596pKLRnvW3OE84doSeknRNnb0NpvGfRwvFvPI+eI3+PfQQQTQQtMo?= =?us-ascii?Q?RhpfNzzZUEGVL6+Z0f1M5ObkBX9fEYiFE/KQkLcRPGJvAJT8C2PsdfAZKvlT?= =?us-ascii?Q?O7xDncMiqyf0ymvGaxk7EkO0LDCBZ7MlukC/tRyF1W80MicFfXebVdW/spv7?= =?us-ascii?Q?nsyzNLio4zMeX8yppz7ZCI+8bTRNXvD4QsMmryzrIP10zbUUfuycMDvWt5oS?= =?us-ascii?Q?Vy19OaZ9psU/LBshY8A/GlrTg76OiIONDf2Yu6ZhzxWv4gbP551p0sBTdEyZ?= =?us-ascii?Q?lwR4iLMhrR3grr4faAAnYSxccNC/W+PKliVeZhhG9dRMhuEQQsaZqYSyytsg?= =?us-ascii?Q?QAgocLCeT7qZ2cONlFTSLqNiHOlPV/sIrfhLvIrVL9bmyixpxBXS0tVI+t1O?= =?us-ascii?Q?P52MEYmMaJcdGdZGronqsmXuqcp1p7JzF4EKaTjDA3knFLR2kVa6sPNCvUhP?= =?us-ascii?Q?+hLksAnCdYv+CGLyXobdBwgpD9fWdtYjKjivZ8A4VNvS19a6xu2k+vHRKS9L?= =?us-ascii?Q?Glh/BVFazChhNibYWhdFdYK0CLzajZEZFvzH9CjfbyGPQD5B+MIm3JLYXRHI?= =?us-ascii?Q?a6KnJyPVCbsmEbt7abYMXhQSS5Q3rWcjWv8/SXwCzqzLGYP/aiuXJuO81Xkz?= =?us-ascii?Q?BSg+mocP0vKw5icQGPmI5IgHPZLGghgK1ix8n/Z08JRhJU6kXXDRWEKOdrTw?= =?us-ascii?Q?ExtwWRAxcmhTlUTbtC0p5waeYaA55+8lzOdbxR72vVljKW5WBqBxJz+qT0rE?= =?us-ascii?Q?b2gxyGfDrP7XycTWAvYj40Pq79kWtbLN2dV04TBlZqZMm9t++1nIOCRff7+O?= =?us-ascii?Q?PCJdev/qDjI7e8QYxNmYxHcJuAQqiOnI0Pw7UNJCZrQVGzI2jwNN2Qiy6ivq?= =?us-ascii?Q?fDSph8x/My8nf2d71JSo4YYPccspe9sIv7qytoeBjX/RFNpNwjRm8waKEp99?= =?us-ascii?Q?2owG+ggRFtqoJ+oEnfRE8/59fQC4PJiHLkKSidF3e2t4UoqICn+g0Y2S91Hf?= =?us-ascii?Q?vPU8yWz6q/bixd7AW060yg/3ghoP+peKaY9qMrbTnTaUZjqTCSqiJRB5t5kL?= =?us-ascii?Q?rxNV+vH5xFmo5Q9YOmU28vZ83ZbHSOPB9EnEhIlX18Eog8bfj/GtjFwkTMvh?= =?us-ascii?Q?maR7L+ahlCsw/862WaX1Oo2PO8pTWCr6wV0hLSXqaX+IE3k7hel7KUnbRQo4?= =?us-ascii?Q?QQ5VvMcMhXSrt7XWpRRSx/1CvrOQca1udYEH/qj1rwZ8n0UNDb1iR0cXP7YE?= =?us-ascii?Q?ydABDnY5AitKJmcVds9HStIIPTRLsMxeuVWj5Wht92tuZXYFOO+jZeJ8473O?= =?us-ascii?Q?j1cV6RJKTko=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?1tIGZs33ibtG6BLdkAWOq86kthAqLD2+X5wJCu0BFfHQecW56cXmFtgFKjHm?= =?us-ascii?Q?zO8T9QKHDHspcni62vVpcb8lWpc0fToVv/Ct/JF3kP2DSHrdZpne0mJXoJTP?= =?us-ascii?Q?7SiygCEIfMcQ0IUYAQcYzFYdi/yBlJrlxawpOqOlmh76mPTknifdMlsb7FAo?= =?us-ascii?Q?AJVEklPsUc3eI+b6KDtriLwkhhLEQf+LRZCE3dHNUhKn/mn9gC/SGEXz7+BE?= =?us-ascii?Q?nkY2Hz9bZPaERSW2UCGAPR4q8xHnFYtHi0K9MBBzwlg1N9oYoctun3Zo8RGr?= =?us-ascii?Q?06qxiKlmHqhqfoCriiophajPgWjxPW5btXTP3aBfPj4POwtfA/QfmjyhgSeT?= =?us-ascii?Q?ZqVszDWexGwtuLtYsIchrjnuiiDkb8x6VIcvlBoNli/89mRvteVDUMzDe3Pv?= =?us-ascii?Q?l69+b+aaYzKSwXpo0nGO8LQEvl4gUlqhnupfwrTEFA3kbm3Vfnu3c2rKRxqM?= =?us-ascii?Q?ORl2uD+2JTeFkZ2dWk8iZTD0lGgmQ3z1Kxy8dyvh9FjHM0BPYW4Uk3jDm2DB?= =?us-ascii?Q?0HFli+r5SoMVer9BhRHA1eE3cSChT83d6J4v9WQOJHh7qzpkANJuIPkW56oM?= =?us-ascii?Q?CCAl2gilzNZjVzwNA0SC3OLXZK+c3+Z7iYPRku0VxAKYc1efo0WkdU6xdx+I?= =?us-ascii?Q?aIL78dCe/CTQb/BBzr2pkwP+hLxqZWHaAG5Q9tobe4IV2RE3xH94zWjU+xhE?= =?us-ascii?Q?M63BYq9OXlgd7hSMAaGRc1VnjHYN+045aQjjj7nO56iWZrWx3ffzSy/4OgiM?= =?us-ascii?Q?PNo4M/a2XqWeJfYIV0FE4b+uytfiMmXteQRg0+p1zQP7Xc6SkFEKU9ZfOi3L?= =?us-ascii?Q?22HzQfXLH0tSiWriXbjD7ANAEW3VU3SWvnhNy+j1jU+mKcCaLwuAyDipK4AL?= =?us-ascii?Q?Rf6TSU50KGcoHQcm7tfS4TAFj5HATrzkCAkMDHnMm7c6yV8xa8X0eeohknxl?= =?us-ascii?Q?8IXTn62wCslA3qqpoDO1cbVggKhaTk/jM1jhGQ5DTfF17I1oCXjzTTYpMHjO?= =?us-ascii?Q?Qn0/rXClDvj3SOCJbpFZMEH5LclrfVpvW2AyKZYnuK/d8U9DJ5i+xhqxyUKM?= =?us-ascii?Q?wqZ2QQj5/iyTFcAGrxDGPOLAqT0j9V9UoIdx6YjWimYiL6yMlyV8+aywk5tN?= =?us-ascii?Q?5Iy4TIpCpCEYAsED3qfFcmniJSJ3qAAty6FDNy8n1SZDDBzHFl4cuFt7fgEk?= =?us-ascii?Q?9mVYfpLsmSsRBV4IbezwzvLz2HKHr/q/7UYR9x76g6MySd9lRmuBtytBUogT?= =?us-ascii?Q?7cd5ZeEuNP9dfeKAYKZtSau/bNoNb6JRmivosgea8f4KM85asz4gZEh7NK3t?= =?us-ascii?Q?kUQ6pVPKsTOHdnU3T1HRWrdIOiX1RFYBpgygWWqqAExe0hjauczyLcx+HTSQ?= =?us-ascii?Q?iaGFnNnh70Xb6KGvndjaEdGquARMRb7i2wUEWQBV1bcKMaI7nHGyQvDpUS/1?= =?us-ascii?Q?k6e9lfAn5POCb8OVxzGAl6AMfZra1jT73ktvOzWbeSKD6G4sJUJixXfVERyZ?= =?us-ascii?Q?AQeBq1vaflWQIjRfV38TuO0XY+BigBg5Fw33eYx7txrOTFQibgyWJOLzlYDq?= =?us-ascii?Q?rIC1cIgIgix8i9ZXbdLCep9NRNu8edL5resJ69fB?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2cebe217-4a01-4087-9dbb-08ddb41ec8e6 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:47.2764 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +ulSmE0Uc6IdrtNZLEu/0VXL53VMriphvPQFTI/UW7WFpm0Lfrefi+LGrKWdPL9VRT3+Xawf3sTUzWlSu+gvtw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Authority-Analysis: v=2.4 cv=Xt36OUF9 c=1 sm=1 tr=0 ts=685c4e69 cx=c_pps a=HdAMwazFWLWhmZ51wATH9A==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=7VGmajfop4gqLxerqI0A:9 X-Proofpoint-ORIG-GUID: HO-bE-bsZIxwnCXHeUzXEzT0gmg5Q_f6 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX8PvEOpjMw5lk CvB1afz1vGUiwPs/KmdNkEfGK4JRS5ah7RXwduqK9QpzgqQZcrfaLpfEgAdy1BRrMRiFUV9qfpN v2pYB40wrpvLOQAnMM0r1z6NRVxBRfoTO3AYwa9oZPpOIPHjk3AIu1IElBw1VVfgyschBTg2ky2 XOIvYcYCQdbnEgA7MV6ECDDO3HD3EzEMJdHgPjhq6QuoFhOVpWWqUxP47p21jxhQ/NawllRTXFZ Z5991yO1w/6kxPf/3uWE28dmRiS9LG4N9M3ZUevaAp7viA+v6Cw1tvsA4bA5M1RyU/+BSWyJ08A G1IuOHIeoX1uaeP9vQOngMgIgT+NQINqX/oaFnIbI17RpgFP8OwnOxI2X0cSKdfmbL+FufyD4Gh DkiQTJmobgb1jbx73y5+SF/W/mg5SgpPqSraEiT2hMHG799wxI7TBhHPuIdQGsXxphmsgE7P X-Proofpoint-GUID: HO-bE-bsZIxwnCXHeUzXEzT0gmg5Q_f6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750880096823116600 Content-Type: text/plain; charset="utf-8" The user container will shortly need access to the underlying vfio-user proxy; set this up. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/container.h | 2 ++ hw/vfio-user/container.c | 43 +++++++++++++++++++++++++++++++--------- 2 files changed, 36 insertions(+), 9 deletions(-) diff --git a/hw/vfio-user/container.h b/hw/vfio-user/container.h index e4a46d2c1b..2bb1fa1343 100644 --- a/hw/vfio-user/container.h +++ b/hw/vfio-user/container.h @@ -10,10 +10,12 @@ #include "qemu/osdep.h" =20 #include "hw/vfio/vfio-container-base.h" +#include "hw/vfio-user/proxy.h" =20 /* MMU container sub-class for vfio-user. */ typedef struct VFIOUserContainer { VFIOContainerBase bcontainer; + VFIOUserProxy *proxy; } VFIOUserContainer; =20 OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserContainer, VFIO_IOMMU_USER); diff --git a/hw/vfio-user/container.c b/hw/vfio-user/container.c index f5bfd54316..b4a5a840b0 100644 --- a/hw/vfio-user/container.c +++ b/hw/vfio-user/container.c @@ -49,15 +49,28 @@ static int vfio_user_query_dirty_bitmap(const VFIOConta= inerBase *bcontainer, =20 static bool vfio_user_setup(VFIOContainerBase *bcontainer, Error **errp) { - error_setg_errno(errp, ENOTSUP, "Not supported"); - return -ENOTSUP; + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + + assert(container->proxy->dma_pgsizes !=3D 0); + bcontainer->pgsizes =3D container->proxy->dma_pgsizes; + bcontainer->dma_max_mappings =3D container->proxy->max_dma; + + /* No live migration support yet. */ + bcontainer->dirty_pages_supported =3D false; + bcontainer->max_dirty_bitmap_size =3D container->proxy->max_bitmap; + bcontainer->dirty_pgsizes =3D container->proxy->migr_pgsize; + + return true; } =20 -static VFIOUserContainer *vfio_user_create_container(Error **errp) +static VFIOUserContainer *vfio_user_create_container(VFIODevice *vbasedev, + Error **errp) { VFIOUserContainer *container; =20 container =3D VFIO_IOMMU_USER(object_new(TYPE_VFIO_IOMMU_USER)); + container->proxy =3D vbasedev->proxy; return container; } =20 @@ -65,16 +78,18 @@ static VFIOUserContainer *vfio_user_create_container(Er= ror **errp) * Try to mirror vfio_container_connect() as much as possible. */ static VFIOUserContainer * -vfio_user_container_connect(AddressSpace *as, Error **errp) +vfio_user_container_connect(AddressSpace *as, VFIODevice *vbasedev, + Error **errp) { VFIOContainerBase *bcontainer; VFIOUserContainer *container; VFIOAddressSpace *space; VFIOIOMMUClass *vioc; + int ret; =20 space =3D vfio_address_space_get(as); =20 - container =3D vfio_user_create_container(errp); + container =3D vfio_user_create_container(vbasedev, errp); if (!container) { goto put_space_exit; } @@ -85,11 +100,17 @@ vfio_user_container_connect(AddressSpace *as, Error **= errp) goto free_container_exit; } =20 + ret =3D ram_block_uncoordinated_discard_disable(true); + if (ret) { + error_setg_errno(errp, -ret, "Cannot set discarding of RAM broken"= ); + goto unregister_container_exit; + } + vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); assert(vioc->setup); =20 if (!vioc->setup(bcontainer, errp)) { - goto unregister_container_exit; + goto enable_discards_exit; } =20 vfio_address_space_insert(space, bcontainer); @@ -108,6 +129,9 @@ listener_release_exit: vioc->release(bcontainer); } =20 +enable_discards_exit: + ram_block_uncoordinated_discard_disable(false); + unregister_container_exit: vfio_cpr_unregister_container(bcontainer); =20 @@ -124,14 +148,15 @@ static void vfio_user_container_disconnect(VFIOUserCo= ntainer *container) { VFIOContainerBase *bcontainer =3D &container->bcontainer; VFIOIOMMUClass *vioc =3D VFIO_IOMMU_GET_CLASS(bcontainer); + VFIOAddressSpace *space =3D bcontainer->space; + + ram_block_uncoordinated_discard_disable(false); =20 vfio_listener_unregister(bcontainer); if (vioc->release) { vioc->release(bcontainer); } =20 - VFIOAddressSpace *space =3D bcontainer->space; - vfio_cpr_unregister_container(bcontainer); object_unref(container); =20 @@ -163,7 +188,7 @@ static bool vfio_user_device_attach(const char *name, V= FIODevice *vbasedev, { VFIOUserContainer *container; =20 - container =3D vfio_user_container_connect(as, errp); + container =3D vfio_user_container_connect(as, vbasedev, errp); if (container =3D=3D NULL) { error_prepend(errp, "failed to connect proxy"); return false; --=20 2.43.0 From nobody Sat Nov 15 14:11:14 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750880085; cv=pass; d=zohomail.com; s=zohoarc; b=P31lR5XiYRkpDWjwm8WB5b7TvxGQ1RUTADwdc4slSpbG2Xfal1tzzEt6R0QsyAL+4+8drWyw4klyB12rHezgHe4zW/O2Dm9MGbgMYt69BvjfGIZBNXCv/Xp6qLFggq3FKtZsGpjbBXwzvlq1O/C5r8p5zXPd7CCILW/sPC8yQVk= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750880085; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=8Sgj0ZLqfnMV1rmzsLAXyKEjHirJgruqwfW3PMUXkTA=; b=Z5DkUFOh2nJXZOz+df/S9fin9VcYfak06jAorHu9uyKd09oQAF6/IcIOQKYw8QVtlBe4SvvdAd+ixecnmAOW/r27fHTMqhR0zPqVBk9ea9vcii1fTUro/FnGg7tQu1auxLIXTFCclQGs/0jITMBK0pxHDq8FamhzG1sJTAbVTNU= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750880085291653.9559983587013; Wed, 25 Jun 2025 12:34:45 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpo-00085i-9Q; Wed, 25 Jun 2025 15:31:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVph-00083v-R2 for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:58 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpf-00040R-3A for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:30:57 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PEoMMc018767; Wed, 25 Jun 2025 12:30:52 -0700 Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04on2107.outbound.protection.outlook.com [40.107.100.107]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 47dva0hhua-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:52 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:50 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:50 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=8Sgj0ZLqfnMV1rmzsLAXyKEjHirJgruqwfW3PMUXk TA=; b=mpilCHA8+PIL8k+Pl6bBYt6JkxsUTkt496g9aSCNsvsuB7630laUtE7mk G50t9ygydvKbxc55eiHbEU9MHJV98aJv8otm5qKG4aThvHo7Yuw2qUdoKJHiYrxm ofRPjOmYwjmCqenRdPfyRNZ7sNxx7a6Hzelo9si6HWy75FOXWTNbUbkm67V2TmU1 zVWNoYTD6akq3EDckrsvke13B5tioaL8/Y1OFKwDoyzF4p+AmvjZGrdRdXUWD/Mq rq9uIaOJQSqNItm7vkAk2DcF8k00k0yBu05k022HfbZvKu5qMCMEcx3e4y4NDOpH TVTJ/OyhtT/sfgnEb9Tf3WFbxzbEw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=slDme3ejL3DQoWfgyYaE4bakt30/dQOOVvhGxprHUzsaYw1Q6rWs/+HBHtEoltOnn7Q7nMg71cfR7mqU93IjDEeXWYaCvFeIoSj4P+M7UopSBEuRB58IwSrOxky9caXd2neYJ3nz1KBes2VlE+kbYbQk5EBw/MoWlHXqwniVs8U8r1ORoKAzVncNqWrpfBy+8giu1NOGtCUAS3trpL7fa62lGTHOAlJxbJwdMV1RIeiva0TEUP3CvSrVuoLlQN1MRoLdOwD5cC0Ah+S5siIgIX9yPWXeMlrE8mH5a6jYWYDhVnCOdYfo/Jrid5wJlBPFRl31RDoN+HeYo074uZrKYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8Sgj0ZLqfnMV1rmzsLAXyKEjHirJgruqwfW3PMUXkTA=; b=NfbdiZmDM+8tN1tApWFRtAiCgR3vXewc18ZSNuqaUd6Nbpekt9x4ol5dajgSkMNZMQ7kP0iRjx5x6PE9oy+RUfVi4Qim441N0B4Empzi7UHwhveeUFiXs/uzJ6SPgQNCNzY6X869q4MlVhUMMWhoJ2bUqdgtbt8mHSuij4c2KFX6AAoFi0fIWfK/rcmqsEdXWVYOsqOREUgfKsba6fbycF4QCfWNIULkGrfetNNBEuNdoNLWfwTXT4kbqnAnpXsUcsYJ9PfThcCjdIW61UYS8YeE+B5908KXIHZNcL/TuOTr5ieGiJsxJcI8j6SGWEJ5F2fCJ8Uqt05SUPFT+iyAgw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8Sgj0ZLqfnMV1rmzsLAXyKEjHirJgruqwfW3PMUXkTA=; b=D2TbManbrwChhTDjrhsy7dXJ+GzNIejR0zxDA8f5ZASDT8UJagOF2QK/rrE/NIs38DdlG6vDugmio1Vn5JpCPvKa/uRNLmbYx12J1/rm1rOen3vPjgUuddZEK37PPkewbGLEfxv1GdP721mAt8LcOrzy396C+XlVH6Snh50yFhwTpIbU2cMTUda7xd4DHBn2U9U/D2uF2QGbwwvOgZJoqC9UqfbAE88Iz1Cgq1/hjb91lFd9o2QiNxfmBHrppWsILvS9wCY83ajZfUgm42c8VSt5Vp0GFX8UDr8s3N5hjZNMm0JDagdc8kJ00ZIhhR4NXgUGHwxejYb0ldxocW0nyQ== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 12/19] vfio-user: implement VFIO_USER_DEVICE_RESET Date: Wed, 25 Jun 2025 20:30:04 +0100 Message-ID: <20250625193012.2316242-13-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: e338a2ff-2009-48c2-2454-08ddb41ecaaf x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?VXcw7IqKNhL4HpxcA8vc4ZUjew7eu+QPEkD9l0oWsJvm74a9O9navVjwufP6?= =?us-ascii?Q?BbfFNdLgq9o+9Yoa955ud2mbm+41Sw6QlyM7Rxh9YJ5uqGDPlllmk+QfbPZV?= =?us-ascii?Q?egCOTzir0V4Msy5zfYhfB2R0WAq7JSzT3THDCxGFk36flur0FyJ36y3ys5FI?= =?us-ascii?Q?n5maPSjPuZ5D51XswA2dX4Sd9TuAnth3V15l9A6ld/yNZcFfxdy6WvpPaXte?= =?us-ascii?Q?MHnBTnIzuvE7m33yDBj79cfkHtweRB6w29p/l1GGhx9GBY4Q5rJ4LkPrjMge?= =?us-ascii?Q?FtX9w2VUBaf/YuUpHPJMaDYHlIn1jy941Yv/ZYor/CKoBxpdZ85jlUnHLdbW?= =?us-ascii?Q?Zt4hN0cspln0Dt3448V+M6zFjRYg/RKqPAfMoVWsO/YrcQqZFUWp0RtMwIYX?= =?us-ascii?Q?KSk35bLUA7SuXu8Y1odNsPytWLkmtzKAQ9KL3yrJOla/IQu30N88fOEpjyBM?= =?us-ascii?Q?xDhRz6QwHSn2seaZFDo9Cgbe2VEPF1t7ll0cd1ISUoZI+q/kGtkbBWMUHo4p?= =?us-ascii?Q?0AcJ/dx2ZSlItuEF/u0kyoTFiJvj5Y4Ch7TWSj7MXEvmg7VnEkBXSvOlvGxj?= =?us-ascii?Q?ayfEAVMt7b7ADE22Sg1M5kkV9AW6IvPZ003qM6VtE6oWP5I+ej4Azv4PUUHc?= =?us-ascii?Q?sxKuAbi9rlTuB3ALV/znQJRB8mxL859xIp/F8RtIEDlvwol/l6ZfiQE7L0Tx?= =?us-ascii?Q?oid8h/eh7kh1U5YIp9Ysev8vIpoF29NeX3pZkVWA3vhWzvdCSlvl4u2yh9Ym?= =?us-ascii?Q?F4p6htLocSML+96S+1sZeYG5+Pm1t9BaZFhVaG/cJ6CDnyzhAolG7SJhBs2B?= =?us-ascii?Q?1kmO4pWpGhSHvDUsNc4W8KD4HJ06bJH84OpJHFv1bXb1UsV9vx3CLx5s7Am9?= =?us-ascii?Q?pYVZI87PLj/gGcihzJ7eyIaK3yIEwdzS+grm7hQx3gxhNNL6Lw34XsUN+lXH?= =?us-ascii?Q?c2fyIlvZwWJCO2hZwxnjTV4AtO/toBaoDCildI+Q589LaD0nP8PmMDc+VX9t?= =?us-ascii?Q?3CJpytlo6WL7eRPNwpXH3Vjzh9IZY+IYIpmc1/Ac07OAzqK1iwTOLzkj246X?= =?us-ascii?Q?DoP2BHZb8YS1olVyUgeCmHdheI8g4Bv1Ke5qpZUbb5qhkbz4bfME892mVQgL?= =?us-ascii?Q?R6hkjnWo6ozqTcUUgljlmgBv++K7IHIItitpellgGnqbgqwLszesS4OAfFCr?= =?us-ascii?Q?LbtuVMTQRmabPoE84rdZy26RJbXoWeV9BhMZT7KQMWHpxcZDRj03BdByOlxR?= =?us-ascii?Q?YJJlcZfKPLcmNR4Irrd7g+C+ZMKNMeGUsgrWqlCWfyjfpmVeNJQ1VEdiMoe8?= =?us-ascii?Q?oBBMaZ4rBNJBdtWvvzwKco6lY6YnRaB3XnDIMnNXR+YYnc0Y4kS9X3Ldh11d?= =?us-ascii?Q?1rbkHtaprYN956dmsLSyZ5wy8lSftyAmYhiqkzbTb5n39F/UegGXm9YdEDPJ?= =?us-ascii?Q?Nb/PW2F4ksc=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?OxQHhO6UeFpd1Dlxdb+N346MSB3lqZmx1v398T9MYCff7Qcd2ZSZoN5cHYGa?= =?us-ascii?Q?Ot+YUAHvk/d53k1ZPoGd2bfX/GgYsRfQbVA0jCinvUmWUmk9yR1hocUe9sKH?= =?us-ascii?Q?UGSqcz3+CITO53SoJlNd22r5vRCNr9/eHiuHWL1KVgu0HTwuHHvPmnjdOEXW?= =?us-ascii?Q?y/QDauCHZ5VPLtOUx2cQJsM1/8d9Q8A3YAfRp3h5NMwu8PQkqkhZC9YA971o?= =?us-ascii?Q?N8UkvSKtmws/yuFgLUiDRLx3hfNLzLfYg5IbhXDH2f2SI5mI5ANJtY2Uhpg3?= =?us-ascii?Q?JiSuIkipVkYFFSWss/mLj1BZrT9EthZEeVCLCuZQW5K1bIc/CzDh1JvLcKv2?= =?us-ascii?Q?G+f8GIU6ZfEbkGxBoQ4qMS6qVa2EL/ogvB5thqyLiV5+0SZBLVigjVGYiiN7?= =?us-ascii?Q?9CYND933pBEMSO7xwx5VAluph0itIFy05u8KeytGLQSQD4S9r7UbkiW0jW8e?= =?us-ascii?Q?SNgJbSLVVXIyg377kerNRjIIVsh9wvYd0NWdMvLdHbmMTH4SCGHaUxdDBcuA?= =?us-ascii?Q?ojvEXkCR5UJkrMSviuCgTU+LOpHueIKQrlL8NehlvM6/NyhxzyrAarp1faIC?= =?us-ascii?Q?pregXd7TNycymwIhDXbwSGXVMMzVqfs+tfERrhVDZAANWDXf4jet+5rTkvzv?= =?us-ascii?Q?a6WB5VJvVIsxrmxCqMnd40EIDnmGSSi6wXcuOAqxf0VPltSnW7aALxpJaAvB?= =?us-ascii?Q?VjHLdU44g3Q2bkfcaanIYZI8hC1Iu51u9JqCBOAPWJNiCklGsAXolHedm9/r?= =?us-ascii?Q?VDj7/KVxwfzDJvRjf8ZVK8UCOHJjMv3M4RXCajMoaAO7R1j1Nk+UxESQiz4x?= =?us-ascii?Q?A1/PaqSdD9+0A8a7n93ZCtvMa/1MQuZmEa0GJbYRqZS7RRGpC6gyw48FwYbc?= =?us-ascii?Q?ODRzY26VykBTU0egHbPFQTFr2W3NbOdhaBGHpsEbPBxxVE2frZe5H36TG6ei?= =?us-ascii?Q?8NvkhgE4ZNOJJ7lpAabQoJCB/KO+LRt9xee9VqIaRKJEmOOQJaOsyP41kpRe?= =?us-ascii?Q?X3w32GR8pfGnoxjiWszMK1ZdimIL4IlcGVmcefImVxzeEGN6vpn1RhRFG7Du?= =?us-ascii?Q?DNLL5G5QvIeQuEIJeFMCt54Ge4JmZnhFKuvzO9XxkgJu51/eWjN/hq+87o84?= =?us-ascii?Q?jRT5s82ikIoIar8T2kzaQvVK717k3An67UJCCZijSi93GBYi4GRl4mq+Vdi1?= =?us-ascii?Q?6PhMvaWO29nQOV5MtyFbqRXaVbSmy2HI/qzdqGrZpDvydIabrVsYvb8Su75e?= =?us-ascii?Q?CY3m100jOcsyztTKKXje8DTAxak8tciFX6ax+NoxDuWpXlYy3hgFqXV1rD99?= =?us-ascii?Q?8csin/CbRSttCugzubn0InvapNcjT4UKUHTd89oF0zP0h8yLLGmojQsm5NTw?= =?us-ascii?Q?m3QjPidBRTNHhmo9pnp51RnIXqeJqPM8uYA8fnVDKAKIdy+G17yVcKH/aveX?= =?us-ascii?Q?WB5+9jqXCqUE+Z+tyxYjIlVFVvoMjrLn2WfYz+PqH3z4FJq/535/gRbP7svC?= =?us-ascii?Q?ur/o0VOP7FIQoPBdoAOcKpVWTZ1QyCzgbg8u5UCa2N7w6lVdto/ChTfubZHc?= =?us-ascii?Q?pQ+0Vm21oGu2UA9K+d6UXgMPzMMqzlYGLJFCp9aP?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: e338a2ff-2009-48c2-2454-08ddb41ecaaf X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:50.2163 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bVZ2eV5+6mgWSZh6Q3m1YR2V3xXjsuNAxadoAurduPeZagtehoxh4BY2C/h+xSMote3iMjUu2yJ4J89j2EBM5A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX8+0Jw3C0+0Cu DFbV5p0K4kRN54mqrPj+n3Auo6Z2FPB9B2nxxpYjYaAO65MsifQhFYhx4FGTfJ2UGmTK1PpOXJ6 470JRNCnsyAFpNi8NmygTE2U2KjtNGdHV+OwJvQT9j2ByiBz20AgkmS+6ihwazkWAqF5+FMWdcw YtkErjG2lRhHLn/b/ZaQT8o1Y5X88mxYU0eJgvRyQkw2Yq/xHG07mqtbqellRmMCi1earAMiiVc hzPlQ7eHYSqyPPk/LuPyoVJuSBBEp6sVh2F4CV+H4p3QX9GolyF9sdsqDSx4vqdjbJcbpcccHdI zPROuDJjtrGUclOLXF3Tv7eNK3VeLdNF9fVstGz3DUWjLshxfPc6QH/Q+b79yD/Rp42330gQOwr kxoaFkW6jxYFgsoNSt0k0pppqzaMVN5tOKli662/42KOhflbtZUFKn3GOTy76bRmahs3Guwm X-Proofpoint-GUID: LVJuEdmVgOw5NXyaa0v4caARX5GvHdxc X-Authority-Analysis: v=2.4 cv=Qohe3Uyd c=1 sm=1 tr=0 ts=685c4e6c cx=c_pps a=SVzXTY2ksShwLJJ6alra1w==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=70NfNXkHNs9el5T2m-kA:9 X-Proofpoint-ORIG-GUID: LVJuEdmVgOw5NXyaa0v4caARX5GvHdxc X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750880086676116600 Content-Type: text/plain; charset="utf-8" Hook this call up to the legacy reset handler for vfio-user-pci. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/device.h | 2 ++ hw/vfio-user/device.c | 18 ++++++++++++++++++ hw/vfio-user/pci.c | 15 +++++++++++++++ 3 files changed, 35 insertions(+) diff --git a/hw/vfio-user/device.h b/hw/vfio-user/device.h index 619c0f3140..d183a3950e 100644 --- a/hw/vfio-user/device.h +++ b/hw/vfio-user/device.h @@ -17,6 +17,8 @@ bool vfio_user_get_device_info(VFIOUserProxy *proxy, struct vfio_device_info *info, Error **errp= ); =20 +void vfio_user_device_reset(VFIOUserProxy *proxy); + extern VFIODeviceIOOps vfio_user_device_io_ops_sock; =20 #endif /* VFIO_USER_DEVICE_H */ diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index f01b3925c5..3a118e7361 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -54,6 +54,24 @@ bool vfio_user_get_device_info(VFIOUserProxy *proxy, return true; } =20 +void vfio_user_device_reset(VFIOUserProxy *proxy) +{ + Error *local_err =3D NULL; + VFIOUserHdr hdr; + + vfio_user_request_msg(&hdr, VFIO_USER_DEVICE_RESET, sizeof(hdr), 0); + + if (!vfio_user_send_wait(proxy, &hdr, NULL, 0, &local_err)) { + error_prepend(&local_err, "%s: ", __func__); + error_report_err(local_err); + return; + } + + if (hdr.flags & VFIO_USER_ERROR) { + error_printf("reset reply error %d\n", hdr.error_reply); + } +} + static int vfio_user_get_region_info(VFIOUserProxy *proxy, struct vfio_region_info *info, VFIOUserFDs *fds) diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index c0f00f15b1..49d12763ab 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -263,6 +263,20 @@ static void vfio_user_instance_finalize(Object *obj) } } =20 +static void vfio_user_pci_reset(DeviceState *dev) +{ + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(dev); + VFIODevice *vbasedev =3D &vdev->vbasedev; + + vfio_pci_pre_reset(vdev); + + if (vbasedev->reset_works) { + vfio_user_device_reset(vbasedev->proxy); + } + + vfio_pci_post_reset(vdev); +} + static const Property vfio_user_pci_dev_properties[] =3D { DEFINE_PROP_UINT32("x-pci-vendor-id", VFIOPCIDevice, vendor_id, PCI_ANY_ID), @@ -310,6 +324,7 @@ static void vfio_user_pci_dev_class_init(ObjectClass *k= lass, const void *data) DeviceClass *dc =3D DEVICE_CLASS(klass); PCIDeviceClass *pdc =3D PCI_DEVICE_CLASS(klass); =20 + device_class_set_legacy_reset(dc, vfio_user_pci_reset); device_class_set_props(dc, vfio_user_pci_dev_properties); =20 object_class_property_add(klass, "socket", "SocketAddress", NULL, --=20 2.43.0 From nobody Sat Nov 15 14:11:14 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750880008; cv=pass; d=zohomail.com; s=zohoarc; b=Iyn80hoKZp5uB16HS9+6hpuHycoKRVUPOfY5tcX+9vIAYvb06z0oCPF91zFD7z5u0M2gE31/GPVgQX8jRbW23+zJ7P630ePtmcUsJm7WZm7UVy0hayUFUpQCIeH0nsqlsInvGmFR+jcybyd50Rell50ZTcgBWCGhgz+eJzDADqA= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750880008; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=g9PhacUvZcUxmruVZsbtYmC6rCRxbgNXibolLRoSReM=; b=lGBliTdKc+R+SsVTB7IXzMizjmjGvpUDHG0E8QWemljcCdY/BYlHj9y8BYOEB5OPfxkCU0gfbSs5cgUUjNT4u/dX8egL+KxKmx/q1dGXZqmzCC17H0yZCVuyZzoeAwQHFYJtz9/l3VnhBO/Jb7M5FEz4hmJYswFNhJrvKpFRN/I= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 175088000791717.456478385036462; Wed, 25 Jun 2025 12:33:27 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpq-00087Y-74; Wed, 25 Jun 2025 15:31:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpo-00085l-9q for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:04 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpk-00046Y-22 for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:04 -0400 Received: from pps.filterd (m0127840.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PDoDdu015699; Wed, 25 Jun 2025 12:30:55 -0700 Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04on2116.outbound.protection.outlook.com [40.107.100.116]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 47gjbh0r93-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:55 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:52 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:52 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=g9PhacUvZcUxmruVZsbtYmC6rCRxbgNXibolLRoSR eM=; b=y2dTbGfEIygrrfCbZHCEwklJSWNViQMvZhjq2HqXy4+h4AQ1HGxgklVAy 1UolKpOxZx84OOoujBDWUWfGv7/+33Rk45t9XGo7lP11WwHF63Xm/UUyKDPI0zzL qzBxxN/0c+CX9ggNCCE07/XuzImhFHJAy9ULr0L+CHmm5TOXn2ePetSL2cfuYFb1 ZbfzJ9aPozgsiafCCRkL5Ejh4k5MVvF3U09998m3UATTufQfizObirtj+rFdUeZK bdjavxjL4Xvh8deXCqE2gp/hTPqcH2ZYQnQ4KFlxjoKD+8HXcXHPyjewPOmx6XiA VLkA0+hTDB+V3Z3tac5ZnhwEDA/Nw== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kQVhUscJ0/GzqhbVCerR5i9xYrUS6Y9T821WeL9qjSYfAlrCf71yc3PvF3jjrPK3aXgyQVWVB1MPY/ToDndl24LP6+Tb79QHE+wSDBBrmhB5WIOHowB/s5+6BQ/ICrw1+35reYVf746za7A1D/kpYSehtRcTxxVg3I2/NfMtSd4pmEIBtLQrH0QONFfC3o7pFCGFF0v+N0Me2mlj54/XlRxqBgoe64GIndm4xmUqF/3tR+zN0Oc0IUd7PQHzP9XHJnwxl9e0eNuPIGpnv9/RzHVYMTaCD8PyVpMt7vllUIekpYfOQaYegwZJzmdyndf0VvkJ9d7gJtWAk7ehipmjmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=g9PhacUvZcUxmruVZsbtYmC6rCRxbgNXibolLRoSReM=; b=ajpCUiyWqeehmpOXZx3PmPnbCQ92JqV01AS4Wx1g03r9C5yy2rYHHIkA7ShgWri6T/Wh31YezkkaJzO8XwaOlxgNJePJEM4dLIX/wGclqDpFfYT6bG6rSdIJ/ieoIdOewHJKeYPPhs4LVSQ1Wk9806ObyH630zeoexsVFdGO7hNErZPStcDMC/uU+J2ecu30ik1XoZiRdF7MfiW1RAzPINvgLpv9bepTPeYruHdX75txEcsXCr0XKtBBWRHfcf/HjEHEvl2ExjrKOxonTHqXqJs1A5kXAjs3mm5nNof7WC3VoO2n7J+Tan62gtDh4Rq3L4SS8lWK0sjGS05TDvdtBQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g9PhacUvZcUxmruVZsbtYmC6rCRxbgNXibolLRoSReM=; b=VoCvih44y6U/5UMEUUoEggjt/DbjZNZV4YCK06/0lNDkeFS9fSDTdoNb2VBODuEeYrfWONVLUz3RRnYcooyCBd+M7ZNA9O7dFYSQ5YO8DaZ5lfhrd6IBqfltmCgtOUPirVME/JeoJI1qpx7w14WL2GiRbPETJwp2wm3JEANEmNqe2b/Iv5eU1amd018hKfh/mDoC9iNgWZupqArKA4DusQLyUd+dZC0kZYkZRcZR0IGzI8c/VpSHnzrjNoYTeJYzPD9Vi6JH8+0NoHtXG5VEMy+ujiUbmOdDVTLtN8cnoF/PZweX+ljsG33LCVFlSu8TYGcpff8P0k+Drts1wHDTIQ== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Jagannathan Raman , Elena Ufimtseva Subject: [PATCH v5 13/19] vfio-user: implement VFIO_USER_DMA_MAP/UNMAP Date: Wed, 25 Jun 2025 20:30:05 +0100 Message-ID: <20250625193012.2316242-14-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: 9f26237e-281d-4d64-2103-08ddb41ecc39 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?QQWNpkj2WltWtrVByn4aabaoyJSBrqvgpCmRb9YzWuK55ITvOSYdbuLZgUI3?= =?us-ascii?Q?7sQB+LGDK7z7QQBSeTTpPffcF2egMMYxOhLmTOzzfquXCrkvygxIEwXCxtIP?= =?us-ascii?Q?dev21RFHC1kxoT0ZfWy9RohoTCFUeE8lAlIDLHORMz985t2qggCtVvrEDwPF?= =?us-ascii?Q?Fo9TcgGL53kccxHqKhfCEQOUci2Wj74jexirO9wXxdMjgnKfANOqxYJqtz+z?= =?us-ascii?Q?c+9v2UthQ8LHg5eWkrzPh9Rx9M2Khvf+cKNwadJkCZerAaRFGbcL/RQwwt/J?= =?us-ascii?Q?rmy2d/qaBGKCaZXY7N9qWAzEZTRXTlXTYQJTcP/O02KosVZVo+MGNAhJBgm/?= =?us-ascii?Q?ZoTklZmhV/ACQvo0rLwzxj2unaeRayJxTLfugv/s2ReDtbJlSE/BqSyeGF+R?= =?us-ascii?Q?nSPdRANdf/bdCsyGp4BBrEj4eh/J/XSywPgMnfgVtIH4YmX1i6TMnoWM8FpY?= =?us-ascii?Q?voUnB5+P26w8NH41TE5fBjHNqt1TxBEaVN1fA5ksit+fqIgutFy1HcVq7zl9?= =?us-ascii?Q?BC/oEDB5p6DS4O42R37kQevBRIVMAMq2jmVB1G9Vj+bL6c6ZO/nKeoiMjZMX?= =?us-ascii?Q?ERIML0wWCgG46X59VSOPt0cBut+isCj5eJIMmMedQALlCstNir6PKYfott1t?= =?us-ascii?Q?97L2T+TD67VyjxIA0I3EH4pUIdBXdf7VT/zr5kSd8I2Paqi7X9LwNmh2yq5+?= =?us-ascii?Q?zW0g8swM/MZ5vwj0L8Ka1Mh58Bb+yH2lLxI/lMK/cYK/zV8pleKIXP1aM9OC?= =?us-ascii?Q?tp6qqm/BNCd5oqM884xvvMr6iBieCLMqActRGuMvzgnupO9vD6wMlCNR/l0q?= =?us-ascii?Q?sSfGLLR3Tu1Csge8GqVX8z7CjbOgwcrasjt3JPgkl0zCnCmXDfols/h4Gj/p?= =?us-ascii?Q?sTTSQk+DRzHw9d+F5Cf96EZgkPYzHIpDn2Q+olqrmykXhrisaSHL6QC+vFJq?= =?us-ascii?Q?DqX+ymc34Xn0eY833l5tu190C0CfHA2qDjcJPo3JRRUEbcKOOdd/RQk7UCBP?= =?us-ascii?Q?YybQZuq82hpCmpPBMr3tV6OYJGcl4Vdkc1k+zcpNPhcqc75U4BEk5LPjRzDQ?= =?us-ascii?Q?O50RPD60wh8xnoViL8igZeUu21lSHMxL8d7mwXPMYPVP/4vgcnyX86CIwB9b?= =?us-ascii?Q?SzD9ku9YFD4J7lpob1d9l9nbyWilNU9BC5DzcrYWcXKKN493naujfHAPBm8C?= =?us-ascii?Q?eotqhe6SnaClXGqz2piA3BBtrH4WMh/xqK8TyMRjBobvhudRt3pyxWyA0MQ0?= =?us-ascii?Q?A4kwHGVvca/o8Lh/JHWl4+ynyuqRnW6shH7IbjovHzoNzDYsk6i0fmJQo+Nx?= =?us-ascii?Q?nk7lkCQjxZFrgAJGHSoZen9ktqzn3o+KEICwtHZ/7/8oOSaMHhixnE0DWiBu?= =?us-ascii?Q?nxx/PgsIPm8yGLZnUmM/F/ZeGibph7/+3suDJsAeWNwy3Y208vtTSelCDdmh?= =?us-ascii?Q?qT98h1aeoDo=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?RtodN0agUxhI71nskm0ojxstrV4+GsYCT64Cc//mcRfL1+tkA3A0DZPE9cSY?= =?us-ascii?Q?8vhQKYWKXjd1fVtJipgwkT1G4M6t95/pHIJR2q3qtCnJAtYIYxUKI9W61I3P?= =?us-ascii?Q?a6mv59rEnkBaEVXplbt9athNk4S1EAEjNdHSUHKLE2XCKbu3GWg/P4mFyMyA?= =?us-ascii?Q?xqvZH2Q8pguRRkfctzOaKu+pQnxykSc6IONNUMM8Yq7KApC5sm9xAIOSjVnY?= =?us-ascii?Q?LytfT5bUtyQO6zDyAwrst9lTANok0QeMsjjUKFHNmX8flTMifbEe9EFLASwI?= =?us-ascii?Q?LIRjGJ00tygGA4gXo4SPIgpbhS2CSaWH5TGemAOjcVjfn3C+/hLmh/K3ityv?= =?us-ascii?Q?b3CbRvTyzsi6UrIwhmXVY2x0peu7ku+fvHiEmkJWfkv7n8dY5JqeKJluITAy?= =?us-ascii?Q?2P7ohovBV3Emg9XQG7n8qKekMth3C1zVrOKZtDhDphDntXtM+uN4Hu0tepjd?= =?us-ascii?Q?5F8oLU8LIXkIsJ+CLGJ0Zv5rnoS4GF0Iey9CJho5IgYUIcCE+gOzMN8K0+Io?= =?us-ascii?Q?mVdrTy71KyLC3/H2w0/yTUIw+niUvQX7wJqilNo9gY+X4HSrJwYAlzQViXb6?= =?us-ascii?Q?1Vy0lJLXtR9KJUrHoIOobUpCX2gn5wmmxOvfrwAT0Myc3x7DDb48d7sCSK7Z?= =?us-ascii?Q?fk7tQaW4NEXlchKAYL2742ce10SP7Br/ih2AIeNsgfndJzcNVCTrk+G+7IJ6?= =?us-ascii?Q?y1kkgPHO2jnLEswr3IImJ710o5clupJ4DCAf6QV3kbORYx+E15DZOCxGs2y1?= =?us-ascii?Q?aEn1kI3jvlv1LGoqJxmKYAjiCaUU1qhl/bTFzuGi7l/vTWLC2dVjyGDqetEF?= =?us-ascii?Q?9Qd3vTHIGqQP2X32g08nHIMcJX8fXaLvDu6WubIM6LS0N7GlRwaaESFfC+8s?= =?us-ascii?Q?AcbGkMNs3HCX3/quDgAjR//QXT70r7z3NlolHtXdhhi5gPqdqxSIXO6oq31L?= =?us-ascii?Q?K/Gv9J9qk/9NoCTgCWT5bw6UOCTylVSS7O2n0KvzavcNqPaTZQoDuptDpJei?= =?us-ascii?Q?gCkrbkQ7vEre2gU7M41x10DIhLFz2gqYs372f7qGKihVmw3+LYAH3Ii4Ew/r?= =?us-ascii?Q?/p6LLlU+Cut+kEu58v4UP1fweB8KTq3+WGjzrIhbnqywnynAGXlAGIk0Ex/0?= =?us-ascii?Q?Poa2+iuW13AUAFbL5VAtFayeeQJPRr+Smx1N3e2rX6QkGkgUfQii3xizfgMM?= =?us-ascii?Q?f3XnTF0S4QexjMBmNsYOeqQXo/5Veax36J6ADIcFSYcduuhXFbPXn05J9SN7?= =?us-ascii?Q?th2thY9M9ZOZUBOcn4h6jkkUxPOIql6q/pf7Y79zQC3TbxytmRdoj/CIdaZp?= =?us-ascii?Q?epxZSu6pdNP5ABuZhstx5raJD2JvhqHujKs/HbCfWkbRu/gPvod9oV4DVFYZ?= =?us-ascii?Q?fERzZP7hEgdj4qKcJrjJBb002djU4dDrO5SE0ev3dQ8YMC++yY4N+DHFSBb4?= =?us-ascii?Q?27LlOltslPERUbHU750fgAdnGIP6Iw+vhgvZCNlxhoxFzziorInwokEqkwMi?= =?us-ascii?Q?Yvf8YBe9llwjHkBfJK9+aLFcbfEEeVbF9by8TJoaOKaD7O/J5Jlxf4Z0fSZv?= =?us-ascii?Q?87ZtXszHApm+WuZFD/ZS6S8NnKZzLdrlYmOVG4DX?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9f26237e-281d-4d64-2103-08ddb41ecc39 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:52.8284 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1U3FlfrGDJJ/sPNlhx5PHzMspYXMXi6Et127aaUnW752CDcxmrMrBMUuVJv2YeWl4nDjkOhhTQyTe1EYntrhNA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX0sDXIlzeAmR+ T16R9An6SKQEccy/F61yhAlhiReUZ5ZCgZZKIUmnKF3R+bdpME2vIHhXyUPMoL+qXLcL2UTh2gJ BdYKKqVkefely4lgagASAPmZmIoi1rCYitJlHy9Sf6j3OzTTOcsUld0Ih/+wJxfeJeSR9q1ZwWC zNHrhQpE7l80T2xs9WRdLH8pFzcB05UQYTb5iaHn9srZWSRTQC1xT/wPWkVg0kMc4ur9o+BtJTZ nTpYDlM8h6wfyR5H43fYI4kRdDQgmOR06NWZxBC4hLuQbglhushbks4HpU5hE3OpZ7AZeLWzUmj jsBmNATXN2OmVMuY1o3tpDVRUNX5sFIZ0999+iTZzPvKv5zxrg9cyuG7fm99nM4PfgOY2rnH3Uu NYgTT+eP6Gx2cbC0uzScKeK5CwQjXzjiO402WX7Ju+RbDDgZZIBwRVccyHhZV3zaV1bTdhgb X-Authority-Analysis: v=2.4 cv=WMl/XmsR c=1 sm=1 tr=0 ts=685c4e6f cx=c_pps a=ynxkwAiy9VFaaSOUVCgc6g==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=RQZ_2NmkAAAA:8 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=VuTGjHEVGySaywlrrqoA:9 a=46pEW5UW3zrkaSsnLxuo:22 X-Proofpoint-ORIG-GUID: sDgHTeq8oExXROuUzSpZ7pA2mwHI_zAz X-Proofpoint-GUID: sDgHTeq8oExXROuUzSpZ7pA2mwHI_zAz X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750880009832116600 Content-Type: text/plain; charset="utf-8" From: John Levon When the vfio-user container gets mapping updates, share them with the vfio-user by sending a message; this can include the region fd, allowing the server to directly mmap() the region as needed. For performance, we only wait for the message responses when we're doing with a series of updates via the listener_commit() callback. Originally-by: John Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/protocol.h | 32 +++++++++ hw/vfio-user/proxy.h | 6 ++ hw/vfio-user/container.c | 135 +++++++++++++++++++++++++++++++++++++- hw/vfio-user/proxy.c | 84 +++++++++++++++++++++++- hw/vfio-user/trace-events | 4 ++ 5 files changed, 257 insertions(+), 4 deletions(-) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 48144b2c33..524f3d633a 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -112,6 +112,31 @@ typedef struct { */ #define VFIO_USER_DEF_MAX_BITMAP (256 * 1024 * 1024) =20 +/* + * VFIO_USER_DMA_MAP + * imported from struct vfio_iommu_type1_dma_map + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint64_t offset; /* FD offset */ + uint64_t iova; + uint64_t size; +} VFIOUserDMAMap; + +/* + * VFIO_USER_DMA_UNMAP + * imported from struct vfio_iommu_type1_dma_unmap + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint64_t iova; + uint64_t size; +} VFIOUserDMAUnmap; + /* * VFIO_USER_DEVICE_GET_INFO * imported from struct vfio_device_info @@ -175,4 +200,11 @@ typedef struct { char data[]; } VFIOUserRegionRW; =20 +/*imported from struct vfio_bitmap */ +typedef struct { + uint64_t pgsize; + uint64_t size; + char data[]; +} VFIOUserBitmap; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index ba1c33aba8..e2fc83ca3b 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -70,6 +70,7 @@ typedef struct VFIOUserProxy { QemuCond close_cv; AioContext *ctx; QEMUBH *req_bh; + bool async_ops; =20 /* * above only changed when BQL is held @@ -99,9 +100,14 @@ void vfio_user_set_handler(VFIODevice *vbasedev, void *reqarg); bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp); =20 +VFIOUserFDs *vfio_user_getfds(int numfds); + void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, uint32_t size, uint32_t flags); +void vfio_user_wait_reqs(VFIOUserProxy *proxy); bool vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, VFIOUserFDs *fds, int rsize, Error **errp); +bool vfio_user_send_nowait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize, Error **errp); =20 #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/container.c b/hw/vfio-user/container.c index b4a5a840b0..3133fef177 100644 --- a/hw/vfio-user/container.c +++ b/hw/vfio-user/container.c @@ -12,23 +12,152 @@ =20 #include "hw/vfio-user/container.h" #include "hw/vfio-user/device.h" +#include "hw/vfio-user/trace.h" #include "hw/vfio/vfio-cpr.h" #include "hw/vfio/vfio-device.h" #include "hw/vfio/vfio-listener.h" #include "qapi/error.h" =20 +/* + * When DMA space is the physical address space, the region add/del listen= ers + * will fire during memory update transactions. These depend on BQL being= held, + * so do any resulting map/demap ops async while keeping BQL. + */ +static void vfio_user_listener_begin(VFIOContainerBase *bcontainer) +{ + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + + container->proxy->async_ops =3D true; +} + +static void vfio_user_listener_commit(VFIOContainerBase *bcontainer) +{ + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + + /* wait here for any async requests sent during the transaction */ + container->proxy->async_ops =3D false; + vfio_user_wait_reqs(container->proxy); +} + static int vfio_user_dma_unmap(const VFIOContainerBase *bcontainer, hwaddr iova, ram_addr_t size, IOMMUTLBEntry *iotlb, bool unmap_all) { - return -ENOTSUP; + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + Error *local_err =3D NULL; + int ret =3D 0; + + VFIOUserDMAUnmap *msgp =3D g_malloc(sizeof(*msgp)); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DMA_UNMAP, sizeof(*msgp), = 0); + msgp->argsz =3D sizeof(struct vfio_iommu_type1_dma_unmap); + msgp->flags =3D unmap_all ? VFIO_DMA_UNMAP_FLAG_ALL : 0; + msgp->iova =3D iova; + msgp->size =3D size; + trace_vfio_user_dma_unmap(msgp->iova, msgp->size, msgp->flags, + container->proxy->async_ops); + + if (container->proxy->async_ops) { + if (!vfio_user_send_nowait(container->proxy, &msgp->hdr, NULL, + 0, &local_err)) { + error_report_err(local_err); + ret =3D -EFAULT; + } else { + ret =3D 0; + } + } else { + if (!vfio_user_send_wait(container->proxy, &msgp->hdr, NULL, + 0, &local_err)) { + error_report_err(local_err); + ret =3D -EFAULT; + } + + if (msgp->hdr.flags & VFIO_USER_ERROR) { + ret =3D -msgp->hdr.error_reply; + } + + g_free(msgp); + } + + return ret; } =20 static int vfio_user_dma_map(const VFIOContainerBase *bcontainer, hwaddr i= ova, ram_addr_t size, void *vaddr, bool readonly, MemoryRegion *mrp) { - return -ENOTSUP; + VFIOUserContainer *container =3D container_of(bcontainer, VFIOUserCont= ainer, + bcontainer); + int fd =3D memory_region_get_fd(mrp); + Error *local_err =3D NULL; + int ret; + + VFIOUserFDs *fds =3D NULL; + VFIOUserDMAMap *msgp =3D g_malloc0(sizeof(*msgp)); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DMA_MAP, sizeof(*msgp), 0); + msgp->argsz =3D sizeof(struct vfio_iommu_type1_dma_map); + msgp->flags =3D VFIO_DMA_MAP_FLAG_READ; + msgp->offset =3D 0; + msgp->iova =3D iova; + msgp->size =3D size; + + /* + * vaddr enters as a QEMU process address; make it either a file offset + * for mapped areas or leave as 0. + */ + if (fd !=3D -1) { + msgp->offset =3D qemu_ram_block_host_offset(mrp->ram_block, vaddr); + } + + if (!readonly) { + msgp->flags |=3D VFIO_DMA_MAP_FLAG_WRITE; + } + + trace_vfio_user_dma_map(msgp->iova, msgp->size, msgp->offset, msgp->fl= ags, + container->proxy->async_ops); + + /* + * The async_ops case sends without blocking. They're later waited for= in + * vfio_send_wait_reqs. + */ + if (container->proxy->async_ops) { + /* can't use auto variable since we don't block */ + if (fd !=3D -1) { + fds =3D vfio_user_getfds(1); + fds->send_fds =3D 1; + fds->fds[0] =3D fd; + } + + if (!vfio_user_send_nowait(container->proxy, &msgp->hdr, fds, + 0, &local_err)) { + error_report_err(local_err); + ret =3D -EFAULT; + } else { + ret =3D 0; + } + } else { + VFIOUserFDs local_fds =3D { 1, 0, &fd }; + + fds =3D fd !=3D -1 ? &local_fds : NULL; + + if (!vfio_user_send_wait(container->proxy, &msgp->hdr, fds, + 0, &local_err)) { + error_report_err(local_err); + ret =3D -EFAULT; + } + + if (msgp->hdr.flags & VFIO_USER_ERROR) { + ret =3D -msgp->hdr.error_reply; + } + + g_free(msgp); + } + + return ret; } =20 static int @@ -218,6 +347,8 @@ static void vfio_iommu_user_class_init(ObjectClass *kla= ss, const void *data) VFIOIOMMUClass *vioc =3D VFIO_IOMMU_CLASS(klass); =20 vioc->setup =3D vfio_user_setup; + vioc->listener_begin =3D vfio_user_listener_begin, + vioc->listener_commit =3D vfio_user_listener_commit, vioc->dma_map =3D vfio_user_dma_map; vioc->dma_unmap =3D vfio_user_dma_unmap; vioc->attach_device =3D vfio_user_device_attach; diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index aed7b22e2a..c8ae8a59b4 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -27,7 +27,6 @@ static IOThread *vfio_user_iothread; static void vfio_user_shutdown(VFIOUserProxy *proxy); static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hd= r, VFIOUserFDs *fds); -static VFIOUserFDs *vfio_user_getfds(int numfds); static void vfio_user_recycle(VFIOUserProxy *proxy, VFIOUserMsg *msg); =20 static void vfio_user_recv(void *opaque); @@ -132,7 +131,7 @@ static void vfio_user_recycle(VFIOUserProxy *proxy, VFI= OUserMsg *msg) QTAILQ_INSERT_HEAD(&proxy->free, msg, next); } =20 -static VFIOUserFDs *vfio_user_getfds(int numfds) +VFIOUserFDs *vfio_user_getfds(int numfds) { VFIOUserFDs *fds =3D g_malloc0(sizeof(*fds) + (numfds * sizeof(int))); =20 @@ -618,6 +617,43 @@ static bool vfio_user_send_queued(VFIOUserProxy *proxy= , VFIOUserMsg *msg, return true; } =20 +/* + * nowait send - vfio_wait_reqs() can wait for it later + * + * Returns false if we did not successfully receive a reply message, in wh= ich + * case @errp will be populated. + * + * In either case, ownership of @hdr and @fds is taken, and the caller must + * *not* free them itself. + */ +bool vfio_user_send_nowait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize, Error **errp) +{ + VFIOUserMsg *msg; + + QEMU_LOCK_GUARD(&proxy->lock); + + msg =3D vfio_user_getmsg(proxy, hdr, fds); + msg->id =3D hdr->id; + msg->rsize =3D rsize ? rsize : hdr->size; + msg->type =3D VFIO_MSG_NOWAIT; + + if (hdr->flags & VFIO_USER_NO_REPLY) { + error_setg_errno(errp, EINVAL, "%s on NO_REPLY message", __func__); + vfio_user_recycle(proxy, msg); + return false; + } + + if (!vfio_user_send_queued(proxy, msg, errp)) { + vfio_user_recycle(proxy, msg); + return false; + } + + proxy->last_nowait =3D msg; + + return true; +} + /* * Returns false if we did not successfully receive a reply message, in wh= ich * case @errp will be populated. @@ -666,6 +702,50 @@ bool vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUse= rHdr *hdr, return ok; } =20 +void vfio_user_wait_reqs(VFIOUserProxy *proxy) +{ + VFIOUserMsg *msg; + + /* + * Any DMA map/unmap requests sent in the middle + * of a memory region transaction were sent nowait. + * Wait for them here. + */ + qemu_mutex_lock(&proxy->lock); + if (proxy->last_nowait !=3D NULL) { + /* + * Change type to WAIT to wait for reply + */ + msg =3D proxy->last_nowait; + msg->type =3D VFIO_MSG_WAIT; + proxy->last_nowait =3D NULL; + while (!msg->complete) { + if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, wait_time)) { + VFIOUserMsgQ *list; + + list =3D msg->pending ? &proxy->pending : &proxy->outgoing; + QTAILQ_REMOVE(list, msg, next); + error_printf("vfio_wait_reqs - timed out\n"); + break; + } + } + + if (msg->hdr->flags & VFIO_USER_ERROR) { + error_printf("vfio_user_wait_reqs - error reply on async "); + error_printf("request: command %x error %s\n", msg->hdr->comma= nd, + strerror(msg->hdr->error_reply)); + } + + /* + * Change type back to NOWAIT to free + */ + msg->type =3D VFIO_MSG_NOWAIT; + vfio_user_recycle(proxy, msg); + } + + qemu_mutex_unlock(&proxy->lock); +} + static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); =20 diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index aa8f3c3d4d..44dde020b3 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -13,3 +13,7 @@ vfio_user_get_region_info(uint32_t index, uint32_t flags,= uint64_t size) " index vfio_user_region_rw(uint32_t region, uint64_t off, uint32_t count) " regio= n %d offset 0x%"PRIx64" count %d" vfio_user_get_irq_info(uint32_t index, uint32_t flags, uint32_t count) " i= ndex %d flags 0x%x count %d" vfio_user_set_irqs(uint32_t index, uint32_t start, uint32_t count, uint32_= t flags) " index %d start %d count %d flags 0x%x" + +# container.c +vfio_user_dma_map(uint64_t iova, uint64_t size, uint64_t off, uint32_t fla= gs, bool async_ops) " iova 0x%"PRIx64" size 0x%"PRIx64" off 0x%"PRIx64" fla= gs 0x%x async_ops %d" +vfio_user_dma_unmap(uint64_t iova, uint64_t size, uint32_t flags, bool as= ync_ops) " iova 0x%"PRIx64" size 0x%"PRIx64" flags 0x%x async_ops %d" --=20 2.43.0 From nobody Sat Nov 15 14:11:14 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750880107; cv=pass; d=zohomail.com; s=zohoarc; b=iudyI0EN4cnEK1RF6Sz847pRB1GWcxCnUWfRAsxYjf1nJnMZRjexzdOu8mOyNhVboWCLRk/3+6pavMW/FE0/eVXUmpbK6jCiZOqt3vef7/MI1YtWKqj1V99Gq4+nr7qoutoURWg0BP0g73SlrSmv2uaB8WR9yW3anf8ZzUusHHM= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750880107; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=jURN3uf0w13uVCJ0T2hHXiHI03MMBd71E1aociHKrPI=; b=Hqm/oddDnVjrudkK3F5BDzcdLV4paKTmWLYAXp86WAyiCOZyykCBL/L576+AUB9/cZk3sx6ZHdK0Xebf8rreKWAbvOtNJLgXJDP1DmNln1Ia3e0YZTjYG4SA+SLsq4N7StFc7BsfPg9Hp7T+jx/Afu4/hGg+jHCbwaUfP2X77dA= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750880107548511.91615939997996; Wed, 25 Jun 2025 12:35:07 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpo-00085n-Jc; Wed, 25 Jun 2025 15:31:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpm-00084m-2y for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:02 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpj-00046C-Bp for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:01 -0400 Received: from pps.filterd (m0127842.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PH0WYC021654; Wed, 25 Jun 2025 12:30:57 -0700 Received: from nam04-dm6-obe.outbound.protection.outlook.com (mail-dm6nam04on2118.outbound.protection.outlook.com [40.107.102.118]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47geghhf92-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:30:57 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:55 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:55 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=jURN3uf0w13uVCJ0T2hHXiHI03MMBd71E1aociHKr PI=; b=B/k0GX3+xHvEdVV5yJFNfqS5X205egf0pNWSCJXcF2v3foRIm9BUJjded gFRRLpKnTvuXn0RMv6NlSSP/Or/h6vL1nNJL1dcoGyeltWUxgZHJ497WgUk1AV+R pOBPoDK0MFFVGlsK77KaZVuDhPc7zNX1iaDMjeP+TCNEFVXOmXXGlTtav5+rTx3R ttQK88PFI5AIDnhxt/aW+7ibb+Yfgoyuin1gH+kD2+oNkxinAwt58CQ6MEuzX85S rRxeAJgweSUt9NGaGWnoV3ddLyY2pOCvzVxg0S+YjxpnyePVmUAwqUsqz3hdtVIk 2piqEVF60UilZPGJXhc1J1Ak7Ckyg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HLUAgiCHGC9yF0W0oFdPh7nnO93qv2IEI/zvMKi/CMPmKI9YB44oDHkBuJsKmIDhGgfkFC6d1YjGlFT+VgMQ9hbNaLPbR8oi1Dtj+01DdOS0G4MmhHVkN4PEpGaIoNIFJUks/sR9qoFjA8vsmwuPDKAk2e2gWSi8qIXrx6Ph6Dz8fLjNHANF5neXFhbcOa2xhGm4GIMcGMLs2JDilb4a2SFDGEJ2jO+wavm86RLxYp/g78B5kRvyY3XW5rYJ1+US56uDMIQU6byM7tdjASnbV1gxHGHFUF/JJASfq4ZDUKqHDZIgObgZBLuE+M2V+fxSlO6x/JGbIvRTOHmNo95zbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jURN3uf0w13uVCJ0T2hHXiHI03MMBd71E1aociHKrPI=; b=csOzU0LprTI+MBhu8fOC+W6BFm4+P1sj8ut+5Hy5bNVn4pyk2AW8zI0ZzOhjDe2QQlQL9vURvs7fR9BpG5DsoHXDebWFBAyVnA0b/1kNbLhcnuPSnzZB2qV87swemhbdeTdfFBzN+Wveu+JMfFKsFcEinp+N/1Tiene1v7JLPeG5AQnKp5BfadoUph6iKu50L6S684y+Ec7pibEDcLNRmVxeooxmn8Azn8fvIcCMCnt7lv53oCPhEeGi3jeLkd0cDENXsA3wNKqAAA/zMKyhHWuGBP9A/y3BY6kQ//mLafS7zT/+HuYMUc/M3nsL5PBHdRigBvrHl2wecRPLST328A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jURN3uf0w13uVCJ0T2hHXiHI03MMBd71E1aociHKrPI=; b=ShPB1jleKbCrVXyd21FkavqkvTjpxgmLZ/XOwWYe6KjfXZMiQLqdhbN98ycW30jE4nGSQpfwZeb1iihB6HrcTFe8q3ln8nGyQORjsdKpCf4nzRtTkxjcfyOP30pL5Aei2sYll/8tHHm7o3yGrmOocz8GWHWQOF7cog7LwLxOoyIpoQVkyNKn8xnLgHFpM9gglHi1ryuWS37bEfstqhsnE15Y1fWy3ycv7KyK9XeB0qjmaXG77yaKhxmsPEUT4q7ulcYO3nxmZfkpPpKuyYr8Zis1Z+LnZyG8WHfM5Yaij+hP2bUZBuBeAfqMobsrJmLXYxaYBpWZOlRU/ft/hrF/OQ== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 14/19] vfio-user: implement VFIO_USER_DMA_READ/WRITE Date: Wed, 25 Jun 2025 20:30:06 +0100 Message-ID: <20250625193012.2316242-15-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: aa751081-95d6-4713-f5de-08ddb41ecdd0 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?xhmgbMSPSATB6L2hP6+j+o1ljXKmDFMdomEA3i8hv03dMLo19RCqfKOV/Tq0?= =?us-ascii?Q?FHorcitYT/3lcdcAuV28qFsBDYukcURt1YDov9wvzltPiUSZk/POJsOE4scC?= =?us-ascii?Q?B046pd5FerA1rf1cKVUNWx1YKRXA2VGm/0teMm0mrtq9Q9CquMzJUXnOWwcz?= =?us-ascii?Q?aEOklCIraokgCqJzxqnB8u5BiE1oD4Wf2duik1Fs+spG8LE8zEdERBb8eIuz?= =?us-ascii?Q?JtQSMzRpdr6WFBN08avgNtMHtiBV2BHl4YAidVENgdAM2j+CSt/3jXKUYtuA?= =?us-ascii?Q?PoH8nbgdWTyarHuwoX8puvv5hj0BHgU9wlgLOTKFwW+5h8u2Kv4uF0MPSPLW?= =?us-ascii?Q?Kxdyf1MNE6oaZ/Z51OLW0kHorCeMd+8NxHBEUTdkc/FHMLmMAHuP+cvLAby2?= =?us-ascii?Q?YdZ/LlP0g9hXCRnY6CL7tE7cf3R6EK9WmupFP/48x4XVLTTdY++5cc9T0C1V?= =?us-ascii?Q?+vGz+DFQMq1Kvn5kO/Eg3SlUrqPECv9aFm2YVFkqzh2vR0gfvB0thsMHxVpN?= =?us-ascii?Q?6r0Ukx7DgB692I6v9Nbd9wdu9PMZO526boo7j6Cpc0kUTtsx7KTrMYnwdJab?= =?us-ascii?Q?Kn0ck7hVBNfu0NeHnlXnVivUda/G+F5PNz3zcvAKWuXSYpaDGnhWakBxhR97?= =?us-ascii?Q?+Euuz1litPE7nkk1OaoZ7/spErIMpNOSdbKBI3nWzGngc7LsfcJkhDrKPFtY?= =?us-ascii?Q?zyybzw0vwLhgKf299gFmTNEiZ3K0585Ue2RxOX/cAohnnetH1LLcdTk8w7mK?= =?us-ascii?Q?MX4l4StXuTg9gKe1HYrI6IYiUy+qZSdbNVozAt21/mL8FSlXr/wn5pD9iHCP?= =?us-ascii?Q?lSSZ/u6MlzAZub7mL81Pp1/ILHHEUCIUiurhbBZvyG+JB39NwJkFroVSNxhn?= =?us-ascii?Q?4R7dK7TFNvuYEHzmfYxH0repA2mXGpe8M6ziGClh+3Rsfq9ggCtD7o2TyqxV?= =?us-ascii?Q?lDBl9u8QQeeLuAgesrMM2SidVcqIsZma9KaVhFCUng3xGUEtxySLAmB1G0gQ?= =?us-ascii?Q?fbVVwgx6oPC1Nyx4Kb9mtwQtXx/fGcqDe1DydJYpgkFaUzRyYArCREC70dEl?= =?us-ascii?Q?qcLorHqWO+nvbeORCP6xqITKUdLago6Np5jxHdgpUuHhYMAxjgCVnCIMLuW3?= =?us-ascii?Q?6DAaCRulX+CMZVHDHZdQOE+5Y/u68pSUiba8rkgl0SuKEHaTnOf32iOp/N1N?= =?us-ascii?Q?QgcWYJw7IBNzEcpw7dRPlva8drlOvptsdV/AGTS78GUOvxsybJimVa9zQQx8?= =?us-ascii?Q?pb89m4wMA8dMKbFYFhmOUkV9FG8DqUJoRd4g2Xs/rIToOkU9//moFhZOygcw?= =?us-ascii?Q?AxbBTh0w8BpeKJYUCgc8v8YsoqlTG4N322VLH3cw0rJZNsWjilat59YxgaOK?= =?us-ascii?Q?Ba/LsRzN+Kljo+RR8cudzRng9l4DCp1mWY1QvjvGxa0q9TuVq4cqBceJU30f?= =?us-ascii?Q?S+S7ottBf+4=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Ogom+2771g6xd/7eydvAUL/ZGh3fmKf0cutHgaxNeSGTzIjSJiK87PeLWJWO?= =?us-ascii?Q?u8LRnkOzk4vDA9J40WLqzBNlfSM211Xkju0GtL6NrAMScAogGKkEJkZtuLs8?= =?us-ascii?Q?zPb4HbW1qP/kfqhyH0TqRlB+onBaT8PUdb0wH4Ii1FxQqYAor4pccaOM/yzC?= =?us-ascii?Q?xGL4vXs4j2kj3OBJhXGLc6/F7Ot7xqrso14rVFCfapTvj9Q5Fj9pI9MMSQsI?= =?us-ascii?Q?Wd4dmUPe/R1ujMIUxIx88Men4DS4L1/4nT4XUrnBPiCioxxXkZqekM0r5x3k?= =?us-ascii?Q?tG9IiJyBy6kbS1464LlwZTJd7aAmB6gsIW0yXx1eOzDxeUTKkxuzJ5g3+1A0?= =?us-ascii?Q?5XkrQ/xTEG5mUl9POIAizlQWQt+UvhzTkyQ2vW+i3nC9p+KJwegEOAwv9Rdt?= =?us-ascii?Q?fhL/xIdWetx9tA7Fxp3ja+Ff+DXNjmcf9La/9LrKwp0nHM7hk1x3Onhcw6B3?= =?us-ascii?Q?i7VbqIbaGxt23S4Afn6Qt3qJnzizFcJHCyhuRwmEE4KWeH6QrU03VadYKsYt?= =?us-ascii?Q?f+kp45cmT8xskx4A+61xfvcwRj25zFyqLxpsZAS0Af3aLTJtvWAFVE+MlXxd?= =?us-ascii?Q?83M+3NMn0J0C9ck5LNiq48ok5cjIhySzoIIQfQpJaKCI+mORtzRRa6N7rRCd?= =?us-ascii?Q?yud8Gbrl3kD/l3iYD772mi6WM7wpqqGsow/MT8xt99KtxnqPi6DBLzGS0IsV?= =?us-ascii?Q?kCUUEcXTgJNL6Wrhg85yWRQPpPeHAS6dHBx4M6THu7CxojC51KWe2SHJTfrj?= =?us-ascii?Q?Mta69aOk856mOIDcQXHXn06RZUMf/xkB+fCl0z0uInMO7iWPjF6lLQfSVSZj?= =?us-ascii?Q?T4kDWjtexCGc+7stI7b/J070NoikpMVwsNlCRaclpSrWExMNbpU7IwzRFaaE?= =?us-ascii?Q?gTswYpywzA4J8eUa/bPiKyE//E3IGwCgnDFlXnLOE+Fi1xdeOoGvFEejuY+d?= =?us-ascii?Q?MJQ3eTeSEskn0exdMwG4gY/0PObfQU0J7HZ9heHhMtJpNqMHm0vJZFsGem6T?= =?us-ascii?Q?8EgdQqrss+OU+xch7LdXUOlRpsJi4eLVOzkiqj85ofphtgqpGr/n1gQSASUh?= =?us-ascii?Q?uc//EOZu1wcNAMIv/qySZOUE2Uuvvb9SLBjVOSY7BV61mN5SD0pTJ3zogrrU?= =?us-ascii?Q?CirFoJwE8tED5j4Yo+eEewEzgJQ4hOv5IETpLZL8Hh8UABbxQFH6GKy3vQvm?= =?us-ascii?Q?HpkfH+sN5JJBijxJ6iDEroiFA0EyVaNMa7Zahb3ebAUpDho/ztKZJ0u5a+w+?= =?us-ascii?Q?wa27IQTgJd54mXECi5dPzxjcymoyhGihGwXdTQkKZDAV2HuAOmmaM3QOfJYB?= =?us-ascii?Q?PPt2dh6qPfDogg6v3jGOVlL8oBzKOhhCETVT+6hTDGKGVZ35fgYxktl6Q0CH?= =?us-ascii?Q?vX4RCdPuy2VOCxnRcgAHHrWS/JWfQGPcHsW2Ei151z4HolsBxUPyUwpQbKvo?= =?us-ascii?Q?jUC4s0tMhE8yOEZvC4MTWZ3+CVvxT7rIDKAAULbkNL3OIBrm7LTkY04HUZZT?= =?us-ascii?Q?CnF7Wh8beOykexwJO9OGLvA6JxYmBQsIkjRq20Rq6luZmrlJajnBnCqFUHJD?= =?us-ascii?Q?jfnZwGr8Znjlc7cgwogyRuNOkZst3aHA5osD0eh0?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: aa751081-95d6-4713-f5de-08ddb41ecdd0 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:55.6618 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bIcE5rtD3jk1I0VMEM7aPkFqKkCFOzOTZ72bKTZLcYx9mx1T53HTUF9XBbbZlc/9BclJFBaFy2v4zUNooQdCXQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Authority-Analysis: v=2.4 cv=fPo53Yae c=1 sm=1 tr=0 ts=685c4e71 cx=c_pps a=iWk1tyVozQW8pnTusJ1YCw==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=vgBDHfCE-WenxT7XOAUA:9 X-Proofpoint-ORIG-GUID: 3KckbvmXBpKCfLpWDQwCumcdBfolj7fp X-Proofpoint-GUID: 3KckbvmXBpKCfLpWDQwCumcdBfolj7fp X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX8P4ngin/pSUP k292SL9p2d67UThrM+uq9o6eIKv5IL9PHHn2VMeCjiEd1bNhj3AB9eGHLeHex0JBNfw8iNsiAXK 7C0kYJfn7BgX9AyiZ5fbCRgmn7a4Lg5kjqpmVjNLs3Jh41u9SgdDZsrWsWbW4Sq0/VJ/wLIvqQh UV/Q4HE76J2s65flgR/CFWNU55+DFzUua1cdN2A/UZgjn/qz8Ype4FErMEQVxIGsM5aw8d6EjYZ VClXeti0kCtuH76+Y3nfPUoSNZfuD0zVlElMUPvnIbg99gUYujbTSLRDAhpYteIoRAvgY+9fdso dwrj33MK7Me5G1bJDj3ecTYgQEhrUxzmn2QRbjj6mkar71La59NMIUiA//o0j3Z1nh2kHouqAqz fWSgG5HS9WvlJu1OlSqzL2PFa3vKlI6G2EmjakDXBgH4c0GSQKxDaP3ktENWt4BSZhQi/vLz X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750880108726116600 Content-Type: text/plain; charset="utf-8" Unlike most other messages, this is a server->client message, for when a server wants to do "DMA"; this is slow, so normally the server has memory directly mapped instead. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/protocol.h | 13 ++++- hw/vfio-user/proxy.h | 3 ++ hw/vfio-user/pci.c | 111 ++++++++++++++++++++++++++++++++++++++++ hw/vfio-user/proxy.c | 97 +++++++++++++++++++++++++++++++++++ 4 files changed, 223 insertions(+), 1 deletion(-) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 524f3d633a..3e9d8e576b 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -200,7 +200,18 @@ typedef struct { char data[]; } VFIOUserRegionRW; =20 -/*imported from struct vfio_bitmap */ +/* + * VFIO_USER_DMA_READ + * VFIO_USER_DMA_WRITE + */ +typedef struct { + VFIOUserHdr hdr; + uint64_t offset; + uint32_t count; + char data[]; +} VFIOUserDMARW; + +/* imported from struct vfio_bitmap */ typedef struct { uint64_t pgsize; uint64_t size; diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index e2fc83ca3b..39092c08c8 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -101,6 +101,7 @@ void vfio_user_set_handler(VFIODevice *vbasedev, bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp); =20 VFIOUserFDs *vfio_user_getfds(int numfds); +void vfio_user_putfds(VFIOUserMsg *msg); =20 void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, uint32_t size, uint32_t flags); @@ -109,5 +110,7 @@ bool vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUser= Hdr *hdr, VFIOUserFDs *fds, int rsize, Error **errp); bool vfio_user_send_nowait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, VFIOUserFDs *fds, int rsize, Error **errp); +void vfio_user_send_reply(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int size= ); +void vfio_user_send_error(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int erro= r); =20 #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 49d12763ab..040660d197 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -9,6 +9,7 @@ #include #include "qemu/osdep.h" #include "qapi-visit-sockets.h" +#include "qemu/error-report.h" =20 #include "hw/qdev-properties.h" #include "hw/vfio/pci.h" @@ -80,6 +81,95 @@ static void vfio_user_msix_teardown(VFIOPCIDevice *vdev) vdev->msix->pba_region =3D NULL; } =20 +static void vfio_user_dma_read(VFIOPCIDevice *vdev, VFIOUserDMARW *msg) +{ + PCIDevice *pdev =3D &vdev->pdev; + VFIOUserProxy *proxy =3D vdev->vbasedev.proxy; + VFIOUserDMARW *res; + MemTxResult r; + size_t size; + + if (msg->hdr.size < sizeof(*msg)) { + vfio_user_send_error(proxy, &msg->hdr, EINVAL); + return; + } + if (msg->count > proxy->max_xfer_size) { + vfio_user_send_error(proxy, &msg->hdr, E2BIG); + return; + } + + /* switch to our own message buffer */ + size =3D msg->count + sizeof(VFIOUserDMARW); + res =3D g_malloc0(size); + memcpy(res, msg, sizeof(*res)); + g_free(msg); + + r =3D pci_dma_read(pdev, res->offset, &res->data, res->count); + + switch (r) { + case MEMTX_OK: + if (res->hdr.flags & VFIO_USER_NO_REPLY) { + g_free(res); + return; + } + vfio_user_send_reply(proxy, &res->hdr, size); + break; + case MEMTX_ERROR: + vfio_user_send_error(proxy, &res->hdr, EFAULT); + break; + case MEMTX_DECODE_ERROR: + vfio_user_send_error(proxy, &res->hdr, ENODEV); + break; + case MEMTX_ACCESS_ERROR: + vfio_user_send_error(proxy, &res->hdr, EPERM); + break; + default: + error_printf("vfio_user_dma_read unknown error %d\n", r); + vfio_user_send_error(vdev->vbasedev.proxy, &res->hdr, EINVAL); + } +} + +static void vfio_user_dma_write(VFIOPCIDevice *vdev, VFIOUserDMARW *msg) +{ + PCIDevice *pdev =3D &vdev->pdev; + VFIOUserProxy *proxy =3D vdev->vbasedev.proxy; + MemTxResult r; + + if (msg->hdr.size < sizeof(*msg)) { + vfio_user_send_error(proxy, &msg->hdr, EINVAL); + return; + } + /* make sure transfer count isn't larger than the message data */ + if (msg->count > msg->hdr.size - sizeof(*msg)) { + vfio_user_send_error(proxy, &msg->hdr, E2BIG); + return; + } + + r =3D pci_dma_write(pdev, msg->offset, &msg->data, msg->count); + + switch (r) { + case MEMTX_OK: + if ((msg->hdr.flags & VFIO_USER_NO_REPLY) =3D=3D 0) { + vfio_user_send_reply(proxy, &msg->hdr, sizeof(msg->hdr)); + } else { + g_free(msg); + } + break; + case MEMTX_ERROR: + vfio_user_send_error(proxy, &msg->hdr, EFAULT); + break; + case MEMTX_DECODE_ERROR: + vfio_user_send_error(proxy, &msg->hdr, ENODEV); + break; + case MEMTX_ACCESS_ERROR: + vfio_user_send_error(proxy, &msg->hdr, EPERM); + break; + default: + error_printf("vfio_user_dma_write unknown error %d\n", r); + vfio_user_send_error(vdev->vbasedev.proxy, &msg->hdr, EINVAL); + } +} + /* * Incoming request message callback. * @@ -87,7 +177,28 @@ static void vfio_user_msix_teardown(VFIOPCIDevice *vdev) */ static void vfio_user_pci_process_req(void *opaque, VFIOUserMsg *msg) { + VFIOPCIDevice *vdev =3D opaque; + VFIOUserHdr *hdr =3D msg->hdr; + + /* no incoming PCI requests pass FDs */ + if (msg->fds !=3D NULL) { + vfio_user_send_error(vdev->vbasedev.proxy, hdr, EINVAL); + vfio_user_putfds(msg); + return; + } =20 + switch (hdr->command) { + case VFIO_USER_DMA_READ: + vfio_user_dma_read(vdev, (VFIOUserDMARW *)hdr); + break; + case VFIO_USER_DMA_WRITE: + vfio_user_dma_write(vdev, (VFIOUserDMARW *)hdr); + break; + default: + error_printf("vfio_user_pci_process_req unknown cmd %d\n", + hdr->command); + vfio_user_send_error(vdev->vbasedev.proxy, hdr, ENOSYS); + } } =20 /* diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index c8ae8a59b4..cb93d9a660 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -347,6 +347,10 @@ static int vfio_user_recv_one(VFIOUserProxy *proxy, Er= ror **errp) *msg->hdr =3D hdr; data =3D (char *)msg->hdr + sizeof(hdr); } else { + if (hdr.size > proxy->max_xfer_size + sizeof(VFIOUserDMARW)) { + error_setg(errp, "vfio_user_recv request larger than max"); + goto err; + } buf =3D g_malloc0(hdr.size); memcpy(buf, &hdr, sizeof(hdr)); data =3D buf + sizeof(hdr); @@ -702,6 +706,40 @@ bool vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUse= rHdr *hdr, return ok; } =20 +/* + * async send - msg can be queued, but will be freed when sent + * + * Returns false on failure, in which case @errp will be populated. + * + * In either case, ownership of @hdr and @fds is taken, and the caller must + * *not* free them itself. + */ +static bool vfio_user_send_async(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, Error **errp) +{ + VFIOUserMsg *msg; + + QEMU_LOCK_GUARD(&proxy->lock); + + msg =3D vfio_user_getmsg(proxy, hdr, fds); + msg->id =3D hdr->id; + msg->rsize =3D 0; + msg->type =3D VFIO_MSG_ASYNC; + + if (!(hdr->flags & (VFIO_USER_NO_REPLY | VFIO_USER_REPLY))) { + error_setg_errno(errp, EINVAL, "%s on sync message", __func__); + vfio_user_recycle(proxy, msg); + return false; + } + + if (!vfio_user_send_queued(proxy, msg, errp)) { + vfio_user_recycle(proxy, msg); + return false; + } + + return true; +} + void vfio_user_wait_reqs(VFIOUserProxy *proxy) { VFIOUserMsg *msg; @@ -746,6 +784,65 @@ void vfio_user_wait_reqs(VFIOUserProxy *proxy) qemu_mutex_unlock(&proxy->lock); } =20 +/* + * Reply to an incoming request. + */ +void vfio_user_send_reply(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int size) +{ + Error *local_err =3D NULL; + + if (size < sizeof(VFIOUserHdr)) { + error_printf("%s: size too small", __func__); + g_free(hdr); + return; + } + + /* + * convert header to associated reply + */ + hdr->flags =3D VFIO_USER_REPLY; + hdr->size =3D size; + + if (!vfio_user_send_async(proxy, hdr, NULL, &local_err)) { + error_report_err(local_err); + } +} + +/* + * Send an error reply to an incoming request. + */ +void vfio_user_send_error(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int erro= r) +{ + Error *local_err =3D NULL; + + /* + * convert header to associated reply + */ + hdr->flags =3D VFIO_USER_REPLY; + hdr->flags |=3D VFIO_USER_ERROR; + hdr->error_reply =3D error; + hdr->size =3D sizeof(*hdr); + + if (!vfio_user_send_async(proxy, hdr, NULL, &local_err)) { + error_report_err(local_err); + } +} + +/* + * Close FDs erroneously received in an incoming request. + */ +void vfio_user_putfds(VFIOUserMsg *msg) +{ + VFIOUserFDs *fds =3D msg->fds; + int i; + + for (i =3D 0; i < fds->recv_fds; i++) { + close(fds->fds[i]); + } + g_free(fds); + msg->fds =3D NULL; +} + static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); =20 --=20 2.43.0 From nobody Sat Nov 15 14:11:14 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750879885; cv=pass; d=zohomail.com; s=zohoarc; b=dpKSiozlw3CXtMdCFLBMJNaXPFi/WC1/Ll6+m7s36YHbUcda7Vf2XjzSdhA570nL3qC30bJ7tkpkETXojjhePhK+bl+Nr93lNNOhoXSx8YFvI6tjwNiJEWUI86rP7il6XUgz1pcuye0oAHFg0BkR6ASy7lUMG2EkPUfLajllzNI= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750879885; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=OYnVMjMjqwV7IClbxFzR8NdWCf4SvGmTzKBwYz86XCQ=; b=CevjKeLj/YM7fL7FIVmQjaULvJxQOxOLQp9Euq9Qd98qPgEDcHuiBNloL2k6M4E5OG0lsyS0/f4r0p+JiUAv5D5IYoG3dQyWBVb0Or2+9EnPxXnDfuoEibk1qV58XJEeYKawr4pV8XDsLAIWVq7zTIRzVIymlnRjTU6RXcH+Jzs= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750879885764512.0498419900395; Wed, 25 Jun 2025 12:31:25 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpr-00088N-Gd; Wed, 25 Jun 2025 15:31:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpp-00086s-Gu for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:05 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpn-0004C2-8S for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:05 -0400 Received: from pps.filterd (m0127839.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PErIon010338; Wed, 25 Jun 2025 12:31:01 -0700 Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04on2136.outbound.protection.outlook.com [40.107.100.136]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 47duth1k8c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:31:00 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:30:58 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:30:58 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=OYnVMjMjqwV7IClbxFzR8NdWCf4SvGmTzKBwYz86X CQ=; b=avDLLxZAuUb71vbev9hd0qYEhI4VT4O5Hz2+q92RgOBW9JCvZChBqTZK7 laZlXa9npEG/whprRAg9gJYryPSdfrCROOdnMNVK9rYcyg5plYdNelbdQSH+EkXa 7EPchICTXP+lXyDLdS+ZMYr2/oACcyPqhRtusji7tdEemDoE8vbpmirlSo3Mn74O FoVB3M5M9j8DXHx78/VcggZQHyb2TOj7wtJJayJc+G1qqNqbqh9s+sPZBfjwIwF5 SDaR0x1RskCdfDvAFgOxlhEK55HkVIJ8SMSMKAYRj19A0YmdvZ/rvu6rzfa4DtKN 3WQ3rHm/lkrA/zpqXqeHS8+sqyqIg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jaEshoSxUAbyofUEXi1JzNAeEL4vC3TDtMiNbzwtAuyv3InP23Sey9fA8+9qvtz/CrtUECArJ/ov6Yjv799YheNRd5anOHP+l6mWO/PObhsosULbyLfgkEu1xdELRUO+sURlcGAvTmL6Ax/ZIZXLLFaCxkyBEhR7JBY67S3/3OCC3awTcu57ylaS/U07CkdBQOVZjhJyNOy6mwGdYLNH7bWhyl9WChRKBWyMFE39Vb9ZtXbRwd0dNgcQfb3RHC+brNKyS+5Fa/7H+RWAVMXpZAaDWglAnYx4qXj/NbtGhcNH7yP3B04lIgIUoTELDdjjY9bMbd+vEBzRIS85U1EZUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OYnVMjMjqwV7IClbxFzR8NdWCf4SvGmTzKBwYz86XCQ=; b=T+b1ixLVJca9J/4KDxt7WGBCsRtyliDSljyKafeakVU2yJBbN4QIZ0y6csOEt74UK4HdJ1cK7EmuHGPGmo88EV3mPLcUlHrW1UcY2OM0d6uPEscJ3vu66NmU4gNm/+5MaY6yirDmcK4jSFVTTZENL9rTLtM8K9JY0qWSSAY2gwQPndCc5ZL1wliCQXG0omrVyHuUKjSmxrEVcBOrbz5hcWTxJOw5XBbTO43hpSC0xc/VtzJll8KNspFCfbIWcEO+RRXCJOu4oMdM0Edsrp4xHWfLun5IvMcEFjbijX5lazm+aFcIP9Mot68HBxbfzyuCOUDi2qR/Vg53Y01lCtVscQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OYnVMjMjqwV7IClbxFzR8NdWCf4SvGmTzKBwYz86XCQ=; b=nePc/zUw98dLIO3HG/iLqBnn8kQjAxpMR9F5BUjeNrWqSAvu9Fh1RCo4xywzUJRQ0EEZUouhMDsQPYJw3/j/Bs8apX1FZ7JuesLsRYrmtO4eK9kEghv/pvGw1LkUBu/1xcDno6tDVFC2Roc7vBnb7lXPM1+sGpXwQ2vZsa6K13vhN0u74AwBb/UCCy4oBv9S36bXxcI2VsJ4AbwtCe2BZVuMF6v3DCh3qoFHKWrBs8YOZCHordZlyPoxwSSrt7IP5YSTXQG5iZrbKVNrrEEs3hcPYL655yqQvi1nejFCa1wQAY+/PuYhSoiqbnprzzzjZ+NCH1/WvhuGVuZs4x1rAw== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 15/19] vfio-user: add 'x-msg-timeout' option Date: Wed, 25 Jun 2025 20:30:07 +0100 Message-ID: <20250625193012.2316242-16-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: 111983de-855a-46e9-65a5-08ddb41ecfb7 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?aECueuyAC7EPI3RYbgjP27LqOQD48+kTAdm2jcNSNeNtvA79nesD8J9aU6xp?= =?us-ascii?Q?w4jT29OX2aB5aXrBqQhFwE2yhcxpBdP/2C6nIsnlVp7+nMUSZRfiEoIfaRTV?= =?us-ascii?Q?NYpe/UGduonsIUzGLBBJP+/NJmgEW5hYPxGnrewDZXnBf0MNhs2tNv6fAZSE?= =?us-ascii?Q?8VWnG61piFssH3X1N4nL9bNT+DT3EPnOxUdx8x+a3QVLyd8cudfvdI4rnZHn?= =?us-ascii?Q?EGIN19mV5+++9FIuCzgGgdYTIb6i378RUaYhXwLPkcCqHiqdBVGfotqLi8Gm?= =?us-ascii?Q?z7CDyfveLPhCpPr6CdIfw60vlUQr4SlB3nukFQUe0EIhZDeN38Lr5r9AP2qp?= =?us-ascii?Q?jGpvLbOQmVOZvUbYDbZigN/TF0RDii2Wb1tmBJqtgTH7DvWIP6GIshNwZP6g?= =?us-ascii?Q?Qx0F05ovwgwUs16Labd+WnvWgTb3UgqA2zJro4R3YYf9T5fFwTWRdGm4EzQ4?= =?us-ascii?Q?WE128dxjToXl3YPFHIUqd19D+cJTMQFTptpeCo/Nlrray/xW0WT4GMZsUBYG?= =?us-ascii?Q?RnFmcTZWOVqG03ZROx87e3y9iYFpNtfrsngL8nXgZmBWubSNbUoBdc0+yXbL?= =?us-ascii?Q?b+Ax1zF3URLPNFRh8hKEoOr1JsKigTV9bwyckYCZZdd4km2+bSW/3HcOsxIz?= =?us-ascii?Q?QlbkphJbkrFdwaoKtqEx+ESJ59ks0vVDSfyhTeV7uSsch5E7l8/N+tRqHY8m?= =?us-ascii?Q?/zL3iQzqeNxJRXTbuNC3Q5U+koSMzPKXWS9xyA/70V+LLaODnYNgIC1Ra0At?= =?us-ascii?Q?qFKMS7J2nhyRUjSYLcuis4vSyNGSdNz1xa6WVajppN9xmeJER5Clob8okZfr?= =?us-ascii?Q?OYQkCf2QVU+PflOCCKGMNsY3grGIzHvwUElP6VGKMBch1Sa5QE826qENNHVZ?= =?us-ascii?Q?eNhFkewym98AhyJjuutMAkKYTXzz1yHQehDKIbgXkySNE19ByhlQYysaeiu4?= =?us-ascii?Q?uKO09YO5xNauIsGsCxTv7qQnMHT0mLWy6koygJlXJHcG5K0MgadLe2xQe4zk?= =?us-ascii?Q?JIZfxW5NlDfvbezudNA8nc4LxNvZWpZ2iAUVTzoOVCbNIaKZjNwfluRmuhvr?= =?us-ascii?Q?4IojUxfPwN1/3W/Q6KpdO8HhHrorvfBbLrj1z4eCtoUFUqxOjBf/36eanHO6?= =?us-ascii?Q?eMWyixAIvX7zbRaz2FpTl36iJAewFJ2EflwWl9tFNU9/G+9C3DkdiV6qdXNh?= =?us-ascii?Q?cWrn0RXFylSKSNwFLUFjplx3T/6357+0yGT27ZANkTyNHqpBHAipTQlEJF9V?= =?us-ascii?Q?CAF4esWsmTIQcdI9zdk+PpuLXfvTiEZKjcaTcSF44xuhBMDTftlDVrcPURng?= =?us-ascii?Q?h5EbxrC/qVWigZsp8tZu3nysUhGDHq6uFvCjbbTxOigR7zEqvKPKJ5tcWOkU?= =?us-ascii?Q?Q71pn3mLuG7bU0xzbvllWTEf6SFoh96CWaCY4splzAuvPy+7CZ5+NYDOL6l6?= =?us-ascii?Q?NmTye0/mJ/Q=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?9IqSC+y8S0Goev0kL63+2/WTUBSRgeEUa5Fe/jS7CCOuUS5R1c4WGY2JuRdx?= =?us-ascii?Q?OGEnKXXDeRwGrBd1wqXH1g8tzsoKIw6t2y4dKyamG+t0B77sV9GXem80756A?= =?us-ascii?Q?biynj5zFvU+eTebaqGD/CYTz+GiPA7DZaADxaYmBEwVaecQX49UjHIKgjmib?= =?us-ascii?Q?TEkrvH+apILtUuoTcdar8khyw6Se2L6rqLVdcLvGiR93j9K4aKh3+dW+iDM4?= =?us-ascii?Q?LnVElnKb3DJo5fq885EVoFAIcRSFtOBqt9dfRmifLFjVcPasgd3opEblzOpQ?= =?us-ascii?Q?obBMaCFHP5CmCXXzjdekcHLE7V3zVNpB8hy6xFnn+vEvBrtolgvrVZ1+U+q6?= =?us-ascii?Q?8Z0kw7b90yjJV04ERMJN0+AIo5dXOmRT1eSc93sHG+Sud6H2KycwToTIb0No?= =?us-ascii?Q?stHP14U2V0lwgzC4st6BNyIihphpYIqVTH9MDD3AoJjZk9XSEe/FceRlF/sn?= =?us-ascii?Q?lPNr5YOX9XflqaS4a+3AYul4LpNKvTPkDhsLzjxrlLmkRSj/BbNJ/F+onhv+?= =?us-ascii?Q?6WP3u2AXf02M25TguqURvVSKDmp2WFylNEPYIM6DS+eiKjyNk5S24P4y4PRx?= =?us-ascii?Q?Fue5mqFpkRae2Ccc9gplyKkzBfH3w7HdIoy8Q38Znx7yt8QjFCa0Pke4Wgp+?= =?us-ascii?Q?UGvDL+MG8n5cUQ1bQX3K4UO1eQDlY8C5HACE1MjPWPGfecHsrJCZSsPVTid1?= =?us-ascii?Q?ZDBvFM2mTWvxNS/yWelbZV92JuoM+lA06Gj/C4yqVCgzhiPyiPAv2b08PVzQ?= =?us-ascii?Q?7EbPEgFZgSggNm62nZkovfoUqq7Lb71Qo1/kwbOiHBHmYwCRRx8GxQbcNane?= =?us-ascii?Q?2F2vbT3MlGuere6BMFWcodMEi4KmxJ7UZVTIhgF3pxSKuuDThR5d+OmXg5eE?= =?us-ascii?Q?X3E9BrvD9LBXG2jp9hXDDUO18DFI3LFphIBpg1UPSRlul8tbBQrOxrqDa3Re?= =?us-ascii?Q?gqsWobT+aTwYT7mZO+hd8uzpRS27z21jc55EdCqsnYk8R5hAa674pbxcq1/G?= =?us-ascii?Q?dxw6mlVDyYBiq6pFau2qUR0X8Lzvkfxxq8I2nqW4W0+91zNH2zLvCTSHlF36?= =?us-ascii?Q?8jj0DaXN115LXRYZ2doV0GEiiOrtQDomgr22NWShpPddOUUqv5MXgQ8BDuLl?= =?us-ascii?Q?S91Wq69IUrPDGRCYYF1+sIyjr5hCcVyP6aLY1rE9B9UjzyV/OQtLcZ8m4SZO?= =?us-ascii?Q?N1kt1A+fkxeVkQTl3j/0E8qJ0OimjKHqKoVtHPUDtpJlzWSvnco9fkGttGHI?= =?us-ascii?Q?teIhrRLTTH5/MYQh9Ab52Tbm9gs3x4lhLS76+vuSp7toPmqNfNqXbjEDXIcw?= =?us-ascii?Q?I0HfCDkIANP1+q91aZhtvd1qP7MbzRuHDmNP1z2F8ilxELvyFtNTfaBCWcfP?= =?us-ascii?Q?Fj0MDdI1zMX9tbjqb9B+Q3DZcAci87nVD2dLtTzg10lGbmkwuJ7hqr9mzh25?= =?us-ascii?Q?NG7BNX89qG9Y/3gas5Xm+4uCTN/jMK7Qu0ljh5dWEpA8/Ck/YYiun7/y+tdh?= =?us-ascii?Q?aPUObANJrh5OR1o8PEqvPaIxMezBZ9KCPB9FoRePJR1WeHyohJfvZcN85/eJ?= =?us-ascii?Q?3X8vHWsSyk6Whj7JWgaOLtd/2hzakSXT/yRib8D2?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 111983de-855a-46e9-65a5-08ddb41ecfb7 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:30:58.7067 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: oabltOSy8q9J10Qo0RnUnimcw6H2AuYAH+FUluhCnnGbjCAbI3bUKT1GnuK6I1neFAOO+UA6nAVdxMQVQQB+ag== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX5OkemK9WFpvY YyPDReeBL0XcD/fzKWBHgRrGWGIUDva42FWws50xh1pPi/BgozFaUzofh1lxRu2aUV0xRI7dVKA EmOLQgpo20bpR5ChxvjwRnpjqETR99ljMV7bsCJbMq0vRt4V0k1JyL8bv0h39I/5yEXpBklrJwY UCgrzp0iAa7K3pBM1EScJ/fXqT8G3G0g3x3L7OccQCVXeiBB64CJjaN5PypsjS0k9mLApQtY/v9 g4rFhR4FlMUvIf6W77AI1WcoHF6xl38vKlPnymWfwGcgrlK4MSUqDUhwa/aAfFm+alq+t+SmzZA yMBS3cMGwIXg2TMEkVKfV341p48dlDg5m3+gjyJmnJN7Fu+6/8ziyySp0dJo/IYIVRYToA/gp+c ko8rQd76ud/TxwQxJiFTZOh/P6ZUQSKijbL/TSRw0j1FEhKmxDePaBAz0KpZ8apgcnqZ2yz9 X-Authority-Analysis: v=2.4 cv=WNp/XmsR c=1 sm=1 tr=0 ts=685c4e74 cx=c_pps a=D0xTqnO8G3g1I/JDCPG9pA==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=Ql6ANm6PklPDYutWWPYA:9 X-Proofpoint-ORIG-GUID: 2A4XuhGc0RTpyyAxeYNGjJeDZEnrctPP X-Proofpoint-GUID: 2A4XuhGc0RTpyyAxeYNGjJeDZEnrctPP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750879888160116600 Content-Type: text/plain; charset="utf-8" By default, the vfio-user subsystem will wait 5 seconds for a message reply from the server. Add an option to allow this to be configurable. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/proxy.h | 1 + hw/vfio-user/pci.c | 5 +++++ hw/vfio-user/proxy.c | 7 ++++--- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 39092c08c8..6b29cd7cd3 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -67,6 +67,7 @@ typedef struct VFIOUserProxy { uint64_t max_bitmap; uint64_t migr_pgsize; int flags; + uint32_t wait_time; QemuCond close_cv; AioContext *ctx; QEMUBH *req_bh; diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 040660d197..f260bea490 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -23,6 +23,7 @@ struct VFIOUserPCIDevice { VFIOPCIDevice device; SocketAddress *socket; bool send_queued; /* all sends are queued */ + uint32_t wait_time; /* timeout for message replies */ }; =20 /* @@ -267,6 +268,9 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Erro= r **errp) proxy->flags |=3D VFIO_PROXY_FORCE_QUEUED; } =20 + /* user specified or 5 sec default */ + proxy->wait_time =3D udev->wait_time; + if (!vfio_user_validate_version(proxy, errp)) { goto error; } @@ -398,6 +402,7 @@ static const Property vfio_user_pci_dev_properties[] = =3D { DEFINE_PROP_UINT32("x-pci-sub-device-id", VFIOPCIDevice, sub_device_id, PCI_ANY_ID), DEFINE_PROP_BOOL("x-send-queued", VFIOUserPCIDevice, send_queued, fals= e), + DEFINE_PROP_UINT32("x-msg-timeout", VFIOUserPCIDevice, wait_time, 5000= ), }; =20 static void vfio_user_pci_set_socket(Object *obj, Visitor *v, const char *= name, diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index cb93d9a660..c3724ba212 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -21,7 +21,6 @@ #include "qemu/main-loop.h" #include "system/iothread.h" =20 -static int wait_time =3D 5000; /* wait up to 5 sec for busy servers */ static IOThread *vfio_user_iothread; =20 static void vfio_user_shutdown(VFIOUserProxy *proxy); @@ -686,7 +685,8 @@ bool vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUser= Hdr *hdr, =20 if (ok) { while (!msg->complete) { - if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, wait_time)) { + if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, + proxy->wait_time)) { VFIOUserMsgQ *list; =20 list =3D msg->pending ? &proxy->pending : &proxy->outgoing; @@ -758,7 +758,8 @@ void vfio_user_wait_reqs(VFIOUserProxy *proxy) msg->type =3D VFIO_MSG_WAIT; proxy->last_nowait =3D NULL; while (!msg->complete) { - if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, wait_time)) { + if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, + proxy->wait_time)) { VFIOUserMsgQ *list; =20 list =3D msg->pending ? &proxy->pending : &proxy->outgoing; --=20 2.43.0 From nobody Sat Nov 15 14:11:14 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750879952; cv=pass; d=zohomail.com; s=zohoarc; b=FhFKTqHFTygs6chvtWQSuyNaM/v7I7RT0xXlrfP4og0gF6SWg5eWDPwoOiHnxsPzWtlOVWXIwtSHmwU8AOHYA80tUxbX1DmfiNWZlOUF4zTkXc9DOgckzq87l5Yyu9Oh9hBzRJ+MOd7jzFJAZlFLCoU+zUN2x1Rd0QjgNpdJD1w= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750879952; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=qc32R3rTh5shTasSV1D3ldFs4qlCdmIFa6pVZyoOBbk=; b=AnPyFoFItGY275Ht4nevcxQnvlLtvzzGLtleCB8Oz6cU/KSJLNxIm+QAaJS65R4d0qpxcrnx11lpr5GvQUS7B4z8ft5ywGcFKw0DA6J8b0qsVFBZZIqoEVbZ+dD4phHIN1SOpqr9ENjJZQd8XIlotuo2xqKnt8ankk1CmWJXdqE= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750879951945377.376582200444; Wed, 25 Jun 2025 12:32:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVpx-0008CP-1R; Wed, 25 Jun 2025 15:31:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVps-00088R-GG for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:10 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpq-0004FW-EJ for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:08 -0400 Received: from pps.filterd (m0127840.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PEqPsu011321; Wed, 25 Jun 2025 12:31:04 -0700 Received: from nam04-bn8-obe.outbound.protection.outlook.com (mail-bn8nam04on2135.outbound.protection.outlook.com [40.107.100.135]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 47gjbh0r9b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:31:03 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by PH0PR02MB8535.namprd02.prod.outlook.com (2603:10b6:510:dd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.28; Wed, 25 Jun 2025 19:31:01 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:31:01 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=qc32R3rTh5shTasSV1D3ldFs4qlCdmIFa6pVZyoOB bk=; b=jgaNHDvr7SoToBaxm49B7h7tFW6fSPSOsUZV7wfDkzhsnRTJyCZqicGTu LX6/RyCcJ5DQoPpNh5T/DtbaYdAfJVUp12kqYzuWACYdAp8bmufu3aFT0UbtCyhx mjZGuIWaESxa4IqjK1zjCZV9ZcjjT00nMRpVHZoIl2dC24vbJ98EmNDBpwT+GmqP PVOzBL098wqRgMHuSt1Hsw1a61K7+E6RFdN7JmQ9Gp6K5v48IR2W7APHgz4f2cKG S6UgcUz3M1lqDRYwSiO0JbYLep35RGS6+dUbpW1O2vtO4lSH51E3H6P/PlYFCBjT tmgwI3JIvOswVEkqgbvmeX5cele1w== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tyZV2DHStmmYij64w9IMw420zLw0M0slZzoS6vk6CU3PeXp5FSOtOT718iw3qFNK8x9BXdZJ6oOrGr0Ab+tMPvtyjFV/vcLt38bXEAg0iDAvr4ezZ10L+48mYjYtcNAzKfyWYm+fb01VzIIS42Wp9hhiRnZRCqirNy7Ipn15RBYg+O7o5ahroRoJWNx3LIahoJrz0lgj/VGBjumiGnMAjSW1T5j2TiNKt7j7uPgLsqb/qaKZWR/QwZ6LxQuE6Bxw4DbBKvQmAp+geC+8WR+zSUHpsv1aqUCdQRU2ZzMrao5AV5HwXcxf29qgAbI0DQcjqil3O5e1x12GpOns2LQfWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qc32R3rTh5shTasSV1D3ldFs4qlCdmIFa6pVZyoOBbk=; b=nINR4UJo2k0DTxv62PA0l+Umh+vqdh/WMEGDVY2PDrL82FqN4seLoA5WpbQexsLFUpcQ7pV69PWAVEsyQx+H4FP0uW1UJ5FMFKkM0ivS901hkvbbzknqlaPvUpHe6gpUxgmbr8rfT2/d60OvXHNRBr76zyC7XZ1zOT6YjaoLQUNiNqouH46tZqZomxb8emsyn5IKdmSJLcGscBbah/Q2fc23v+mutjYFfPBDVca4tRujjI7o9nKRNK66O3wkxsqfsPHslGc8LPfj47bZ/AEfmXZn5PJOC34jlH5980OfKssKFwWEPCvFrOmUc74eGT9RdHhoRPi9VhHZUBxUPBtKtg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qc32R3rTh5shTasSV1D3ldFs4qlCdmIFa6pVZyoOBbk=; b=fKJv39Z7NKK7zU6PVHp8i8pKorPmg+C4v6PgpuDj0xqkIUAxtvdAvKOCk8vOGK7ZCzN2DrL//PSauPLR0N5teIT+XQ1v83EpOJuJQdK/giltaGYbPwVGXcRBuM6fioDsSCUIAIRasQzh+9+m9r1TUwATFV3Wt7UIb1D46w6ydSLrzWYP9URCBr2AkzhSFRRLKexZv0NJLzrIduYIhMyvZcZS/2KjnO/nYUjO9ImEKcJW2x3+Zg0PXGgkqcgevDlZP107GKx6aHy3N9UAL7hZUiMF+z+9FNJ7F9q3amq9P3CCmdUPBjK3mba7DeYzi1M4/FAHDoD6gdWvyd9QCg0pSQ== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PATCH v5 16/19] vfio-user: support posted writes Date: Wed, 25 Jun 2025 20:30:08 +0100 Message-ID: <20250625193012.2316242-17-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|PH0PR02MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: 4034be1a-242d-407f-8b62-08ddb41ed17b x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?WLyf3wfXjMc9h4S04o/rXuTDuo3bqa2DiV8EJiUfZIwKQ0KSCjKWJqkeIwXn?= =?us-ascii?Q?hwyE/NDcyCRyKQROWSByMkXyQwU8MOxJ4T73J5VWBDLJwGx34y5rHtaMlg9z?= =?us-ascii?Q?pVLwQVdl9Ezv7cn7u49IeFaG6XHrEJZDWyFogKx804kiWYyNRNO7XY40T2cC?= =?us-ascii?Q?nzlCLuaSkyzWIdETcT0SYeTvjdUiB4wcvtfuhmkhRjckk8SowWMztilfGkJF?= =?us-ascii?Q?6hr+KnuyImvfVTz5Liy6U+SEqr3vyOJJBrNFOFiNUOES79bsiYPuRVv0drYm?= =?us-ascii?Q?W6E8Cty1hNaPU0Nz67xtnOFzUs5kx901ERG2ZLq9XqvDho3qcLaREeMUgve0?= =?us-ascii?Q?h3qkOTROx6smEGIcXRpCDNrvg+1MP2SB1jfPLLp5jt/Gf8xhWi2xra3JAYVy?= =?us-ascii?Q?72ic9CyZPbL/9mCLkx7ijBPAxHt2/sIr5GKAISN0NpYQoNq8YEHSDtyd9vtz?= =?us-ascii?Q?ZC/jN9V6UN1LoUssbAz0iRzzlHbxojsoQZVAppP6mYvKmq0hZsnivce6A3ay?= =?us-ascii?Q?CbNuCdtTOhhx4Is9SiL1tPIRqDqADRSL/o/5RuNuSOBSWi2lRow4wuxZ+4fN?= =?us-ascii?Q?AYB9HguEThQT04ub4qnB90NEvSZIqIkCAYPTE+3Qr/60FAa5O30DImlR5cXR?= =?us-ascii?Q?OeTs4LtFMVufpDTNJ0sjWnCnSyBXU8zGVNr5M4ZmEQ1ruGENidYqNzm2iUAH?= =?us-ascii?Q?uZp5sB0t76NkExyFlsK6MQacZ89zOhnq45DJTG8YAmVPKm7ubSkpnP8o+YmB?= =?us-ascii?Q?I69x8ArW8FepDiSW1W5u1TALug10A4flOr7PFZRo7Uh7zIvAB4uCfgR47lbg?= =?us-ascii?Q?8NFiuI45nsQh16vKAdmP4WnNJkNIyI2ONbRCAhq7zgPdrxws5sgJIsgANx5x?= =?us-ascii?Q?CmCDJghcUUyUcsGxyLKIFZG6SalweiwLjQhqdZ6TANkCFgtyy5Vi2u8UvzU7?= =?us-ascii?Q?P2F0ugT6i5MNlqHobpAboG6nzWgZkCfpKRdiG2kOhqH9vMz8iO9cWvlSWko7?= =?us-ascii?Q?eSavhBB/oPE6RB8VpN1lA+ojOSr8oOfMfUj/YRoOjgiZOYfBVpiTfybLYHS4?= =?us-ascii?Q?8AH2c+Xgfn3qTWubL2OBx8cG5W1efPbMDaAz/vMpYBXpqU2kvwlz1A8sMNl3?= =?us-ascii?Q?r2HDFx33K9olhmHfOeng5gl0e1EfnMo8249+Bpp+N1GdYfaToxRfnNAH50m0?= =?us-ascii?Q?3kOM1RQ4ggqd/14YLYNd9Dzq3l8coT+kit8gsKJ4y7fjMnv6ufpMosUJmKgy?= =?us-ascii?Q?ehnaWVBnSDPA7NBtfIQhvbSBnBhPBEr3kvCBkYY/wZcy1PTpV+4dz6BDleNY?= =?us-ascii?Q?akWI0IGxa+5cIa1WYOXuUSl4jPZKe4v1bg9E/G7h5xZeSlUwmVl+4uJkjUrH?= =?us-ascii?Q?Vn/J6vimv063CAg5fFQN93tIF8iwRNfxvM8HOApMtkQlbWAhqi6CpNVRWyUj?= =?us-ascii?Q?SWY8kb6yCwc=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(366016)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?w9TkEnettePSEuTNmmHj/NRT3pDxWOhcayN7wwaUK2gSAX0vfbdknrOl1W6E?= =?us-ascii?Q?dJnlPO3iNtpe6m3jnIVC14AZAl4soyBagV4wx211QTjNI8/ZFB7fxX0Tmv4t?= =?us-ascii?Q?AiOP6errQpt5JjcUyiqcHb3tFBWRN+x0FOw1EZyq+4Y/mTxDuEq/QLQjTUB3?= =?us-ascii?Q?KDCy1MGH5O3w6n1jSVnajYoBB+IRVV8/k4P8lfqy2HrdNviQ7JZntul414yW?= =?us-ascii?Q?7okFTHl3FcL34vy7j9icVIFaDTlV/WQabKi3Vr1ejAYsSQTBEEEBUbjQrLKS?= =?us-ascii?Q?4LfWz6JlEoRHRrQxtfy98myXKo8HBS4AAHWk0FLyQnTFMz+l5rOf38q/Z9ZU?= =?us-ascii?Q?A7NDIBFRusZGGM4qBzunKMhbBg7wTnekBAxPpB1BePsEcm76KWw2Rm3CExa4?= =?us-ascii?Q?DqeK1mBZBuprZmFVtbb4OI6z80IWcYmloYYx3eKlfsfUezXIIOn7nQvNCGJV?= =?us-ascii?Q?0oCDp85STYitxsawOwWmH3SHARqO17DGNj1IdrzRRWgkWRpEWKraosNFeNWh?= =?us-ascii?Q?WdaC5xzeKx/1Vuqaiix0KSS3N2QMs3Yihm24NHojPTwrqQTk0E+URR11hl9Q?= =?us-ascii?Q?rYDzrBCbCgnM96zhfJRUl5QozRfUZ48zVinyTemLAqrIkHSLaZ6luBZZ+iUt?= =?us-ascii?Q?qyzOYL6CniG4onZgV1PTEtPgwT7c9dMjvK1Jao/hZzcjCR9k+bjhnKLA8K/4?= =?us-ascii?Q?k4HcoqHapwsFruYoalWXqkhOBZvY3Nc/NpuFLHp0kdVngGlXge6v8swA5gYb?= =?us-ascii?Q?WmYpmPWFeKT+pGb7TuW0Be5v/Vx2ShacHBIZuaUE7R1pocAlX1dThGCaHKUI?= =?us-ascii?Q?Xw65HZfNpNEIkL+QDAggE0sEq5q+2H/gwMkCq1cQuSyGvhTYJRHJaNvTDT2u?= =?us-ascii?Q?MBKos/A5tEWmsPrHBO/S3seFeKLK1X77C5hc4g/R1+mDtpjXTYAGXrBI3vOP?= =?us-ascii?Q?9htZem39QBxb/fitFf53wXHhej/XWeshVubiH+QEbmhIFV0sZCXQF0UT6UL/?= =?us-ascii?Q?R1Kupqbpgw+Dv7qVS9uYMLEtZPYQ7cdCrXwCRxfxA5Vd0cRbNB9pCBcq/XYV?= =?us-ascii?Q?0Oo1CrkF5dG5jA2sgDUcFbWXLkpcGNhmLcoXsIToLgTscoq7AGY/nCYRsIpd?= =?us-ascii?Q?Y3AqS6qgQ+gVaUr7MZPSrISCf5hu/xbrcebRNKLZ34HWnfhg5pxGSVPdgs07?= =?us-ascii?Q?RcatpRlUtlRwU1hwfQbzmT09Qqagqt69shymuTci+uBwGXff1/VtZiw+b226?= =?us-ascii?Q?eEUST8l9Ya+Jvl9g750O5VddNbezG6wGfg6NlB/E+xC8ITed1nlIGoC8uNLL?= =?us-ascii?Q?Yq9sdVWN+H2ihMwFETLmqQRj7M69FgPyVQ3q4H8iFXLdvGmkEG4MoaGx/MEF?= =?us-ascii?Q?RB3/Da9Yz7nvEOKVzlJ5+f0C3gR9wQ+FD5q5P9k05DMuyjl9gZ4WPzpflLJt?= =?us-ascii?Q?JAcH558IcnpHQp8KFeFzZAIoHDxXtKDuwojj9M6c7qtOA+exwc4pGIEQIfvP?= =?us-ascii?Q?8idtytIqRU4WkAaunHRZkuMruKC0n9rgj3VaeGA42uMq28z6e8RoO1n9XHd3?= =?us-ascii?Q?W5/sjpvpiy+kxfCb152+P9n0GtzymCGow+8sBDve?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4034be1a-242d-407f-8b62-08ddb41ed17b X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:31:01.5761 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6G9Yhvx1Wgb+KmYxtKRFLQgrJ2VjBPYPzhkISHr9jn4WHCIWbrcY86eEqGDllAwNQKboHEDleKGQuwcptg6OBQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR02MB8535 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX7st+P78Vil2x +S8cncpbJ1wnVtBLYMP71Sh5FghInWc83NGgVByVeaEcx9Prb6V2zwS7iC6p8Ndr87XU/euRkfi MXofbUdSbQQpT5ajPxSGQHN75Figgk0DPcDC4mygr31akyivkF5NpeHIc+9NMTDfg8hjU2qjpIl Kvh8jCcdGs9fFCHgePUhAoHFZccqWx4+3ID1eYhlUx+qa6bBbQDlej+w2WMkB+OT/iZdVAnaxJV EZO3gqW4a0Eykky6doq1Xig1EvIOgXLW0WpycbH3zult4bztUm/vbO1OQE+fsLyZLv2SRav7qOd 4fCbdwDgyNnj2S475G5/FfVZm0ru4+Oq2V38eMewbArtbvwtnbDvDWmdzaXFBt11xf3bP8cr9kz vjVbMNnzGeaw9714iAmvYBo8EAv0KvXYXEPrh2JPFnDzxiRnJs7frMeo8ZAwznw6aGiOnBed X-Authority-Analysis: v=2.4 cv=WMl/XmsR c=1 sm=1 tr=0 ts=685c4e77 cx=c_pps a=pKTBGjKhDztqafJ6BpA6Kw==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=64Cc0HZtAAAA:8 a=5Eg6v1c09lA0JFFDL-8A:9 X-Proofpoint-ORIG-GUID: UmhDkEglbGqBcy-BzXe4A7e0q5i13_4i X-Proofpoint-GUID: UmhDkEglbGqBcy-BzXe4A7e0q5i13_4i X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750879954955116600 Content-Type: text/plain; charset="utf-8" Support an asynchronous send of a vfio-user socket message (no wait for a reply) when the write is posted. This is only safe when no regions are mappable by the VM. Add an option to explicitly disable this as well. Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/proxy.h | 6 ++++++ hw/vfio-user/device.c | 45 +++++++++++++++++++++++++++++++++++++++---- hw/vfio-user/pci.c | 6 ++++++ hw/vfio-user/proxy.c | 12 ++++++++++-- 4 files changed, 63 insertions(+), 6 deletions(-) diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 6b29cd7cd3..0418f58bf1 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -91,6 +91,7 @@ typedef struct VFIOUserProxy { /* VFIOProxy flags */ #define VFIO_PROXY_CLIENT 0x1 #define VFIO_PROXY_FORCE_QUEUED 0x4 +#define VFIO_PROXY_NO_POST 0x8 =20 typedef struct VFIODevice VFIODevice; =20 @@ -104,6 +105,8 @@ bool vfio_user_validate_version(VFIOUserProxy *proxy, E= rror **errp); VFIOUserFDs *vfio_user_getfds(int numfds); void vfio_user_putfds(VFIOUserMsg *msg); =20 +void vfio_user_disable_posted_writes(VFIOUserProxy *proxy); + void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, uint32_t size, uint32_t flags); void vfio_user_wait_reqs(VFIOUserProxy *proxy); @@ -111,6 +114,9 @@ bool vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUser= Hdr *hdr, VFIOUserFDs *fds, int rsize, Error **errp); bool vfio_user_send_nowait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, VFIOUserFDs *fds, int rsize, Error **errp); +bool vfio_user_send_async(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, Error **errp); + void vfio_user_send_reply(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int size= ); void vfio_user_send_error(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int erro= r); =20 diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index 3a118e7361..aa07eac330 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -110,10 +110,21 @@ static int vfio_user_get_region_info(VFIOUserProxy *p= roxy, trace_vfio_user_get_region_info(msgp->index, msgp->flags, msgp->size); =20 memcpy(info, &msgp->argsz, info->argsz); + + /* + * If at least one region is directly mapped into the VM, then we can = no + * longer rely on the sequential nature of vfio-user request handling = to + * ensure that posted writes are completed before a subsequent read. I= n this + * case, disable posted write support. This is a per-device property, = not + * per-region. + */ + if (info->flags & VFIO_REGION_INFO_FLAG_MMAP) { + vfio_user_disable_posted_writes(proxy); + } + return 0; } =20 - static int vfio_user_device_io_get_region_info(VFIODevice *vbasedev, struct vfio_region_info *in= fo, int *fd) @@ -312,33 +323,58 @@ static int vfio_user_device_io_region_read(VFIODevice= *vbasedev, uint8_t index, return msgp->count; } =20 +/* + * If this is a posted write, and VFIO_PROXY_NO_POST is not set, then we a= re OK + * to send the write to the socket without waiting for the server's reply: + * a subsequent read (of any region) will not pass the posted write, as all + * messages are handled sequentially. + */ static int vfio_user_device_io_region_write(VFIODevice *vbasedev, uint8_t = index, off_t off, unsigned count, void *data, bool post) { - g_autofree VFIOUserRegionRW *msgp =3D NULL; + VFIOUserRegionRW *msgp =3D NULL; VFIOUserProxy *proxy =3D vbasedev->proxy; int size =3D sizeof(*msgp) + count; Error *local_err =3D NULL; + int flags =3D 0; int ret; =20 if (count > proxy->max_xfer_size) { return -EINVAL; } =20 + if (proxy->flags & VFIO_PROXY_NO_POST) { + post =3D false; + } + + if (post) { + flags |=3D VFIO_USER_NO_REPLY; + } + msgp =3D g_malloc0(size); - vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_WRITE, size, 0); + vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_WRITE, size, flags); msgp->offset =3D off; msgp->region =3D index; msgp->count =3D count; memcpy(&msgp->data, data, count); trace_vfio_user_region_rw(msgp->region, msgp->offset, msgp->count); =20 - /* Ignore post: all writes are synchronous/non-posted. */ + /* async send will free msg after it's sent */ + if (post) { + if (!vfio_user_send_async(proxy, &msgp->hdr, NULL, &local_err)) { + error_prepend(&local_err, "%s: ", __func__); + error_report_err(local_err); + return -EFAULT; + } + + return count; + } =20 if (!vfio_user_send_wait(proxy, &msgp->hdr, NULL, 0, &local_err)) { error_prepend(&local_err, "%s: ", __func__); error_report_err(local_err); + g_free(msgp); return -EFAULT; } =20 @@ -348,6 +384,7 @@ static int vfio_user_device_io_region_write(VFIODevice = *vbasedev, uint8_t index, ret =3D count; } =20 + g_free(msgp); return ret; } =20 diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index f260bea490..be71c77729 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -24,6 +24,7 @@ struct VFIOUserPCIDevice { SocketAddress *socket; bool send_queued; /* all sends are queued */ uint32_t wait_time; /* timeout for message replies */ + bool no_post; /* all region writes are sync */ }; =20 /* @@ -268,6 +269,10 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Err= or **errp) proxy->flags |=3D VFIO_PROXY_FORCE_QUEUED; } =20 + if (udev->no_post) { + proxy->flags |=3D VFIO_PROXY_NO_POST; + } + /* user specified or 5 sec default */ proxy->wait_time =3D udev->wait_time; =20 @@ -403,6 +408,7 @@ static const Property vfio_user_pci_dev_properties[] = =3D { sub_device_id, PCI_ANY_ID), DEFINE_PROP_BOOL("x-send-queued", VFIOUserPCIDevice, send_queued, fals= e), DEFINE_PROP_UINT32("x-msg-timeout", VFIOUserPCIDevice, wait_time, 5000= ), + DEFINE_PROP_BOOL("x-no-posted-writes", VFIOUserPCIDevice, no_post, fal= se), }; =20 static void vfio_user_pci_set_socket(Object *obj, Visitor *v, const char *= name, diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index c3724ba212..7ce8573abb 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -714,8 +714,8 @@ bool vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUser= Hdr *hdr, * In either case, ownership of @hdr and @fds is taken, and the caller must * *not* free them itself. */ -static bool vfio_user_send_async(VFIOUserProxy *proxy, VFIOUserHdr *hdr, - VFIOUserFDs *fds, Error **errp) +bool vfio_user_send_async(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, Error **errp) { VFIOUserMsg *msg; =20 @@ -844,6 +844,14 @@ void vfio_user_putfds(VFIOUserMsg *msg) msg->fds =3D NULL; } =20 +void +vfio_user_disable_posted_writes(VFIOUserProxy *proxy) +{ + WITH_QEMU_LOCK_GUARD(&proxy->lock) { + proxy->flags |=3D VFIO_PROXY_NO_POST; + } +} + static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); =20 --=20 2.43.0 From nobody Sat Nov 15 14:11:14 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750879905; cv=pass; d=zohomail.com; s=zohoarc; b=aP4ERHidHHDNtHqXmrjgnxfJeiIEn9QM48gaQGi2sQGwibOkGulBGVMr9IF0e9GEQFzH0dKT+JpxVspxviOZ4gNne+fknpByx77CJQJDQuz6dwNenz432M6hzDOmCKCoPqdm2967Owmrb9YL6NXYuSHYwqLw2JVpvvlYa370eu0= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750879905; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Z/e9VLMgCEHUaKTkvcLXsmzB6Ct2x8k1UMhh8itltNE=; b=XwT6JoKuMxgybZJKJsWgkSlh20YUJ2FZ7H4uBUsJH0+ao0+MAkrgEf0ZrVsg28A5pOqhd2zMnjJRqfHot0cIZAuW4B4ZxmyA6q94F2I+B/rL/WbZJ8hmYzihP+kxfUaaXS77RulOE9DsqYnefNz1lKbOSZTkN7Srb/zAr8S+EFM= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750879905682477.6674487970415; Wed, 25 Jun 2025 12:31:45 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVq1-0008K6-Aw; Wed, 25 Jun 2025 15:31:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpx-0008CU-02 for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:13 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpu-0004OS-Gp for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:12 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PErU52026909; Wed, 25 Jun 2025 12:31:08 -0700 Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10on2095.outbound.protection.outlook.com [40.107.93.95]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 47dva0hhuw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:31:07 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by SA0PR02MB7308.namprd02.prod.outlook.com (2603:10b6:806:d8::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.29; Wed, 25 Jun 2025 19:31:04 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:31:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=Z/e9VLMgCEHUaKTkvcLXsmzB6Ct2x8k1UMhh8itlt NE=; b=owV8bixdPYgYHlrsf7WapR0bSrbQQ9oC+D1mPDW3lcuCRk7RqCTuzMdZA ybZovQeRIIweNFsrXI0PwAygs7GD6OhZ0S50h2bE7bAHXrR9UW9ahRaYmCw+e2uA NMsQNGMCUgmpt9G5mopsGsE+TrjtBWYnAgkYFaDCJdoiTgY91PLFxqNaYYEvbUMt fzeju7Itiyky7qe/qSrLXLf3w/v5lREvRygYcZSyYrp0kTXXT2DmTwgavE3k9ddp EYGK3jeXFEH+RFK2WJz+vasgkWJI1+rjl3Y40ovEnVV/r78Pah0a0XUefVesed/B +wUPw1zrxgKI7okM9XURxYP4/0a7Q== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=juV77C3zR5cawgtN6thvD1BxgnYZV0shLM69GI+/FLaPJTDHq7vrBiOumf4P5qCn9B+S0zZRJfR5XXDGU/w9RHQW5VOitDNVGwJgyTlLozhztQnMlXGSOPGsRM7jYfSCRuVN0KPoSgTWizzho7K4ivm/ULHQJu1U+pDFmAnigTA2l6tUy6ZhGuwoGu1U7xQ/OGsx3HXBKqH7N/HZDSSTK48B66nrR7zzfZXGmVTgnIw4ZCVsjJ5E7dtC8lhmpudbdCoIwZMPgdq2Ko7ma6DygpMB7vvGWTL0L+FUhBuVJ4//SYxJMp92oAFadwFV374aCVyQxZfMTXxZlYvAdlc++w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Z/e9VLMgCEHUaKTkvcLXsmzB6Ct2x8k1UMhh8itltNE=; b=TfVnMPsk2JZJPn44CAdVNYnph+Rcbp+tc+CJSkuMvD8PLJ1C/8exJ6tNEXJEMF3SbUBLsIH9e/HwONGj66il3KsIpmVrmpv+34de1g8oTwHaOyzfeos9w+Vio9Pa7Todzmx8FU1BAwJybR36WEeLPVopy9JTf8M0rBgpjo7b1/PcUMIH0Jh+4O4CD1pNMx6Yg1pRLdiwOml+8//2hSyXaV7vX4MMyIxmXbUkSVgE4UipcsnZfDbGwtQhqfps59axrLZBMvguRRjpUzmuuqnd/KSfMkp9CS7rq4P6BDbNgoXjre12Vihng7rH4RvEUelsf6NgSDrUwbGU4xySzrFbww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z/e9VLMgCEHUaKTkvcLXsmzB6Ct2x8k1UMhh8itltNE=; b=StJaiVQjetrgqH1r5wDVqe6G3R625YVfVIdfLveEFw7St/LxW5GXkxbKtj2Y/L9yhuFEjYR4ddBHVYi5n7JiDCZ2DBe4kptVtPWMKVBh9VWN4r2+ppnJgjsZ1x99NdKYzExAC4r+iBXuzXmyT9DfnE/iYz8jzt2+drsy//6h+mCMfRb+qrDKaI5uOB8nraYqXHW3aB3DhGcRxDfXDbHooBDLwrkwCaDR9CHHilWk+d/ZmzwJbLpVWmESVDIfrPFQMnohsdqeIWqN0MU+E+x3GbtAtquYVUDdQ8eHA1uoos9g22KX70b8SPA8fjlfXUFE9prTbYfGQ7+dLUqp/FT0jw== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Johnson , Elena Ufimtseva , Jagannathan Raman Subject: [PATCH v5 17/19] vfio-user: add coalesced posted writes Date: Wed, 25 Jun 2025 20:30:09 +0100 Message-ID: <20250625193012.2316242-18-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|SA0PR02MB7308:EE_ X-MS-Office365-Filtering-Correlation-Id: 880e45e5-7bd9-486b-9414-08ddb41ed2e9 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?+GQiUI9um+TIXEOK1sVHGa83dFsrySlxLIN1v6joMn5cf9JS2W60i4zaryay?= =?us-ascii?Q?dH7E7d70IKKRv9asmP+HUxGy5wKmngQyRm5Voyqgnld3nmV152g1mvMny6d+?= =?us-ascii?Q?vVvDmYoa1kkxLFDuq7pgVWO+Hav5qVVA+CB9m8KJcswZiMAdQb9MMn/NMGFx?= =?us-ascii?Q?EREiReyUqvO91+csQFxNdBFfx1+HKedeBaLOJLiIAU1DFW7gNZoC0RVoU0BT?= =?us-ascii?Q?PzzvjD6jjoO+kRZDfaxmMow8ValMjcXW9MLrx/oLpiMsa/3m6tB5f0Tn9mtv?= =?us-ascii?Q?Uuf6qnTk9Vq5C0BIuWWQWlB4sbJ6vjc1OqHkZhptFMdbdOnEB98XfZ5nmcmz?= =?us-ascii?Q?lZFv54ypW295uhgpK3DEPRbAayzYJjzgxaKoeiz8VxeGP86ZqDdiEApn9Mbx?= =?us-ascii?Q?uoux/7g7FX6wocGTyABioD45SyXg7mjQD9+ikr46+UmWjVunA2Og1OWwrrE8?= =?us-ascii?Q?MASPLWMm9Jx775fK6oXA0T6wkFhjuj93ogjp1J3yAey1h0PyhKjfHcTGoHnI?= =?us-ascii?Q?QtGU4God4dV93NeDDPMsUswjkkk23A5aAoRDEWX4S9LHXywGqFKWvWfKZ0rJ?= =?us-ascii?Q?ERarBdHQ+fLStPAPuPQwRuiXGPkBFHv6UC51zERtEhK6lunrp/VMSJdG0uW8?= =?us-ascii?Q?wucrXG6r7SGpquG7U1Asiiouzkyr8PsEu7a1CwT+4SNq4x4YGmzCEZNSMpvV?= =?us-ascii?Q?+nBR627wQBif9qtTsGzMh97uE2pGvjFZzLzP0JsB5/Ad7MYtHik5cvGnAR9e?= =?us-ascii?Q?byNhN4gdpxP/CbKfmzNSZncOPEKnSLj4iZl5ly7buFg0A2gZOAcOVsOlDKYR?= =?us-ascii?Q?URdNodYre4D+d8VDAa0BQu2ctxK7vVvKXkowzYITKzc2RKZG2mbujFCDH2ef?= =?us-ascii?Q?aPVS5pvN9XGI0WoE11tmAEqvw/OGZJRM43uXyV3KIMf+6thjueXy0BJ0Nmke?= =?us-ascii?Q?SFPKe4VX/n66++p5wuQhXuAWlAWvcJ4rjtRvXhfBG9EMGU/q7aHM6jWoDbSJ?= =?us-ascii?Q?h48GBLlSVHUnInG/pZsTqNll7NbpnjJN8Dsu6qVlGQEUMIleMIEha25PwUhM?= =?us-ascii?Q?/uy/0LD9OflHilxQjuwgHZX0zTSuYdAgyDulblFqopcgXDTZIPrI4M+K5VHV?= =?us-ascii?Q?CQSPWkm39u4a2no1p1NozPhxeEMro6H6l9olhibh2qpTSQAsVuuNqjEDb0bx?= =?us-ascii?Q?LRcqr7fzke3mnnRuTclcjE/HmYAdsVtZnKsKENH4wQr6O1boMqHpy22iiXGb?= =?us-ascii?Q?sbm9ACyCDkrp1VorbNpXCaPAItLJlyJO24YKeJa7HCMyZn+AhneEho47eiFQ?= =?us-ascii?Q?wnNvAiwGNv4fFLMgJeTutGhBivoLifRIY75Brvm2bTcAzBf8I1VwH3ZfZ90c?= =?us-ascii?Q?035M1yba8QOBIdXb426Lv5xUC8A0t9xVnzz+j3gJ/IkKzXgv3A=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(376014)(7416014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ywdl14d4JhzpGDrWzwKSBCNLdfoty3AuegNHlHRKGq2pQZOeiVExTfW0S5MZ?= =?us-ascii?Q?29EnViExtJ5mDcjsqo1pFGp79BnwxtEA+XBzFbJhKHmu4gpYOZ3wWhrbQMyz?= =?us-ascii?Q?TVWD8E7nTi08YFheotirMKdqZ2H+pv0rpaCGmH6Wp6hZD6wuV3AR/IjMhIFS?= =?us-ascii?Q?3HKftLwG/mt6aUyDUXYvWXRbI1jfRIZD7aUhc9/u/+4sx/lCFBajzh3nIsBq?= =?us-ascii?Q?Dmp1R+7SgaAR6VwKHFv96Rb4GzNVq9bCRmEyz29LCMobLVRp//Dgef27/6Vd?= =?us-ascii?Q?TUG4gvrxj6b030puyGFl7MnR/Se3k1tlfeRNG8XvgWGfLDSraxhqnxffLSiw?= =?us-ascii?Q?OETnN311+t6BpA98XHLaX4YYEbkcT0niWLvPZxU3gDF1h9PuSiaOKrHjs3I4?= =?us-ascii?Q?wwqfyCYdz5g+9tLVgTi78YYUaob4nohjJZ54lAxCpsaCyQ3CRucDDJIRibh/?= =?us-ascii?Q?Ad0If5nkgfz4eb2tLZ86Q3Ta4NPGGlnhDEY468EqRs6fuNLV+e7Uqf+A2F7m?= =?us-ascii?Q?MAymZyatkN4z3jKwaI/rF/7lXDZrvepOqWSJ8irLpaihTT5kPlzauACbS2bp?= =?us-ascii?Q?dEGc9vtJiyQIoM4I6Z/+XLRIrcHHX8aTtY28XmwLk7yLl1A5ZX8VusueGN92?= =?us-ascii?Q?K+15ycKyDMzloXY704U6fLNdAAOXOauOmRy207VBFL8eAZf1YOaxTDduFJ/S?= =?us-ascii?Q?c1EVZyOqtwx2/i/lJyfmTF09ImBebiIeGQLRFew4biA8bygw+tshdRKfZNVp?= =?us-ascii?Q?W9bIV1j+JZ1c+p+/iLTD/oaSVOKNkxZDaOaQL6809fW5kKh6b2wI5S04Dqus?= =?us-ascii?Q?tGymTb4bHoB0tWYdLr7yx0b3g4GWjaIOnH4dy1elRzv9LaxUR0Z1Y/E2TwEA?= =?us-ascii?Q?9Z2mGViIBgN0cgG+WOI+EpYDsPqZo4KbhC+oQLo8Lsdp8ie5KOx/uqczrqtY?= =?us-ascii?Q?ZtHYHHJWyQR7e5WHNUDTUIozTYl3BsmbvedngQxNiF2BeFpHKOFSnXgjnTow?= =?us-ascii?Q?J8gvG/oFTlOPfeNMUMaeIhKheGWYFAfnmoRQOYFziMIigOGiz7uvrNUSeYts?= =?us-ascii?Q?XGJpe2B7vQ+YHMEWjd7sa9wj94EKek+aXgYv2r4KB1m1FJ2tDsv0XQGI2BCs?= =?us-ascii?Q?nKKGeKDXRgISKsoWSU0e/ZlvdJPy25sRiOEg1QutGokUWZWqHsz2bq09FXiu?= =?us-ascii?Q?UvYEedM/sp97FOwuThztMd1s2ZYcht5mPPdiCQrtTahZAnKtEGLagMx+rya0?= =?us-ascii?Q?ghfhGOOfsp/Gk/ILvWluoZRzIaaOYgIBBBPCB/FV2gKo7fJyc5R7v0dax+U/?= =?us-ascii?Q?9cZil3lf/FrZ5EMqP/TmNE27Yq0D8QV3v/qvaLHVTd6WTrxpVeC6ZMpkod4J?= =?us-ascii?Q?cdzBl9BIVY7V2sQsJHEmqQdk/fg5NSvPtlIcv3FVqrvOAdzwOeefCWQQhqwE?= =?us-ascii?Q?bvJ8I+G6nBv6f96Jg6rShTNSnH9O9WeGjitix97jVVnNLqZfHvefkS6uCg/E?= =?us-ascii?Q?VhWSeC6QaJUmucKPVnd6TCe79gSSHacvGH6bXmtEwXAS2lRfBexrkzAZVjME?= =?us-ascii?Q?V3NQbk9rDWmNl8LYOov6aUkVOyo7TU7vg4FTsHnk?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 880e45e5-7bd9-486b-9414-08ddb41ed2e9 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:31:04.0592 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jv1uLSr3+q8gxEkpWQU75e6eAzWYhZJ1q4Yv5+PQh6sIGziAP0M0eRDdvtcRfzgqaAbvsDciPbNaYYZMJmAWgw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR02MB7308 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX4CMPZ6uUbRg1 GidxXy5gthbkQKHABRQRkYW/KCLfHCRAfUyaN++jIuyAquFGpD3oViDBynOmuxfkn7vd+LZcbCU ktDHaV2/SCuUPHmFzZuJp2sGqoluKvpjkkd686kMiNTuP7OEgNRZ5Md2HgLs3cjDEbx/y5WLnpC ED7evedRJ+EayCIjcp/op59ThClHb7YlwC7acBuUaerm7QlMMZX6V0IoEj2sMZGto6OKkPZlI53 ju8sXIZHMtbFjNecCFrJ+5ILWJppESafhvfLQT2xuEYpaeFMS3epDJjlk0QT7eu/BDeOmlgBmHO YCBgf/I4t9XVqfoa7XlwMgA2UyEiTMB1CnAjr4zM9I3FL6FVahho+B7vGDZddIBJj85hDdtqAYA y0I3k6NheJokuqwwO6CzX53yE386uviAf39aCjrv6QEUfhKzYWWNNvRnNR7IsGghlcFhuuWQ X-Proofpoint-GUID: 72P9zzuRx2ByM6wR-GDyjDEnwtRU8voJ X-Authority-Analysis: v=2.4 cv=Qohe3Uyd c=1 sm=1 tr=0 ts=685c4e7b cx=c_pps a=DIZmGg09oD0WH6Bj2PzylQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=-qxVb9sXb9au51uT7-gA:9 X-Proofpoint-ORIG-GUID: 72P9zzuRx2ByM6wR-GDyjDEnwtRU8voJ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750879906805116600 Content-Type: text/plain; charset="utf-8" Add new message to send multiple writes to server in a single message. Prevents the outgoing queue from overflowing when a long latency operation is followed by a series of posted writes. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio-user/protocol.h | 21 ++++++++++ hw/vfio-user/proxy.h | 12 ++++++ hw/vfio-user/device.c | 40 +++++++++++++++++++ hw/vfio-user/proxy.c | 84 +++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 1 + 5 files changed, 158 insertions(+) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 3e9d8e576b..3249a4a6b6 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -39,6 +39,7 @@ enum vfio_user_command { VFIO_USER_DMA_WRITE =3D 12, VFIO_USER_DEVICE_RESET =3D 13, VFIO_USER_DIRTY_PAGES =3D 14, + VFIO_USER_REGION_WRITE_MULTI =3D 15, VFIO_USER_MAX, }; =20 @@ -72,6 +73,7 @@ typedef struct { #define VFIO_USER_CAP_PGSIZES "pgsizes" #define VFIO_USER_CAP_MAP_MAX "max_dma_maps" #define VFIO_USER_CAP_MIGR "migration" +#define VFIO_USER_CAP_MULTI "write_multiple" =20 /* "migration" members */ #define VFIO_USER_CAP_PGSIZE "pgsize" @@ -218,4 +220,23 @@ typedef struct { char data[]; } VFIOUserBitmap; =20 +/* + * VFIO_USER_REGION_WRITE_MULTI + */ +#define VFIO_USER_MULTI_DATA 8 +#define VFIO_USER_MULTI_MAX 200 + +typedef struct { + uint64_t offset; + uint32_t region; + uint32_t count; + char data[VFIO_USER_MULTI_DATA]; +} VFIOUserWROne; + +typedef struct { + VFIOUserHdr hdr; + uint64_t wr_cnt; + VFIOUserWROne wrs[VFIO_USER_MULTI_MAX]; +} VFIOUserWRMulti; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 0418f58bf1..61e64a0020 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -85,6 +85,8 @@ typedef struct VFIOUserProxy { VFIOUserMsg *last_nowait; VFIOUserMsg *part_recv; size_t recv_left; + VFIOUserWRMulti *wr_multi; + int num_outgoing; enum proxy_state state; } VFIOUserProxy; =20 @@ -92,6 +94,11 @@ typedef struct VFIOUserProxy { #define VFIO_PROXY_CLIENT 0x1 #define VFIO_PROXY_FORCE_QUEUED 0x4 #define VFIO_PROXY_NO_POST 0x8 +#define VFIO_PROXY_USE_MULTI 0x16 + +/* coalescing high and low water marks for VFIOProxy num_outgoing */ +#define VFIO_USER_OUT_HIGH 1024 +#define VFIO_USER_OUT_LOW 128 =20 typedef struct VFIODevice VFIODevice; =20 @@ -120,4 +127,9 @@ bool vfio_user_send_async(VFIOUserProxy *proxy, VFIOUse= rHdr *hdr, void vfio_user_send_reply(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int size= ); void vfio_user_send_error(VFIOUserProxy *proxy, VFIOUserHdr *hdr, int erro= r); =20 +void vfio_user_flush_multi(VFIOUserProxy *proxy); +void vfio_user_create_multi(VFIOUserProxy *proxy); +void vfio_user_add_multi(VFIOUserProxy *proxy, uint8_t index, + off_t offset, uint32_t count, void *data); + #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/device.c b/hw/vfio-user/device.c index aa07eac330..0609a7dc25 100644 --- a/hw/vfio-user/device.c +++ b/hw/vfio-user/device.c @@ -9,6 +9,8 @@ #include "qemu/osdep.h" #include "qapi/error.h" #include "qemu/error-report.h" +#include "qemu/lockable.h" +#include "qemu/thread.h" =20 #include "hw/vfio-user/device.h" #include "hw/vfio-user/trace.h" @@ -337,6 +339,7 @@ static int vfio_user_device_io_region_write(VFIODevice = *vbasedev, uint8_t index, VFIOUserProxy *proxy =3D vbasedev->proxy; int size =3D sizeof(*msgp) + count; Error *local_err =3D NULL; + bool can_multi; int flags =3D 0; int ret; =20 @@ -352,6 +355,43 @@ static int vfio_user_device_io_region_write(VFIODevice= *vbasedev, uint8_t index, flags |=3D VFIO_USER_NO_REPLY; } =20 + /* write eligible to be in a WRITE_MULTI msg ? */ + can_multi =3D (proxy->flags & VFIO_PROXY_USE_MULTI) && post && + count <=3D VFIO_USER_MULTI_DATA; + + /* + * This should be a rare case, so first check without the lock, + * if we're wrong, vfio_send_queued() will flush any posted writes + * we missed here + */ + if (proxy->wr_multi !=3D NULL || + (proxy->num_outgoing > VFIO_USER_OUT_HIGH && can_multi)) { + + /* + * re-check with lock + * + * if already building a WRITE_MULTI msg, + * add this one if possible else flush pending before + * sending the current one + * + * else if outgoing queue is over the highwater, + * start a new WRITE_MULTI message + */ + WITH_QEMU_LOCK_GUARD(&proxy->lock) { + if (proxy->wr_multi !=3D NULL) { + if (can_multi) { + vfio_user_add_multi(proxy, index, off, count, data); + return count; + } + vfio_user_flush_multi(proxy); + } else if (proxy->num_outgoing > VFIO_USER_OUT_HIGH && can_mul= ti) { + vfio_user_create_multi(proxy); + vfio_user_add_multi(proxy, index, off, count, data); + return count; + } + } + } + msgp =3D g_malloc0(size); vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_WRITE, size, flags); msgp->offset =3D off; diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 7ce8573abb..c418954440 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -13,12 +13,14 @@ #include "hw/vfio-user/proxy.h" #include "hw/vfio-user/trace.h" #include "qapi/error.h" +#include "qobject/qbool.h" #include "qobject/qdict.h" #include "qobject/qjson.h" #include "qobject/qnum.h" #include "qemu/error-report.h" #include "qemu/lockable.h" #include "qemu/main-loop.h" +#include "qemu/thread.h" #include "system/iothread.h" =20 static IOThread *vfio_user_iothread; @@ -445,6 +447,7 @@ static ssize_t vfio_user_send_one(VFIOUserProxy *proxy,= Error **errp) } =20 QTAILQ_REMOVE(&proxy->outgoing, msg, next); + proxy->num_outgoing--; if (msg->type =3D=3D VFIO_MSG_ASYNC) { vfio_user_recycle(proxy, msg); } else { @@ -481,6 +484,11 @@ static void vfio_user_send(void *opaque) } qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, vfio_user_recv, NULL, NULL, proxy); + + /* queue empty - send any pending multi write msgs */ + if (proxy->wr_multi !=3D NULL) { + vfio_user_flush_multi(proxy); + } } } =20 @@ -579,11 +587,18 @@ static bool vfio_user_send_queued(VFIOUserProxy *prox= y, VFIOUserMsg *msg, { int ret; =20 + /* older coalesced writes go first */ + if (proxy->wr_multi !=3D NULL && + ((msg->hdr->flags & VFIO_USER_TYPE) =3D=3D VFIO_USER_REQUEST)) { + vfio_user_flush_multi(proxy); + } + /* * Unsent outgoing msgs - add to tail */ if (!QTAILQ_EMPTY(&proxy->outgoing)) { QTAILQ_INSERT_TAIL(&proxy->outgoing, msg, next); + proxy->num_outgoing++; return true; } =20 @@ -598,6 +613,7 @@ static bool vfio_user_send_queued(VFIOUserProxy *proxy,= VFIOUserMsg *msg, =20 if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { QTAILQ_INSERT_HEAD(&proxy->outgoing, msg, next); + proxy->num_outgoing =3D 1; qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, vfio_user_recv, proxy->ctx, vfio_user_send, proxy); @@ -1151,12 +1167,27 @@ static bool check_migr(VFIOUserProxy *proxy, QObjec= t *qobj, Error **errp) return caps_parse(proxy, qdict, caps_migr, errp); } =20 +static bool check_multi(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QBool *qb =3D qobject_to(QBool, qobj); + + if (qb =3D=3D NULL) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MULTI); + return false; + } + if (qbool_get_bool(qb)) { + proxy->flags |=3D VFIO_PROXY_USE_MULTI; + } + return true; +} + static struct cap_entry caps_cap[] =3D { { VFIO_USER_CAP_MAX_FDS, check_max_fds }, { VFIO_USER_CAP_MAX_XFER, check_max_xfer }, { VFIO_USER_CAP_PGSIZES, check_pgsizes }, { VFIO_USER_CAP_MAP_MAX, check_max_dma }, { VFIO_USER_CAP_MIGR, check_migr }, + { VFIO_USER_CAP_MULTI, check_multi }, { NULL } }; =20 @@ -1215,6 +1246,7 @@ static GString *caps_json(void) qdict_put_int(capdict, VFIO_USER_CAP_MAX_XFER, VFIO_USER_DEF_MAX_XFER); qdict_put_int(capdict, VFIO_USER_CAP_PGSIZES, VFIO_USER_DEF_PGSIZE); qdict_put_int(capdict, VFIO_USER_CAP_MAP_MAX, VFIO_USER_DEF_MAP_MAX); + qdict_put_bool(capdict, VFIO_USER_CAP_MULTI, true); =20 qdict_put_obj(dict, VFIO_USER_CAP, QOBJECT(capdict)); =20 @@ -1270,3 +1302,55 @@ bool vfio_user_validate_version(VFIOUserProxy *proxy= , Error **errp) trace_vfio_user_version(msgp->major, msgp->minor, msgp->capabilities); return true; } + +void vfio_user_flush_multi(VFIOUserProxy *proxy) +{ + VFIOUserMsg *msg; + VFIOUserWRMulti *wm =3D proxy->wr_multi; + Error *local_err =3D NULL; + + proxy->wr_multi =3D NULL; + + /* adjust size for actual # of writes */ + wm->hdr.size -=3D (VFIO_USER_MULTI_MAX - wm->wr_cnt) * sizeof(VFIOUser= WROne); + + msg =3D vfio_user_getmsg(proxy, &wm->hdr, NULL); + msg->id =3D wm->hdr.id; + msg->rsize =3D 0; + msg->type =3D VFIO_MSG_ASYNC; + trace_vfio_user_wrmulti("flush", wm->wr_cnt); + + if (!vfio_user_send_queued(proxy, msg, &local_err)) { + error_report_err(local_err); + vfio_user_recycle(proxy, msg); + } +} + +void vfio_user_create_multi(VFIOUserProxy *proxy) +{ + VFIOUserWRMulti *wm; + + wm =3D g_malloc0(sizeof(*wm)); + vfio_user_request_msg(&wm->hdr, VFIO_USER_REGION_WRITE_MULTI, + sizeof(*wm), VFIO_USER_NO_REPLY); + proxy->wr_multi =3D wm; +} + +void vfio_user_add_multi(VFIOUserProxy *proxy, uint8_t index, + off_t offset, uint32_t count, void *data) +{ + VFIOUserWRMulti *wm =3D proxy->wr_multi; + VFIOUserWROne *w1 =3D &wm->wrs[wm->wr_cnt]; + + w1->offset =3D offset; + w1->region =3D index; + w1->count =3D count; + memcpy(&w1->data, data, count); + + wm->wr_cnt++; + trace_vfio_user_wrmulti("add", wm->wr_cnt); + if (wm->wr_cnt =3D=3D VFIO_USER_MULTI_MAX || + proxy->num_outgoing < VFIO_USER_OUT_LOW) { + vfio_user_flush_multi(proxy); + } +} diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 44dde020b3..abb67f4c11 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -13,6 +13,7 @@ vfio_user_get_region_info(uint32_t index, uint32_t flags,= uint64_t size) " index vfio_user_region_rw(uint32_t region, uint64_t off, uint32_t count) " regio= n %d offset 0x%"PRIx64" count %d" vfio_user_get_irq_info(uint32_t index, uint32_t flags, uint32_t count) " i= ndex %d flags 0x%x count %d" vfio_user_set_irqs(uint32_t index, uint32_t start, uint32_t count, uint32_= t flags) " index %d start %d count %d flags 0x%x" +vfio_user_wrmulti(const char *s, uint64_t wr_cnt) " %s count 0x%"PRIx64 =20 # container.c vfio_user_dma_map(uint64_t iova, uint64_t size, uint64_t off, uint32_t fla= gs, bool async_ops) " iova 0x%"PRIx64" size 0x%"PRIx64" off 0x%"PRIx64" fla= gs 0x%x async_ops %d" --=20 2.43.0 From nobody Sat Nov 15 14:11:14 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750879929; cv=pass; d=zohomail.com; s=zohoarc; b=oAaFSUd9kxFcYtxx/u7bJJThHjPR8gOuw47F1mUhPEWUNhGrC3c7z1KAR5usSm5T+D/+vtSX9YUPYLZQtzIVAeOP0I//32PsA8D+0VBoXlJMYAhOCeIEbtKnkHurMRV21OTRGRCyl4CjDUnGMdHfXcqlNTCYtgk9XR2rQ3V8ul4= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750879929; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Ts/w8AhN1+z0KUmhSHPsWe7gANtjE18DYXmyv7iOTto=; b=nOZy9gYJQWFJBUpFixR3xFZ4XnaWxjROBK/k0tyAh7i+oHvCKKJx511FeHdF2m8lNJeSmZ6hSR6WH0gJPXEOUZAsHMg3P20GlKm0elureLwzfcEUaKUh2d9sDcdjAopo8Y7PGbqB7CGpF20kjNyqgGJPxZZDWMx2ca09SqyS22k= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750879929313995.6560599804637; Wed, 25 Jun 2025 12:32:09 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVq5-0008U8-H7; Wed, 25 Jun 2025 15:31:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpx-0008Ce-3l for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:13 -0400 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpv-0004QK-5b for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:12 -0400 Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PErU54026909; Wed, 25 Jun 2025 12:31:09 -0700 Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10on2095.outbound.protection.outlook.com [40.107.93.95]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 47dva0hhuw-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:31:08 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by SA0PR02MB7308.namprd02.prod.outlook.com (2603:10b6:806:d8::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.29; Wed, 25 Jun 2025 19:31:06 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:31:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=Ts/w8AhN1+z0KUmhSHPsWe7gANtjE18DYXmyv7iOT to=; b=YFrjEW7MFL+rjUIQWcqvgzuZl0i0yM7XtkfSwlFkT4mW+lHrjnLJW16x1 ncSFde4DAbU3wNMOUEya74pHtyjOszdgx/RgmR3RKeRVVwT1UgrUKScK9l0PFWTj NKrKrma6H0kdB4aSojNjraNoAjNOivA5ucmvOb1Xpemi+FEvPXQSG/Np+Z35G7dc t1WvXYZjYqivjwkbk6qtVnFJNu9Brzevj5lcKT2RFLMRsyEBHuQLeBHwUE6JgOXS 3Lga7Li7cI5t4r2UaJnOoSkhVGJVOqMPq9uvy9Jo+SwPQcYJSaV/Vk9A6oxYk4+8 auv7DR8w7uBF586eMS6ZWHIxmir2A== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tsxo/lemeu7jtrQtk2m1Blw/6nKUQDubgupcO2wwMBcK/QiHwukDlGrbFw5KmVkW83Uy/kaSztzR8yCebfdy/vY7lsdYFvrnjWFF2jUG5a+XzE4YpG6dQKavYzWDjD5B7XftX+nAO52QDDxwljq9vHZ8BfMWlBGpl+Gr7xcHQBpfSreF05IPz9nhtOzl/xVxz2TzvX8WGSTMKuZs3SEJgvnv97kvNf/JOPRiN6z0GwTQaAdaYzJy4U0wTIujKNdz0GdRMCFn/GKLv8L1wozw1I/MpwO6gUsQhOzT5mYkU6xRfVCMnzA2tLT+WFcr2EFe8c8JvDmgC7ZtKzHAXmcqdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ts/w8AhN1+z0KUmhSHPsWe7gANtjE18DYXmyv7iOTto=; b=m1DU+09lIglmPhpfSDrSpTI+oC3Ew0j5rfbksVJ0gCzatfNQGP6EPnv6IfQBhA2mo1Y7nuuagk2bby+0W9hXmC8ElJ6S0Oj4SbYeXamptUbmHkVTou0rMvzRWnRdRzEZGF94/A/Y/aE0aq0TDxG1bnx346M/mNQ4spzvRfJW3W0Gss+XZidBVEUefqo+8VJiDi88nl/+2pFIrLeCtarhCT8IpCzLMqfgdZL8m1Gl94204fD7JhRSZMksK2EyHkAySE+8VpPIDeNVKIyrDTk8Ag3sc/KwDlhKvaHsdZeWmuq+bz+N9xGyOdtrf+jMjl4d/sKL5hY9DbJQ9jnlsDqepQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ts/w8AhN1+z0KUmhSHPsWe7gANtjE18DYXmyv7iOTto=; b=PIuz2H0qWZqFPius5qVMeFygnVAlyna2LWhuuSapc8NUhrhA1QBEfti3IEqn89lLk94zZqclKYF1yxF+a/049Y+1p1pLffuwCK1UUGovi2qeo/TFIrJ6YccKsCnulyv4SszyT5MYmi4+nNG84Wfdh8IQxsajfgfxtZSlZCHsJjSoGq5o+AJjNV6twPQcBodR/MNSB+MLa/vfALA32CqRdrF7cCsRl5UJK5pmsVd9nvUHMKjnt9jZQAAbWJZWV9W0dcSMeAg/Wsh7mHrR/a55+32J2VWnUjdJN6e0hQxxfDe9Ry23OrkfnJ3b5+FXChuqw8bKc1tWtAXbNulnf0l8aw== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PATCH v5 18/19] docs: add vfio-user documentation Date: Wed, 25 Jun 2025 20:30:10 +0100 Message-ID: <20250625193012.2316242-19-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|SA0PR02MB7308:EE_ X-MS-Office365-Filtering-Correlation-Id: 84df9379-0182-4074-d6fc-08ddb41ed46b x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Fc4jNK8Sn5BZ2N4WUicb8bVgfeJAQKsOXfnOQhk1mh2XiF/5ZJvNV0R8pQsh?= =?us-ascii?Q?2ACjqGh43vxFhzhPioqsVY7uRLzQnxv5CEDiFjnzfcDy24oTQ2n7kAcgUD3+?= =?us-ascii?Q?cdeYgEH4FT+tk+Zh010rFDslPmUFEwM44iwoiNGM9Eoz0adrvI2rLwMxltzq?= =?us-ascii?Q?nS+SFcpTaqdc/PH9zEYYo/9OpdrNp6T1ccptBB7e8jJEK7Ti2hw7Gdj6WY+w?= =?us-ascii?Q?LnZhvwf0AtA+XUPBmTj5KhXW/O0/DUlSnjZg6kQs5pj/48Xd0gwgcnwBHltP?= =?us-ascii?Q?eKxyLcScgtcaYlDOYvJ/LZWRgZkXVEoOZ7kqj/3mtwRLLAMAkEa90WNqJKXl?= =?us-ascii?Q?M/qtRru0UKCYo8f4nrmJJNYynDpG9QbzNKchGbc3nKFK5TLtfVh6UkmTu8V4?= =?us-ascii?Q?fjFGvbmi8G/FbdJjiSV0dUjoGPlAp64F50pLVrF5bvMM6vBg/MNWiao1jcdw?= =?us-ascii?Q?78l96qNznNw+HKA35i4XYSGjXr9493QZwwxx4ejp9gKFJeGDZrKBsNASP3f3?= =?us-ascii?Q?ILfMMG9nwRVYhRUbWMEApD/Fu34OdR2bwHmbiU22CU1m+bJL3x/qqW6DCwOk?= =?us-ascii?Q?d3E1/M26/hs6vh4TRlHaavruHtxYdXYBk+4k+zwIKDBJAv6JdZ3d8X/M0FQq?= =?us-ascii?Q?MBZ7QQZzJzLKsWUDRTquMgtxPoO3/+Nf+KLX44Zhbv7taQcX5aMmYylFoke2?= =?us-ascii?Q?OOlDFFd2iAO8JPp1ZriadFr9JQrB9UTiHPJXTH+xYbdX0mBF/DrA5ESCYW8+?= =?us-ascii?Q?ntjhRWaBiH2b17jrAbkPMqOskAySxB3KpTJjTWgylwllkl0bQPEDMk6fcZt1?= =?us-ascii?Q?pc3jwOSzDRrq1pyFblP3UaFmsEFB4KLqf+e8aenqpeUwCwtb9D4dOVUij0JZ?= =?us-ascii?Q?njJQ4TyvIxduOX4y4jsgGJdTk6Z7JG25TgHrtD9rkPzEiM0luswOa2UxehKC?= =?us-ascii?Q?eb95l0O4E3ScChLozL23UK9d19IyhBTSgU+/8OsZ30FrUtN2n6bHJ0hKtbwa?= =?us-ascii?Q?PskH2JsBTjfdCjRWyK4DAYhyfpSgLJg4YB70H0xgZngOdv9APM1Hl0QSzOUJ?= =?us-ascii?Q?3bSB1CTSpkCnY990kApojMskE61uspWCNP0RiZmUlW5iLUpC7FyJj4alOgid?= =?us-ascii?Q?0OMmN/6IuxzHLu8Y2n0NE4A19JGvhX02Fo9uwpRkFwXiqOvul/Kikf2zUxcv?= =?us-ascii?Q?XqCYLEvOJ0a7KnJqU+CbF3ITE6szghDYwtta9SiEs+S1K8C+JIKgQC68ncK8?= =?us-ascii?Q?7uRLxp3F7obgMUKdfAW4wtQpKWnVN00Ecbk1XxSghMYWoS795tLU9tHqc286?= =?us-ascii?Q?YmQK0qaqU5sadOMyWNTXIw6HHvv6LPF+RdsA4KlAGOBHvjj1bSPYvpWi3NCG?= =?us-ascii?Q?B9cwHx+Bdvffb+qOK6D/bCIIUSFPikwGnFQuL9hwZxVeA08kgZuLA3MfypQr?= =?us-ascii?Q?fK3XZ3YRH/8=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?uoIJR2mye5zPftQGUugRLutdfG/lIEAsnMXTL/2enmxkqEAF8mGRaJ/QikfU?= =?us-ascii?Q?sqayeRRVcyjieKifdfjvArqmbqaHkSJ9P5DjKb957tDQ97au3YqR97zJBX4m?= =?us-ascii?Q?IICNi5rHfTNAgqV+HSO5t0jsXPyMNl4vFRyo6cL4g/91xX+hDpmr0VurF+Q/?= =?us-ascii?Q?PnkWnpZmkwJqufRpd+6q3ji8Ti5TQrrf8x2/qv7L/umH8E1WSK6tWSHaCdyH?= =?us-ascii?Q?yqsIzIaipNQx5gfA/b2T2i6agQnGqRRtBc5rL3YBs0hWty2VkJuH+Vz3YRyT?= =?us-ascii?Q?kjkk/ooyWgO1IlFroXCse01RDQdZRGuR+2mj1eR2sMdbY+6wEx3aPbhGvUa5?= =?us-ascii?Q?PXIlCOyraIcFj7HD0UBhVNHjYuZWHZZsh0NeDccAvgx1GskUru0ZbNsbKFfD?= =?us-ascii?Q?rusj170JTXwSxXwSX/JMTgapPRF6PDMnnKh3Qf4BS39CKukHPayhSCPn6CsY?= =?us-ascii?Q?36GCDlzKjzMA6Nh4TjblgqywIY4cHn6I/ECodzIDaxN7R0EOB9CB3dsn9TIy?= =?us-ascii?Q?FI7MCe4cGqX3MM1nlQ6m9hhOml60LkT7OS56T/r8IdRwnKMSHww7PLw5Nzct?= =?us-ascii?Q?UZc/2a2ogw8sJ6D19aJeiO8flGFin1ZHeb+6yz0hmvujpTthj6VAhp35TFc1?= =?us-ascii?Q?QuQdvN9h/D9h6Buq39+WTw+zkJjz9iDJJiiWk46QsO+WDvuJtHbRmHFZGXui?= =?us-ascii?Q?tjO4lbkDP2QOk8BxUOtLY0EQjFYfporZ7kWDwf0tKR4NbQjr7VsyEiABiASy?= =?us-ascii?Q?gm8P8k/EOvcAaDYEc+46lqds7duVcJTaVYIHh2/ZNv1dfm2hqV+AXJgBh5Wl?= =?us-ascii?Q?pFToIY6NBNEmx4WQIA+o3RoDw1ucSnR1HC7l5eTjsF2BZfbzaf2GFn7WdXtd?= =?us-ascii?Q?0WruO9TmWYgRwe20yjMeYCVN2If4p/ghKCmtXuY/d/jjVeQq6PsCbDr2+Vj5?= =?us-ascii?Q?WAYtPciUALA8XC0wJ+ImM+qyCK3gkzDlddat2h1gJe70fjOQGk43wUq0eC2B?= =?us-ascii?Q?i1zBG9GsinUfIGmZKb5Ff05PXhQGh9sXwCF1lMSSb9RUiGDC5oMkArinY6ZQ?= =?us-ascii?Q?MO0pXu6hd8RQwtySR4eEFWQncgxNp/ZB0X37Vl2S0iv0TPSpOYJiBW306BRG?= =?us-ascii?Q?K1LvYTNCehP6hj/va59IV9e1cumnvNj9D+9USDPm19Px6bA5r/YFKZb5Nb0s?= =?us-ascii?Q?+nyBMMsXZvvrOpgom/xtmXWyA+Te9erQ/ntE9voEuBJSDxbf4jU5tYWTQC4F?= =?us-ascii?Q?tM710dENwAxYAQPQlrCppRgx0nLRlu4XJx7zqKYu4wdfU27Se7bNyqzGc7nH?= =?us-ascii?Q?qZjoGw/YDmwcIuOc9Ax4lQl2dfKRBTC/TTlu+ipLXTDHuL3t9SbaNCg1YUid?= =?us-ascii?Q?47G2OZ9L2KN20c+ddbXql96/HP9Mft2lb5b60MMyIIePUzXp10a5e4V5LGwK?= =?us-ascii?Q?byx8H06k7glZnGtY5xkWD9OJvblKc9Up8H5c9gfu7Fs0LSPVfsaas1lp+C5R?= =?us-ascii?Q?4y1eEZge3u59se18Dv0UmLyUknRR0fdsOfwQsEyVkTrf54MMs+/8kdpg1Izu?= =?us-ascii?Q?9e5wHvHUk3FOpTiXCbyRDUTaquFnEU/CG1+nI5dJ?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 84df9379-0182-4074-d6fc-08ddb41ed46b X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:31:06.5505 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ccn+yJwDdG5hyOmFwzZR40TylJRqmlmicf0kCGPTzE42UC7J2Ff1pEfZBPkKS0Vp83jKtLTcPXiRV5CtkJQCjA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR02MB7308 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX2Wr9XsncJm3P 1kU16ZdxH511GHVTE2kJqVQ6RyGDcndjpVYViC5UqQQQhOgOvrF7BZYWDBWnOEeaybtOLDlCoxH TZu77s5itIthd96f70esJWnMUamTm3RzEYoc7NmD1E37aPxrOuUb/AfrwRzxWSRobU2FSOXtAZt rOS0LSH6RhrH2WAyla7Wjyl4I3d0CFNAcvR/xsXwQ0wkDJvNn0ClYD3LvkzjGJLQBAjLfS6FV/P aWqEu7OUDghClO9khs+SJkUEV2GE/LqD9Cm64s59qYrCGW3qp0+3qJS5AoEdNVvKMaExUZ9OEEl X7wlMwLoUuy0v3qDoL2dbknaNtW9vJxlkRT6w0l9ZRzSYCXKIjvv5ie/KRmIPdiV+wbXftZiqT/ MnFF4NLGtOHpO26ExRBtvKYz4j3Sf0ViaCck33RoVeYDfNmdc5qfozdA3v04f2+jIWIIZVOD X-Proofpoint-GUID: zZF_yTVd8t81n0Dc9a0jWtt7EUR-8PpE X-Authority-Analysis: v=2.4 cv=Qohe3Uyd c=1 sm=1 tr=0 ts=685c4e7d cx=c_pps a=DIZmGg09oD0WH6Bj2PzylQ==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=NEAV23lmAAAA:8 a=64Cc0HZtAAAA:8 a=LOSiUy74ef7l4vre4JwA:9 X-Proofpoint-ORIG-GUID: zZF_yTVd8t81n0Dc9a0jWtt7EUR-8PpE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750879930574116600 Content-Type: text/plain; charset="utf-8" Add some basic documentation on vfio-user usage. Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- docs/system/device-emulation.rst | 1 + docs/system/devices/vfio-user.rst | 26 ++++++++++++++++++++++++++ 2 files changed, 27 insertions(+) create mode 100644 docs/system/devices/vfio-user.rst diff --git a/docs/system/device-emulation.rst b/docs/system/device-emulatio= n.rst index a1b0d7997e..911381643f 100644 --- a/docs/system/device-emulation.rst +++ b/docs/system/device-emulation.rst @@ -85,6 +85,7 @@ Emulated Devices devices/can.rst devices/ccid.rst devices/cxl.rst + devices/vfio-user.rst devices/ivshmem.rst devices/ivshmem-flat.rst devices/keyboard.rst diff --git a/docs/system/devices/vfio-user.rst b/docs/system/devices/vfio-u= ser.rst new file mode 100644 index 0000000000..b6dcaa5615 --- /dev/null +++ b/docs/system/devices/vfio-user.rst @@ -0,0 +1,26 @@ +.. SPDX-License-Identifier: GPL-2.0-or-later + +=3D=3D=3D=3D=3D=3D=3D=3D=3D +vfio-user +=3D=3D=3D=3D=3D=3D=3D=3D=3D + +QEMU includes a ``vfio-user`` client. The ``vfio-user`` specification allo= ws for +implementing (PCI) devices in userspace outside of QEMU; it is similar to +``vhost-user`` in this respect (see :doc:`vhost-user`), but can emulate ar= bitrary +PCI devices, not just ``virtio``. Whereas ``vfio`` is handled by the host +kernel, ``vfio-user``, while similar in implementation, is handled entirel= y in +userspace. + +For example, SPDK includes a virtual PCI NVMe controller implementation; by +setting up a ``vfio-user`` UNIX socket between QEMU and SPDK, a VM can sen= d NVMe +I/O to the SPDK process. + +Presuming a suitable ``vfio-user`` server has opened a socket at +``/tmp/vfio-user.sock``, a device can be configured with for example: + +.. code-block:: console + +-device '{"driver": "vfio-user-pci","socket": {"path": "/tmp/vfio-user.soc= k", "type": "unix"}}' + +See `libvfio-user `_ for further +information. --=20 2.43.0 From nobody Sat Nov 15 14:11:14 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass(p=none dis=none) header.from=nutanix.com ARC-Seal: i=2; a=rsa-sha256; t=1750879977; cv=pass; d=zohomail.com; s=zohoarc; b=gNkhYU7aUGG9XmuwIWsg+KQtCP8cmIWsHXTuHeVo7vVvQrjS2fKVEvc/lZM1qCtRStY3CxWXewoB3P/mAyPev4rqoIbopedLIKvYbdI+Aqevw1uABrOWiJ/58G7Wi/Qu4coYJqUL4O82OZ2lRfPo3UgHCeGnbWFOFefcLRziBAk= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1750879977; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=r6KvPjY4FhMxhd37xxCaFM69wdD44/bHpUwCkeEAGC8=; b=SdKopEWxav/1FGT5AQ96ggQMy5hGA784ou2r5JVBAGxIW7kX39rf03Bd1A8kyZEjJIusCw78NLvNgcv5xQd/i+VKNgTpY541OBUevLkaMz9CULA0dMaX3YB8JSNhCiq6Et6ygFo5lytzO5RUjfbARt9D6NSiFhmuzsUaOOAqIzQ= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nutanix.com); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1750879977525979.8253697629982; Wed, 25 Jun 2025 12:32:57 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uUVq7-0008Vo-1Q; Wed, 25 Jun 2025 15:31:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVq3-0008PG-Bf for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:20 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uUVpy-0004Rg-E8 for qemu-devel@nongnu.org; Wed, 25 Jun 2025 15:31:19 -0400 Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 55PDssmf025796; Wed, 25 Jun 2025 12:31:13 -0700 Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10on2092.outbound.protection.outlook.com [40.107.94.92]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 47gjdtrsbn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jun 2025 12:31:12 -0700 (PDT) Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by SA0PR02MB7308.namprd02.prod.outlook.com (2603:10b6:806:d8::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.29; Wed, 25 Jun 2025 19:31:09 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%2]) with mapi id 15.20.8880.015; Wed, 25 Jun 2025 19:31:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=r6KvPjY4FhMxhd37xxCaFM69wdD44/bHpUwCkeEAG C8=; b=VzKYZXtGh/HA+zRPv5P7Tpa21k9t0HuX2Ec1VTgNYL2SPVQ2NJZfm9KtC bbA5hUezNWp/NoGqj/XxNBIufctE7suZbEqOTouqMPdUy/Iu5VXcjtd+Aa9H2Tsj hpZxcpYcwcD/chqYGyKZW48idRSSEZ4cbLebCYTnn0YiQ1LtT0LMF28kZFkGuPrU 6H4a0dklqeqOjXagmGK82/Xafc66c+tDb2+0c5n8BO4TALf/mzS5pnBY4KCbm//r WHkt1p8prqqSYzHkXhfc47qmVE1TyxlnpdBkqGN2lTgAjthnhJaNmXpFsXfbJeNF xuzwllEC7cemDxKi2qFkbGbQQujwQ== ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kodnDqU3M4iZ3yeyiSoD/Zh/to7PfFFx6Mafe2xxtk43IoPKG2zPCXq9PI7CcdjJ3gDJSzh6dSCni6sJ5nZMi7IgLm6PHyAi2JsO1hK1WCKBqtemtMryRQuv//o3ykUzsTuhZrE0eaG4KsGJwYNxCOkKCR6PKfpMySztDeZdt4QdXwM/AskL3QJCV87OwBQG8tf2MgPsivnoXbPCBPXNezbvShctyDjO2pgQHvr4fRQGOJOkB1vSQi1ZAADpeV/SSod6jO2ztPkgcpLChHEZuL8xtXlc2bnWkhcLLOpF9strPWj/ubwSWojviPvqdzKJ50n3lm3FhO7iKZkUUlCENw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=r6KvPjY4FhMxhd37xxCaFM69wdD44/bHpUwCkeEAGC8=; b=NiNgir4PN1ntbGrehrFkDqBUdEZj6wPcU+9QtKddGVUPwhk/lzq32PZ7N9BAx62nk/joRuyWnYBv7yGPiS4KbYY3227wIxdr0FYWx7k/WXdRZQr9Sbd/cgsp2uzuv/NOYl428D3Ofx7YaJmoJSQHCxpBDIL4zpIMp+0SoZFBMXPb+G8WjTnlmDLBU4s9xYNP6lY7EEBv4HPh8cPrIr9UtjoTY+PxigQEWyx/8IDmFNwFxOglHK4nXKlgeh+WNxR4K5mJZkzMBYClPrOs3YV0c/sp+E3VCOxXrpFT9PiQyLsLuEjWTC1vJVRuBrIhJmdyAZYxC46KFLJskYk9APeLpQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=r6KvPjY4FhMxhd37xxCaFM69wdD44/bHpUwCkeEAGC8=; b=PggG9acqR0f4aOJysBrloeZD7/IrerFA6pLwUK3+zgkVoKUsASkku1AWvI5cledy4sn44/1ws1nOPvnbsHE0nItusGF+tD4T+rUOAp/SYLehovIl0nw64bOMTwDruwTJ1qBGBt4PhSy4UpeJBVc8aRMqnJnepoRSVl1dQPHyaKAX6LD1mBFq9rvUd2hbsSsQ9t8i4BqojTFo/NborUG6FPYDdVXkCLMY+O3tgA1tPqDrYRZ8uJ/uP1edvRM8HPpeaVeotqPgBjSwAP1IXTjbFsOWZnLDYto8Ml3gpQjdlNgYjylDURs3zElvRZYIWztgIb3/ToxcAgzJEP8weHospw== From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , John Levon , Thanos Makatos , Paolo Bonzini , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PATCH v5 19/19] vfio-user: introduce vfio-user protocol specification Date: Wed, 25 Jun 2025 20:30:11 +0100 Message-ID: <20250625193012.2316242-20-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250625193012.2316242-1-john.levon@nutanix.com> References: <20250625193012.2316242-1-john.levon@nutanix.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AS4P190CA0019.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d0::18) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|SA0PR02MB7308:EE_ X-MS-Office365-Filtering-Correlation-Id: f0f3f741-3fa0-41b3-a3fc-08ddb41ed5c6 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?tiT3qSggw/7jwSk6yTqDHazhxF9Ibv8Ptdp8ZIOPz5Vklv7nPkIevqt4fo21?= =?us-ascii?Q?rGmPaTC6OpY0grHM7QxecNJ76cB0lZtqQh7hRpoRtcIMZA9JMrD/L++QFhF5?= =?us-ascii?Q?PYjQ/QGSbOjFx2EOm6QI3a1KyeatUt8E30Ufvf4o+j9xuS9aQIXswjaFef+I?= =?us-ascii?Q?7dRNcG6Gcwwh9pJeJ5DlnJWhTlkUfC7c/1Np/7UZ9EGMHsbfAovVhfZjT2FJ?= =?us-ascii?Q?l75x9SsOe1U+nyluJbLh9nivzi0k0o4TkTAeO3tUaxDtVjuOqWUbbpsIUDld?= =?us-ascii?Q?HDrby0l+wMuYBQxOQwvCI/76qO+80LPMXh1Ck9UGiPfHp+1FlYUT6Kkg5ZQB?= =?us-ascii?Q?Kl1QUDCkUD3+NbOvq8/zq9ensrcmmL9f+qMNqPwgmkjo2EIDB5lGztXEbTvP?= =?us-ascii?Q?PKJmYffE2aTvejZa+/8m2kjrY2hxqMCp9zRTFF3Qix6kHUnx3u5d3gIYy5md?= =?us-ascii?Q?r45CyfSvdh5dSpaAJVuTyAmvUHuSKRc9g1B3mQM1mJz3+/5C/U2kljTbIIxR?= =?us-ascii?Q?XD8TkE2s2oxVzjHN1kwwHLOR25QMg3ytbDvKaRkX+WrKXPBqnM+Lk49DfyFh?= =?us-ascii?Q?An2oYosiETWenQ9jY9ZK1ADHmNocVmw2bQ4cglTVE+7lE1E0R/P1ZeG/qkXq?= =?us-ascii?Q?whuNf6EUcCUiOtO/SGNBe9IvYWGeJ5ORhNfNbWIC72Rs9r9GsKLZO8vwQQ8u?= =?us-ascii?Q?O/tp0JfLyqr1uADRK4m2sNX3HnbuUr25CCLonZi5iVg+UjgoqIu9FtvbhcSX?= =?us-ascii?Q?jU8uQiACibsKAfJc+TTk//q0503bLTiI/CfDdYYMHc+E3OdyPHK/rSiA4iY0?= =?us-ascii?Q?x13R7CA1EfFOXGjiZ8hypIObvbU08SNv+sK4NxrO8BSYS9MpA2+cP0uM7VUg?= =?us-ascii?Q?Gs5eskHlruHu+tFZoZ8oql1tf/aT4xX6AU0Wv/bhKwiZjMz2JaEYjSTmPHX9?= =?us-ascii?Q?8RCOjl2l29svYYGlRGqjYZ/quE0X6S125pXTOzMkB0TUrVdedmqksd7KaZal?= =?us-ascii?Q?35MT3NFVu036G50teAxOgNyMD/e1b2k63zKB+NBihvY6F2NKdK+IO91PIcBj?= =?us-ascii?Q?TgsQhelaQup+jBVJMc5KcOpo8JYkqLvn8/6URk/oxRKF1Iy6oukaV9aGC2LM?= =?us-ascii?Q?Zq1XrsxCXZCayljOIPwrW4pdGcNXDin6GSht9NrFD67gxpDbmFdtlsOM2j98?= =?us-ascii?Q?yZzM4qDnVNnJS8k9UNdCzEZGPRImb60lQw0MrVbJ7T5gnvlkhx7dVYBTmzFk?= =?us-ascii?Q?WiqUkZTT5JXO1mdbCGueiP7+v1J9Hl0SSlFU+ht2e6lA64FmekLwlZwdZFCZ?= =?us-ascii?Q?4pCpPTrCqBi/eXC2p/NuUWw+dnXjCBBzYw5AKkc5NseB48CBM0mWq8MRltRy?= =?us-ascii?Q?nwgFqYcJfDASelmd2GvTfaNYprmMy5fl1kCD4j712UJ2RKaOpVnN+3u9zQEM?= =?us-ascii?Q?th8AB3s2oEQ=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?/WveFWF6EFRbsakB7eZa8ozHvgeAB0a0w/EkQqLedMrP6QKjA75NOdcNapC9?= =?us-ascii?Q?w4Tz/igqhq6KBNdz7hHirYYOQsd9trpU/gdRYF3M1Z4dzlNPO6SEYB+OuPws?= =?us-ascii?Q?ImIkHfHV+2tptsn2/Ae1rQaUGA7HE9kdigR8zSe4BDUdfTWHU9IBj63TKAEt?= =?us-ascii?Q?VOevhaTdN3Fc4vTXAIkigUFwKS6xniZSPRQApOeK3DoRaVI2NUPAzTeoFCes?= =?us-ascii?Q?ICx53KB6iYjvTP7atS0EiWRP/VUtO7j+RWhwBkrAnWHhT8GsBSb2oEEpKocd?= =?us-ascii?Q?7bfx53MHusRIYDheFIowIkV67O1o91+7MOXu3Fk3YiJQ22kX3W8KTcFSdaBP?= =?us-ascii?Q?KLrt+FvZ6ciFK4GMbQS5aqTgq/C3vqKQq8snYWyeHzCyuEr6XlKuZ5ZgeDkR?= =?us-ascii?Q?v7zxmb+GoM709dB7Dh+LZqCfW1PVWzyFgZmZnyAQC8p5XuZzVFrnWikyQMGT?= =?us-ascii?Q?Xa3uU5x/N2/1GF8liQMnvzRH7fSWQorKmkNuxxwmP7Q3DpncNshRtf47OL5E?= =?us-ascii?Q?q3356tR4phE5wxhzTa4rxARLOCK9q3sCF18+LD5GfJKY3jrHv699tCoIReWG?= =?us-ascii?Q?HqLOWrGP9dAvAojH3CmMq3lgjRJYhMe2XDgRtF3b5XbKzIkcUDG5ijV9Uesv?= =?us-ascii?Q?noIf9/JnfiHgC1255UfaeGlRyss2TxXcf6N6YGbBBKNs357G6hanHZTeYjg5?= =?us-ascii?Q?pSN41oTjCMb5/jCdep2XqW2VuOR0Ia31ksMFD8sIcNXfXTBnpkQBKXTsNG8W?= =?us-ascii?Q?QyAy4IBnIjJeJq/VRNMBpityuSpqPrLVk8PgULCNmiZixecJmtN2ypjHDGWn?= =?us-ascii?Q?BZiOgL37MImuKYXGERJxlKx2TeskxWytBg3hPOh+km9ZWzapTFjoZ+yvEPfD?= =?us-ascii?Q?vIQWWNacT0fI11Km+lD+/xtLGSEGWLLt1sllu6LuheN33lvqxuF7ES9JOy+D?= =?us-ascii?Q?k/B9LdOrHsKK91vB6+vKwe2TUEkeCrXl0vXb6MBFbl8EsG0y9CxBuvd4EJd/?= =?us-ascii?Q?wezuX3niyIfE0XDH6E9rKjzWLojM5RR7fba9ZXIG9O0cBUVxgzmoGpF25eKS?= =?us-ascii?Q?TNTJWf/Tp9Xhan3Vop1EJp4JkNhAv5c45eiRvTWu8lUGsxPo9/a3NQqr+56b?= =?us-ascii?Q?RG+Rm31gvCW2ylmaRCg8SfOxjXtGSVigZVoTrvHePPVZzVMf3r492YUXPiIB?= =?us-ascii?Q?m+l2fx0TEWScbfXgPR8AUu/xncsVRGyl5zRWdE6K2piA1/t3zLJZkQ9Cxeyt?= =?us-ascii?Q?AYS56SfU89eVeakJxucr/0ce+I4E1Tk42oQKL8ekk5WUIukFmyUEHTvZMl6K?= =?us-ascii?Q?XkT7E4/KETeukyRj5HitGPk6O0XvWfDRSaEXtGN89JIAHDMvzttimo3DO2RB?= =?us-ascii?Q?i09yXVh5GMrRw896vkEkzqFoNlecy9K+f1KFkAtQzg/qlAAH7HmHCgQaZ/Ac?= =?us-ascii?Q?8tBn9gimBjdIIvh3U2EJfA4sP0ZgJgID392GLsiXy4LmIqijkb+BlTb2gish?= =?us-ascii?Q?HpSlHyr4HYieDtfl0bHUtnL+ahXKQ6eQX1m6ffPw5q/qohFYLf87bcWoUMQ8?= =?us-ascii?Q?smFNGgztu6ZRdjWTfLTF9zoOD/RU3ohLOSKcjXo/?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: f0f3f741-3fa0-41b3-a3fc-08ddb41ed5c6 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2025 19:31:08.9740 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: T+gX0WhBF6EU2twv2nPakg26rqfmELd5wgbsZPUgA+5fVRdJtJ7CG2FB0GPgM/bsXCmTmLjr7NddE/FwWLLvXQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR02MB7308 X-Proofpoint-ORIG-GUID: TflVVVGRXq4L9pOKck2mk1yV0cxUJpn7 X-Authority-Analysis: v=2.4 cv=Fcs3xI+6 c=1 sm=1 tr=0 ts=685c4e80 cx=c_pps a=odStxYl4fA30BBTO3gVnUA==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=6IFa9wvqVegA:10 a=0kUYKlekyDsA:10 a=VwQbUJbxAAAA:8 a=pGLkceISAAAA:8 a=64Cc0HZtAAAA:8 a=h9SPtozqHe0Wh_k42joA:9 a=x5O-bmgOEKQMiMGR:21 a=N0yriQgkpQ4A:10 a=HplO-upLQ7EA:10 X-Proofpoint-GUID: TflVVVGRXq4L9pOKck2mk1yV0cxUJpn7 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNjI1MDE0NyBTYWx0ZWRfX93Qi3KSQGvTA 5CLT6cDc9zxgETYSMQmoNVR3NHr5gsGG67YuiWznM/vxRQiISMe3XkITj++ZS3MXef7ZHio+RHF wNiF4DvSWdXSgtPmWOSt8lxQc4nYipFitWP7/2auOrI7gdcVpEKpzTnWnH5iRyMVOexegXeoAdL oDbKj7RAbijBUbBwispV16ENMR+2dVefStDfFv42xg8TSMzjMRfDiIgWHESr+b47BYX9d2PObEw d4+/gGmCVMEUs1K1Xh30sN4cIzejzEcO8wlLo/djvXxs9s4YOEafAP5Kg1cZ14mzm/0uXs/6+Qg uQwm9MbOLpdWXDRVqtTLhuJ4pfw0zRAqcAGVAhb+nXZ8DFPE97p4eH7RNgF7oMIGSqnA+09UlsH /ATKb/Agnr/UPA4125fsM0R7QuHPg1Oc5LklkKwqM9W4syLJE51fsKYvD/cBJpjxIyl4tLPZ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-06-25_06,2025-06-25_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, WEIRD_QUOTING=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @nutanix.com) X-ZM-MESSAGEID: 1750879979433116600 Content-Type: text/plain; charset="utf-8" From: Thanos Makatos This patch introduces the vfio-user protocol specification (formerly known as VFIO-over-socket), which is designed to allow devices to be emulated outside QEMU, in a separate process. vfio-user reuses the existing VFIO defines, structs and concepts. It has been earlier discussed as an RFC in: "RFC: use VFIO over a UNIX domain socket to implement device offloading" Signed-off-by: Thanos Makatos Signed-off-by: John Levon Reviewed-by: C=C3=A9dric Le Goater --- MAINTAINERS | 3 +- docs/interop/index.rst | 1 + docs/interop/vfio-user.rst | 1520 ++++++++++++++++++++++++++++++++++++ 3 files changed, 1523 insertions(+), 1 deletion(-) create mode 100644 docs/interop/vfio-user.rst diff --git a/MAINTAINERS b/MAINTAINERS index 2369391004..1b73b8b394 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -4246,7 +4246,6 @@ F: hw/remote/proxy-memory-listener.c F: include/hw/remote/proxy-memory-listener.h F: hw/remote/iohub.c F: include/hw/remote/iohub.h -F: subprojects/libvfio-user F: hw/remote/vfio-user-obj.c F: include/hw/remote/vfio-user-obj.h F: hw/remote/iommu.c @@ -4257,6 +4256,8 @@ VFIO-USER: M: John Levon M: Thanos Makatos S: Supported +F: docs/interop/vfio-user.rst +F: docs/system/devices/vfio-user.rst F: hw/vfio-user/* F: include/hw/vfio-user/* F: subprojects/libvfio-user diff --git a/docs/interop/index.rst b/docs/interop/index.rst index 972f3e49ce..d830c5c410 100644 --- a/docs/interop/index.rst +++ b/docs/interop/index.rst @@ -25,6 +25,7 @@ are useful for making QEMU interoperate with other softwa= re. qemu-ga-ref qemu-qmp-ref qemu-storage-daemon-qmp-ref + vfio-user vhost-user vhost-user-gpu vhost-vdpa diff --git a/docs/interop/vfio-user.rst b/docs/interop/vfio-user.rst new file mode 100644 index 0000000000..0b06f026b0 --- /dev/null +++ b/docs/interop/vfio-user.rst @@ -0,0 +1,1520 @@ +.. include:: +.. SPDX-License-Identifier: GPL-2.0-or-later + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D +vfio-user Protocol Specification +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D + +.. contents:: Table of Contents + +Introduction +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +vfio-user is a protocol that allows a device to be emulated in a separate +process outside of a Virtual Machine Monitor (VMM). vfio-user devices cons= ist +of a generic VFIO device type, living inside the VMM, which we call the cl= ient, +and the core device implementation, living outside the VMM, which we call = the +server. + +The vfio-user specification is partly based on the +`Linux VFIO ioctl interface `_. + +VFIO is a mature and stable API, backed by an extensively used framework. = The +existing VFIO client implementation in QEMU (``qemu/hw/vfio/``) can be lar= gely +re-used, though there is nothing in this specification that requires that +particular implementation. None of the VFIO kernel modules are required for +supporting the protocol, on either the client or server side. Some source +definitions in VFIO are re-used for vfio-user. + +The main idea is to allow a virtual device to function in a separate proce= ss in +the same host over a UNIX domain socket. A UNIX domain socket (``AF_UNIX``= ) is +chosen because file descriptors can be trivially sent over it, which in tu= rn +allows: + +* Sharing of client memory for DMA with the server. +* Sharing of server memory with the client for fast MMIO. +* Efficient sharing of eventfd's for triggering interrupts. + +Other socket types could be used which allow the server to run in a separa= te +guest in the same host (``AF_VSOCK``) or remotely (``AF_INET``). Theoretic= ally +the underlying transport does not necessarily have to be a socket, however= we do +not examine such alternatives. In this protocol version we focus on using = a UNIX +domain socket and introduce basic support for the other two types of socke= ts +without considering performance implications. + +While passing of file descriptors is desirable for performance reasons, su= pport +is not necessary for either the client or the server in order to implement= the +protocol. There is always an in-band, message-passing fall back mechanism. + +Overview +=3D=3D=3D=3D=3D=3D=3D=3D + +VFIO is a framework that allows a physical device to be securely passed th= rough +to a user space process; the device-specific kernel driver does not drive = the +device at all. Typically, the user space process is a VMM and the device = is +passed through to it in order to achieve high performance. VFIO provides a= n API +and the required functionality in the kernel. QEMU has adopted VFIO to all= ow a +guest to directly access physical devices, instead of emulating them in +software. + +vfio-user reuses the core VFIO concepts defined in its API, but implements= them +as messages to be sent over a socket. It does not change the kernel-based = VFIO +in any way, in fact none of the VFIO kernel modules need to be loaded to u= se +vfio-user. It is also possible for the client to concurrently use the curr= ent +kernel-based VFIO for one device, and vfio-user for another device. + +VFIO Device Model +----------------- + +A device under VFIO presents a standard interface to the user process. Man= y of +the VFIO operations in the existing interface use the ``ioctl()`` system c= all, and +references to the existing interface are called the ``ioctl()`` implementa= tion in +this document. + +The following sections describe the set of messages that implement the vfi= o-user +interface over a socket. In many cases, the messages are analogous to data +structures used in the ``ioctl()`` implementation. Messages derived from t= he +``ioctl()`` will have a name derived from the ``ioctl()`` command name. E= .g., the +``VFIO_DEVICE_GET_INFO`` ``ioctl()`` command becomes a +``VFIO_USER_DEVICE_GET_INFO`` message. The purpose of this reuse is to sh= are as +much code as feasible with the ``ioctl()`` implementation``. + +Connection Initiation +^^^^^^^^^^^^^^^^^^^^^ + +After the client connects to the server, the initial client message is +``VFIO_USER_VERSION`` to propose a protocol version and set of capabilitie= s to +apply to the session. The server replies with a compatible version and set= of +capabilities it supports, or closes the connection if it cannot support the +advertised version. + +Device Information +^^^^^^^^^^^^^^^^^^ + +The client uses a ``VFIO_USER_DEVICE_GET_INFO`` message to query the serve= r for +information about the device. This information includes: + +* The device type and whether it supports reset (``VFIO_DEVICE_FLAGS_``), +* the number of device regions, and +* the device presents to the client the number of interrupt types the devi= ce + supports. + +Region Information +^^^^^^^^^^^^^^^^^^ + +The client uses ``VFIO_USER_DEVICE_GET_REGION_INFO`` messages to query the +server for information about the device's regions. This information descri= bes: + +* Read and write permissions, whether it can be memory mapped, and whether= it + supports additional capabilities (``VFIO_REGION_INFO_CAP_``). +* Region index, size, and offset. + +When a device region can be mapped by the client, the server provides a fi= le +descriptor which the client can ``mmap()``. The server is responsible for +polling for client updates to memory mapped regions. + +Region Capabilities +""""""""""""""""""" + +Some regions have additional capabilities that cannot be described adequat= ely +by the region info data structure. These capabilities are returned in the +region info reply in a list similar to PCI capabilities in a PCI device's +configuration space. + +Sparse Regions +"""""""""""""" +A region can be memory-mappable in whole or in part. When only a subset of= a +region can be mapped by the client, a ``VFIO_REGION_INFO_CAP_SPARSE_MMAP`` +capability is included in the region info reply. This capability describes +which portions can be mapped by the client. + +.. Note:: + For example, in a virtual NVMe controller, sparse regions can be used so + that accesses to the NVMe registers (found in the beginning of BAR0) are + trapped (an infrequent event), while allowing direct access to the door= bells + (an extremely frequent event as every I/O submission requires a write to + BAR0), found in the next page after the NVMe registers in BAR0. + +Device-Specific Regions +""""""""""""""""""""""" + +A device can define regions additional to the standard ones (e.g. PCI inde= xes +0-8). This is achieved by including a ``VFIO_REGION_INFO_CAP_TYPE`` capabi= lity +in the region info reply of a device-specific region. Such regions are ref= lected +in ``struct vfio_user_device_info.num_regions``. Thus, for PCI devices this +value can be equal to, or higher than, ``VFIO_PCI_NUM_REGIONS``. + +Region I/O via file descriptors +------------------------------- + +For unmapped regions, region I/O from the client is done via +``VFIO_USER_REGION_READ/WRITE``. As an optimization, ioeventfds or ioregi= onfds +may be configured for sub-regions of some regions. A client may request +information on these sub-regions via ``VFIO_USER_DEVICE_GET_REGION_IO_FDS`= `; by +configuring the returned file descriptors as ioeventfds or ioregionfds, the +server can be directly notified of I/O (for example, by KVM) without takin= g a +trip through the client. + +Interrupts +^^^^^^^^^^ + +The client uses ``VFIO_USER_DEVICE_GET_IRQ_INFO`` messages to query the se= rver +for the device's interrupt types. The interrupt types are specific to the = bus +the device is attached to, and the client is expected to know the capabili= ties +of each interrupt type. The server can signal an interrupt by directly inj= ecting +interrupts into the guest via an event file descriptor. The client configu= res +how the server signals an interrupt with ``VFIO_USER_SET_IRQS`` messages. + +Device Read and Write +^^^^^^^^^^^^^^^^^^^^^ + +When the guest executes load or store operations to an unmapped device reg= ion, +the client forwards these operations to the server with +``VFIO_USER_REGION_READ`` or ``VFIO_USER_REGION_WRITE`` messages. The serv= er +will reply with data from the device on read operations or an acknowledgem= ent on +write operations. See `Read and Write Operations`_. + +Client memory access +-------------------- + +The client uses ``VFIO_USER_DMA_MAP`` and ``VFIO_USER_DMA_UNMAP`` messages= to +inform the server of the valid DMA ranges that the server can access on be= half +of a device (typically, VM guest memory). DMA memory may be accessed by the +server via ``VFIO_USER_DMA_READ`` and ``VFIO_USER_DMA_WRITE`` messages ove= r the +socket. In this case, the "DMA" part of the naming is a misnomer. + +Actual direct memory access of client memory from the server is possible i= f the +client provides file descriptors the server can ``mmap()``. Note that ``mm= ap()`` +privileges cannot be revoked by the client, therefore file descriptors sho= uld +only be exported in environments where the client trusts the server not to +corrupt guest memory. + +See `Read and Write Operations`_. + +Client/server interactions +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D + +Socket +------ + +A server can serve: + +1) one or more clients, and/or +2) one or more virtual devices, belonging to one or more clients. + +The current protocol specification requires a dedicated socket per +client/server connection. It is a server-side implementation detail whethe= r a +single server handles multiple virtual devices from the same or multiple +clients. The location of the socket is implementation-specific. Multiplexi= ng +clients, devices, and servers over the same socket is not supported in this +version of the protocol. + +Authentication +-------------- + +For ``AF_UNIX``, we rely on OS mandatory access controls on the socket fil= es, +therefore it is up to the management layer to set up the socket as require= d. +Socket types that span guests or hosts will require a proper authentication +mechanism. Defining that mechanism is deferred to a future version of the +protocol. + +Command Concurrency +------------------- + +A client may pipeline multiple commands without waiting for previous comma= nd +replies. The server will process commands in the order they are received.= A +consequence of this is if a client issues a command with the *No_reply* bi= t, +then subsequently issues a command without *No_reply*, the older command w= ill +have been processed before the reply to the younger command is sent by the +server. The client must be aware of the device's capability to process +concurrent commands if pipelining is used. For example, pipelining allows +multiple client threads to concurrently access device regions; the client = must +ensure these accesses obey device semantics. + +An example is a frame buffer device, where the device may allow concurrent +access to different areas of video memory, but may have indeterminate beha= vior +if concurrent accesses are performed to command or status registers. + +Note that unrelated messages sent from the server to the client can appear= in +between a client to server request/reply and vice versa. + +Implementers should be prepared for certain commands to exhibit potentially +unbounded latencies. For example, ``VFIO_USER_DEVICE_RESET`` may take an +arbitrarily long time to complete; clients should take care not to block +unnecessarily. + +Socket Disconnection Behavior +----------------------------- +The server and the client can disconnect from each other, either intention= ally +or unexpectedly. Both the client and the server need to know how to handle= such +events. + +Server Disconnection +^^^^^^^^^^^^^^^^^^^^ +A server disconnecting from the client may indicate that: + +1) A virtual device has been restarted, either intentionally (e.g. because= of a + device update) or unintentionally (e.g. because of a crash). +2) A virtual device has been shut down with no intention to be restarted. + +It is impossible for the client to know whether or not a failure is +intermittent or innocuous and should be retried, therefore the client shou= ld +reset the VFIO device when it detects the socket has been disconnected. +Error recovery will be driven by the guest's device error handling +behavior. + +Client Disconnection +^^^^^^^^^^^^^^^^^^^^ +The client disconnecting from the server primarily means that the client +has exited. Currently, this means that the guest is shut down so the devic= e is +no longer needed therefore the server can automatically exit. However, the= re +can be cases where a client disconnection should not result in a server ex= it: + +1) A single server serving multiple clients. +2) A multi-process QEMU upgrading itself step by step, which is not yet + implemented. + +Therefore in order for the protocol to be forward compatible, the server s= hould +respond to a client disconnection as follows: + + - all client memory regions are unmapped and cleaned up (including closin= g any + passed file descriptors) + - all IRQ file descriptors passed from the old client are closed + - the device state should otherwise be retained + +The expectation is that when a client reconnects, it will re-establish IRQ= and +client memory mappings. + +If anything happens to the client (such as qemu really did exit), the cont= rol +stack will know about it and can clean up resources accordingly. + +Security Considerations +----------------------- + +Speaking generally, vfio-user clients should not trust servers, and vice v= ersa. +Standard tools and mechanisms should be used on both sides to validate inp= ut and +prevent against denial of service scenarios, buffer overflow, etc. + +Request Retry and Response Timeout +---------------------------------- +A failed command is a command that has been successfully sent and has been +responded to with an error code. Failure to send the command in the first = place +(e.g. because the socket is disconnected) is a different type of error exa= mined +earlier in the disconnect section. + +.. Note:: + QEMU's VFIO retries certain operations if they fail. While this makes s= ense + for real HW, we don't know for sure whether it makes sense for virtual + devices. + +Defining a retry and timeout scheme is deferred to a future version of the +protocol. + +Message sizes +------------- + +Some requests have an ``argsz`` field. In a request, it defines the maximum +expected reply payload size, which should be at least the size of the fixed +reply payload headers defined here. The *request* payload size is defined = by the +usual ``msg_size`` field in the header, not the ``argsz`` field. + +In a reply, the server sets ``argsz`` field to the size needed for a full +payload size. This may be less than the requested maximum size. This may be +larger than the requested maximum size: in that case, the full payload is = not +included in the reply, but the ``argsz`` field in the reply indicates the = needed +size, allowing a client to allocate a larger buffer for holding the reply = before +trying again. + +In addition, during negotiation (see `Version`_), the client and server m= ay +each specify a ``max_data_xfer_size`` value; this defines the maximum data= that +may be read or written via one of the ``VFIO_USER_DMA/REGION_READ/WRITE`` +messages; see `Read and Write Operations`_. + +Protocol Specification +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +To distinguish from the base VFIO symbols, all vfio-user symbols are prefi= xed +with ``vfio_user`` or ``VFIO_USER``. In this revision, all data is in the +endianness of the host system, although this may be relaxed in future +revisions in cases where the client and server run on different hosts +with different endianness. + +Unless otherwise specified, all sizes should be presumed to be in bytes. + +.. _Commands: + +Commands +-------- +The following table lists the VFIO message command IDs, and whether the +message command is sent from the client or the server. + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Name Command Request Direction +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +``VFIO_USER_VERSION`` 1 client -> server +``VFIO_USER_DMA_MAP`` 2 client -> server +``VFIO_USER_DMA_UNMAP`` 3 client -> server +``VFIO_USER_DEVICE_GET_INFO`` 4 client -> server +``VFIO_USER_DEVICE_GET_REGION_INFO`` 5 client -> server +``VFIO_USER_DEVICE_GET_REGION_IO_FDS`` 6 client -> server +``VFIO_USER_DEVICE_GET_IRQ_INFO`` 7 client -> server +``VFIO_USER_DEVICE_SET_IRQS`` 8 client -> server +``VFIO_USER_REGION_READ`` 9 client -> server +``VFIO_USER_REGION_WRITE`` 10 client -> server +``VFIO_USER_DMA_READ`` 11 server -> client +``VFIO_USER_DMA_WRITE`` 12 server -> client +``VFIO_USER_DEVICE_RESET`` 13 client -> server +``VFIO_USER_REGION_WRITE_MULTI`` 15 client -> server +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Header +------ + +All messages, both command messages and reply messages, are preceded by a +16-byte header that contains basic information about the message. The head= er is +followed by message-specific data described in the sections below. + ++----------------+--------+-------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| Message ID | 0 | 2 | ++----------------+--------+-------------+ +| Command | 2 | 2 | ++----------------+--------+-------------+ +| Message size | 4 | 4 | ++----------------+--------+-------------+ +| Flags | 8 | 4 | ++----------------+--------+-------------+ +| | +-----+------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0-3 | Type | | +| | +-----+------------+ | +| | | 4 | No_reply | | +| | +-----+------------+ | +| | | 5 | Error | | +| | +-----+------------+ | ++----------------+--------+-------------+ +| Error | 12 | 4 | ++----------------+--------+-------------+ +| | 16 | variable | ++----------------+--------+-------------+ + +* *Message ID* identifies the message, and is echoed in the command's reply + message. Message IDs belong entirely to the sender, can be re-used (even + concurrently) and the receiver must not make any assumptions about their + uniqueness. +* *Command* specifies the command to be executed, listed in Commands_. It = is + also set in the reply header. +* *Message size* contains the size of the entire message, including the he= ader. +* *Flags* contains attributes of the message: + + * The *Type* bits indicate the message type. + + * *Command* (value 0x0) indicates a command message. + * *Reply* (value 0x1) indicates a reply message acknowledging a previ= ous + command with the same message ID. + * *No_reply* in a command message indicates that no reply is needed for = this + command. This is commonly used when multiple commands are sent, and o= nly + the last needs acknowledgement. + * *Error* in a reply message indicates the command being acknowledged had + an error. In this case, the *Error* field will be valid. + +* *Error* in a reply message is an optional UNIX errno value. It may be ze= ro + even if the Error bit is set in Flags. It is reserved in a command messa= ge. + +Each command message in Commands_ must be replied to with a reply message, +unless the message sets the *No_Reply* bit. The reply consists of the hea= der +with the *Reply* bit set, plus any additional data. + +If an error occurs, the reply message must only include the reply header. + +As the header is standard in both requests and replies, it is not included= in +the command-specific specifications below; each message definition should = be +appended to the standard header, and the offsets are given from the end of= the +standard header. + +``VFIO_USER_VERSION`` +--------------------- + +.. _Version: + +This is the initial message sent by the client after the socket connection= is +established; the same format is used for the server's reply. + +Upon establishing a connection, the client must send a ``VFIO_USER_VERSION= `` +message proposing a protocol version and a set of capabilities. The server +compares these with the versions and capabilities it supports and sends a +``VFIO_USER_VERSION`` reply according to the following rules. + +* The major version in the reply must be the same as proposed. If the clie= nt + does not support the proposed major, it closes the connection. +* The minor version in the reply must be equal to or less than the minor + version proposed. +* The capability list must be a subset of those proposed. If the server + requires a capability the client did not include, it closes the connecti= on. + +The protocol major version will only change when incompatible protocol cha= nges +are made, such as changing the message format. The minor version may change +when compatible changes are made, such as adding new messages or capabilit= ies, +Both the client and server must support all minor versions less than the +maximum minor version it supports. E.g., an implementation that supports +version 1.3 must also support 1.0 through 1.2. + +When making a change to this specification, the protocol version number mu= st +be included in the form "added in version X.Y" + +Request +^^^^^^^ + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D =3D=3D=3D= =3D +Name Offset Size +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D =3D=3D=3D= =3D +version major 0 2 +version minor 2 2 +version data 4 variable (including terminating NUL). Optional. +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D =3D=3D=3D= =3D + +The version data is an optional UTF-8 encoded JSON byte array with the fol= lowing +format: + ++--------------+--------+-----------------------------------+ +| Name | Type | Description | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| capabilities | object | Contains common capabilities that | +| | | the sender supports. Optional. | ++--------------+--------+-----------------------------------+ + +Capabilities: + ++--------------------+---------+------------------------------------------= ------+ +| Name | Type | Description = | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| max_msg_fds | number | Maximum number of file descriptors that c= an be | +| | | received by the sender in one message. = | +| | | Optional. If not specified then the recei= ver | +| | | must assume a value of ``1``. = | ++--------------------+---------+------------------------------------------= ------+ +| max_data_xfer_size | number | Maximum ``count`` for data transfer messa= ges; | +| | | see `Read and Write Operations`_. Optiona= l, | +| | | with a default value of 1048576 bytes. = | ++--------------------+---------+------------------------------------------= ------+ +| pgsizes | number | Page sizes supported in DMA map operation= s | +| | | or'ed together. Optional, with a default = value | +| | | of supporting only 4k pages. = | ++--------------------+---------+------------------------------------------= ------+ +| max_dma_maps | number | Maximum number DMA map windows that can b= e | +| | | valid simultaneously. Optional, with a = | +| | | value of 65535 (64k-1). = | ++--------------------+---------+------------------------------------------= ------+ +| migration | object | Migration capability parameters. If missi= ng | +| | | then migration is not supported by the se= nder. | ++--------------------+---------+------------------------------------------= ------+ +| write_multiple | boolean | ``VFIO_USER_REGION_WRITE_MULTI`` messages= | +| | | are supported if the value is ``true``. = | ++--------------------+---------+------------------------------------------= ------+ + +The migration capability contains the following name/value pairs: + ++-----------------+--------+----------------------------------------------= ----+ +| Name | Type | Description = | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+ +| pgsize | number | Page size of dirty pages bitmap. The smallest= | +| | | between the client and the server is used. = | ++-----------------+--------+----------------------------------------------= ----+ +| max_bitmap_size | number | Maximum bitmap size in ``VFIO_USER_DIRTY_PAGE= S`` | +| | | and ``VFIO_DMA_UNMAP`` messages. Optional, = | +| | | with a default value of 256MB. = | ++-----------------+--------+----------------------------------------------= ----+ + +Reply +^^^^^ + +The same message format is used in the server's reply with the semantics +described above. + +``VFIO_USER_DMA_MAP`` +--------------------- + +This command message is sent by the client to the server to inform it of t= he +memory regions the server can access. It must be sent before the server can +perform any DMA to the client. It is normally sent directly after the vers= ion +handshake is completed, but may also occur when memory is added to the cli= ent, +or if the client uses a vIOMMU. + +Request +^^^^^^^ + +The request payload for this message is a structure of the following forma= t: + ++-------------+--------+-------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+-------------+ +| flags | 4 | 4 | ++-------------+--------+-------------+ +| | +-----+------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | readable | | +| | +-----+------------+ | +| | | 1 | writeable | | +| | +-----+------------+ | ++-------------+--------+-------------+ +| offset | 8 | 8 | ++-------------+--------+-------------+ +| address | 16 | 8 | ++-------------+--------+-------------+ +| size | 24 | 8 | ++-------------+--------+-------------+ + +* *argsz* is the size of the above structure. Note there is no reply paylo= ad, + so this field differs from other message types. +* *flags* contains the following region attributes: + + * *readable* indicates that the region can be read from. + + * *writeable* indicates that the region can be written to. + +* *offset* is the file offset of the region with respect to the associated= file + descriptor, or zero if the region is not mappable +* *address* is the base DMA address of the region. +* *size* is the size of the region. + +This structure is 32 bytes in size, so the message size is 16 + 32 bytes. + +If the DMA region being added can be directly mapped by the server, a file +descriptor must be sent as part of the message meta-data. The region can be +mapped via the mmap() system call. On ``AF_UNIX`` sockets, the file descri= ptor +must be passed as ``SCM_RIGHTS`` type ancillary data. Otherwise, if the D= MA +region cannot be directly mapped by the server, no file descriptor must be= sent +as part of the message meta-data and the DMA region can be accessed by the +server using ``VFIO_USER_DMA_READ`` and ``VFIO_USER_DMA_WRITE`` messages, +explained in `Read and Write Operations`_. A command to map over an existi= ng +region must be failed by the server with ``EEXIST`` set in error field in = the +reply. + +Reply +^^^^^ + +There is no payload in the reply message. + +``VFIO_USER_DMA_UNMAP`` +----------------------- + +This command message is sent by the client to the server to inform it that= a +DMA region, previously made available via a ``VFIO_USER_DMA_MAP`` command +message, is no longer available for DMA. It typically occurs when memory is +subtracted from the client or if the client uses a vIOMMU. The DMA region = is +described by the following structure: + +Request +^^^^^^^ + +The request payload for this message is a structure of the following forma= t: + ++--------------+--------+------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++--------------+--------+------------------------+ +| flags | 4 | 4 | ++--------------+--------+------------------------+ +| address | 8 | 8 | ++--------------+--------+------------------------+ +| size | 16 | 8 | ++--------------+--------+------------------------+ + +* *argsz* is the maximum size of the reply payload. +* *flags* is unused in this version. +* *address* is the base DMA address of the DMA region. +* *size* is the size of the DMA region. + +The address and size of the DMA region being unmapped must match exactly a +previous mapping. + +Reply +^^^^^ + +Upon receiving a ``VFIO_USER_DMA_UNMAP`` command, if the file descriptor is +mapped then the server must release all references to that DMA region befo= re +replying, which potentially includes in-flight DMA transactions. + +The server responds with the original DMA entry in the request. + + +``VFIO_USER_DEVICE_GET_INFO`` +----------------------------- + +This command message is sent by the client to the server to query for basic +information about the device. + +Request +^^^^^^^ + ++-------------+--------+--------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+--------------------------+ +| flags | 4 | 4 | ++-------------+--------+--------------------------+ +| | +-----+-------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_DEVICE_FLAGS_RESET | | +| | +-----+-------------------------+ | +| | | 1 | VFIO_DEVICE_FLAGS_PCI | | +| | +-----+-------------------------+ | ++-------------+--------+--------------------------+ +| num_regions | 8 | 4 | ++-------------+--------+--------------------------+ +| num_irqs | 12 | 4 | ++-------------+--------+--------------------------+ + +* *argsz* is the maximum size of the reply payload +* all other fields must be zero. + +Reply +^^^^^ + ++-------------+--------+--------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+--------------------------+ +| flags | 4 | 4 | ++-------------+--------+--------------------------+ +| | +-----+-------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_DEVICE_FLAGS_RESET | | +| | +-----+-------------------------+ | +| | | 1 | VFIO_DEVICE_FLAGS_PCI | | +| | +-----+-------------------------+ | ++-------------+--------+--------------------------+ +| num_regions | 8 | 4 | ++-------------+--------+--------------------------+ +| num_irqs | 12 | 4 | ++-------------+--------+--------------------------+ + +* *argsz* is the size required for the full reply payload (16 bytes today) +* *flags* contains the following device attributes. + + * ``VFIO_DEVICE_FLAGS_RESET`` indicates that the device supports the + ``VFIO_USER_DEVICE_RESET`` message. + * ``VFIO_DEVICE_FLAGS_PCI`` indicates that the device is a PCI device. + +* *num_regions* is the number of memory regions that the device exposes. +* *num_irqs* is the number of distinct interrupt types that the device sup= ports. + +This version of the protocol only supports PCI devices. Additional devices= may +be supported in future versions. + +``VFIO_USER_DEVICE_GET_REGION_INFO`` +------------------------------------ + +This command message is sent by the client to the server to query for +information about device regions. The VFIO region info structure is define= d in +```` (``struct vfio_region_info``). + +Request +^^^^^^^ + ++------------+--------+------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+ +| argsz | 0 | 4 | ++------------+--------+------------------------------+ +| flags | 4 | 4 | ++------------+--------+------------------------------+ +| index | 8 | 4 | ++------------+--------+------------------------------+ +| cap_offset | 12 | 4 | ++------------+--------+------------------------------+ +| size | 16 | 8 | ++------------+--------+------------------------------+ +| offset | 24 | 8 | ++------------+--------+------------------------------+ + +* *argsz* the maximum size of the reply payload +* *index* is the index of memory region being queried, it is the only field + that is required to be set in the command message. +* all other fields must be zero. + +Reply +^^^^^ + ++------------+--------+------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+ +| argsz | 0 | 4 | ++------------+--------+------------------------------+ +| flags | 4 | 4 | ++------------+--------+------------------------------+ +| | +-----+-----------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_REGION_INFO_FLAG_READ | | +| | +-----+-----------------------------+ | +| | | 1 | VFIO_REGION_INFO_FLAG_WRITE | | +| | +-----+-----------------------------+ | +| | | 2 | VFIO_REGION_INFO_FLAG_MMAP | | +| | +-----+-----------------------------+ | +| | | 3 | VFIO_REGION_INFO_FLAG_CAPS | | +| | +-----+-----------------------------+ | ++------------+--------+------------------------------+ ++------------+--------+------------------------------+ +| index | 8 | 4 | ++------------+--------+------------------------------+ +| cap_offset | 12 | 4 | ++------------+--------+------------------------------+ +| size | 16 | 8 | ++------------+--------+------------------------------+ +| offset | 24 | 8 | ++------------+--------+------------------------------+ + +* *argsz* is the size required for the full reply payload (region info str= ucture + plus the size of any region capabilities) +* *flags* are attributes of the region: + + * ``VFIO_REGION_INFO_FLAG_READ`` allows client read access to the region. + * ``VFIO_REGION_INFO_FLAG_WRITE`` allows client write access to the regi= on. + * ``VFIO_REGION_INFO_FLAG_MMAP`` specifies the client can mmap() the reg= ion. + When this flag is set, the reply will include a file descriptor in its + meta-data. On ``AF_UNIX`` sockets, the file descriptors will be passed= as + ``SCM_RIGHTS`` type ancillary data. + * ``VFIO_REGION_INFO_FLAG_CAPS`` indicates additional capabilities found= in the + reply. + +* *index* is the index of memory region being queried, it is the only field + that is required to be set in the command message. +* *cap_offset* describes where additional region capabilities can be found. + cap_offset is relative to the beginning of the VFIO region info structur= e. + The data structure it points is a VFIO cap header defined in + ````. +* *size* is the size of the region. +* *offset* is the offset that should be given to the mmap() system call for + regions with the MMAP attribute. It is also used as the base offset when + mapping a VFIO sparse mmap area, described below. + +VFIO region capabilities +"""""""""""""""""""""""" + +The VFIO region information can also include a capabilities list. This lis= t is +similar to a PCI capability list - each entry has a common header that +identifies a capability and where the next capability in the list can be f= ound. +The VFIO capability header format is defined in ```` (``stru= ct +vfio_info_cap_header``). + +VFIO cap header format +"""""""""""""""""""""" + ++---------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D+ +| id | 0 | 2 | ++---------+--------+------+ +| version | 2 | 2 | ++---------+--------+------+ +| next | 4 | 4 | ++---------+--------+------+ + +* *id* is the capability identity. +* *version* is a capability-specific version number. +* *next* specifies the offset of the next capability in the capability lis= t. It + is relative to the beginning of the VFIO region info structure. + +VFIO sparse mmap cap header +""""""""""""""""""""""""""" + ++------------------+----------------------------------+ +| Name | Value | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| id | VFIO_REGION_INFO_CAP_SPARSE_MMAP | ++------------------+----------------------------------+ +| version | 0x1 | ++------------------+----------------------------------+ +| next | | ++------------------+----------------------------------+ +| sparse mmap info | VFIO region info sparse mmap | ++------------------+----------------------------------+ + +This capability is defined when only a subrange of the region supports +direct access by the client via mmap(). The VFIO sparse mmap area is defin= ed in +```` (``struct vfio_region_sparse_mmap_area`` and ``struct +vfio_region_info_cap_sparse_mmap``). + +VFIO region info cap sparse mmap +"""""""""""""""""""""""""""""""" + ++----------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D= =3D+ +| nr_areas | 0 | 4 | ++----------+--------+------+ +| reserved | 4 | 4 | ++----------+--------+------+ +| offset | 8 | 8 | ++----------+--------+------+ +| size | 16 | 8 | ++----------+--------+------+ +| ... | | | ++----------+--------+------+ + +* *nr_areas* is the number of sparse mmap areas in the region. +* *offset* and size describe a single area that can be mapped by the clien= t. + There will be *nr_areas* pairs of offset and size. The offset will be ad= ded to + the base offset given in the ``VFIO_USER_DEVICE_GET_REGION_INFO`` to for= m the + offset argument of the subsequent mmap() call. + +The VFIO sparse mmap area is defined in ```` (``struct +vfio_region_info_cap_sparse_mmap``). + + +``VFIO_USER_DEVICE_GET_REGION_IO_FDS`` +-------------------------------------- + +Clients can access regions via ``VFIO_USER_REGION_READ/WRITE`` or, if prov= ided, by +``mmap()`` of a file descriptor provided by the server. + +``VFIO_USER_DEVICE_GET_REGION_IO_FDS`` provides an alternative access mech= anism via +file descriptors. This is an optional feature intended for performance +improvements where an underlying sub-system (such as KVM) supports communi= cation +across such file descriptors to the vfio-user server, without needing to +round-trip through the client. + +The server returns an array of sub-regions for the requested region. Each +sub-region describes a span (offset and size) of a region, along with the +requested file descriptor notification mechanism to use. Each sub-region = in the +response message may choose to use a different method, as defined below. = The +two mechanisms supported in this specification are ioeventfds and ioregion= fds. + +The server in addition returns a file descriptor in the ancillary data; cl= ients +are expected to configure each sub-region's file descriptor with the reque= sted +notification method. For example, a client could configure KVM with the +requested ioeventfd via a ``KVM_IOEVENTFD`` ``ioctl()``. + +Request +^^^^^^^ + ++-------------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+------+ +| flags | 4 | 4 | ++-------------+--------+------+ +| index | 8 | 4 | ++-------------+--------+------+ +| count | 12 | 4 | ++-------------+--------+------+ + +* *argsz* the maximum size of the reply payload +* *index* is the index of memory region being queried +* all other fields must be zero + +The client must set ``flags`` to zero and specify the region being queried= in +the ``index``. + +Reply +^^^^^ + ++-------------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+------+ +| flags | 4 | 4 | ++-------------+--------+------+ +| index | 8 | 4 | ++-------------+--------+------+ +| count | 12 | 4 | ++-------------+--------+------+ +| sub-regions | 16 | ... | ++-------------+--------+------+ + +* *argsz* is the size of the region IO FD info structure plus the + total size of the sub-region array. Thus, each array entry "i" is at off= set + i * ((argsz - 32) / count). Note that currently this is 40 bytes for bot= h IO + FD types, but this is not to be relied on. As elsewhere, this indicates = the + full reply payload size needed. +* *flags* must be zero +* *index* is the index of memory region being queried +* *count* is the number of sub-regions in the array +* *sub-regions* is the array of Sub-Region IO FD info structures + +The reply message will additionally include at least one file descriptor i= n the +ancillary data. Note that more than one sub-region may share the same file +descriptor. + +Note that it is the client's responsibility to verify the requested values= (for +example, that the requested offset does not exceed the region's bounds). + +Each sub-region given in the response has one of two possible structures, +depending whether *type* is ``VFIO_USER_IO_FD_TYPE_IOEVENTFD`` or +``VFIO_USER_IO_FD_TYPE_IOREGIONFD``: + +Sub-Region IO FD info format (ioeventfd) +"""""""""""""""""""""""""""""""""""""""" + ++-----------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D+ +| offset | 0 | 8 | ++-----------+--------+------+ +| size | 8 | 8 | ++-----------+--------+------+ +| fd_index | 16 | 4 | ++-----------+--------+------+ +| type | 20 | 4 | ++-----------+--------+------+ +| flags | 24 | 4 | ++-----------+--------+------+ +| padding | 28 | 4 | ++-----------+--------+------+ +| datamatch | 32 | 8 | ++-----------+--------+------+ + +* *offset* is the offset of the start of the sub-region within the region + requested ("physical address offset" for the region) +* *size* is the length of the sub-region. This may be zero if the access s= ize is + not relevant, which may allow for optimizations +* *fd_index* is the index in the ancillary data of the FD to use for ioeve= ntfd + notification; it may be shared. +* *type* is ``VFIO_USER_IO_FD_TYPE_IOEVENTFD`` +* *flags* is any of: + + * ``KVM_IOEVENTFD_FLAG_DATAMATCH`` + * ``KVM_IOEVENTFD_FLAG_PIO`` + * ``KVM_IOEVENTFD_FLAG_VIRTIO_CCW_NOTIFY`` (FIXME: makes sense?) + +* *datamatch* is the datamatch value if needed + +See https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt, *4.59 +KVM_IOEVENTFD* for further context on the ioeventfd-specific fields. + +Sub-Region IO FD info format (ioregionfd) +""""""""""""""""""""""""""""""""""""""""" + ++-----------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D+ +| offset | 0 | 8 | ++-----------+--------+------+ +| size | 8 | 8 | ++-----------+--------+------+ +| fd_index | 16 | 4 | ++-----------+--------+------+ +| type | 20 | 4 | ++-----------+--------+------+ +| flags | 24 | 4 | ++-----------+--------+------+ +| padding | 28 | 4 | ++-----------+--------+------+ +| user_data | 32 | 8 | ++-----------+--------+------+ + +* *offset* is the offset of the start of the sub-region within the region + requested ("physical address offset" for the region) +* *size* is the length of the sub-region. This may be zero if the access s= ize is + not relevant, which may allow for optimizations; ``KVM_IOREGION_POSTED_W= RITES`` + must be set in *flags* in this case +* *fd_index* is the index in the ancillary data of the FD to use for ioreg= ionfd + messages; it may be shared +* *type* is ``VFIO_USER_IO_FD_TYPE_IOREGIONFD`` +* *flags* is any of: + + * ``KVM_IOREGION_PIO`` + * ``KVM_IOREGION_POSTED_WRITES`` + +* *user_data* is an opaque value passed back to the server via a message o= n the + file descriptor + +For further information on the ioregionfd-specific fields, see: +https://lore.kernel.org/kvm/cover.1613828726.git.eafanasova@gmail.com/ + +(FIXME: update with final API docs.) + +``VFIO_USER_DEVICE_GET_IRQ_INFO`` +--------------------------------- + +This command message is sent by the client to the server to query for +information about device interrupt types. The VFIO IRQ info structure is +defined in ```` (``struct vfio_irq_info``). + +Request +^^^^^^^ + ++-------+--------+---------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------+--------+---------------------------+ +| flags | 4 | 4 | ++-------+--------+---------------------------+ +| | +-----+--------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_IRQ_INFO_EVENTFD | | +| | +-----+--------------------------+ | +| | | 1 | VFIO_IRQ_INFO_MASKABLE | | +| | +-----+--------------------------+ | +| | | 2 | VFIO_IRQ_INFO_AUTOMASKED | | +| | +-----+--------------------------+ | +| | | 3 | VFIO_IRQ_INFO_NORESIZE | | +| | +-----+--------------------------+ | ++-------+--------+---------------------------+ +| index | 8 | 4 | ++-------+--------+---------------------------+ +| count | 12 | 4 | ++-------+--------+---------------------------+ + +* *argsz* is the maximum size of the reply payload (16 bytes today) +* index is the index of IRQ type being queried (e.g. ``VFIO_PCI_MSIX_IRQ_I= NDEX``) +* all other fields must be zero + +Reply +^^^^^ + ++-------+--------+---------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------+--------+---------------------------+ +| flags | 4 | 4 | ++-------+--------+---------------------------+ +| | +-----+--------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_IRQ_INFO_EVENTFD | | +| | +-----+--------------------------+ | +| | | 1 | VFIO_IRQ_INFO_MASKABLE | | +| | +-----+--------------------------+ | +| | | 2 | VFIO_IRQ_INFO_AUTOMASKED | | +| | +-----+--------------------------+ | +| | | 3 | VFIO_IRQ_INFO_NORESIZE | | +| | +-----+--------------------------+ | ++-------+--------+---------------------------+ +| index | 8 | 4 | ++-------+--------+---------------------------+ +| count | 12 | 4 | ++-------+--------+---------------------------+ + +* *argsz* is the size required for the full reply payload (16 bytes today) +* *flags* defines IRQ attributes: + + * ``VFIO_IRQ_INFO_EVENTFD`` indicates the IRQ type can support server ev= entfd + signalling. + * ``VFIO_IRQ_INFO_MASKABLE`` indicates that the IRQ type supports the ``= MASK`` + and ``UNMASK`` actions in a ``VFIO_USER_DEVICE_SET_IRQS`` message. + * ``VFIO_IRQ_INFO_AUTOMASKED`` indicates the IRQ type masks itself after= being + triggered, and the client must send an ``UNMASK`` action to receive new + interrupts. + * ``VFIO_IRQ_INFO_NORESIZE`` indicates ``VFIO_USER_SET_IRQS`` operations= setup + interrupts as a set, and new sub-indexes cannot be enabled without dis= abling + the entire type. +* index is the index of IRQ type being queried +* count describes the number of interrupts of the queried type. + +``VFIO_USER_DEVICE_SET_IRQS`` +----------------------------- + +This command message is sent by the client to the server to set actions for +device interrupt types. The VFIO IRQ set structure is defined in +```` (``struct vfio_irq_set``). + +Request +^^^^^^^ + ++-------+--------+------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------+--------+------------------------------+ +| flags | 4 | 4 | ++-------+--------+------------------------------+ +| | +-----+-----------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_IRQ_SET_DATA_NONE | | +| | +-----+-----------------------------+ | +| | | 1 | VFIO_IRQ_SET_DATA_BOOL | | +| | +-----+-----------------------------+ | +| | | 2 | VFIO_IRQ_SET_DATA_EVENTFD | | +| | +-----+-----------------------------+ | +| | | 3 | VFIO_IRQ_SET_ACTION_MASK | | +| | +-----+-----------------------------+ | +| | | 4 | VFIO_IRQ_SET_ACTION_UNMASK | | +| | +-----+-----------------------------+ | +| | | 5 | VFIO_IRQ_SET_ACTION_TRIGGER | | +| | +-----+-----------------------------+ | ++-------+--------+------------------------------+ +| index | 8 | 4 | ++-------+--------+------------------------------+ +| start | 12 | 4 | ++-------+--------+------------------------------+ +| count | 16 | 4 | ++-------+--------+------------------------------+ +| data | 20 | variable | ++-------+--------+------------------------------+ + +* *argsz* is the size of the VFIO IRQ set request payload, including any *= data* + field. Note there is no reply payload, so this field differs from other + message types. +* *flags* defines the action performed on the interrupt range. The ``DATA`` + flags describe the data field sent in the message; the ``ACTION`` flags + describe the action to be performed. The flags are mutually exclusive for + both sets. + + * ``VFIO_IRQ_SET_DATA_NONE`` indicates there is no data field in the com= mand. + The action is performed unconditionally. + * ``VFIO_IRQ_SET_DATA_BOOL`` indicates the data field is an array of boo= lean + bytes. The action is performed if the corresponding boolean is true. + * ``VFIO_IRQ_SET_DATA_EVENTFD`` indicates an array of event file descrip= tors + was sent in the message meta-data. These descriptors will be signalled= when + the action defined by the action flags occurs. In ``AF_UNIX`` sockets,= the + descriptors are sent as ``SCM_RIGHTS`` type ancillary data. + If no file descriptors are provided, this de-assigns the specified + previously configured interrupts. + * ``VFIO_IRQ_SET_ACTION_MASK`` indicates a masking event. It can be used= with + ``VFIO_IRQ_SET_DATA_BOOL`` or ``VFIO_IRQ_SET_DATA_NONE`` to mask an in= terrupt, + or with ``VFIO_IRQ_SET_DATA_EVENTFD`` to generate an event when the gu= est masks + the interrupt. + * ``VFIO_IRQ_SET_ACTION_UNMASK`` indicates an unmasking event. It can be= used + with ``VFIO_IRQ_SET_DATA_BOOL`` or ``VFIO_IRQ_SET_DATA_NONE`` to unmas= k an + interrupt, or with ``VFIO_IRQ_SET_DATA_EVENTFD`` to generate an event = when the + guest unmasks the interrupt. + * ``VFIO_IRQ_SET_ACTION_TRIGGER`` indicates a triggering event. It can b= e used + with ``VFIO_IRQ_SET_DATA_BOOL`` or ``VFIO_IRQ_SET_DATA_NONE`` to trigg= er an + interrupt, or with ``VFIO_IRQ_SET_DATA_EVENTFD`` to generate an event = when the + server triggers the interrupt. + +* *index* is the index of IRQ type being setup. +* *start* is the start of the sub-index being set. +* *count* describes the number of sub-indexes being set. As a special case= , a + count (and start) of 0, with data flags of ``VFIO_IRQ_SET_DATA_NONE`` di= sables + all interrupts of the index. +* *data* is an optional field included when the + ``VFIO_IRQ_SET_DATA_BOOL`` flag is present. It contains an array of bool= eans + that specify whether the action is to be performed on the corresponding + index. It's used when the action is only performed on a subset of the ra= nge + specified. + +Not all interrupt types support every combination of data and action flags. +The client must know the capabilities of the device and IRQ index before it +sends a ``VFIO_USER_DEVICE_SET_IRQ`` message. + +In typical operation, a specific IRQ may operate as follows: + +1. The client sends a ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_EVENTFD|VFIO_IRQ_SET_ACTION_TRIGGER)`` alo= ng + with an eventfd. This associates the IRQ with a particular eventfd on t= he + server side. + +#. The client may send a ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_EVENTFD|VFIO_IRQ_SET_ACTION_MASK/UNMASK)``= along + with another eventfd. This associates the given eventfd with the + mask/unmask state on the server side. + +#. The server may trigger the IRQ by writing 1 to the eventfd. + +#. The server may mask/unmask an IRQ which will write 1 to the correspondi= ng + mask/unmask eventfd, if there is one. + +5. A client may trigger a device IRQ itself, by sending a + ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_NONE/BOOL|VFIO_IRQ_SET_ACTION_TRIGGER)``. + +6. A client may mask or unmask the IRQ, by sending a + ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_NONE/BOOL|VFIO_IRQ_SET_ACTION_MASK/UNMASK)= ``. + +Reply +^^^^^ + +There is no payload in the reply. + +.. _Read and Write Operations: + +Note that all of these operations must be supported by the client and/or s= erver, +even if the corresponding memory or device region has been shared as mappa= ble. + +The ``count`` field must not exceed the value of ``max_data_xfer_size`` of= the +peer, for both reads and writes. + +``VFIO_USER_REGION_READ`` +------------------------- + +If a device region is not mappable, it's not directly accessible by the cl= ient +via ``mmap()`` of the underlying file descriptor. In this case, a client c= an +read from a device region with this message. + +Request +^^^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ + +* *offset* into the region being accessed. +* *region* is the index of the region being accessed. +* *count* is the size of the data to be transferred. + +Reply +^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ +| data | 16 | variable | ++--------+--------+----------+ + +* *offset* into the region accessed. +* *region* is the index of the region accessed. +* *count* is the size of the data transferred. +* *data* is the data that was read from the device region. + +``VFIO_USER_REGION_WRITE`` +-------------------------- + +If a device region is not mappable, it's not directly accessible by the cl= ient +via mmap() of the underlying fd. In this case, a client can write to a dev= ice +region with this message. + +Request +^^^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ +| data | 16 | variable | ++--------+--------+----------+ + +* *offset* into the region being accessed. +* *region* is the index of the region being accessed. +* *count* is the size of the data to be transferred. +* *data* is the data to write + +Reply +^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ + +* *offset* into the region accessed. +* *region* is the index of the region accessed. +* *count* is the size of the data transferred. + +``VFIO_USER_DMA_READ`` +----------------------- + +If the client has not shared mappable memory, the server can use this mess= age to +read from guest memory. + +Request +^^^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 8 | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. This address = must have + been previously exported to the server with a ``VFIO_USER_DMA_MAP`` mess= age. +* *count* is the size of the data to be transferred. + +Reply +^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 8 | ++---------+--------+----------+ +| data | 16 | variable | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. +* *count* is the size of the data transferred. +* *data* is the data read. + +``VFIO_USER_DMA_WRITE`` +----------------------- + +If the client has not shared mappable memory, the server can use this mess= age to +write to guest memory. + +Request +^^^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 8 | ++---------+--------+----------+ +| data | 16 | variable | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. This address = must have + been previously exported to the server with a ``VFIO_USER_DMA_MAP`` mess= age. +* *count* is the size of the data to be transferred. +* *data* is the data to write + +Reply +^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 4 | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. +* *count* is the size of the data transferred. + +``VFIO_USER_DEVICE_RESET`` +-------------------------- + +This command message is sent from the client to the server to reset the de= vice. +Neither the request or reply have a payload. + +``VFIO_USER_REGION_WRITE_MULTI`` +-------------------------------- + +This message can be used to coalesce multiple device write operations +into a single messgage. It is only used as an optimization when the +outgoing message queue is relatively full. + +Request +^^^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| wr_cnt | 0 | 8 | ++---------+--------+----------+ +| wrs | 8 | variable | ++---------+--------+----------+ + +* *wr_cnt* is the number of device writes coalesced in the message +* *wrs* is an array of device writes defined below + +Single Device Write Format +"""""""""""""""""""""""""" + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ +| data | 16 | 8 | ++--------+--------+----------+ + +* *offset* into the region being accessed. +* *region* is the index of the region being accessed. +* *count* is the size of the data to be transferred. This format can + only describe writes of 8 bytes or less. +* *data* is the data to write. + +Reply +^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| wr_cnt | 0 | 8 | ++---------+--------+----------+ + +* *wr_cnt* is the number of device writes completed. + + +Appendices +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Unused VFIO ``ioctl()`` commands +-------------------------------- + +The following VFIO commands do not have an equivalent vfio-user command: + +* ``VFIO_GET_API_VERSION`` +* ``VFIO_CHECK_EXTENSION`` +* ``VFIO_SET_IOMMU`` +* ``VFIO_GROUP_GET_STATUS`` +* ``VFIO_GROUP_SET_CONTAINER`` +* ``VFIO_GROUP_UNSET_CONTAINER`` +* ``VFIO_GROUP_GET_DEVICE_FD`` +* ``VFIO_IOMMU_GET_INFO`` + +However, once support for live migration for VFIO devices is finalized some +of the above commands may have to be handled by the client in their +corresponding vfio-user form. This will be addressed in a future protocol +version. + +VFIO groups and containers +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The current VFIO implementation includes group and container idioms that +describe how a device relates to the host IOMMU. In the vfio-user +implementation, the IOMMU is implemented in SW by the client, and is not +visible to the server. The simplest idea would be that the client put each +device into its own group and container. + +Backend Program Conventions +--------------------------- + +vfio-user backend program conventions are based on the vhost-user ones. + +* The backend program must not daemonize itself. +* No assumptions must be made as to what access the backend program has on= the + system. +* File descriptors 0, 1 and 2 must exist, must have regular + stdin/stdout/stderr semantics, and can be redirected. +* The backend program must honor the SIGTERM signal. +* The backend program must accept the following commands line options: + + * ``--socket-path=3DPATH``: path to UNIX domain socket, + * ``--fd=3DFDNUM``: file descriptor for UNIX domain socket, incompatible= with + ``--socket-path`` +* The backend program must be accompanied with a JSON file stored under + ``/usr/share/vfio-user``. + +TODO add schema similar to docs/interop/vhost-user.json. --=20 2.43.0