From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132873; cv=pass; d=zohomail.com; s=zohoarc; b=IgQ9R6cJ/MDu+8FUfvzprbsY4JB7tSoJ8d+6BhPWsu2iy0stlkvZxoHMBsFmyd8Q6R9zJgIxSBN/WHzLWIMkfJIN91Y97mTYmgxHBbNGeDx8FxONI5RpUEUYG+dM4KOel5n0A9GsrIkpV/bZv36MTbsRiW/E+FPFHzTP/tTXxdI= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132873; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=VK4HpqJzexgufdeGvvyFkzlbHQd0SjE/hKmlvmuY7Vw=; b=SW/UDsZas9UjvF7CHaofMt6vuBwig3QE6k7TSYqnyxmXLQQCngOXCEGjSUYn/OGEp1vwkf4d9nfKCE91CJx5wNdxIa0Y2rvlgPxtytSOm0MwG4Zwlzap8VPIynG+/ZiZHs/nvboVQcsGw8/GmzXhrFy5Xej6kfxbvVccikFU6r4= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629132873476609.5849922891562; Mon, 16 Aug 2021 09:54:33 -0700 (PDT) Received: from localhost ([::1]:53336 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfsd-0003zN-HG for importer@patchew.org; Mon, 16 Aug 2021 12:54:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37224) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhi-00010q-BS for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:14 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:49738) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhY-0007y3-Vn for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:14 -0400 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGaeJP011290; Mon, 16 Aug 2021 16:43:03 GMT Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 3af83029gn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:03 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeu1k137465; Mon, 16 Aug 2021 16:43:01 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2168.outbound.protection.outlook.com [104.47.55.168]) by userp3020.oracle.com with ESMTP id 3aeqksfrwu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:00 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by BYAPR10MB2565.namprd10.prod.outlook.com (2603:10b6:a02:b2::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.19; Mon, 16 Aug 2021 16:42:55 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:42:55 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:42:55 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=VK4HpqJzexgufdeGvvyFkzlbHQd0SjE/hKmlvmuY7Vw=; b=tawX31y1KVxis0h1yGTnIOTHfWy+cv+w5szBxG7hkumWKFw18/5FHdNhGILPVlZ3xbhW /YQ1Grho605+JzFO0XpQHk89TfUT8RTKPKBVIktDmcdPof9e8ouLm7Huu7Ec3Bvva0UL G9EnFZsCkQSqUgbvH6LD51cIMSOfw40NDdWr6odtLWfykO8bDG1f7UTI6h9fMnHOSVjB 4vxJKBf+MaTNILx7W5LhroQBzkw5uurU2GI+aQors14TLDQRHJQh+wDIVF6xxQwJEwUb DcT2d8fNYsIUDFah24ILa3C+80AfAeTSFYIICMhnxeW6jDb9Uk71unCdtSBLzyag7F3/ fg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=VK4HpqJzexgufdeGvvyFkzlbHQd0SjE/hKmlvmuY7Vw=; b=yIZoFJ/WeNnZtGzuLaiaGeX2wie3H6TeJ+LFYxLig9LQQxaZxxDMVkp73z+RJdokd06O Br7FzdE9qgKYimCcvfe74LTw3JKDjseMLubdNwqoJHBYKJVV7sH3Jf1BPHuJ0Df1xXj0 bhHGLI2CH6pGlcF2TTGu+qm8EihI2+97q5GJtZEtYcFhKcoqoxxMlwRe/OQJeLeGjBhh JQ3PRVitxPvflGnsENPQ9XkwZjTymhSoxVYdNaT6MTZ7tSJ9vzvg6zwFsmqDsBhCwQgx EUicqs4f9eg6cLoG1pxGh5EIlgnzg4lywd4vg8JZ9j/GByW5/1xBG2P1o0ttTdSA3XJI ug== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iUeQH9O08eZWVH5Emwn2RjnZfjywa54vdLKH71ypbg0RANkueBQSUHmjUV7lzFyp2jnky66K8S4QUOdOuXEVYYq+ifxYpi97Cmf60dpTSc2hFMhVk5TDU49paK4XG+CJSzydATf2WFJskzShFXYQt0poMrufBt4DFV3Vau7kE8j9oWutLvdJjBknLoYdjrbU0SgwpmJ0ZlbQqhbEk2T4knDETbST418/Y8h4gtJNd55i4N+OOGmMc685ewFrdyAoBcx/gvYN3ECM5Top6P91QIF4tsuBIvp5PzNHEW/GSA5vL5h2taqI7Wb3TJ9BKLi+fVl1JRB/J6qwwowKpzzDNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VK4HpqJzexgufdeGvvyFkzlbHQd0SjE/hKmlvmuY7Vw=; b=nKsxGOYS6ii1ZJPrK/1+EcAcTHvdC+PJBM/mHQAkAVqS4U18zpZrl6ENeEQfNcc6cLVs/jlaLFeWAFf+vyDNA4KzHR7qX65nUWJdS1wp84dHjZiE2O4Rxj6dd4CuCqSBD8nR29sa4wIzcCx9oGcCdHPPIPTj1uoDKHbnbO8pUwIfcdnw+CyJOOE+nNMHUYvVObbuJ3ttonmCOrrJYBmbPo4OmJtFR710YEe83dwY89Ac/2mOt4dhHwKxfM4lQeuNkHNyApUPfuYSIe01aCWhf8gQ8XqEw6TP4JAZQBpoiwFq+NU7Kzmu6VYKzt6BrVA8GozRFZlatjoywwPqmpI9IQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VK4HpqJzexgufdeGvvyFkzlbHQd0SjE/hKmlvmuY7Vw=; b=Ru18iiu6SjzjWOrVY2SCIsVJ6bJVJxZ379jt1R9WrSq4bXCsfub+TTKzqO0GQLEdCevfLdyH2tIPJXiN5vkpzrlV2xBD14ykUHrnE4/J2/IhnywP2nrPaNdpSAy1KsSm75MkGnvgBexe/C/18GxXnY/K3rdciaeYuusyaee4H4U= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 01/16] vfio-user: introduce vfio-user protocol specification Date: Mon, 16 Aug 2021 09:42:34 -0700 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 254ed045-3fe1-497a-aed3-08d960d4e5d4 X-MS-TrafficTypeDiagnostic: BYAPR10MB2565: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ilXB+GKI0tbqyUDJxsyXz4kSo678jNlkwGGjwRmyxKp/2ZjSIxeJpm7/yzySEGuVho3iSjQUG5wGtSEKjTPtW28NITJ9IR5+1Tk7747cetrTbIR7oA+26YsG3z3Z+Wl/xVolShNC44EBEDHXYv+lFR/8Hhj/7Ez2pJspH63LVSnKOwtnzXqq3AobRFR7I44bV0ZcEAB8ZCG29Phd22bMV7uu9rwIRg4X0E8yhHsm6L7k9oX/zgUZD0vzdWNZvNbiKLiwVBi7SPoNeq9O099IPnw9EIxelTdyzy8w3f77KXWP2J7xRYcBOKz3W8aUvj+NSX5xJlVL3mB0vIPq98RbHsCawwFNlAsJPrAPoHUeawGdiP/39Jl8tfuo0Y0imWBO4Jsr5T6iET7bgCYQWpKzOFjNxoxb/nIExr03fRAYs9yAxFlyrZrFqNyOCDOOYdOWXKxdgr9ixwgV/CyfN0cs+g4/Xt4A7COjpYIyDlozjpwEhDxheAcw7uVdrLtKV/lVNUmO/KYKM547dyRLuvPTQeh0Hj4SLssYP8PIZ+HFIsjjfhbwyFCK1T/e7DfHmyUVTAoOpqUFYcxEqzS8S65onUMZ/G4F2rie7D0Bul5v1UbhmpbaceA09B5SW4+HCTyudU5ZFwAJzJka+BMy9nLfMEleIBPyJeIDN3IkdpQuXR6Y3O7xID2eOumTBOpf5AetDtUq8QEc+lfI3RWotCIU62gKsm2dPm8M27q1GhoBb6TWtTcPSHXYUw/qNOImj645CW1QOowenVYIk+a7v0Pwhg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(6486002)(316002)(38100700002)(44832011)(2616005)(6512007)(2906002)(6916009)(52116002)(5660300002)(966005)(508600001)(6506007)(66946007)(4326008)(66476007)(66556008)(6666004)(30864003)(186003)(86362001)(8676002)(8936002)(36756003)(83380400001)(579004)(559001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?rPyVYY9Wx7W4BBRwbebk4fRmaBYnk0Wt5Z7VyJLDHvCRjc7FpOGYJi7zspg3?= =?us-ascii?Q?VcI7130m00EAr79sBjqNHqu+0SV1H8J1skJxVOppJi1GTD/tgbq6kONo2x7x?= =?us-ascii?Q?qXBROJvCxZrxcZ3W5No7Uv9mr4vhGPGgA0FQiTCzjmnh4B0Q6eVVoTLTOReo?= =?us-ascii?Q?1ji/oQgfdem+imJMwjFAIWqAQpRItiSyYuWlsZOFe65G5DGBggMIH2XK53nj?= =?us-ascii?Q?KsEecoOj36HpmZ8Dcv0dUw2F9fvV0bKGoxglm2kGtKZHWVDW7RvrxCTJ5FTW?= =?us-ascii?Q?UPrmEXO2NtIltFrLhI9Jc5WM6bDJ1pU7mClXeZKKPOIJG0iFKYVznw71eABU?= =?us-ascii?Q?v16NMjqMtG76CK+ALENqELeVo8EjDHrFnUQ4hUww9Ry0D77skx85gI23hCCS?= =?us-ascii?Q?Q1gnCupH4CbeBhYvf7HlbeuZlfkZs9E+eoSKfnFcv/i515xFWU9H7cjrVcIX?= =?us-ascii?Q?I5ujLjbkeN4x2GYfi4wAv16jFCcKv0jd4gmV5I+REonlMI4r8dLk863cBmbO?= =?us-ascii?Q?Mj89/f5XbKvnSYjF+8PF/arLJeZEW+Tu+bkDcxcEVpOAVhYGVYSl9fs/27hc?= =?us-ascii?Q?FzqyLJkJXZRZ3qp32MpkAMgQDf2xbD7aoCiKrrjxeB1AoePTKFi5OJvn1JoY?= =?us-ascii?Q?RFciTdxFnFDRCKjOG7DDe9C4AHAojbSZFTyZSE66V18+Lml+bvKO5fsjylGH?= =?us-ascii?Q?5WUM+aM8JVWR766+UJNPgfDyc2/gcZb0PLodtFtDSttONbdym9FojPwveGki?= =?us-ascii?Q?/7Hm56Twv4VhmXXJ+rQvLRMP+NdlExJYtNFG5zpNa88YnVSSmupZUkqc1wLz?= =?us-ascii?Q?gVCnu8P7QupN2EIVGkqa9LtgQtiScus2bXCRJXUnGaI2jaJMRt+06LdIJnU9?= =?us-ascii?Q?ipTiwBDCMxKJzGoDGppbihrVDczhlY0oRMQ/F5L5MMcjCexdDz0ezVNXLFjC?= =?us-ascii?Q?7T5ltwdepEZ8a+ZTNL1QfE2XqgPvjxUm16j1BGQnGq7rHa+FxgXFy3nZmP+R?= =?us-ascii?Q?/jhFxGq4U4x5pHd3olpjYZYukw7DY+Te0aWGzAckoSjxpCPW3nzrjF9AEljv?= =?us-ascii?Q?WicUX27ALI/FUfXkibZIj6PWKMeJbE+7GIIaE0t//YLaOd5EiH3J2xli/bLo?= =?us-ascii?Q?Yttb9u/owZR4Yay3Ma9Wa7Mgq8zEoc8W90VgjA9Ld0voAybAhsWv03KmfCai?= =?us-ascii?Q?GdiWkx173/MCfRtQgFNSiheaZCrJ3Ods/hao5S/ZphV13IVKJah0x2xabhdP?= =?us-ascii?Q?z8RaIwpFX3ZMZsKxsLzwnFS0GONSLQb0pLrjNtxXvEVmAHJwmhAx/qNsYPDb?= =?us-ascii?Q?HMudJMoI16ioWYuSEpKO7QXmeVh8XSbUqdfbkAwUGlJWFeW5dGwtJ229UkVR?= =?us-ascii?Q?1Gq/yk7i7FxdDDTEcE84syMOvuKu?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 254ed045-3fe1-497a-aed3-08d960d4e5d4 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:42:55.7708 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9lWDrznxZlwkB7nLuqqt5bG5tC/r5iaCXRN4k9p16683dGGczm0vJg/rAtJKdRZReqEJQS0vnJWGGRX7PlbJRtg6tvDdXRS/qh8NkIbWyD4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2565 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 malwarescore=0 mlxscore=0 spamscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-ORIG-GUID: b3CZXGB2kNzqAF6ztMS0yQ1kxSOXlY8b X-Proofpoint-GUID: b3CZXGB2kNzqAF6ztMS0yQ1kxSOXlY8b Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, WEIRD_QUOTING=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132874410100001 Content-Type: text/plain; charset="utf-8" From: Thanos Makatos This patch introduces the vfio-user protocol specification (formerly known as VFIO-over-socket), which is designed to allow devices to be emulated outside QEMU, in a separate process. vfio-user reuses the existing VFIO defines, structs and concepts. This patch is sourced from: https://patchwork.kernel.org/project/qemu-devel/patch/20210614104608.212276= -1-thanos.makatos@nutanix.com/ It has been earlier discussed as an RFC in: "RFC: use VFIO over a UNIX domain socket to implement device offloading" Signed-off-by: John G Johnson Signed-off-by: Thanos Makatos Signed-off-by: John Levon --- docs/devel/index.rst | 1 + docs/devel/vfio-user.rst | 1809 ++++++++++++++++++++++++++++++++++++++ MAINTAINERS | 6 + 3 files changed, 1816 insertions(+) create mode 100644 docs/devel/vfio-user.rst diff --git a/docs/devel/index.rst b/docs/devel/index.rst index 5522db7241..304ca1c12f 100644 --- a/docs/devel/index.rst +++ b/docs/devel/index.rst @@ -44,3 +44,4 @@ modifying QEMU's source code. vfio-migration qapi-code-gen writing-qmp-commands + vfio-user diff --git a/docs/devel/vfio-user.rst b/docs/devel/vfio-user.rst new file mode 100644 index 0000000000..0b2acec101 --- /dev/null +++ b/docs/devel/vfio-user.rst @@ -0,0 +1,1809 @@ +.. include:: +******************************** +vfio-user Protocol Specification +******************************** + +-------------- +Version_ 0.9.1 +-------------- + +.. contents:: Table of Contents + +Introduction +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +vfio-user is a protocol that allows a device to be emulated in a separate +process outside of a Virtual Machine Monitor (VMM). vfio-user devices cons= ist +of a generic VFIO device type, living inside the VMM, which we call the cl= ient, +and the core device implementation, living outside the VMM, which we call = the +server. + +The vfio-user specification is partly based on the +`Linux VFIO ioctl interface `_. + +VFIO is a mature and stable API, backed by an extensively used framework. = The +existing VFIO client implementation in QEMU (``qemu/hw/vfio/``) can be lar= gely +re-used, though there is nothing in this specification that requires that +particular implementation. None of the VFIO kernel modules are required for +supporting the protocol, on either the client or server side. Some source +definitions in VFIO are re-used for vfio-user. + +The main idea is to allow a virtual device to function in a separate proce= ss in +the same host over a UNIX domain socket. A UNIX domain socket (``AF_UNIX``= ) is +chosen because file descriptors can be trivially sent over it, which in tu= rn +allows: + +* Sharing of client memory for DMA with the server. +* Sharing of server memory with the client for fast MMIO. +* Efficient sharing of eventfd's for triggering interrupts. + +Other socket types could be used which allow the server to run in a separa= te +guest in the same host (``AF_VSOCK``) or remotely (``AF_INET``). Theoretic= ally +the underlying transport does not necessarily have to be a socket, however= we do +not examine such alternatives. In this protocol version we focus on using = a UNIX +domain socket and introduce basic support for the other two types of socke= ts +without considering performance implications. + +While passing of file descriptors is desirable for performance reasons, su= pport +is not necessary for either the client or the server in order to implement= the +protocol. There is always an in-band, message-passing fall back mechanism. + +Overview +=3D=3D=3D=3D=3D=3D=3D=3D + +VFIO is a framework that allows a physical device to be securely passed th= rough +to a user space process; the device-specific kernel driver does not drive = the +device at all. Typically, the user space process is a VMM and the device = is +passed through to it in order to achieve high performance. VFIO provides a= n API +and the required functionality in the kernel. QEMU has adopted VFIO to all= ow a +guest to directly access physical devices, instead of emulating them in +software. + +vfio-user reuses the core VFIO concepts defined in its API, but implements= them +as messages to be sent over a socket. It does not change the kernel-based = VFIO +in any way, in fact none of the VFIO kernel modules need to be loaded to u= se +vfio-user. It is also possible for the client to concurrently use the curr= ent +kernel-based VFIO for one device, and vfio-user for another device. + +VFIO Device Model +----------------- + +A device under VFIO presents a standard interface to the user process. Man= y of +the VFIO operations in the existing interface use the ``ioctl()`` system c= all, and +references to the existing interface are called the ``ioctl()`` implementa= tion in +this document. + +The following sections describe the set of messages that implement the vfi= o-user +interface over a socket. In many cases, the messages are analogous to data +structures used in the ``ioctl()`` implementation. Messages derived from t= he +``ioctl()`` will have a name derived from the ``ioctl()`` command name. E= .g., the +``VFIO_DEVICE_GET_INFO`` ``ioctl()`` command becomes a +``VFIO_USER_DEVICE_GET_INFO`` message. The purpose of this reuse is to sh= are as +much code as feasible with the ``ioctl()`` implementation``. + +Connection Initiation +^^^^^^^^^^^^^^^^^^^^^ + +After the client connects to the server, the initial client message is +``VFIO_USER_VERSION`` to propose a protocol version and set of capabilitie= s to +apply to the session. The server replies with a compatible version and set= of +capabilities it supports, or closes the connection if it cannot support the +advertised version. + +Device Information +^^^^^^^^^^^^^^^^^^ + +The client uses a ``VFIO_USER_DEVICE_GET_INFO`` message to query the serve= r for +information about the device. This information includes: + +* The device type and whether it supports reset (``VFIO_DEVICE_FLAGS_``), +* the number of device regions, and +* the device presents to the client the number of interrupt types the devi= ce + supports. + +Region Information +^^^^^^^^^^^^^^^^^^ + +The client uses ``VFIO_USER_DEVICE_GET_REGION_INFO`` messages to query the +server for information about the device's regions. This information descri= bes: + +* Read and write permissions, whether it can be memory mapped, and whether= it + supports additional capabilities (``VFIO_REGION_INFO_CAP_``). +* Region index, size, and offset. + +When a device region can be mapped by the client, the server provides a fi= le +descriptor which the client can ``mmap()``. The server is responsible for +polling for client updates to memory mapped regions. + +Region Capabilities +""""""""""""""""""" + +Some regions have additional capabilities that cannot be described adequat= ely +by the region info data structure. These capabilities are returned in the +region info reply in a list similar to PCI capabilities in a PCI device's +configuration space. + +Sparse Regions +"""""""""""""" +A region can be memory-mappable in whole or in part. When only a subset of= a +region can be mapped by the client, a ``VFIO_REGION_INFO_CAP_SPARSE_MMAP`` +capability is included in the region info reply. This capability describes +which portions can be mapped by the client. + +.. Note:: + For example, in a virtual NVMe controller, sparse regions can be used so + that accesses to the NVMe registers (found in the beginning of BAR0) are + trapped (an infrequent event), while allowing direct access to the door= bells + (an extremely frequent event as every I/O submission requires a write to + BAR0), found in the next page after the NVMe registers in BAR0. + +Device-Specific Regions +""""""""""""""""""""""" + +A device can define regions additional to the standard ones (e.g. PCI inde= xes +0-8). This is achieved by including a ``VFIO_REGION_INFO_CAP_TYPE`` capabi= lity +in the region info reply of a device-specific region. Such regions are ref= lected +in ``struct vfio_user_device_info.num_regions``. Thus, for PCI devices this +value can be equal to, or higher than, ``VFIO_PCI_NUM_REGIONS``. + +Region I/O via file descriptors +------------------------------- + +For unmapped regions, region I/O from the client is done via +``VFIO_USER_REGION_READ/WRITE``. As an optimization, ioeventfds or ioregi= onfds +may be configured for sub-regions of some regions. A client may request +information on these sub-regions via ``VFIO_USER_DEVICE_GET_REGION_IO_FDS`= `; by +configuring the returned file descriptors as ioeventfds or ioregionfds, the +server can be directly notified of I/O (for example, by KVM) without takin= g a +trip through the client. + +Interrupts +^^^^^^^^^^ + +The client uses ``VFIO_USER_DEVICE_GET_IRQ_INFO`` messages to query the se= rver +for the device's interrupt types. The interrupt types are specific to the = bus +the device is attached to, and the client is expected to know the capabili= ties +of each interrupt type. The server can signal an interrupt by directly inj= ecting +interrupts into the guest via an event file descriptor. The client configu= res +how the server signals an interrupt with ``VFIO_USER_SET_IRQS`` messages. + +Device Read and Write +^^^^^^^^^^^^^^^^^^^^^ + +When the guest executes load or store operations to an unmapped device reg= ion, +the client forwards these operations to the server with +``VFIO_USER_REGION_READ`` or ``VFIO_USER_REGION_WRITE`` messages. The serv= er +will reply with data from the device on read operations or an acknowledgem= ent on +write operations. See `Read and Write Operations`_. + +Client memory access +-------------------- + +The client uses ``VFIO_USER_DMA_MAP`` and ``VFIO_USER_DMA_UNMAP`` messages= to +inform the server of the valid DMA ranges that the server can access on be= half +of a device (typically, VM guest memory). DMA memory may be accessed by the +server via ``VFIO_USER_DMA_READ`` and ``VFIO_USER_DMA_WRITE`` messages ove= r the +socket. In this case, the "DMA" part of the naming is a misnomer. + +Actual direct memory access of client memory from the server is possible i= f the +client provides file descriptors the server can ``mmap()``. Note that ``mm= ap()`` +privileges cannot be revoked by the client, therefore file descriptors sho= uld +only be exported in environments where the client trusts the server not to +corrupt guest memory. + +See `Read and Write Operations`_. + +Client/server interactions +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D + +Socket +------ + +A server can serve: + +1) one or more clients, and/or +2) one or more virtual devices, belonging to one or more clients. + +The current protocol specification requires a dedicated socket per +client/server connection. It is a server-side implementation detail whethe= r a +single server handles multiple virtual devices from the same or multiple +clients. The location of the socket is implementation-specific. Multiplexi= ng +clients, devices, and servers over the same socket is not supported in this +version of the protocol. + +Authentication +-------------- + +For ``AF_UNIX``, we rely on OS mandatory access controls on the socket fil= es, +therefore it is up to the management layer to set up the socket as require= d. +Socket types than span guests or hosts will require a proper authentication +mechanism. Defining that mechanism is deferred to a future version of the +protocol. + +Command Concurrency +------------------- + +A client may pipeline multiple commands without waiting for previous comma= nd +replies. The server will process commands in the order they are received.= A +consequence of this is if a client issues a command with the *No_reply* bi= t, +then subsequently issues a command without *No_reply*, the older command w= ill +have been processed before the reply to the younger command is sent by the +server. The client must be aware of the device's capability to process +concurrent commands if pipelining is used. For example, pipelining allows +multiple client threads to concurrently access device regions; the client = must +ensure these accesses obey device semantics. + +An example is a frame buffer device, where the device may allow concurrent +access to different areas of video memory, but may have indeterminate beha= vior +if concurrent accesses are performed to command or status registers. + +Note that unrelated messages sent from the server to the client can appear= in +between a client to server request/reply and vice versa. + +Implementers should be prepared for certain commands to exhibit potentially +unbounded latencies. For example, ``VFIO_USER_DEVICE_RESET`` may take an +arbitrarily long time to complete; clients should take care not to block +unnecessarily. + +Socket Disconnection Behavior +----------------------------- +The server and the client can disconnect from each other, either intention= ally +or unexpectedly. Both the client and the server need to know how to handle= such +events. + +Server Disconnection +^^^^^^^^^^^^^^^^^^^^ +A server disconnecting from the client may indicate that: + +1) A virtual device has been restarted, either intentionally (e.g. because= of a + device update) or unintentionally (e.g. because of a crash). +2) A virtual device has been shut down with no intention to be restarted. + +It is impossible for the client to know whether or not a failure is +intermittent or innocuous and should be retried, therefore the client shou= ld +reset the VFIO device when it detects the socket has been disconnected. +Error recovery will be driven by the guest's device error handling +behavior. + +Client Disconnection +^^^^^^^^^^^^^^^^^^^^ +The client disconnecting from the server primarily means that the client +has exited. Currently, this means that the guest is shut down so the devic= e is +no longer needed therefore the server can automatically exit. However, the= re +can be cases where a client disconnection should not result in a server ex= it: + +1) A single server serving multiple clients. +2) A multi-process QEMU upgrading itself step by step, which is not yet + implemented. + +Therefore in order for the protocol to be forward compatible, the server s= hould +respond to a client disconnection as follows: + + - all client memory regions are unmapped and cleaned up (including closin= g any + passed file descriptors) + - all IRQ file descriptors passed from the old client are closed + - the device state should otherwise be retained + +The expectation is that when a client reconnects, it will re-establish IRQ= and +client memory mappings. + +If anything happens to the client (such as qemu really did exit), the cont= rol +stack will know about it and can clean up resources accordingly. + +Security Considerations +----------------------- + +Speaking generally, vfio-user clients should not trust servers, and vice v= ersa. +Standard tools and mechanisms should be used on both sides to validate inp= ut and +prevent against denial of service scenarios, buffer overflow, etc. + +Request Retry and Response Timeout +---------------------------------- +A failed command is a command that has been successfully sent and has been +responded to with an error code. Failure to send the command in the first = place +(e.g. because the socket is disconnected) is a different type of error exa= mined +earlier in the disconnect section. + +.. Note:: + QEMU's VFIO retries certain operations if they fail. While this makes s= ense + for real HW, we don't know for sure whether it makes sense for virtual + devices. + +Defining a retry and timeout scheme is deferred to a future version of the +protocol. + +Message sizes +------------- + +Some requests have an ``argsz`` field. In a request, it defines the maximum +expected reply payload size, which should be at least the size of the fixed +reply payload headers defined here. The *request* payload size is defined = by the +usual ``msg_size`` field in the header, not the ``argsz`` field. + +In a reply, the server sets ``argsz`` field to the size needed for a full +payload size. This may be less than the requested maximum size. This may be +larger than the requested maximum size: in that case, the full payload is = not +included in the reply, but the ``argsz`` field in the reply indicates the = needed +size, allowing a client to allocate a larger buffer for holding the reply = before +trying again. + +In addition, during negotiation (see `Version`_), the client and server m= ay +each specify a ``max_data_xfer_size`` value; this defines the maximum data= that +may be read or written via one of the ``VFIO_USER_DMA/REGION_READ/WRITE`` +messages; see `Read and Write Operations`_. + +Protocol Specification +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +To distinguish from the base VFIO symbols, all vfio-user symbols are prefi= xed +with ``vfio_user`` or ``VFIO_USER``. In this revision, all data is in the +little-endian format, although this may be relaxed in future revisions in = cases +where the client and server are both big-endian. + +Unless otherwise specified, all sizes should be presumed to be in bytes. + +.. _Commands: + +Commands +-------- +The following table lists the VFIO message command IDs, and whether the +message command is sent from the client or the server. + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Name Command Request Direction +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +``VFIO_USER_VERSION`` 1 client -> server +``VFIO_USER_DMA_MAP`` 2 client -> server +``VFIO_USER_DMA_UNMAP`` 3 client -> server +``VFIO_USER_DEVICE_GET_INFO`` 4 client -> server +``VFIO_USER_DEVICE_GET_REGION_INFO`` 5 client -> server +``VFIO_USER_DEVICE_GET_REGION_IO_FDS`` 6 client -> server +``VFIO_USER_DEVICE_GET_IRQ_INFO`` 7 client -> server +``VFIO_USER_DEVICE_SET_IRQS`` 8 client -> server +``VFIO_USER_REGION_READ`` 9 client -> server +``VFIO_USER_REGION_WRITE`` 10 client -> server +``VFIO_USER_DMA_READ`` 11 server -> client +``VFIO_USER_DMA_WRITE`` 12 server -> client +``VFIO_USER_DEVICE_RESET`` 13 client -> server +``VFIO_USER_DIRTY_PAGES`` 14 client -> server +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Header +------ + +All messages, both command messages and reply messages, are preceded by a +16-byte header that contains basic information about the message. The head= er is +followed by message-specific data described in the sections below. + ++----------------+--------+-------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| Message ID | 0 | 2 | ++----------------+--------+-------------+ +| Command | 2 | 2 | ++----------------+--------+-------------+ +| Message size | 4 | 4 | ++----------------+--------+-------------+ +| Flags | 8 | 4 | ++----------------+--------+-------------+ +| | +-----+------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0-3 | Type | | +| | +-----+------------+ | +| | | 4 | No_reply | | +| | +-----+------------+ | +| | | 5 | Error | | +| | +-----+------------+ | ++----------------+--------+-------------+ +| Error | 12 | 4 | ++----------------+--------+-------------+ +| | 16 | variable | ++----------------+--------+-------------+ + +* *Message ID* identifies the message, and is echoed in the command's reply + message. Message IDs belong entirely to the sender, can be re-used (even + concurrently) and the receiver must not make any assumptions about their + uniqueness. +* *Command* specifies the command to be executed, listed in Commands_. It = is + also set in the reply header. +* *Message size* contains the size of the entire message, including the he= ader. +* *Flags* contains attributes of the message: + + * The *Type* bits indicate the message type. + + * *Command* (value 0x0) indicates a command message. + * *Reply* (value 0x1) indicates a reply message acknowledging a previ= ous + command with the same message ID. + * *No_reply* in a command message indicates that no reply is needed for = this + command. This is commonly used when multiple commands are sent, and o= nly + the last needs acknowledgement. + * *Error* in a reply message indicates the command being acknowledged had + an error. In this case, the *Error* field will be valid. + +* *Error* in a reply message is an optional UNIX errno value. It may be ze= ro + even if the Error bit is set in Flags. It is reserved in a command messa= ge. + +Each command message in Commands_ must be replied to with a reply message, +unless the message sets the *No_Reply* bit. The reply consists of the hea= der +with the *Reply* bit set, plus any additional data. + +If an error occurs, the reply message must only include the reply header. + +As the header is standard in both requests and replies, it is not included= in +the command-specific specifications below; each message definition should = be +appended to the standard header, and the offsets are given from the end of= the +standard header. + +``VFIO_USER_VERSION`` +--------------------- + +.. _Version: + +This is the initial message sent by the client after the socket connection= is +established; the same format is used for the server's reply. + +Upon establishing a connection, the client must send a ``VFIO_USER_VERSION= `` +message proposing a protocol version and a set of capabilities. The server +compares these with the versions and capabilities it supports and sends a +``VFIO_USER_VERSION`` reply according to the following rules. + +* The major version in the reply must be the same as proposed. If the clie= nt + does not support the proposed major, it closes the connection. +* The minor version in the reply must be equal to or less than the minor + version proposed. +* The capability list must be a subset of those proposed. If the server + requires a capability the client did not include, it closes the connecti= on. + +The protocol major version will only change when incompatible protocol cha= nges +are made, such as changing the message format. The minor version may change +when compatible changes are made, such as adding new messages or capabilit= ies, +Both the client and server must support all minor versions less than the +maximum minor version it supports. E.g., an implementation that supports +version 1.3 must also support 1.0 through 1.2. + +When making a change to this specification, the protocol version number mu= st +be included in the form "added in version X.Y" + +Request +^^^^^^^ + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D =3D=3D=3D= =3D +Name Offset Size +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D =3D=3D=3D= =3D +version major 0 2 +version minor 2 2 +version data 4 variable (including terminating NUL). Optional. +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D =3D=3D=3D= =3D + +The version data is an optional UTF-8 encoded JSON byte array with the fol= lowing +format: + ++--------------+--------+-----------------------------------+ +| Name | Type | Description | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| capabilities | object | Contains common capabilities that | +| | | the sender supports. Optional. | ++--------------+--------+-----------------------------------+ + +Capabilities: + ++--------------------+--------+-------------------------------------------= -----+ +| Name | Type | Description = | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| max_msg_fds | number | Maximum number of file descriptors that ca= n be | +| | | received by the sender in one message. = | +| | | Optional. If not specified then the receiv= er | +| | | must assume a value of ``1``. = | ++--------------------+--------+-------------------------------------------= -----+ +| max_data_xfer_size | number | Maximum ``count`` for data transfer messag= es; | +| | | see `Read and Write Operations`_. Optional= , | +| | | with a default value of 1048576 bytes. = | ++--------------------+--------+-------------------------------------------= -----+ +| migration | object | Migration capability parameters. If missin= g | +| | | then migration is not supported by the sen= der. | ++--------------------+--------+-------------------------------------------= -----+ + +The migration capability contains the following name/value pairs: + ++--------+--------+-----------------------------------------------+ +| Name | Type | Description | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| pgsize | number | Page size of dirty pages bitmap. The smallest | +| | | between the client and the server is used. | ++--------+--------+-----------------------------------------------+ + +Reply +^^^^^ + +The same message format is used in the server's reply with the semantics +described above. + +``VFIO_USER_DMA_MAP`` +--------------------- + +This command message is sent by the client to the server to inform it of t= he +memory regions the server can access. It must be sent before the server can +perform any DMA to the client. It is normally sent directly after the vers= ion +handshake is completed, but may also occur when memory is added to the cli= ent, +or if the client uses a vIOMMU. + +Request +^^^^^^^ + +The request payload for this message is a structure of the following forma= t: + ++-------------+--------+-------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+-------------+ +| flags | 4 | 4 | ++-------------+--------+-------------+ +| | +-----+------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | readable | | +| | +-----+------------+ | +| | | 1 | writeable | | +| | +-----+------------+ | ++-------------+--------+-------------+ +| offset | 8 | 8 | ++-------------+--------+-------------+ +| address | 16 | 8 | ++-------------+--------+-------------+ +| size | 24 | 8 | ++-------------+--------+-------------+ + +* *argsz* is the size of the above structure. Note there is no reply paylo= ad, + so this field differs from other message types. +* *flags* contains the following region attributes: + + * *readable* indicates that the region can be read from. + + * *writeable* indicates that the region can be written to. + +* *offset* is the file offset of the region with respect to the associated= file + descriptor, or zero if the region is not mappable +* *address* is the base DMA address of the region. +* *size* is the size of the region. + +This structure is 32 bytes in size, so the message size is 16 + 32 bytes. + +If the DMA region being added can be directly mapped by the server, a file +descriptor must be sent as part of the message meta-data. The region can be +mapped via the mmap() system call. On ``AF_UNIX`` sockets, the file descri= ptor +must be passed as ``SCM_RIGHTS`` type ancillary data. Otherwise, if the D= MA +region cannot be directly mapped by the server, no file descriptor must be= sent +as part of the message meta-data and the DMA region can be accessed by the +server using ``VFIO_USER_DMA_READ`` and ``VFIO_USER_DMA_WRITE`` messages, +explained in `Read and Write Operations`_. A command to map over an existi= ng +region must be failed by the server with ``EEXIST`` set in error field in = the +reply. + +Reply +^^^^^ + +There is no payload in the reply message. + +``VFIO_USER_DMA_UNMAP`` +----------------------- + +This command message is sent by the client to the server to inform it that= a +DMA region, previously made available via a ``VFIO_USER_DMA_MAP`` command +message, is no longer available for DMA. It typically occurs when memory is +subtracted from the client or if the client uses a vIOMMU. The DMA region = is +described by the following structure: + +Request +^^^^^^^ + +The request payload for this message is a structure of the following forma= t: + ++--------------+--------+------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++--------------+--------+------------------------+ +| flags | 4 | 4 | ++--------------+--------+------------------------+ +| | +-----+-----------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | get dirty page bitmap | | +| | +-----+-----------------------+ | ++--------------+--------+------------------------+ +| address | 8 | 8 | ++--------------+--------+------------------------+ +| size | 16 | 8 | ++--------------+--------+------------------------+ + +* *argsz* is the maximum size of the reply payload. +* *flags* contains the following DMA region attributes: + + * *get dirty page bitmap* indicates that a dirty page bitmap must be + populated before unmapping the DMA region. The client must provide a + `VFIO Bitmap`_ structure, explained below, immediately following this + entry. + +* *address* is the base DMA address of the DMA region. +* *size* is the size of the DMA region. + +The address and size of the DMA region being unmapped must match exactly a +previous mapping. The size of request message depends on whether or not the +*get dirty page bitmap* bit is set in Flags: + +* If not set, the size of the total request message is: 16 + 24. + +* If set, the size of the total request message is: 16 + 24 + 16. + +.. _VFIO Bitmap: + +VFIO Bitmap Format +"""""""""""""""""" + ++--------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D+ +| pgsize | 0 | 8 | ++--------+--------+------+ +| size | 8 | 8 | ++--------+--------+------+ + +* *pgsize* is the page size for the bitmap, in bytes. +* *size* is the size for the bitmap, in bytes, excluding the VFIO bitmap h= eader. + +Reply +^^^^^ + +Upon receiving a ``VFIO_USER_DMA_UNMAP`` command, if the file descriptor is +mapped then the server must release all references to that DMA region befo= re +replying, which potentially includes in-flight DMA transactions. + +The server responds with the original DMA entry in the request. If the +*get dirty page bitmap* bit is set in flags in the request, then +the server also includes the `VFIO Bitmap`_ structure sent in the request, +followed by the corresponding dirty page bitmap, where each bit represents +one page of size *pgsize* in `VFIO Bitmap`_ . + +The total size of the total reply message is: +16 + 24 + (16 + *size* in `VFIO Bitmap`_ if *get dirty page bitmap* is set= ). + +``VFIO_USER_DEVICE_GET_INFO`` +----------------------------- + +This command message is sent by the client to the server to query for basic +information about the device. + +Request +^^^^^^^ + ++-------------+--------+--------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+--------------------------+ +| flags | 4 | 4 | ++-------------+--------+--------------------------+ +| | +-----+-------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_DEVICE_FLAGS_RESET | | +| | +-----+-------------------------+ | +| | | 1 | VFIO_DEVICE_FLAGS_PCI | | +| | +-----+-------------------------+ | ++-------------+--------+--------------------------+ +| num_regions | 8 | 4 | ++-------------+--------+--------------------------+ +| num_irqs | 12 | 4 | ++-------------+--------+--------------------------+ + +* *argsz* is the maximum size of the reply payload +* all other fields must be zero. + +Reply +^^^^^ + ++-------------+--------+--------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+--------------------------+ +| flags | 4 | 4 | ++-------------+--------+--------------------------+ +| | +-----+-------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_DEVICE_FLAGS_RESET | | +| | +-----+-------------------------+ | +| | | 1 | VFIO_DEVICE_FLAGS_PCI | | +| | +-----+-------------------------+ | ++-------------+--------+--------------------------+ +| num_regions | 8 | 4 | ++-------------+--------+--------------------------+ +| num_irqs | 12 | 4 | ++-------------+--------+--------------------------+ + +* *argsz* is the size required for the full reply payload (16 bytes today) +* *flags* contains the following device attributes. + + * ``VFIO_DEVICE_FLAGS_RESET`` indicates that the device supports the + ``VFIO_USER_DEVICE_RESET`` message. + * ``VFIO_DEVICE_FLAGS_PCI`` indicates that the device is a PCI device. + +* *num_regions* is the number of memory regions that the device exposes. +* *num_irqs* is the number of distinct interrupt types that the device sup= ports. + +This version of the protocol only supports PCI devices. Additional devices= may +be supported in future versions. + +``VFIO_USER_DEVICE_GET_REGION_INFO`` +------------------------------------ + +This command message is sent by the client to the server to query for +information about device regions. The VFIO region info structure is define= d in +```` (``struct vfio_region_info``). + +Request +^^^^^^^ + ++------------+--------+------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+ +| argsz | 0 | 4 | ++------------+--------+------------------------------+ +| flags | 4 | 4 | ++------------+--------+------------------------------+ +| index | 8 | 4 | ++------------+--------+------------------------------+ +| cap_offset | 12 | 4 | ++------------+--------+------------------------------+ +| size | 16 | 8 | ++------------+--------+------------------------------+ +| offset | 24 | 8 | ++------------+--------+------------------------------+ + +* *argsz* the maximum size of the reply payload +* *index* is the index of memory region being queried, it is the only field + that is required to be set in the command message. +* all other fields must be zero. + +Reply +^^^^^ + ++------------+--------+------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+ +| argsz | 0 | 4 | ++------------+--------+------------------------------+ +| flags | 4 | 4 | ++------------+--------+------------------------------+ +| | +-----+-----------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_REGION_INFO_FLAG_READ | | +| | +-----+-----------------------------+ | +| | | 1 | VFIO_REGION_INFO_FLAG_WRITE | | +| | +-----+-----------------------------+ | +| | | 2 | VFIO_REGION_INFO_FLAG_MMAP | | +| | +-----+-----------------------------+ | +| | | 3 | VFIO_REGION_INFO_FLAG_CAPS | | +| | +-----+-----------------------------+ | ++------------+--------+------------------------------+ ++------------+--------+------------------------------+ +| index | 8 | 4 | ++------------+--------+------------------------------+ +| cap_offset | 12 | 4 | ++------------+--------+------------------------------+ +| size | 16 | 8 | ++------------+--------+------------------------------+ +| offset | 24 | 8 | ++------------+--------+------------------------------+ + +* *argsz* is the size required for the full reply payload (region info str= ucture + plus the size of any region capabilities) +* *flags* are attributes of the region: + + * ``VFIO_REGION_INFO_FLAG_READ`` allows client read access to the region. + * ``VFIO_REGION_INFO_FLAG_WRITE`` allows client write access to the regi= on. + * ``VFIO_REGION_INFO_FLAG_MMAP`` specifies the client can mmap() the reg= ion. + When this flag is set, the reply will include a file descriptor in its + meta-data. On ``AF_UNIX`` sockets, the file descriptors will be passed= as + ``SCM_RIGHTS`` type ancillary data. + * ``VFIO_REGION_INFO_FLAG_CAPS`` indicates additional capabilities found= in the + reply. + +* *index* is the index of memory region being queried, it is the only field + that is required to be set in the command message. +* *cap_offset* describes where additional region capabilities can be found. + cap_offset is relative to the beginning of the VFIO region info structur= e. + The data structure it points is a VFIO cap header defined in + ````. +* *size* is the size of the region. +* *offset* is the offset that should be given to the mmap() system call for + regions with the MMAP attribute. It is also used as the base offset when + mapping a VFIO sparse mmap area, described below. + +VFIO region capabilities +"""""""""""""""""""""""" + +The VFIO region information can also include a capabilities list. This lis= t is +similar to a PCI capability list - each entry has a common header that +identifies a capability and where the next capability in the list can be f= ound. +The VFIO capability header format is defined in ```` (``stru= ct +vfio_info_cap_header``). + +VFIO cap header format +"""""""""""""""""""""" + ++---------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D+ +| id | 0 | 2 | ++---------+--------+------+ +| version | 2 | 2 | ++---------+--------+------+ +| next | 4 | 4 | ++---------+--------+------+ + +* *id* is the capability identity. +* *version* is a capability-specific version number. +* *next* specifies the offset of the next capability in the capability lis= t. It + is relative to the beginning of the VFIO region info structure. + +VFIO sparse mmap cap header +""""""""""""""""""""""""""" + ++------------------+----------------------------------+ +| Name | Value | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| id | VFIO_REGION_INFO_CAP_SPARSE_MMAP | ++------------------+----------------------------------+ +| version | 0x1 | ++------------------+----------------------------------+ +| next | | ++------------------+----------------------------------+ +| sparse mmap info | VFIO region info sparse mmap | ++------------------+----------------------------------+ + +This capability is defined when only a subrange of the region supports +direct access by the client via mmap(). The VFIO sparse mmap area is defin= ed in +```` (``struct vfio_region_sparse_mmap_area`` and ``struct +vfio_region_info_cap_sparse_mmap``). + +VFIO region info cap sparse mmap +"""""""""""""""""""""""""""""""" + ++----------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D= =3D+ +| nr_areas | 0 | 4 | ++----------+--------+------+ +| reserved | 4 | 4 | ++----------+--------+------+ +| offset | 8 | 8 | ++----------+--------+------+ +| size | 16 | 9 | ++----------+--------+------+ +| ... | | | ++----------+--------+------+ + +* *nr_areas* is the number of sparse mmap areas in the region. +* *offset* and size describe a single area that can be mapped by the clien= t. + There will be *nr_areas* pairs of offset and size. The offset will be ad= ded to + the base offset given in the ``VFIO_USER_DEVICE_GET_REGION_INFO`` to for= m the + offset argument of the subsequent mmap() call. + +The VFIO sparse mmap area is defined in ```` (``struct +vfio_region_info_cap_sparse_mmap``). + +VFIO region type cap header +""""""""""""""""""""""""""" + ++------------------+---------------------------+ +| Name | Value | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| id | VFIO_REGION_INFO_CAP_TYPE | ++------------------+---------------------------+ +| version | 0x1 | ++------------------+---------------------------+ +| next | | ++------------------+---------------------------+ +| region info type | VFIO region info type | ++------------------+---------------------------+ + +This capability is defined when a region is specific to the device. + +VFIO region info type cap +""""""""""""""""""""""""" + +The VFIO region info type is defined in ```` +(``struct vfio_region_info_cap_type``). + ++---------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D+ +| type | 0 | 4 | ++---------+--------+------+ +| subtype | 4 | 4 | ++---------+--------+------+ + +The only device-specific region type and subtype supported by vfio-user is +``VFIO_REGION_TYPE_MIGRATION`` (3) and ``VFIO_REGION_SUBTYPE_MIGRATION`` (= 1). + +``VFIO_USER_DEVICE_GET_REGION_IO_FDS`` +-------------------------------------- + +Clients can access regions via ``VFIO_USER_REGION_READ/WRITE`` or, if prov= ided, by +``mmap()`` of a file descriptor provided by the server. + +``VFIO_USER_DEVICE_GET_REGION_IO_FDS`` provides an alternative access mech= anism via +file descriptors. This is an optional feature intended for performance +improvements where an underlying sub-system (such as KVM) supports communi= cation +across such file descriptors to the vfio-user server, without needing to +round-trip through the client. + +The server returns an array of sub-regions for the requested region. Each +sub-region describes a span (offset and size) of a region, along with the +requested file descriptor notification mechanism to use. Each sub-region = in the +response message may choose to use a different method, as defined below. = The +two mechanisms supported in this specification are ioeventfds and ioregion= fds. + +The server in addition returns a file descriptor in the ancillary data; cl= ients +are expected to configure each sub-region's file descriptor with the reque= sted +notification method. For example, a client could configure KVM with the +requested ioeventfd via a ``KVM_IOEVENTFD`` ``ioctl()``. + +Request +^^^^^^^ + ++-------------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+------+ +| flags | 4 | 4 | ++-------------+--------+------+ +| index | 8 | 4 | ++-------------+--------+------+ +| count | 12 | 4 | ++-------------+--------+------+ + +* *argsz* the maximum size of the reply payload +* *index* is the index of memory region being queried +* all other fields must be zero + +The client must set ``flags`` to zero and specify the region being queried= in +the ``index``. + +Reply +^^^^^ + ++-------------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------------+--------+------+ +| flags | 4 | 4 | ++-------------+--------+------+ +| index | 8 | 4 | ++-------------+--------+------+ +| count | 12 | 4 | ++-------------+--------+------+ +| sub-regions | 16 | ... | ++-------------+--------+------+ + +* *argsz* is the size of the region IO FD info structure plus the + total size of the sub-region array. Thus, each array entry "i" is at off= set + i * ((argsz - 32) / count). Note that currently this is 40 bytes for bot= h IO + FD types, but this is not to be relied on. As elsewhere, this indicates = the + full reply payload size needed. +* *flags* must be zero +* *index* is the index of memory region being queried +* *count* is the number of sub-regions in the array +* *sub-regions* is the array of Sub-Region IO FD info structures + +The reply message will additionally include at least one file descriptor i= n the +ancillary data. Note that more than one sub-region may share the same file +descriptor. + +Note that it is the client's responsibility to verify the requested values= (for +example, that the requested offset does not exceed the region's bounds). + +Each sub-region given in the response has one of two possible structures, +depending whether *type* is ``VFIO_USER_IO_FD_TYPE_IOEVENTFD`` or +``VFIO_USER_IO_FD_TYPE_IOREGIONFD``: + +Sub-Region IO FD info format (ioeventfd) +"""""""""""""""""""""""""""""""""""""""" + ++-----------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D+ +| offset | 0 | 8 | ++-----------+--------+------+ +| size | 8 | 8 | ++-----------+--------+------+ +| fd_index | 16 | 4 | ++-----------+--------+------+ +| type | 20 | 4 | ++-----------+--------+------+ +| flags | 24 | 4 | ++-----------+--------+------+ +| padding | 28 | 4 | ++-----------+--------+------+ +| datamatch | 32 | 8 | ++-----------+--------+------+ + +* *offset* is the offset of the start of the sub-region within the region + requested ("physical address offset" for the region) +* *size* is the length of the sub-region. This may be zero if the access s= ize is + not relevant, which may allow for optimizations +* *fd_index* is the index in the ancillary data of the FD to use for ioeve= ntfd + notification; it may be shared. +* *type* is ``VFIO_USER_IO_FD_TYPE_IOEVENTFD`` +* *flags* is any of: + + * ``KVM_IOEVENTFD_FLAG_DATAMATCH`` + * ``KVM_IOEVENTFD_FLAG_PIO`` + * ``KVM_IOEVENTFD_FLAG_VIRTIO_CCW_NOTIFY`` (FIXME: makes sense?) + +* *datamatch* is the datamatch value if needed + +See https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt, *4.59 +KVM_IOEVENTFD* for further context on the ioeventfd-specific fields. + +Sub-Region IO FD info format (ioregionfd) +""""""""""""""""""""""""""""""""""""""""" + ++-----------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D+ +| offset | 0 | 8 | ++-----------+--------+------+ +| size | 8 | 8 | ++-----------+--------+------+ +| fd_index | 16 | 4 | ++-----------+--------+------+ +| type | 20 | 4 | ++-----------+--------+------+ +| flags | 24 | 4 | ++-----------+--------+------+ +| padding | 28 | 4 | ++-----------+--------+------+ +| user_data | 32 | 8 | ++-----------+--------+------+ + +* *offset* is the offset of the start of the sub-region within the region + requested ("physical address offset" for the region) +* *size* is the length of the sub-region. This may be zero if the access s= ize is + not relevant, which may allow for optimizations; ``KVM_IOREGION_POSTED_W= RITES`` + must be set in *flags* in this case +* *fd_index* is the index in the ancillary data of the FD to use for ioreg= ionfd + messages; it may be shared +* *type* is ``VFIO_USER_IO_FD_TYPE_IOREGIONFD`` +* *flags* is any of: + + * ``KVM_IOREGION_PIO`` + * ``KVM_IOREGION_POSTED_WRITES`` + +* *user_data* is an opaque value passed back to the server via a message o= n the + file descriptor + +For further information on the ioregionfd-specific fields, see: +https://lore.kernel.org/kvm/cover.1613828726.git.eafanasova@gmail.com/ + +(FIXME: update with final API docs.) + +``VFIO_USER_DEVICE_GET_IRQ_INFO`` +--------------------------------- + +This command message is sent by the client to the server to query for +information about device interrupt types. The VFIO IRQ info structure is +defined in ```` (``struct vfio_irq_info``). + +Request +^^^^^^^ + ++-------+--------+---------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------+--------+---------------------------+ +| flags | 4 | 4 | ++-------+--------+---------------------------+ +| | +-----+--------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_IRQ_INFO_EVENTFD | | +| | +-----+--------------------------+ | +| | | 1 | VFIO_IRQ_INFO_MASKABLE | | +| | +-----+--------------------------+ | +| | | 2 | VFIO_IRQ_INFO_AUTOMASKED | | +| | +-----+--------------------------+ | +| | | 3 | VFIO_IRQ_INFO_NORESIZE | | +| | +-----+--------------------------+ | ++-------+--------+---------------------------+ +| index | 8 | 4 | ++-------+--------+---------------------------+ +| count | 12 | 4 | ++-------+--------+---------------------------+ + +* *argsz* is the maximum size of the reply payload (16 bytes today) +* index is the index of IRQ type being queried (e.g. ``VFIO_PCI_MSIX_IRQ_I= NDEX``) +* all other fields must be zero + +Reply +^^^^^ + ++-------+--------+---------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------+--------+---------------------------+ +| flags | 4 | 4 | ++-------+--------+---------------------------+ +| | +-----+--------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_IRQ_INFO_EVENTFD | | +| | +-----+--------------------------+ | +| | | 1 | VFIO_IRQ_INFO_MASKABLE | | +| | +-----+--------------------------+ | +| | | 2 | VFIO_IRQ_INFO_AUTOMASKED | | +| | +-----+--------------------------+ | +| | | 3 | VFIO_IRQ_INFO_NORESIZE | | +| | +-----+--------------------------+ | ++-------+--------+---------------------------+ +| index | 8 | 4 | ++-------+--------+---------------------------+ +| count | 12 | 4 | ++-------+--------+---------------------------+ + +* *argsz* is the size required for the full reply payload (16 bytes today) +* *flags* defines IRQ attributes: + + * ``VFIO_IRQ_INFO_EVENTFD`` indicates the IRQ type can support server ev= entfd + signalling. + * ``VFIO_IRQ_INFO_MASKABLE`` indicates that the IRQ type supports the ``= MASK`` + and ``UNMASK`` actions in a ``VFIO_USER_DEVICE_SET_IRQS`` message. + * ``VFIO_IRQ_INFO_AUTOMASKED`` indicates the IRQ type masks itself after= being + triggered, and the client must send an ``UNMASK`` action to receive new + interrupts. + * ``VFIO_IRQ_INFO_NORESIZE`` indicates ``VFIO_USER_SET_IRQS`` operations= setup + interrupts as a set, and new sub-indexes cannot be enabled without dis= abling + the entire type. +* index is the index of IRQ type being queried +* count describes the number of interrupts of the queried type. + +``VFIO_USER_DEVICE_SET_IRQS`` +----------------------------- + +This command message is sent by the client to the server to set actions for +device interrupt types. The VFIO IRQ set structure is defined in +```` (``struct vfio_irq_set``). + +Request +^^^^^^^ + ++-------+--------+------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------+--------+------------------------------+ +| flags | 4 | 4 | ++-------+--------+------------------------------+ +| | +-----+-----------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_IRQ_SET_DATA_NONE | | +| | +-----+-----------------------------+ | +| | | 1 | VFIO_IRQ_SET_DATA_BOOL | | +| | +-----+-----------------------------+ | +| | | 2 | VFIO_IRQ_SET_DATA_EVENTFD | | +| | +-----+-----------------------------+ | +| | | 3 | VFIO_IRQ_SET_ACTION_MASK | | +| | +-----+-----------------------------+ | +| | | 4 | VFIO_IRQ_SET_ACTION_UNMASK | | +| | +-----+-----------------------------+ | +| | | 5 | VFIO_IRQ_SET_ACTION_TRIGGER | | +| | +-----+-----------------------------+ | ++-------+--------+------------------------------+ +| index | 8 | 4 | ++-------+--------+------------------------------+ +| start | 12 | 4 | ++-------+--------+------------------------------+ +| count | 16 | 4 | ++-------+--------+------------------------------+ +| data | 20 | variable | ++-------+--------+------------------------------+ + +* *argsz* is the size of the VFIO IRQ set request payload, including any *= data* + field. Note there is no reply payload, so this field differs from other + message types. +* *flags* defines the action performed on the interrupt range. The ``DATA`` + flags describe the data field sent in the message; the ``ACTION`` flags + describe the action to be performed. The flags are mutually exclusive for + both sets. + + * ``VFIO_IRQ_SET_DATA_NONE`` indicates there is no data field in the com= mand. + The action is performed unconditionally. + * ``VFIO_IRQ_SET_DATA_BOOL`` indicates the data field is an array of boo= lean + bytes. The action is performed if the corresponding boolean is true. + * ``VFIO_IRQ_SET_DATA_EVENTFD`` indicates an array of event file descrip= tors + was sent in the message meta-data. These descriptors will be signalled= when + the action defined by the action flags occurs. In ``AF_UNIX`` sockets,= the + descriptors are sent as ``SCM_RIGHTS`` type ancillary data. + If no file descriptors are provided, this de-assigns the specified + previously configured interrupts. + * ``VFIO_IRQ_SET_ACTION_MASK`` indicates a masking event. It can be used= with + ``VFIO_IRQ_SET_DATA_BOOL`` or ``VFIO_IRQ_SET_DATA_NONE`` to mask an in= terrupt, + or with ``VFIO_IRQ_SET_DATA_EVENTFD`` to generate an event when the gu= est masks + the interrupt. + * ``VFIO_IRQ_SET_ACTION_UNMASK`` indicates an unmasking event. It can be= used + with ``VFIO_IRQ_SET_DATA_BOOL`` or ``VFIO_IRQ_SET_DATA_NONE`` to unmas= k an + interrupt, or with ``VFIO_IRQ_SET_DATA_EVENTFD`` to generate an event = when the + guest unmasks the interrupt. + * ``VFIO_IRQ_SET_ACTION_TRIGGER`` indicates a triggering event. It can b= e used + with ``VFIO_IRQ_SET_DATA_BOOL`` or ``VFIO_IRQ_SET_DATA_NONE`` to trigg= er an + interrupt, or with ``VFIO_IRQ_SET_DATA_EVENTFD`` to generate an event = when the + server triggers the interrupt. + +* *index* is the index of IRQ type being setup. +* *start* is the start of the sub-index being set. +* *count* describes the number of sub-indexes being set. As a special case= , a + count (and start) of 0, with data flags of ``VFIO_IRQ_SET_DATA_NONE`` di= sables + all interrupts of the index. +* *data* is an optional field included when the + ``VFIO_IRQ_SET_DATA_BOOL`` flag is present. It contains an array of bool= eans + that specify whether the action is to be performed on the corresponding + index. It's used when the action is only performed on a subset of the ra= nge + specified. + +Not all interrupt types support every combination of data and action flags. +The client must know the capabilities of the device and IRQ index before it +sends a ``VFIO_USER_DEVICE_SET_IRQ`` message. + +In typical operation, a specific IRQ may operate as follows: + +1. The client sends a ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_EVENTFD|VFIO_IRQ_SET_ACTION_TRIGGER)`` alo= ng + with an eventfd. This associates the IRQ with a particular eventfd on t= he + server side. + +#. The client may send a ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_EVENTFD|VFIO_IRQ_SET_ACTION_MASK/UNMASK)``= along + with another eventfd. This associates the given eventfd with the + mask/unmask state on the server side. + +#. The server may trigger the IRQ by writing 1 to the eventfd. + +#. The server may mask/unmask an IRQ which will write 1 to the correspondi= ng + mask/unmask eventfd, if there is one. + +5. A client may trigger a device IRQ itself, by sending a + ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_NONE/BOOL|VFIO_IRQ_SET_ACTION_TRIGGER)``. + +6. A client may mask or unmask the IRQ, by sending a + ``VFIO_USER_DEVICE_SET_IRQ`` message with + ``flags=3D(VFIO_IRQ_SET_DATA_NONE/BOOL|VFIO_IRQ_SET_ACTION_MASK/UNMASK)= ``. + +Reply +^^^^^ + +There is no payload in the reply. + +.. _Read and Write Operations: + +Note that all of these operations must be supported by the client and/or s= erver, +even if the corresponding memory or device region has been shared as mappa= ble. + +The ``count`` field must not exceed the value of ``max_data_xfer_size`` of= the +peer, for both reads and writes. + +``VFIO_USER_REGION_READ`` +------------------------- + +If a device region is not mappable, it's not directly accessible by the cl= ient +via ``mmap()`` of the underlying file descriptor. In this case, a client c= an +read from a device region with this message. + +Request +^^^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ + +* *offset* into the region being accessed. +* *region* is the index of the region being accessed. +* *count* is the size of the data to be transferred. + +Reply +^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ +| data | 16 | variable | ++--------+--------+----------+ + +* *offset* into the region accessed. +* *region* is the index of the region accessed. +* *count* is the size of the data transferred. +* *data* is the data that was read from the device region. + +``VFIO_USER_REGION_WRITE`` +-------------------------- + +If a device region is not mappable, it's not directly accessible by the cl= ient +via mmap() of the underlying fd. In this case, a client can write to a dev= ice +region with this message. + +Request +^^^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ +| data | 16 | variable | ++--------+--------+----------+ + +* *offset* into the region being accessed. +* *region* is the index of the region being accessed. +* *count* is the size of the data to be transferred. +* *data* is the data to write + +Reply +^^^^^ + ++--------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D+ +| offset | 0 | 8 | ++--------+--------+----------+ +| region | 8 | 4 | ++--------+--------+----------+ +| count | 12 | 4 | ++--------+--------+----------+ + +* *offset* into the region accessed. +* *region* is the index of the region accessed. +* *count* is the size of the data transferred. + +``VFIO_USER_DMA_READ`` +----------------------- + +If the client has not shared mappable memory, the server can use this mess= age to +read from guest memory. + +Request +^^^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 8 | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. This address = must have + been previously exported to the server with a ``VFIO_USER_DMA_MAP`` mess= age. +* *count* is the size of the data to be transferred. + +Reply +^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 8 | ++---------+--------+----------+ +| data | 16 | variable | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. +* *count* is the size of the data transferred. +* *data* is the data read. + +``VFIO_USER_DMA_WRITE`` +----------------------- + +If the client has not shared mappable memory, the server can use this mess= age to +write to guest memory. + +Request +^^^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 8 | ++---------+--------+----------+ +| data | 16 | variable | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. This address = must have + been previously exported to the server with a ``VFIO_USER_DMA_MAP`` mess= age. +* *count* is the size of the data to be transferred. +* *data* is the data to write + +Reply +^^^^^ + ++---------+--------+----------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| address | 0 | 8 | ++---------+--------+----------+ +| count | 8 | 4 | ++---------+--------+----------+ + +* *address* is the client DMA memory address being accessed. +* *count* is the size of the data transferred. + +``VFIO_USER_DEVICE_RESET`` +-------------------------- + +This command message is sent from the client to the server to reset the de= vice. +Neither the request or reply have a payload. + +``VFIO_USER_DIRTY_PAGES`` +------------------------- + +This command is analogous to ``VFIO_IOMMU_DIRTY_PAGES``. It is sent by the= client +to the server in order to control logging of dirty pages, usually during a= live +migration. + +Dirty page tracking is optional for server implementation; clients should = not +rely on it. + +Request +^^^^^^^ + ++-------+--------+-----------------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++-------+--------+-----------------------------------------+ +| flags | 4 | 4 | ++-------+--------+-----------------------------------------+ +| | +-----+----------------------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= + | +| | | 0 | VFIO_IOMMU_DIRTY_PAGES_FLAG_START | | +| | +-----+----------------------------------------+ | +| | | 1 | VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP | | +| | +-----+----------------------------------------+ | +| | | 2 | VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP | | +| | +-----+----------------------------------------+ | ++-------+--------+-----------------------------------------+ + +* *argsz* is the size of the VFIO dirty bitmap info structure for + ``START/STOP``; and for ``GET_BITMAP``, the maximum size of the reply pa= yload + +* *flags* defines the action to be performed by the server: + + * ``VFIO_IOMMU_DIRTY_PAGES_FLAG_START`` instructs the server to start lo= gging + pages it dirties. Logging continues until explicitly disabled by + ``VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP``. + + * ``VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP`` instructs the server to stop logg= ing + dirty pages. + + * ``VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP`` requests the server to retu= rn + the dirty bitmap for a specific IOVA range. The IOVA range is specifie= d by + a "VFIO Bitmap Range" structure, which must immediately follow this + "VFIO Dirty Pages" structure. See `VFIO Bitmap Range Format`_. + This operation is only valid if logging of dirty pages has been previo= usly + started. + + These flags are mutually exclusive with each other. + +This part of the request is analogous to VFIO's ``struct +vfio_iommu_type1_dirty_bitmap``. + +.. _VFIO Bitmap Range Format: + +VFIO Bitmap Range Format +"""""""""""""""""""""""" + ++--------+--------+------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D+ +| iova | 0 | 8 | ++--------+--------+------+ +| size | 8 | 8 | ++--------+--------+------+ +| bitmap | 16 | 24 | ++--------+--------+------+ + +* *iova* is the IOVA offset + +* *size* is the size of the IOVA region + +* *bitmap* is the VFIO Bitmap explained in `VFIO Bitmap`_. + +This part of the request is analogous to VFIO's ``struct +vfio_iommu_type1_dirty_bitmap_get``. + +Reply +^^^^^ + +For ``VFIO_IOMMU_DIRTY_PAGES_FLAG_START`` or +``VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP``, there is no reply payload. + +For ``VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP``, the reply payload is as fo= llows: + ++--------------+--------+-----------------------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ +| argsz | 0 | 4 | ++--------------+--------+-----------------------------------------+ +| flags | 4 | 4 | ++--------------+--------+-----------------------------------------+ +| | +-----+----------------------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+ | +| | | 2 | VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP | | +| | +-----+----------------------------------------+ | ++--------------+--------+-----------------------------------------+ +| bitmap range | 8 | 40 | ++--------------+--------+-----------------------------------------+ +| bitmap | 48 | variable | ++--------------+--------+-----------------------------------------+ + +* *argsz* is the size required for the full reply payload (dirty pages str= ucture + + bitmap range structure + actual bitmap) +* *flags* is ``VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP`` +* *bitmap range* is the same bitmap range struct provided in the request, = as + defined in `VFIO Bitmap Range Format`_. +* *bitmap* is the actual dirty pages bitmap corresponding to the range req= uest + +VFIO Device Migration Info +-------------------------- + +A device may contain a migration region (of type +``VFIO_REGION_TYPE_MIGRATION``). The beginning of the region must contain +``struct vfio_device_migration_info``, defined in ````. This +subregion is accessed like any other part of a standard vfio-user region +using ``VFIO_USER_REGION_READ``/``VFIO_USER_REGION_WRITE``. + ++---------------+--------+-----------------------------+ +| Name | Offset | Size | ++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D+= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+ +| device_state | 0 | 4 | ++---------------+--------+-----------------------------+ +| | +-----+----------------------------+ | +| | | Bit | Definition | | +| | +=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | +| | | 0 | VFIO_DEVICE_STATE_RUNNING | | +| | +-----+----------------------------+ | +| | | 1 | VFIO_DEVICE_STATE_SAVING | | +| | +-----+----------------------------+ | +| | | 2 | VFIO_DEVICE_STATE_RESUMING | | +| | +-----+----------------------------+ | ++---------------+--------+-----------------------------+ +| reserved | 4 | 4 | ++---------------+--------+-----------------------------+ +| pending_bytes | 8 | 8 | ++---------------+--------+-----------------------------+ +| data_offset | 16 | 8 | ++---------------+--------+-----------------------------+ +| data_size | 24 | 8 | ++---------------+--------+-----------------------------+ + +* *device_state* defines the state of the device: + + The client initiates device state transition by writing the intended sta= te. + The server must respond only after it has successfully transitioned to t= he new + state. If an error occurs then the server must respond to the + ``VFIO_USER_REGION_WRITE`` operation with the Error field set accordingl= y and + must remain at the previous state, or in case of internal error it must + transition to the error state, defined as + ``VFIO_DEVICE_STATE_RESUMING | VFIO_DEVICE_STATE_SAVING``. The client mu= st + re-read the device state in order to determine it afresh. + + The following device states are defined: + + +-----------+---------+----------+-----------------------------------+ + | _RESUMING | _SAVING | _RUNNING | Description | + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ + | 0 | 0 | 0 | Device is stopped. | + +-----------+---------+----------+-----------------------------------+ + | 0 | 0 | 1 | Device is running, default state. | + +-----------+---------+----------+-----------------------------------+ + | 0 | 1 | 0 | Stop-and-copy state | + +-----------+---------+----------+-----------------------------------+ + | 0 | 1 | 1 | Pre-copy state | + +-----------+---------+----------+-----------------------------------+ + | 1 | 0 | 0 | Resuming | + +-----------+---------+----------+-----------------------------------+ + | 1 | 0 | 1 | Invalid state | + +-----------+---------+----------+-----------------------------------+ + | 1 | 1 | 0 | Error state | + +-----------+---------+----------+-----------------------------------+ + | 1 | 1 | 1 | Invalid state | + +-----------+---------+----------+-----------------------------------+ + + Valid state transitions are shown in the following table: + + +-------------------------+---------+---------+---------------+---------= -+----------+ + | |darr| From / To |rarr| | Stopped | Running | Stop-and-copy | Pre-copy= | Resuming | + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D+ + | Stopped | \- | 1 | 0 | 0 = | 0 | + +-------------------------+---------+---------+---------------+---------= -+----------+ + | Running | 1 | \- | 1 | 1 = | 1 | + +-------------------------+---------+---------+---------------+---------= -+----------+ + | Stop-and-copy | 1 | 1 | \- | 0 = | 0 | + +-------------------------+---------+---------+---------------+---------= -+----------+ + | Pre-copy | 0 | 0 | 1 | \- = | 0 | + +-------------------------+---------+---------+---------------+---------= -+----------+ + | Resuming | 0 | 1 | 0 | 0 = | \- | + +-------------------------+---------+---------+---------------+---------= -+----------+ + + A device is migrated to the destination as follows: + + * The source client transitions the device state from the running state = to + the pre-copy state. This transition is optional for the client but mus= t be + supported by the server. The source server starts sending device state= data + to the source client through the migration region while the device is + running. + + * The source client transitions the device state from the running state = or the + pre-copy state to the stop-and-copy state. The source server stops the + device, saves device state and sends it to the source client through t= he + migration region. + + The source client is responsible for sending the migration data to the + destination client. + + A device is resumed on the destination as follows: + + * The destination client transitions the device state from the running s= tate + to the resuming state. The destination server uses the device state da= ta + received through the migration region to resume the device. + + * The destination client provides saved device state to the destination + server and then transitions the device to back to the running state. + +* *reserved* This field is reserved and any access to it must be ignored b= y the + server. + +* *pending_bytes* Remaining bytes to be migrated by the server. This field= is + read only. + +* *data_offset* Offset in the migration region where the client must: + + * read from, during the pre-copy or stop-and-copy state, or + + * write to, during the resuming state. + + This field is read only. + +* *data_size* Contains the size, in bytes, of the amount of data copied to: + + * the source migration region by the source server during the pre-copy or + stop-and copy state, or + + * the destination migration region by the destination client during the + resuming state. + +Device-specific data must be stored at any position after +``struct vfio_device_migration_info``. Note that the migration region can = be +memory mappable, even partially. In practise, only the migration data port= ion +can be memory mapped. + +The client processes device state data during the pre-copy and the +stop-and-copy state in the following iterative manner: + + 1. The client reads ``pending_bytes`` to mark a new iteration. Repeated = reads + of this field is an idempotent operation. If there are no migration d= ata + to be consumed then the next step depends on the current device state: + + * pre-copy: the client must try again. + + * stop-and-copy: this procedure can end and the device can now start + resuming on the destination. + + 2. The client reads ``data_offset``; at this point the server must make + available a portion of migration data at this offset to be read by the + client, which must happen *before* completing the read operation. The + amount of data to be read must be stored in the ``data_size`` field, = which + the client reads next. + + 3. The client reads ``data_size`` to determine the amount of migration d= ata + available. + + 4. The client reads and processes the migration data. + + 5. Go to step 1. + +Note that the client can transition the device from the pre-copy state to = the +stop-and-copy state at any time; ``pending_bytes`` does not need to become= zero. + +The client initializes the device state on the destination by setting the +device state in the resuming state and writing the migration data to the +destination migration region at ``data_offset`` offset. The client can wri= te the +source migration data in an iterative manner and the server must consume t= his +data before completing each write operation, updating the ``data_offset`` = field. +The server must apply the source migration data on the device resume state= . The +client must write data on the same order and transaction size as read. + +If an error occurs then the server must fail the read or write operation. = It is +an implementation detail of the client how to handle errors. + +Appendices +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Unused VFIO ``ioctl()`` commands +-------------------------------- + +The following VFIO commands do not have an equivalent vfio-user command: + +* ``VFIO_GET_API_VERSION`` +* ``VFIO_CHECK_EXTENSION`` +* ``VFIO_SET_IOMMU`` +* ``VFIO_GROUP_GET_STATUS`` +* ``VFIO_GROUP_SET_CONTAINER`` +* ``VFIO_GROUP_UNSET_CONTAINER`` +* ``VFIO_GROUP_GET_DEVICE_FD`` +* ``VFIO_IOMMU_GET_INFO`` + +However, once support for live migration for VFIO devices is finalized some +of the above commands may have to be handled by the client in their +corresponding vfio-user form. This will be addressed in a future protocol +version. + +VFIO groups and containers +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The current VFIO implementation includes group and container idioms that +describe how a device relates to the host IOMMU. In the vfio-user +implementation, the IOMMU is implemented in SW by the client, and is not +visible to the server. The simplest idea would be that the client put each +device into its own group and container. + +Backend Program Conventions +--------------------------- + +vfio-user backend program conventions are based on the vhost-user ones. + +* The backend program must not daemonize itself. +* No assumptions must be made as to what access the backend program has on= the + system. +* File descriptors 0, 1 and 2 must exist, must have regular + stdin/stdout/stderr semantics, and can be redirected. +* The backend program must honor the SIGTERM signal. +* The backend program must accept the following commands line options: + + * ``--socket-path=3DPATH``: path to UNIX domain socket, + * ``--fd=3DFDNUM``: file descriptor for UNIX domain socket, incompatible= with + ``--socket-path`` +* The backend program must be accompanied with a JSON file stored under + ``/usr/share/vfio-user``. + +TODO add schema similar to docs/interop/vhost-user.json. diff --git a/MAINTAINERS b/MAINTAINERS index 694973ed23..d838b9e3f2 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1879,6 +1879,12 @@ F: hw/vfio/ap.c F: docs/system/s390x/vfio-ap.rst L: qemu-s390x@nongnu.org =20 +vfio-user +M: John G Johnson +M: Thanos Makatos +S: Supported +F: docs/devel/vfio-user.rst + vhost M: Michael S. Tsirkin S: Supported --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132738; cv=pass; d=zohomail.com; s=zohoarc; b=OFh3t2lo//EGJcfojatb+PGGFnRR24HvBBGn3hXjY7Ru0t5Fk+ntWA/NYlrb75RF1SXYVnTSpsjSbYPBK8rPgdaUAwimOJpE1Ewt5SGJ0etyNvUUHpSStrt5CG+IGRiZDdmpMHPVNOYKl5zcK4ZJgg4OaYUK1fLmZyGwqwq4xow= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132738; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=KcjsSD+Wq2I/FtUzkAJgb2ONemFI/sZTtoqwRT3vav4=; b=YPoRYGmGQ0a2v5lyzjE6+3R2681kXvBZlVz1WMEPH0jOtvrixmB0hFTc9ow72VM9eRE5eAJcd/3UXc8GTuNI4fmRysGPkArfGF9NSfaiHwC7W+PSphF7rsL1NrogOQjP4as1ShLgn/dzVIdhEnDUpXKyzQTGkVe+epCG8a9Kj38= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629132738285727.5258487347977; Mon, 16 Aug 2021 09:52:18 -0700 (PDT) Received: from localhost ([::1]:47284 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfqT-0008SI-6e for importer@patchew.org; Mon, 16 Aug 2021 12:52:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37134) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhe-0000oX-HM for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:10 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:35300) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhY-0007vK-PS for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:10 -0400 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGaE6R000648; Mon, 16 Aug 2021 16:43:01 GMT Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3af1q9anur-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:00 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeuOL025726; Mon, 16 Aug 2021 16:42:59 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2102.outbound.protection.outlook.com [104.47.55.102]) by userp3030.oracle.com with ESMTP id 3ae2xxgrw2-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:42:59 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by BYAPR10MB2565.namprd10.prod.outlook.com (2603:10b6:a02:b2::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.19; Mon, 16 Aug 2021 16:42:56 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:42:56 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:42:56 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=KcjsSD+Wq2I/FtUzkAJgb2ONemFI/sZTtoqwRT3vav4=; b=dPGpK6XIokMzcf0R9LwRAKU13q+OIMxwgtmdxWmWunZcVbKgyf2FuSvvSJU/H8/g0QHj esXX2mrBsESo5gN9JpJQ9zMAoKfen7ka4zsR5bZT8efd9kOsVWyAQYXT4jPBFoD+Se2U HQCkk1SAQp8aA0w8P7eF1RuUmvLkiVhM3Pnb23RxE3Zlf0gYLIM7grOEt00CEcxh9KsI Tro/Ms4GayIgBYfXe6rS4glbAIMCd69Pb0OQxBBEPVsr6yIv/TJEYSEXt6IAidlsUag4 twv+Qv7nWmCFm6ymVauaUR2k2dcg7mgbdaXYwKG47o4CjwY0YxTX6Ual+DQdw9bKtBw8 YQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=KcjsSD+Wq2I/FtUzkAJgb2ONemFI/sZTtoqwRT3vav4=; b=nsZaWmXHTLpzRIwBQxqofngbdtBEZGLpnq6O8erG2q8WS/qE3gZoDjaorl0+w5k9vUgm Kx/cvwCTeXih/pSuScKTBvougMGTrlo3b/6IB5qsOJ72Ns3s23ZdZIboc88rQ1nIDgdk KDvaZ7dHzJxJStS+sEhAx+47bjQqZRY4p4VSun1NyYX33zBrwSaEJ1KkP/OIZqkD8Ny6 koc+fltVD4/HiQtcSLDIvvktyN71hXJUQDduNVrqmRTLkgCvdc8gT9FRXQR/hab7nb1z tQVRTTCaTH08Ejwye7GcuU4d0sxqUCk2GEa2HeDIrnXLplz5TYhtr2CMNRBdjE1EHxfh Fg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V9r9krC7R9rJV/fzdW7KWs89mH0Te+T/GEoP3DL2VYEQnsBDqhNhnPHl8k4YQXdURRM2CjmYs0ie8QIb7oABKG85DQhiVOd278/bXOt7vBRzGG5y+0jLRXI07gA7nmp0t+n09hdH9RvZ2oE6ngNtngGLdNyX7BZhDwkGKfXmDcceylkd2pCTgwNzZ8FRn3caoV3aPKP/lyzAtqW33rsxS7D/UqVxRjvCIxHcSwQKCe9Gm38LAJN/FdNHnr7SKFcHCFJp/ysFvbbaUq/ysupv8c7kkxWs+BzQH/L/2WQp5LUiezLpqCctdTFTuxlV7RQ3ld2QPCaWi+WvjWdJB06dGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KcjsSD+Wq2I/FtUzkAJgb2ONemFI/sZTtoqwRT3vav4=; b=g5qHaViMWXmVjm8vzu9nEYhtDtiTDgqNcKTvVnsmoKYzGu/j6angUTE/L7uKU9Uh1KCEKphHvXhs0eTG4YYNszErbuLOZSzPbHVORg2c1xztu9dvmDBDhSEghwFlSmeuMBNRqgkX3OiG77p5vkL+5vIltgR1nOO3gTtgYhXu6wCAp9HCbs87pPsA4E9yd8H6Ipb522ff11A2nYZqpeAj/HlAIxBb2++OfmuPk3uKHT56J8bvrmhYLXDb20KZDmaWwG63lWLq1z6kH3xKZBz/sFwSb5Rix0DB/q0H8Kat43IfaIY7JoMvO8NN1lswzn3jfJjdscuQFMUW957xDWWU+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KcjsSD+Wq2I/FtUzkAJgb2ONemFI/sZTtoqwRT3vav4=; b=Oc5/HdCVzsA15t/jtN52XapI4IDSZRkWl3la7TbMiVq/rU1HNNbuC3wGxmbT1lFTIE6YX+GMx5QdhziyTo0pTv7J5Ix4VBvGtDW3tsSVQo8a0iqWh5lG8RmQSiPdLkTI8o1VQuBdnmAx7lzv7PI2fl+PIdG38yvNYlRelPeV42g= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 02/16] vfio-user: add VFIO base abstract class Date: Mon, 16 Aug 2021 09:42:35 -0700 Message-Id: <3610c39c2e12132e01a23752d161e7209be12f38.1629131628.git.elena.ufimtseva@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 55adc678-9e03-4070-29e9-08d960d4e63b X-MS-TrafficTypeDiagnostic: BYAPR10MB2565: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2582; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KlpfkIvGMDuobs4JPzmk0BI0gflBMBXcMl+8qGDzXMr1ybiw/yYCcLlVCnmSdg6SWUdZWubD0Qcnl2ct4qX6wU5PpzMjZ8RK3xSom+zXkd3RT9bVRqyAIogYtpodS15wLgkxmp6DsGpAv8Ox8SF1ahuMEh2T7qvnvtMFdpOEUxQV3/7bbykOGzmeKZWf5PT0K0vJ9CYPYPBkF9dOgVCevj1RZReHTBy8Fn3CUmcSkUyr/bcBnh8L2uQMreuUEA822A4BVlHhDygu4/rN6d/ESKfBgZVYlSjmJAAjRug77zr2GuKJnx+78V01IfM+ivBQUF7wKa8KwH2Dt6cwuvpmWs2GW7vFQf7wfE/o6uWZUUs62AeWNNVG5mmgWRZUKHCWk3XIzjhyuE1tmqeYQ/jB5lRSA68NdniUMUr7ALd+N6hF1/FLPZJCh8aBo23SgrynVvpO2KAEwrsYHq4Sgm9uj58JOpzeSEF/lj6Ze5NN5uG04vc/+ETpLDk7JyVLfiGPasacUYgi0xmApvUxfZ5HTbjOHEKqIuNvGTnL9IM32PLBoif+yqSzjwYwk0LQXA/xoWwE6nguEULw4n6gzmYX/XdsmH423c3ZWaYkXa4MOJWJZv05zKNzFHgSPfQIE7UmZToueNtJvJoWvVablhGuUA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(6486002)(316002)(38100700002)(44832011)(2616005)(6512007)(2906002)(6916009)(52116002)(5660300002)(508600001)(6506007)(66946007)(4326008)(66476007)(66556008)(6666004)(186003)(86362001)(8676002)(8936002)(36756003)(83380400001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?xNhU9V3yqzyrpVb1nWSqZ6jDxw5WuQWkdLIOZN85g0PfdMB1CBprqKo3WaT2?= =?us-ascii?Q?8MDZGOS6kDSOLaq94H6Zry5VMyT0suVgnJgbONZo5e3NYz54aHwnLyjiQ0zT?= =?us-ascii?Q?2+VVTySbZhiJsNAZFXF1r2ufpGLpejhNY/mWSY0NfpRTFoN/+pLq8GFr4m5o?= =?us-ascii?Q?jFpOfI1W7SdGRX1RYKeV6oZ48/9H2bKn8oC+ajJaD98AdwDi2PTqaR2DWHMt?= =?us-ascii?Q?ilW6gwVyswd9rPWojYPoX1DwPudduO2k1ZFL/KKaqvLEAHZWZfvkxDt/zkpB?= =?us-ascii?Q?wTJ/LPKk3R1NyfYvVkUYCUUYgYwplBV1ANtOs6wE0nqTT3Z6GVuQlTFhyue7?= =?us-ascii?Q?uZdxMFomoGCJ2bhpQy3bJcUVG7lLAfCF5EmLg+aBV3d42/uwr2yJ31b4Exmq?= =?us-ascii?Q?3qktM7JVFUJ+Dw2P7NTEWoxGD6fz/8akSYbRkUxTbTNq02XP0l4CZd+lspQr?= =?us-ascii?Q?HxLtjZZpgPH1erus6hKZN0b5TnStZbNdoNHnFi7hA6OqHY1OWgxDxPfn225s?= =?us-ascii?Q?CGj/mv3eXIWFvfJgNEnkKwSd1XoP2a1Kf/vS788wQyf1vdn56VL+QfeCnsOg?= =?us-ascii?Q?FyjLilXCjxZMx9OBiyAszwY1r+ww44DJfQ9DVYmR0wMhLJgwjuqNjdis9TTB?= =?us-ascii?Q?A99ceSyxHeSRYyDZ04utkOO0yih7QessphwmE2ixohCW59ZBv5u6VrXDVTA/?= =?us-ascii?Q?inn/Idc/GiqYY8zM6tZ216QQh04uPJVmLSb2xr4NTgdeBF1gEGgf2ZBS2Et7?= =?us-ascii?Q?20sySWv1kkF2WOT0eWt/GroQ/eHWVXdTy6OtSwyBH+nTR7xiPSdtWQ7298XV?= =?us-ascii?Q?sHOUlQ+78AT9rb2EcVMlS8k66L8HBSSzPTB06/YNmUuak0xeComHs/fLC641?= =?us-ascii?Q?T0F9vv7feKEZzqFmmJCqbQ9PVWto9F5hv+yxO/eCBh8DCgzuB4uH70vC/aOr?= =?us-ascii?Q?z7X9OZy8LQ70WhDIr9zXkibsBUCzr2jvd2aPqaU9e5laM61/Io8400WUXcuP?= =?us-ascii?Q?Cnx0CAvMHVw2tO1ovEAKi3Bc7YPF54D6Q2ey4z2AgKHhXnRt6x5LRXcRpLBi?= =?us-ascii?Q?Qi5o+geXlSatEOSezT/tfDLylZX7XE2HVvJWY4USSBB9ne/VCsof0wxlZfAz?= =?us-ascii?Q?9KJLadnJ/8yI1hiQdt9oWm3U5vTh5td6N9WNQ9sAyhjv4H1XDzubT4rLg4sN?= =?us-ascii?Q?VgUENND24oQOVzJbZOml3BdGT62HeDzPDE1SJGzNmox7RJgwAw9bM3AuuJFy?= =?us-ascii?Q?8bmX+CHfGGN72Lif+0VurwAWUgS/P0rLa4Pg29s7/rr2uJkhkY3rEbNP6O77?= =?us-ascii?Q?CgPqII34mcxrpT2ffUUMcZyLDIsm8dtoIuwQ1WWYvv+eyKQxVfnn7CaXTatL?= =?us-ascii?Q?nhqTmmwi3E8YBlRYckHLiIbI7xZr?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 55adc678-9e03-4070-29e9-08d960d4e63b X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:42:56.2965 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ijze4Or8Mxo1wwhhnLr4rrCUWqSce8kFcahQ4dJ5Xm/rCjwx1+dbGFC5gr9d75KQVVpfQ1fBKv7DTXmRkT8w5r4nqC66GZi592CCg2QGsPw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2565 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 adultscore=0 suspectscore=0 phishscore=0 mlxlogscore=999 malwarescore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-GUID: DKMMYzoQrwgKi5jFKEfBfJv1BKRz83l3 X-Proofpoint-ORIG-GUID: DKMMYzoQrwgKi5jFKEfBfJv1BKRz83l3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132740134100001 Content-Type: text/plain; charset="utf-8" From: John Johnson Add an abstract base class both the kernel driver and user socket implementations can use to share code. Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman --- hw/vfio/pci.h | 16 +++++++++++-- hw/vfio/pci.c | 63 ++++++++++++++++++++++++++++++++------------------- 2 files changed, 54 insertions(+), 25 deletions(-) diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index 64777516d1..bbc78aaeb3 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -114,8 +114,13 @@ typedef struct VFIOMSIXInfo { unsigned long *pending; } VFIOMSIXInfo; =20 -#define TYPE_VFIO_PCI "vfio-pci" -OBJECT_DECLARE_SIMPLE_TYPE(VFIOPCIDevice, VFIO_PCI) +/* + * TYPE_VFIO_PCI_BASE is an abstract type used to share code + * between VFIO implementations that use a kernel driver + * with those that use user sockets. + */ +#define TYPE_VFIO_PCI_BASE "vfio-pci-base" +OBJECT_DECLARE_SIMPLE_TYPE(VFIOPCIDevice, VFIO_PCI_BASE) =20 struct VFIOPCIDevice { PCIDevice pdev; @@ -175,6 +180,13 @@ struct VFIOPCIDevice { Notifier irqchip_change_notifier; }; =20 +#define TYPE_VFIO_PCI "vfio-pci" +OBJECT_DECLARE_SIMPLE_TYPE(VFIOKernPCIDevice, VFIO_PCI) + +struct VFIOKernPCIDevice { + VFIOPCIDevice device; +}; + /* Use uin32_t for vendor & device so PCI_ANY_ID expands and cannot match = hw */ static inline bool vfio_pci_is(VFIOPCIDevice *vdev, uint32_t vendor, uint3= 2_t device) { diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index e1ea1d8a23..bea95efc33 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -231,7 +231,7 @@ static void vfio_intx_update(VFIOPCIDevice *vdev, PCIIN= TxRoute *route) =20 static void vfio_intx_routing_notifier(PCIDevice *pdev) { - VFIOPCIDevice *vdev =3D VFIO_PCI(pdev); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); PCIINTxRoute route; =20 if (vdev->interrupt !=3D VFIO_INT_INTx) { @@ -457,7 +457,7 @@ static void vfio_update_kvm_msi_virq(VFIOMSIVector *vec= tor, MSIMessage msg, static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr, MSIMessage *msg, IOHandler *handler) { - VFIOPCIDevice *vdev =3D VFIO_PCI(pdev); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); VFIOMSIVector *vector; int ret; =20 @@ -542,7 +542,7 @@ static int vfio_msix_vector_use(PCIDevice *pdev, =20 static void vfio_msix_vector_release(PCIDevice *pdev, unsigned int nr) { - VFIOPCIDevice *vdev =3D VFIO_PCI(pdev); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); VFIOMSIVector *vector =3D &vdev->msi_vectors[nr]; =20 trace_vfio_msix_vector_release(vdev->vbasedev.name, nr); @@ -1063,7 +1063,7 @@ static const MemoryRegionOps vfio_vga_ops =3D { */ static void vfio_sub_page_bar_update_mapping(PCIDevice *pdev, int bar) { - VFIOPCIDevice *vdev =3D VFIO_PCI(pdev); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); VFIORegion *region =3D &vdev->bars[bar].region; MemoryRegion *mmap_mr, *region_mr, *base_mr; PCIIORegion *r; @@ -1109,7 +1109,7 @@ static void vfio_sub_page_bar_update_mapping(PCIDevic= e *pdev, int bar) */ uint32_t vfio_pci_read_config(PCIDevice *pdev, uint32_t addr, int len) { - VFIOPCIDevice *vdev =3D VFIO_PCI(pdev); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); uint32_t emu_bits =3D 0, emu_val =3D 0, phys_val =3D 0, val; =20 memcpy(&emu_bits, vdev->emulated_config_bits + addr, len); @@ -1142,7 +1142,7 @@ uint32_t vfio_pci_read_config(PCIDevice *pdev, uint32= _t addr, int len) void vfio_pci_write_config(PCIDevice *pdev, uint32_t addr, uint32_t val, int len) { - VFIOPCIDevice *vdev =3D VFIO_PCI(pdev); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); uint32_t val_le =3D cpu_to_le32(val); =20 trace_vfio_pci_write_config(vdev->vbasedev.name, addr, val, len); @@ -2782,7 +2782,7 @@ static void vfio_unregister_req_notifier(VFIOPCIDevic= e *vdev) =20 static void vfio_realize(PCIDevice *pdev, Error **errp) { - VFIOPCIDevice *vdev =3D VFIO_PCI(pdev); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); VFIODevice *vbasedev_iter; VFIOGroup *group; char *tmp, *subsys, group_path[PATH_MAX], *group_name; @@ -3105,7 +3105,7 @@ error: =20 static void vfio_instance_finalize(Object *obj) { - VFIOPCIDevice *vdev =3D VFIO_PCI(obj); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); VFIOGroup *group =3D vdev->vbasedev.group; =20 vfio_display_finalize(vdev); @@ -3125,7 +3125,7 @@ static void vfio_instance_finalize(Object *obj) =20 static void vfio_exitfn(PCIDevice *pdev) { - VFIOPCIDevice *vdev =3D VFIO_PCI(pdev); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); =20 vfio_unregister_req_notifier(vdev); vfio_unregister_err_notifier(vdev); @@ -3144,7 +3144,7 @@ static void vfio_exitfn(PCIDevice *pdev) =20 static void vfio_pci_reset(DeviceState *dev) { - VFIOPCIDevice *vdev =3D VFIO_PCI(dev); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(dev); =20 trace_vfio_pci_reset(vdev->vbasedev.name); =20 @@ -3184,7 +3184,7 @@ post_reset: static void vfio_instance_init(Object *obj) { PCIDevice *pci_dev =3D PCI_DEVICE(obj); - VFIOPCIDevice *vdev =3D VFIO_PCI(obj); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); =20 device_add_bootindex_property(obj, &vdev->bootindex, "bootindex", NULL, @@ -3253,28 +3253,24 @@ static Property vfio_pci_dev_properties[] =3D { DEFINE_PROP_END_OF_LIST(), }; =20 -static void vfio_pci_dev_class_init(ObjectClass *klass, void *data) +static void vfio_pci_base_dev_class_init(ObjectClass *klass, void *data) { DeviceClass *dc =3D DEVICE_CLASS(klass); PCIDeviceClass *pdc =3D PCI_DEVICE_CLASS(klass); =20 - dc->reset =3D vfio_pci_reset; - device_class_set_props(dc, vfio_pci_dev_properties); - dc->desc =3D "VFIO-based PCI device assignment"; + dc->desc =3D "VFIO PCI base device"; set_bit(DEVICE_CATEGORY_MISC, dc->categories); - pdc->realize =3D vfio_realize; pdc->exit =3D vfio_exitfn; pdc->config_read =3D vfio_pci_read_config; pdc->config_write =3D vfio_pci_write_config; } =20 -static const TypeInfo vfio_pci_dev_info =3D { - .name =3D TYPE_VFIO_PCI, +static const TypeInfo vfio_pci_base_dev_info =3D { + .name =3D TYPE_VFIO_PCI_BASE, .parent =3D TYPE_PCI_DEVICE, - .instance_size =3D sizeof(VFIOPCIDevice), - .class_init =3D vfio_pci_dev_class_init, - .instance_init =3D vfio_instance_init, - .instance_finalize =3D vfio_instance_finalize, + .instance_size =3D 0, + .abstract =3D true, + .class_init =3D vfio_pci_base_dev_class_init, .interfaces =3D (InterfaceInfo[]) { { INTERFACE_PCIE_DEVICE }, { INTERFACE_CONVENTIONAL_PCI_DEVICE }, @@ -3282,6 +3278,26 @@ static const TypeInfo vfio_pci_dev_info =3D { }, }; =20 +static void vfio_pci_dev_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc =3D DEVICE_CLASS(klass); + PCIDeviceClass *pdc =3D PCI_DEVICE_CLASS(klass); + + dc->reset =3D vfio_pci_reset; + device_class_set_props(dc, vfio_pci_dev_properties); + dc->desc =3D "VFIO-based PCI device assignment"; + pdc->realize =3D vfio_realize; +} + +static const TypeInfo vfio_pci_dev_info =3D { + .name =3D TYPE_VFIO_PCI, + .parent =3D TYPE_VFIO_PCI_BASE, + .instance_size =3D sizeof(VFIOKernPCIDevice), + .class_init =3D vfio_pci_dev_class_init, + .instance_init =3D vfio_instance_init, + .instance_finalize =3D vfio_instance_finalize, +}; + static Property vfio_pci_dev_nohotplug_properties[] =3D { DEFINE_PROP_BOOL("ramfb", VFIOPCIDevice, enable_ramfb, false), DEFINE_PROP_END_OF_LIST(), @@ -3298,12 +3314,13 @@ static void vfio_pci_nohotplug_dev_class_init(Objec= tClass *klass, void *data) static const TypeInfo vfio_pci_nohotplug_dev_info =3D { .name =3D TYPE_VFIO_PCI_NOHOTPLUG, .parent =3D TYPE_VFIO_PCI, - .instance_size =3D sizeof(VFIOPCIDevice), + .instance_size =3D sizeof(VFIOKernPCIDevice), .class_init =3D vfio_pci_nohotplug_dev_class_init, }; =20 static void register_vfio_pci_dev_type(void) { + type_register_static(&vfio_pci_base_dev_info); type_register_static(&vfio_pci_dev_info); type_register_static(&vfio_pci_nohotplug_dev_info); } --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132287; cv=pass; d=zohomail.com; s=zohoarc; b=YmOaqLZ2DTZYerr3BMJluV3V60KVHxFCHYjDEOFsHItbnE6YiJkCos40yUQy+XkAEKlwPNrwOKqSKAYuf5SxTaN5tv0CS5WPNzchZUUEi78O+r9/TXmT9ypQPoQMuIu4MCvXpi4eF6jyzpq+S5zpmySX4rLzZWccHxNe7fCl55g= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132287; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=HU+WVZn20enFNMZWdVa5bfgLHRj1V4MI+wkyDyMte0Y=; b=LnOaBtaKAE52OgbgCLp9xcpvjP1jkudC0S855+j5vzTPsTnzlOD+ijx7mP+Y3SyqEt0ARImihBp1wWaFkTmV0J6gyS/vL8JzFZWXvKttGkPabte6cCwMIGPzaM5BL7sFAEZisAVoYbebRGREBXKQps/InbnzkhBEIlUhHiQiW3g= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629132287224800.5602576025458; Mon, 16 Aug 2021 09:44:47 -0700 (PDT) Received: from localhost ([::1]:55206 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfjB-0003LT-CR for importer@patchew.org; Mon, 16 Aug 2021 12:44:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37010) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhZ-0000dk-Uy for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:06 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:32166) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhX-0007uU-FY for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:05 -0400 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGaAhs021058; Mon, 16 Aug 2021 16:43:00 GMT Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3afgmb9jyr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:00 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeijA113651; Mon, 16 Aug 2021 16:42:59 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2103.outbound.protection.outlook.com [104.47.58.103]) by aserp3020.oracle.com with ESMTP id 3ae5n609gk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:42:59 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by BYAPR10MB3221.namprd10.prod.outlook.com (2603:10b6:a03:14f::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.19; Mon, 16 Aug 2021 16:42:57 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:42:57 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:42:56 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=HU+WVZn20enFNMZWdVa5bfgLHRj1V4MI+wkyDyMte0Y=; b=yCNoOFoHUrEDmZmMKgVPVyRYB7bJQbrP9vPA1wimH5vaDAKglVD+XS3sllmkPbc4kQqL c0nnBx3u3NcS+HQeJ8qsQJLMn2tAHsAjOBUfKj+IEKO3RuuVXjoj1qLG3rVJJcNVg7jQ xbM6PXjtKiukIbRziJYxJmfq3VEpfZKovygCZLo9jTWRtPO3hzi9dj11wkzR2hB69VsW G6ByizbphwVVuY+/h1Ga47/cKs2gw79rOjmeReLVVwy1BwIdmn9uJyZOTUsF+PgeITjv qNJvkZ/pKiYEIkFR6RrxmOZoYFhEL/TX1RUrB3ntN8NVTdBdgmkGEeeQlnJso3NjH/sb Jg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=HU+WVZn20enFNMZWdVa5bfgLHRj1V4MI+wkyDyMte0Y=; b=bHAesTW/SIunEFvYfZFZdBhmTqHBVlqSczVw6CD5kP3kzpX9g7dueF/B2TEZMPXxaqy2 zVl5SsnS4eutCb5o7Lffb+oqkHwzm9BflNZqw8Bapim1ix7t0bzZR0X7VDRpFG0BylV7 LlhMqLzPyUm/zfN8m5QmtqvPosZtCzeK8g6CtUllf+8p0WrfMyTyN8rMwor2G2RQ2Qha lH+HRF/OnSzAo+XmJPpyAg2GDGR0DKc3VDQzS9Zs7aYj+XhG++EUuPJ8/DmFC8IWvbdU l9OlfUqSvj4kgFnssEWAGTm+KsMxuqKJjYY3BovJcw4zEq0kDRGqjTnv12wJhgBhH557 +g== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h6HQETKFi+hK0bFK6FSJMDqPJgfNwhdWvTfWtwbTNMBXW4m+3+sNBtS6pgneFFzVVgh2s1zNPzPm9nN5GgeWtIVN4PbP0DZBlcqU5rE+e3nNUkW1KiVVdM8p/Xu+X90DLYV91ZDnYeqND6GQUuqJ0ok7h/+8XG63ukkNlciV+xK+KYvTZ97RNf4QqMRCOs+xJ2FDAQRnwQnJZBlUjwzsFBwerkMvuzy5YDpsGzffadgDFkLAl4ADS73dpKXZdRpAa3BMe4rzxINvKUHgUVZlWpheCcJz+3ala2ohap69tAfoFh0SV/BtnjKpuKwwxh73X5WdziqD2o/iTvCHrhtiYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HU+WVZn20enFNMZWdVa5bfgLHRj1V4MI+wkyDyMte0Y=; b=SiZRLKjQKHo95wv2cjhdM1Q7fkR4q8TIaJDz1q7RcKT4iKTFPVReFb4UxLoGXZSNonQKKxuQvQ3q8H9GJbDpzaoZyTD2oon5wYRsKI/8SlDLDtldwHJ+aoB/FV70IkIU2hS81qXKNhaP8qGHh6XSj8bsmD5B89XYWagWzkOJaYCTX3CquirJ3oYYjKNYP8VdryO4MlXZEXE/VMC1bGT/wCt++zLzdre/TXk0qhaZhvTG5aalx9VCR7EI7CK1Zj+SqGtCw41nHHKjkL94oVyzWdJZmdSlyeP90K0AThvA0Ojb7RwYxsMuO9h7PMfz0khQlidi+1JHnZWINWDXjP9LTQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HU+WVZn20enFNMZWdVa5bfgLHRj1V4MI+wkyDyMte0Y=; b=epXmIH/ONiqfxCY8uH0RoM3jxsUGBXdzvWM2GWvNHYsc1o5JysY+Yo47nQ1J0lazdITxN2h3EBqXj+k/HzzS8///B1SUi0jP9s6Yg1x2LrrglWm3/PH9OaLsu4uq4qf4mdfdnXTmgWEtg88J7evkNTa7VHK5C6TKHTFj1UeMNNs= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 03/16] vfio-user: Define type vfio_user_pci_dev_info Date: Mon, 16 Aug 2021 09:42:36 -0700 Message-Id: <4153fb93ac5c353911a1321efc0a02f9f402acd5.1629131628.git.elena.ufimtseva@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8a8465e7-6639-43ec-0e86-08d960d4e68d X-MS-TrafficTypeDiagnostic: BYAPR10MB3221: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1186; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: bSu5LiiNnUFhnoeE8UIHZXNP4EhFo9rvdcy+jYKA1At4/YGlf2xWnWKMliwXH9A1Nl5XdVIJja3wKRKo+/x0jC2z/xlC4W5d1kh+Be/t0hhENyFmhOg+frhf9xNtAUvM9Lf6MvrXsaBBm7lFs7D78V8rBjPIaDncbnuDUzgoZGkRmXD4MUarUvpmKyaEz9Dwn+0ASGCTnJtgF7h4ThL/B04X8jgoMJjdsCGKaZ4OB9aWC00cbuJOKdhxBOZvPVz2i8PoiPq62KlSnB/Q/yRN8WPOGYQc+NaZ08JIey0obaYyF/LnDtZsP3uUAKsimNBl7imLvWQtPGiOrOV5GaKYvMdhWd4DayYXHfUCDS+fpXOwEVP4MbDN8F3WkB8hbk8bdF+uOZvVjC+ywBibN/79Qi9Zah+Nidyahr/fnIWyzrjfA5oPwGLZ9UIU7XJA5hpMYJ7cdLEjx2pv4MZGDPeoLMM7nFaHzq4wIhoztkRVAyRUikV2GnyN5WWzRW25y8HHOfEuGAO4vIrkZvrFnd13FifmDCMxLKNXqBvzhT7cCI+VvsUSamjk+knVZP8jB/dShndthRx9LSOZOFrSOPuMSUd1oUUV/bnhGGGEG065xTufCYVLD2Lnc7ia0py7J0JnPxxobsekjH8aHfp7QvWbSA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(39860400002)(376002)(346002)(366004)(396003)(136003)(2616005)(6916009)(66556008)(66476007)(66946007)(52116002)(478600001)(36756003)(5660300002)(6506007)(6666004)(6512007)(186003)(316002)(6486002)(86362001)(44832011)(8936002)(8676002)(38100700002)(4326008)(2906002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?I13HZtuLLRZzi89PHSCxCjtRdsE/gRzrrKKxXCaKLPpV7nlACyNts0Y2FKym?= =?us-ascii?Q?8gS0FfSOs50O/I58ADcqUfqguMi+90SqpfR3G5y85KoRQ72tNYgULSypNKma?= =?us-ascii?Q?mxGtVaXKZB9Vjc9q4aavROaQmntI4XCMp8VV+wN6IG2E+p2ev/ye2VA2Y4UX?= =?us-ascii?Q?UtbFJE3kMK8XNUntIiASURBmmKxPjJAoRiLpbQBPXqUeqkgOpA8e4k3JKrmD?= =?us-ascii?Q?F+waRKqgs5052vaTdBwTH3u+kD5j5WPzaf85Y5m23DQ60GMr6DEie000nAhY?= =?us-ascii?Q?GFJJKQi7BUjdIzfADeWlijnkd5XyIjKKJKzSJG9Gt+JGnUQVs4gbTSG8tGqX?= =?us-ascii?Q?sSY5VB0jNRXXBjUG7IgO8NPzS/3vkmvhlonPHDoIq4jxapnE2nWV0tnlJxkc?= =?us-ascii?Q?JbI0W8Vj2Y2GF2d/F28IdY24Dp+YE9+cUncALDWg8/CfHZDuCKblyaC4C+NH?= =?us-ascii?Q?uGAoB6OKI/4pu3NbSHfbr8iCA5r4N8ADzxcQV0Z1fxcI555+xPfiWvz+FPjT?= =?us-ascii?Q?xUsV8CRuGcW3brGvuFSrRk7gAZawexZtTSXDpYX9SrE5G1/Fyb3xLeQSngNN?= =?us-ascii?Q?hOfWlHEsnWyhQ6mduQf/UQPmvChLTQZUEMNgdAkS+z8rFvTKiI2Qv4eryZob?= =?us-ascii?Q?TA4FK5/VYsHYhhbpmEuT+wxWiZQkjDO3/ECVVHD4HfyHErwExXnggAOEC0Os?= =?us-ascii?Q?sdpHILPzlNAihySZ+fIqsqqIl6cw0J6Sp+l85esQYSI+ABPM3eBbv5PaVMeo?= =?us-ascii?Q?h07knqPSOnFj69DuvddMBUN3EggGvS+l4BlykaPsmJuMtLKOZRXgXwDCvdtZ?= =?us-ascii?Q?rjCWlrypQ/UFDwnuYb8NVAh+eZC6IJ59QxUhaymIRFcidNxkjlBG+szMZtNC?= =?us-ascii?Q?zc97wEJQ9gpFXx+xThiX1Mi2G5I7zdTleSdRO5/y6v4G+Qw6WmxzupJ5CQEZ?= =?us-ascii?Q?fBtON5qUrFjc9YOsBiyhl1jqvofOAxIi0mAL/jCTv656A7+KVdSxKYuobiQc?= =?us-ascii?Q?du2WoEKlSxf25C0DDE/yoQDmg9X3hqS7rSLrNYfqpuTJ6ez+A9oRfcBPhglo?= =?us-ascii?Q?Tm2EaSDxI6vUb6vtqIWgKX9f86rdqOkUZXDMboZ+CALOhHmZNAH2LXDJHeXM?= =?us-ascii?Q?6rn8U2y+ajXQZFmXoHhqlDq/VLYIwTHv9i0JdXI+NpijVZ8P0zh0DdCrOC+M?= =?us-ascii?Q?ndACPElstC/AJhIP7xX9QejSVdd0C7xo3Za44sCqStKsADxvOKQ/fJvtoZG9?= =?us-ascii?Q?nLqdQsLft2HJqVB+pjTl4pNU/TBb3GVYGRsedQ/EyZo3sg5X3zzTQ4ubSbG6?= =?us-ascii?Q?LJFDONt5X3kT5tqsS56CUFAPWB6DDjOyOAVLP/dQ+zH8bZTNjINbs9PlXutU?= =?us-ascii?Q?BnZnqRLUKG/s01Dn8j6f+rAs3Yjy?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8a8465e7-6639-43ec-0e86-08d960d4e68d X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:42:56.8292 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1ZvnPMdum1+EnrJXOHbPYsiSt+sF8+o1MaNC9H4Zau3y/M8lji49pfJ0OlIfh2y9lvNuM6j63tIdaD/km4edzdOsm3SmCKHU6+OY2oF/wCc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3221 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-ORIG-GUID: YJicBTfLWVRwpKWVZL02A1fwAXrmG_d9 X-Proofpoint-GUID: YJicBTfLWVRwpKWVZL02A1fwAXrmG_d9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132288399100001 Content-Type: text/plain; charset="utf-8" From: John Johnson New class for vfio-user with its class and instance constructors and destructors, and its pci ops. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/pci.h | 9 ++++++ hw/vfio/pci.c | 86 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 95 insertions(+) diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index bbc78aaeb3..08ac6475a4 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -187,6 +187,15 @@ struct VFIOKernPCIDevice { VFIOPCIDevice device; }; =20 +#define TYPE_VFIO_USER_PCI "vfio-user-pci" +OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserPCIDevice, VFIO_USER_PCI) + +struct VFIOUserPCIDevice { + VFIOPCIDevice device; + char *sock_name; + bool secure_dma; /* disable shared mem for DMA */ +}; + /* Use uin32_t for vendor & device so PCI_ANY_ID expands and cannot match = hw */ static inline bool vfio_pci_is(VFIOPCIDevice *vdev, uint32_t vendor, uint3= 2_t device) { diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index bea95efc33..d642aafb7f 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3326,3 +3326,89 @@ static void register_vfio_pci_dev_type(void) } =20 type_init(register_vfio_pci_dev_type) + + +/* + * vfio-user routines. + */ + +/* + * Emulated devices don't use host hot reset + */ +static int vfio_user_pci_no_reset(VFIODevice *vbasedev) +{ + error_printf("vfio-user - no hot reset\n"); + return 0; +} + +static void vfio_user_pci_not_needed(VFIODevice *vbasedev) +{ + vbasedev->needs_reset =3D false; +} + +static VFIODeviceOps vfio_user_pci_ops =3D { + .vfio_compute_needs_reset =3D vfio_user_pci_not_needed, + .vfio_hot_reset_multi =3D vfio_user_pci_no_reset, + .vfio_eoi =3D vfio_intx_eoi, + .vfio_get_object =3D vfio_pci_get_object, + .vfio_save_config =3D vfio_pci_save_config, + .vfio_load_config =3D vfio_pci_load_config, +}; + +static void vfio_user_pci_realize(PCIDevice *pdev, Error **errp) +{ + ERRP_GUARD(); + VFIOUserPCIDevice *udev =3D VFIO_USER_PCI(pdev); + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); + VFIODevice *vbasedev =3D &vdev->vbasedev; + + if (!udev->sock_name) { + error_setg(errp, "No socket specified"); + error_append_hint(errp, "Use -device vfio-user-pci,socket=3D= \n"); + return; + } + + vbasedev->name =3D g_strdup_printf("VFIO user <%s>", udev->sock_name); + vbasedev->dev =3D DEVICE(vdev); + vbasedev->fd =3D -1; + vbasedev->type =3D VFIO_DEVICE_TYPE_PCI; + vbasedev->no_mmap =3D false; + vbasedev->ops =3D &vfio_user_pci_ops; + +} + +static void vfio_user_instance_finalize(Object *obj) +{ +} + +static Property vfio_user_pci_dev_properties[] =3D { + DEFINE_PROP_STRING("socket", VFIOUserPCIDevice, sock_name), + DEFINE_PROP_BOOL("secure-dma", VFIOUserPCIDevice, secure_dma, false), + DEFINE_PROP_END_OF_LIST(), +}; + +static void vfio_user_pci_dev_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc =3D DEVICE_CLASS(klass); + PCIDeviceClass *pdc =3D PCI_DEVICE_CLASS(klass); + + device_class_set_props(dc, vfio_user_pci_dev_properties); + dc->desc =3D "VFIO over socket PCI device assignment"; + pdc->realize =3D vfio_user_pci_realize; +} + +static const TypeInfo vfio_user_pci_dev_info =3D { + .name =3D TYPE_VFIO_USER_PCI, + .parent =3D TYPE_VFIO_PCI_BASE, + .instance_size =3D sizeof(VFIOUserPCIDevice), + .class_init =3D vfio_user_pci_dev_class_init, + .instance_init =3D vfio_instance_init, + .instance_finalize =3D vfio_user_instance_finalize, +}; + +static void register_vfio_user_dev_type(void) +{ + type_register_static(&vfio_user_pci_dev_info); +} + +type_init(register_vfio_user_dev_type) --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132306; cv=pass; d=zohomail.com; s=zohoarc; b=nAt+Iy5SBUv6Pie+gxo0enGyOAkor7+IYRHIJB6AAEBUSmxhlGKPYpj4UbhIaXP2q1f+HbYnVBti7QNx8kETbMtZARG54uKEEf2kD6nSHxWcuT/AnS2nGrrUec9fpMhRZRdbQMB0wDCN+YjNnOER5faMCkqXzlZ7/8xtMKyzXUo= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132306; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=bHVoFr/SonDv8WbdTf1Kb6rfL+P7v0JCZowJuVtb15A=; b=Uzm/pyxxBNhtrkUiToWsAWdrLz1XvYflfbzDHsaCZ0i9SSHuEhzyVKSOTSGW4LfdID96IaGqs0qufd8/qIs7Q2wJBNxZa7MCG8K5kaptLTNVfZLjIBv2ICqHor7Z5WIu06USWx+a33CbYF69a6SJuybLbLZrjZQxy306jJTH350= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629132306417337.38434865293596; Mon, 16 Aug 2021 09:45:06 -0700 (PDT) Received: from localhost ([::1]:56752 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfjV-0004Le-6r for importer@patchew.org; Mon, 16 Aug 2021 12:45:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37012) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfha-0000e9-6A for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:06 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:33086) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhX-0007uY-9M for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:05 -0400 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGZhsd008102; Mon, 16 Aug 2021 16:43:01 GMT Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3afgpghjpk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:00 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeijB113651; Mon, 16 Aug 2021 16:42:59 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2103.outbound.protection.outlook.com [104.47.58.103]) by aserp3020.oracle.com with ESMTP id 3ae5n609gk-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:42:59 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by BYAPR10MB3221.namprd10.prod.outlook.com (2603:10b6:a03:14f::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.19; Mon, 16 Aug 2021 16:42:57 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:42:57 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:42:56 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=bHVoFr/SonDv8WbdTf1Kb6rfL+P7v0JCZowJuVtb15A=; b=gtiAxBX69Qnv4giWZxCS6aaWmiLr7T6vYivggc8IZtCfIKxwiQDcdDU+EWeGkQRiWX7Z gg8BIwIJZSbSKxjnBPrjneA1xI0poO9FnDatfwGDYo11kZKZaV1P8/klDwzXbZtsTKSi HAFMqS9eIG9hlnHpGdZkJuU/0QeJPOz8fYfhftQeXSJ1XlJrvW2a5AeGBsim1NfvGxEF XUxeGcEReC4Urt3O/bG3K8VuFE5ifNtGZCVr7uk0yGTOBACEtmeHfdoZQVQVIuGcW6+/ adQZVaP8nWlW/iOF5SZpvfLlRtIJiABrxjwrDR/wNEeNJgAHiwQwwedQGsVKKC4SCk/e Ng== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=corp-2020-01-29; bh=bHVoFr/SonDv8WbdTf1Kb6rfL+P7v0JCZowJuVtb15A=; b=mO2koF+1IP0aYiGSYpMiFZGz952RczCF0B5NtBJA0Ckm/4gPG/BWN3azHq7oTR4/lVAq 0Xo/dXqm6uGkYj7HdyrlN4AeiLIKGErlyjDXpuFTo6FJrTFXgevoP/2VPmoMzqOjivXW 4GUR6PD/TFKxKXKc+aEiY2jDpqcMMvaWivxqmQcOWsSysITRqSei8DaIzPsBpBSqNGN+ eLRrOHEF29Sx99J7OMflDzDkhIeAS4WpWNe/qY5pOhDXbb9Y1Xi1Gi2uMbgYYgtCUNKn wgOOyQHFEGQ6nMCFpl7hchTu39zBv/h+alYtHdur2ouVKlynzSfeG2I/X7b3FiUi5PX2 ow== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T+HjycCWk8RntAqr2Mp1v+Ji6iSqcYdL+7NKUol8Ype6J/1jZMDxVp3iGA0mel7pKK5LZW01xKSSLK9dtmdL1aZm8L8ucnrBoZzW0wkW520X2BP875x5QEUOZoi9hvZZ/npQHj6CuK9ChuAqSho1j7BlRnxQfaWedbOzKcuUv8WLXX2BwR2hNN982D7b/Ps/2UkZkuiyftMRvjpKkAWSVu27/8AF+xNNlr1BmFQG+JYze36a/3zQxg8ws7pYEVNvhG/We+euA9xWLvWjxYD7Ue5cTo/tylb30tC11PXp3ZYmKryFuCR+Lp7WrIBNLbdbE+WenpsnjO+s86gH22F3RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bHVoFr/SonDv8WbdTf1Kb6rfL+P7v0JCZowJuVtb15A=; b=Y2nIDJzOVS1N99blE6LMxVL1WbVsfIArd/glBRbDVZrbmSVEwaN4lm+/lOj1AKggo9/JGwUiFPsFFCA49c2rETNPsitd4Hh6MIP9L0oTzqBWpdexNjoL2ydun0WqfTgYNmvJ+QoPo3FyfIww/8+0iWhQK8wokN+5kzG2bnhMqhUyg5Jf7gufvnqNKGSJh2qU6Did+vaSp+P1MilAhsT0MLodIbeTch5M29FFHUTUV8NQmBAX+Jj59IsI+U2KuqVMXsgPFg78UiSI2zJ7CC6gDB5IdnPyTdFPAaos4+OdxVUvJQzp4Uo1jRb8SBZkhZ2zwj2MfUF0/fZ6P+QaHGE7bw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bHVoFr/SonDv8WbdTf1Kb6rfL+P7v0JCZowJuVtb15A=; b=upVZ1GvXrRG8CaeRaTqRz/ATXP+yxK/PQDYqR3jq3fMJXjqRg8UHHHiD7gszOvKP8pHqpAtweErlBJpXAgxhri8u9+TZbSaKgWL597p9TVq6vLThvSmXImMrHqcdhhRs4vjW4vhIvF3iTP4/HVfnn70f7D0dpdUiENiriVYDOiA= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server Date: Mon, 16 Aug 2021 09:42:37 -0700 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 062b21a8-2c59-4fbd-98ed-08d960d4e6d2 X-MS-TrafficTypeDiagnostic: BYAPR10MB3221: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:792; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kd/DSYq1O/rdNA3fAF3eTogp5v5GwSuTO1Ro8DZlA8z+SpvsviuEm+S373J4pSEWNYlgcI1krSpNjqAZ/iqjV59qRqgpAWUgtmfFsLMzGL4KUYSc3CBY7KRGQEC7W3C3SRkJlkET5BnEaGZNd1t9S3qa+7zQk/AlVCYu0CUpPwk1MKu5apXI89eNt8ed6CBIlybgaXa751MnyxMiHpa863+a8L/xwgq8vbisZ2o9OW8xtQd5amEj9oEx+xrVTmD3G164cFK9fYb0Cfr1tcwYyIK98AV3dxjA7uUXpJZAbuZLvhoetSD0bsfIG/B8+aqBPvSz7vzbN0B0a6h1sSAeGF4DcyXX7c1LQUs6NEs6nrPpyWuVh5IHB9+FyGZIlXuvmBgZqyfOxuucM5ZmVlBfL1mCCa3Lo9o7LLYXApAHJN8oBOsXXNNqBGfRukhY8PFSJl8/J9iJeJHgjpA94gbe7h7JLuIl56/bm2qL1YaH5W4oyAmNPVOq50QVPsPsQyVTnoTIdhT+s2XVW9g67LGPCnvwJKeEDYkefn1IMnVpY+q1EjdGR7s+89S8kPgUbbqxSQQeRaKX53Qygfsb07hL/OArBk74rMjW9UKSfjwzj4s0IWqw+/eum5A3ecbnt0kMKKqtbpvoim+S4e+IQdIwoQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(39860400002)(376002)(346002)(366004)(396003)(136003)(2616005)(6916009)(66556008)(66476007)(66946007)(52116002)(83380400001)(478600001)(36756003)(5660300002)(6506007)(6666004)(6512007)(186003)(316002)(6486002)(86362001)(44832011)(8936002)(8676002)(38100700002)(4326008)(2906002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SlBWdTMxeTZtZ0FQbzhJcVNoY0FRSlpGNkp4Q3BoLzAxNFQya20rSkdjb0pY?= =?utf-8?B?RUJrbkRNem5PMEtoNzFlWFpUS3VJUmZZOGpRdGhGK1ErVVA4MU1oN1c2b0pI?= =?utf-8?B?cjhGV2NKOCtPWWdkanM3aDJvaUF2dzIxdG0wNzJpZ05CUHQ4akRzNldpZ1ho?= =?utf-8?B?dmVTRlpPTXJoa0psc3lyZ0owSGxwS2ZkbnFvcmlkM1RDYzgvcG05TXJCcTRr?= =?utf-8?B?OEU2OWExSFZWTlUzWVpKd3hKb0xyY2tuS0tKbkUvYmdJLy9VbHJGWm4ycUQ3?= =?utf-8?B?WkZLOHdwTWZoVEZSWjNQaEhFcGxQRm96QkVMM3VseVFNMy8zdUNzMlJYOUcr?= =?utf-8?B?QjNRVHFqTjhKYStxNHl5QmJERmEvQ1IzYlg2UUt6SFlyczM4cnZ2ajBIeUsy?= =?utf-8?B?RDNJbVJWVFc1ZnFiN3lnckZRdmFTM2xoM3RVYlZTUG5Ub3hyNGwzUjd3QzR2?= =?utf-8?B?U2ZTcW5HS0Q4YWpGVmNoM0o4dVk0YThHdGJ4bEw1eHZ2UXlqd0pZblVDUStY?= =?utf-8?B?dFNpZ3EzNFV5TVpvSkFhNlltK2doRUd4ZUx6OWpLbmRZbDhPZXBNSU9aTU53?= =?utf-8?B?RFhnNmo2K0ZMZmU4S3pKOGhMalUvT0NES3VzKzZuWDUvdTlzVGVXQnZvblU1?= =?utf-8?B?TTEyb1BKRjUwWHV6eWRCSjRtdVlJNExCYisxMVhPNEo0YzFFUTYyYTJ6c0J4?= =?utf-8?B?M1dlZkZ0SnUvaTJGMHRJRVJHZGZQRTlvU1BmMlNDSGFXSldJcnNpdTM4cGxo?= =?utf-8?B?V081ZVpjaHQ0NHJ5YmU5SGZiODhMYTFva2NVczN6QTRiWVN0am1zL0twRG9i?= =?utf-8?B?Tkp1cEFEMFhKUlB1aXh5Qk1YZndzWXkweXNzQUcxT0pZVXRHNXJhYVgzWTBa?= =?utf-8?B?S1M1SXpNcFFDc1FPTHZrdERBMnR3MzBFSzZZc01MOWFyMG9uMTdIakdNT1ZT?= =?utf-8?B?ZEFzYXdyZWR4Wm1KbU5NSlovRG94VFBRMnlZRDhGQk1qTUhES3EwZjYzbzBI?= =?utf-8?B?eHQwT0Y4VzV1NUxGYU9CZWF4REp2L0JLT05jVnp3RFMrQUpwd1hrZnpvMTR3?= =?utf-8?B?OXpkRDVRS0JCSUszR01jcXRGZEZXN0VmUG96aHNGcWpZeWRRWTdxeldrTHdq?= =?utf-8?B?T1d3K1AyY29pbUFxZDhBdlp1SXhKMStlUUZKQ0g1M09KZk1LV2oxeFptY3Jo?= =?utf-8?B?YWJDZUpWb3BZSlJJb09SSVhDa25pa3V5Mml5TlYzTjFLRXVRWjV5SWJ5eGFn?= =?utf-8?B?Mkx2YWRXWVMrNS9BVHlKMEppMDdUMVQ4S0J1dFNydldPS2I1c1c4UFluT3F5?= =?utf-8?B?M09OMkp2c0FaTkc1dUc5eUhIcHNWQk1vWThIRForZnBkY2xwdm1oZnVRWG5U?= =?utf-8?B?RDN2TzNPUUxsUlpaZFl4VGV1YkJkaFo0YmxMdDFHNXJqenp1RklCVWN1UGEy?= =?utf-8?B?aFZxaUxzc2lPQnllemxxb0MyWGc2K0F3ZEtuRVNPY25BRGZIVGQ2NWc4QXF2?= =?utf-8?B?SFU0LzJIS1VuSWJhWmFEdVRVTzRna1EzTkptaW5GZldXMTVjWE1uM1J0RUNL?= =?utf-8?B?cXBqb0VhNVNFZUlsS0t6WlFkN2RBU3FPYVVTbkhCWGt3UjZlWi9RK09ZMUFo?= =?utf-8?B?WVg5TTZyV3Vvc0FvQ2d5YkJONEJzVE05R1ozdXdNcWtsZEV0WnRhL0Ivekps?= =?utf-8?B?bjNPR0RIL2tWV2s2dWVRVzd1TitndDlQWkZKTGZZUHhmR2xYSitMR1RPWEp5?= =?utf-8?B?ekxTL2NhWWt3ODJVQTZSUzZMekttUG9IYm1XY0JHaVdXaHU5VkxBamRHSGFN?= =?utf-8?B?MDJKZmFOMllzMlZSc05SNGwvWGxwS1FjUDU1QzhGcUJ3c1MyeFF4bHgzYklZ?= =?utf-8?Q?NcGiYF4MPf02u?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 062b21a8-2c59-4fbd-98ed-08d960d4e6d2 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:42:57.2769 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BGMtzRAw/I8joIGmNr9COxHgHFmXAOCrPVRr9KJnNiU4z9f2DraZyjCUtmNfp0rx3Y9YgOlcbh0hyKCEElUoAYE3KVhR6x/OJiQCy1mcpyw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3221 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-GUID: oIFuVEKSr94_acTkya3rXAa8bHXfNF4M X-Proofpoint-ORIG-GUID: oIFuVEKSr94_acTkya3rXAa8bHXfNF4M Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132308253100001 From: John Johnson Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman --- hw/vfio/user.h | 66 ++++++++++++++ include/hw/vfio/vfio-common.h | 2 + hw/vfio/pci.c | 29 ++++++ hw/vfio/user.c | 160 ++++++++++++++++++++++++++++++++++ MAINTAINERS | 4 + hw/vfio/meson.build | 1 + 6 files changed, 262 insertions(+) create mode 100644 hw/vfio/user.h create mode 100644 hw/vfio/user.c diff --git a/hw/vfio/user.h b/hw/vfio/user.h new file mode 100644 index 0000000000..62b2d03d56 --- /dev/null +++ b/hw/vfio/user.h @@ -0,0 +1,66 @@ +#ifndef VFIO_USER_H +#define VFIO_USER_H + +/* + * vfio protocol over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + */ + +typedef struct { + int send_fds; + int recv_fds; + int *fds; +} VFIOUserFDs; + +typedef struct VFIOUserReply { + QTAILQ_ENTRY(VFIOUserReply) next; + VFIOUserFDs *fds; + uint32_t rsize; + uint32_t id; + QemuCond cv; + bool complete; + bool nowait; +} VFIOUserReply; + + +enum proxy_state { + VFIO_PROXY_CONNECTED =3D 1, + VFIO_PROXY_RECV_ERROR =3D 2, + VFIO_PROXY_CLOSING =3D 3, + VFIO_PROXY_CLOSED =3D 4, +}; + +typedef struct VFIOProxy { + QLIST_ENTRY(VFIOProxy) next; + char *sockname; + struct QIOChannel *ioc; + int (*request)(void *opaque, char *buf, VFIOUserFDs *fds); + void *reqarg; + int flags; + QemuCond close_cv; + + /* + * above only changed when BQL is held + * below are protected by per-proxy lock + */ + QemuMutex lock; + QTAILQ_HEAD(, VFIOUserReply) free; + QTAILQ_HEAD(, VFIOUserReply) pending; + VFIOUserReply *last_nowait; + enum proxy_state state; + bool close_wait; +} VFIOProxy; + +/* VFIOProxy flags */ +#define VFIO_PROXY_CLIENT 0x1 +#define VFIO_PROXY_SECURE 0x2 + +VFIOProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp); +void vfio_user_disconnect(VFIOProxy *proxy); + +#endif /* VFIO_USER_H */ diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 8af11b0a76..f43dc6e5d0 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -75,6 +75,7 @@ typedef struct VFIOAddressSpace { } VFIOAddressSpace; =20 struct VFIOGroup; +typedef struct VFIOProxy VFIOProxy; =20 typedef struct VFIOContainer { VFIOAddressSpace *space; @@ -143,6 +144,7 @@ typedef struct VFIODevice { VFIOMigration *migration; Error *migration_blocker; OnOffAuto pre_copy_dirty_page_tracking; + VFIOProxy *proxy; } VFIODevice; =20 struct VFIODeviceOps { diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index d642aafb7f..7c2d245ca5 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -42,6 +42,7 @@ #include "qapi/error.h" #include "migration/blocker.h" #include "migration/qemu-file.h" +#include "hw/vfio/user.h" =20 #define TYPE_VFIO_PCI_NOHOTPLUG "vfio-pci-nohotplug" =20 @@ -3361,13 +3362,35 @@ static void vfio_user_pci_realize(PCIDevice *pdev, = Error **errp) VFIOUserPCIDevice *udev =3D VFIO_USER_PCI(pdev); VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); VFIODevice *vbasedev =3D &vdev->vbasedev; + SocketAddress addr; + VFIOProxy *proxy; + Error *err =3D NULL; =20 + /* + * TODO: make option parser understand SocketAddress + * and use that instead of having scaler options + * for each socket type. + */ if (!udev->sock_name) { error_setg(errp, "No socket specified"); error_append_hint(errp, "Use -device vfio-user-pci,socket=3D= \n"); return; } =20 + memset(&addr, 0, sizeof(addr)); + addr.type =3D SOCKET_ADDRESS_TYPE_UNIX; + addr.u.q_unix.path =3D udev->sock_name; + proxy =3D vfio_user_connect_dev(&addr, &err); + if (!proxy) { + error_setg(errp, "Remote proxy not found"); + return; + } + vbasedev->proxy =3D proxy; + + if (udev->secure_dma) { + proxy->flags |=3D VFIO_PROXY_SECURE; + } + vbasedev->name =3D g_strdup_printf("VFIO user <%s>", udev->sock_name); vbasedev->dev =3D DEVICE(vdev); vbasedev->fd =3D -1; @@ -3379,6 +3402,12 @@ static void vfio_user_pci_realize(PCIDevice *pdev, E= rror **errp) =20 static void vfio_user_instance_finalize(Object *obj) { + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); + VFIODevice *vbasedev =3D &vdev->vbasedev; + + vfio_put_device(vdev); + + vfio_user_disconnect(vbasedev->proxy); } =20 static Property vfio_user_pci_dev_properties[] =3D { diff --git a/hw/vfio/user.c b/hw/vfio/user.c new file mode 100644 index 0000000000..3bd304e036 --- /dev/null +++ b/hw/vfio/user.c @@ -0,0 +1,160 @@ +/* + * vfio protocol over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include +#include + +#include "qemu/error-report.h" +#include "qapi/error.h" +#include "qemu/main-loop.h" +#include "hw/hw.h" +#include "hw/vfio/vfio-common.h" +#include "hw/vfio/vfio.h" +#include "qemu/sockets.h" +#include "io/channel.h" +#include "io/channel-socket.h" +#include "io/channel-util.h" +#include "sysemu/iothread.h" +#include "user.h" + +static IOThread *vfio_user_iothread; +static void vfio_user_shutdown(VFIOProxy *proxy); + + +/* + * Functions called by main, CPU, or iothread threads + */ + +static void vfio_user_shutdown(VFIOProxy *proxy) +{ + qio_channel_shutdown(proxy->ioc, QIO_CHANNEL_SHUTDOWN_READ, NULL); +} + + +/* + * Functions only called by iothread + */ + +static void vfio_user_cb(void *opaque) +{ + VFIOProxy *proxy =3D opaque; + + qemu_mutex_lock(&proxy->lock); + proxy->state =3D VFIO_PROXY_CLOSED; + qemu_mutex_unlock(&proxy->lock); + qemu_cond_signal(&proxy->close_cv); +} + + +/* + * Functions called by main or CPU threads + */ + +static QLIST_HEAD(, VFIOProxy) vfio_user_sockets =3D + QLIST_HEAD_INITIALIZER(vfio_user_sockets); + +VFIOProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp) +{ + VFIOProxy *proxy; + QIOChannelSocket *sioc; + QIOChannel *ioc; + char *sockname; + + if (addr->type !=3D SOCKET_ADDRESS_TYPE_UNIX) { + error_setg(errp, "vfio_user_connect - bad address family"); + return NULL; + } + sockname =3D addr->u.q_unix.path; + + sioc =3D qio_channel_socket_new(); + ioc =3D QIO_CHANNEL(sioc); + if (qio_channel_socket_connect_sync(sioc, addr, errp)) { + object_unref(OBJECT(ioc)); + return NULL; + } + qio_channel_set_blocking(ioc, true, NULL); + + proxy =3D g_malloc0(sizeof(VFIOProxy)); + proxy->sockname =3D sockname; + proxy->ioc =3D ioc; + proxy->flags =3D VFIO_PROXY_CLIENT; + proxy->state =3D VFIO_PROXY_CONNECTED; + qemu_cond_init(&proxy->close_cv); + + if (vfio_user_iothread =3D=3D NULL) { + vfio_user_iothread =3D iothread_create("VFIO user", errp); + } + + qemu_mutex_init(&proxy->lock); + QTAILQ_INIT(&proxy->free); + QTAILQ_INIT(&proxy->pending); + QLIST_INSERT_HEAD(&vfio_user_sockets, proxy, next); + + return proxy; +} + +void vfio_user_disconnect(VFIOProxy *proxy) +{ + VFIOUserReply *r1, *r2; + + qemu_mutex_lock(&proxy->lock); + + /* our side is quitting */ + if (proxy->state =3D=3D VFIO_PROXY_CONNECTED) { + vfio_user_shutdown(proxy); + if (!QTAILQ_EMPTY(&proxy->pending)) { + error_printf("vfio_user_disconnect: outstanding requests\n"); + } + } + object_unref(OBJECT(proxy->ioc)); + proxy->ioc =3D NULL; + + proxy->state =3D VFIO_PROXY_CLOSING; + QTAILQ_FOREACH_SAFE(r1, &proxy->pending, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->pending, r1, next); + g_free(r1); + } + QTAILQ_FOREACH_SAFE(r1, &proxy->free, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->free, r1, next); + g_free(r1); + } + + /* + * Make sure the iothread isn't blocking anywhere + * with a ref to this proxy by waiting for a BH + * handler to run after the proxy fd handlers were + * deleted above. + */ + proxy->close_wait =3D 1; + aio_bh_schedule_oneshot(iothread_get_aio_context(vfio_user_iothread), + vfio_user_cb, proxy); + + /* drop locks so the iothread can make progress */ + qemu_mutex_unlock_iothread(); + qemu_cond_wait(&proxy->close_cv, &proxy->lock); + + /* we now hold the only ref to proxy */ + qemu_mutex_unlock(&proxy->lock); + qemu_cond_destroy(&proxy->close_cv); + qemu_mutex_destroy(&proxy->lock); + + qemu_mutex_lock_iothread(); + + QLIST_REMOVE(proxy, next); + if (QLIST_EMPTY(&vfio_user_sockets)) { + iothread_destroy(vfio_user_iothread); + vfio_user_iothread =3D NULL; + } + + g_free(proxy); +} diff --git a/MAINTAINERS b/MAINTAINERS index d838b9e3f2..f429bab391 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1882,8 +1882,12 @@ L: qemu-s390x@nongnu.org vfio-user M: John G Johnson M: Thanos Makatos +M: Elena Ufimtseva +M: Jagannathan Raman S: Supported F: docs/devel/vfio-user.rst +F: hw/vfio/user.c +F: hw/vfio/user.h =20 vhost M: Michael S. Tsirkin diff --git a/hw/vfio/meson.build b/hw/vfio/meson.build index da9af297a0..739b30be73 100644 --- a/hw/vfio/meson.build +++ b/hw/vfio/meson.build @@ -8,6 +8,7 @@ vfio_ss.add(when: 'CONFIG_VFIO_PCI', if_true: files( 'display.c', 'pci-quirks.c', 'pci.c', + 'user.c', )) vfio_ss.add(when: 'CONFIG_VFIO_CCW', if_true: files('ccw.c')) vfio_ss.add(when: 'CONFIG_VFIO_PLATFORM', if_true: files('platform.c')) --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132502; cv=pass; d=zohomail.com; s=zohoarc; b=D6iSwK29sjHQf9mq8iC7SkZFYg30rGXVxa8PwoiqI1Z0YkmJtwLtpkUiuBkm3LnX2rUJ+Fkuqxsb7kAqx/75lJ7Tg0Td0f6ElxIfQPyZN9eIzpPbnUyKZNrkAKwuEtmKO480GA/q9o3zBb6C05tL2GOc09KBwuqMjb3xMaKhI2A= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132502; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ujoH7bCSKiw2S++WAYDzYW+rsvzgBNv0frY7sMGl0U0=; b=b/+2P6khXiK5u9Hexwy574zwfr2qpJLRbbkZTuFp4PmhGEXvG0RqVicMvbE3N2WlfoDmGfmLYKXpXEk/6WckzvqeIFxTd9M3AbbMWoby9sCTzfV6QAIXMSbk861nVBemzt4tBJqd/lT7ABRS6lLSj0fYWU4LLmqhIFp34vBGowI= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 162913250212064.55176608756642; Mon, 16 Aug 2021 09:48:22 -0700 (PDT) Received: from localhost ([::1]:37300 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfme-0001lT-Ru for importer@patchew.org; Mon, 16 Aug 2021 12:48:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37078) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhc-0000jF-Je for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:08 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:34316) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhY-0007uh-3V for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:08 -0400 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGaA9n021054; Mon, 16 Aug 2021 16:43:01 GMT Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3afgmb9jyx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:01 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeijC113651; Mon, 16 Aug 2021 16:43:00 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2103.outbound.protection.outlook.com [104.47.58.103]) by aserp3020.oracle.com with ESMTP id 3ae5n609gk-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:00 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by BYAPR10MB3221.namprd10.prod.outlook.com (2603:10b6:a03:14f::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.19; Mon, 16 Aug 2021 16:42:58 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:42:57 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:42:57 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=ujoH7bCSKiw2S++WAYDzYW+rsvzgBNv0frY7sMGl0U0=; b=KJEbxjiEjovko16lh+hxCDUiOYtkzFcT4hyQbmL35qq53C+STQOCQyDlA3UhVbdzLPAG pp4sabPzZRQQVpwwH2Syg5PI8GkwwBm0b/Cij5fBDcSVRhx5xArz0EehGmzU9+ap6kgw hoSO6+7TzM1NhyrD6RFs3iXgMu6g93s2yNQJIs5s6z7vWTnNG/39HPvNW6bXibkj8w2H 1LbavsWQDBcEnGWsm6oRODYHxNHKHuK5AsW95jemfj62tyrvrRO9NMsLzp8XmG7ibYRW mDiyLJvDGvLe4BDHxW1msouVVNAHzxElh2JP0cWvGO0OXNB8uenKLT6S+RMvs1jTPeKj dA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=corp-2020-01-29; bh=ujoH7bCSKiw2S++WAYDzYW+rsvzgBNv0frY7sMGl0U0=; b=HJEXkH2PeXCYKiwPY0hVQ18Xs8I6b7HW40Km0mVXG8HaSByrIcP/6eg5AMW4AMK2h8xQ UkSKAchVLVVkE/1SffzcvDGvN/CNoNgZrrTpuI4zufwJRgZPg8Z2c6edlV6zdE31J3Nx KSHvJdk9QoFsYWqWzxCdXF8ALAnPyFt80dxrX8i4UOxA0Jq+iNI0hj0xVoenQv88WK6u BpHTNka7Nw+hrxb6A5SvBCvuVuXBXkdW7e2wIH7iE2TG4EOfDpPZmt8uzcpYboJbiYS1 zdRxVCB7phomrZeRlBaO20DrpfEhltbwUZjWZP5HS6O12ZAupry2S1m4T9An/RxiXAfH IA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MI+fXm4lLL2F86LNXPAH9q/mbGzdOEtbcf+v11XzYGuD1tegnsZHoS+J57nB/0/PH2rShakAtReT3on8kXnNKcUrH4dbEIZUSTUxBzUvBfJBlF+zPfk9V6CbwzlNPF1AviOW0v390N20YaQnZRSKOkSzv+TTm23Cub3SCFHdvdIDpFd4R/hvN8TG2aaOIvOqqlKO/oRaAUW8vldgK0ZT9fb9APy9Jd76eVutwO0MSxglnbwqtLi7lJ4cqOpOHpm3axOTQ8lxz32+86d1/XwreteupUGjVR1NAe3XWmYTTR9F0qKCXzyM69NpY8mBXvT/dIvEUGkEAD4k0giTTRHmHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ujoH7bCSKiw2S++WAYDzYW+rsvzgBNv0frY7sMGl0U0=; b=nlNx6trd/9Udl/840rEtK9rNR2+KDb96KZIpzFJy4Tkg21PS5Z4BmmPRZqO8PEYUz9Tz/xDY6fEkMTM9Klw1uSNqF6eI3IT7Jt5mf2crnrERKCxS45vPT8cWEiagoXUjPY/nhIqRXps8rlp4Cb8mJAZxF9eSP0SyI/HZDAhFpw99xYwpjZ2CW2mg4go8sHExE+aTCKmdmFlnXbhtrOu80GBIcmDSdtRMMyc6vIQNnxF4MhAeo5pU95vzGnu+cJORJD1GU3wJmlTP/ppw1mRJwgXSaVVBpgUNkdM3RPvUP3EbCOTsVJEKRLLhY9ps383Ru3DFerQ97Zlo+IaDgETvTA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ujoH7bCSKiw2S++WAYDzYW+rsvzgBNv0frY7sMGl0U0=; b=B4kg3BVppp9EtbKOtq/wMwD6MybBpppSdWQ7fVJnA8RBQRV9SGYBlJ2RbFdQ9Yf11rEw7EqmTycahbH2t6FAsQSwmoJGa3sHINfv1VLIHGvflcTSQorB6uboNWxMrBbzw+wDHobFoYUhoJgQguzFvMUl31vn1WXJBR21NWWGKkA= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 05/16] vfio-user: define VFIO Proxy and communication functions Date: Mon, 16 Aug 2021 09:42:38 -0700 Message-Id: <3d3befd308bacb7ec9f3ccd8f99e5184261279b2.1629131628.git.elena.ufimtseva@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8ec3c874-e9e4-4ef3-1202-08d960d4e717 X-MS-TrafficTypeDiagnostic: BYAPR10MB3221: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0zsCA0XJL5UwntX+fMsaz84GhgJF4A/ZF5A5kXm7P4mO4Ssx1pf+ykZ+9DTOVOw0KK7v05wns5atpzTXITDWB/RDSH/tb/8M6b8GlIC9SAfgbZHCKTpGaKF6K37gXxZm0/VcQEfCQ1dRJMcJ1/1Lzq2RH/S+1Udss8VyroUkd6cfZV5Y26VzlpbC5LNgxG+eE8IaRg5xFYu+5SwxplNagjrjUN3NJoRMdu78b4adv5hkhQv71eBBwBrsGopn5a1XofrqjsQRytVsCUyDJ0UTkYFk6h/Ds0+lJs8UFfrxOa1vkMM9Bx3iTU7nyFbokKp3WSpV+IGGAR+8WPp/S0V8AzPHw+6OOpL60BK9m08BFwiDKUOrt/lVczC32cglJbuOoJNgHuR0TfBUWl4WE6zlbCd3LVuBNOxZ3w+KBFFTwkJNCy0c5EvG1VYlEZ4HKY2y0GD/8C1Ba/7uQumogbVadM9UPVI1vpQVlE8X2Z9cjDNgR2Yo7btM1pbxJUhAaft9zE+zoQa/r/M7NJaHP2htMSBe2CcNRZb4K6RQfAttFcLFSBmvte82BQO/MCOzANzHdLd8LQbqBRKQOnbIS3bWvTkk993r+oNs9CuvyNf28mdUxH/rPwywFNcmVmuynyPU13129C4Rv6CEEaPkaQYSfA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(39860400002)(376002)(346002)(366004)(396003)(136003)(2616005)(6916009)(66556008)(66476007)(66946007)(52116002)(30864003)(83380400001)(478600001)(36756003)(5660300002)(6506007)(6666004)(6512007)(186003)(316002)(6486002)(86362001)(44832011)(8936002)(8676002)(38100700002)(4326008)(2906002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dmUzSEw5dkFRVVIzNktwN2ZhRG5JYnBTL0JRb010SnRxYnM4bFpRNWRVVkJ0?= =?utf-8?B?b3VkcmVJZTFkY1MzWHZxUDA2OHdWZFRsTGZ5OFNXZHZqUnFZa3NIVHhCNUtS?= =?utf-8?B?clJMSUh2VmZJM09GMldTaCs3K1RzSENkY3UwWjFLK2JnbUlkblVMYmdRcHpU?= =?utf-8?B?WUFVazh2Z3dQdGZ0V0VRSVpXd0ErbG5DTjUzWlhzQm12cThrQnYwQmRMaU1a?= =?utf-8?B?emRHUFptY0pMd3V2OFZMZlpJcnRHUkJRSGVzbEZjWE04NndJUUNjaFd2bWd5?= =?utf-8?B?K0JNNUtXUUZqaWQvNThOWkwzaHBiR3VPWmR4MjVUZnBnQUhCVjZCbWJzbHM3?= =?utf-8?B?Rk5MdHBWZnYxcG5PZzJuM1FuZ3VLS2ZKOGVGREVNTFd0OHlYQ3ZKWW13bmxz?= =?utf-8?B?TDB6TGxuQkxZYTZCUnoxeVcwai91emFVcVl0SmVPcjN2dkZrYm9ZS05DQ042?= =?utf-8?B?dTlZVngrM1dpcWdjb2VyOGY0QzZPOWlyQTRreEZpa25mWWNURktTdWNBMktD?= =?utf-8?B?Z3hnbU5VMmFkOW1GYzJUQlYxZkdOSWdQbkgydGtacjQzRXM5ZUo4Sk9HWHBI?= =?utf-8?B?QkpJc1ZpVjd3RnNSSXBhcmt2S0J1cmhXOHNnem1ycXlYSXRRTFlVK2NZem41?= =?utf-8?B?bkt1WndqWHRHT1VydFlJY1Bzc05NaUpnNktVUFY4TmF1aUl2bUZqR3BRVWli?= =?utf-8?B?b2xoMU1QYkVnY2I3eHR4d1FmTXllOHhQa0FJSEFSQ2JDM3RlNFBSaVJsNEJP?= =?utf-8?B?N0ttNzhPU3BpdXlUVTNySCt4SzBoQktrOU5xb08xK2VNUGR2d0U4OXhTWEsw?= =?utf-8?B?OXlhY2RrWVJId3pvM3JFMEI5OENCQWhTMklxdVIwYnJVZ1hlS2RrYmcvc3Ez?= =?utf-8?B?MU5ZV3pOcXRWaEZXcTVpWndSb1Z5SjIyMTVEdjdqSldJVERjbW5hRnc2VFVS?= =?utf-8?B?OXMvWWZha3ErdnJIcUVPb0dnQlBjRDQzMXBZemJwUkFYb0RiWUFOL3c0UDRu?= =?utf-8?B?OEZhaVVFS1F2dFpGeUN2elhMWEs4Qm5JSCttS3NVSHRjRFkvVmFoUkdxUTRj?= =?utf-8?B?R3o4QU1OKzNxNEZjMXVqUDJoenpERGo5cmpZR0w4ZnArYWw0NHRCelZlaTMy?= =?utf-8?B?NGhEemJ0WmFHVjRnOTZzVjk2bDAxNXBpTklaa3ZyUmJncG9rZGRuektuUWFC?= =?utf-8?B?c2d3ZXBxZ1ZsRm1GaWN3Z0RCZHp6dWppY1hPaUkwSVN6YWZsK295YnRsTlFF?= =?utf-8?B?RFV2Zm9yY1hEZ1VsVXFWaDRxWFZyeVE1TXE1MTFFSmU1Nk9ERE1EKzNBU3lC?= =?utf-8?B?VlFEN2JUeCs5Mzk3YlpWdkV3aGJiZFRudG1VWjd4Y0tNNVJEMlkveFZwRTF4?= =?utf-8?B?dWpaZ0NCQUFQeStVaEtaNlBqR1NTTTVDMEpQMmpkeG9rZGk0SURpQ1JNeGJo?= =?utf-8?B?SExqQVpyUmNHRmNIeWtKRFVSUGdONGt1R0RwNThFazhxZDdVMFhTT2lXRkI5?= =?utf-8?B?M054S2Z0RXRjRk0wOHZvOTVKUE01RnFocUEyQ1kxQ3pXRFdTSlFleURheXdm?= =?utf-8?B?V09ucldwVFpWU3NxUUptL3ptZnNwRFVSWmZmYWEvLzZZM0hHTlNJYkdWYWRI?= =?utf-8?B?anBkUkJUSy9xZlFOMENGRVlqbHFCcEZtd2NWMEU3OVRNKy82dGR6NWVsbVh2?= =?utf-8?B?ZEtQUkxOQ2RjYkhZRTlvSlJ6YWJibU40MjNpNFc2Um44QytkdUtNU1dFSlV5?= =?utf-8?B?MWNmN0hISVA2eUgrdDhXRlNhMlZjRFZkQmQyeEhqcDM0dUxxWjRLeU12TmVn?= =?utf-8?B?ek1xcVJpYVd1RXgzdUExV293VXJ5U05wemRYQjRvaHlSTnpvRERINk9URGh3?= =?utf-8?Q?8UEiF7nuDxNc1?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8ec3c874-e9e4-4ef3-1202-08d960d4e717 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:42:57.7597 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: j95TseykZUMVeKCmFNyGIGKv6Hsmaq2/Ls8jlHzE8/cJHO083BP60mddnLa5CrLUPWhmha3OqX80WFyl6G7Fi/W33EozQfGq7WiKK8TYXHM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3221 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-ORIG-GUID: 3EqtAwqddRKT-csdXOJdSZEoa9o1xvIb X-Proofpoint-GUID: 3EqtAwqddRKT-csdXOJdSZEoa9o1xvIb Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132502810100001 From: John Johnson Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 62 +++++++++ hw/vfio/user.h | 8 ++ hw/vfio/pci.c | 6 + hw/vfio/user.c | 289 ++++++++++++++++++++++++++++++++++++++++ MAINTAINERS | 1 + 5 files changed, 366 insertions(+) create mode 100644 hw/vfio/user-protocol.h diff --git a/hw/vfio/user-protocol.h b/hw/vfio/user-protocol.h new file mode 100644 index 0000000000..27062cb910 --- /dev/null +++ b/hw/vfio/user-protocol.h @@ -0,0 +1,62 @@ +#ifndef VFIO_USER_PROTOCOL_H +#define VFIO_USER_PROTOCOL_H + +/* + * vfio protocol over a UNIX socket. + * + * Copyright =C2=A9 2018, 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + * Each message has a standard header that describes the command + * being sent, which is almost always a VFIO ioctl(). + * + * The header may be followed by command-specific data, such as the + * region and offset info for read and write commands. + */ + +typedef struct { + uint16_t id; + uint16_t command; + uint32_t size; + uint32_t flags; + uint32_t error_reply; +} VFIOUserHdr; + +/* VFIOUserHdr commands */ +enum vfio_user_command { + VFIO_USER_VERSION =3D 1, + VFIO_USER_DMA_MAP =3D 2, + VFIO_USER_DMA_UNMAP =3D 3, + VFIO_USER_DEVICE_GET_INFO =3D 4, + VFIO_USER_DEVICE_GET_REGION_INFO =3D 5, + VFIO_USER_DEVICE_GET_REGION_IO_FDS =3D 6, + VFIO_USER_DEVICE_GET_IRQ_INFO =3D 7, + VFIO_USER_DEVICE_SET_IRQS =3D 8, + VFIO_USER_REGION_READ =3D 9, + VFIO_USER_REGION_WRITE =3D 10, + VFIO_USER_DMA_READ =3D 11, + VFIO_USER_DMA_WRITE =3D 12, + VFIO_USER_DEVICE_RESET =3D 13, + VFIO_USER_DIRTY_PAGES =3D 14, + VFIO_USER_MAX, +}; + +/* VFIOUserHdr flags */ +#define VFIO_USER_REQUEST 0x0 +#define VFIO_USER_REPLY 0x1 +#define VFIO_USER_TYPE 0xF + +#define VFIO_USER_NO_REPLY 0x10 +#define VFIO_USER_ERROR 0x20 + + +#define VFIO_USER_DEF_MAX_FDS 8 +#define VFIO_USER_MAX_MAX_FDS 16 + +#define VFIO_USER_DEF_MAX_XFER (1024 * 1024) +#define VFIO_USER_MAX_MAX_XFER (64 * 1024 * 1024) + + +#endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio/user.h b/hw/vfio/user.h index 62b2d03d56..905e374e12 100644 --- a/hw/vfio/user.h +++ b/hw/vfio/user.h @@ -11,6 +11,8 @@ * */ =20 +#include "user-protocol.h" + typedef struct { int send_fds; int recv_fds; @@ -19,6 +21,7 @@ typedef struct { =20 typedef struct VFIOUserReply { QTAILQ_ENTRY(VFIOUserReply) next; + VFIOUserHdr *msg; VFIOUserFDs *fds; uint32_t rsize; uint32_t id; @@ -62,5 +65,10 @@ typedef struct VFIOProxy { =20 VFIOProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp); void vfio_user_disconnect(VFIOProxy *proxy); +void vfio_user_set_reqhandler(VFIODevice *vbasdev, + int (*handler)(void *opaque, char *buf, + VFIOUserFDs *fds), + void *reqarg); +void vfio_user_send_reply(VFIOProxy *proxy, char *buf, int ret); =20 #endif /* VFIO_USER_H */ diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 7c2d245ca5..7005d9f891 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3333,6 +3333,11 @@ type_init(register_vfio_pci_dev_type) * vfio-user routines. */ =20 +static int vfio_user_pci_process_req(void *opaque, char *buf, VFIOUserFDs = *fds) +{ + return 0; +} + /* * Emulated devices don't use host hot reset */ @@ -3386,6 +3391,7 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Er= ror **errp) return; } vbasedev->proxy =3D proxy; + vfio_user_set_reqhandler(vbasedev, vfio_user_pci_process_req, vdev); =20 if (udev->secure_dma) { proxy->flags |=3D VFIO_PROXY_SECURE; diff --git a/hw/vfio/user.c b/hw/vfio/user.c index 3bd304e036..2fcc77d997 100644 --- a/hw/vfio/user.c +++ b/hw/vfio/user.c @@ -25,8 +25,15 @@ #include "sysemu/iothread.h" #include "user.h" =20 +static uint64_t max_xfer_size =3D VFIO_USER_DEF_MAX_XFER; static IOThread *vfio_user_iothread; + static void vfio_user_shutdown(VFIOProxy *proxy); +static void vfio_user_recv(void *opaque); +static void vfio_user_send_locked(VFIOProxy *proxy, VFIOUserHdr *msg, + VFIOUserFDs *fds); +static void vfio_user_send(VFIOProxy *proxy, VFIOUserHdr *msg, + VFIOUserFDs *fds); =20 =20 /* @@ -36,6 +43,67 @@ static void vfio_user_shutdown(VFIOProxy *proxy); static void vfio_user_shutdown(VFIOProxy *proxy) { qio_channel_shutdown(proxy->ioc, QIO_CHANNEL_SHUTDOWN_READ, NULL); + qio_channel_set_aio_fd_handler(proxy->ioc, + iothread_get_aio_context(vfio_user_ioth= read), + NULL, NULL, NULL); +} + +static void vfio_user_send_locked(VFIOProxy *proxy, VFIOUserHdr *msg, + VFIOUserFDs *fds) +{ + struct iovec iov =3D { + .iov_base =3D msg, + .iov_len =3D msg->size, + }; + size_t numfds =3D 0; + int msgleft, ret, *fdp =3D NULL; + char *buf; + Error *local_err =3D NULL; + + if (proxy->state !=3D VFIO_PROXY_CONNECTED) { + msg->flags |=3D VFIO_USER_ERROR; + msg->error_reply =3D ECONNRESET; + return; + } + + if (fds !=3D NULL && fds->send_fds !=3D 0) { + numfds =3D fds->send_fds; + fdp =3D fds->fds; + } + + ret =3D qio_channel_writev_full(proxy->ioc, &iov, 1, fdp, numfds, &loc= al_err); + if (ret < 0) { + goto err; + } + if (ret =3D=3D msg->size) { + return; + } + + buf =3D iov.iov_base + ret; + msgleft =3D iov.iov_len - ret; + do { + ret =3D qio_channel_write(proxy->ioc, buf, msgleft, &local_err); + if (ret < 0) { + goto err; + } + buf +=3D ret; + msgleft -=3D ret; + } while (msgleft !=3D 0); + return; + +err: + msg->flags |=3D VFIO_USER_ERROR; + msg->error_reply =3D EIO; + error_report_err(local_err); +} + +static void vfio_user_send(VFIOProxy *proxy, VFIOUserHdr *msg, + VFIOUserFDs *fds) +{ + + qemu_mutex_lock(&proxy->lock); + vfio_user_send_locked(proxy, msg, fds); + qemu_mutex_unlock(&proxy->lock); } =20 =20 @@ -43,6 +111,213 @@ static void vfio_user_shutdown(VFIOProxy *proxy) * Functions only called by iothread */ =20 +void vfio_user_send_reply(VFIOProxy *proxy, char *buf, int ret) +{ + VFIOUserHdr *hdr =3D (VFIOUserHdr *)buf; + + /* + * convert header to associated reply + * positive ret is reply size, negative is error code + */ + hdr->flags =3D VFIO_USER_REPLY; + if (ret >=3D sizeof(VFIOUserHdr)) { + hdr->size =3D ret; + } else if (ret < 0) { + hdr->flags |=3D VFIO_USER_ERROR; + hdr->error_reply =3D -ret; + hdr->size =3D sizeof(*hdr); + } else { + error_printf("vfio_user_send_reply - size too small\n"); + return; + } + vfio_user_send(proxy, hdr, NULL); +} + +void vfio_user_recv(void *opaque) +{ + VFIODevice *vbasedev =3D opaque; + VFIOProxy *proxy =3D vbasedev->proxy; + VFIOUserReply *reply =3D NULL; + g_autofree int *fdp =3D NULL; + VFIOUserFDs reqfds =3D { 0, 0, fdp }; + VFIOUserHdr msg; + struct iovec iov =3D { + .iov_base =3D &msg, + .iov_len =3D sizeof(msg), + }; + bool isreply; + int i, ret; + size_t msgleft, numfds =3D 0; + char *data =3D NULL; + g_autofree char *buf =3D NULL; + Error *local_err =3D NULL; + + qemu_mutex_lock(&proxy->lock); + if (proxy->state =3D=3D VFIO_PROXY_CLOSING) { + qemu_mutex_unlock(&proxy->lock); + return; + } + + ret =3D qio_channel_readv_full(proxy->ioc, &iov, 1, &fdp, &numfds, + &local_err); + if (ret <=3D 0) { + /* read error or other side closed connection */ + goto fatal; + } + + if (ret < sizeof(msg)) { + error_setg(&local_err, "vfio_user_recv short read of header"); + goto err; + } + if (msg.size < sizeof(VFIOUserHdr)) { + error_setg(&local_err, "vfio_user_recv bad header size"); + goto err; + } + + /* + * For replies, find the matching pending request + */ + switch (msg.flags & VFIO_USER_TYPE) { + case VFIO_USER_REQUEST: + isreply =3D 0; + break; + case VFIO_USER_REPLY: + isreply =3D 1; + break; + default: + error_setg(&local_err, "vfio_user_recv unknown message type"); + goto err; + } + + if (isreply) { + QTAILQ_FOREACH(reply, &proxy->pending, next) { + if (msg.id =3D=3D reply->id) { + break; + } + } + if (reply =3D=3D NULL) { + error_setg(&local_err, "vfio_user_recv unexpected reply"); + goto err; + } + QTAILQ_REMOVE(&proxy->pending, reply, next); + + /* + * Process any received FDs + */ + if (numfds !=3D 0) { + if (reply->fds =3D=3D NULL || reply->fds->recv_fds < numfds) { + error_setg(&local_err, "vfio_user_recv unexpected FDs"); + goto err; + } + reply->fds->recv_fds =3D numfds; + memcpy(reply->fds->fds, fdp, numfds * sizeof(int)); + } + + } else { + /* + * The client doesn't expect any FDs in requests, but + * they will be expected on the server + */ + if (numfds !=3D 0 && (proxy->flags & VFIO_PROXY_CLIENT)) { + error_setg(&local_err, "vfio_user_recv fd in client reply"); + goto err; + } + reqfds.recv_fds =3D numfds; + } + + /* + * put the whole message into a single buffer + */ + if (isreply) { + if (msg.size > reply->rsize) { + error_setg(&local_err, + "vfio_user_recv reply larger than recv buffer"); + goto fatal; + } + *reply->msg =3D msg; + data =3D (char *)reply->msg + sizeof(msg); + } else { + if (msg.size > max_xfer_size) { + error_setg(&local_err, "vfio_user_recv request larger than max= "); + goto fatal; + } + buf =3D g_malloc0(msg.size); + memcpy(buf, &msg, sizeof(msg)); + data =3D buf + sizeof(msg); + } + + msgleft =3D msg.size - sizeof(msg); + if (msgleft !=3D 0) { + ret =3D qio_channel_read(proxy->ioc, data, msgleft, &local_err); + if (ret < 0) { + goto fatal; + } + if (ret !=3D msgleft) { + error_setg(&local_err, "vfio_user_recv short read of msg body"= ); + goto err; + } + } + + /* + * Replies signal a waiter, requests get processed by vfio code + * that may assume the iothread lock is held. + */ + if (isreply) { + reply->complete =3D 1; + if (!reply->nowait) { + qemu_cond_signal(&reply->cv); + } else { + if (msg.flags & VFIO_USER_ERROR) { + error_printf("vfio_user_rcv error reply on async request "= ); + error_printf("command %x error %s\n", msg.command, + strerror(msg.error_reply)); + } + /* just free it if no one is waiting */ + reply->nowait =3D 0; + if (proxy->last_nowait =3D=3D reply) { + proxy->last_nowait =3D NULL; + } + g_free(reply->msg); + QTAILQ_INSERT_HEAD(&proxy->free, reply, next); + } + qemu_mutex_unlock(&proxy->lock); + } else { + qemu_mutex_unlock(&proxy->lock); + qemu_mutex_lock_iothread(); + /* + * make sure proxy wasn't closed while we waited + * checking state without holding the proxy lock is safe + * since it's only set to CLOSING when BQL is held + */ + if (proxy->state !=3D VFIO_PROXY_CLOSING) { + ret =3D proxy->request(proxy->reqarg, buf, &reqfds); + if (ret < 0 && !(msg.flags & VFIO_USER_NO_REPLY)) { + vfio_user_send_reply(proxy, buf, ret); + } + } + qemu_mutex_unlock_iothread(); + } + return; + +fatal: + vfio_user_shutdown(proxy); + proxy->state =3D VFIO_PROXY_RECV_ERROR; + +err: + for (i =3D 0; i < numfds; i++) { + close(fdp[i]); + } + if (reply !=3D NULL) { + /* force an error to keep sending thread from hanging */ + reply->msg->flags |=3D VFIO_USER_ERROR; + reply->msg->error_reply =3D EINVAL; + reply->complete =3D 1; + qemu_cond_signal(&reply->cv); + } + qemu_mutex_unlock(&proxy->lock); + error_report_err(local_err); +} + static void vfio_user_cb(void *opaque) { VFIOProxy *proxy =3D opaque; @@ -101,6 +376,20 @@ VFIOProxy *vfio_user_connect_dev(SocketAddress *addr, = Error **errp) return proxy; } =20 +void vfio_user_set_reqhandler(VFIODevice *vbasedev, + int (*handler)(void *opaque, char *buf, + VFIOUserFDs *fds), + void *reqarg) +{ + VFIOProxy *proxy =3D vbasedev->proxy; + + proxy->request =3D handler; + proxy->reqarg =3D reqarg; + qio_channel_set_aio_fd_handler(proxy->ioc, + iothread_get_aio_context(vfio_user_ioth= read), + vfio_user_recv, NULL, vbasedev); +} + void vfio_user_disconnect(VFIOProxy *proxy) { VFIOUserReply *r1, *r2; diff --git a/MAINTAINERS b/MAINTAINERS index f429bab391..52d37dd088 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1888,6 +1888,7 @@ S: Supported F: docs/devel/vfio-user.rst F: hw/vfio/user.c F: hw/vfio/user.h +F: hw/vfio/user-protocol.h =20 vhost M: Michael S. Tsirkin --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132475; cv=pass; d=zohomail.com; s=zohoarc; b=Oj6Yfvxh+sAr3cKVjUTKx5X4aZXfgHkaNB2nNyPbarsclV1NA7lhXuBdKnwgj5Z6k15iyoO/FTmy30yWg//JaL8V2edFA+ui97Wfok1nMH+TYYQuLh83Vt8REQnuljmQWlLS6cfmRxa0OXF6XJCU4Rt0YGeDqOtIeQGMJLxzA+c= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132475; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=wurk93t3EgNHShjLwZYYsyPz+MrafcioJ7ppfdsGyhk=; b=M8EsCIZxdCYn+cFMQWUW5T+AdmH12qeQ2rXjO5tAOJjjCbSeGNuVAOc/+kIA2/V+R4xt5Ub59XPZBcc5i/l6W/N7EuTP1eDYDbLc+ElntybaTPhudZww8Lmbrg0CKGE2iFYZy/B/ioIflt8HyKx7j+Ribf/DoWdD9d3iNagsBeI= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629132475851588.1335273733041; Mon, 16 Aug 2021 09:47:55 -0700 (PDT) Received: from localhost ([::1]:35726 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfmE-0000kv-9a for importer@patchew.org; Mon, 16 Aug 2021 12:47:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37060) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhc-0000iA-6M for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:08 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:36808) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhX-0007xB-Lf for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:07 -0400 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGaDUi000575; Mon, 16 Aug 2021 16:43:02 GMT Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3af1q9anuw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:01 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeijD113651; Mon, 16 Aug 2021 16:43:01 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2103.outbound.protection.outlook.com [104.47.58.103]) by aserp3020.oracle.com with ESMTP id 3ae5n609gk-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:00 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by BYAPR10MB3221.namprd10.prod.outlook.com (2603:10b6:a03:14f::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.19; Mon, 16 Aug 2021 16:42:58 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:42:58 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:42:57 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=wurk93t3EgNHShjLwZYYsyPz+MrafcioJ7ppfdsGyhk=; b=yFE8kNoS7OZl8IAfyGUcvABGJMs0B8txERGOuzJR9WzKrMUFtX/OYAj6lXPqaIF4rBgd G7svVm9m4G1TgbtmqbGek82B3npQHfmy43ULW51CwiLmqkHkGHQ1Y5TDVE+lbgul9ifQ WRlIFDYFJr1A0LdJHXDBSQf5i/qpyfZAFQ/T7TjLROwKUMAKSoGj2JEZFXrHt/6ZF81Q DdIiuSgY7njgvo3uZdKRFFGrrjdcyrU616N9L4RjU+Gu2PZOm0+19rrdVTVsQe8ydaF+ ngWLGfxhuI2UhsLEIFaTaY3wC1hze2nKpFQoWILDjn1jqGj4Fw/vaErqA9NeBVxg3hme ZA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=wurk93t3EgNHShjLwZYYsyPz+MrafcioJ7ppfdsGyhk=; b=SgXhP8i6LSMKUZz68bqKmkV22m0eLhprEpzYd6NIpqenOzHZsSIMmSwrIAGBpnPkeWNn Gbdtrq2iQ21z4YSGPnfgLq+XtPEw6cbnIIlTeQI9sdUEgY8XPiriD5FZQi37H0fD78bp yxe3ryvlKLZZsMUxYJjo5yeAoWm+dnh1dOuUkBwG2RJZnXr6bzx79+q8eym+4epU/EWj TF7HooYPnyNu6VLvbfeIW1TpWmVJvABFpw9CziHw8I1XpGbEC/EUNGHs8yLWbjVENiXq gULrbabNW+6yJDLw6AumT1hbIPDE3ZeOFVco7s41ftyaLChEYnCF3Lne3dyC3iFCU+4H jg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=G5RYhsXAZiUJ7PJ08QtSW1vu1JtjV9OIdEdNM1U+G9m1INYYZPZAzqubEuNYtT5qOMU0lYZ51jTwLPxnozZ1oez0fepC7pm4ynf0p7q9suK0aSwehxgDrDJqnAGol4LwE8srN1NvTKMaAqpaMmi2DsNJwhujdqaJHdRZ/G0VTT3QeSkKQkXfChRAEScoj7DEdQUe2n9wdTrejit90mPT0Jl3r8m/RgGBQ05OjwPvUSb/ezXtVcvo1oBF/lu3gXH+LNE2WMVKEwoAb+FhZAhvUgBV9IV3/SwsC1VXHoKf/lWcznFi1V6HAIJEhR9MHzBCSDLORCYcem+siR8puEi85w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wurk93t3EgNHShjLwZYYsyPz+MrafcioJ7ppfdsGyhk=; b=K/9+Upc5XkqDH9adaXFCVqEcxcku8sJFxL91RRV9cYDXS+JcXoRjxvb511Govn79I1j6vOdlsyiH4MDjnJfCNIA3OTUtltM4A/aj6IX9rmZ0nCRYlnb3HcaLIhcEjRdP8FUV4yAOWCuIy+Yb5pJ7jamml1SRRmcnpCnMl7avUFScr9+x/QU/6gKBY7yi5QJdg6Lw5bGkTRgWuUfUrjc8EyP1M2rkYnN0YfDShCf72AK444DynPold5+0bPtpCHyXyIgjJM2m/fmpb4eaKEswuqFbvyd4DZszYv/90FRb1qVM1/osngLCseSkEM1AxJBlGVePTnBlHZG8zkYawM53Cw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wurk93t3EgNHShjLwZYYsyPz+MrafcioJ7ppfdsGyhk=; b=J0QmalywJKNTDM4/WF12whivR1NkfV7K7oUJYy8F7CSEodQxXDd0NwpMusEslys0qHpB8PG6ecIH04W3ZQ6M8yBp8EKp3gn029+yE7/NN+UxcBlpiHjcPh3kaLBn1Mvkp6asOd0f+mU9Y6aDOamnoDPUtxn/Pzys95nLEzWucuc= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 06/16] vfio-user: negotiate version with remote server Date: Mon, 16 Aug 2021 09:42:39 -0700 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 939681f2-6134-4cb6-8220-08d960d4e75f X-MS-TrafficTypeDiagnostic: BYAPR10MB3221: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:873; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HeK6ZdnVt6Pnd4buy7pEm2C6454sl1eMbOwXXfmZGILSeob5hPwKcR6ijWE6zQIu5Xq1THlHKmH/HJ9IdAK7pDZGx5Zc+SEJq2ljYpfD2mIAm4us+AmD3sobzrNF0c70s/lsHtfeSvEeEcUfT8/9x9bjjyscsHGUV0OHzM4a8mc8KCqGEj/QoEzNoLS9WUsURF1WSmS7aYIBwuumoluBQHliXlVG57m4BZlxTmdg/nPqpDkEKcSJbFn0O2qiF5dSgG6AUoDx4GfuYq3Hff3SiCD9T0s0vj7C6v8U41goIHktzkxcWTfbQG8FLb3WTFklQlewZLlOKyC9x46hiqReddTk9d79HovyQx0BLjqYQ/M3+soB8JqWZ2qDFK7TF6gLzOd3fjRbBRgVXbO0MO83AXVSgURu4HxFnciwSY3RFKuuLjTkqqjG4OcV1JOmKO9uBhGKDYOA5xndFVz+DqNk14HKr7z89b34h0W0kn0UPLw2PrfCTx1enjsyeSEkFg9aWu9H66WOYi4Z8povrg2gHoZjnb0AxPD5rdAw2Ay2SFZ2jMZzUltqnO5gDnSE2twVBzFJCjua/u8jFO9dIWEctZD32Z6hpJhOpPfeOvcSHDxOg6pV67hlGfO/6hU02FmoRvfoGcYA9Fx88n81EAw43Q== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(39860400002)(376002)(346002)(366004)(396003)(136003)(2616005)(6916009)(66556008)(66476007)(66946007)(52116002)(83380400001)(478600001)(36756003)(5660300002)(6506007)(6666004)(6512007)(186003)(316002)(6486002)(86362001)(44832011)(8936002)(8676002)(38100700002)(4326008)(2906002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?WhpJ0+8QuK7pvrGIgm5mO2gk8x49uPB7cCcpCNbc7EdirI5uZLIhv6Y121yz?= =?us-ascii?Q?Bs7BOzgP7b9gyHOaFE3ydoWqirlv9JPdkDBiaJB7lEeBMaXjzm5R5VFFIz83?= =?us-ascii?Q?IfbeAML7q683uDmJhC/tMdigW0ZiVcUwfinAQCWyGYkyU2v4DNQGgyTG2rTS?= =?us-ascii?Q?5keamV21AslmmUKwZDARTXgWVG0Gdea9O35NzdzolTF6MHtWxBh/Rq6wHO7O?= =?us-ascii?Q?WXI0VaVXVitcT5pDKIQIYQc7TnXlngZ5ZTO/lYYfwGg8nH40EYx2I7A+D/Vk?= =?us-ascii?Q?qWKcy2zHERlsD3FW+6j3gESJXU7tWfkiQByUj3r+6nV6HbEIVIUOA+FtNqYR?= =?us-ascii?Q?9y5f4zWd+o/0oXvVuBIi9YhD3hr8tDSMP+C8tKJD1VnjLifxegsWLHrWrKpf?= =?us-ascii?Q?r2NMjqF/zYl/S2DNiwxAFT3le5hnMo4oib42wRfVwhGVKD/CnBi5znR/2uV3?= =?us-ascii?Q?va1bGDQxD9A/WXCPL64XVRoK3UgltF1ApFYXmGILqsNf9MlXKtqVXASnu4jS?= =?us-ascii?Q?q/gDrcQwlxgg533IBBnFgDrx9RomCpOyJVCuRw25SHWowhjzQohFW77SbGS0?= =?us-ascii?Q?iM198FB/YXFQ7+QzdMQEoLhpEC47rYP67+dnvO5o+hznbQEOSdwAr3PCvHmx?= =?us-ascii?Q?BccdMXlfufGGHV/nqAFEHWsIP9Eq95a56kcqBgROE3zXiBZWYAXC/ijzQLK3?= =?us-ascii?Q?QlCipgeOu/CaeErR5IeetmBBAvJOpmNFAkC1muI1hNBiG/RalhocU/MweJTc?= =?us-ascii?Q?+3KZ1d/twf4ocsg/+aoan9qKlfUJATSWWB225LQBixExozdle+9mIcPqobO5?= =?us-ascii?Q?xA7tHRchUEy60O5lQqZLa4AygSA3h/DnWeKrZ/Ol1HXZ3YlFKX2qkesi3Rrv?= =?us-ascii?Q?ebChE8envNj9hbDlD2BQPWrwRhYMg3IrOPZi8ie7Me78PVqymX5L60i1x4gO?= =?us-ascii?Q?X07dEhAbRq7kSWmTswO6+KknT3un7BChCxp19RXRyH/9cqpISXZx2KuO8Pbo?= =?us-ascii?Q?fpzoMCZeEF208wHnf1McBQsj6gc4txHInxCPoa5w2XdKf8YMC1vT4OpQNp1n?= =?us-ascii?Q?1+44nhzK21JYpMf89zMMxXWS8hEO2OCsx4kN8Ho3PuPNNZ48lC0zuiG7QBnV?= =?us-ascii?Q?pUvFqBjqrJLQS9w5kKCB8srNMoHacAjsWsW7d0RDwNZ6v49oC3DB31Kc7F4i?= =?us-ascii?Q?yeRCMYjyiMYEmbFy/Iloort/JsDaHVyT3nZrZvSEdpcoTd9nq9g2fg/w7SEb?= =?us-ascii?Q?QS1mSUGlSFOkPbyIX0ybUbe3/5yD7D+WvCU8exJ5uFqecwN8v7W+4r7KigPR?= =?us-ascii?Q?DyD8dMc/IVoKhJmldvjoeUcpJ3yB3NZTBJL6n8QU1XTZlE+5HnxSW9R7K+RU?= =?us-ascii?Q?Rai0FP69JOtqwNRfTFBjhXzOCGiI?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 939681f2-6134-4cb6-8220-08d960d4e75f X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:42:58.6541 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: y5+PNTGiUgwTyLXxUePO9cVlxzFhAbXThzQOIbcNYuINaa0m/41FsRT+Nb/oLPJgDUFBI/7VksWHLvQDUhZ1SXAgYuusn9HoXpkLPcpmD7w= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3221 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-GUID: MLR0wU_GX5YYoJf4hzscZrCrhsnCo7BD X-Proofpoint-ORIG-GUID: MLR0wU_GX5YYoJf4hzscZrCrhsnCo7BD Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -8 X-Spam_score: -0.9 X-Spam_bar: / X-Spam_report: (-0.9 / 5.0 requ) DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132477114100001 Content-Type: text/plain; charset="utf-8" From: John Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson --- hw/vfio/user-protocol.h | 23 ++++ hw/vfio/user.h | 1 + hw/vfio/pci.c | 9 ++ hw/vfio/user.c | 267 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 300 insertions(+) diff --git a/hw/vfio/user-protocol.h b/hw/vfio/user-protocol.h index 27062cb910..14b762d1ad 100644 --- a/hw/vfio/user-protocol.h +++ b/hw/vfio/user-protocol.h @@ -52,6 +52,29 @@ enum vfio_user_command { #define VFIO_USER_ERROR 0x20 =20 =20 +/* + * VFIO_USER_VERSION + */ +typedef struct { + VFIOUserHdr hdr; + uint16_t major; + uint16_t minor; + char capabilities[]; +} VFIOUserVersion; + +#define VFIO_USER_MAJOR_VER 0 +#define VFIO_USER_MINOR_VER 0 + +#define VFIO_USER_CAP "capabilities" + +/* "capabilities" members */ +#define VFIO_USER_CAP_MAX_FDS "max_msg_fds" +#define VFIO_USER_CAP_MAX_XFER "max_data_xfer_size" +#define VFIO_USER_CAP_MIGR "migration" + +/* "migration" member */ +#define VFIO_USER_CAP_PGSIZE "pgsize" + #define VFIO_USER_DEF_MAX_FDS 8 #define VFIO_USER_MAX_MAX_FDS 16 =20 diff --git a/hw/vfio/user.h b/hw/vfio/user.h index 905e374e12..cab957ba7a 100644 --- a/hw/vfio/user.h +++ b/hw/vfio/user.h @@ -70,5 +70,6 @@ void vfio_user_set_reqhandler(VFIODevice *vbasdev, VFIOUserFDs *fds), void *reqarg); void vfio_user_send_reply(VFIOProxy *proxy, char *buf, int ret); +int vfio_user_validate_version(VFIODevice *vbasedev, Error **errp); =20 #endif /* VFIO_USER_H */ diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 7005d9f891..eae33e746f 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3397,6 +3397,12 @@ static void vfio_user_pci_realize(PCIDevice *pdev, E= rror **errp) proxy->flags |=3D VFIO_PROXY_SECURE; } =20 + vfio_user_validate_version(vbasedev, &err); + if (err !=3D NULL) { + error_propagate(errp, err); + goto error; + } + vbasedev->name =3D g_strdup_printf("VFIO user <%s>", udev->sock_name); vbasedev->dev =3D DEVICE(vdev); vbasedev->fd =3D -1; @@ -3404,6 +3410,9 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Er= ror **errp) vbasedev->no_mmap =3D false; vbasedev->ops =3D &vfio_user_pci_ops; =20 +error: + vfio_user_disconnect(proxy); + error_prepend(errp, VFIO_MSG_PREFIX, vdev->vbasedev.name); } =20 static void vfio_user_instance_finalize(Object *obj) diff --git a/hw/vfio/user.c b/hw/vfio/user.c index 2fcc77d997..e89464a571 100644 --- a/hw/vfio/user.c +++ b/hw/vfio/user.c @@ -23,9 +23,16 @@ #include "io/channel-socket.h" #include "io/channel-util.h" #include "sysemu/iothread.h" +#include "qapi/qmp/qdict.h" +#include "qapi/qmp/qjson.h" +#include "qapi/qmp/qnull.h" +#include "qapi/qmp/qstring.h" +#include "qapi/qmp/qnum.h" #include "user.h" =20 static uint64_t max_xfer_size =3D VFIO_USER_DEF_MAX_XFER; +static uint64_t max_send_fds =3D VFIO_USER_DEF_MAX_FDS; +static int wait_time =3D 1000; /* wait 1 sec for replies */ static IOThread *vfio_user_iothread; =20 static void vfio_user_shutdown(VFIOProxy *proxy); @@ -34,7 +41,14 @@ static void vfio_user_send_locked(VFIOProxy *proxy, VFIO= UserHdr *msg, VFIOUserFDs *fds); static void vfio_user_send(VFIOProxy *proxy, VFIOUserHdr *msg, VFIOUserFDs *fds); +static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags); +static void vfio_user_send_recv(VFIOProxy *proxy, VFIOUserHdr *msg, + VFIOUserFDs *fds, int rsize, int flags); =20 +/* vfio_user_send_recv flags */ +#define NOWAIT 0x1 /* do not wait for reply */ +#define NOIOLOCK 0x2 /* do not drop iolock */ =20 /* * Functions called by main, CPU, or iothread threads @@ -333,6 +347,79 @@ static void vfio_user_cb(void *opaque) * Functions called by main or CPU threads */ =20 +static void vfio_user_send_recv(VFIOProxy *proxy, VFIOUserHdr *msg, + VFIOUserFDs *fds, int rsize, int flags) +{ + VFIOUserReply *reply; + bool iolock =3D 0; + + if (msg->flags & VFIO_USER_NO_REPLY) { + error_printf("vfio_user_send_recv on async message\n"); + return; + } + + /* + * We may block later, so use a per-proxy lock and let + * the iothreads run while we sleep unless told no to + */ + QEMU_LOCK_GUARD(&proxy->lock); + if (!(flags & NOIOLOCK)) { + iolock =3D qemu_mutex_iothread_locked(); + if (iolock) { + qemu_mutex_unlock_iothread(); + } + } + + reply =3D QTAILQ_FIRST(&proxy->free); + if (reply !=3D NULL) { + QTAILQ_REMOVE(&proxy->free, reply, next); + reply->complete =3D 0; + } else { + reply =3D g_malloc0(sizeof(*reply)); + qemu_cond_init(&reply->cv); + } + reply->msg =3D msg; + reply->fds =3D fds; + reply->id =3D msg->id; + reply->rsize =3D rsize ? rsize : msg->size; + QTAILQ_INSERT_TAIL(&proxy->pending, reply, next); + + vfio_user_send_locked(proxy, msg, fds); + if (!(msg->flags & VFIO_USER_ERROR)) { + if (!(flags & NOWAIT)) { + while (reply->complete =3D=3D 0) { + if (!qemu_cond_timedwait(&reply->cv, &proxy->lock, wait_ti= me)) { + msg->flags |=3D VFIO_USER_ERROR; + msg->error_reply =3D ETIMEDOUT; + break; + } + } + QTAILQ_INSERT_HEAD(&proxy->free, reply, next); + } else { + reply->nowait =3D 1; + proxy->last_nowait =3D reply; + } + } else { + QTAILQ_INSERT_HEAD(&proxy->free, reply, next); + } + + if (iolock) { + qemu_mutex_lock_iothread(); + } +} + +static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags) +{ + static uint16_t next_id; + + hdr->id =3D qatomic_fetch_inc(&next_id); + hdr->command =3D cmd; + hdr->size =3D size; + hdr->flags =3D (flags & ~VFIO_USER_TYPE) | VFIO_USER_REQUEST; + hdr->error_reply =3D 0; +} + static QLIST_HEAD(, VFIOProxy) vfio_user_sockets =3D QLIST_HEAD_INITIALIZER(vfio_user_sockets); =20 @@ -447,3 +534,183 @@ void vfio_user_disconnect(VFIOProxy *proxy) =20 g_free(proxy); } + +struct cap_entry { + const char *name; + int (*check)(QObject *qobj, Error **errp); +}; + +static int caps_parse(QDict *qdict, struct cap_entry caps[], Error **errp) +{ + QObject *qobj; + struct cap_entry *p; + + for (p =3D caps; p->name !=3D NULL; p++) { + qobj =3D qdict_get(qdict, p->name); + if (qobj !=3D NULL) { + if (p->check(qobj, errp)) { + return -1; + } + qdict_del(qdict, p->name); + } + } + + /* warning, for now */ + if (qdict_size(qdict) !=3D 0) { + error_printf("spurious capabilities\n"); + } + return 0; +} + +static int check_pgsize(QObject *qobj, Error **errp) +{ + QNum *qn =3D qobject_to(QNum, qobj); + uint64_t pgsize; + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &pgsize)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_PGSIZE); + return -1; + } + return pgsize =3D=3D 4096 ? 0 : -1; +} + +static struct cap_entry caps_migr[] =3D { + { VFIO_USER_CAP_PGSIZE, check_pgsize }, + { NULL } +}; + +static int check_max_fds(QObject *qobj, Error **errp) +{ + QNum *qn =3D qobject_to(QNum, qobj); + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &max_send_fds) || + max_send_fds > VFIO_USER_MAX_MAX_FDS) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_FDS); + return -1; + } + return 0; +} + +static int check_max_xfer(QObject *qobj, Error **errp) +{ + QNum *qn =3D qobject_to(QNum, qobj); + + if (qn =3D=3D NULL || !qnum_get_try_uint(qn, &max_xfer_size) || + max_xfer_size > VFIO_USER_MAX_MAX_XFER) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_XFER); + return -1; + } + return 0; +} + +static int check_migr(QObject *qobj, Error **errp) +{ + QDict *qdict =3D qobject_to(QDict, qobj); + + if (qdict =3D=3D NULL || caps_parse(qdict, caps_migr, errp)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_FDS); + return -1; + } + return 0; +} + +static struct cap_entry caps_cap[] =3D { + { VFIO_USER_CAP_MAX_FDS, check_max_fds }, + { VFIO_USER_CAP_MAX_XFER, check_max_xfer }, + { VFIO_USER_CAP_MIGR, check_migr }, + { NULL } +}; + +static int check_cap(QObject *qobj, Error **errp) +{ + QDict *qdict =3D qobject_to(QDict, qobj); + + if (qdict =3D=3D NULL || caps_parse(qdict, caps_cap, errp)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP); + return -1; + } + return 0; +} + +static struct cap_entry ver_0_0[] =3D { + { VFIO_USER_CAP, check_cap }, + { NULL } +}; + +static int caps_check(int minor, const char *caps, Error **errp) +{ + QObject *qobj; + QDict *qdict; + int ret; + + qobj =3D qobject_from_json(caps, NULL); + if (qobj =3D=3D NULL) { + error_setg(errp, "malformed capabilities %s", caps); + return -1; + } + qdict =3D qobject_to(QDict, qobj); + if (qdict =3D=3D NULL) { + error_setg(errp, "capabilities %s not an object", caps); + qobject_unref(qobj); + return -1; + } + ret =3D caps_parse(qdict, ver_0_0, errp); + + qobject_unref(qobj); + return ret; +} + +static GString *caps_json(void) +{ + QDict *dict =3D qdict_new(); + QDict *capdict =3D qdict_new(); + QDict *migdict =3D qdict_new(); + GString *str; + + qdict_put_int(migdict, VFIO_USER_CAP_PGSIZE, 4096); + qdict_put_obj(capdict, VFIO_USER_CAP_MIGR, QOBJECT(migdict)); + + qdict_put_int(capdict, VFIO_USER_CAP_MAX_FDS, VFIO_USER_MAX_MAX_FDS); + qdict_put_int(capdict, VFIO_USER_CAP_MAX_XFER, VFIO_USER_DEF_MAX_XFER); + + qdict_put_obj(dict, VFIO_USER_CAP, QOBJECT(capdict)); + + str =3D qobject_to_json(QOBJECT(dict)); + qobject_unref(dict); + return str; +} + +int vfio_user_validate_version(VFIODevice *vbasedev, Error **errp) +{ + g_autofree VFIOUserVersion *msgp; + GString *caps; + int size, caplen; + + caps =3D caps_json(); + caplen =3D caps->len + 1; + size =3D sizeof(*msgp) + caplen; + msgp =3D g_malloc0(size); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_VERSION, size, 0); + msgp->major =3D VFIO_USER_MAJOR_VER; + msgp->minor =3D VFIO_USER_MINOR_VER; + memcpy(&msgp->capabilities, caps->str, caplen); + g_string_free(caps, true); + + vfio_user_send_recv(vbasedev->proxy, &msgp->hdr, NULL, 0, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + error_setg_errno(errp, msgp->hdr.error_reply, "version reply"); + return -1; + } + + if (msgp->major !=3D VFIO_USER_MAJOR_VER || + msgp->minor > VFIO_USER_MINOR_VER) { + error_setg(errp, "incompatible server version"); + return -1; + } + if (caps_check(msgp->minor, (char *)msgp + sizeof(*msgp), errp) !=3D 0= ) { + return -1; + } + + return 0; +} --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132473; cv=pass; d=zohomail.com; s=zohoarc; b=lYFXXR1njjKzqlq82UrvRNznPnPWV+r8p1ViNNXv+wom0xpI7H9NW5HtiSvLomft7+X/y/Vz94LXu4xH4HopvN6eum9Ek6Si2v6kNDhJnnBKVXjgaM6nuH226J6H1ZN17PHR4RiOVSMaG+NykLBjUrOfmukcLssoWlT0d1qO+rQ= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132473; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=W4lpY0dj94sm3O/uZtUx8TQVK8x1Yaa9DO9CP6I9Zno=; b=DlUF7PROa1E/CXDYVk2+Q65P6uSobn69VUq8j2Y0KbQb7U1ebZJSWr3wVDiChoaWe4oVNLoA6441pPKVHr9mO3PiTcsB8V3SNrzTokXGdPkqY0F6LykLWHPRt0GwnwXSHnAKivH9WtTLuMvpP+hStKA/+91zydQw/nksEfNSNI4= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629132473053412.8526883055107; Mon, 16 Aug 2021 09:47:53 -0700 (PDT) Received: from localhost ([::1]:35634 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfmB-0000hm-V1 for importer@patchew.org; Mon, 16 Aug 2021 12:47:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37048) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhb-0000gj-Oa for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:07 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:47136) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhX-0007w9-M1 for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:07 -0400 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGbX1d029459; Mon, 16 Aug 2021 16:43:02 GMT Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3af3kxtg0s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:01 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeo5s159237; Mon, 16 Aug 2021 16:43:00 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2106.outbound.protection.outlook.com [104.47.58.106]) by aserp3030.oracle.com with ESMTP id 3ae3ve34y4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:00 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by SJ0PR10MB4765.namprd10.prod.outlook.com (2603:10b6:a03:2af::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.15; Mon, 16 Aug 2021 16:42:59 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:42:59 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:42:58 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=W4lpY0dj94sm3O/uZtUx8TQVK8x1Yaa9DO9CP6I9Zno=; b=rOEfmXBrKFRL2uR+LtmrtLsqJkAwmzNRbcvS2rx5TEtmWq2Lim3XXel0H6ViWxxNoGRU 7u5mFIRYaf5B+ky/iqMLQBZGmJL1qc3nkR0Nbrz4+guwt5D/N1F9caXGQ8fyliar4VzK i356gjkgeVCZOc0+xeEBb0w5XLa8DN+pptfOxuzdrDSdcSBE72mWkC3ukTfIdUGwQIrA UVGera3pmfm2ZQTCEMY4zpE0qGIZKO8aBxTQ0lz+VN9DNBm/nUsbnCSgrunQdHPWtrRc OemD7sxe2qRpNXrnpcX0D6ptxtKdeJEaOk28E+OhsrrEWAJCeycy5mW3hmuNocUQb6jl ng== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=W4lpY0dj94sm3O/uZtUx8TQVK8x1Yaa9DO9CP6I9Zno=; b=fg4ekc9a9Kxh50Z/WWI55wQwZBP67v7OWLkuEbFOzKlZfEtRyM6DTiSidoJ7zvrAZL8H 7IUSxRq+zP4rez+gZXmWruyVmhMXUxkkG9VqoO+moyzQH/XSldjEJ1YWOxVqU/2W9Gnd rbYIqScdQ6ldp65JpTQyPSHEs4hK2if0pGeiuzBXy5JOVr/HhOTVZesKhUT0Ly9puolR 3kBgOJO1FogNBDUK27uvP8AbpR11K/MQVc2lJKWUrHQXRJehk/c1eKkr99UInt2chvkF XuQ0lZDSqmdhh1DdaWCJkM8p8ZcWKecjYhQul9cM5EKIoFk15IaKVlfYm0ny1Lut1yOJ fg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DopNxm+FCOJnSi6HRPFSdqVLFseRKDt3CoIC17kjXkx5MHOW7AJWk6OPukeqiwZNTv8sYaPazOXmcWcn/dXAgX5BrtaOMfWy3bX+0uxKnSD6GleCHqPhmovhrK6AabtvLUDQkBNDKBBVNSnw1J1fW3HCimXWscEgpj30tgZ2KIf1HuXT/BP/JnEFPCO3OJGIls6rreIEkiKMPgyVaOs1CCgj5BAeEB6bs7ofEs6Samzr59TY7mjSi8YthRpHZOLv7albjZqZage0SLGvHdjo6th/LKN8osDtuztA534vFtHgCr3p/0NtU8V4o0tIcqooherZMHnAkYytyVATwrHgXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W4lpY0dj94sm3O/uZtUx8TQVK8x1Yaa9DO9CP6I9Zno=; b=NNd8Du8MyoAc0a+zPXlb+9aYvY8TTGac+hq98uxlTI/E3mB1yl3+XWGdr5RKzMtE6AW22V7iecx3Y984YTljR06pL5JWm47NEQiq0XWQcmsaow3D/egR0cCcfPlIvtlYY0i+kcMLwALKOnOXVh86nMhaQeyIxYCC61X3ldL+sDjaigqvUsNSIQoRDcQOdMPVgFUmQigTpcmDUKszcoZeAkJXxyI+UrNCZykofY7m8VK6GjijBb4ZP8MqJovn65gqhU5VS4Jrdw7FVxFF6j66tux90jlHIuk4aCcfPfRqKo7aJsVKAHSc4SsTnHhVXytfL4Vi1zN9OCP9//FMjdGJCg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W4lpY0dj94sm3O/uZtUx8TQVK8x1Yaa9DO9CP6I9Zno=; b=OlY3CYU2w5h+KxDhg06fp/DV1p0vV4PNTg37fXLsWfSh/ue4aEn8Mb5RVOHKF0tTOzXZW+Hgm2rjQ2W4fpLm6ldgZ3SD84oYpcJM0QBCKzvVgpKupOOHHDZ4m5oakPgQdYozbTWSQHTUekJ2aFbiJWi1Wf+8xtDeXdEAcLrvTs4= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 07/16] vfio-user: get device info Date: Mon, 16 Aug 2021 09:42:40 -0700 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 620908f6-de42-4fed-f59c-08d960d4e7f4 X-MS-TrafficTypeDiagnostic: SJ0PR10MB4765: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:669; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VrleL+Y+SkdABFTZqjAgI8vXtgjWZRbBS/nYW29S3oo8+sal4gyLT53wrmIV1QAkbs54VOGxNqRenbkpBkC3zyGe1mjOd6puTgP1MgO+U6hSzv5qQkPWEUbEjJc6MKxIuRtznnl+4XS66f0x0grHGNfb3/cDRcYv0zrbzKgu6zBqjevk729jmRviBU4yOb5lin0HoiDjEw1kpNrhLnSfiIkeOxrebNP/GZYKJYkgf7x0gfeaYetVLyRY44q7vEDIfbSm7L4wlHKL1a4ChYzwfATBVYTk5IgBaYJSjFK7G555SLqBWMlnf1Lh9xesx8ZtYCdMRBpqxM8F03KCBO5vmf95/XNapmhb/47EV9Ne7leI/M2Sqp9cTKrOOf+LNrNbSfAvbjcqaXQSgkwYAIgBLBM/gpQKtpO5iRLR9OhsTJYmLGptI0dLpfrcQsqjQGTJUeqdxu8k6iQDOHS7TIHiP4Evs3HXeOI7AXQsWn3PKdMjS5eNd1l0+DbrlL/ypdXwSXlQaT9QE1lRINVcDBLoF4F4taX2VRSNtPQoYcHtVxlu1U9y1L4th8cb60aaCUNkpiW8Zi9h5EBmvkey+LtuLLuoke3JJHIx4z4xe5W95xH4xMCPtPxbh79WX4e0tVN9cDVD+i4exeoWQmwfNoVIrA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(186003)(2906002)(86362001)(66556008)(4326008)(38100700002)(8676002)(508600001)(66476007)(6486002)(66946007)(6506007)(5660300002)(44832011)(2616005)(6512007)(6666004)(36756003)(6916009)(316002)(52116002)(83380400001)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?+1mzodtN4htY4PE4CWwzi8JmQTXuWGaI1qeuTskF8g2XI9NeTTeDe3TiR8Sq?= =?us-ascii?Q?koUO/ttbwAqI5lhJeM80hvFsmzcKNBkamHkCVP+40nn8Yvwat9SsNcz40MVe?= =?us-ascii?Q?20g7RdDj91CKhzdIHwCcW8De8nYSEtTS8oW5IgnCDFnfbUtolXMXAfL0rKgt?= =?us-ascii?Q?hk/tgLjT+SWmIAB8S7a1F0U3EENUGm1B9Qfmo8V/SOjdEi6nxc+Maow0VDpr?= =?us-ascii?Q?euKCaKV95SaWXWJimuy5r3EmbRbnYZnfAABBxfhNKlbuAp7IVV47JWx03KRG?= =?us-ascii?Q?RvG8P20UjNNKoT1zI442I1eM+7I6Bd5miymDmRmI1KxCoJkPkC/9M2z9WsUl?= =?us-ascii?Q?vnjbcbW/DAEWmN4LwHNtsUwEceBB+bT2ZmvegoXa3S5h7FEed4rIFYmqIl20?= =?us-ascii?Q?DKxgf6kj4olZoDFqpUv2IFS1MgV+EBcHr7dnCBSvsE5gkbkWQg2CHtTJDPJ/?= =?us-ascii?Q?cJaYWPg6JqP3eWoe6zSfFXCm4kE19kP8HLTRA7TmXgcCex/jl/yZ82PXjEW6?= =?us-ascii?Q?+Ey8aEnUxmiJycSOi0FVeU0GihFjYuImsrGGKHYon995DmiPzxnQCHGAfjiN?= =?us-ascii?Q?79xIViACRTRY3NGm3JMU7OsV2WkOIQrcpTAOlNl4QST9Zbw1cQNdZ2K5Erv+?= =?us-ascii?Q?NbcfgI3G0KT+XzkdVfW3K6vVipAZzJ5uJEG0X6pbL+tlMREv2vOS/SWXlUH+?= =?us-ascii?Q?zLBeScwx26iSya8s6o16aI+A5SW64wrU2d1uUQm3Z5FuNS6uoAjfya3PB7W0?= =?us-ascii?Q?T3fi+FSWN1QGaYPKr6aOOu5O3EGRCAEOvXGQ4ssoWqNK4q4zetTwFarhiw7Q?= =?us-ascii?Q?povWh7Mrd5tOAgCR63Ft3AqYDk8Izc6b0fmOBzZP0bFK1l2KetMSC5vVEAUX?= =?us-ascii?Q?G0k1xYHVeyQVfLkKVWW8QOA3aA9qA6V7VsDM8vk333zzCDFuPyXza+sXImwp?= =?us-ascii?Q?x+EuM6aOSgbboESg4J/m+fbX3ZyvhaRjBncw6wsuju+enp+NO1KfhbB2bsOs?= =?us-ascii?Q?upwITkxJy07CHZM2zdVNUA/mL82tjgiGo+YCO2HlfDCLi871tKWUI08J0lt1?= =?us-ascii?Q?WtPPlF1GA2t5LxGQCIk5Iy77Y/sPT4iEhZSuQZeYemN6EjvMDumY6V+K/W/t?= =?us-ascii?Q?30ztsS44EYBts+cOdnHSd/jgTjNm6M0GBWmcO/+mhqLApvD48UFVzGGLLzuK?= =?us-ascii?Q?Vd78EBj3kYats2uNYXxXCIUhn7EdvEqokG5xowl2LS83fVu6Wxgf5NdJwnxd?= =?us-ascii?Q?S+aPjdInw232xOEj48DoP1bNIn7A28eV97df7KAp725XG23HFMS0Sjvm8xbd?= =?us-ascii?Q?gsQLuclSNkFPuFplmoxVKpz10X+kEqyl9mUE/QgrDNeMkhF2dQIr5b8PS7Cx?= =?us-ascii?Q?IRYJmcaFGcUWgcA5xD/8ScUsRdrL?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 620908f6-de42-4fed-f59c-08d960d4e7f4 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:42:59.2008 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xx4DGMYG9ilVo/+o0PKhIyy09Eqhl7GeBJOmj/Th4vHWNavBgMpnFypqoVsi2JQ0nu1qh6XeYl8Sw1KWc0D2eum+szpu7tEwKr6w6DFmygk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4765 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 bulkscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-GUID: ZhRxo4LBAlc5gDS3M7WOofluhnF4Cmqk X-Proofpoint-ORIG-GUID: ZhRxo4LBAlc5gDS3M7WOofluhnF4Cmqk Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132474864100001 Content-Type: text/plain; charset="utf-8" From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 13 +++++++++++++ hw/vfio/user.h | 1 + hw/vfio/pci.c | 13 +++++++++++++ hw/vfio/user.c | 20 ++++++++++++++++++++ 4 files changed, 47 insertions(+) diff --git a/hw/vfio/user-protocol.h b/hw/vfio/user-protocol.h index 14b762d1ad..13e44ebf1c 100644 --- a/hw/vfio/user-protocol.h +++ b/hw/vfio/user-protocol.h @@ -82,4 +82,17 @@ typedef struct { #define VFIO_USER_MAX_MAX_XFER (64 * 1024 * 1024) =20 =20 +/* + * VFIO_USER_DEVICE_GET_INFO + * imported from struct_device_info + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t num_regions; + uint32_t num_irqs; + uint32_t cap_offset; +} VFIOUserDeviceInfo; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio/user.h b/hw/vfio/user.h index cab957ba7a..82044e7e78 100644 --- a/hw/vfio/user.h +++ b/hw/vfio/user.h @@ -71,5 +71,6 @@ void vfio_user_set_reqhandler(VFIODevice *vbasdev, void *reqarg); void vfio_user_send_reply(VFIOProxy *proxy, char *buf, int ret); int vfio_user_validate_version(VFIODevice *vbasedev, Error **errp); +int vfio_user_get_info(VFIODevice *vbasedev); =20 #endif /* VFIO_USER_H */ diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index eae33e746f..63aa2441f0 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3369,6 +3369,7 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Er= ror **errp) VFIODevice *vbasedev =3D &vdev->vbasedev; SocketAddress addr; VFIOProxy *proxy; + int ret; Error *err =3D NULL; =20 /* @@ -3410,6 +3411,18 @@ static void vfio_user_pci_realize(PCIDevice *pdev, E= rror **errp) vbasedev->no_mmap =3D false; vbasedev->ops =3D &vfio_user_pci_ops; =20 + ret =3D vfio_user_get_info(&vdev->vbasedev); + if (ret) { + error_setg_errno(errp, -ret, "get info failure"); + goto error; + } + + vfio_populate_device(vdev, &err); + if (err) { + error_propagate(errp, err); + goto error; + } + error: vfio_user_disconnect(proxy); error_prepend(errp, VFIO_MSG_PREFIX, vdev->vbasedev.name); diff --git a/hw/vfio/user.c b/hw/vfio/user.c index e89464a571..b584b8e0f2 100644 --- a/hw/vfio/user.c +++ b/hw/vfio/user.c @@ -714,3 +714,23 @@ int vfio_user_validate_version(VFIODevice *vbasedev, E= rror **errp) =20 return 0; } + +int vfio_user_get_info(VFIODevice *vbasedev) +{ + VFIOUserDeviceInfo msg; + + memset(&msg, 0, sizeof(msg)); + vfio_user_request_msg(&msg.hdr, VFIO_USER_DEVICE_GET_INFO, sizeof(msg)= , 0); + msg.argsz =3D sizeof(struct vfio_device_info); + + vfio_user_send_recv(vbasedev->proxy, &msg.hdr, NULL, 0, 0); + if (msg.hdr.flags & VFIO_USER_ERROR) { + return -msg.hdr.error_reply; + } + + vbasedev->num_irqs =3D msg.num_irqs; + vbasedev->num_regions =3D msg.num_regions; + vbasedev->flags =3D msg.flags; + vbasedev->reset_works =3D !!(msg.flags & VFIO_DEVICE_FLAGS_RESET); + return 0; +} --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132709; cv=pass; d=zohomail.com; s=zohoarc; b=kCxpxXUws5achuly2feCAtfScvuF9nmzOOCOG32feEIvpz1SJYIqmHr2t7DO9BkRFvngKcAc0OuxbATGN2T+6ss3XVRZuSO1zh5j9w+qHlELKV1nF9rASSd0CBskWBXERbt4Pruzpr6fR1jy7wjrnEMAQIr4EXqtwdzcHgRL4eI= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132709; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Z+HYnZBTg8/iVO2ADTDXw+BbrtUybeoKYSv9IN97OaE=; b=TB2g5sCmngJQPFCzk41FGX2TIwFKRiFqnIs5bUCEi55V8sjKQBsGIWBzW3j+cjoi0p9Pswsth8nBi2al3o6GTdFiZJezpZcDf3pclbv12ehjC9AOgg9xWj4LqLbswd31Ot3l4Qjj2rjQabr7Y9DJYS7/uGKQrWauWh4CimwH3Qg= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629132709433122.47497335582739; Mon, 16 Aug 2021 09:51:49 -0700 (PDT) Received: from localhost ([::1]:46164 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfq0-0007j0-AL for importer@patchew.org; Mon, 16 Aug 2021 12:51:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37108) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhd-0000l1-9E for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:09 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:47936) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhX-0007x8-U7 for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:09 -0400 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGbZUE029485; Mon, 16 Aug 2021 16:43:02 GMT Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3af3kxtg0t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:02 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeo5t159237; Mon, 16 Aug 2021 16:43:01 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2106.outbound.protection.outlook.com [104.47.58.106]) by aserp3030.oracle.com with ESMTP id 3ae3ve34y4-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:01 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by SJ0PR10MB4765.namprd10.prod.outlook.com (2603:10b6:a03:2af::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.15; Mon, 16 Aug 2021 16:42:59 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:42:59 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:42:59 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=Z+HYnZBTg8/iVO2ADTDXw+BbrtUybeoKYSv9IN97OaE=; b=An1LXz2+MAiGckgOlh/4RWEusKrzS3FrQhgkZc87qCen8WRpKdQ5kECu0oJiT8EA+2AR 1WX/T8IPqEtK+7S5BQufCKI0DlQTOPJjpuzjK56d9VclbFELVRDZ1ueiB56t+A7udLwZ GI1wnnEdXRAxoNlOjcK889/wh8KkEzm/mSrbcsoOqAk+rQyB4l08cGmMcXsgQTSmp2ME Fy8e3xPqvtvyg6SasGig/JksxfLSyTmKxqcq8BKIXMIhKJb29Ngd/AtubH3Q/DoS766x puisixNrgg0jjHtIEZPd/cGCYFzb4ubCTZR86IxKWEvGpSJ+R1znUBLLGTkFGNx2/CqL TQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=Z+HYnZBTg8/iVO2ADTDXw+BbrtUybeoKYSv9IN97OaE=; b=KMqLPWRGEGCKd9GkW5uz+/4cNOZZ95bE2SpZXMk9sQXcQ6kEK13sjsIET6cVxFksN9h5 n6cFdTJiLv4OyUgiR34hT373DV5WMR0AqeU5XMI3B0KmvYXK6I+sRxMNFarDZOsqYsaV QENKmuboY6Y690b+xgwniFSw58uPmt5T9GiqbMxsz17KT7/imFzJGg/FzCIuQn5pqUNy Djz4oj/YWcujmjPnkYI4MZnzx+eOThQhmRCV7vEx0FYApOmaVu7YyU8Dzt01yrvt1U9Z m2MvfN9W9iRYx+TBFZX795VUA3MrZHDKUvjbgdOsCOL4GAS14GSiMGz0spw4UP4Gkgrb hA== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AXz7yXqZ+T3fFrewj3I2eFbsnwuHY1DugQ6iiqCgzIE0fJHmt896QmWxk8TMtsI7e4ijEJPotbczkD/wNJWHpfN5YpMdQq1nfkxgYFFGbVwnFGrH4+Jb0uv1/gCllfNO8ZZBMEgs4J5Wz84iWAXHrWcAfCY+y9ix0nWj0qiT4Ojte3dzNVA3/zGhQ0aQzJSj5da0/wVHxEQ9BN8Phb0mMxy4jq2X2MjXxJK9PuQUxya2ROcxb6Uzm7IWNnutmWYlMq8iBoZNA/dBUNBvY4SlBS0k2gMtQjilQyAaadkrCfs2sKqk14n+5Uxdq/0MeHTdn2Zx2Lwwuopr/fhFYAeYWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z+HYnZBTg8/iVO2ADTDXw+BbrtUybeoKYSv9IN97OaE=; b=anROrCoSKPhxFnbyxpIfMM2aHxhqbYM1vpH4YEFxeaYbLeraizAKLaDB/6YtgWadPC8u9MWFL9IyPkpd6/mO2wz1NuUMnzZjOcldtOzaPSPe2ums/WThu0evWGNIy84dJGfiiJNLUnPPtVAUcYqUGz3+rq6ptK8dvqlfHEpQICsyG/JnpFm6QkgjPicbC/+nXSePKDPNsR5O+I3hzUfDgcn8XGSfGLZu3T1u57lGp9KGiqqnMEPEEb/lhclms2lPbzeXakYrSoqI/JY87Hf2wRpD5qvtMw06ep0Q5bdQmfq0J7ZFpUuSQ8+nAv7dFlY4G5z8ppXIS8AObfO2ue0wxw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z+HYnZBTg8/iVO2ADTDXw+BbrtUybeoKYSv9IN97OaE=; b=FS1Jr5A1v99ZmEHv2KHmgEa5oQYAKMNCLrsF4Rz/gVjUPoTcqnRW/J5ab9miw5vnOgCmpMQQwKyQNasUiYNKeDaDvQHsQa3hnDvaNZa0dGnh57o2nzBsB9Tc/9//lBm4NTYQpJ5tiskePb5yhICiJycC43eNXSPQt6WOnPLUCUc= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 08/16] vfio-user: get region info Date: Mon, 16 Aug 2021 09:42:41 -0700 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0448f41a-ba4e-4662-170d-08d960d4e846 X-MS-TrafficTypeDiagnostic: SJ0PR10MB4765: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:324; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JEZoHPer/nkBxQJisYyA42A1gmyEN/4ZTZNC9ZlWZXxIKagQjjSnx8EXrq8NPiJyHsAv4LiyHWM72xIVRs/lDkm/stOGOuZpTt9Gg5E9CbKWld4oARZqBZbk+lNTxxFPj1ad4oDfb+wBVxRq7WMk4BKje899SzgqMKDiler80eH8uLayPNysfnW+dSwjCKOsl15Wqkutp/duf5/O0IiXu+bY+UaLOF/yBijNF6AMiZ8HIRQT13B3He3BvuH4I11ywCUAKKrKaWp22k4icMgMOLX2HDzzeHRjo/avmrP73ogvud6gbIGbhqatRk86FEOuOlgVWRpZsGOF9uZpsvMZPurmn4W4xI/vWfWeWDvw+kXldCGKgy/nGZzutDPMxwmCbFN4iImzgCvuJYtiPwOeOda+1sARVX2hmbP2oqvG3sUwDERBM+WJczv1Zg8kHIZOOxyVqvB50x+QG3mT1N5SeZye56mOKtBRjwyGI7ESyODM/FGsvv11Ujuj8SGQV7wWflGAx2c95bOmVkj5eE7Y5+Y51UZ7uYbtKlog6iDLT1x5oPu/tZLWgATMQjboNQ1xc0G4bZhMoXVW1bQbcWnXgnHL/WwuMaeGqm7eJK8ggA+erLNgkXSrVYMMSbr7j47SH14iZVwOgZbvYFKFqLu4HQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(186003)(2906002)(86362001)(66556008)(4326008)(38100700002)(8676002)(508600001)(66476007)(6486002)(66946007)(6506007)(5660300002)(44832011)(2616005)(6512007)(6666004)(36756003)(6916009)(316002)(52116002)(83380400001)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ds4ImlLk4inMKv3nEv/3rd9sb777gf54GN8VDzbO9KRJXtFoMXDnIo/OdaL4?= =?us-ascii?Q?ko/JwK4lwrl94QZKxgK8zXKSgCDdZjcDLE1dggoAlnh4b/suug2n7kxZj0og?= =?us-ascii?Q?ccihSdxjhgc6iZnBmBAULu/4dLJO++53BQfUd46fXNEyTBU4mMSPg0RI47Vw?= =?us-ascii?Q?B0a/YMi8yrvbci3zLT43EU2I++df8zeO73UgvD+LBEeJ0BbTmY30gcMXgQ0s?= =?us-ascii?Q?3TuJuoE/ncQ4D3BG7bjx41r64ci+dRxYUhlt07bYCjly43w8NBCcYt//knQj?= =?us-ascii?Q?c8KlEuQx+BKOy633Ewk7bBV+Cd50RtZoizhD+R5WkKyep1yG3mlGoPE5AhWr?= =?us-ascii?Q?o0k9t0p2OLBL6eRWqHIkhInozHxZfbKhXe3/FPkJLd0ho+XHJP+aaLFxoK7G?= =?us-ascii?Q?o7p2mL7D/zFHIb1lAKyAGFUnHchNKR0bE2IXyCaEKBC0mQ88CWmmGHEm6eMC?= =?us-ascii?Q?X9ny+YL8DUnIh7tVWgRh2F1o5Mub3VdT5+SptYT8g408JHy3P8cKcYwnNUNh?= =?us-ascii?Q?NmNtuhpyB08XBm1yBz2Sm6iOwjMBb3/uiWjS0PtkBNhFla9Ec9MEgy4cuxDO?= =?us-ascii?Q?UxX1DP9NFKffp7oKK4ITRGE9/7IQO7lFMPAtVryrRyLP8h9OUUCZaQmAbYZw?= =?us-ascii?Q?xiUKwFc7TXR/BwssPWOuAh6MxVyY8FKgSjtoo4bUEbkaFn5Yhxxcff0tumTw?= =?us-ascii?Q?nI81j31QdCholNjh48lHdd9JxmfLa/yLzDK5VO6hkwBlU4WePAgn4X/LDQct?= =?us-ascii?Q?PWeeciAC+Tkrx+9eoBseZv10iI/7LoKnklrSsm+iabBuQ0sW9AN8Au1258F6?= =?us-ascii?Q?Im7iIPFKuqpSmPa2qTc7KS/uDZdwFexIQLkUWmfrJa10TrziE1eR20dpTTlq?= =?us-ascii?Q?F15iD+Sr1O2WtK2lORbVbisSyzl26nmVLBHg3qr9XN4rs/+bziNK1K8gAO8P?= =?us-ascii?Q?T8dEIbslbLo9xY+JUmx9EdmqUvOxw74/dJ9yTmHM227tYD0qpbrEClOnCVhy?= =?us-ascii?Q?Ai2E8LJqH103LHwrBDm+Lj7IyC4vaAnciJLKNMr+cE1DAY84RdUQWLxmaLwV?= =?us-ascii?Q?lEc/J60slnd8tMsiTWngHaQTeP76QloKqZsTC3DNLo+x6BBzoVveiaGgq6qi?= =?us-ascii?Q?7yss6/CUo6cZmfzB1My8Boc6ry+W2uOrPTQFYnTDa5Ls91hOxKT4BYMgr90h?= =?us-ascii?Q?4ey5ZolJ8t69FKaKNsC0NmeAFFxfJ24pfH3N5VkoSmscLDK/uhP9QiQ20mhg?= =?us-ascii?Q?oKfvIzXZ01bPYLhEDEFZiFK8a+i0PHm+Z2UdddmZyAYvVuwSlBljENs3PgYi?= =?us-ascii?Q?jtCjgdvu7mnrIwMR8G/kBBXz8zdnpAaD67drUa1Elkzy0tyz9KskxQ1JGQpM?= =?us-ascii?Q?iy/gLKa2Xfh1QzZPRkG++5wAne+F?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0448f41a-ba4e-4662-170d-08d960d4e846 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:42:59.7055 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PfCVIzAyztpbmOgLjV8helwtS4QD1Xn0IFxDRQPeJo3fDNyiJKTZMcqCr+BH3oUVnT7TnyXY4FY104gz1MC6XxkruzPC8PgvOifmeUyUq64= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4765 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 bulkscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-GUID: wsVNoaoUIxlHJvBjLZHiSh8-jXPNQxFi X-Proofpoint-ORIG-GUID: wsVNoaoUIxlHJvBjLZHiSh8-jXPNQxFi Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132710509100001 Content-Type: text/plain; charset="utf-8" From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 14 +++++++ hw/vfio/user.h | 2 + include/hw/vfio/vfio-common.h | 3 ++ hw/vfio/common.c | 76 ++++++++++++++++++++++++++++++++++- hw/vfio/user.c | 33 +++++++++++++++ 5 files changed, 126 insertions(+), 2 deletions(-) diff --git a/hw/vfio/user-protocol.h b/hw/vfio/user-protocol.h index 13e44ebf1c..104bf4ff31 100644 --- a/hw/vfio/user-protocol.h +++ b/hw/vfio/user-protocol.h @@ -95,4 +95,18 @@ typedef struct { uint32_t cap_offset; } VFIOUserDeviceInfo; =20 +/* + * VFIO_USER_DEVICE_GET_REGION_INFO + * imported from struct_vfio_region_info + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t index; + uint32_t cap_offset; + uint64_t size; + uint64_t offset; +} VFIOUserRegionInfo; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio/user.h b/hw/vfio/user.h index 82044e7e78..f0122539ba 100644 --- a/hw/vfio/user.h +++ b/hw/vfio/user.h @@ -72,5 +72,7 @@ void vfio_user_set_reqhandler(VFIODevice *vbasdev, void vfio_user_send_reply(VFIOProxy *proxy, char *buf, int ret); int vfio_user_validate_version(VFIODevice *vbasedev, Error **errp); int vfio_user_get_info(VFIODevice *vbasedev); +int vfio_user_get_region_info(VFIODevice *vbasedev, int index, + struct vfio_region_info *info, VFIOUserFDs *= fds); =20 #endif /* VFIO_USER_H */ diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index f43dc6e5d0..bdd25a546c 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -56,6 +56,7 @@ typedef struct VFIORegion { uint32_t nr_mmaps; VFIOMmap *mmaps; uint8_t nr; /* cache the region number for debug */ + int remfd; /* fd if exported from remote process */ } VFIORegion; =20 typedef struct VFIOMigration { @@ -145,6 +146,8 @@ typedef struct VFIODevice { Error *migration_blocker; OnOffAuto pre_copy_dirty_page_tracking; VFIOProxy *proxy; + struct vfio_region_info **regions; + int *regfds; } VFIODevice; =20 struct VFIODeviceOps { diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 8728d4d5c2..7d667b0533 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -40,6 +40,7 @@ #include "trace.h" #include "qapi/error.h" #include "migration/migration.h" +#include "hw/vfio/user.h" =20 VFIOGroupList vfio_group_list =3D QLIST_HEAD_INITIALIZER(vfio_group_list); @@ -1514,6 +1515,16 @@ bool vfio_get_info_dma_avail(struct vfio_iommu_type1= _info *info, return true; } =20 +static int vfio_get_region_info_remfd(VFIODevice *vbasedev, int index) +{ + struct vfio_region_info *info; + + if (vbasedev->regions =3D=3D NULL || vbasedev->regions[index] =3D=3D N= ULL) { + vfio_get_region_info(vbasedev, index, &info); + } + return vbasedev->regfds !=3D NULL ? vbasedev->regfds[index] : -1; +} + static int vfio_setup_region_sparse_mmaps(VFIORegion *region, struct vfio_region_info *info) { @@ -1567,6 +1578,7 @@ int vfio_region_setup(Object *obj, VFIODevice *vbased= ev, VFIORegion *region, region->size =3D info->size; region->fd_offset =3D info->offset; region->nr =3D index; + region->remfd =3D vfio_get_region_info_remfd(vbasedev, index); =20 if (region->size) { region->mem =3D g_new0(MemoryRegion, 1); @@ -1610,6 +1622,7 @@ int vfio_region_mmap(VFIORegion *region) { int i, prot =3D 0; char *name; + int fd; =20 if (!region->mem) { return 0; @@ -1618,9 +1631,11 @@ int vfio_region_mmap(VFIORegion *region) prot |=3D region->flags & VFIO_REGION_INFO_FLAG_READ ? PROT_READ : 0; prot |=3D region->flags & VFIO_REGION_INFO_FLAG_WRITE ? PROT_WRITE : 0; =20 + fd =3D region->remfd !=3D -1 ? region->remfd : region->vbasedev->fd; + for (i =3D 0; i < region->nr_mmaps; i++) { region->mmaps[i].mmap =3D mmap(NULL, region->mmaps[i].size, prot, - MAP_SHARED, region->vbasedev->fd, + MAP_SHARED, fd, region->fd_offset + region->mmaps[i].offset); if (region->mmaps[i].mmap =3D=3D MAP_FAILED) { @@ -2397,6 +2412,23 @@ int vfio_get_device(VFIOGroup *group, const char *na= me, =20 void vfio_put_base_device(VFIODevice *vbasedev) { + if (vbasedev->regions !=3D NULL) { + int i; + + for (i =3D 0; i < vbasedev->num_regions; i++) { + if (vbasedev->regfds !=3D NULL && vbasedev->regfds[i] !=3D -1)= { + close(vbasedev->regfds[i]); + } + g_free(vbasedev->regions[i]); + } + g_free(vbasedev->regions); + vbasedev->regions =3D NULL; + if (vbasedev->regfds !=3D NULL) { + g_free(vbasedev->regfds); + vbasedev->regfds =3D NULL; + } + } + if (!vbasedev->group) { return; } @@ -2410,6 +2442,24 @@ int vfio_get_region_info(VFIODevice *vbasedev, int i= ndex, struct vfio_region_info **info) { size_t argsz =3D sizeof(struct vfio_region_info); + int fd =3D -1; + int ret; + + /* create region cache */ + if (vbasedev->regions =3D=3D NULL) { + vbasedev->regions =3D g_new0(struct vfio_region_info *, + vbasedev->num_regions); + if (vbasedev->proxy !=3D NULL) { + vbasedev->regfds =3D g_new0(int, vbasedev->num_regions); + } + } + /* check cache */ + if (vbasedev->regions[index] !=3D NULL) { + *info =3D g_malloc0(vbasedev->regions[index]->argsz); + memcpy(*info, vbasedev->regions[index], + vbasedev->regions[index]->argsz); + return 0; + } =20 *info =3D g_malloc0(argsz); =20 @@ -2417,7 +2467,17 @@ int vfio_get_region_info(VFIODevice *vbasedev, int i= ndex, retry: (*info)->argsz =3D argsz; =20 - if (ioctl(vbasedev->fd, VFIO_DEVICE_GET_REGION_INFO, *info)) { + if (vbasedev->proxy !=3D NULL) { + VFIOUserFDs fds =3D { 0, 1, &fd}; + + ret =3D vfio_user_get_region_info(vbasedev, index, *info, &fds); + } else { + ret =3D ioctl(vbasedev->fd, VFIO_DEVICE_GET_REGION_INFO, *info); + if (ret < 0) { + ret =3D -errno; + } + } + if (ret !=3D 0) { g_free(*info); *info =3D NULL; return -errno; @@ -2426,10 +2486,22 @@ retry: if ((*info)->argsz > argsz) { argsz =3D (*info)->argsz; *info =3D g_realloc(*info, argsz); + if (fd !=3D -1) { + close(fd); + fd =3D -1; + } =20 goto retry; } =20 + /* fill cache */ + vbasedev->regions[index] =3D g_malloc0(argsz); + memcpy(vbasedev->regions[index], *info, argsz); + *vbasedev->regions[index] =3D **info; + if (vbasedev->regfds !=3D NULL) { + vbasedev->regfds[index] =3D fd; + } + return 0; } =20 diff --git a/hw/vfio/user.c b/hw/vfio/user.c index b584b8e0f2..91b51f37df 100644 --- a/hw/vfio/user.c +++ b/hw/vfio/user.c @@ -734,3 +734,36 @@ int vfio_user_get_info(VFIODevice *vbasedev) vbasedev->reset_works =3D !!(msg.flags & VFIO_DEVICE_FLAGS_RESET); return 0; } + +int vfio_user_get_region_info(VFIODevice *vbasedev, int index, + struct vfio_region_info *info, VFIOUserFDs *= fds) +{ + g_autofree VFIOUserRegionInfo *msgp =3D NULL; + int size; + + /* data returned can be larger than vfio_region_info */ + if (info->argsz < sizeof(*info)) { + error_printf("vfio_user_get_region_info argsz too small\n"); + return -EINVAL; + } + if (fds !=3D NULL && fds->send_fds !=3D 0) { + error_printf("vfio_user_get_region_info can't send FDs\n"); + return -EINVAL; + } + + size =3D info->argsz + sizeof(VFIOUserHdr); + msgp =3D g_malloc0(size); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DEVICE_GET_REGION_INFO, + sizeof(*msgp), 0); + msgp->argsz =3D info->argsz; + msgp->index =3D info->index; + + vfio_user_send_recv(vbasedev->proxy, &msgp->hdr, fds, size, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + + memcpy(info, &msgp->argsz, info->argsz); + return 0; +} --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132290; cv=pass; d=zohomail.com; s=zohoarc; b=MSpLgBJ46D8idvTdQ6+/QxXLagp1vTzC5zQI7RuFKWyAukQITWKtuzzbQ9XH+Mj36Jr6f/rTH0jaI0Dz2bp7s8jgEbdcF56HM8L6SBWx+/SyyHvnNJ0C46mxE8/NjoUcpcZAQQ6jSYgPtEfADxXLAIFVzetL87LyaRcjTH/R6LU= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132290; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=grNf/V8qZcO6D7heJPv97ZzLoNAJ/jpkLVPA5jc+de4=; b=jH+g7wVl2k0E0nyDNEETmB0DFE+ZqGLe719beF/GLNgcN7lwW5VXV9r5upfitN2bdo/VKhRVflBk+tjAmo/Q08z7TfdBSTF0l8AbUz6JVgZ5R96Og2ozMHK23UNSjk9OgEuWNeZGx/g9dsfPKABRi8693YDUYjiSrPu8rHvmBY4= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629132290596852.9270482722698; Mon, 16 Aug 2021 09:44:50 -0700 (PDT) Received: from localhost ([::1]:55616 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfjF-0003bS-Fy for importer@patchew.org; Mon, 16 Aug 2021 12:44:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37128) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhd-0000lw-O8 for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:09 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:48514) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhY-0007xJ-AY for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:09 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGbwOP029362; Mon, 16 Aug 2021 16:43:03 GMT Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3afdbd1wrb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:02 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeo5u159237; Mon, 16 Aug 2021 16:43:01 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2106.outbound.protection.outlook.com [104.47.58.106]) by aserp3030.oracle.com with ESMTP id 3ae3ve34y4-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:01 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by SJ0PR10MB4765.namprd10.prod.outlook.com (2603:10b6:a03:2af::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.15; Mon, 16 Aug 2021 16:43:00 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:43:00 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:43:00 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=grNf/V8qZcO6D7heJPv97ZzLoNAJ/jpkLVPA5jc+de4=; b=Ezc3Bnk9jFKsgzi4fhgZadx1A/Gxf3tVoyqxN7ZhmH1khtn0FIktobxcImiistCSov7P 1m1lqfzhwjbLMBvyiqSZVmzLlXxr5gGg0Yuv5N+BwCNfEOgnsXLoteTi3kgReaB0KDiq 1Q8+RpMF/+t105Borh4ZLCy1SoGyLbNTLXhqPlhUmaaOl/DzCsRF6pY+nsX/cJaGeYlk Nha8KgYVEVwOv3Ymj0YaTPA9zRKninsNFRGCcip+amLeLWpbFWo5lMirLHCJzgZhvvoi kQuaW6xC7bPIYZGQBGxG362y5LktW9NF7eVD0SEpPTHWHlnIMsWaWWKSqnT9YfdRFV7c tg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=grNf/V8qZcO6D7heJPv97ZzLoNAJ/jpkLVPA5jc+de4=; b=UFoYG92JwuxOVEeAHAg3WvzceEta279i+yxdXh8XS2Vjpu6ViQSb9q+X+9sJdqAikFmL S7nqFhs5hW3CRbDLdAIgS54+nFWVd9Qei9jN2xnKQENLloBUHGUu6zj+gOpeAH38GY1n +hHajiIBvmQpQ0DF5KpYuunPbXOrEnyhIyCceRJw13l8C8lqjXevimQLdFA7MsOYGnA+ 0Qa4VOHOsbCb06/GJSz34WmWBS1ECTKdQ2ZiNqF9Ti74qF+mjB+SDdzzOyBO/Faz3NjP v+hF+2ctdJUUYSS0Lvwbzu1+YRicl6+n7mCsu3uzIyEtK0Aco0P1KsrtZhY0rAHEXduW oQ== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CpKORUEjqDFoRNNo8cRksLU45jvTKYwrGbjRex2sz93+JJ62GFtBVvwWw1FOk3Cizqk92dIoYaX/Bd9eomRZFZD8TSi7alutYYKbSCftikbM9qMi60jtw5Hf2DuUqaTKVu4A1uSpBCpGyBXRePd80dCyzvfF0GgXzqcrfonV01f14vez+YaLsuC55BMYoR48PPWQwDNWXETuHaU4zEEO973uSk6apfJ17HYiuMptIFx+KcPMld6fJmsLHSk+VZuSRCzGPrNWhy54XUbHcfPttc+kyhDO/0f5APqhjkv8GdGvpLkfmc4b5FqkMgN0fffT0gyhjUpNCEqYAgT4Aecryw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=grNf/V8qZcO6D7heJPv97ZzLoNAJ/jpkLVPA5jc+de4=; b=UjS1JxM/NajHMLvL9jptt9wzr8I6BkeXPRl0CuDsmKyZLf2+wmkaysXNj5S0QCLiCpmcqDaxUAlvbb2cHcSSGTxrUYTdtXoqxSU1+AFJYBfTMDSUvPBdiCzTeW33YslKuc780rITAGCNhj3boZgeaqle6Jfrv6DRNWluHvKiBOQKR1H4U0MJe72M0yutQ8Xeh6U8Gw87ekzDs597O70NX24HaiECcbXMmFH5j/tcZyUJHycLaJsL097iYUEnhUCX9rY8UxUCtYIOWKrghcoeSldP2ScnFqGz7uh9+S/y3Z0umdNOW8Lt9n1sAPjOELKxAZrOExInH7B+qi7C2c5VNw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=grNf/V8qZcO6D7heJPv97ZzLoNAJ/jpkLVPA5jc+de4=; b=piLV5m2GwP75nKOLEEIIhfAdj0YI3pfPKduKl8UWbOMxgOT7PvArG+WxQoyywYUnD0jnBgaK1sbBzFvghABew1XARRNJeKcy9UuZ+t3qpSpr+8RpucifYqEQgMbBts17zh2Rf/tM+GXhixK3ou4InmZfR/g/K3ak8x1kXtQAYuw= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 09/16] vfio-user: region read/write Date: Mon, 16 Aug 2021 09:42:42 -0700 Message-Id: <92fb16181e71a1c4049f9995294dbd7ff4270627.1629131628.git.elena.ufimtseva@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 92af5206-e312-4b99-702c-08d960d4e8a1 X-MS-TrafficTypeDiagnostic: SJ0PR10MB4765: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:78; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AeAs5MlZgHgobQGGj5eXIWj3a94NW5KU03vl596IRFPi3Z2aU3nlOYmuhuWEZaLZab+5Cm8wqHZNjuI7E5Ri2IvnA52TZVdUeY8DdPoDQuoh8IRIhC6HlDx9INhuUvAJpIy9x+hhSzpffiSKnZtq29f41Kf0dQNsofIQQ0Xi3veIELIWOoVwkBIVXb0EkNWwd0CL/8I2sLbavz1cSK3p4CSJztsWV1IrOWwozO7WzfczEbauSWNl9CD/YuCyYUESEss+s4NA6b1jl6JBRIBiD8MAHIy/JB0WuPJE8ZQtHA30QxSxfyc1wCbEE4/DCUlugbdO38XxLzHMr6xD9ApRsPpkd7yNq27jUHrfPkrQkJwaI/3aXYtOV69+IYZKP6wZ/vUDoOEYwTcBXTC0fyd9djP/Ed78+LSzkYXeFOHfZHiwpkaFEaOWUfyRirZEK552tAFk8WpPxwWqQE10WyK9EvFaugCdh1KX8SqwMr8SnD0QZhE4b7sgha5/3HrH2VUYoYpTzWRjDrIPlZQzgmIy6wnTzQyA07fVmwAekm1HKR/NJ1CgAYwIUlu9b6G06Q8AgLjHFwUPsLMuIxwzNOr3hFVL+xqI7qwFX1eOcknNotiwP8OGCtbBoAIqFEZJavU2kVgWFAIeTNDSdVL23K532g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(186003)(2906002)(86362001)(66556008)(4326008)(38100700002)(8676002)(508600001)(66476007)(6486002)(66946007)(6506007)(5660300002)(44832011)(2616005)(6512007)(6666004)(36756003)(6916009)(316002)(52116002)(83380400001)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?YZtnBy2DDANasd37hsdW1D6Sut8F88gd9g47pFNuLNR2rhRxZrl//uiIm3Ns?= =?us-ascii?Q?LyjxOoJf2Tv46y9I/bgId6i07tq7HvDf9JUeWE95RJGh5oeuQ+x2seWfDOOH?= =?us-ascii?Q?AmrGLFRKNULxgKu0ct0WiK9WW5R+i8ErIEURbfevGlli8jJbzGOU0yZSTDOd?= =?us-ascii?Q?gL5XtixYUbI+stU2v0InPQat4wF/ToJlF2F/GAzfuyiDTy9e/mp30jQPy2lF?= =?us-ascii?Q?1+ELge8FAl2ZSMmG/Hm78VlOW2it7+gWYmzxD2/cxhCFytmEszJ1UQs2zIXh?= =?us-ascii?Q?X5xdDiwLiuCuP5q4Wx5oiL+RKUcFg9nvr/KL2lpB4rNx0dY7heCYuVvhK/dL?= =?us-ascii?Q?BS5YUdnNQfVwI+y1cn6s+UrtQ5BqtUlH7o6G4g6PZ33DGAdFGO6ErF1NQ1BV?= =?us-ascii?Q?K6Ue4FrQmqTOHrY2IDIcxhVyPwkcCYAEEnmCBKK0EQ3O4FvCOJWcAmOB0wUw?= =?us-ascii?Q?uMNF4jYcGbeoR2sHrCs3/BnpeqHyJ1lhQwZJ2c4DIDrpP6babwrCnPY+vRZ6?= =?us-ascii?Q?C/UUUjCLdRy6YhZGasFAlAwqJzWwLE+xPYf6/pP2TaOBY7qC0y9u9kzXME4R?= =?us-ascii?Q?4wnYjKBmYWOaxqk42tY4D7tFSN1Ezinriie4BssGtMfhWoKT1eYadFyQy5RM?= =?us-ascii?Q?CqLlPkqyxuSwp6/SV9R7U+YJksSe0+QPqlw0jKh8PwIATV3aefII4N2COs7J?= =?us-ascii?Q?JQqqAyJoEiZgwiK2t+8sHnR7J2C4U2Xx4ergkJAyUDsXInMImBCSdLTKVm71?= =?us-ascii?Q?cJkag9nHc5/oo6sJWwHbrvBlj0FwPYaGNos3pKGbYv0IxFrU4SBKwNLqRy6B?= =?us-ascii?Q?duQjTyWHdPwof1u2kJcIOrhhzxiYuOkTcQP0laNIRK905EY0sIxyiYbfsJo8?= =?us-ascii?Q?crdXPcmbY7zySpLQsjgGSmHN5WjvMI50Ni5tjSJEsgsV79RP5eFCHKBzgIlc?= =?us-ascii?Q?01V75XDGzZ3zrKY3XAGD+UP+ZTeR1PSyK01Wte8HBQ/Uty+QZdo7Fbae1/FQ?= =?us-ascii?Q?MJfN5ivtnLrTgO3NxbQ9nXLMnYx2vzlCAMdihmjscL9umCjWUlT/I6mJstkU?= =?us-ascii?Q?JExvo7oVPGqNm0Yw77bJSWcvDBwC1ZkMZ/AXAcl2vDZUlXYmsKgbuShgFHxh?= =?us-ascii?Q?WG78JyRNOILKohBOb7I73k4Fq4+CpH0CiFgXqN7iLD7qAk7hQPT7wqTXnXju?= =?us-ascii?Q?3szYHMJBTYhATxAVVUOrhx5qI2++ukC8+z4nCkT699r7YgcnX+QMmct5P+fh?= =?us-ascii?Q?YYrcoh5vGbpqaxSUpT/RERdi4O9cIk+1KftuFPZLns1vn/9kbgqi+zzbn10j?= =?us-ascii?Q?IhFg2/5pEQvK6XAmEx3QGjg0K0RWxoGngKGrfTeKLPaxhW8P95Kwak9X8Aon?= =?us-ascii?Q?X2MT/ARsZtXz2XB/PEMwscN+8deg?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 92af5206-e312-4b99-702c-08d960d4e8a1 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:43:00.3182 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RgdWpWy558yQyAoc41wRK+AVonTsXymTjusJ2Lbdio6M75oDT5xXJJkxjl6o/ulAdq/mSGsKlrmVktIblj6VnEyMBBc11IshDdTFn/rvOlg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4765 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 bulkscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-ORIG-GUID: ymYekk42WEl1Z8yo9NPhEw2tR022hJBt X-Proofpoint-GUID: ymYekk42WEl1Z8yo9NPhEw2tR022hJBt Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132292512100005 Content-Type: text/plain; charset="utf-8" From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 12 ++++++++++++ hw/vfio/user.h | 4 ++++ hw/vfio/common.c | 16 +++++++++++++-- hw/vfio/user.c | 43 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 73 insertions(+), 2 deletions(-) diff --git a/hw/vfio/user-protocol.h b/hw/vfio/user-protocol.h index 104bf4ff31..56904cf872 100644 --- a/hw/vfio/user-protocol.h +++ b/hw/vfio/user-protocol.h @@ -109,4 +109,16 @@ typedef struct { uint64_t offset; } VFIOUserRegionInfo; =20 +/* + * VFIO_USER_REGION_READ + * VFIO_USER_REGION_WRITE + */ +typedef struct { + VFIOUserHdr hdr; + uint64_t offset; + uint32_t region; + uint32_t count; + char data[]; +} VFIOUserRegionRW; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio/user.h b/hw/vfio/user.h index f0122539ba..02f832a173 100644 --- a/hw/vfio/user.h +++ b/hw/vfio/user.h @@ -74,5 +74,9 @@ int vfio_user_validate_version(VFIODevice *vbasedev, Erro= r **errp); int vfio_user_get_info(VFIODevice *vbasedev); int vfio_user_get_region_info(VFIODevice *vbasedev, int index, struct vfio_region_info *info, VFIOUserFDs *= fds); +int vfio_user_region_read(VFIODevice *vbasedev, uint32_t index, uint64_t o= ffset, + uint32_t count, void *data); +int vfio_user_region_write(VFIODevice *vbasedev, uint32_t index, + uint64_t offset, uint32_t count, void *data); =20 #endif /* VFIO_USER_H */ diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 7d667b0533..a8b1ea9358 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -215,6 +215,7 @@ void vfio_region_write(void *opaque, hwaddr addr, uint32_t dword; uint64_t qword; } buf; + int ret; =20 switch (size) { case 1: @@ -234,7 +235,12 @@ void vfio_region_write(void *opaque, hwaddr addr, break; } =20 - if (pwrite(vbasedev->fd, &buf, size, region->fd_offset + addr) !=3D si= ze) { + if (vbasedev->proxy !=3D NULL) { + ret =3D vfio_user_region_write(vbasedev, region->nr, addr, size, &= data); + } else { + ret =3D pwrite(vbasedev->fd, &buf, size, region->fd_offset + addr); + } + if (ret !=3D size) { error_report("%s(%s:region%d+0x%"HWADDR_PRIx", 0x%"PRIx64 ",%d) failed: %m", __func__, vbasedev->name, region->nr, @@ -266,8 +272,14 @@ uint64_t vfio_region_read(void *opaque, uint64_t qword; } buf; uint64_t data =3D 0; + int ret; =20 - if (pread(vbasedev->fd, &buf, size, region->fd_offset + addr) !=3D siz= e) { + if (vbasedev->proxy !=3D NULL) { + ret =3D vfio_user_region_read(vbasedev, region->nr, addr, size, &b= uf); + } else { + ret =3D pread(vbasedev->fd, &buf, size, region->fd_offset + addr); + } + if (ret !=3D size) { error_report("%s(%s:region%d+0x%"HWADDR_PRIx", %d) failed: %m", __func__, vbasedev->name, region->nr, addr, size); diff --git a/hw/vfio/user.c b/hw/vfio/user.c index 91b51f37df..83235b2411 100644 --- a/hw/vfio/user.c +++ b/hw/vfio/user.c @@ -767,3 +767,46 @@ int vfio_user_get_region_info(VFIODevice *vbasedev, in= t index, memcpy(info, &msgp->argsz, info->argsz); return 0; } + +int vfio_user_region_read(VFIODevice *vbasedev, uint32_t index, uint64_t o= ffset, + uint32_t count, void *data) +{ + g_autofree VFIOUserRegionRW *msgp =3D NULL; + int size =3D sizeof(*msgp) + count; + + msgp =3D g_malloc0(size); + vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_READ, sizeof(*msgp)= , 0); + msgp->offset =3D offset; + msgp->region =3D index; + msgp->count =3D count; + + vfio_user_send_recv(vbasedev->proxy, &msgp->hdr, NULL, size, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } else if (msgp->count > count) { + return -E2BIG; + } else { + memcpy(data, &msgp->data, msgp->count); + } + + return msgp->count; +} + +int vfio_user_region_write(VFIODevice *vbasedev, uint32_t index, + uint64_t offset, uint32_t count, void *data) +{ + g_autofree VFIOUserRegionRW *msgp =3D NULL; + int size =3D sizeof(*msgp) + count; + + msgp =3D g_malloc0(size); + vfio_user_request_msg(&msgp->hdr, VFIO_USER_REGION_WRITE, size, + VFIO_USER_NO_REPLY); + msgp->offset =3D offset; + msgp->region =3D index; + msgp->count =3D count; + memcpy(&msgp->data, data, count); + + vfio_user_send(vbasedev->proxy, &msgp->hdr, NULL); + + return count; +} --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132892; cv=pass; d=zohomail.com; s=zohoarc; b=LrFozZgyihfemaGFYwraFR6HIo4ymOa0rYnPHuBcw7yChwiiqcBoErlxl/3rpB4WbWWwZbdVIMPh/yNxj1s5ven7OHFvuMh3LOOK8lXaC72XWsgAYr+Ibut586Q3qj4x9jldy38w82cD/kq3tHhhOQVYmJFgeZqIKatq/eI0FF0= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132892; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=D4ZqAyq4SE/ICj/ozzOgWBRxaqFlE2xHi8oF3tQZWe8=; b=kg321fYFTJ6MLRVoQIZ5vsk+775ZosMpZ6KJJnIbOXo0pgQqskRj24GCQu9L2GES8YQVErv52hGZsY1c9d1chRJKCPyvHEP4nA4wj2QHoFNVFI0QnN2hd7pemy/HUsRGuul2GsXs91PIXMlGnfxSeJS5ddWoIPS2HjZzWjK1pZw= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629132892971141.58558356372737; Mon, 16 Aug 2021 09:54:52 -0700 (PDT) Received: from localhost ([::1]:54032 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfsx-0004RZ-Ah for importer@patchew.org; Mon, 16 Aug 2021 12:54:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37152) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhf-0000qg-A9 for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:11 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:37958) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhY-0007xg-9l for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:10 -0400 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGZhse008102; Mon, 16 Aug 2021 16:43:03 GMT Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3afgpghjpt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:03 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeo5v159237; Mon, 16 Aug 2021 16:43:02 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2106.outbound.protection.outlook.com [104.47.58.106]) by aserp3030.oracle.com with ESMTP id 3ae3ve34y4-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:02 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by SJ0PR10MB4765.namprd10.prod.outlook.com (2603:10b6:a03:2af::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.15; Mon, 16 Aug 2021 16:43:01 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:43:01 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:43:00 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=D4ZqAyq4SE/ICj/ozzOgWBRxaqFlE2xHi8oF3tQZWe8=; b=vbwLr3kFKbTRlDuI64RscayhLij23ulaHPWrFiU3SPDQkJM2hb32IQhKq3NeJvFC4W7G vu2uGnbwHJs0kZAfwLa5ARhqXSDry+o7CxicBRgAC2OyCHhp/rV9EvLr9rN9sKn+2gau qYDbtjUpYXRdLAHeisz+9JqsftRUaSwueKTGO0iweEDpnsJgE20NdOmjFu7zD09QBUQM HiXZULaiuQNZgJgeVqEa1szfNhoa+rSrfLRdTbDSmyp58sE0tA8AkKRZdBDGfaVFFhM6 FE+6vuB9ljCseurNGCF7t210G4RejY3mw7Vgcg6zYwSzr67YqDp83gLHtUCo/nAb3HVI nA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=D4ZqAyq4SE/ICj/ozzOgWBRxaqFlE2xHi8oF3tQZWe8=; b=zpqoUqy9n/uAa+btmsD0g9Bp+eFVFHYTJVOBfwZuPG4G9rH0fX/7Ia5dL3AhRTAXVIzh bhPwFvLo+wbI8l6kbVPiyvp1hVXef0im2C9SBThTkqS9tbNMdljr70vEGufALNWgHELs HeGSDqaOiY/n6OTeBxCMH3uhTCE19q4flnN01TI2IVOwglPxEB0DzBqKXmlK+kXS/qPM NYhJahOIWQ/fdb41p+f66iFd/QYJ8wlpRBQ9CUvhKbGPmJ1aod/deGbWHcoUk+T/36Qr GJFQlxATWvK6vYv+4PcUU9Jg9qeJdqjLH238QqXwfT5nZS5YSX/1UDq7eKMwZB+ey2lN pQ== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LfULiFQ/6ehwivAjs+yoCAZFXKU7w37DwWrKSUJPPMyBDG8UqeLY9I4cg38hwWhaDhUJLq7aZzzG4CC+gA4sXSpz9NH/edw0ic0LtKD4szG2l9vOppZZuB/uk6Eqrr65JWOCUiIqDGoVt545J7SZ0BkT/wqP4DkSMMxTdMBmkKzqR/ynXef2AcIziZftuHhMAgIF7DftvR0kxvzK+3UhMDLawiY+3XOks7ba4zQ5Wdp48pUDhvb7Iwh3PVK88OmWcfEEGW5rVzY4DgAAjMR1EGmJEPMa1cZjO41bGT6AxRzGEgaApyafVDyIPgEAdfmjHZLWEdDbXEAvAEhyjLgCpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=D4ZqAyq4SE/ICj/ozzOgWBRxaqFlE2xHi8oF3tQZWe8=; b=eyLH7X59JGqijT8elRKhJD/1BHd8MrB6zSAIxxvWmnaQqoHmUsdSviBjcrrGvfIkM6JYDXJVrfLvRYJaVKkMw5SXYpYfWwcYb3fCI6/JYYfJYnS8mhfDbN/Qs5DZ2qCnbzf/sP7+bT2RVtsfBZ+o1S9L4wFljmrqYaDTeGdjMKFu6akuZswRslxXxzeQNw3ErWxET4vnisyB6+Xp9Xbq38k6ml++LxgGvD+PesumpYHj9ERZa4jjAju2sa6IOACoO9eUXE5Uw6m9JHxKgzEQs/BlnLjLT8zVeHDRX/Rf1w2kMIekU+RLq3vP6q010LryhAo5NcaNPtqoMDwTVzwoPg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=D4ZqAyq4SE/ICj/ozzOgWBRxaqFlE2xHi8oF3tQZWe8=; b=zNv/VQtiwRemo0ZBRZ+6RXJHHSGyVfxvXFC9qiA4isB0pk7yBXoFnQpQiu/H+V4eZiT6fMG3chA30RE5fSIdTnmDhh7OjgpprY6at2gvqV+kpw45SAI7ar0RAo6DpvQPcerOu6reFJUZABhO7Yzv+FzFpq3bmemFbmCGtN5S5dc= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 10/16] vfio-user: pci_user_realize PCI setup Date: Mon, 16 Aug 2021 09:42:43 -0700 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b888b83c-7149-45b9-a241-08d960d4e8fc X-MS-TrafficTypeDiagnostic: SJ0PR10MB4765: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1201; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4IKBKuJt4QWdpYBA5zdfB3EbGHlR91ziNefq2Uj61+FVAuAQObL5071IEBMlrEDGfhxkWBLFxWJW3C0mDO+7uYUtVuW5f+2LxxNzan3JDcKNJjoxe2rWLzmNHYQNsx9BUpjmXOtv6eVl+0bD0D3MEqeYQjZP+E2J4MsT+cmc1m0UogVNFlohZEk1AHaZom4Q2Pu0xQNTL7nK1MSNy0h2XbnwsRNY52j6ONcVN79zaGTEvxaarEp2e3a/iTDF5AZegkRt8qwSWH8h7YVz1h2lwBpW9TD55CN2LM+alJ3RY1y62psufQYX4GuW/5Wyj5s+2ScaYLSovlJQJfVQ3IFeLl1z6UEVWChdMzmK5Oe2FYX0L2ThAL/OOdMiHLNA73SP6RBRzIE14xYsEoQerpxGg2+lbIl16EMTWu2H3N67vXOajmKC04yExuCpzKwisc/aHShwLesEMgA6xvJeqDjGVREQFasFUP/gqiOxaqL58qo8X9iHLggs17A0m2iBu6N9nwOvK25va0FNR6m2nDE+FHnZ83RJS3h2iHF8TMaQv8dYXwayoyv0x4CFDxx0VKvOMKnT7UT56eiFtMB2wxFZKRjjNbRAxTt1HC1tD95Gf59HGYzdrWaAOm3js2XOxnDu7lwzkZf09Kj1PJtoaxu2Og== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(186003)(2906002)(86362001)(66556008)(4326008)(38100700002)(8676002)(508600001)(66476007)(6486002)(66946007)(6506007)(5660300002)(44832011)(2616005)(6512007)(6666004)(36756003)(6916009)(316002)(52116002)(83380400001)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?4XOSCZZQpgO9V4bUKxzxndPMZlyRjr2VW0aFopvlMvf6wl7rEtUlq03+CyAY?= =?us-ascii?Q?k6XMloUqZJwBduh9/1tMWqb8IF1evlQKieSNZHMcnC8Rl9KLF7uarsoROh34?= =?us-ascii?Q?Q1CerEl+a4jRJNDj/RDNtVz5sQ4uDC3s0LC25fgjgzU5UlGeTDCtLvfSkXRQ?= =?us-ascii?Q?ZQgiyFEVawy+S+5IDEKvNIDyZx9ShM2UZElaX29V/GkslHzXo+v6tm4fmcsW?= =?us-ascii?Q?ZCc7yq+fKRDUvZhcGw7eYBjUdQKwLJRgjr/IRyA1B0mH4rT3ql8Q3FmhTfjn?= =?us-ascii?Q?IHoDnSbMhO6q4mRRJG+PneKVAu4NvuY8eZZ0lU2MImLzQNb8BCDzfykvT4ea?= =?us-ascii?Q?bdcXtH9YumZOYgpiEi3JRu1613bomhGa81EeR8JS3qNb737UsedFJxqKWKOo?= =?us-ascii?Q?IdLseqD+4JbOBCLzjVWM+OdDwNsk+sIU8j/rweiBELLIxcEUnkHRLmCyLM98?= =?us-ascii?Q?Fm88mX92Eo94mwKnVNxUgKre+BsfwJDnw8fOtvqryT2fU6AAv1h6AXMTcls+?= =?us-ascii?Q?h8jdipIjwmKQlDzi8sv0xeMU3tSJqbmiiFX6LAgqmepZvGv2s5bJyjYn+bzH?= =?us-ascii?Q?v9PVfugXNZlX97yA7o7gk47lbN/IWXhpwY312Z3BpJI5E0ZvqUxlrstL+lyO?= =?us-ascii?Q?W0auCJa7MPaQdMBYfQxWyDDA70uz+vSOPW4Tn/eUfL0XHtQoKTfSRWutk2sY?= =?us-ascii?Q?57nnKjg7aJWlwPOkDiWq3b9XdQN6ozNLs33q2At2ucLPI14iQPjmxTEf2Z8z?= =?us-ascii?Q?0ML1EoQ6kcLYgEjDfUApEQVySnPjh40uxkd9f+KLVBhH5L+SPQStr2eKrENJ?= =?us-ascii?Q?cQWF3ycdh+b3RL8Wxqz0uYcbbpYrFVURjy/ACKbzVw+NfmfUxsquAWksj/px?= =?us-ascii?Q?F2wREfpIiXvzVPB6SVMcYblbI8XmDSk2v1A5EinEtW8dgRN5Vq43NhjUTG18?= =?us-ascii?Q?8H1yEGBbTjuwUvnrO48S/InL0FU/XoZmvVcup8UuZvd2C5c36qIGzoYJ8oxI?= =?us-ascii?Q?lcF9HcwjN/jzrhKkHTeDi0JbN7kGHQoYwDvKFOZ2x3OoOWBLJEZSAqwznLxs?= =?us-ascii?Q?LO6YrKifnUA58WZoQcYah1gGK6biiUPNvn+zzjfr0EtORrJ5p+Y4arKnuV9h?= =?us-ascii?Q?b+u3AyrbV2im4JQhvVlSjD2pKEJQ5h5EL2LFkcjTJbDV9bL7OKvi52lzV1In?= =?us-ascii?Q?K8LOPC4+H8IIAzZFZsWPVABf0Y4NSoe86c0aUqZfC074i0/M3XB9DREGjBtM?= =?us-ascii?Q?nAtJ02C0cPmzD8eicq+T8R0OoPAl+lz+g3VfvwWT2hXPzYVIBs7r7H7EuxSl?= =?us-ascii?Q?Z1JLmUpBnklZ/8SY+3EkUxgbMW5XzUXawVRz2zIhoHudO73Xq0nZcmpXuIUt?= =?us-ascii?Q?olt8XavCDjxDIlDK58EvItUqSH5f?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b888b83c-7149-45b9-a241-08d960d4e8fc X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:43:00.9298 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zSOn4p5ocsv1DHd/YO2apmDElhdqMX15BaM/pDkbukK6mp14r9IUtdYS86mXji7ZbnKnUdX+9EOR1+uU2XM5XB729wtwh71a3JW1v51vxoo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4765 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 bulkscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-GUID: mW2HA9vHsKV2Rp7bDGmaU9vQle4qp64t X-Proofpoint-ORIG-GUID: mW2HA9vHsKV2Rp7bDGmaU9vQle4qp64t Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132894662100001 Content-Type: text/plain; charset="utf-8" From: John Johnson PCI BARs read from remote device PCI config reads/writes sent to remote server Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Reviewed-by: Stefan Hajnoczi --- hw/vfio/pci.c | 210 +++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 175 insertions(+), 35 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 63aa2441f0..ea0df8be65 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -807,8 +807,14 @@ static void vfio_pci_load_rom(VFIOPCIDevice *vdev) memset(vdev->rom, 0xff, size); =20 while (size) { - bytes =3D pread(vdev->vbasedev.fd, vdev->rom + off, - size, vdev->rom_offset + off); + if (vdev->vbasedev.proxy !=3D NULL) { + bytes =3D vfio_user_region_read(&vdev->vbasedev, + VFIO_PCI_ROM_REGION_INDEX, + off, size, vdev->rom + off); + } else { + bytes =3D pread(vdev->vbasedev.fd, vdev->rom + off, + size, vdev->rom_offset + off); + } if (bytes =3D=3D 0) { break; } else if (bytes > 0) { @@ -927,12 +933,28 @@ static void vfio_pci_size_rom(VFIOPCIDevice *vdev) * Use the same size ROM BAR as the physical device. The contents * will get filled in later when the guest tries to read it. */ - if (pread(fd, &orig, 4, offset) !=3D 4 || - pwrite(fd, &size, 4, offset) !=3D 4 || - pread(fd, &size, 4, offset) !=3D 4 || - pwrite(fd, &orig, 4, offset) !=3D 4) { - error_report("%s(%s) failed: %m", __func__, vdev->vbasedev.name); - return; + if (vdev->vbasedev.proxy !=3D NULL) { + if (vfio_user_region_read(&vdev->vbasedev, VFIO_PCI_CONFIG_REGION_= INDEX, + PCI_ROM_ADDRESS, 4, &orig) !=3D 4 || + vfio_user_region_write(&vdev->vbasedev, + VFIO_PCI_CONFIG_REGION_INDEX, + PCI_ROM_ADDRESS, 4, &size) !=3D 4 || + vfio_user_region_read(&vdev->vbasedev, VFIO_PCI_CONFIG_REGION_= INDEX, + PCI_ROM_ADDRESS, 4, &size) !=3D 4 || + vfio_user_region_write(&vdev->vbasedev, + VFIO_PCI_CONFIG_REGION_INDEX, + PCI_ROM_ADDRESS, 4, &orig) !=3D 4) { + error_report("%s(%s) failed: %m", __func__, vdev->vbasedev.nam= e); + return; + } + } else { + if (pread(fd, &orig, 4, offset) !=3D 4 || + pwrite(fd, &size, 4, offset) !=3D 4 || + pread(fd, &size, 4, offset) !=3D 4 || + pwrite(fd, &orig, 4, offset) !=3D 4) { + error_report("%s(%s) failed: %m", __func__, vdev->vbasedev.nam= e); + return; + } } =20 size =3D ~(le32_to_cpu(size) & PCI_ROM_ADDRESS_MASK) + 1; @@ -1123,8 +1145,14 @@ uint32_t vfio_pci_read_config(PCIDevice *pdev, uint3= 2_t addr, int len) if (~emu_bits & (0xffffffffU >> (32 - len * 8))) { ssize_t ret; =20 - ret =3D pread(vdev->vbasedev.fd, &phys_val, len, - vdev->config_offset + addr); + if (vdev->vbasedev.proxy !=3D NULL) { + ret =3D vfio_user_region_read(&vdev->vbasedev, + VFIO_PCI_CONFIG_REGION_INDEX, + addr, len, &phys_val); + } else { + ret =3D pread(vdev->vbasedev.fd, &phys_val, len, + vdev->config_offset + addr); + } if (ret !=3D len) { error_report("%s(%s, 0x%x, 0x%x) failed: %m", __func__, vdev->vbasedev.name, addr, len); @@ -1145,12 +1173,20 @@ void vfio_pci_write_config(PCIDevice *pdev, { VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(pdev); uint32_t val_le =3D cpu_to_le32(val); + int ret; =20 trace_vfio_pci_write_config(vdev->vbasedev.name, addr, val, len); =20 /* Write everything to VFIO, let it filter out what we can't write */ - if (pwrite(vdev->vbasedev.fd, &val_le, len, vdev->config_offset + addr) - !=3D len) { + if (vdev->vbasedev.proxy !=3D NULL) { + ret =3D vfio_user_region_write(&vdev->vbasedev, + VFIO_PCI_CONFIG_REGION_INDEX, + addr, len, &val_le); + } else { + ret =3D pwrite(vdev->vbasedev.fd, &val_le, len, + vdev->config_offset + addr); + } + if (ret !=3D len) { error_report("%s(%s, 0x%x, 0x%x, 0x%x) failed: %m", __func__, vdev->vbasedev.name, addr, val, len); } @@ -1240,10 +1276,15 @@ static int vfio_msi_setup(VFIOPCIDevice *vdev, int = pos, Error **errp) int ret, entries; Error *err =3D NULL; =20 - if (pread(vdev->vbasedev.fd, &ctrl, sizeof(ctrl), - vdev->config_offset + pos + PCI_CAP_FLAGS) !=3D sizeof(ctrl)= ) { - error_setg_errno(errp, errno, "failed reading MSI PCI_CAP_FLAGS"); - return -errno; + if (vdev->vbasedev.proxy !=3D NULL) { + /* during setup, config space was initialized from remote */ + memcpy(&ctrl, vdev->pdev.config + pos + PCI_CAP_FLAGS, sizeof(ctrl= )); + } else { + if (pread(vdev->vbasedev.fd, &ctrl, sizeof(ctrl), + vdev->config_offset + pos + PCI_CAP_FLAGS) !=3D sizeof(c= trl)) { + error_setg_errno(errp, errno, "failed reading MSI PCI_CAP_FLAG= S"); + return -errno; + } } ctrl =3D le16_to_cpu(ctrl); =20 @@ -1456,22 +1497,30 @@ static void vfio_msix_early_setup(VFIOPCIDevice *vd= ev, Error **errp) return; } =20 - if (pread(fd, &ctrl, sizeof(ctrl), - vdev->config_offset + pos + PCI_MSIX_FLAGS) !=3D sizeof(ctrl= )) { - error_setg_errno(errp, errno, "failed to read PCI MSIX FLAGS"); - return; - } + if (vdev->vbasedev.proxy !=3D NULL) { + /* during setup, config space was initialized from remote */ + memcpy(&ctrl, vdev->pdev.config + pos + PCI_MSIX_FLAGS, sizeof(ctr= l)); + memcpy(&table, vdev->pdev.config + pos + PCI_MSIX_TABLE, sizeof(ta= ble)); + memcpy(&pba, vdev->pdev.config + pos + PCI_MSIX_PBA, sizeof(pba)); + } else { + if (pread(fd, &ctrl, sizeof(ctrl), + vdev->config_offset + pos + PCI_MSIX_FLAGS) !=3D sizeof(= ctrl)) { + error_setg_errno(errp, errno, "failed to read PCI MSIX FLAGS"); + return; + } =20 - if (pread(fd, &table, sizeof(table), - vdev->config_offset + pos + PCI_MSIX_TABLE) !=3D sizeof(tabl= e)) { - error_setg_errno(errp, errno, "failed to read PCI MSIX TABLE"); - return; - } + if (pread(fd, &table, sizeof(table), + vdev->config_offset + pos + + PCI_MSIX_TABLE) !=3D sizeof(table)) { + error_setg_errno(errp, errno, "failed to read PCI MSIX TABLE"); + return; + } =20 - if (pread(fd, &pba, sizeof(pba), - vdev->config_offset + pos + PCI_MSIX_PBA) !=3D sizeof(pba)) { - error_setg_errno(errp, errno, "failed to read PCI MSIX PBA"); - return; + if (pread(fd, &pba, sizeof(pba), + vdev->config_offset + pos + PCI_MSIX_PBA) !=3D sizeof(pb= a)) { + error_setg_errno(errp, errno, "failed to read PCI MSIX PBA"); + return; + } } =20 ctrl =3D le16_to_cpu(ctrl); @@ -1619,11 +1668,17 @@ static void vfio_bar_prepare(VFIOPCIDevice *vdev, i= nt nr) } =20 /* Determine what type of BAR this is for registration */ - ret =3D pread(vdev->vbasedev.fd, &pci_bar, sizeof(pci_bar), - vdev->config_offset + PCI_BASE_ADDRESS_0 + (4 * nr)); - if (ret !=3D sizeof(pci_bar)) { - error_report("vfio: Failed to read BAR %d (%m)", nr); - return; + if (vdev->vbasedev.proxy !=3D NULL) { + /* during setup, config space was initialized from remote */ + memcpy(&pci_bar, vdev->pdev.config + PCI_BASE_ADDRESS_0 + (4 * nr), + sizeof(pci_bar)); + } else { + ret =3D pread(vdev->vbasedev.fd, &pci_bar, sizeof(pci_bar), + vdev->config_offset + PCI_BASE_ADDRESS_0 + (4 * nr)); + if (ret !=3D sizeof(pci_bar)) { + error_report("vfio: Failed to read BAR %d (%m)", nr); + return; + } } =20 pci_bar =3D le32_to_cpu(pci_bar); @@ -3423,6 +3478,91 @@ static void vfio_user_pci_realize(PCIDevice *pdev, E= rror **errp) goto error; } =20 + /* Get a copy of config space */ + ret =3D vfio_user_region_read(vbasedev, VFIO_PCI_CONFIG_REGION_INDEX, = 0, + MIN(pci_config_size(pdev), vdev->config_si= ze), + pdev->config); + if (ret < (int)MIN(pci_config_size(&vdev->pdev), vdev->config_size)) { + error_setg_errno(errp, -ret, "failed to read device config space"); + goto error; + } + + /* vfio emulates a lot for us, but some bits need extra love */ + vdev->emulated_config_bits =3D g_malloc0(vdev->config_size); + + /* QEMU can choose to expose the ROM or not */ + memset(vdev->emulated_config_bits + PCI_ROM_ADDRESS, 0xff, 4); + /* QEMU can also add or extend BARs */ + memset(vdev->emulated_config_bits + PCI_BASE_ADDRESS_0, 0xff, 6 * 4); + vdev->vendor_id =3D pci_get_word(pdev->config + PCI_VENDOR_ID); + vdev->device_id =3D pci_get_word(pdev->config + PCI_DEVICE_ID); + + /* QEMU can change multi-function devices to single function, or rever= se */ + vdev->emulated_config_bits[PCI_HEADER_TYPE] =3D + PCI_HEADER_TYPE_MULTI_FUNCTI= ON; + + /* Restore or clear multifunction, this is always controlled by QEMU */ + if (vdev->pdev.cap_present & QEMU_PCI_CAP_MULTIFUNCTION) { + vdev->pdev.config[PCI_HEADER_TYPE] |=3D PCI_HEADER_TYPE_MULTI_FUNC= TION; + } else { + vdev->pdev.config[PCI_HEADER_TYPE] &=3D ~PCI_HEADER_TYPE_MULTI_FUN= CTION; + } + + /* + * Clear host resource mapping info. If we choose not to register a + * BAR, such as might be the case with the option ROM, we can get + * confusing, unwritable, residual addresses from the host here. + */ + memset(&vdev->pdev.config[PCI_BASE_ADDRESS_0], 0, 24); + memset(&vdev->pdev.config[PCI_ROM_ADDRESS], 0, 4); + + vfio_pci_size_rom(vdev); + + vfio_bars_prepare(vdev); + + vfio_msix_early_setup(vdev, &err); + if (err) { + error_propagate(errp, err); + goto error; + } + + vfio_bars_register(vdev); + + ret =3D vfio_add_capabilities(vdev, errp); + if (ret) { + goto out_teardown; + } + + /* QEMU emulates all of MSI & MSIX */ + if (pdev->cap_present & QEMU_PCI_CAP_MSIX) { + memset(vdev->emulated_config_bits + pdev->msix_cap, 0xff, + MSIX_CAP_LENGTH); + } + + if (pdev->cap_present & QEMU_PCI_CAP_MSI) { + memset(vdev->emulated_config_bits + pdev->msi_cap, 0xff, + vdev->msi_cap_size); + } + + if (vdev->pdev.config[PCI_INTERRUPT_PIN] !=3D 0) { + vdev->intx.mmap_timer =3D timer_new_ms(QEMU_CLOCK_VIRTUAL, + vfio_intx_mmap_enable, vdev); + pci_device_set_intx_routing_notifier(&vdev->pdev, + vfio_intx_routing_notifier); + vdev->irqchip_change_notifier.notify =3D vfio_irqchip_change; + kvm_irqchip_add_change_notifier(&vdev->irqchip_change_notifier); + ret =3D vfio_intx_enable(vdev, errp); + if (ret) { + goto out_deregister; + } + } + +out_deregister: + pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); + kvm_irqchip_remove_change_notifier(&vdev->irqchip_change_notifier); +out_teardown: + vfio_teardown_msi(vdev); + vfio_bars_exit(vdev); error: vfio_user_disconnect(proxy); error_prepend(errp, VFIO_MSG_PREFIX, vdev->vbasedev.name); --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132477; cv=pass; d=zohomail.com; s=zohoarc; b=kpL/8O6P0s+0BVu043zlZQcEO/XlCW8cWyptouGSidm9Aw5FfXelHeza4qLVeX1QXotXRxIU+T26FMfUKnQhxF+ddF8De82h+MKIgHQ+ECakRfkoRs+PB0cv4Ch7AfkjMthaS6UNCm/z9byCjgjKjOAwkjtFKC+ug7KK/vSBGOs= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132477; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=nUV3Y0uIESmLz7Zkqh1tpA3nnyQpn+3Fz/PyO5rgOHU=; b=AMOB1i5hBn38riwOYuQNBc1YwK/E8IwtqFEn0jS6IUqtjdNYcFnpbuKp0SJB7McgtH2DcV9mQb8sUPe6Zb3ku4WYj+UL+Sn2d1SLq/YsI5amFkKjBMXmjGfzb+fH7GxGLZ3VRGGMJYpAwZxUNHzi6Wgg3Zx0BarA0Ncvp230VdQ= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629132477394464.4903879427827; Mon, 16 Aug 2021 09:47:57 -0700 (PDT) Received: from localhost ([::1]:35982 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfmG-0000uY-AW for importer@patchew.org; Mon, 16 Aug 2021 12:47:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37130) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhe-0000nX-7l for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:10 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:38904) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhY-0007y8-Lw for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:09 -0400 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGaA9q021054; Mon, 16 Aug 2021 16:43:03 GMT Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3afgmb9k04-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:03 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeo5w159237; Mon, 16 Aug 2021 16:43:02 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2106.outbound.protection.outlook.com [104.47.58.106]) by aserp3030.oracle.com with ESMTP id 3ae3ve34y4-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:02 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by SJ0PR10MB4765.namprd10.prod.outlook.com (2603:10b6:a03:2af::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.15; Mon, 16 Aug 2021 16:43:01 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:43:01 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:43:01 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=nUV3Y0uIESmLz7Zkqh1tpA3nnyQpn+3Fz/PyO5rgOHU=; b=gnFaoNCCsUe62fPezMnI+Rt7lr7nA8k8wgGq8iB4/NOWWHVWwaDe9BKE9604Z3HXwPpH b2yKLuZGZQ0su16qpPTrCkhZ4esTdf34VVnPhwlY10Lw0cb1B+ZC9mn8yEs1pmBeAzX5 aI24w/M8cxrI6gxEtf1AnkqvvfRFRzfnajAGQEqiR+MPPy2N6Bd3/72EhF9TDwNAPFh8 mx4Hw/U5KNAyoQlL1ln3lgvTbF4kHiR5//ARa59dYxAp2plMB6eu3hd3E3QVRnXcU6xc qcsmoa7Eue8Xw2GA/wPRuwbtNbW6aXP5D471EdXlPA0Ffr/xgASsF6rUxVKANm9tWb6H cA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=nUV3Y0uIESmLz7Zkqh1tpA3nnyQpn+3Fz/PyO5rgOHU=; b=Z+RLqUyUj4EekElBmqBRd9A5pbSf5jO5jjT+nkUo8/T0IXnWgPX6BJcf2QZrgovyJ2rp dh2zz1cg9JXF1rhkmuqu8UfwpP2iKsjVKFfh0caAZOHIDUMMJIDpKI3rbVOe6Ga5RPDO 7lG4tQvD8xsohqqr4JoFnBUunMrMpl/P3dCz3yv22oHFbe2EWlTE6lZ5w+dKcHLbnWOj rcU6OQF7mmtwjKYcSX7m4Ggc49Ov5iGPt4YKOHWFW+Mwi1Enw3kFpmX6mGnSS5YeDbSs ypG/rmGPU7hdx2XhSgdyQMFk4tDqa3DC/bIoPtDazSrq5B7aqi7gelnplBjjSN8IMqD+ gg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F8Drh/648U9JmUfzCD4PNHZs6T1SVGjA1jIx2dzW6Wyyj4NLFPulB8Fv33AHvBshQVPIA6sc2qfaawRd/KhXGiYICtVddGnac80sUJpmqsXLcCHaUDTwPZDf3qnYE/xfGPitxY2ji++uDPSxeJSSyNinHifDNaMnI9raHvJejPUhwzeBeyQLC7HCBWE2sVVpmMtzJanTChKYxoGe2VjugRVuSUZRspV0qZjwVr+eFaICBTO2ySzGE7I4GbIcarb+jUOph4ndBZGVjMw1rkL+DXkbKdHOcobi/nQDQ4V4+DEJbehXN+JUNpkXEJVNLHbClwRfhXcJ8xlxNKyqsjAR9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nUV3Y0uIESmLz7Zkqh1tpA3nnyQpn+3Fz/PyO5rgOHU=; b=H0EWHP/YN9CWEsYgTjXoFeAdGcxr1Jk4NIMa02jzon+16U42WOvHPzPMfB+jtTP7StBNGVheCY/HrFu5BAvTQxvKYT8gT7E0OzoINh5pq/DyVoCBkcbpJ78hv8Hvg+HzEXteCvosbwPJ54BoCVb8+tdhvFs9wa6gXIV/T/m2yQYxEl9aelg48buMFw+uvh29mRjcHypGO4n4Cx1PAnTFieW7Iu+ptTnHCzgfBtI96onFy5108DRUlVYY0iSpCYz31eYNuHx8yo7z4Gi0Rj96ZqNFYLUVcsxwLIxhysozH5snTeFS8iAklYFQBgdwW5fWXwtYpqcoWQuMhbFn5/nK8g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nUV3Y0uIESmLz7Zkqh1tpA3nnyQpn+3Fz/PyO5rgOHU=; b=O0X69LWGS9s7BWZk7bz27oLtVfqH19qtPrZ02K+R0IwcqquZtzQKCM+PxvAI23MTvz97Mz9vteKZJL33iNpeacjv+0jh+qr1b61keD3GJmfTKDPM/CykRr12yYw6ErVw5UveCzKT9Rq4nAoRr9ncK3tGyfroxsLXlrQUiWX7sP4= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 11/16] vfio-user: get and set IRQs Date: Mon, 16 Aug 2021 09:42:44 -0700 Message-Id: <4d292fab645b18b8a20f0f5a83356ae7fc0efa28.1629131628.git.elena.ufimtseva@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 669b6871-2c9a-4f4b-3d45-08d960d4e94c X-MS-TrafficTypeDiagnostic: SJ0PR10MB4765: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6430; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: S6lD3zrhA9HFLZRhecQfWEhWwzY92bKIiyI3mtt1U1+VefYwrCNFkYPtY729Uz/1VMbrERKCoRnUnpe8TU1TpfLLRN+dsjGZjLffqs38MUDosLtVU0YMvSS4opCkyPGdzV/txGswoJ0/IQk9MPRCjc0CrpxroiPWMiRbTk2xjrYUrqKMsx2nExN1F2jBZS8JirUX4zSxc2rq4cE/szf9002TX9S1QZcRWByhScRnXOySFBhkZJVeyag+gjU5BdGZS8AQcHC2uHXfd9d47IIriAD45no8i/kQ94fuO7FeKje1jCDBDDBtr/gU8oUBacsX4wXA3IqAtY/kWnlsd/N1VUtyDb5OvqTl2z1Sqvk7aXUrVt30uy0Uuvyx08gdlxTG4y2mvr9sdOOrTFzypSIp5qLmYJEs+peG260H+Z51C40kSe4N67kXkjPArFPi3au5S14SNyBFDVcSvQiHwnJ2xumu/872JSK0B4FHfu7c8RFgCVcSXvnC3l1FiUV/U20RodlitywdsWq1eEJlhfWIRCNKAHFhpCQBkRqQejnSNYMOFqW9/rdaTfj2bTwP80eqwt6lYHPAphkXtZskWkxk4BK21+xYJ1x59WLJdqD2PCdHU2DgVeBtl+R4j3EE6MwW3hGcVXnODLcnQprZ1MubmA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(186003)(2906002)(86362001)(66556008)(4326008)(38100700002)(8676002)(508600001)(66476007)(6486002)(66946007)(6506007)(5660300002)(44832011)(2616005)(6512007)(6666004)(36756003)(6916009)(316002)(52116002)(83380400001)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?LwqJ8yNtBvqrUNMsBupJSH2DZELQkfTqCXSTypPO9fmmpcbMlCSMROMchafr?= =?us-ascii?Q?4U/GquNJfAo51Sn/g/GOp9Obx2X4kxFNE0eOtUTtp0fjCAitdGuNx6sIl/x7?= =?us-ascii?Q?+AiCi2xvK30Osfnd5zyOrXDWoK2f1Iviq+/Jy84BXdPG6+Ge5watkM6+haxX?= =?us-ascii?Q?4qt8S5GFnK2fp9GZZGS45h1OF4/IWYB3fpjcTqlMK6SAWnKmjGNcra1OKLHM?= =?us-ascii?Q?1laS7LrkCir+fufySDqoFHBDeylT47uEZMgNC+ncJn8g73iowZImpE7OUfOZ?= =?us-ascii?Q?nOJPxVtyCGmuneo9iFMosXxupcexYncZClaKqMa4XcSdlp/SEnVg6rnKz04r?= =?us-ascii?Q?6E3uFSZ6WFdzOLvjYXe5lckd6cCgCVArBQqHZow61GBTFG9WmCzPep+BEYQO?= =?us-ascii?Q?mB58CqVezyrNQ+yTpgHFgx7G6pBDTiVHbWsLW0PmvGVj8x2Z/s3WrPxKRhiP?= =?us-ascii?Q?EEK3KIcdyrcCMaBbBJIpL7g52VseEylpr8y5o4BqZXF+zxw7YgVJwy3N3mGb?= =?us-ascii?Q?1Jug83mVhXoZSzrwKpgyEHjoptQO3Kzzf4u8nTskGUlME3hUSGiS/9QVgn6A?= =?us-ascii?Q?8pO1z0Sw2W9DRTG+WIMsbVRrVpjC1h7KxVGnzOrhUXPS/jeE4SizFdw0IGD2?= =?us-ascii?Q?qg2GjorvobtB+g4ucgUgGMmKlcisn7+bDo8cAb6Xbl2mOWx+jmyo+rEEG8AO?= =?us-ascii?Q?HgcZzoWL0jexocbWCCC9I+DFFe/h58wmEGC0maMfeDQ9pXcx+RgQI4MSjkfQ?= =?us-ascii?Q?LWUrJ4ZoigNzPl/KMtcxpmgdciZwjKR2t834YNgibLmLyKFTPQvazeZxyTYS?= =?us-ascii?Q?KjOIwuzaMPK3Jt059iDi8bRddmCD292t+gPJ1xTv4v++qRdC+UVmKiokUhFv?= =?us-ascii?Q?LBP9tIgr2Ua2dBHAQj5oC3khgmfoVuI6OtOQ4f/SHaqTDwYW6Q9SXr07Cj6U?= =?us-ascii?Q?m6wkexL/zYuqu4o1v9rQZQVgYr/Tc2a32relBDvgiwLsSkttBs1FUM27ZCXZ?= =?us-ascii?Q?kakHEmnZiVFx8289q4Z1esFw2aZ6sTiQfpovYWlLQybpGLA1jo1Jxl4P0lpb?= =?us-ascii?Q?yz0nHA7IbZkMo/H8XWUaQN3l5nOmPPV+JKwuTVkvi19Md7ZHa3MHnKeHHpiY?= =?us-ascii?Q?JSLQmYZx8kt6UHOgRMbDUo1UeDRNnZqmAy1z4vaZ30lKmOmyqbvwJQ/BMy8m?= =?us-ascii?Q?3sFjLw7c2ZCgS6Fq0das/3yCeJn8OO4oQpKTsDhdAtFTkpbMYc0ihgT8ff60?= =?us-ascii?Q?jhVnlx8WUGE7muwpshxhZ0R1G/Ayz0nxU2NzsMMOidt//5LHWrX0eow7DArO?= =?us-ascii?Q?1EuaNqEnL7ioBIYSPF41t4l32Naai16nthJRxBAN05Ztw0Pic7MOWq96jdFu?= =?us-ascii?Q?wzEOabgkRNfsbEsYCdYLRkN8tMxZ?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 669b6871-2c9a-4f4b-3d45-08d960d4e94c X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:43:01.4695 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 10fU2+Ux1N7evxwTktAFVlHoFpEYcurBe66Q29AE4pn6+9Ok/CNQIb72ilh0AOfuS2TzfX5JrAp2pY8QS6W+TLpkwUtjrAB6mdlOZpiV+lo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4765 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 bulkscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-ORIG-GUID: U3V5AmsFVs7sHS8ti7MBGc1sU_s0Q2PV X-Proofpoint-GUID: U3V5AmsFVs7sHS8ti7MBGc1sU_s0Q2PV Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -8 X-Spam_score: -0.9 X-Spam_bar: / X-Spam_report: (-0.9 / 5.0 requ) DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132479344100004 Content-Type: text/plain; charset="utf-8" From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 25 ++++++++++ hw/vfio/user.h | 2 + hw/vfio/common.c | 26 ++++++++-- hw/vfio/pci.c | 31 ++++++++++-- hw/vfio/user.c | 106 ++++++++++++++++++++++++++++++++++++++++ 5 files changed, 181 insertions(+), 9 deletions(-) diff --git a/hw/vfio/user-protocol.h b/hw/vfio/user-protocol.h index 56904cf872..5614efa0a4 100644 --- a/hw/vfio/user-protocol.h +++ b/hw/vfio/user-protocol.h @@ -109,6 +109,31 @@ typedef struct { uint64_t offset; } VFIOUserRegionInfo; =20 +/* + * VFIO_USER_DEVICE_GET_IRQ_INFO + * imported from struct vfio_irq_info + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t index; + uint32_t count; +} VFIOUserIRQInfo; + +/* + * VFIO_USER_DEVICE_SET_IRQS + * imported from struct vfio_irq_set + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint32_t index; + uint32_t start; + uint32_t count; +} VFIOUserIRQSet; + /* * VFIO_USER_REGION_READ * VFIO_USER_REGION_WRITE diff --git a/hw/vfio/user.h b/hw/vfio/user.h index 02f832a173..248ad80943 100644 --- a/hw/vfio/user.h +++ b/hw/vfio/user.h @@ -74,6 +74,8 @@ int vfio_user_validate_version(VFIODevice *vbasedev, Erro= r **errp); int vfio_user_get_info(VFIODevice *vbasedev); int vfio_user_get_region_info(VFIODevice *vbasedev, int index, struct vfio_region_info *info, VFIOUserFDs *= fds); +int vfio_user_get_irq_info(VFIODevice *vbasedev, struct vfio_irq_info *inf= o); +int vfio_user_set_irqs(VFIODevice *vbasedev, struct vfio_irq_set *irq); int vfio_user_region_read(VFIODevice *vbasedev, uint32_t index, uint64_t o= ffset, uint32_t count, void *data); int vfio_user_region_write(VFIODevice *vbasedev, uint32_t index, diff --git a/hw/vfio/common.c b/hw/vfio/common.c index a8b1ea9358..9fe3e05dc6 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -71,7 +71,11 @@ void vfio_disable_irqindex(VFIODevice *vbasedev, int ind= ex) .count =3D 0, }; =20 - ioctl(vbasedev->fd, VFIO_DEVICE_SET_IRQS, &irq_set); + if (vbasedev->proxy !=3D NULL) { + vfio_user_set_irqs(vbasedev, &irq_set); + } else { + ioctl(vbasedev->fd, VFIO_DEVICE_SET_IRQS, &irq_set); + } } =20 void vfio_unmask_single_irqindex(VFIODevice *vbasedev, int index) @@ -84,7 +88,11 @@ void vfio_unmask_single_irqindex(VFIODevice *vbasedev, i= nt index) .count =3D 1, }; =20 - ioctl(vbasedev->fd, VFIO_DEVICE_SET_IRQS, &irq_set); + if (vbasedev->proxy !=3D NULL) { + vfio_user_set_irqs(vbasedev, &irq_set); + } else { + ioctl(vbasedev->fd, VFIO_DEVICE_SET_IRQS, &irq_set); + } } =20 void vfio_mask_single_irqindex(VFIODevice *vbasedev, int index) @@ -97,7 +105,11 @@ void vfio_mask_single_irqindex(VFIODevice *vbasedev, in= t index) .count =3D 1, }; =20 - ioctl(vbasedev->fd, VFIO_DEVICE_SET_IRQS, &irq_set); + if (vbasedev->proxy !=3D NULL) { + vfio_user_set_irqs(vbasedev, &irq_set); + } else { + ioctl(vbasedev->fd, VFIO_DEVICE_SET_IRQS, &irq_set); + } } =20 static inline const char *action_to_str(int action) @@ -178,8 +190,12 @@ int vfio_set_irq_signaling(VFIODevice *vbasedev, int i= ndex, int subindex, pfd =3D (int32_t *)&irq_set->data; *pfd =3D fd; =20 - if (ioctl(vbasedev->fd, VFIO_DEVICE_SET_IRQS, irq_set)) { - ret =3D -errno; + if (vbasedev->proxy !=3D NULL) { + ret =3D vfio_user_set_irqs(vbasedev, irq_set); + } else { + if (ioctl(vbasedev->fd, VFIO_DEVICE_SET_IRQS, irq_set)) { + ret =3D -errno; + } } g_free(irq_set); =20 diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index ea0df8be65..282de6a30b 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -403,7 +403,11 @@ static int vfio_enable_vectors(VFIOPCIDevice *vdev, bo= ol msix) fds[i] =3D fd; } =20 - ret =3D ioctl(vdev->vbasedev.fd, VFIO_DEVICE_SET_IRQS, irq_set); + if (vdev->vbasedev.proxy !=3D NULL) { + ret =3D vfio_user_set_irqs(&vdev->vbasedev, irq_set); + } else { + ret =3D ioctl(vdev->vbasedev.fd, VFIO_DEVICE_SET_IRQS, irq_set); + } =20 g_free(irq_set); =20 @@ -2675,7 +2679,13 @@ static void vfio_populate_device(VFIOPCIDevice *vdev= , Error **errp) =20 irq_info.index =3D VFIO_PCI_ERR_IRQ_INDEX; =20 - ret =3D ioctl(vdev->vbasedev.fd, VFIO_DEVICE_GET_IRQ_INFO, &irq_info); + if (vbasedev->proxy !=3D NULL) { + ret =3D vfio_user_get_irq_info(vbasedev, &irq_info); + } else { + ret =3D ioctl(vdev->vbasedev.fd, VFIO_DEVICE_GET_IRQ_INFO, &irq_in= fo); + } + + if (ret) { /* This can fail for an old kernel or legacy PCI dev */ trace_vfio_populate_device_get_irq_info_failure(strerror(errno)); @@ -2794,8 +2804,16 @@ static void vfio_register_req_notifier(VFIOPCIDevice= *vdev) return; } =20 - if (ioctl(vdev->vbasedev.fd, - VFIO_DEVICE_GET_IRQ_INFO, &irq_info) < 0 || irq_info.count <= 1) { + if (vdev->vbasedev.proxy !=3D NULL) { + if (vfio_user_get_irq_info(&vdev->vbasedev, &irq_info) < 0) { + return; + } + } else { + if (ioctl(vdev->vbasedev.fd, VFIO_DEVICE_GET_IRQ_INFO, &irq_info) = < 0) { + return; + } + } + if (irq_info.count < 1) { return; } =20 @@ -3557,6 +3575,11 @@ static void vfio_user_pci_realize(PCIDevice *pdev, E= rror **errp) } } =20 + vfio_register_err_notifier(vdev); + vfio_register_req_notifier(vdev); + + return; + out_deregister: pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); kvm_irqchip_remove_change_notifier(&vdev->irqchip_change_notifier); diff --git a/hw/vfio/user.c b/hw/vfio/user.c index 83235b2411..b68ca1279d 100644 --- a/hw/vfio/user.c +++ b/hw/vfio/user.c @@ -768,6 +768,112 @@ int vfio_user_get_region_info(VFIODevice *vbasedev, i= nt index, return 0; } =20 +int vfio_user_get_irq_info(VFIODevice *vbasedev, struct vfio_irq_info *inf= o) +{ + VFIOUserIRQInfo msg; + + memset(&msg, 0, sizeof(msg)); + vfio_user_request_msg(&msg.hdr, VFIO_USER_DEVICE_GET_IRQ_INFO, + sizeof(msg), 0); + msg.argsz =3D info->argsz; + msg.index =3D info->index; + + vfio_user_send_recv(vbasedev->proxy, &msg.hdr, NULL, 0, 0); + if (msg.hdr.flags & VFIO_USER_ERROR) { + return -msg.hdr.error_reply; + } + + memcpy(info, &msg.argsz, sizeof(*info)); + return 0; +} + +static int irq_howmany(int *fdp, int cur, int max) +{ + int n =3D 0; + + if (fdp[cur] !=3D -1) { + do { + n++; + } while (n < max && fdp[cur + n] !=3D -1 && n < max_send_fds); + } else { + do { + n++; + } while (n < max && fdp[cur + n] =3D=3D -1 && n < max_send_fds); + } + + return n; +} + +int vfio_user_set_irqs(VFIODevice *vbasedev, struct vfio_irq_set *irq) +{ + g_autofree VFIOUserIRQSet *msgp =3D NULL; + uint32_t size, nfds, send_fds, sent_fds; + + if (irq->argsz < sizeof(*irq)) { + error_printf("vfio_user_set_irqs argsz too small\n"); + return -EINVAL; + } + + /* + * Handle simple case + */ + if ((irq->flags & VFIO_IRQ_SET_DATA_EVENTFD) =3D=3D 0) { + size =3D sizeof(VFIOUserHdr) + irq->argsz; + msgp =3D g_malloc0(size); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DEVICE_SET_IRQS, size,= 0); + msgp->argsz =3D irq->argsz; + msgp->flags =3D irq->flags; + msgp->index =3D irq->index; + msgp->start =3D irq->start; + msgp->count =3D irq->count; + + vfio_user_send_recv(vbasedev->proxy, &msgp->hdr, NULL, 0, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + + return 0; + } + + /* + * Calculate the number of FDs to send + * and adjust argsz + */ + nfds =3D (irq->argsz - sizeof(*irq)) / sizeof(int); + irq->argsz =3D sizeof(*irq); + msgp =3D g_malloc0(sizeof(*msgp)); + /* + * Send in chunks if over max_send_fds + */ + for (sent_fds =3D 0; nfds > sent_fds; sent_fds +=3D send_fds) { + VFIOUserFDs *arg_fds, loop_fds; + + /* must send all valid FDs or all invalid FDs in single msg */ + send_fds =3D irq_howmany((int *)irq->data, sent_fds, nfds - sent_f= ds); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DEVICE_SET_IRQS, + sizeof(*msgp), 0); + msgp->argsz =3D irq->argsz; + msgp->flags =3D irq->flags; + msgp->index =3D irq->index; + msgp->start =3D irq->start + sent_fds; + msgp->count =3D send_fds; + + loop_fds.send_fds =3D send_fds; + loop_fds.recv_fds =3D 0; + loop_fds.fds =3D (int *)irq->data + sent_fds; + arg_fds =3D loop_fds.fds[0] !=3D -1 ? &loop_fds : NULL; + + vfio_user_send_recv(vbasedev->proxy, &msgp->hdr, arg_fds, 0, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + return -msgp->hdr.error_reply; + } + } + + return 0; +} + int vfio_user_region_read(VFIODevice *vbasedev, uint32_t index, uint64_t o= ffset, uint32_t count, void *data) { --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629133043; cv=pass; d=zohomail.com; s=zohoarc; b=k/upjV8IKJUWKm9XPv4HjQ5TgyMHOHRDkLJYI47A+RMq5ZjDxhyr7XnmVa4byfPvyWyQJmotW3iQ6YUfqSBnq7NtJ1VnZHN+5u4qqZ+iDjqhwucMp8kafxiQaszXcuW/QpElHUwo50QMg10Q3X62roNEgQpT9vWFyUc4kduVSv4= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629133043; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=qGePs2QFNSfTvuBH7+YB/8qpJE2m7WykWxFo6Uuiu3o=; b=hTrALV+V9J40hyQaNynprKGYVRarFRJ+WqzkhqrDcMbcZJSYc8FTSkIY2y0W8hBEr6igkIMaCJaAn38cA7maCvf9PdzG3b1R3Brgv4sMj69pDHpqz4lN/+l93T34ojsfgxx5Yd/M0PiH5IgvF9rjmWGZB6hhVRCPEBiyXVEWh4w= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629133043252570.440024219118; Mon, 16 Aug 2021 09:57:23 -0700 (PDT) Received: from localhost ([::1]:60524 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfvO-0000Kb-5H for importer@patchew.org; Mon, 16 Aug 2021 12:57:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37166) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhf-0000rt-N4 for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:11 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:40068) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhZ-0007yP-DC for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:11 -0400 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGaFJO000687; Mon, 16 Aug 2021 16:43:04 GMT Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3af1q9anv0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:04 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeo5x159237; Mon, 16 Aug 2021 16:43:03 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2106.outbound.protection.outlook.com [104.47.58.106]) by aserp3030.oracle.com with ESMTP id 3ae3ve34y4-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:03 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by SJ0PR10MB4765.namprd10.prod.outlook.com (2603:10b6:a03:2af::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.15; Mon, 16 Aug 2021 16:43:02 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:43:02 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:43:01 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=qGePs2QFNSfTvuBH7+YB/8qpJE2m7WykWxFo6Uuiu3o=; b=gc6MdU+eOfiCF28+mSky+2BJstuPqxrCtuXOhpYgLvrGylnWmFwhEgBKtxtP+AiZ5YAd ArPYZRJi/OpiKwJ5k8sOpFBtff8llsRX2ep38JNYVY3PQsnuOVtke71Q0Im/QniNtXVF 5DUnq/iF23h1m8VnOt+r616Nb3OaA5krlo2IE7ooIjUBEzDZXoUlsphFSRMZWBnVvb38 QGX6dWWpWOl/pxNTu5xIQ9TotyelMqE7ftEs1BO8rRrELcvzctHLNGQNHVDNSPOLBvdE KQKgAzzAYhIPYRA1xK2xorPErJRsM7l6LRxhlWtgbq3hFSAw+8ITNK69KpLnjk+NcOnl tA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=qGePs2QFNSfTvuBH7+YB/8qpJE2m7WykWxFo6Uuiu3o=; b=G32KNQz1z/+BohJ5NZE5d97LAJWDlDxBgl06Egbmo+QyU7+KKPg+y1yOwxp/mqEz5+6e aP16StKvznq7lOd79B22BkZ+dRVYuqn8J3s9eY/8ihRi+tvPVcuTFGjLuyBFNYK8k+Gl zJgS7LqyaSP8r3ULtQ5mpqTZfhskdo/EL1GjZSVflCCJjG0qkxLayFbdkPxf/blqwltT xez+1FOpcdj87EqLNDoUQ+J6XK356HetzTSvSAyGe72IArltIbrxB7gjG2x4x2IRANfk f5SsJKMkv6Sav+UPWTGgGBR5c/sRhlgag9tSaDxPLbPHRvcHqUWRkVPNvQGr83b1nnnL og== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F50STqsKGHZCa4qnxX0k/hoMzwKhqjv8EZkgx2iQnzTfF27Yim3WYktAq8NWkmmj8IpqDGH/9rNAZC43hInvYpuwBIdEKc4sxVnk2lBo+liGB6UuGCaOoYUfmMFqqzWNlLZYrKulVcgtiiJATYJXpkSAVIuOGzqZIGPp8yQlDZjNh8kqnXuSGXOT1ExwRrv7mfQiMqT629pinB9gZ14hIhuqkAGJGWB2870u0m8w+TLh1A8wzu8tob++hGTQ6Wlwcabl3oZFGSlGvQbSgFHA6NDRQ3IDqE9pX9GMvgg31/po89a1z8IXZsIw+o0utHQPoMbfnpEj+uuBSSgBdVzEgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qGePs2QFNSfTvuBH7+YB/8qpJE2m7WykWxFo6Uuiu3o=; b=XvxacVjcb+MgL6TOiSCMoVqrvBNJ57SSSbSB18T9Jo8pg7LmZ9iShBtZh21tkvjriAuaVZNj772kFi8DM1AdoKNlHatnGhZRmAAlqXekgkwdB7fUd0hwDfaN0+XSWQlgB2zbfIdWTjQY1yN+PBttLR5Gisz4Vgzs+0zBFA9YmsZDtn432BbeyIy2S4+1t6p9sZaOzSYjHdADm2BSztXiX4v8zNlT28gbpeQk5dSaClEb9T9UPdVDHEqZU8/MD9zsljyE/eytRKGVD/IcNiw+JdHQc4GEfYxh1I67NMXtRM3hVKGOBEpvUXqRFmCKgoQVuxtCR5e1PYEDUObTdk9qyA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qGePs2QFNSfTvuBH7+YB/8qpJE2m7WykWxFo6Uuiu3o=; b=rJtFRQgl2gVA47f9Jc2KQ4GHRtEYJe/W8URT8pjWUXPV/z1PJ9oGqFFOyccM+xKAxN5Q5CWQFYjDY8w8ot9LSI7m8qIk9/CR1mdvJ2cHCYAe3kTabQH7wzG+mP9v4Q7z4VXvTTedVZPASqG8di5gIIT98TeV45lzYLFvz3ZidZg= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 12/16] vfio-user: proxy container connect/disconnect Date: Mon, 16 Aug 2021 09:42:45 -0700 Message-Id: <7f6595dcd88bfc0b459d3befd615a569635bae7d.1629131628.git.elena.ufimtseva@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fd93c160-5cd9-4990-03cf-08d960d4e99a X-MS-TrafficTypeDiagnostic: SJ0PR10MB4765: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1751; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mbFngX9tuTfBFS2HKkr7hZGalHDeFBjZzaAzbSLYcAwMOn3rSJV8L8R/MhGqM7ktLN25lNTZMNW+QMPQna+cfUeioTf74jzxQi22ra/nSpaM4LwEENyq0qX7DXI59Y5aRyLnhr7HiPm5wB5/5vuAnkb6w/t3j5A8z/k6zsPojkWqXY/xBddzeBm4MvJFUuPkC7W1Cl0GYY6BEu6uOywbNFtGmrllqKfxmfXurnK8tRfWbqMwv9+4+iysuc82+MG8fIpLYl7QkDJ09twuKglsrSGro/SDSzZmmOs0FcNl+iEiFZQIO84sE1wP/Jp9ilmwMKJgTss5T55HBfuoJJIeE9ym8l8mNW4+Ota9QGLFtJ8ZeJqyYMC3blZd2O697Ex6C9BJmw1Ivny3camU/PsUpt/Zq3zue4W/DwD/s0T0XB+2OTrRKAhB2HnyZY6WOb4HZsnNJXSVMUtmxqI6yW1diWUIlB6IFJMDw1yuWMbLWSJ6k47Z4OtWpG7JlAxYLW5LFGZ0IOvdB9Du9jUGRTWxFOMvMu82G7NDd9rF/DGbFd+MV4ORHmJ1/cQXS8s1nn3d/rigIOGUXou0RgRQ3v7DogtE73f2e8GLFOsVG9VRP9wa0ZYaKrMD/m2FosUK8+mmlJWTirY3Iq+Ypxr4hmi9ww== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(186003)(2906002)(86362001)(66556008)(4326008)(38100700002)(8676002)(508600001)(66476007)(6486002)(66946007)(6506007)(5660300002)(44832011)(2616005)(6512007)(6666004)(36756003)(6916009)(316002)(52116002)(83380400001)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?cx10qUFyuC1D66I15ksf+IU802yCUiELQxciIunvOq0kslz33cupOJtw/BLr?= =?us-ascii?Q?snUTi50RAtdn+UycPyfCrJT/+DaIAjszZr+bkHcpO8tPEX3kb4xOCZOFmf+O?= =?us-ascii?Q?Kv7gXtsOcQ0Nt20vnEJ+GTbBaXKxNxcAReT+g5qYBnWeEdd6zQsmai2xSsjf?= =?us-ascii?Q?v6LGDwcpe0LOov2dhH8HusdaPCDJx9/6WIvYAU9FEF/ZNWieO0OaWlpJpc9r?= =?us-ascii?Q?1ZD9pTrbamhsWWEho7qKacZrKeT6VYeFaxyJ7RlyQDV4cbksAYkphz2ezRAv?= =?us-ascii?Q?wZk41LW7Btbmaj73O3yB1K+zyF6NigF7QpgUqN9df+QkOrCx00z5/DSx9dQB?= =?us-ascii?Q?jfr+rITuCj1KoR9ySwKM4OzxVxUsFQhHy0pgc4hmKTDGjWa85QLB6CrQ4gkJ?= =?us-ascii?Q?Ah+NZKn66zhkRWD0F4/vlpGT1O9x1JYpS/xaVpebwLUwxptLi/qmyn0ydnfb?= =?us-ascii?Q?ZZuI5+bGd4csawboaKZM4MF9/eCiSRKji2o87yc0nMAfg3pZBZEuknYpE1p+?= =?us-ascii?Q?N9oKXlwYZqeq+rVIkB8GWta+u1xtL++qkT3Xmak4AvL41OY6MOh23HAI/s/1?= =?us-ascii?Q?mAeZGm9pn3CwTxWefVRXTAUJIsaEK9OH5qclc8Cr458WXFdy8p7tDc4hELqZ?= =?us-ascii?Q?gKI6ohwRJiZhMnyA+ojBsnziJqLGNp65rPyQRhFdIrQOMVlmso42pC1cPN9u?= =?us-ascii?Q?Bn6raaXwBckpi1nCgeSe6RxNBeBdxq1TMmYn6z5axTtATatLErBu0st6METg?= =?us-ascii?Q?5njO2hl7xw5WybvIqczm4Xy3L368SjROA/CVzeUBual328OW2HY/8d6RCQsk?= =?us-ascii?Q?GverQlyxceswZDHikGTwPANz68B1+G7SU1LuPKjRUf0MS2dav5z8ALC6buWQ?= =?us-ascii?Q?FsKl3EMclb0faG/J2lY0yVPK1FWp+se+r6Cb1G4e1Vjon+0aJpMjwZBnPpQ3?= =?us-ascii?Q?3+pxoxL+rJIXwjUkKvPRX1xsymnt+ND6XH7C9chtGY3CMNssAgNiV5f2hhfG?= =?us-ascii?Q?vC5knTvg35b1nfo5XshjuqJx82IVp6/nf2rHLWpINfvG1UFZZEmrEEFmR14m?= =?us-ascii?Q?HBJf3tJ/Oq0Sc+/0UfbsHnNHLmeXo/2vIa2rgfTOkRE+s5Y6mgvwx0io2tYX?= =?us-ascii?Q?M0HqvtjU/e7XkCBqtfIbfrlvmE1gw0a5I1n5iL3w0wzRAsxmV3UqX9Kssutw?= =?us-ascii?Q?nnhYi0M4Tqm7xxbTk9JDnv+pAyKdALIRxaaygAW5ABWLt30QMF/L0TeI5A5P?= =?us-ascii?Q?12ptV7EPnTHhKOlCi2ow/2UIoCnWhIpzPhwFjpO9aDTpTTYpQ2h8BZyT8cBz?= =?us-ascii?Q?i1RBdu/cxuH4H/rF6ngaCxN28koyZn3MlMDTklLJvto6LDhbPT3Kct7JsrW3?= =?us-ascii?Q?Lpa/H+AWFdGYVRlz4oN5P1e8+v/+?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: fd93c160-5cd9-4990-03cf-08d960d4e99a X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:43:01.9463 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lQ3sPeC7R80UvcvlceQeg/QpmPtFaDGtgc1JZO/U8ZD8OGsKvPTm66CdxBP8pvziPIU8jJFvwABKgPN2PVvGtp2hB/FQs5iV2Iq8DgWoVLc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4765 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 bulkscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-GUID: LeNAUWSeXRzR7ZqcyHZfFvjnHvi-TK6z X-Proofpoint-ORIG-GUID: LeNAUWSeXRzR7ZqcyHZfFvjnHvi-TK6z Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629133044739100001 Content-Type: text/plain; charset="utf-8" From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- include/hw/vfio/vfio-common.h | 3 ++ hw/vfio/common.c | 84 +++++++++++++++++++++++++++++++++++ hw/vfio/pci.c | 22 +++++++++ 3 files changed, 109 insertions(+) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index bdd25a546c..688660c28d 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -91,6 +91,7 @@ typedef struct VFIOContainer { uint64_t max_dirty_bitmap_size; unsigned long pgsizes; unsigned int dma_max_mappings; + VFIOProxy *proxy; QLIST_HEAD(, VFIOGuestIOMMU) giommu_list; QLIST_HEAD(, VFIOHostDMAWindow) hostwin_list; QLIST_HEAD(, VFIOGroup) group_list; @@ -217,6 +218,8 @@ VFIOGroup *vfio_get_group(int groupid, AddressSpace *as= , Error **errp); void vfio_put_group(VFIOGroup *group); int vfio_get_device(VFIOGroup *group, const char *name, VFIODevice *vbasedev, Error **errp); +void vfio_connect_proxy(VFIOProxy *proxy, VFIOGroup *group, AddressSpace *= as); +void vfio_disconnect_proxy(VFIOGroup *group); =20 extern const MemoryRegionOps vfio_region_ops; typedef QLIST_HEAD(VFIOGroupList, VFIOGroup) VFIOGroupList; diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 9fe3e05dc6..57b9e111e6 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -2249,6 +2249,55 @@ put_space_exit: return ret; } =20 +void vfio_connect_proxy(VFIOProxy *proxy, VFIOGroup *group, AddressSpace *= as) +{ + VFIOAddressSpace *space; + VFIOContainer *container; + + if (QLIST_EMPTY(&vfio_group_list)) { + qemu_register_reset(vfio_reset_handler, NULL); + } + + QLIST_INSERT_HEAD(&vfio_group_list, group, next); + + /* + * try to mirror vfio_connect_container() + * as much as possible + */ + + space =3D vfio_get_address_space(as); + + container =3D g_malloc0(sizeof(*container)); + container->space =3D space; + container->fd =3D -1; + QLIST_INIT(&container->giommu_list); + QLIST_INIT(&container->hostwin_list); + container->proxy =3D proxy; + + /* + * The proxy uses a SW IOMMU in lieu of the HW one + * used in the ioctl() version. Use TYPE1 with the + * target's page size for maximum capatibility + */ + container->iommu_type =3D VFIO_TYPE1_IOMMU; + vfio_host_win_add(container, 0, (hwaddr)-1, TARGET_PAGE_SIZE); + container->pgsizes =3D TARGET_PAGE_SIZE; + + container->dirty_pages_supported =3D true; + container->max_dirty_bitmap_size =3D VFIO_USER_DEF_MAX_XFER; + container->dirty_pgsizes =3D TARGET_PAGE_SIZE; + + QLIST_INIT(&container->group_list); + QLIST_INSERT_HEAD(&space->containers, container, next); + + group->container =3D container; + QLIST_INSERT_HEAD(&container->group_list, group, container_next); + + container->listener =3D vfio_memory_listener; + memory_listener_register(&container->listener, container->space->as); + container->initialized =3D true; +} + static void vfio_disconnect_container(VFIOGroup *group) { VFIOContainer *container =3D group->container; @@ -2291,6 +2340,41 @@ static void vfio_disconnect_container(VFIOGroup *gro= up) } } =20 +void vfio_disconnect_proxy(VFIOGroup *group) +{ + VFIOContainer *container =3D group->container; + VFIOAddressSpace *space =3D container->space; + VFIOGuestIOMMU *giommu, *tmp; + + /* + * try to mirror vfio_disconnect_container() + * as much as possible, knowing each device + * is in one group and one container + */ + + QLIST_REMOVE(group, container_next); + group->container =3D NULL; + + /* + * Explicitly release the listener first before unset container, + * since unset may destroy the backend container if it's the last + * group. + */ + memory_listener_unregister(&container->listener); + + QLIST_REMOVE(container, next); + + QLIST_FOREACH_SAFE(giommu, &container->giommu_list, giommu_next, tmp) { + memory_region_unregister_iommu_notifier( + MEMORY_REGION(giommu->iommu), &giommu->n); + QLIST_REMOVE(giommu, giommu_next); + g_free(giommu); + } + + g_free(container); + vfio_put_address_space(space); +} + VFIOGroup *vfio_get_group(int groupid, AddressSpace *as, Error **errp) { VFIOGroup *group; diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 282de6a30b..2c9fcb2fa9 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3442,6 +3442,7 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Er= ror **errp) VFIODevice *vbasedev =3D &vdev->vbasedev; SocketAddress addr; VFIOProxy *proxy; + VFIOGroup *group =3D NULL; int ret; Error *err =3D NULL; =20 @@ -3484,6 +3485,19 @@ static void vfio_user_pci_realize(PCIDevice *pdev, E= rror **errp) vbasedev->no_mmap =3D false; vbasedev->ops =3D &vfio_user_pci_ops; =20 + /* + * each device gets its own group and container + * make them unrelated to any host IOMMU groupings + */ + group =3D g_malloc0(sizeof(*group)); + group->fd =3D -1; + group->groupid =3D -1; + QLIST_INIT(&group->device_list); + QLIST_INSERT_HEAD(&group->device_list, vbasedev, next); + vbasedev->group =3D group; + + vfio_connect_proxy(proxy, group, pci_device_iommu_address_space(pdev)); + ret =3D vfio_user_get_info(&vdev->vbasedev); if (ret) { error_setg_errno(errp, -ret, "get info failure"); @@ -3587,6 +3601,9 @@ out_teardown: vfio_teardown_msi(vdev); vfio_bars_exit(vdev); error: + if (group !=3D NULL) { + vfio_disconnect_proxy(group); + } vfio_user_disconnect(proxy); error_prepend(errp, VFIO_MSG_PREFIX, vdev->vbasedev.name); } @@ -3595,6 +3612,11 @@ static void vfio_user_instance_finalize(Object *obj) { VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(obj); VFIODevice *vbasedev =3D &vdev->vbasedev; + VFIOGroup *group =3D vbasedev->group; + + vfio_disconnect_proxy(group); + g_free(group); + vbasedev->group =3D NULL; =20 vfio_put_device(vdev); =20 --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132915; cv=pass; d=zohomail.com; s=zohoarc; b=fUNTsD6C5vGPSnZdEErtWr+O072z0I5CwR9MbEAXkWKXUmM8b86arzjpiVp4rIcvNWWuQ7xbwE9PVZXgKL+3ZB4of4islZB7D8BFdJg1UDqe0OM4Al++TYT5KBpL5rIK1F6hcIn52bw+ks/O5BtxJncHPdz7kDpNNJC4HnCU0R4= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132915; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=R+/8Xn6UCRdNqx3X44X9ogFDXli8hym8wKlPB0YaaRw=; b=Nym65KKeEwf6e7GRE4T28jOQYAjiVx7K6LMFJZo49CnJrQrJM0Ky4se3LFn9CkZxTDNeNgn5YYJNnpvdSVouIRTCz/eGDCEwfiMtPbFSX8BtYKsq7dac9rRo/c5AqURt5jfSfKk0+7JB9HwbdlPg0BAexYe1gINRi377wTNfHAg= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629132915172705.954528849651; Mon, 16 Aug 2021 09:55:15 -0700 (PDT) Received: from localhost ([::1]:54850 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFftJ-0004z0-OB for importer@patchew.org; Mon, 16 Aug 2021 12:55:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37226) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhi-00011D-Db for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:14 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:46090) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhc-000812-49 for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:14 -0400 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGaE6T000648; Mon, 16 Aug 2021 16:43:06 GMT Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3af1q9anv2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:06 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeiuQ113652; Mon, 16 Aug 2021 16:43:04 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2108.outbound.protection.outlook.com [104.47.58.108]) by aserp3020.oracle.com with ESMTP id 3ae5n609mu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:04 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by SJ0PR10MB4765.namprd10.prod.outlook.com (2603:10b6:a03:2af::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.15; Mon, 16 Aug 2021 16:43:02 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:43:02 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:43:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=R+/8Xn6UCRdNqx3X44X9ogFDXli8hym8wKlPB0YaaRw=; b=gZ44t6Zcv54un4jlD4l6Q0u8M07ACkppMosZ9kTN1RCQff7pE75hXfV7+kjd3bWhEEAc KshH+ORqoq5oplKvPMJqH6hBWliYgi24ker3Du3Vuq1OBET4Kw0zfJ9D3GPB//kGE8cv psstx2E2r5PSYkw7oYrG0G56GVh8FCBUJm2J8grlrUJ6vFmhi7HAj09FL6ZNG5oUBl8C rCxJvoVI6NYyjIGkryPMV/XhB6J2CXwRBCTqSgaCRER+/MSXrftjU7+E3vYiGrXczcZY O7D4gzzU0G4NZ0BGrChHJUwTAyu2+T7rrwGT5RAHJuaJJu9NVVtXYyWJQC68SZYO/yUM TQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=R+/8Xn6UCRdNqx3X44X9ogFDXli8hym8wKlPB0YaaRw=; b=tB0wOVf8T7qqJzsrWEuVuFti6WIUY5xch6Di/suWBEIqyfEh5muQNKCCcHcAp7DYZ7Ps MhEXh86rbrDcR2iU3AGRwz4cY0AMU8y1Y7yO6OFoNL5qErSC7QxinUvLDT0L7EEyiWyw 4MZYQXljcJJCgkHQG4H9zad2PZEiIeZSkFmiHDpsVGRWJFsW2f2Bqz6GD8j5xRTl2zXL LW/LmkbxztWOMlbEOkfACP2R/JiRRrPaOhQvHWvB2hWPihjobw0QotHWYL+AQDELtTNc cml79GZ9oMQgj+6TqNGKYozlyffWlpljNQHGdnIcZK45fTBRX8w34Zxr6YEYw/AbFQwa Ag== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=G/tcG7o7XWVKhFctcQj9Wvd57M30H+0aAvfZuTlZ+hBzCLRooFuyF+hN6sS376uC5oBOarVQOg84png0FJSAezlK8pVUXKKcL+kFdrWWN40lKgS7Po86p/hxdJI387DrImuNjOPFPSg8Vun8w02DSwFHn3gb7MXs6EzlWnVnnTQsN0/ncn+Ya3SvkrtaDh9/TVVwoIbiCg4YhKWTZUna5xenY3wsYBxZzsxRI4X2nU/NqwCQ65/k6XzuLJq2SH8/epNKA1+dfmlSF0/IGQT6FeTDRXKMLpG3ucuy8+rBYnQSaOk7xohQ7lgfHIn/XTsIPS7ZohmpukpmXrXsAtii9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=R+/8Xn6UCRdNqx3X44X9ogFDXli8hym8wKlPB0YaaRw=; b=diNz6X+ZzuLQ0vHfFnnX7caKY8hhmCQiej7B8dKJMI175vGZ//b5OtmFd9K5GaRRBOlamOldQPQLWt2DESdRx5i8pyIgGJbqaAsvhPdnLKO0BsCU6++iEvWFR4chHIyTxa2CR4YHixJic02dPM4YqrpCin4lCNJ/CfckD9cT58rUTTWzBRgwCgTDjA4ULXJZe4bJ7F/Vr3b2+VZQMIXGoLXtPXOmapUFrWLsYgbkuMgn92LpFS5KVt1554af9hlXnaB9s10DG0nMjpJZJ3o+oVMPb+J7w8FGIq5A5WMSF3EFiRtAhlfEfmgws5iTQNC1+Zug1oYvQERbT/SIBa3Pkg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=R+/8Xn6UCRdNqx3X44X9ogFDXli8hym8wKlPB0YaaRw=; b=cX4418KNxnBWESQOfGaPz7y0fMvy1Wz3s0EYZaab49HGVgOVCdaIUCqj5uxrle/hpA+Pb8N45FGCES41qjGVJIswvaqIsvYH0dXdHyOOmdTKl2V7m2w3272sjXGGTqNXi0ErxdH/SGYEeKv9zkj2Vl6Hykjx3i0nWuu9XADzbjA= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 13/16] vfio-user: dma map/unmap operations Date: Mon, 16 Aug 2021 09:42:46 -0700 Message-Id: <8960370c1ce02089099702878ed6fb6c3d42ad06.1629131628.git.elena.ufimtseva@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 17d240b4-106a-4a5b-bc4f-08d960d4e9f6 X-MS-TrafficTypeDiagnostic: SJ0PR10MB4765: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1332; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Ec2VJpIvPTbMvh7QaHJ3h1S57cAHAqtd7ZxGFECqlLKzJtTpQq0sJXCkAr2cyRDzEkZmrrFPuz1i40khqBU7td3m6vwE08Dzx5oB0/M0do84f5D4oFN+PubMdA1TCfagK2H5zz7c8DztNl3CNEG54qS3tPia/s9miIRwiUpIWmb1ZLtE2IjBF1OYrhuecTDgFpGH5zTf8z98BZ7aF3svhiv+kH77aboTCbXPjy2FER+RGHyIwTj81xtWHTE0NWoqgzvGJjuieHIe1TIC/FOGeD8AbNCY1CHMnqU/PpHP7e7eS8j4zl5dDnXD7DpJI40oKcNu89mp3IoDKBIWhACyxsNxlrx83lgXmBTvF630wxgOyegkVWHdq9w9Z10T/v1ueboYxD14Lgi8y1+peV8USWAkFWlfwBNn3OStoT6leph7Hz8Izw35NO9WZsuc3GM8CY5xct9/RTOJgB7jp9AfPlPg2MK/7C9rbe1vWGuCFhu+ii8dps5mM1ShftT4aZSV+kYUojA34KQWmY0k8eqVAK5225bKEpMAQMwA7TlSEBXci+7Fsu9Ci1xKTl9/BsjbrB4AvatXGjBloIhukFBKNfKo4aamiSAfZAHqkqlOO7gbaWlF+hqgscUSf4GWJWFwYy+45BBMwjkPwgOzjYgA1g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(186003)(2906002)(86362001)(30864003)(66556008)(4326008)(38100700002)(8676002)(508600001)(66476007)(6486002)(66946007)(6506007)(5660300002)(44832011)(2616005)(6512007)(6666004)(36756003)(6916009)(316002)(52116002)(83380400001)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?AWnQHpEXy6OCoe91mOoAvAnL1GYu9qG914LZSb+cc8ZuEnF3J5W4ULm5HZ3j?= =?us-ascii?Q?AW0MwSNWtMjalw3/8cuJtyyu2tTrorbFkVpRTVmF2VlaVgW6L9xhIkD3LjPr?= =?us-ascii?Q?W2YJyqRqQREbTjdiOQkAZDuwbaj8+kqWQN/aS5em/W8fzrmIDaDl832z2N7g?= =?us-ascii?Q?TyfmGkVd9E06N53mdhzZ5TbypYa9hH3MczMsLPbbtEcj1/hrl9FclzQMpvCI?= =?us-ascii?Q?AgUWNysdgEP8LxKVQvbpGIGNsQunel8XapDec7naPbJ0kM5700q/5EiSFuVy?= =?us-ascii?Q?D9NriCbx3Aax8DUSY2w/UOCM76hsZb3WWU2d2v3/14PX3mjLui6lInp7Clfb?= =?us-ascii?Q?SNAH0pP44wO+7hFgXIn7i0ZnBmQSE4VhKTVIFsmzSgf8Rw9FGLyKozj9CKOJ?= =?us-ascii?Q?/QO7kdNoIyeyQuPuZ4n4FMfEC5u6voQo45QRKq8/trSkKsAEQasE9yby+uW3?= =?us-ascii?Q?GQBkf8gHIPY6v9c1X2DaCK6zH3k/BxH0Iyww/HeWM/DpdBnqxD5O8T7LX8Mu?= =?us-ascii?Q?g5XqSGVYAzAGlAfWyiBZYgZok2LCwcldmVrzDWhC5X+TmyTbiI88v3iSP/yt?= =?us-ascii?Q?scnrzYyNUoLDpm3oHBiXT2DHzne/elEVldmlmrp+qbX7P8d/y1ayQfGjJScE?= =?us-ascii?Q?FqvVfC+abI4dTLwA/mBDSOp6sabl2J/1gxlrAHscqMZokvSWm0omxHC0BDKY?= =?us-ascii?Q?+WwKjTON0pUutpa/wjbEbFjNK0vdFPyvi5bNR1kNvrpRQEJ+qLX2zD/AoRIH?= =?us-ascii?Q?jiPs/cX76Em2RBSXo4d6ricrN1W4XMKIxuUMq41U8YQMMWR5oIR+ZFECpVTg?= =?us-ascii?Q?Hk/iQhV9ri3pYYiZHmtt1Nl2FgBsdjndL1LsNd7uVhmHhy7WucmlW7vTwWFX?= =?us-ascii?Q?G66qP+tgK8g1lXJjetU5XzHrxN7sEeQ9zhdu9AFcSmovQ3l8NM5X7K1VVvLx?= =?us-ascii?Q?ctlj9CnVDVCvgOIL10ymIcKwN3N7HmmFf5Baf04EPrgEj3ENPiI9TrmnkitN?= =?us-ascii?Q?jfFMgDyl5PuNmk/zAgQmdw7FduTSZDpG9LwubqW9qtnb645QZL7ndNCV0RZX?= =?us-ascii?Q?IK0uRP+DTvGVl5jkdfRkb/+Dx0z/sJ8Gi0A282cCqqdkjfAR7532cd1SkAG2?= =?us-ascii?Q?JNkm7cVhIjEDtauVwfhcnI82e98DqPv2rRSFjX63/6wtYggGIeqVljQPeP3b?= =?us-ascii?Q?rDr8V7s+M7F1OlVZJa8dyhMZApY3hGEbwrqKlQbSDlOm63uqCz4NAh5dpWte?= =?us-ascii?Q?N/aT22OO1ERTftxYp1MDw66lsH1ahQoAOs1G8ULS78xTnJU1vMwmoENJWRVJ?= =?us-ascii?Q?spjt4fZFr/Y5iRQMzPuYbLp4ttJyqOuqIyEA6sdi/1vw+qZgNh2UDvwc5zvy?= =?us-ascii?Q?cTjjR0SrqvzjRzcmNr0lXkNmK+5q?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 17d240b4-106a-4a5b-bc4f-08d960d4e9f6 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:43:02.5579 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SSTYhRKcRw3t2m8Zn/ZsgERbSczrkxIWFX/c5kVeRAku/QSZpkqrao6t0C/zC7eExQ3ghYz54mb78uj1Fm4b20xX9pKVA1K7LXWcOSk3EFQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4765 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-GUID: AXjR6khvQVIOmDGrzgaIG8VVwaQ_L6pt X-Proofpoint-ORIG-GUID: AXjR6khvQVIOmDGrzgaIG8VVwaQ_L6pt Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132917524100001 Content-Type: text/plain; charset="utf-8" From: John Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson --- hw/vfio/user-protocol.h | 32 +++++++++ hw/vfio/user.h | 6 ++ include/hw/vfio/vfio-common.h | 1 + hw/vfio/common.c | 71 ++++++++++++++++--- hw/vfio/user.c | 124 ++++++++++++++++++++++++++++++++++ 5 files changed, 226 insertions(+), 8 deletions(-) diff --git a/hw/vfio/user-protocol.h b/hw/vfio/user-protocol.h index 5614efa0a4..ca53fce5f4 100644 --- a/hw/vfio/user-protocol.h +++ b/hw/vfio/user-protocol.h @@ -82,6 +82,31 @@ typedef struct { #define VFIO_USER_MAX_MAX_XFER (64 * 1024 * 1024) =20 =20 +/* + * VFIO_USER_DMA_MAP + * imported from struct vfio_iommu_type1_dma_map + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint64_t offset; /* FD offset */ + uint64_t iova; + uint64_t size; +} VFIOUserDMAMap; + +/* + * VFIO_USER_DMA_UNMAP + * imported from struct vfio_iommu_type1_dma_unmap + */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; + uint64_t iova; + uint64_t size; +} VFIOUserDMAUnmap; + /* * VFIO_USER_DEVICE_GET_INFO * imported from struct_device_info @@ -146,4 +171,11 @@ typedef struct { char data[]; } VFIOUserRegionRW; =20 +/*imported from struct vfio_bitmap */ +typedef struct { + uint64_t pgsize; + uint64_t size; + char data[]; +} VFIOUserBitmap; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio/user.h b/hw/vfio/user.h index 248ad80943..7786ab57c5 100644 --- a/hw/vfio/user.h +++ b/hw/vfio/user.h @@ -71,6 +71,11 @@ void vfio_user_set_reqhandler(VFIODevice *vbasdev, void *reqarg); void vfio_user_send_reply(VFIOProxy *proxy, char *buf, int ret); int vfio_user_validate_version(VFIODevice *vbasedev, Error **errp); +int vfio_user_dma_map(VFIOProxy *proxy, struct vfio_iommu_type1_dma_map *m= ap, + VFIOUserFDs *fds, bool will_commit); +int vfio_user_dma_unmap(VFIOProxy *proxy, + struct vfio_iommu_type1_dma_unmap *unmap, + struct vfio_bitmap *bitmap, bool will_commit); int vfio_user_get_info(VFIODevice *vbasedev); int vfio_user_get_region_info(VFIODevice *vbasedev, int index, struct vfio_region_info *info, VFIOUserFDs *= fds); @@ -80,5 +85,6 @@ int vfio_user_region_read(VFIODevice *vbasedev, uint32_t = index, uint64_t offset, uint32_t count, void *data); int vfio_user_region_write(VFIODevice *vbasedev, uint32_t index, uint64_t offset, uint32_t count, void *data); +void vfio_user_drain_reqs(VFIOProxy *proxy); =20 #endif /* VFIO_USER_H */ diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 688660c28d..13d1d14c3b 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -87,6 +87,7 @@ typedef struct VFIOContainer { Error *error; bool initialized; bool dirty_pages_supported; + bool will_commit; uint64_t dirty_pgsizes; uint64_t max_dirty_bitmap_size; unsigned long pgsizes; diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 57b9e111e6..a532e52bcf 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -427,6 +427,7 @@ static int vfio_dma_unmap_bitmap(VFIOContainer *contain= er, struct vfio_iommu_type1_dma_unmap *unmap; struct vfio_bitmap *bitmap; uint64_t pages =3D REAL_HOST_PAGE_ALIGN(size) / qemu_real_host_page_si= ze; + bool will_commit =3D container->will_commit; int ret; =20 unmap =3D g_malloc0(sizeof(*unmap) + sizeof(*bitmap)); @@ -460,7 +461,11 @@ static int vfio_dma_unmap_bitmap(VFIOContainer *contai= ner, goto unmap_exit; } =20 - ret =3D ioctl(container->fd, VFIO_IOMMU_UNMAP_DMA, unmap); + if (container->proxy !=3D NULL) { + ret =3D vfio_user_dma_unmap(container->proxy, unmap, bitmap, will_= commit); + } else { + ret =3D ioctl(container->fd, VFIO_IOMMU_UNMAP_DMA, unmap); + } if (!ret) { cpu_physical_memory_set_dirty_lebitmap((unsigned long *)bitmap->da= ta, iotlb->translated_addr, pages); @@ -487,12 +492,17 @@ static int vfio_dma_unmap(VFIOContainer *container, .iova =3D iova, .size =3D size, }; + bool will_commit =3D container->will_commit; =20 if (iotlb && container->dirty_pages_supported && vfio_devices_all_running_and_saving(container)) { return vfio_dma_unmap_bitmap(container, iova, size, iotlb); } =20 + if (container->proxy !=3D NULL) { + return vfio_user_dma_unmap(container->proxy, &unmap, NULL, will_co= mmit); + } + while (ioctl(container->fd, VFIO_IOMMU_UNMAP_DMA, &unmap)) { /* * The type1 backend has an off-by-one bug in the kernel (71a7d3d7= 8e3c @@ -519,7 +529,7 @@ static int vfio_dma_unmap(VFIOContainer *container, return 0; } =20 -static int vfio_dma_map(VFIOContainer *container, hwaddr iova, +static int vfio_dma_map(VFIOContainer *container, MemoryRegion *mr, hwaddr= iova, ram_addr_t size, void *vaddr, bool readonly) { struct vfio_iommu_type1_dma_map map =3D { @@ -529,11 +539,30 @@ static int vfio_dma_map(VFIOContainer *container, hwa= ddr iova, .iova =3D iova, .size =3D size, }; + bool will_commit =3D container->will_commit; =20 if (!readonly) { map.flags |=3D VFIO_DMA_MAP_FLAG_WRITE; } =20 + if (container->proxy !=3D NULL) { + VFIOUserFDs fds; + int fd; + + fd =3D memory_region_get_fd(mr); + if (fd !=3D -1 && !(container->proxy->flags & VFIO_PROXY_SECURE)) { + fds.send_fds =3D 1; + fds.recv_fds =3D 0; + fds.fds =3D &fd; + map.vaddr =3D qemu_ram_block_host_offset(mr->ram_block, vaddr); + + return vfio_user_dma_map(container->proxy, &map, &fds, will_co= mmit); + } else { + map.vaddr =3D 0; + return vfio_user_dma_map(container->proxy, &map, NULL, will_co= mmit); + } + } + /* * Try the mapping, if it fails with EBUSY, unmap the region and try * again. This shouldn't be necessary, but we sometimes see it in @@ -602,7 +631,8 @@ static bool vfio_listener_skipped_section(MemoryRegionS= ection *section) =20 /* Called with rcu_read_lock held. */ static bool vfio_get_xlat_addr(IOMMUTLBEntry *iotlb, void **vaddr, - ram_addr_t *ram_addr, bool *read_only) + ram_addr_t *ram_addr, bool *read_only, + MemoryRegion **mrp) { MemoryRegion *mr; hwaddr xlat; @@ -683,6 +713,10 @@ static bool vfio_get_xlat_addr(IOMMUTLBEntry *iotlb, v= oid **vaddr, *read_only =3D !writable || mr->readonly; } =20 + if (mrp !=3D NULL) { + *mrp =3D mr; + } + return true; } =20 @@ -690,6 +724,7 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, IOM= MUTLBEntry *iotlb) { VFIOGuestIOMMU *giommu =3D container_of(n, VFIOGuestIOMMU, n); VFIOContainer *container =3D giommu->container; + MemoryRegion *mr; hwaddr iova =3D iotlb->iova + giommu->iommu_offset; void *vaddr; int ret; @@ -708,7 +743,7 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, IOM= MUTLBEntry *iotlb) if ((iotlb->perm & IOMMU_RW) !=3D IOMMU_NONE) { bool read_only; =20 - if (!vfio_get_xlat_addr(iotlb, &vaddr, NULL, &read_only)) { + if (!vfio_get_xlat_addr(iotlb, &vaddr, NULL, &read_only, &mr)) { goto out; } /* @@ -718,7 +753,7 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, IOM= MUTLBEntry *iotlb) * of vaddr will always be there, even if the memory object is * destroyed and its backing memory munmap-ed. */ - ret =3D vfio_dma_map(container, iova, + ret =3D vfio_dma_map(container, mr, iova, iotlb->addr_mask + 1, vaddr, read_only); if (ret) { @@ -780,7 +815,7 @@ static int vfio_ram_discard_notify_populate(RamDiscardL= istener *rdl, section->offset_within_address_space; vaddr =3D memory_region_get_ram_ptr(section->mr) + start; =20 - ret =3D vfio_dma_map(vrdl->container, iova, next - start, + ret =3D vfio_dma_map(vrdl->container, section->mr, iova, next - st= art, vaddr, section->readonly); if (ret) { /* Rollback */ @@ -888,6 +923,24 @@ static void vfio_unregister_ram_discard_listener(VFIOC= ontainer *container, g_free(vrdl); } =20 +static void vfio_listener_begin(MemoryListener *listener) +{ + VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + + container->will_commit =3D 1; +} + +static void vfio_listener_commit(MemoryListener *listener) +{ + VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + + /* wait for any async requests sent during the transaction */ + if (container->proxy !=3D NULL) { + vfio_user_drain_reqs(container->proxy); + } + container->will_commit =3D 0; +} + static void vfio_listener_region_add(MemoryListener *listener, MemoryRegionSection *section) { @@ -1080,7 +1133,7 @@ static void vfio_listener_region_add(MemoryListener *= listener, } } =20 - ret =3D vfio_dma_map(container, iova, int128_get64(llsize), + ret =3D vfio_dma_map(container, section->mr, iova, int128_get64(llsize= ), vaddr, section->readonly); if (ret) { error_setg(&err, "vfio_dma_map(%p, 0x%"HWADDR_PRIx", " @@ -1346,7 +1399,7 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifier= *n, IOMMUTLBEntry *iotlb) } =20 rcu_read_lock(); - if (vfio_get_xlat_addr(iotlb, NULL, &translated_addr, NULL)) { + if (vfio_get_xlat_addr(iotlb, NULL, &translated_addr, NULL, NULL)) { int ret; =20 ret =3D vfio_get_dirty_bitmap(container, iova, iotlb->addr_mask + = 1, @@ -1463,6 +1516,8 @@ static void vfio_listener_log_sync(MemoryListener *li= stener, } =20 static const MemoryListener vfio_memory_listener =3D { + .begin =3D vfio_listener_begin, + .commit =3D vfio_listener_commit, .region_add =3D vfio_listener_region_add, .region_del =3D vfio_listener_region_del, .log_global_start =3D vfio_listener_log_global_start, diff --git a/hw/vfio/user.c b/hw/vfio/user.c index b68ca1279d..06bcd46e60 100644 --- a/hw/vfio/user.c +++ b/hw/vfio/user.c @@ -408,6 +408,47 @@ static void vfio_user_send_recv(VFIOProxy *proxy, VFIO= UserHdr *msg, } } =20 +void vfio_user_drain_reqs(VFIOProxy *proxy) +{ + VFIOUserReply *reply; + bool iolock =3D 0; + + /* + * Any DMA map/unmap requests sent in the middle + * of a memory region transaction were sent async. + * Wait for them here. + */ + QEMU_LOCK_GUARD(&proxy->lock); + if (proxy->last_nowait !=3D NULL) { + iolock =3D qemu_mutex_iothread_locked(); + if (iolock) { + qemu_mutex_unlock_iothread(); + } + + reply =3D proxy->last_nowait; + reply->nowait =3D 0; + while (reply->complete =3D=3D 0) { + if (!qemu_cond_timedwait(&reply->cv, &proxy->lock, wait_time))= { + error_printf("vfio_drain_reqs - timed out\n"); + break; + } + } + + if (reply->msg->flags & VFIO_USER_ERROR) { + error_printf("vfio_user_rcv error reply on async request "); + error_printf("command %x error %s\n", reply->msg->command, + strerror(reply->msg->error_reply)); + } + proxy->last_nowait =3D NULL; + g_free(reply->msg); + QTAILQ_INSERT_HEAD(&proxy->free, reply, next); + } + + if (iolock) { + qemu_mutex_lock_iothread(); + } +} + static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, uint32_t size, uint32_t flags) { @@ -715,6 +756,89 @@ int vfio_user_validate_version(VFIODevice *vbasedev, E= rror **errp) return 0; } =20 +int vfio_user_dma_map(VFIOProxy *proxy, struct vfio_iommu_type1_dma_map *m= ap, + VFIOUserFDs *fds, bool will_commit) +{ + VFIOUserDMAMap *msgp =3D g_malloc(sizeof(*msgp)); + int ret, flags; + + /* commit will wait, so send async without dropping BQL */ + flags =3D will_commit ? (NOIOLOCK | NOWAIT) : 0; + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_DMA_MAP, sizeof(*msgp), 0); + msgp->argsz =3D map->argsz; + msgp->flags =3D map->flags; + msgp->offset =3D map->vaddr; + msgp->iova =3D map->iova; + msgp->size =3D map->size; + + vfio_user_send_recv(proxy, &msgp->hdr, fds, 0, flags); + ret =3D (msgp->hdr.flags & VFIO_USER_ERROR) ? -msgp->hdr.error_reply := 0; + + if (!(flags & NOWAIT)) { + g_free(msgp); + } + return ret; +} + +int vfio_user_dma_unmap(VFIOProxy *proxy, + struct vfio_iommu_type1_dma_unmap *unmap, + struct vfio_bitmap *bitmap, bool will_commit) +{ + struct { + VFIOUserDMAUnmap msg; + VFIOUserBitmap bitmap; + } *msgp =3D NULL; + int msize, rsize, flags; + + if (bitmap =3D=3D NULL && (unmap->flags & + VFIO_DMA_UNMAP_FLAG_GET_DIRTY_BITMAP)) { + error_printf("vfio_user_dma_unmap mismatched flags and bitmap\n"); + return -EINVAL; + } + + /* can't drop BQL until commit */ + flags =3D will_commit ? NOIOLOCK : 0; + + /* + * If a dirty bitmap is returned, allocate extra space for it + * otherwise, just send the unmap request + */ + if (bitmap !=3D NULL) { + msize =3D sizeof(*msgp); + rsize =3D msize + bitmap->size; + msgp =3D g_malloc0(rsize); + msgp->bitmap.pgsize =3D bitmap->pgsize; + msgp->bitmap.size =3D bitmap->size; + } else { + /* can only send async if no bitmap returned */ + flags |=3D will_commit ? NOWAIT : 0; + msize =3D rsize =3D sizeof(VFIOUserDMAUnmap); + msgp =3D g_malloc0(rsize); + } + + vfio_user_request_msg(&msgp->msg.hdr, VFIO_USER_DMA_UNMAP, msize, flag= s); + msgp->msg.argsz =3D unmap->argsz; + msgp->msg.flags =3D unmap->flags; + msgp->msg.iova =3D unmap->iova; + msgp->msg.size =3D unmap->size; + + vfio_user_send_recv(proxy, &msgp->msg.hdr, NULL, rsize, flags); + if (msgp->msg.hdr.flags & VFIO_USER_ERROR) { + g_free(msgp); + return -msgp->msg.hdr.error_reply; + } + + if (bitmap !=3D NULL) { + memcpy(bitmap->data, &msgp->bitmap.data, bitmap->size); + } + if (!(flags & NOWAIT)) { + g_free(msgp); + } + + return 0; +} + int vfio_user_get_info(VFIODevice *vbasedev) { VFIOUserDeviceInfo msg; --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132681; cv=pass; d=zohomail.com; s=zohoarc; b=dk8bIzs8+B//ngB/O8CYYaB10I6cNsfVPIigYRmmp58okyZTg/i3DZtQdWMXIPMchBF2cqNXaF8h/uVgTwD9WOsmgXVwa8/4XPRHwnmVbsQO4gSXTfnRLLg1kAP4qssf3vBjskbQ7k7Q6ad55nIrngKZb4P6h9MOwQhgWK9BJbg= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132681; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=nmiwh8p0OoAJzqrtdn4jXK/dOs6qH6Bv7KKpzspOa/M=; b=PhpwCtvNnJrkZV+3MhDbpfGfkgkkr74JfV6BrHZYDquFHnZZ6fDYM/cRdmh5XMObSq71yurDhdjCO+N4rGM/ZfHEgt4lSP55zNQrmUANTU0uu/Uc3fD0xtex2bQ19FRuQHZ9CCvoeXVVFREXqD9KW3N0XiF8zXQNNi7j/Pld/KY= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629132681106852.3931507726613; Mon, 16 Aug 2021 09:51:21 -0700 (PDT) Received: from localhost ([::1]:44692 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfpX-0006fT-VL for importer@patchew.org; Mon, 16 Aug 2021 12:51:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37214) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhh-0000yp-JW for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:13 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:55508) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhc-00080w-As for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:13 -0400 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGbX1f029459; Mon, 16 Aug 2021 16:43:06 GMT Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3af3kxtg10-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:06 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeiuR113652; Mon, 16 Aug 2021 16:43:05 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2108.outbound.protection.outlook.com [104.47.58.108]) by aserp3020.oracle.com with ESMTP id 3ae5n609mu-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:05 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by SJ0PR10MB4765.namprd10.prod.outlook.com (2603:10b6:a03:2af::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.15; Mon, 16 Aug 2021 16:43:03 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:43:03 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:43:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=nmiwh8p0OoAJzqrtdn4jXK/dOs6qH6Bv7KKpzspOa/M=; b=PSk1f6/CLsx1HgpvgJAQpYRz59RlcLV4S4f6cltJiJEZgjLt1j93Sb/erPSfQUP4hbcH bQFLF7yf5d9GtpYuVTO2EeiwhwxJv8p17Lw+yYKKK5GMbZh+2eKEPFYzh+mKuQCx3z0t jSu33eZEA28S+akFDXwASGB3xLXfI9NoMySMScHQJzXVlJbYnFn6owkxiAOwu90NR/kX jOSGkCzfRv5S1hzxchKSpHdv1GSiZajrbL4gLUAQG3Kx2OB/jQA303CrfO8+Jw6T6EKB UYSz2SJ2BS54oNFYRUvle9XcY5iEhXwbQHzEXX29S16XzFC4IRrcLqiU4le7sv2iaU98 XQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=nmiwh8p0OoAJzqrtdn4jXK/dOs6qH6Bv7KKpzspOa/M=; b=GCWaF3au+n/ojY+lKyDmwJtODOhRRh5FxzRLPSwqYgXPIL46hddQYDW10CDHmyIotx78 FNKppfnONxJbGAXsVO0BDiEBpYe1Db4zRFw3svQg94OTla+B3T/4XYcYeyGTsP0r9KtH LPR2rX6juNAIHJ3BuA/LRMs+j2MNmvPgvObAvxdhxmOWktVwy+xdDcRpf9ohyGdS9kqA 1rIH3OEcfg4T6Le0V/GcWoTZfCYj2o5KLQYKxA4Dz1qCTavLlTP764fVDQT3oUrsC9Hp 3zD/BvZYZt/Mb8w1KIh/Te9F7FNJiHTepK1tIUSd6KGkE2Sey/r8D1mWEkSwFhGNtiAO 0A== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mONJUyB1nRBK/X7KQnfKF8Bp1vrTfbu9wmBeJCDFcds8QfUZbfOJQXcy/BWu36D/SBxWO816wh7Jkw6bYfdS1ax59FUJLHrLYNQXLIuk4XoUHK3dfIav9Fxd9O9d9JAGd1OWI4HACDX7UZcXBThBRyLZ9iiP9rZabIDsR0MWlCMKGpr/2kaACnAPiDm2EkfyQCupMXHKCFn4n4bbWPBmhw3mW/h8RwhGdHm9m5kJjliR8125NQgcmz97wwbzBQ/SHtuVSC26Ys1JrMDnHScI1Y9Q0cUSMkcS4ktfd3wGEl6mmuXX8zrflxFHabgpY35JIuWJ9ntCf37oWfzPHKUhzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nmiwh8p0OoAJzqrtdn4jXK/dOs6qH6Bv7KKpzspOa/M=; b=Xbwpnuu549emb2MxaDOIUCjn7w4MLXPorQmvM/X7kM72DdHJIONSh/2oFelKmodcaioIEOkmZzaD1ECDbjGA+2L68B7tdb+wLa7vfhBxNHE/W6OzbRZKrQ/DNGcEwTtU/+o5416cSbtSwLAsfzUWmtfqTGSKJfnXwVlDeOvOs7uvWRlQwiVFG50sFSggER3bUv3zbagoWdy9ZiblluR/JZc4rApkWrPb8ANGMj0Fq0VoFvnkSjcqHicWea0xZwBOOwaxpsNBqZ3rVOD7DDofu8zjsroW5JVTZSerMeOgiNzdcjdhpCXlV4vhejNbEdUQ3mhDWoCkunPiBVPLCYQG9A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nmiwh8p0OoAJzqrtdn4jXK/dOs6qH6Bv7KKpzspOa/M=; b=u8NKmG1RC66ISucnQRrWWq4/ShTwdbiLW+9Z5VNTt1RnXNWCSwC5SHl74R3+wDE3Xkx+w10qLgMiZHHtbGq2OABHZRd8tKTYHRhffSAScirJ8lPUv4I72peK2V2J5H7CqPRxSm4hu5xgLBEH+1hlekkn3eXW2UAUcVX+SoHOf4g= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 14/16] vfio-user: dma read/write operations Date: Mon, 16 Aug 2021 09:42:47 -0700 Message-Id: <7b21118256af2cb3d0dfe45b1e4ef9683fabccb5.1629131628.git.elena.ufimtseva@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 321be7c0-0342-4b19-6423-08d960d4ea49 X-MS-TrafficTypeDiagnostic: SJ0PR10MB4765: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:962; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ynqDGncm38DR+tjBZEW7cEQtxSzNwsWYRVhG2tktWOKz4hMHMsLOzIXZZysG6cc5IxwD+fdVL/UdvCCQ7SYPl+wQUSR4IGAFLFsPUYbZ6TWFVQF/9qDQqxo9Erx8CHcIM/u+7DWPAxh+hS4UutsMhPWYEfUV3W3m1X4DTYGzLGCCX1WsY0HOfApVfPdRkhY9suFbG+a7IGpimrx9PTab9ZHMDGLQGd9hlCnmrcqibRQDwP6lBYYQWQI0Z1yB6pE5wxAx+3xWq5Dlm0xcYt/Y5eECyfONnXlsIe49ong5KKaoTVWyB1gKgtAummPoBblVHuH5FEAfvXwv3tM+QNBQH9tkaY58MmLs9CB+J3I+6VUNYkz+bahYLF9skmz0ucRPO+uKyjNFRKtS2lV4boJvmSBYqYomHw58ni0U+BEaArO+Jg9bAdoqXocbgTu6f1e2AjApA+7Q99EyETSkWuPx4BiMlPIWa0ysLmZrCZ07HTgWFEwpFhrZnCBGrfuR3iVn9p5w9CpPvxjs5RCEwwq7C+/nEHNsK+cMUbPPVcNXGlx0wI/gavCmci/0BgCGAI1roeC5nuKQA0plMAapwZAcEOHLxxFz5rILq4CTlDkTFfOQ38pSsJSi5VEePFVoUP8x+UHC8bG0+CsjZY807vi3pg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(186003)(2906002)(86362001)(66556008)(4326008)(38100700002)(8676002)(508600001)(66476007)(6486002)(66946007)(6506007)(5660300002)(44832011)(2616005)(6512007)(6666004)(36756003)(6916009)(316002)(52116002)(83380400001)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?mWCH18PknI++ABn1aurzv0Gj0sqzJBftp8akXBsBnmzsEN21HnY66oGRRctv?= =?us-ascii?Q?+MRPkt4DQJ168ExueN4TdQdnUD8WTN32FfatwLIOJGKh3s36C7utPbWVGNIt?= =?us-ascii?Q?2NfjhRDYl+WKwcA7rzHA8iKX0R08Lab1SI1Jv+eVHvRfPMc/glnE60N2qvgw?= =?us-ascii?Q?QnYrQ2HhYosfN6scRGRdUzf2KXHcBDbvy9hxg4MKy5Ppt/pPShv5toD/YXxK?= =?us-ascii?Q?hUyBSwK5Slsy0O8aSmW8hbxpsJyKchU8M24aIjecjlsqJa4xgDmBVqfjU532?= =?us-ascii?Q?v5iD7ldwe8bk5mViz+zVn8x9ihMdjj78qLTMRaux17P/DVXUMRF5jDJ56Y0O?= =?us-ascii?Q?JyItZiLjWGH1FFnvrI+f7K1W0mUPgt8JUxTzgoZgcbbMd1XZUrGUcuuN6m99?= =?us-ascii?Q?JNjZWY0nc4bzJzwpV3Wj0XIarwTXhjMdkK77yb+7Hxm8nZqaZrB1+mYKLjly?= =?us-ascii?Q?D7qhqOciB+gEFp7n7Uw8aNipqf9tAjYBBICfl4rpzrLaowfhZyQ5Ki7R17+z?= =?us-ascii?Q?Rl+4t9IIUcntBhBwLZ27rBFXV/WjdvsKzY1sJLkbntje46R6uX/8tiQvg+8s?= =?us-ascii?Q?y7CKeOOvKWnv+D8TzHJtdRR8o+fwHy2v+oIzAmhK3HneuwryTpzn5SD3y2Pb?= =?us-ascii?Q?tsReJdK5OESbCeQuTx29fpvk3jV8GOtnMLUupXVcLFN/eOXqGZLVHjqS55Ld?= =?us-ascii?Q?Yof1HVwOLmThCrTDJ7VvTp87p+nyB6VZvRCvd9KtfKONFl/Mvq0HPIplPoxq?= =?us-ascii?Q?Vzc/ZiFkUBOSWzlB27zYXUaYoMqv0aKJDSm8cVXxSHgbi02ew/y5kWoAgddd?= =?us-ascii?Q?lwJtdknXj9VkP+FaFCuu1mofwSAhOY88kQoFSPPDYn8I6lWklPeWEi9s3/NN?= =?us-ascii?Q?bV7c+3N0+GlVZhKvHF/pRukac3083dLVvmw3oSyZxEMsm2yugZY3Bezxkqaz?= =?us-ascii?Q?CPZFKOFCKK2ldwZf6+5SvcjTHtldTCWUBiZf0gDj1chvPRnc7v4FhfHoREDh?= =?us-ascii?Q?FscrW5OxGOTXlLWOBWSMHKZxuz1omvufvjAsNtCaT1A5N483aV+uYvAexCmC?= =?us-ascii?Q?vRkOrpM6JDh194W4i6v9JQrVX1B0HDpJmg75LBe16KHz3YAjeNFaySX12h5f?= =?us-ascii?Q?XfTCnIm07rgjzvfQDS5MNz10haLJ41xgxGAQBjoioCXYyyjhxMX/MlvSrsfu?= =?us-ascii?Q?OBuOwDSZDHWJiKMhThIkGg/RHn+bKRSVn0ax+RyfHBTXj4OGk45YHnwRqSXH?= =?us-ascii?Q?5pna/q4xuakVV4uxTT2sCprf6cPdP5D2IHBalEDfsIaC+Qv3RC/2WfLo2RPr?= =?us-ascii?Q?zbX+4oZeSdcP+k3ew0feLN5zPdNdd0vo8gByRBHkrMtoEx4im6Z1VLn8QjPE?= =?us-ascii?Q?8V/ORLesYU0lw3OriVdvyJcpWcN1?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 321be7c0-0342-4b19-6423-08d960d4ea49 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:43:03.0876 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3TwkrLXeAn0/D7mhHBsSZ+w6+P29Wch9/zYC9DgMtsFS1CRI7CCxVpAZcivsCntndLbUugFT5tyk7oYPrYNQoNiZ5L6MKhvcTHWwxC6IzfY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4765 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-GUID: 1BRbLtOUmRBkw1CHLxE7Rjz3WMV4-ent X-Proofpoint-ORIG-GUID: 1BRbLtOUmRBkw1CHLxE7Rjz3WMV4-ent Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132682573100001 Content-Type: text/plain; charset="utf-8" From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 11 +++++++ hw/vfio/user.h | 1 + hw/vfio/pci.c | 63 ++++++++++++++++++++++++++++++++++++++++- hw/vfio/user.c | 7 ++++- 4 files changed, 80 insertions(+), 2 deletions(-) diff --git a/hw/vfio/user-protocol.h b/hw/vfio/user-protocol.h index ca53fce5f4..c5d9473f8f 100644 --- a/hw/vfio/user-protocol.h +++ b/hw/vfio/user-protocol.h @@ -171,6 +171,17 @@ typedef struct { char data[]; } VFIOUserRegionRW; =20 +/* + * VFIO_USER_DMA_READ + * VFIO_USER_DMA_WRITE + */ +typedef struct { + VFIOUserHdr hdr; + uint64_t offset; + uint32_t count; + char data[]; +} VFIOUserDMARW; + /*imported from struct vfio_bitmap */ typedef struct { uint64_t pgsize; diff --git a/hw/vfio/user.h b/hw/vfio/user.h index 7786ab57c5..32e8b70d28 100644 --- a/hw/vfio/user.h +++ b/hw/vfio/user.h @@ -65,6 +65,7 @@ typedef struct VFIOProxy { =20 VFIOProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp); void vfio_user_disconnect(VFIOProxy *proxy); +uint64_t vfio_user_max_xfer(void); void vfio_user_set_reqhandler(VFIODevice *vbasdev, int (*handler)(void *opaque, char *buf, VFIOUserFDs *fds), diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 2c9fcb2fa9..29a874c066 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3406,11 +3406,72 @@ type_init(register_vfio_pci_dev_type) * vfio-user routines. */ =20 -static int vfio_user_pci_process_req(void *opaque, char *buf, VFIOUserFDs = *fds) +static int vfio_user_dma_read(VFIOPCIDevice *vdev, VFIOUserDMARW *msg) { + PCIDevice *pdev =3D &vdev->pdev; + char *buf; + int size =3D msg->count + sizeof(VFIOUserDMARW); + + if (msg->hdr.flags & VFIO_USER_NO_REPLY) { + return -EINVAL; + } + if (msg->count > vfio_user_max_xfer()) { + return -E2BIG; + } + + buf =3D g_malloc0(size); + memcpy(buf, msg, sizeof(*msg)); + + pci_dma_read(pdev, msg->offset, buf + sizeof(*msg), msg->count); + + vfio_user_send_reply(vdev->vbasedev.proxy, buf, size); + g_free(buf); return 0; } =20 +static int vfio_user_dma_write(VFIOPCIDevice *vdev, + VFIOUserDMARW *msg) +{ + PCIDevice *pdev =3D &vdev->pdev; + char *buf =3D (char *)msg + sizeof(*msg); + + /* make sure transfer count isn't larger than the message data */ + if (msg->count > msg->hdr.size - sizeof(*msg)) { + return -E2BIG; + } + + pci_dma_write(pdev, msg->offset, buf, msg->count); + + if ((msg->hdr.flags & VFIO_USER_NO_REPLY) =3D=3D 0) { + vfio_user_send_reply(vdev->vbasedev.proxy, (char *)msg, + sizeof(msg->hdr)); + } + return 0; +} + +static int vfio_user_pci_process_req(void *opaque, char *buf, VFIOUserFDs = *fds) +{ + VFIOPCIDevice *vdev =3D opaque; + VFIOUserHdr *hdr =3D (VFIOUserHdr *)buf; + int ret; + + if (fds->recv_fds !=3D 0) { + return -EINVAL; + } + switch (hdr->command) { + case VFIO_USER_DMA_READ: + ret =3D vfio_user_dma_read(vdev, (VFIOUserDMARW *)hdr); + break; + case VFIO_USER_DMA_WRITE: + ret =3D vfio_user_dma_write(vdev, (VFIOUserDMARW *)hdr); + break; + default: + error_printf("vfio_user_process_req unknown cmd %d\n", hdr->comman= d); + ret =3D -ENOSYS; + } + return ret; +} + /* * Emulated devices don't use host hot reset */ diff --git a/hw/vfio/user.c b/hw/vfio/user.c index 06bcd46e60..fcc041959c 100644 --- a/hw/vfio/user.c +++ b/hw/vfio/user.c @@ -54,6 +54,11 @@ static void vfio_user_send_recv(VFIOProxy *proxy, VFIOUs= erHdr *msg, * Functions called by main, CPU, or iothread threads */ =20 +uint64_t vfio_user_max_xfer(void) +{ + return max_xfer_size; +} + static void vfio_user_shutdown(VFIOProxy *proxy) { qio_channel_shutdown(proxy->ioc, QIO_CHANNEL_SHUTDOWN_READ, NULL); @@ -251,7 +256,7 @@ void vfio_user_recv(void *opaque) *reply->msg =3D msg; data =3D (char *)reply->msg + sizeof(msg); } else { - if (msg.size > max_xfer_size) { + if (msg.size > max_xfer_size + sizeof(VFIOUserDMARW)) { error_setg(&local_err, "vfio_user_recv request larger than max= "); goto fatal; } --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629132675; cv=pass; d=zohomail.com; s=zohoarc; b=Hulo/DqsEEiRe/3YQsRDJEfY66G/9+KgBET6DQbIHwAT0mTYW9O3i5mnOZ1JwHn8CORmeHzsG7cvRcKGjMppfyGKTx/70WZhxo9nytRbDJLGkrjp+l4+cSjxGNqYrG3XN8Hz+FGF8R3TFT1+iUrLki2/8zmWJRtjl5Q+sowiVvs= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629132675; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=gK+Q0/rA5s16W+k1rf3tURI3UrxB7LD4G/Aog3Zoeyk=; b=ml2p7OxwUCqG8GXfPgb3xO7fcn95RGaN7EWBpTWEtd6PpMzel1ruDJNoSHiXH7vQOilrOse1cOA1hM7HQcFlcuNZwxyCeEtFh+RPFGGyFvO3VEOimhVXdnSfinZ+jDHgEHoWb4U2aVGknTCTm1dHSdIPqNZ3LXJPPWGpxhwDppg= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629132675737273.32968450094165; Mon, 16 Aug 2021 09:51:15 -0700 (PDT) Received: from localhost ([::1]:44254 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfpS-0006OB-9Y for importer@patchew.org; Mon, 16 Aug 2021 12:51:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37176) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhg-0000vD-Ib for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:12 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:45702) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhb-000810-Pw for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:12 -0400 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGZfZ6008090; Mon, 16 Aug 2021 16:43:06 GMT Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3afgpghjq2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:06 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeiuS113652; Mon, 16 Aug 2021 16:43:05 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2108.outbound.protection.outlook.com [104.47.58.108]) by aserp3020.oracle.com with ESMTP id 3ae5n609mu-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:05 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by SJ0PR10MB4765.namprd10.prod.outlook.com (2603:10b6:a03:2af::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.15; Mon, 16 Aug 2021 16:43:03 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:43:03 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:43:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=gK+Q0/rA5s16W+k1rf3tURI3UrxB7LD4G/Aog3Zoeyk=; b=MKNTClW7qizuI8SW16g1By3eofYdoZmuzg/eMd+HXURfYRfYHHyjpPZqoZw9CQL1vTxm zED8psAEm8f9FzVTmUCKZWr+RPOAwZhPYt2AgHF/4W/XR9rKNZgkAqddFBjDj8wQSNVe M+RLqTKoLOGVZgzzHTHF4NHcMqH/KrqVtyGqX2Vvph5kPDUiaRbzWjqAPMdmBD1bn2KO cqRb8O8w4xhk0ZpFU7JqveSCjbRFye3IGf7W/J7zKwdp01FJ7h7U6y6hcOAHPreTeOgZ 9EBdNL2kbWW6q9+Iyy72HfYjc3pA9sJFSHJ9dtU9eFbmPlOgmsRIifhpWqGOp6eBZXqm CQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=gK+Q0/rA5s16W+k1rf3tURI3UrxB7LD4G/Aog3Zoeyk=; b=oC2GmxKr7MtYe4tyE15V07HfaNwAPj0jyWEX+kNfJH2aL2zafQ4/YG6iX14cuDvmkJOh 6jcXB7nVPQp+B5jvHTrUg2j/SaUtvJGWljmDOeQpVr1XjqXhgB4I0X4QvYukGDzT8PXn r3JlEUHpjij5U7hbyeJFS6j2rRm14HtMw6sDMWaxgsNekp3An+qO5W/V5l8JZbsSP9ho 3XQkJ4+LkjSUIuWpWI7lCBz9wMuRZ0j3f1S5AObmyQDKQhAJvNKfUr1nMWt5T3Z2fqkh WKHk43Y6yQY9r339lPNBidGQ7F4gronWiLlclbIpuOSCsmrgLpIIzADjW770le1JiLF0 Qg== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Hc5hcOl6Fv7y2iJPiQjBIFoFphWKJeBt2KDsL1XIKJVrfMkd96MmSYjQkYv0ZNuyq51AC7T5MqyVXupogba15KoIxZnsJADd/FpgVpk5OtUTEPwAMUCezp5/dHb9w3xtpZh/k0o7EMN1HwCURVbsJD9vBlqvEWX1YwfFuTBl/RZ1yYlvQNlZZNxa0WTT994wqX5yJBXGrmvLh2UBlKcvEFTiq7X2KVSbJBgjwmjs9L2CiwGQM+lFOyg/xbQMhOhx80k9gxFxQkjWQTz6iZha1MjIUe2Nxdbp0RtOVpDEmx0/N2lFWsS9kmkPneaZUXrxPEJ/ofe/CyBXdQs2SEFAhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gK+Q0/rA5s16W+k1rf3tURI3UrxB7LD4G/Aog3Zoeyk=; b=Jq/i33fjCRnQza2cO9Jvz4kR2ph03wytaT1vhOdaWeTSel+oa8LuKMGJ5HMxwwtLg4r8dZdDX4x7KY4hhenG6MzGfHw+Pkaf5m7HvckTXu2sdYxu2SCN/5b96CMnoS5gc+//TcYV8CV1ESOnZXp/cSD4u8RPPK6hWh9q5fqS1FYXZkOZACHCI24HgkVCIhrQUqS6yjFLv2hLzaKfCh7IweXcUkjuozK/Z6o9AoCI1ym3H9YV9QQStqfEbuIGxW5sWWj/VDpGw/ntJfIU3fwN7gz2FN/V+my35Eao/nigP8sg8J8aWgfh2elNYyo3rMt6BHRNSC4CmuMLC9I8zZ23ig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gK+Q0/rA5s16W+k1rf3tURI3UrxB7LD4G/Aog3Zoeyk=; b=oiyHiytyDCe4diT+Fg5GE8fvu6ExLCJQK5OKBCh+Drs6LNw9/df2ney/o00jMmtxAWOCBLGoKELNBOccn1mbFAABzrWU+GMZE3eGCVQpabnvV2cmM77UIKhJB4JVUXoNXwPrsdbc0HAeFdfcDWQaLZyaogahXSywSNGZYTKzXS8= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 15/16] vfio-user: pci reset Date: Mon, 16 Aug 2021 09:42:48 -0700 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8aed286d-24d3-4e40-2ab4-08d960d4ea97 X-MS-TrafficTypeDiagnostic: SJ0PR10MB4765: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:183; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: E0U5mPtOVIU/v5M/IPeb953MiTn/2s+XkFwWAktW9wCsXiFAxXYFhTwTo6jdDyH6bprUjpVGWv9zs6KX0uJNLWTbvbfy159ArFvQZ7Gsc4DXScAjjlaE180n/yU/5sUxm5Lj+D4iMeVjJJsS4+WK7AJFOX9Ys0ijWqtM9bSrCc/44rxj0ijnrhCRZgUJ92T7XFmReIt1x8+1h1kztlvjU/xXuP7cuA8D5ys9CruHJs4AebMlMn0m/JPV1ChdpTlUyYpOFzas6bqgeFLEFFh82IurfwmfpSEqOujbBajqCvLJkopCX0JongMQ61VJaYtruJnNTmKzah42RA/mA2mYGR2vcTv7RJQIcelWr0pCgbDmdIbN4wqLsHAkcZEBQ4O+hk49ntPM7ZURsMNKi1J4OBZqXCT36lYJ5MMePcuCOSo/RAg6PusPdZu3VZpzAt8faLEk7gdGaNC8Y4SZbahp36wcx8bz67HafTrYVeLZxyR4A2YOpfY8gLs7vldjyz7G8dgaCou9ZYh38JsSe5AMUhADyMOs9tSJnV63luUllNMxXlxCcBQ82GC/9qIm/L7YyeD36BnFSAB3JMID21PXzwMYhR66P9O5NPACIgMF4lDLJ78bNdaSzYls4C3QevBSILCE37FtdwSN7X6yNGc3/A== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(186003)(2906002)(86362001)(66556008)(4326008)(38100700002)(8676002)(508600001)(66476007)(6486002)(66946007)(6506007)(5660300002)(44832011)(2616005)(6512007)(6666004)(36756003)(6916009)(316002)(52116002)(83380400001)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?xGwKZQqyRzSCxLG7+EHJ43cz8k5VaIpsmD4MLXouwmopVbsc0rSQmk90lMJw?= =?us-ascii?Q?tec3Jqf50b/5M5S55iH6IurJX6eJ0ZID1t9AHs/toWtKjwpeVUoeJpzbXwkI?= =?us-ascii?Q?KokXF3ftZXq675IRZOQcYRIwWzNye7212CVc6rj8DGACGj1n67lJH3GTV0Qt?= =?us-ascii?Q?fk2owyIidrhCa1lB0ET50m6nrMcQ/icbx1DQ5Kd+oH/uPyl+PwJSAts7AZYn?= =?us-ascii?Q?sKHo/xArOXUIadgtAYrxY/TwL67NXt6YxfPMqQS7JkBT3wOdzVu3+3ZQI5sF?= =?us-ascii?Q?AXcU8A2MnXyXJK0w3ekb0BP0fH4JjpcbinXTTuuTJt+C6brmR37pywFH7mXC?= =?us-ascii?Q?iEktnfNZ407CVa4fQFRTfSzCJ1wZrH9JPRaAhD/k2PMoxFeTFMPFy0M9JyTP?= =?us-ascii?Q?KR27ewUQPC2H2NaLiaE14FbleqflaubmTsNy3b/zvkHCH//TalNDXXjSAh6K?= =?us-ascii?Q?b17sBX5xDepIOg6TnO5qCkCPael6B2FkxwfuzSj0lI/NcXNZ4TbR/oS95HS2?= =?us-ascii?Q?urCgtAzG/2fbYugbGehgo8YjLg54tbnWMaHuA0bDeKmApHxIOixlUjRI7NCQ?= =?us-ascii?Q?jq12nIG/Z5YU0L3HgMdxWzd+LahHcuRR07pYh6Ad+aHoki7YS0m+75/tkCKk?= =?us-ascii?Q?G/qj9xRFcdTIpITA7TcqRjs4lxN4LSxqbzGYFjXxzLiF2vg0Q4rpFqtrOtwZ?= =?us-ascii?Q?BjUHMGtCl21es9V0JY7OrOIz4LTvTeQVwJz2wRpYf4Jz/fDfraLpLIlPdfjA?= =?us-ascii?Q?tcp3LmtZlX88s1LXM5rZ17YtdaDHzpSUsJMqidoXWeLpnjfSmIRWM1S6ahyH?= =?us-ascii?Q?snOUlhkU/g0lmtLyh40HSM0XIpw9x086kbqJW5EL13Xy/Jr+qFSJ7V7t38y7?= =?us-ascii?Q?OgPOYK+tt2DfSoTee7ps3R/dDScUPp/xXys/tJokdHinnA16fIdra7gGuYM6?= =?us-ascii?Q?RieYx4lFfwUhNuyoC3+x5dOdXbfqfGlL2GId3cPbZeLRFiS8x7vQq9xcsn8B?= =?us-ascii?Q?mg22prSbdlpGouMthRjJNwTC0WlCszVD4w4G/2G17CcexyKhbN/WROayLqEV?= =?us-ascii?Q?DkfGbU+2e0oZH7tGd4d2Ran27Al2Ba7pIVofAl7nY/KCq0evDFeuO7gU1Mik?= =?us-ascii?Q?oLrzBjZPa+rT075nIVANPJ9ZAGcczBtIFbqsN2W36GaMkXaVihr2LznahSga?= =?us-ascii?Q?vOvXwFFILYr3ho34UREbODvs0PSfnhD4ExgQNOIkZHwykiDDPlq+iPYjxynd?= =?us-ascii?Q?klMDyk7cqxlw2LygawgocMA2k6rFn5geXZB4PaGBt5fkkOfL2Z7PDGcndrr9?= =?us-ascii?Q?miG5vgYNZIjtYarAViSiD2ISY+z0r99hQLU3mXQyyOJpzRaQgcZmERNFUA4A?= =?us-ascii?Q?zaTFNN548ZagNlWTtgt6MkU9vfKP?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8aed286d-24d3-4e40-2ab4-08d960d4ea97 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:43:03.6013 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YvdoONj2+iKXTj1voFcxQfI1GGKR/jdlEXhLeWCEuLS+uOCcpsr8+gfiMm7gLc/PlFa/UL1zRaug+98IfzHA7nSVKKtV126uqMWTf3+6I2A= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4765 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-GUID: iI5wjs3fZpTrd9P67XQLNCynDIJL7nSk X-Proofpoint-ORIG-GUID: iI5wjs3fZpTrd9P67XQLNCynDIJL7nSk Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629132677379100001 Content-Type: text/plain; charset="utf-8" From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user.h | 1 + hw/vfio/pci.c | 29 ++++++++++++++++++++++++++--- hw/vfio/user.c | 12 ++++++++++++ 3 files changed, 39 insertions(+), 3 deletions(-) diff --git a/hw/vfio/user.h b/hw/vfio/user.h index 32e8b70d28..5d4d0a43ba 100644 --- a/hw/vfio/user.h +++ b/hw/vfio/user.h @@ -86,6 +86,7 @@ int vfio_user_region_read(VFIODevice *vbasedev, uint32_t = index, uint64_t offset, uint32_t count, void *data); int vfio_user_region_write(VFIODevice *vbasedev, uint32_t index, uint64_t offset, uint32_t count, void *data); +void vfio_user_reset(VFIODevice *vbasedev); void vfio_user_drain_reqs(VFIOProxy *proxy); =20 #endif /* VFIO_USER_H */ diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 29a874c066..4b933ed10f 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -2229,8 +2229,9 @@ static void vfio_pci_pre_reset(VFIOPCIDevice *vdev) =20 static void vfio_pci_post_reset(VFIOPCIDevice *vdev) { + VFIODevice *vbasedev =3D &vdev->vbasedev; Error *err =3D NULL; - int nr; + int ret, nr; =20 vfio_intx_enable(vdev, &err); if (err) { @@ -2238,11 +2239,18 @@ static void vfio_pci_post_reset(VFIOPCIDevice *vdev) } =20 for (nr =3D 0; nr < PCI_NUM_REGIONS - 1; ++nr) { - off_t addr =3D vdev->config_offset + PCI_BASE_ADDRESS_0 + (4 * nr); + off_t addr =3D PCI_BASE_ADDRESS_0 + (4 * nr); uint32_t val =3D 0; uint32_t len =3D sizeof(val); =20 - if (pwrite(vdev->vbasedev.fd, &val, len, addr) !=3D len) { + if (vbasedev->proxy !=3D NULL) { + ret =3D vfio_user_region_write(vbasedev, VFIO_PCI_CONFIG_REGIO= N_INDEX, + addr, len, &val); + } else { + ret =3D pwrite(vdev->vbasedev.fd, &val, len, + vdev->config_offset + addr); + } + if (ret !=3D len) { error_report("%s(%s) reset bar %d failed: %m", __func__, vdev->vbasedev.name, nr); } @@ -3684,6 +3692,20 @@ static void vfio_user_instance_finalize(Object *obj) vfio_user_disconnect(vbasedev->proxy); } =20 +static void vfio_user_pci_reset(DeviceState *dev) +{ + VFIOPCIDevice *vdev =3D VFIO_PCI_BASE(dev); + VFIODevice *vbasedev =3D &vdev->vbasedev; + + vfio_pci_pre_reset(vdev); + + if (vbasedev->reset_works) { + vfio_user_reset(vbasedev); + } + + vfio_pci_post_reset(vdev); +} + static Property vfio_user_pci_dev_properties[] =3D { DEFINE_PROP_STRING("socket", VFIOUserPCIDevice, sock_name), DEFINE_PROP_BOOL("secure-dma", VFIOUserPCIDevice, secure_dma, false), @@ -3695,6 +3717,7 @@ static void vfio_user_pci_dev_class_init(ObjectClass = *klass, void *data) DeviceClass *dc =3D DEVICE_CLASS(klass); PCIDeviceClass *pdc =3D PCI_DEVICE_CLASS(klass); =20 + dc->reset =3D vfio_user_pci_reset; device_class_set_props(dc, vfio_user_pci_dev_properties); dc->desc =3D "VFIO over socket PCI device assignment"; pdc->realize =3D vfio_user_pci_realize; diff --git a/hw/vfio/user.c b/hw/vfio/user.c index fcc041959c..7de2125346 100644 --- a/hw/vfio/user.c +++ b/hw/vfio/user.c @@ -1045,3 +1045,15 @@ int vfio_user_region_write(VFIODevice *vbasedev, uin= t32_t index, =20 return count; } + +void vfio_user_reset(VFIODevice *vbasedev) +{ + VFIOUserHdr msg; + + vfio_user_request_msg(&msg, VFIO_USER_DEVICE_RESET, sizeof(msg), 0); + + vfio_user_send_recv(vbasedev->proxy, &msg, NULL, 0, 0); + if (msg.flags & VFIO_USER_ERROR) { + error_printf("reset reply error %d\n", msg.error_reply); + } +} --=20 2.25.1 From nobody Sat May 11 20:39:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=2; a=rsa-sha256; t=1629133151; cv=pass; d=zohomail.com; s=zohoarc; b=fg65amyYibD+Osc+q042GT3joVW3zuBvKt8flKi66wY8fqfvGaUhoyW8i0R1hQSme7kWw5PLFJcTSBo4NL7u4zvM3ZM9iytuQ7+2wXaKZMW2PThDIUWGCAEsFjVkaOEsdEn/nVi7ZwHEFPCSziYmHuFYhHlsH6zRS7Rp26ov8tw= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1629133151; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=dEIQoWq1svlzZmaE3BBG0U9ChO0Ig3r7lzdRFu1zmXw=; b=j6Y9MujYo6KW9lBNh5HsKc3GQVzkECLStz14yj5gwpbBpKIG3cN22QPK3TfUN07PwJ1WwuQ8YmmREtIHl/AeLcKMDQ3LiNAdVftCBaP7RycGVG9ZpJVP2hnG1Iyr9ink64dVo0UwLi1EiyWX4dL5UR2FlwIUmulvdrZ/vZ0t+bc= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=oracle.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1629133151365319.11645073326076; Mon, 16 Aug 2021 09:59:11 -0700 (PDT) Received: from localhost ([::1]:34630 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFfx8-0001tv-9R for importer@patchew.org; Mon, 16 Aug 2021 12:59:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37230) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhi-00011t-N3 for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:14 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:57618) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFfhd-00081V-5a for qemu-devel@nongnu.org; Mon, 16 Aug 2021 12:43:14 -0400 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17GGbZUF029485; Mon, 16 Aug 2021 16:43:07 GMT Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3af3kxtg13-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:07 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 17GGeiuT113652; Mon, 16 Aug 2021 16:43:06 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2108.outbound.protection.outlook.com [104.47.58.108]) by aserp3020.oracle.com with ESMTP id 3ae5n609mu-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Aug 2021 16:43:06 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) by SJ0PR10MB4765.namprd10.prod.outlook.com (2603:10b6:a03:2af::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.15; Mon, 16 Aug 2021 16:43:04 +0000 Received: from BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539]) by BYAPR10MB2869.namprd10.prod.outlook.com ([fe80::5891:a7dc:5c2b:5539%7]) with mapi id 15.20.4415.023; Mon, 16 Aug 2021 16:43:04 +0000 Received: from nuker.hsd1.ca.comcast.net (2601:642:4000:a09:154a:3cd:532c:f36d) by BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Mon, 16 Aug 2021 16:43:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=dEIQoWq1svlzZmaE3BBG0U9ChO0Ig3r7lzdRFu1zmXw=; b=OKDYLn5jlPLLKJw0oYntG/Vss9HE+NyQ8eODHM9n6tq6/5c6RfZjKvd9xlHhE/Vrwg7E Q/bdqrzKAbbHC66RhBNojxR8Z2FZJlCt/98DkRTzt9M8bEoHNQfDueajU6bLnHBD22rY iSMqel09Ps1E1xPvUoAf4+tqd0C/ta+m2Vls5NXRT0Fg8dorDAE3lpdWAnT9dnZTYr8C gOUiiQIDOfSCfhEhU1N0K+U3J5o/6Dn3eMnZSmjzGXwXt0M71DqdY+1fXzMoN9DR60aL mor4UvQlcLWYNtKBhd8LzJN5dmsN+BczaD68YlMlQHnW/mZzOjjUvCPs0P6y2G1oqWEk FA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=dEIQoWq1svlzZmaE3BBG0U9ChO0Ig3r7lzdRFu1zmXw=; b=hlG96/HiLKSUUcNKBHlXx2XrOAJu9O3TkzmzdKakGALAdMFpd2yNaifLVh8GxvPK/yRJ 7lSg6tG5FxGVVWdeTWPh1f8w6Quf4EvOBtRU6aDx7jwD6IdxGTPiwE5+SiA24iZzooMc BcZJoe+mxN33VND3LEFXkwZeCj2aHiNI2XVls2xBO37OUQSBjQu3v3Qddkyqb9ksUl5k Hn4auw5p5fUpw0crhuWgBGKm/6QIjY8T+oVDu1BHJHat6RIcgEv4+tw1Q3p+mShOJr9H yMnMzZ9exTHeTwQm3ptclluUvAMv004JX+D7F+FDVaUDVNCN8jkrCUcU8x4pjyWWU/7j 6w== ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WNQyAwaxvOwS0hknTBmwjhzYQwwIBuQPe0EDuCD+fdRdsDf36gPUD2Xz6EndfLAwknpkrbKbjevyjNSmdnjUf4PUv1Axb1dvkvUsIaoV5P5ZdAcBBRJI+W8MTuBdoey1gRICveSjl4q4nKAxouDV3hxBAiH+F16VnTz/kxpWGmr59OCGKeX57pCppDQg+1Ez5my9qLCWiENlurZfAD7e+VTAKnKfVapZ6uroSJQLZiMg3crC7Ib57urr/qZ0OiWwUDdtuxo0Z5y8lKm5PqPTh0GhuiFsWcC6MY807vMK6WvXL1nX9e8AaJvKPmmCwwvVFWpN79Xwz8R5UE+jUoA21A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dEIQoWq1svlzZmaE3BBG0U9ChO0Ig3r7lzdRFu1zmXw=; b=aAp/X6AX+V2iTJsxQbG1xLOwOv2xAzg/r7FKfwmqBHJnmieQc2v79TmQbvFKCu2Hz7uu4DDyczglgpOA0jip7nxiSEu7UKWfRRXRseaTDroIzfbDrpNVRiVdXZPEbIioT6Belk5CXbXR4wA/+QLqoCH6JKE1YvXKp0P9OpAr6z9qxop0ET6YsePNmtsYYK/2rlhSPoPmz3372HTiMMtc6i5ftx3S23rAxaeyKWrj0+5KomPB2CvGo0S/sleq60m++wUrMQUIToUJDaymO42oZVxuIPLucDsQnHiMzkOJTUYquENN7Khh1CZvi90m/tOjzCwaNH5GvcNyVLHKmO1FeA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dEIQoWq1svlzZmaE3BBG0U9ChO0Ig3r7lzdRFu1zmXw=; b=aNTr/5y7I/m6XzXRDp2sZ65wzHZICmZLmc8uH9XtR/ehwOlyfPe9ZWzeSBN6aSjz3alzVeCTugKPWHXfW7bSL+gOaJdx04SDu+WzSQeth6mRWzdeZ6t5K9+PpmmqTZ3vl0cdKsGkohiGby0UcKFQYMMHChIgN5Wo4uH75F4Kd7g= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; From: Elena Ufimtseva To: qemu-devel@nongnu.org Subject: [PATCH RFC v2 16/16] vfio-user: migration support Date: Mon, 16 Aug 2021 09:42:49 -0700 Message-Id: <42758d8b56b1036d86cf2252c175071c1094a8b0.1629131628.git.elena.ufimtseva@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BY3PR03CA0026.namprd03.prod.outlook.com (2603:10b6:a03:39a::31) To BYAPR10MB2869.namprd10.prod.outlook.com (2603:10b6:a03:85::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 97624a51-8ec2-42bd-f8ee-08d960d4eaf2 X-MS-TrafficTypeDiagnostic: SJ0PR10MB4765: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:31; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pJPeXVDf5qfRZjqVXwEEnVMKvbGmKOn3qwwKkklYYbhOQFJseg3vdiViVA9GIE9f7vwHJKLXCpJdO0D8j0yxZxTDW2JeygDSiZdns6yKKVudlCRCw119dOyHEm+LQbMHsMWX2hP3RsLioq7NZu+fvY1tr3ZXQ9pEzSnwj/oYaJfEscvR+m1zseI+foBjwvu4qatcWDvKmRbttWO1GO+GWZUgXFu75Y5GnooXylyCFsSn4wR7V7UrpIYf6FzyJZ5bE1A5C4ftairpmGeJ2XZqlFb6Eg8ErdENseUp4DjrS+gBuJQ/PdRyGg5e0L1A7egx3TRqnIvzLZMrrJMiZrcQHYIeSG6oD8ieC2XqJM4R+Uzbrv5aN1sPh9L3jy5sah9sFLTNy7Tntj8Pg1ajKk448Ut5kBqlcOETq2aZXdkaR8R8ijIEKJopqVNEub97PcCrJSchJpNBOTpprGvC4F/0W4ZRi/fUK0i9eYOrY7VvqfCwxzLKbR05crp+pGecdsr8vlwW/Uv5lVTZfg239obeQ+N9NcyqizKcZt2aPbCr8Ba46e5HN0rrcUugcvNtWmPoCNyVQArcYT6oM5X/POTSmm6YRhM7SEYL9TmsjOKy0KHdqN0WX1G5ZCVflUoaeCpLG9kwpjbRaekKPNSJVu5qGA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2869.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(186003)(2906002)(86362001)(66556008)(4326008)(38100700002)(8676002)(508600001)(66476007)(6486002)(66946007)(6506007)(5660300002)(44832011)(2616005)(6512007)(6666004)(36756003)(6916009)(316002)(52116002)(83380400001)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?s+eeu+JresGXjtcgOvyAW6TR9fN/xf7yIhwa67hJM+9FUhjxQfSoeQ02REFZ?= =?us-ascii?Q?Vf0XRuBeoR3KlMwtZjOLAB1PScIRTvgQH0XycRjQ5Iw0XqSg1zsRyb/2Jt+K?= =?us-ascii?Q?mhcuMHAtjtbuUgbnTJRxXXbNeNWRjX/B9mQSI8zY/VgQQPe7Gq0lhNuxHX9t?= =?us-ascii?Q?TShL54XUUyestTUGaHKJcp2q30uJndtBtX+5q8iOx/Baq6/wsPrznRsUwRx1?= =?us-ascii?Q?rSteIeOV/xGyF2RmVTk6geCZK+kqWky1TJjKBW3iS+KYoOvHRmihFkPlyCoO?= =?us-ascii?Q?1ZNIZ465FqanI9Sv+g7+0MIfxwt4NBNbEuq6gVqG97WTzpu+/yI+qI73BhN+?= =?us-ascii?Q?VSwxgFLwhLz/04Iq07zYU/R1sg2gasAErxujzZw6CLqbPzAdYCRnc4BrR6Z+?= =?us-ascii?Q?f7HO5sLulITARLNWnSo6Dg6gBn/Xnf/qg8gRZvsrsHNVtPHjaXmTP9k3NQKY?= =?us-ascii?Q?vEnlybUQySx32A1PRQAIKvCVmMEhGtwwr8yhx4Vl85hqS0mfjPlRwyuX9OWp?= =?us-ascii?Q?pCn3fqy2gYTWqz7/Sn5JmdJ0ehCsepWhHOAqxxfDdB+efTr3XFR5QuLuRO4h?= =?us-ascii?Q?b4cH4kZi1agvIRB0Q5md4c20oWiJ94eoTd3x1E07a8XC2PW0gQQGYEmOBaS1?= =?us-ascii?Q?tfle6thA78N4J/QVmtgmwjgWrE2nc8PxDyZdcX/EFX7hsq4X2MGjjtzpQLCt?= =?us-ascii?Q?Jr0zMuQznh2Qgh450KP85/1JxL0/fqUtcmhLyLA2vU5W9WTGrYMxbGVn28wy?= =?us-ascii?Q?zizgbtb830Dp9D3AtjGpNa6W8SV5sdBXDwxOQYyd1riUarLaXKK54LkoZ1HZ?= =?us-ascii?Q?kgFabUrWU4hBhDLq9tpZ8ys1xKw30puBg4WFgQGUOzy61+YX5ejUWtqQJaPW?= =?us-ascii?Q?rMWcFXCybk636vpCZcgiyNdg3O9qM6+iSXDVSy+NaHTDIs+lyRpezOdJnjr6?= =?us-ascii?Q?BQG0jfGH3rt9ZXFluTo81FerLEgRuYjClNNuOkQp3KACXLYq+sHGKbSD4EBn?= =?us-ascii?Q?/9ScjbY5S7Ga5plB0UyhSFQshfi+wRP0AYOb0/L/HPU08hkHfACRka1GZOHc?= =?us-ascii?Q?Ix0fTiQe7vJ7bhoClJfRMoVnZGQGoO+o6Qu3eRpFz+1n2b+tGc/msM4LYsVB?= =?us-ascii?Q?CP67e9SFJdJv8nWWm4c2jaC/xptImVs63HqJxbzZ8zXD86k9LSJM07qLgwXx?= =?us-ascii?Q?WMzARGkbrYnOHrO/Gmfoy9S4UD7dCUXncEy2Tzp0HFI2DKlWzdCwZMC8KCYe?= =?us-ascii?Q?eOwYkIYMQhIkNXKwRB3ccz2f72Yi4jv+nMz7x1TunRArtHMA5n2+VMgEbruW?= =?us-ascii?Q?zXClBJSkEAO3/L15Ft0soPNk7OlQPBVY88e6bDFVUExRaBTTv+eh2dpQV7FW?= =?us-ascii?Q?kCdROrKtnNmDONc3ETmlTNne3WZo?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 97624a51-8ec2-42bd-f8ee-08d960d4eaf2 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2869.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2021 16:43:04.2320 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: efrAWQHbgL6kA5Zpf9yvfTfYVcoat4IHOpvd5HXIsOj22enzWU4ILHi7LAIdaYBDGRW9bwEH/etUbPWwoy43wSf03Ti7Upo7EUua14VTANQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4765 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10078 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108160105 X-Proofpoint-GUID: AHfl2hyeULaEDUMwpNRoxHwtbzZBwaKq X-Proofpoint-ORIG-GUID: AHfl2hyeULaEDUMwpNRoxHwtbzZBwaKq Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=elena.ufimtseva@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L3=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.onmicrosoft.com) X-ZM-MESSAGEID: 1629133153008100001 Content-Type: text/plain; charset="utf-8" From: John Johnson Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 18 +++++++++++++++++ hw/vfio/user.h | 3 +++ hw/vfio/common.c | 23 ++++++++++++++++----- hw/vfio/migration.c | 34 +++++++++++++++++-------------- hw/vfio/pci.c | 12 +++++++++++ hw/vfio/user.c | 45 +++++++++++++++++++++++++++++++++++++++++ 6 files changed, 115 insertions(+), 20 deletions(-) diff --git a/hw/vfio/user-protocol.h b/hw/vfio/user-protocol.h index c5d9473f8f..bad067a570 100644 --- a/hw/vfio/user-protocol.h +++ b/hw/vfio/user-protocol.h @@ -182,6 +182,10 @@ typedef struct { char data[]; } VFIOUserDMARW; =20 +/* + * VFIO_USER_DIRTY_PAGES + */ + /*imported from struct vfio_bitmap */ typedef struct { uint64_t pgsize; @@ -189,4 +193,18 @@ typedef struct { char data[]; } VFIOUserBitmap; =20 +/* imported from struct vfio_iommu_type1_dirty_bitmap_get */ +typedef struct { + uint64_t iova; + uint64_t size; + VFIOUserBitmap bitmap; +} VFIOUserBitmapRange; + +/* imported from struct vfio_iommu_type1_dirty_bitmap */ +typedef struct { + VFIOUserHdr hdr; + uint32_t argsz; + uint32_t flags; +} VFIOUserDirtyPages; + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio/user.h b/hw/vfio/user.h index 5d4d0a43ba..905e0ee28d 100644 --- a/hw/vfio/user.h +++ b/hw/vfio/user.h @@ -87,6 +87,9 @@ int vfio_user_region_read(VFIODevice *vbasedev, uint32_t = index, uint64_t offset, int vfio_user_region_write(VFIODevice *vbasedev, uint32_t index, uint64_t offset, uint32_t count, void *data); void vfio_user_reset(VFIODevice *vbasedev); +int vfio_user_dirty_bitmap(VFIOProxy *proxy, + struct vfio_iommu_type1_dirty_bitmap *bitmap, + struct vfio_iommu_type1_dirty_bitmap_get *range= ); void vfio_user_drain_reqs(VFIOProxy *proxy); =20 #endif /* VFIO_USER_H */ diff --git a/hw/vfio/common.c b/hw/vfio/common.c index a532e52bcf..09d0147df2 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1303,10 +1303,19 @@ static void vfio_set_dirty_page_tracking(VFIOContai= ner *container, bool start) dirty.flags =3D VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP; } =20 - ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, &dirty); - if (ret) { - error_report("Failed to set dirty tracking flag 0x%x errno: %d", - dirty.flags, errno); + if (container->proxy !=3D NULL) { + ret =3D vfio_user_dirty_bitmap(container->proxy, &dirty, NULL); + if (ret) { + error_report("Failed to set dirty tracking flag 0x%x errno: %d= ", + dirty.flags, -ret); + } + } else { + ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, &dirty); + if (ret) { + error_report("Failed to set dirty tracking flag 0x%x errno: %d= ", + dirty.flags, errno); + ret =3D -errno; + } } } =20 @@ -1356,7 +1365,11 @@ static int vfio_get_dirty_bitmap(VFIOContainer *cont= ainer, uint64_t iova, goto err_out; } =20 - ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, dbitmap); + if (container->proxy !=3D NULL) { + ret =3D vfio_user_dirty_bitmap(container->proxy, dbitmap, range); + } else { + ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, dbitmap); + } if (ret) { error_report("Failed to get dirty bitmap for iova: 0x%"PRIx64 " size: 0x%"PRIx64" err: %d", (uint64_t)range->iova, diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 82f654afb6..89926a3b01 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -27,6 +27,7 @@ #include "pci.h" #include "trace.h" #include "hw/hw.h" +#include "user.h" =20 /* * Flags to be used as unique delimiters for VFIO devices in the migration @@ -49,10 +50,18 @@ static int64_t bytes_transferred; static inline int vfio_mig_access(VFIODevice *vbasedev, void *val, int cou= nt, off_t off, bool iswrite) { + VFIORegion *region =3D &vbasedev->migration->region; int ret; =20 - ret =3D iswrite ? pwrite(vbasedev->fd, val, count, off) : - pread(vbasedev->fd, val, count, off); + if (vbasedev->proxy !=3D NULL) { + ret =3D iswrite ? + vfio_user_region_write(vbasedev, region->nr, off, count, val) : + vfio_user_region_read(vbasedev, region->nr, off, count, val); + } else { + off +=3D region->fd_offset; + ret =3D iswrite ? pwrite(vbasedev->fd, val, count, off) : + pread(vbasedev->fd, val, count, off); + } if (ret < count) { error_report("vfio_mig_%s %d byte %s: failed at offset 0x%" HWADDR_PRIx", err: %s", iswrite ? "write" : "read", c= ount, @@ -111,9 +120,7 @@ static int vfio_migration_set_state(VFIODevice *vbasede= v, uint32_t mask, uint32_t value) { VFIOMigration *migration =3D vbasedev->migration; - VFIORegion *region =3D &migration->region; - off_t dev_state_off =3D region->fd_offset + - VFIO_MIG_STRUCT_OFFSET(device_state); + off_t dev_state_off =3D VFIO_MIG_STRUCT_OFFSET(device_state); uint32_t device_state; int ret; =20 @@ -201,13 +208,13 @@ static int vfio_save_buffer(QEMUFile *f, VFIODevice *= vbasedev, uint64_t *size) int ret; =20 ret =3D vfio_mig_read(vbasedev, &data_offset, sizeof(data_offset), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_offs= et)); + VFIO_MIG_STRUCT_OFFSET(data_offset)); if (ret < 0) { return ret; } =20 ret =3D vfio_mig_read(vbasedev, &data_size, sizeof(data_size), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_si= ze)); + VFIO_MIG_STRUCT_OFFSET(data_size)); if (ret < 0) { return ret; } @@ -233,8 +240,7 @@ static int vfio_save_buffer(QEMUFile *f, VFIODevice *vb= asedev, uint64_t *size) } buf_allocated =3D true; =20 - ret =3D vfio_mig_read(vbasedev, buf, sec_size, - region->fd_offset + data_offset); + ret =3D vfio_mig_read(vbasedev, buf, sec_size, data_offset); if (ret < 0) { g_free(buf); return ret; @@ -269,7 +275,7 @@ static int vfio_load_buffer(QEMUFile *f, VFIODevice *vb= asedev, =20 do { ret =3D vfio_mig_read(vbasedev, &data_offset, sizeof(data_offset), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_offs= et)); + VFIO_MIG_STRUCT_OFFSET(data_offset)); if (ret < 0) { return ret; } @@ -309,8 +315,7 @@ static int vfio_load_buffer(QEMUFile *f, VFIODevice *vb= asedev, qemu_get_buffer(f, buf, sec_size); =20 if (buf_alloc) { - ret =3D vfio_mig_write(vbasedev, buf, sec_size, - region->fd_offset + data_offset); + ret =3D vfio_mig_write(vbasedev, buf, sec_size, data_offse= t); g_free(buf); =20 if (ret < 0) { @@ -322,7 +327,7 @@ static int vfio_load_buffer(QEMUFile *f, VFIODevice *vb= asedev, } =20 ret =3D vfio_mig_write(vbasedev, &report_size, sizeof(report_size), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_si= ze)); + VFIO_MIG_STRUCT_OFFSET(data_size)); if (ret < 0) { return ret; } @@ -334,12 +339,11 @@ static int vfio_load_buffer(QEMUFile *f, VFIODevice *= vbasedev, static int vfio_update_pending(VFIODevice *vbasedev) { VFIOMigration *migration =3D vbasedev->migration; - VFIORegion *region =3D &migration->region; uint64_t pending_bytes =3D 0; int ret; =20 ret =3D vfio_mig_read(vbasedev, &pending_bytes, sizeof(pending_bytes), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(pending_byt= es)); + VFIO_MIG_STRUCT_OFFSET(pending_bytes)); if (ret < 0) { migration->pending_bytes =3D 0; return ret; diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 4b933ed10f..976fb89786 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3658,6 +3658,13 @@ static void vfio_user_pci_realize(PCIDevice *pdev, E= rror **errp) } } =20 + if (!pdev->failover_pair_id) { + ret =3D vfio_migration_probe(&vdev->vbasedev, errp); + if (ret) { + error_report("%s: Migration disabled", vdev->vbasedev.name); + } + } + vfio_register_err_notifier(vdev); vfio_register_req_notifier(vdev); =20 @@ -3709,6 +3716,11 @@ static void vfio_user_pci_reset(DeviceState *dev) static Property vfio_user_pci_dev_properties[] =3D { DEFINE_PROP_STRING("socket", VFIOUserPCIDevice, sock_name), DEFINE_PROP_BOOL("secure-dma", VFIOUserPCIDevice, secure_dma, false), + DEFINE_PROP_BOOL("x-enable-migration", VFIOPCIDevice, + vbasedev.enable_migration, false), + DEFINE_PROP_ON_OFF_AUTO("x-pre-copy-dirty-page-tracking", VFIOPCIDevic= e, + vbasedev.pre_copy_dirty_page_tracking, + ON_OFF_AUTO_ON), DEFINE_PROP_END_OF_LIST(), }; =20 diff --git a/hw/vfio/user.c b/hw/vfio/user.c index 7de2125346..486f7c0fe7 100644 --- a/hw/vfio/user.c +++ b/hw/vfio/user.c @@ -1057,3 +1057,48 @@ void vfio_user_reset(VFIODevice *vbasedev) error_printf("reset reply error %d\n", msg.error_reply); } } + +int vfio_user_dirty_bitmap(VFIOProxy *proxy, + struct vfio_iommu_type1_dirty_bitmap *cmd, + struct vfio_iommu_type1_dirty_bitmap_get *dbitm= ap) +{ + g_autofree struct { + VFIOUserDirtyPages msg; + VFIOUserBitmapRange range; + } *msgp =3D NULL; + int msize, rsize; + + /* + * If just the command is sent, the returned bitmap isn't needed. + * The bitmap structs are different from the ioctl() versions, + * ioctl() returns the bitmap in a local VA + */ + if (dbitmap !=3D NULL) { + msize =3D sizeof(*msgp); + rsize =3D msize + dbitmap->bitmap.size; + msgp =3D g_malloc0(rsize); + msgp->range.iova =3D dbitmap->iova; + msgp->range.size =3D dbitmap->size; + msgp->range.bitmap.pgsize =3D dbitmap->bitmap.pgsize; + msgp->range.bitmap.size =3D dbitmap->bitmap.size; + } else { + msize =3D rsize =3D sizeof(VFIOUserDirtyPages); + msgp =3D g_malloc0(rsize); + } + + vfio_user_request_msg(&msgp->msg.hdr, VFIO_USER_DIRTY_PAGES, msize, 0); + msgp->msg.argsz =3D msize - sizeof(msgp->msg.hdr); + msgp->msg.flags =3D cmd->flags; + + vfio_user_send_recv(proxy, &msgp->msg.hdr, NULL, rsize, 0); + if (msgp->msg.hdr.flags & VFIO_USER_ERROR) { + return -msgp->msg.hdr.error_reply; + } + + if (dbitmap !=3D NULL) { + memcpy(dbitmap->bitmap.data, &msgp->range.bitmap.data, + dbitmap->bitmap.size); + } + + return 0; +} --=20 2.25.1