From nobody Thu Sep 19 21:14:18 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70E9BC77B61 for ; Thu, 13 Apr 2023 08:19:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230423AbjDMITT (ORCPT ); Thu, 13 Apr 2023 04:19:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57182 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230369AbjDMITP (ORCPT ); Thu, 13 Apr 2023 04:19:15 -0400 Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2071.outbound.protection.outlook.com [40.107.21.71]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A3CEE7280; Thu, 13 Apr 2023 01:19:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UHBCKc/ir14PXfh8Py2Ox87vrx5GamNqHLfYUOpktlkB5D4jBcilKM/6uOvz7xv3+MxXJ2ChRl19kyshXQ3X8fzp/dBl7875i730OZdQuYBQ4Sly+GKRC9ykHa9RpWkIEntm6JKYfV1/5vFNzNpzgJoa1QRo3Zg1YjArLOXpN2VFXPB2A33RFuGQcBeP4DT0S2Th6RPth/YT/tXWgIIDtNyxWe68T5+pWKdzcoxgAK6hc79NqaJr65fALLiBCZjZpfHko9ZM0bKvQNXed4bfbEjKD6S3JcW85Vv3p//O9E4Xjsm5YiT0woI4V/BWd4/SbXNUackOJXW1g3yUabTvWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nVhA+aLyeR7D47AsiAwjmzAFFy5taKpvwm6WQopnOnI=; b=FXd1Gpda+azk6DX6tp4dgu11CDIg9ReY8bIO13nQfdn4DGOpWmduRVPh7MW8o+ZoFzUNWZcchB1ilJcIWm12lZXtY9SWaJcVcFmEzPnftUzLbNG6SlAbTA5QQJlCQO1B1czbY1rjQJJM8GR6Lp9b9lYr6TRGVge1cety0z2aWA61iTflkfOc2oKBIy01LhbRWXkEcbZam49+XFDH3Ehc/Vt+1Xy4wD4aBwpLwAA6YREma43PqyyYd+uxYTtpbQ/DdxA4fEBPUt8NqwMee36k81ZJQ+DjzNgoaOL0XKSwzt7st1pl369sloHYtqdaf9egKBEnVvhoF49TVqBKPeUVtA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=solid-run.com; dmarc=pass action=none header.from=solid-run.com; dkim=pass header.d=solid-run.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=solidrn.onmicrosoft.com; s=selector1-solidrn-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nVhA+aLyeR7D47AsiAwjmzAFFy5taKpvwm6WQopnOnI=; b=p9KNXoGZD+4ZwGwjmetnrumALna2dqEGTUbzf/BW7sfkpWmdBdEtCsLazx7gSKTiJpU8D0mOnWVrmA1U2pWOhuYnDs+c2W20yqgqcVaTikV1XQAYMjpYk87ifVD5GkdCocx7hBM5BhKQOUiMpsKAnpcL2zmKR/yWDmxafCEOW6U= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=solid-run.com; Received: from AM0PR04MB4723.eurprd04.prod.outlook.com (2603:10a6:208:c0::20) by DBBPR04MB7721.eurprd04.prod.outlook.com (2603:10a6:10:1f6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.30; Thu, 13 Apr 2023 08:19:11 +0000 Received: from AM0PR04MB4723.eurprd04.prod.outlook.com ([fe80::54c9:6706:9dc6:d977]) by AM0PR04MB4723.eurprd04.prod.outlook.com ([fe80::54c9:6706:9dc6:d977%5]) with mapi id 15.20.6298.030; Thu, 13 Apr 2023 08:19:11 +0000 From: Alvaro Karsz To: mst@redhat.com, jasowang@redhat.com Cc: virtualization@lists.linux-foundation.org, cohuck@redhat.com, pasic@linux.ibm.com, farman@linux.ibm.com, linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, yan@daynix.com, viktor@daynix.com, Xuan Zhuo Subject: [PATCH 1/2] virtio: add VIRTIO_F_NOTIFICATION_DATA feature support Date: Thu, 13 Apr 2023 11:18:54 +0300 Message-Id: <20230413081855.36643-2-alvaro.karsz@solid-run.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230413081855.36643-1-alvaro.karsz@solid-run.com> References: <20230413081855.36643-1-alvaro.karsz@solid-run.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: ZR0P278CA0054.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:1d::23) To AM0PR04MB4723.eurprd04.prod.outlook.com (2603:10a6:208:c0::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM0PR04MB4723:EE_|DBBPR04MB7721:EE_ X-MS-Office365-Filtering-Correlation-Id: 2e6f3c8c-da86-4879-ab01-08db3bf7c2e6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wYORrxztA6yeedvxZwcH+D4Su0xET2tTYyxXIET+gVyhYy7nsZ3I5eEjar6RcCaYXubZW39wxcT4t9616XRB7+5hXzjkMEkdetDSSmVghrZhpCEhmKt5ZaYxCEH1D/nichjNewgJlz5e3uie068FnHn2kylTz79FePtDy3cRDkXT/b88mNM3dEh50URpU85uKotmrwQ0b5Gd9rQtDOUOvk0Zamywhqt+E2ttGsW+lmhuNUdS0h8m0ZOCBclae8i6bNoGmRc27dkMC/2OLZ+IK7j8LY1MhsKTQyg0Cvx077l29q1Y/2EQBY4gsZMNcywJ8FKPWh1Zi3WeovXSZy9OjvjzuuICWY6SsVs3G6kMTFuBg/Ab/VB/v9nWChQUVYgkfn6hUZ6yYi34a0stO/GENBEf7YY8/+cK9GToqx7u4hGAxpdx3fYvMCRBZd6pTtRljkEfG2Ied6FBP2xxyiuuRSD8dyz+XHF7NOMwaXDhIikupmOyqgo6nmo79LN2WN6U2+LBBuyRHuKj171zoKAHbRxUG0Vmk1hBu3/3yK+xO61Ov9SuAJdt8DTK0o2UBA8n6UM06ZS4G1AfysYQd0yAbbjBa0wsQzyJhi4Zo1/5sNWOFWESQM/n2LRgBazUK7n+ X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB4723.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(136003)(396003)(346002)(376002)(39840400004)(366004)(451199021)(6666004)(52116002)(6486002)(66476007)(66556008)(66946007)(4326008)(36756003)(2906002)(7416002)(44832011)(86362001)(41300700001)(5660300002)(8676002)(8936002)(316002)(38100700002)(38350700002)(478600001)(6512007)(6506007)(1076003)(26005)(2616005)(186003)(83380400001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?oc916p8XdbckAQ7HwB6eUzD6l7rBn63v7NW1kY8e/2oSELGOB1iRvVps93Op?= =?us-ascii?Q?pAxyOOW8CbQa4SHOM7k3vztdd/eXs7kiruIGBWXObtJG+g6adsiuDsOKtFwl?= =?us-ascii?Q?hMbk73leUJ0G7FmCNksqdl6ORUGdgFD4BkMKR57GziPJUV123ldpfoU5Z6Ky?= =?us-ascii?Q?nX4gwlIr4LjWLJRhd81hXsPkvtkEuEVlr2IDHtcQoHF8PdHNiLIzIBqxemiy?= =?us-ascii?Q?EsMGn4OOo7DJTZFgv2NYY6eqaaxgs/0tRc7bMg2noiRwHNTU8M5cTet64jig?= =?us-ascii?Q?1zZ7ibgOVUzynPQirddOWym0VYJME/FP3m9cvZAb45FJX5QirGy/93kBzwFP?= =?us-ascii?Q?SsHAKgUe0jJ9GOgI7wu4XZA4Kc21jrmJaxUuEYX4r4hGv1+5DpVGKb8bTLpO?= =?us-ascii?Q?FcHnUgf2ZpVKO8WY3ZxFkIT9bcxAO2wVlCtlhWGd5XLHt0bYVODDAbqBhwXT?= =?us-ascii?Q?99Xam2XaSf7OyBeIHzwKbqDUZxSf4mRYNdVkwPnSBRRFns5io4SxBbmlfbpT?= =?us-ascii?Q?j1hJA0Nl01GKNQimZxq2sYlI9GkRqmveCdACZRxkJL9Zg6bl/COy+UxRgWEo?= =?us-ascii?Q?1g5kIozJpAGgZAOdIsvBQ+VeXC4i0J8zBuf5hQQIuyN5eRogfEYcWea2VRb6?= =?us-ascii?Q?/Pal1kgKCBxrotIX9lXmnfafXnG3SzGCHH/LvAUPhha6mUh3ztDUVTh1FIz/?= =?us-ascii?Q?mZzEVhqDStcr5A26r022IBkjig+bye+QMPaW1wSk875JRzize5t06YleixoJ?= =?us-ascii?Q?ZiCX5BLdQ1aTjJvoQSk9Ru0Zpoii3t0in41ivpmW+E8GVQ/TvSWaIcDEzmqn?= =?us-ascii?Q?OSeKZXFegtL9NEoiSSANc6V2dqd3swCmgiztwXGHpM/UIECO/A2cIMNdXDHS?= =?us-ascii?Q?x3IWus3+rQnpe12szEEn54VXk6JMr2OIrlD6l8Ms43HJxPs3Mi0rX5OmB4pq?= =?us-ascii?Q?DmdtcuTj45+GHeHndTKWdIydawReb7cu9+cXTFdshex4LRhW8IOzdal2AiNW?= =?us-ascii?Q?rV5iSHfNlxEzMa/vXXysRgkwnIWdfpay3Mpzr0m7VDOO9WhjjvnYDu1LOCXe?= =?us-ascii?Q?wH2PUN6AEh/eY5zdYF51xceMQxy0TqWaWLkHqmBNzt2ZEjPBSM1E0ZaPB1FX?= =?us-ascii?Q?lNLSQYU54vDGCieUg4MN/RbUanS9e0UqhFBXIQ/9vCcbRkKZLwFrqYi51hxW?= =?us-ascii?Q?coJbUwMaKFFFozSeH58Tok7JDV5PPFjNTOcrGVRueCVlxh+bCEnxAZufRxMt?= =?us-ascii?Q?ypjGfcgx6xI5LHKpL3u/bUE2ZGl5GP/A5TeNdEXF3loJ0xVVSQH1y032pIwv?= =?us-ascii?Q?2OSEbvJo6QPnKqiZS3lww4YIj7aV9urN1dFY2EwALcBbwTD0L9tMMnzkNO/p?= =?us-ascii?Q?0IvB6wfwqbPyj6x09fj3+sXGGEZcvGCeI8Hh3LNoO7wVURlgaMNT1EInC51g?= =?us-ascii?Q?1aQTdbHI/KUQJmzoXf4yjp4Tol9w20DhR+BoRaklJRvFLlKmy/yspMbHT+Fa?= =?us-ascii?Q?34bQBgqLCRpv5U/9TRsPlxlG8w6ZYLMjU1imBQj4tyYMv62MUes5iymCnqWx?= =?us-ascii?Q?QZY3waIZScKPYt5TeexjhUd4KAyhMk6EHEF5Vkuddhj3YCrKE9q4UGuXxt/b?= =?us-ascii?Q?lg=3D=3D?= X-OriginatorOrg: solid-run.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2e6f3c8c-da86-4879-ab01-08db3bf7c2e6 X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB4723.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2023 08:19:11.8158 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a4a8aaf3-fd27-4e27-add2-604707ce5b82 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VcDK/M3U4pYcPj1pibDBdXh4uIzeu43R6I3Lyr4o+qMbfesvmpM3VYq7eQAMTC9DnKfu6qSPYXF75e18caEVVTyS+lEcV/B36hzuJB9w7hM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR04MB7721 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Viktor Prutyanov According to VirtIO spec v1.2, VIRTIO_F_NOTIFICATION_DATA feature indicates that the driver passes extra data along with the queue notifications. In a split queue case, the extra data is 16-bit available index. In a packed queue case, the extra data is 1-bit wrap counter and 15-bit available index. Add support for this feature for MMIO, channel I/O and modern PCI transports. Signed-off-by: Viktor Prutyanov Acked-by: Jason Wang Reviewed-by: Xuan Zhuo --- drivers/s390/virtio/virtio_ccw.c | 22 +++++++++++++++++++--- drivers/virtio/virtio_mmio.c | 18 +++++++++++++++++- drivers/virtio/virtio_pci_modern.c | 17 ++++++++++++++++- drivers/virtio/virtio_ring.c | 19 +++++++++++++++++++ include/linux/virtio_ring.h | 2 ++ include/uapi/linux/virtio_config.h | 6 ++++++ 6 files changed, 79 insertions(+), 5 deletions(-) diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_= ccw.c index 954fc31b4bc..02922768b12 100644 --- a/drivers/s390/virtio/virtio_ccw.c +++ b/drivers/s390/virtio/virtio_ccw.c @@ -391,7 +391,7 @@ static void virtio_ccw_drop_indicator(struct virtio_ccw= _device *vcdev, ccw_device_dma_free(vcdev->cdev, thinint_area, sizeof(*thinint_area)); } =20 -static bool virtio_ccw_kvm_notify(struct virtqueue *vq) +static inline bool virtio_ccw_do_kvm_notify(struct virtqueue *vq, u32 data) { struct virtio_ccw_vq_info *info =3D vq->priv; struct virtio_ccw_device *vcdev; @@ -402,12 +402,22 @@ static bool virtio_ccw_kvm_notify(struct virtqueue *v= q) BUILD_BUG_ON(sizeof(struct subchannel_id) !=3D sizeof(unsigned int)); info->cookie =3D kvm_hypercall3(KVM_S390_VIRTIO_CCW_NOTIFY, *((unsigned int *)&schid), - vq->index, info->cookie); + data, info->cookie); if (info->cookie < 0) return false; return true; } =20 +static bool virtio_ccw_kvm_notify(struct virtqueue *vq) +{ + return virtio_ccw_do_kvm_notify(vq, vq->index); +} + +static bool virtio_ccw_kvm_notify_with_data(struct virtqueue *vq) +{ + return virtio_ccw_do_kvm_notify(vq, vring_notification_data(vq)); +} + static int virtio_ccw_read_vq_conf(struct virtio_ccw_device *vcdev, struct ccw1 *ccw, int index) { @@ -495,6 +505,7 @@ static struct virtqueue *virtio_ccw_setup_vq(struct vir= tio_device *vdev, struct ccw1 *ccw) { struct virtio_ccw_device *vcdev =3D to_vc_device(vdev); + bool (*notify)(struct virtqueue *vq); int err; struct virtqueue *vq =3D NULL; struct virtio_ccw_vq_info *info; @@ -502,6 +513,11 @@ static struct virtqueue *virtio_ccw_setup_vq(struct vi= rtio_device *vdev, unsigned long flags; bool may_reduce; =20 + if (__virtio_test_bit(vdev, VIRTIO_F_NOTIFICATION_DATA)) + notify =3D virtio_ccw_kvm_notify_with_data; + else + notify =3D virtio_ccw_kvm_notify; + /* Allocate queue. */ info =3D kzalloc(sizeof(struct virtio_ccw_vq_info), GFP_KERNEL); if (!info) { @@ -524,7 +540,7 @@ static struct virtqueue *virtio_ccw_setup_vq(struct vir= tio_device *vdev, may_reduce =3D vcdev->revision > 0; vq =3D vring_create_virtqueue(i, info->num, KVM_VIRTIO_CCW_RING_ALIGN, vdev, true, may_reduce, ctx, - virtio_ccw_kvm_notify, callback, name); + notify, callback, name); =20 if (!vq) { /* For now, we fail if we can't get the requested size. */ diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c index 3ff746e3f24..dd4424c1423 100644 --- a/drivers/virtio/virtio_mmio.c +++ b/drivers/virtio/virtio_mmio.c @@ -285,6 +285,16 @@ static bool vm_notify(struct virtqueue *vq) return true; } =20 +static bool vm_notify_with_data(struct virtqueue *vq) +{ + struct virtio_mmio_device *vm_dev =3D to_virtio_mmio_device(vq->vdev); + u32 data =3D vring_notification_data(vq); + + writel(data, vm_dev->base + VIRTIO_MMIO_QUEUE_NOTIFY); + + return true; +} + /* Notify all virtqueues on an interrupt. */ static irqreturn_t vm_interrupt(int irq, void *opaque) { @@ -363,12 +373,18 @@ static struct virtqueue *vm_setup_vq(struct virtio_de= vice *vdev, unsigned int in const char *name, bool ctx) { struct virtio_mmio_device *vm_dev =3D to_virtio_mmio_device(vdev); + bool (*notify)(struct virtqueue *vq); struct virtio_mmio_vq_info *info; struct virtqueue *vq; unsigned long flags; unsigned int num; int err; =20 + if (__virtio_test_bit(vdev, VIRTIO_F_NOTIFICATION_DATA)) + notify =3D vm_notify_with_data; + else + notify =3D vm_notify; + if (!name) return NULL; =20 @@ -397,7 +413,7 @@ static struct virtqueue *vm_setup_vq(struct virtio_devi= ce *vdev, unsigned int in =20 /* Create the vring */ vq =3D vring_create_virtqueue(index, num, VIRTIO_MMIO_VRING_ALIGN, vdev, - true, true, ctx, vm_notify, callback, name); + true, true, ctx, notify, callback, name); if (!vq) { err =3D -ENOMEM; goto error_new_virtqueue; diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci= _modern.c index 9e496e288cf..05deba5153b 100644 --- a/drivers/virtio/virtio_pci_modern.c +++ b/drivers/virtio/virtio_pci_modern.c @@ -288,6 +288,15 @@ static u16 vp_config_vector(struct virtio_pci_device *= vp_dev, u16 vector) return vp_modern_config_vector(&vp_dev->mdev, vector); } =20 +static bool vp_notify_with_data(struct virtqueue *vq) +{ + u32 data =3D vring_notification_data(vq); + + iowrite32(data, (void __iomem *)vq->priv); + + return true; +} + static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, struct virtio_pci_vq_info *info, unsigned int index, @@ -298,10 +307,16 @@ static struct virtqueue *setup_vq(struct virtio_pci_d= evice *vp_dev, { =20 struct virtio_pci_modern_device *mdev =3D &vp_dev->mdev; + bool (*notify)(struct virtqueue *vq); struct virtqueue *vq; u16 num; int err; =20 + if (__virtio_test_bit(&vp_dev->vdev, VIRTIO_F_NOTIFICATION_DATA)) + notify =3D vp_notify_with_data; + else + notify =3D vp_notify; + if (index >=3D vp_modern_get_num_queues(mdev)) return ERR_PTR(-EINVAL); =20 @@ -321,7 +336,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_dev= ice *vp_dev, vq =3D vring_create_virtqueue(index, num, SMP_CACHE_BYTES, &vp_dev->vdev, true, true, ctx, - vp_notify, callback, name); + notify, callback, name); if (!vq) return ERR_PTR(-ENOMEM); =20 diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 41144b5246a..7749bd5f99f 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2752,6 +2752,23 @@ void vring_del_virtqueue(struct virtqueue *_vq) } EXPORT_SYMBOL_GPL(vring_del_virtqueue); =20 +u32 vring_notification_data(struct virtqueue *_vq) +{ + struct vring_virtqueue *vq =3D to_vvq(_vq); + u16 next; + + if (vq->packed_ring) + next =3D (vq->packed.next_avail_idx & + ~(-(1 << VRING_PACKED_EVENT_F_WRAP_CTR))) | + vq->packed.avail_wrap_counter << + VRING_PACKED_EVENT_F_WRAP_CTR; + else + next =3D vq->split.avail_idx_shadow; + + return next << 16 | _vq->index; +} +EXPORT_SYMBOL_GPL(vring_notification_data); + /* Manipulates transport-specific feature bits. */ void vring_transport_features(struct virtio_device *vdev) { @@ -2771,6 +2788,8 @@ void vring_transport_features(struct virtio_device *v= dev) break; case VIRTIO_F_ORDER_PLATFORM: break; + case VIRTIO_F_NOTIFICATION_DATA: + break; default: /* We don't understand this bit. */ __virtio_clear_bit(vdev, i); diff --git a/include/linux/virtio_ring.h b/include/linux/virtio_ring.h index 8b95b69ef69..2550c9170f4 100644 --- a/include/linux/virtio_ring.h +++ b/include/linux/virtio_ring.h @@ -117,4 +117,6 @@ void vring_del_virtqueue(struct virtqueue *vq); void vring_transport_features(struct virtio_device *vdev); =20 irqreturn_t vring_interrupt(int irq, void *_vq); + +u32 vring_notification_data(struct virtqueue *_vq); #endif /* _LINUX_VIRTIO_RING_H */ diff --git a/include/uapi/linux/virtio_config.h b/include/uapi/linux/virtio= _config.h index 3c05162bc98..2c712c65416 100644 --- a/include/uapi/linux/virtio_config.h +++ b/include/uapi/linux/virtio_config.h @@ -99,6 +99,12 @@ */ #define VIRTIO_F_SR_IOV 37 =20 +/* + * This feature indicates that the driver passes extra data (besides + * identifying the virtqueue) in its device notifications. + */ +#define VIRTIO_F_NOTIFICATION_DATA 38 + /* * This feature indicates that the driver can reset a queue individually. */ --=20 2.34.1 From nobody Thu Sep 19 21:14:18 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED211C77B61 for ; Thu, 13 Apr 2023 08:19:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230458AbjDMIT2 (ORCPT ); Thu, 13 Apr 2023 04:19:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57208 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230387AbjDMITQ (ORCPT ); Thu, 13 Apr 2023 04:19:16 -0400 Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2071.outbound.protection.outlook.com [40.107.21.71]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3980B98; Thu, 13 Apr 2023 01:19:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gtTNaLmVtWrPX5cnyCwRSMvNC4qcahUiOlS4l8DFBLLH4effIV2LXBRJypUq1BZRF3zggQFfcGpNRrgKih1BMAaM57/ryhvZTCwEyKZqlJHf3wik4BHrVz+Fpd2G1sLpMQpBiSDJD20Cd+K4xCBOgU9DFD4ICBFSseM0T6WYvYiPsaR7fCGiF3fEB4lF2eX3AtbmB4PhVGzdo+e0WSYoxlmPGRqHLLcKZXGIUMq7H9sxKp2J1EwqLTZnjsMxM0bbZ2jVvXbwHvtDhjn4MjhECKTWLZYJZ/6d1TM48DpSVZDffsgAbC09/DofP5D3SPLfP4kQIlQBPf25aDG+CTFHSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=N174125ILgz38cpAln85Q8rftQqSNm/zgKL0yeDc/UI=; b=E/LwciEnyTXg7iJ8rs4K0eXhvqX0bqkacBqucDAxCZRhu3V5XDpTor/MbcDO9wYXjeuNzSH8/Cu64+lepQTu9sbGYvEaQixwfcLnHKEDiLpO8rdLUW4R2YNnZkpOpDevMNTt8IqemUcpzGqTK+tjxaHkePFt1Qoqfcvdj1y8Goul+GWyNNYqdXvYUZlcV1pzML+uW+xE2QaD7czNFwIkqpioxytLDwAvZ3//wBhk6l4H4MOJc7Inc1ttNNczHaS2W5Dt1zbG3IoI7bNJY6drYIJ2hYRF49p7tWf3BRvIhWiADPY26PMPsUp6D+qY2EqxhXCvWcZn5NvifCwLqRluWg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=solid-run.com; dmarc=pass action=none header.from=solid-run.com; dkim=pass header.d=solid-run.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=solidrn.onmicrosoft.com; s=selector1-solidrn-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N174125ILgz38cpAln85Q8rftQqSNm/zgKL0yeDc/UI=; b=TXKN3QaAJPANstwDLFYUuHxW4/9J7Yvxhevla6mKyZq0ZzS9dbsxPc5Ac96xwIMLUCmo8QC+xf9NlQDmcjVZW3bh4zd/oFld8VJSAMAhFC5kgkDYaGbvEK1KuqBXzo5DxucjmCLBFxXdt2FIgLRvFmNCvO2w0ol+HKEwf/qUm+U= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=solid-run.com; Received: from AM0PR04MB4723.eurprd04.prod.outlook.com (2603:10a6:208:c0::20) by DBBPR04MB7721.eurprd04.prod.outlook.com (2603:10a6:10:1f6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.30; Thu, 13 Apr 2023 08:19:13 +0000 Received: from AM0PR04MB4723.eurprd04.prod.outlook.com ([fe80::54c9:6706:9dc6:d977]) by AM0PR04MB4723.eurprd04.prod.outlook.com ([fe80::54c9:6706:9dc6:d977%5]) with mapi id 15.20.6298.030; Thu, 13 Apr 2023 08:19:13 +0000 From: Alvaro Karsz To: mst@redhat.com, jasowang@redhat.com Cc: virtualization@lists.linux-foundation.org, cohuck@redhat.com, pasic@linux.ibm.com, farman@linux.ibm.com, linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, yan@daynix.com, viktor@daynix.com, Alvaro Karsz Subject: [PATCH 2/2] virtio-vdpa: add VIRTIO_F_NOTIFICATION_DATA feature support Date: Thu, 13 Apr 2023 11:18:55 +0300 Message-Id: <20230413081855.36643-3-alvaro.karsz@solid-run.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230413081855.36643-1-alvaro.karsz@solid-run.com> References: <20230413081855.36643-1-alvaro.karsz@solid-run.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: ZR0P278CA0054.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:1d::23) To AM0PR04MB4723.eurprd04.prod.outlook.com (2603:10a6:208:c0::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM0PR04MB4723:EE_|DBBPR04MB7721:EE_ X-MS-Office365-Filtering-Correlation-Id: 4344d1b9-3b2e-4f73-fbce-08db3bf7c3ff X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pthIlGDCP8GebUhMWbDibezoEbzWh50JNkVTXjkYZ8uvZo8wr+mIk+SvOwLwzw8efeEkXs5qSvErS7ClYRxV86U8vbg7DIwFMotnObzFz2ojK5nsm0DUgSsIA/PO/OYruY7Gb26JGgJqH5othIpajXPXfSE5pQjrxSrO3VE7w2me3lNOwGlt79MTyEPEZfEdM4U4vNLuYO7R6xHMUw95inrVTNU6NwuFQRHpGlIdzND/8Gv82Dvj94oph4Wmd/pwGxHYc1w7aiX0HAO/4TObH3oRH7hgWTMHoiE6HTCoctieN/TvGIKIbQqWA39qNRYZvtqnjnhWoA6bVQCoxFQZjI4t2MXF9RjZcteG+PO86ZIbb0DP6HWk9soaaM68pJfBWd033Ud9oVhpEt0ZUS03b6qGrCs9+KG3c/CVaEjuOFvVxqPYbhayFA/Np9qXXh9tQsaoTXGVOrRpbeiPdgXTitepCI7nzmMH2zvCSLeZZ+34ihNFdC1sIwBNyr59qliZo63EfOUxCI9ousAjs1Y2Nm6tYiK4vYjOH7H/QsLdzj8Gz7YteKfpG+8zL4T1+xii6cp1zqiT9d43zP+h70ulAW28imacbd+UuzrgY2WhRL0a6iPpT0QVa8cUz6g3SKCJ X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB4723.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(136003)(396003)(346002)(376002)(39840400004)(366004)(451199021)(6666004)(52116002)(6486002)(66476007)(66556008)(66946007)(4326008)(36756003)(2906002)(7416002)(44832011)(86362001)(41300700001)(5660300002)(8676002)(8936002)(316002)(38100700002)(38350700002)(478600001)(6512007)(6506007)(1076003)(26005)(2616005)(186003)(107886003)(66899021)(83380400001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?8yi89h6SVWgUkl1g1O57kE7SYDzoBXmtf4YKPiAs03kJh6Tzd10uAPHLYuVC?= =?us-ascii?Q?93dFr5clPcyDjS89BU9gXSzCelCX19MZ+2G1kXl+s53vEj0aBcEMh5OncoiU?= =?us-ascii?Q?dQcQGo+v6Dv3PAtkCjJdwk8W4s54cmHWxYNHp/XtgiJ/Ky+xUiVTx5ecVSe2?= =?us-ascii?Q?s/D7XkNpw6J30YoKO5XG+QYXiQVPfo2aTrA/ge6jYzEBZjXywyciwJQZ7lBm?= =?us-ascii?Q?CGN53sFQgvmuAED6wVMgdpdAHM63evLruVyaWgVi3nlefExkrRceDfmHCnU+?= =?us-ascii?Q?pdTy1QJddDAkYHYz2ULqESbWKcOnPc9TqlNcKPdqa4+q+e7X+TO9Zm37TSZV?= =?us-ascii?Q?Zg4xi7AStIPm/M+6JRTBBl7f4R8SZqTNp6uJhXa1CHDqoMkZJQhKHSItXQg9?= =?us-ascii?Q?276+GDHPbfvgyaYYc29bIgl5EhFeFFmMjJxvHTvtOy4p4ViuSrR0G0V4e7Wa?= =?us-ascii?Q?7Nr1ntnDsDn1JJzAk7FkcUSBMayL1XAyEkccEzlo8tPSzrsjkzT0U0PTXQ4P?= =?us-ascii?Q?EdUGEzKUMvQr4Y1bvPVZM9SM9rjgyWO5nLtz+Jc8OG9w/ihRbVlYLg44n9qB?= =?us-ascii?Q?Aqeo+otmmXPQLiqzBNBGXmo1ddYC0cWlsVMxIEFvwHOISCJgqSe1teLjHDDL?= =?us-ascii?Q?PToY7P6FH6IW4GQSMP6QrZzg8GblvR6X64fGszAO2/IrvY+6ckNycOEsFZEZ?= =?us-ascii?Q?URvFoEcL9+iMDOGIFtcvCDfoB/LXistOQOmvnocpTsijCJ8MLCPIgm0jQ3f3?= =?us-ascii?Q?NlHsEqKYjyrlVT5i5rA5ef6xM5AWFI71qtAF59ScABQ74OU0VYKpz9/SPPr1?= =?us-ascii?Q?735JKod+Wu7Ek2JXZHWfeBa44+YDMJ8Dlad4PYvKascSwK9N/QMwtmsR5Zhi?= =?us-ascii?Q?8Z4YllMSBEvEtrRs9B2rohBZ4j0iuKjR0VJxD40rWs06DEDjEvktgECBAajr?= =?us-ascii?Q?dlkX3A1zV5Et8/XRUO3yc5q9+Esz3xiyEQNZF7922jFBImtgn8PH5KP/Glt3?= =?us-ascii?Q?IwdMJXzNlWWu0a8FPQMG9Q9slTiniwO9xPbjYujKqIH5Vlpe6ZAiIOa76+xo?= =?us-ascii?Q?jTzohB+SraRU7lNElQ/UWlrTkR24np8SGMZkpa1A4NAGO66xclRTWqJ95wtE?= =?us-ascii?Q?hX0Cq58+P9H0ctiQ8ns81zDhA8c5DQ+accrdpPFS2xbz0E9Mb0O7w5lqH8US?= =?us-ascii?Q?ZnZRF4265ELSl6k7ijI5ZceMwKbkVYqkPr577imijfmobNENOR9cGsGqjhSb?= =?us-ascii?Q?8UNrS+bP5ZVUTP70KvhINr0Gc0yZwO4htbKLZxeTqIuMEKkUH5b0CH9xZDEi?= =?us-ascii?Q?6UijCQaURNaeNun91d81zo8sLXaryMPKoMuh5fzrceP2/AnVSrtFrp/ffrrm?= =?us-ascii?Q?u5xWpSYisiop0urvsAeUZ2KM/vpvZlUyFaSlWnnx1LPXBI0GxNxW8WyrJ/rd?= =?us-ascii?Q?ph/rsSqaes9eZz7lgk+yb3k4eTZRI7SilGE9V7iKKSY1JznzB3bISj/oeehr?= =?us-ascii?Q?U5kxmQabqUhGOnScrF0jL942Bc5pj2Rj+YrZADNZ91P9+IdxI+8uNnjRW/VQ?= =?us-ascii?Q?rdQzMZbijF8yGRVJbKJCgRGEeXTQfAj9YK+clgst8eus2Ec46pkUuq58Y1EY?= =?us-ascii?Q?9Q=3D=3D?= X-OriginatorOrg: solid-run.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4344d1b9-3b2e-4f73-fbce-08db3bf7c3ff X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB4723.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2023 08:19:13.6481 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a4a8aaf3-fd27-4e27-add2-604707ce5b82 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gRNZOUTPJ4Sos5ZJAnJahNVxxsTITNfzJWlJVkYrPChbBlT7XPt/ED3HWD7coAWhfbSR/V3gcya8YWyuNve10CWByPIF2PuIsM8uv0+8Gzo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR04MB7721 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add VIRTIO_F_NOTIFICATION_DATA support for vDPA transport. If this feature is negotiated, the driver passes extra data when kicking a virtqueue. A device that offers this feature needs to implement the kick_vq_with_data callback. kick_vq_with_data receives the vDPA device and data. data includes: 16 bits vqn and 16 bits next available index for split virtqueues. 16 bits vqs, 15 least significant bits of next available index and 1 bit next_wrap for packed virtqueues. This patch follows a patch [1] by Viktor Prutyanov which adds support for the MMIO, channel I/O and modern PCI transports. Signed-off-by: Alvaro Karsz Acked-by: Jason Wang --- drivers/virtio/virtio_vdpa.c | 23 +++++++++++++++++++++-- include/linux/vdpa.h | 9 +++++++++ 2 files changed, 30 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c index d7f5af62dda..737c1f36d32 100644 --- a/drivers/virtio/virtio_vdpa.c +++ b/drivers/virtio/virtio_vdpa.c @@ -112,6 +112,17 @@ static bool virtio_vdpa_notify(struct virtqueue *vq) return true; } =20 +static bool virtio_vdpa_notify_with_data(struct virtqueue *vq) +{ + struct vdpa_device *vdpa =3D vd_get_vdpa(vq->vdev); + const struct vdpa_config_ops *ops =3D vdpa->config; + u32 data =3D vring_notification_data(vq); + + ops->kick_vq_with_data(vdpa, data); + + return true; +} + static irqreturn_t virtio_vdpa_config_cb(void *private) { struct virtio_vdpa_device *vd_dev =3D private; @@ -138,6 +149,7 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsign= ed int index, struct device *dma_dev; const struct vdpa_config_ops *ops =3D vdpa->config; struct virtio_vdpa_vq_info *info; + bool (*notify)(struct virtqueue *vq) =3D virtio_vdpa_notify; struct vdpa_callback cb; struct virtqueue *vq; u64 desc_addr, driver_addr, device_addr; @@ -154,6 +166,14 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsig= ned int index, if (index >=3D vdpa->nvqs) return ERR_PTR(-ENOENT); =20 + /* We cannot accept VIRTIO_F_NOTIFICATION_DATA without kick_vq_with_data = */ + if (__virtio_test_bit(vdev, VIRTIO_F_NOTIFICATION_DATA)) { + if (ops->kick_vq_with_data) + notify =3D virtio_vdpa_notify_with_data; + else + __virtio_clear_bit(vdev, VIRTIO_F_NOTIFICATION_DATA); + } + /* Queue shouldn't already be set up. */ if (ops->get_vq_ready(vdpa, index)) return ERR_PTR(-ENOENT); @@ -183,8 +203,7 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsign= ed int index, dma_dev =3D vdpa_get_dma_dev(vdpa); vq =3D vring_create_virtqueue_dma(index, max_num, align, vdev, true, may_reduce_num, ctx, - virtio_vdpa_notify, callback, - name, dma_dev); + notify, callback, name, dma_dev); if (!vq) { err =3D -ENOMEM; goto error_new_virtqueue; diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 43f59ef10cc..04cdaad77dd 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -143,6 +143,14 @@ struct vdpa_map_file { * @kick_vq: Kick the virtqueue * @vdev: vdpa device * @idx: virtqueue index + * @kick_vq_with_data: Kick the virtqueue and supply extra data + * (only if VIRTIO_F_NOTIFICATION_DATA is negotiated) + * @vdev: vdpa device + * @data for split virtqueue: + * 16 bits vqn and 16 bits next available index. + * @data for packed virtqueue: + * 16 bits vqn, 15 least significant bits of + * next available index and 1 bit next_wrap. * @set_vq_cb: Set the interrupt callback function for * a virtqueue * @vdev: vdpa device @@ -300,6 +308,7 @@ struct vdpa_config_ops { u64 device_area); void (*set_vq_num)(struct vdpa_device *vdev, u16 idx, u32 num); void (*kick_vq)(struct vdpa_device *vdev, u16 idx); + void (*kick_vq_with_data)(struct vdpa_device *vdev, u32 data); void (*set_vq_cb)(struct vdpa_device *vdev, u16 idx, struct vdpa_callback *cb); void (*set_vq_ready)(struct vdpa_device *vdev, u16 idx, bool ready); --=20 2.34.1