From nobody Tue Sep 9 16:08:53 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECDEDC64EC7 for ; Tue, 28 Feb 2023 09:42:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230260AbjB1JmO (ORCPT ); Tue, 28 Feb 2023 04:42:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59032 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230000AbjB1JmN (ORCPT ); Tue, 28 Feb 2023 04:42:13 -0500 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F7D912855 for ; Tue, 28 Feb 2023 01:42:12 -0800 (PST) Received: by mail-pf1-x42d.google.com with SMTP id fd25so5306743pfb.1 for ; Tue, 28 Feb 2023 01:42:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VZXmNRcPsICL6kN8QABaXbPWSPAMlFY7UslgXOPbl2g=; b=WZm1/ppM9bXo+zrQbhLrwZvlq2t4bwNPkt4KUj6yPa5gKCwz8hlkwCNxGY3/hdZBLs KyXwByvuLOAC+7T0rNGW/cSpoyGkNBHdKZFtS8QZA4wobdHYVgRwMYD5IJZRpIFlm5Go pdPu+E58xD2ArAgqnwyJChUBVUhs2RxrXSuEsbA/4pn8AKWmqMGjJE+5L/LNtGdFF9me 6f3yh0poY2vWE4vGs4uGGs1Fc8fOg8rENBJuSyy0r2hwicETvbZS/zfbLlsaBWwjngJR jxb7VJU7pMmi1cB7Pt+t96Ut3llgUL1lCK7XZEahCoZV8FOIBXLOSw/bZOV/5i+HiG5l eP6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VZXmNRcPsICL6kN8QABaXbPWSPAMlFY7UslgXOPbl2g=; b=ZZTiLkRK57mZYyZcZK1XOlwFNSFnRT+DEt1R5yBPKBgRRaKc0L7VzJuTz2YYpxktKX WUfDagQ+A1rWrRpRRD5TI+r8lozk5QyrZz1x/xk25lEKHICFtG1UVY0Yzn3MnWfryke9 mi1AQuV6QqJIuq1WPTOzkyrBCtkNu0wH1VLjHMkLJoaa7EPsFz/uhejyuytf346oNm6H ArxkuiuLaYj2/1BR2mswk3Qjg4UkOcmVfm69rSWv6yKrVTRySt3w39DWD9xG4IDh4XrU btUwIN8A8Q4UmW6oSXfkxBcX0hEIolqYHGVwsBPrwCso0sTTl3fuRAMAS12NWYOWiKfG /llg== X-Gm-Message-State: AO0yUKUlAgj2J+mpDePHjhmb1tWoGiNbegtKM/tRCZoRAbvgNjPOEp6y ec97hTtCf/z0gV9QHWsOToG7K4dlNYplBUs= X-Google-Smtp-Source: AK7set+3yAVpuE0+W31PocteZQbp7Jfg/gw72pcGhpvkIIBjNzUWy2DSGF/+10Ov4ubcwXv5mpZFrg== X-Received: by 2002:a62:546:0:b0:5a8:ecb1:bfe with SMTP id 67-20020a620546000000b005a8ecb10bfemr1992722pff.2.1677577331997; Tue, 28 Feb 2023 01:42:11 -0800 (PST) Received: from localhost ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id r24-20020a62e418000000b00571f66721aesm5616892pfh.42.2023.02.28.01.42.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 01:42:11 -0800 (PST) From: Xie Yongji To: mst@redhat.com, jasowang@redhat.com, tglx@linutronix.de, hch@lst.de Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 01/11] lib/group_cpus: Export group_cpus_evenly() Date: Tue, 28 Feb 2023 17:41:00 +0800 Message-Id: <20230228094110.37-2-xieyongji@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230228094110.37-1-xieyongji@bytedance.com> References: <20230228094110.37-1-xieyongji@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Export group_cpus_evenly() so that some modules can make use of it to group CPUs evenly according to NUMA and CPU locality. Signed-off-by: Xie Yongji Acked-by: Jason Wang --- lib/group_cpus.c | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/group_cpus.c b/lib/group_cpus.c index 9c837a35fef7..aa3f6815bb12 100644 --- a/lib/group_cpus.c +++ b/lib/group_cpus.c @@ -426,3 +426,4 @@ struct cpumask *group_cpus_evenly(unsigned int numgrps) return masks; } #endif /* CONFIG_SMP */ +EXPORT_SYMBOL_GPL(group_cpus_evenly); --=20 2.20.1 From nobody Tue Sep 9 16:08:53 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D8D7C64EC7 for ; Tue, 28 Feb 2023 09:42:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231296AbjB1Jmu (ORCPT ); Tue, 28 Feb 2023 04:42:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59150 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231142AbjB1Jmn (ORCPT ); Tue, 28 Feb 2023 04:42:43 -0500 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10DCC2D140 for ; Tue, 28 Feb 2023 01:42:16 -0800 (PST) Received: by mail-pl1-x62e.google.com with SMTP id bh1so9740923plb.11 for ; Tue, 28 Feb 2023 01:42:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=43XkzLpWZVinLMc6qbysNTMc1WeQ5tARy2ulZHhS0yc=; b=OnIaS9R0UH4qWdL4l1OHkEdbUQV4LfcylF1rlRaJBO8Ks+LLgfX5lMXhpXOcgSxvTK 5fhOnk4BFkUvtamOmAaK8iO/aUReCim2zcyr5CIqFvfxKZhKSchItfK+QzSt3Mqul9U7 2z6bbK/bOpXqdidNHsDAV2ZxNHXR8aXMhgBfuXDSOcyUlMZV4uuvVzZGOxaoKodcsxgj 6sT+Min8zS6xY0vJXUYv5yvBAtEO45plqfro7nvIt6sHFLDCe5PuLangDy0p8EHpq45n 057ojK75FIon17rtVgMPjB4gcMzinWNzTVqG71VkvhLu9QjwBvdVRwYKgAlaYyhvPraF m/Fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=43XkzLpWZVinLMc6qbysNTMc1WeQ5tARy2ulZHhS0yc=; b=BGSeR8KXqr3dzJn2a1WyoCz6FlK2LY8uDYRv9cMik4dceSEkNo3+AA/tnGPgS35tU0 JHiWj7QT+pH0MtcnG2Zlcn27oLza45/WT8b9dxQTeGRhEOdD6dMo2HFHfdJEKuD1egZZ 0ZLOi1BGNfp/nsm1i+97K+WouiaTKDf3HZ5U9AnnnkJJwZAQj9vTAsQ0EwQzcc8paYze JvfH6xxZnaran/IXKxXA6kAcFCCkFvBZEZc9qCD6N16ao/5I77HDbyfj3h3nbwlTNr93 mUkAjmf1FMCEoX01qKHv/mzF81sSpdO6wzcZAZsHD0LkWWLL//Ubj1HDw7FuGLZIvEyX R2zA== X-Gm-Message-State: AO0yUKXvvXDiERBprCeekg/Vm733kMoGUTEVhsX/+h13nMcnkRq4i6ai xFwhJcaCy5hbR8Bc6k2mh/KV X-Google-Smtp-Source: AK7set/yNW1VZ/u49W8Z+8V2zMpN8qjFOJ8xhFYg4XwxkuqbHEW18Pzu/sqtedB4kVIqsJhptc/Dsg== X-Received: by 2002:a05:6a20:7aa7:b0:cc:120c:b259 with SMTP id u39-20020a056a207aa700b000cc120cb259mr2046843pzh.39.1677577335520; Tue, 28 Feb 2023 01:42:15 -0800 (PST) Received: from localhost ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id y21-20020a63b515000000b00502e48db9aesm5346378pge.53.2023.02.28.01.42.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 01:42:15 -0800 (PST) From: Xie Yongji To: mst@redhat.com, jasowang@redhat.com, tglx@linutronix.de, hch@lst.de Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 02/11] vdpa: Add set/get_vq_affinity callbacks in vdpa_config_ops Date: Tue, 28 Feb 2023 17:41:01 +0800 Message-Id: <20230228094110.37-3-xieyongji@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230228094110.37-1-xieyongji@bytedance.com> References: <20230228094110.37-1-xieyongji@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This introduces set/get_vq_affinity callbacks in vdpa_config_ops to support interrupt affinity management for vdpa device drivers. Signed-off-by: Xie Yongji Acked-by: Jason Wang --- drivers/virtio/virtio_vdpa.c | 28 ++++++++++++++++++++++++++++ include/linux/vdpa.h | 13 +++++++++++++ 2 files changed, 41 insertions(+) diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c index d7f5af62ddaa..f72696b4c1c2 100644 --- a/drivers/virtio/virtio_vdpa.c +++ b/drivers/virtio/virtio_vdpa.c @@ -337,6 +337,32 @@ static const char *virtio_vdpa_bus_name(struct virtio_= device *vdev) return dev_name(&vdpa->dev); } =20 +static int virtio_vdpa_set_vq_affinity(struct virtqueue *vq, + const struct cpumask *cpu_mask) +{ + struct virtio_vdpa_device *vd_dev =3D to_virtio_vdpa_device(vq->vdev); + struct vdpa_device *vdpa =3D vd_dev->vdpa; + const struct vdpa_config_ops *ops =3D vdpa->config; + unsigned int index =3D vq->index; + + if (ops->set_vq_affinity) + return ops->set_vq_affinity(vdpa, index, cpu_mask); + + return 0; +} + +static const struct cpumask * +virtio_vdpa_get_vq_affinity(struct virtio_device *vdev, int index) +{ + struct vdpa_device *vdpa =3D vd_get_vdpa(vdev); + const struct vdpa_config_ops *ops =3D vdpa->config; + + if (ops->get_vq_affinity) + return ops->get_vq_affinity(vdpa, index); + + return NULL; +} + static const struct virtio_config_ops virtio_vdpa_config_ops =3D { .get =3D virtio_vdpa_get, .set =3D virtio_vdpa_set, @@ -349,6 +375,8 @@ static const struct virtio_config_ops virtio_vdpa_confi= g_ops =3D { .get_features =3D virtio_vdpa_get_features, .finalize_features =3D virtio_vdpa_finalize_features, .bus_name =3D virtio_vdpa_bus_name, + .set_vq_affinity =3D virtio_vdpa_set_vq_affinity, + .get_vq_affinity =3D virtio_vdpa_get_vq_affinity, }; =20 static void virtio_vdpa_release_dev(struct device *_d) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 43f59ef10cc9..d61f369f9cd6 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -250,6 +250,15 @@ struct vdpa_map_file { * @vdev: vdpa device * Returns the iova range supported by * the device. + * @set_vq_affinity: Set the irq affinity of virtqueue (optional) + * @vdev: vdpa device + * @idx: virtqueue index + * @cpu_mask: irq affinity mask + * Returns integer: success (0) or error (< 0) + * @get_vq_affinity: Get the irq affinity of virtqueue (optional) + * @vdev: vdpa device + * @idx: virtqueue index + * Returns the irq affinity mask * @set_group_asid: Set address space identifier for a * virtqueue group (optional) * @vdev: vdpa device @@ -340,6 +349,10 @@ struct vdpa_config_ops { const void *buf, unsigned int len); u32 (*get_generation)(struct vdpa_device *vdev); struct vdpa_iova_range (*get_iova_range)(struct vdpa_device *vdev); + int (*set_vq_affinity)(struct vdpa_device *vdev, u16 idx, + const struct cpumask *cpu_mask); + const struct cpumask *(*get_vq_affinity)(struct vdpa_device *vdev, + u16 idx); =20 /* DMA ops */ int (*set_map)(struct vdpa_device *vdev, unsigned int asid, --=20 2.20.1 From nobody Tue Sep 9 16:08:53 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DA76C64EC7 for ; Tue, 28 Feb 2023 09:42:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230179AbjB1JmX (ORCPT ); Tue, 28 Feb 2023 04:42:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230274AbjB1JmV (ORCPT ); Tue, 28 Feb 2023 04:42:21 -0500 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B20DB2CFE8 for ; Tue, 28 Feb 2023 01:42:19 -0800 (PST) Received: by mail-pf1-x429.google.com with SMTP id n5so5269026pfv.11 for ; Tue, 28 Feb 2023 01:42:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4LeXo4wlRPLjaFCDUlWd/vTm6oCQxlpO5FWsfhmVL4c=; b=jCF/GGa9lIkRMXEETq9mILXcYfSU+l7wJJCBS/zOo80F2sCHGtOCUJYzhpcVmI5iSJ 7o4Mi57hybiDlC4/jTo4q0MnvArNeUPFnz3V3PHt6GJmPlMoFRQrdmP/iJ8HPv+TbrqM HZX1aD7DvYy12DY1TP+jsbMCyoUlAuLBXx5ocYH6kaYVzwm4gy7z7//J2vZ2rgdyHvzg kiFaAik8PxswTSDDlDNz3EMhlPwkHwLp20/eVb/vwa4yGijprJa61KLUTWsEB/EvKktr FQyCwFFElfzKeQYZ6tMam6/e35BV0dPnAblV6Dktp82JlmCJ3se/54OII1yIUvgNouLE mylg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4LeXo4wlRPLjaFCDUlWd/vTm6oCQxlpO5FWsfhmVL4c=; b=cgwCy1shfML8F6qVqDgLGnP6c1C6mkR7W3S5OyooZ9aiVMM9/dXx9VbKfmtAu762ke LsPC3Vn/jXgyUk9u7tszTiJwC2Hjd5PM7OSi6UQx846Bjo6nkkv1DpNx6NRPE5AN/IZB 3VQxInJXkQxd5GTVlCQhcEvwSvfEJZV2yaadBgwJbDoLzdzN/rATHgpeEZL7fKWlmAqC daxzF1+IDSwejKg0jO146xLsBUu09gRVvAEmhhyXVrX6+edTF242Ll081/O4mCp7rMnf 3ND5KHpo/MkN8cWiugvkh1isA+KWaIYcvV6elPR5T38znWpWoI6IA2zFEctUPaTDI1Nj Rh4w== X-Gm-Message-State: AO0yUKVwGvs1twsf6FE67yi1+5RHVtsH46IhBjECdAe5qkibOXnm6pFJ GwlJs/1wXyO54etv67TVTed67qgamrq/HT0= X-Google-Smtp-Source: AK7set8aNyyzTj9qxjcMFF4Vf7GHi0pJZ/mlBlA4cU1sVgDRWvMC7HsMR9n9VIATIYVeAZZ+GVHvag== X-Received: by 2002:aa7:8428:0:b0:600:cc40:2589 with SMTP id q8-20020aa78428000000b00600cc402589mr1758849pfn.3.1677577339176; Tue, 28 Feb 2023 01:42:19 -0800 (PST) Received: from localhost ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id w3-20020aa78583000000b005abc30d9445sm5609518pfn.180.2023.02.28.01.42.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 01:42:18 -0800 (PST) From: Xie Yongji To: mst@redhat.com, jasowang@redhat.com, tglx@linutronix.de, hch@lst.de Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 03/11] vdpa: Add set_irq_affinity callback in vdpa_config_ops Date: Tue, 28 Feb 2023 17:41:02 +0800 Message-Id: <20230228094110.37-4-xieyongji@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230228094110.37-1-xieyongji@bytedance.com> References: <20230228094110.37-1-xieyongji@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This introduces set_irq_affinity callback in vdpa_config_ops so that vdpa device driver can get the interrupt affinity hint from the virtio device driver. The interrupt affinity hint would be needed by the interrupt affinity spreading mechanism. Signed-off-by: Xie Yongji --- drivers/virtio/virtio_vdpa.c | 4 ++++ include/linux/vdpa.h | 9 +++++++++ 2 files changed, 13 insertions(+) diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c index f72696b4c1c2..9eee8afabda8 100644 --- a/drivers/virtio/virtio_vdpa.c +++ b/drivers/virtio/virtio_vdpa.c @@ -282,9 +282,13 @@ static int virtio_vdpa_find_vqs(struct virtio_device *= vdev, unsigned int nvqs, struct virtio_vdpa_device *vd_dev =3D to_virtio_vdpa_device(vdev); struct vdpa_device *vdpa =3D vd_get_vdpa(vdev); const struct vdpa_config_ops *ops =3D vdpa->config; + struct irq_affinity default_affd =3D { 0 }; struct vdpa_callback cb; int i, err, queue_idx =3D 0; =20 + if (ops->set_irq_affinity) + ops->set_irq_affinity(vdpa, desc ? desc : &default_affd); + for (i =3D 0; i < nvqs; ++i) { if (!names[i]) { vqs[i] =3D NULL; diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index d61f369f9cd6..10bd22387276 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -259,6 +259,13 @@ struct vdpa_map_file { * @vdev: vdpa device * @idx: virtqueue index * Returns the irq affinity mask + * @set_irq_affinity: Pass the irq affinity hint (best effort) + * from the virtio device driver to vdpa + * driver (optional). + * Needed by the interrupt affinity spreading + * mechanism. + * @vdev: vdpa device + * @desc: irq affinity hint * @set_group_asid: Set address space identifier for a * virtqueue group (optional) * @vdev: vdpa device @@ -353,6 +360,8 @@ struct vdpa_config_ops { const struct cpumask *cpu_mask); const struct cpumask *(*get_vq_affinity)(struct vdpa_device *vdev, u16 idx); + void (*set_irq_affinity)(struct vdpa_device *vdev, + struct irq_affinity *desc); =20 /* DMA ops */ int (*set_map)(struct vdpa_device *vdev, unsigned int asid, --=20 2.20.1 From nobody Tue Sep 9 16:08:53 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05708C64EC7 for ; Tue, 28 Feb 2023 09:43:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231221AbjB1JnB (ORCPT ); Tue, 28 Feb 2023 04:43:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230386AbjB1Jmz (ORCPT ); Tue, 28 Feb 2023 04:42:55 -0500 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 647CC2CFC6 for ; Tue, 28 Feb 2023 01:42:23 -0800 (PST) Received: by mail-pj1-x1032.google.com with SMTP id u3-20020a17090a450300b00239db6d7d47so1896249pjg.4 for ; Tue, 28 Feb 2023 01:42:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sPERv8sXud0KxJcvENIsE+Ngn3x88OA7gINkyBoEiFY=; b=PDJpl+ldwuucI1/uzqbjlrW/O5HF0qhP66bBRMokCzBCnN+t3cSLIYouzDqz9Nf5Ps pFCsGNyykMkKM4AWTND+EAXjOsC85j6nfq9VC7ZtUeAIYaiR+TpGPPfQuHZ+vDtTL6bR 6tMN21/xSKfJB+U272N/5FhlKZQZQURhP2H2Eas/ro9UyQh0YMNZgxCkBNTE+rOD6aw4 +QsaqFlsBDi8fPf5UhdCL5NdLQtHo0ZvZZ6y19CoEp+Zm7RACTyUHFp6z8hG3LVFlrXC R0wYdsQsehOv/kHQ/Hmx2CdCeodUHGKCqAUToQeRwD86fOKid9gep9FhAh+mu0q8qGcF tk7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sPERv8sXud0KxJcvENIsE+Ngn3x88OA7gINkyBoEiFY=; b=4ZybLOzqj3o9UMF9Uwmxz0WBvRvIqu3h7p1wKQ9eMemgFHLIvp+HZp06cMwViDl7HP fcf2J+TPET2uq/tzWG43nlPBn0GMX7pO/MKy5awQAdPXSm8KeWkqrMkrZIu6FKGUQf1N WQlaixHpr5dhcKnmwOHPXH8ao/MdxmHTGLqPLGMFB9rSKV2PZGSSMOtXxVe55n9mUe1g 7XsLiJsJEkmsWwIEMjMPg4k6uCYyx2rX209p+DZ42H6c08s9gH+vyB7/N1ADiBNllfXf CIJCCrA9amloZ1QobU9Xza2VFfdv8Win4C6rEiRi+kK3oap8vBA4U4um6rAxhKfQz8/8 Z0iw== X-Gm-Message-State: AO0yUKUxpKr0oz5SWfe7Rep2xgqpTu+Z2Ld4E3zTRMxq27KGqSGNYsSD gL2oeBkCbkIa0Yp6eGJjWs+m X-Google-Smtp-Source: AK7set/aFITGNY+YtZ83ReOfFYQ+Ol58CDALtb3zGAzrhqo50WDsMJ08vZG4fTrooH5e98s2tRxg3Q== X-Received: by 2002:a17:903:22ca:b0:19d:90f:6c6c with SMTP id y10-20020a17090322ca00b0019d090f6c6cmr2816251plg.17.1677577342819; Tue, 28 Feb 2023 01:42:22 -0800 (PST) Received: from localhost ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id v14-20020a17090331ce00b0019906c4c9dcsm5856719ple.98.2023.02.28.01.42.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 01:42:22 -0800 (PST) From: Xie Yongji To: mst@redhat.com, jasowang@redhat.com, tglx@linutronix.de, hch@lst.de Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 04/11] vduse: Refactor allocation for vduse virtqueues Date: Tue, 28 Feb 2023 17:41:03 +0800 Message-Id: <20230228094110.37-5-xieyongji@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230228094110.37-1-xieyongji@bytedance.com> References: <20230228094110.37-1-xieyongji@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Allocate memory for vduse virtqueues one by one instead of doing one allocation for all of them. This is a preparation for adding sysfs interface for virtqueues. Signed-off-by: Xie Yongji Acked-by: Jason Wang --- drivers/vdpa/vdpa_user/vduse_dev.c | 98 ++++++++++++++++++++---------- 1 file changed, 66 insertions(+), 32 deletions(-) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vd= use_dev.c index 0c3b48616a9f..98359d87a06f 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -76,7 +76,7 @@ struct vduse_umem { struct vduse_dev { struct vduse_vdpa *vdev; struct device *dev; - struct vduse_virtqueue *vqs; + struct vduse_virtqueue **vqs; struct vduse_iova_domain *domain; char *name; struct mutex lock; @@ -434,7 +434,7 @@ static void vduse_dev_reset(struct vduse_dev *dev) flush_work(&dev->inject); =20 for (i =3D 0; i < dev->vq_num; i++) { - struct vduse_virtqueue *vq =3D &dev->vqs[i]; + struct vduse_virtqueue *vq =3D dev->vqs[i]; =20 vq->ready =3D false; vq->desc_addr =3D 0; @@ -466,7 +466,7 @@ static int vduse_vdpa_set_vq_address(struct vdpa_device= *vdpa, u16 idx, u64 device_area) { struct vduse_dev *dev =3D vdpa_to_vduse(vdpa); - struct vduse_virtqueue *vq =3D &dev->vqs[idx]; + struct vduse_virtqueue *vq =3D dev->vqs[idx]; =20 vq->desc_addr =3D desc_area; vq->driver_addr =3D driver_area; @@ -500,7 +500,7 @@ static void vduse_vq_kick_work(struct work_struct *work) static void vduse_vdpa_kick_vq(struct vdpa_device *vdpa, u16 idx) { struct vduse_dev *dev =3D vdpa_to_vduse(vdpa); - struct vduse_virtqueue *vq =3D &dev->vqs[idx]; + struct vduse_virtqueue *vq =3D dev->vqs[idx]; =20 if (!eventfd_signal_allowed()) { schedule_work(&vq->kick); @@ -513,7 +513,7 @@ static void vduse_vdpa_set_vq_cb(struct vdpa_device *vd= pa, u16 idx, struct vdpa_callback *cb) { struct vduse_dev *dev =3D vdpa_to_vduse(vdpa); - struct vduse_virtqueue *vq =3D &dev->vqs[idx]; + struct vduse_virtqueue *vq =3D dev->vqs[idx]; =20 spin_lock(&vq->irq_lock); vq->cb.callback =3D cb->callback; @@ -524,7 +524,7 @@ static void vduse_vdpa_set_vq_cb(struct vdpa_device *vd= pa, u16 idx, static void vduse_vdpa_set_vq_num(struct vdpa_device *vdpa, u16 idx, u32 n= um) { struct vduse_dev *dev =3D vdpa_to_vduse(vdpa); - struct vduse_virtqueue *vq =3D &dev->vqs[idx]; + struct vduse_virtqueue *vq =3D dev->vqs[idx]; =20 vq->num =3D num; } @@ -533,7 +533,7 @@ static void vduse_vdpa_set_vq_ready(struct vdpa_device = *vdpa, u16 idx, bool ready) { struct vduse_dev *dev =3D vdpa_to_vduse(vdpa); - struct vduse_virtqueue *vq =3D &dev->vqs[idx]; + struct vduse_virtqueue *vq =3D dev->vqs[idx]; =20 vq->ready =3D ready; } @@ -541,7 +541,7 @@ static void vduse_vdpa_set_vq_ready(struct vdpa_device = *vdpa, static bool vduse_vdpa_get_vq_ready(struct vdpa_device *vdpa, u16 idx) { struct vduse_dev *dev =3D vdpa_to_vduse(vdpa); - struct vduse_virtqueue *vq =3D &dev->vqs[idx]; + struct vduse_virtqueue *vq =3D dev->vqs[idx]; =20 return vq->ready; } @@ -550,7 +550,7 @@ static int vduse_vdpa_set_vq_state(struct vdpa_device *= vdpa, u16 idx, const struct vdpa_vq_state *state) { struct vduse_dev *dev =3D vdpa_to_vduse(vdpa); - struct vduse_virtqueue *vq =3D &dev->vqs[idx]; + struct vduse_virtqueue *vq =3D dev->vqs[idx]; =20 if (dev->driver_features & BIT_ULL(VIRTIO_F_RING_PACKED)) { vq->state.packed.last_avail_counter =3D @@ -569,7 +569,7 @@ static int vduse_vdpa_get_vq_state(struct vdpa_device *= vdpa, u16 idx, struct vdpa_vq_state *state) { struct vduse_dev *dev =3D vdpa_to_vduse(vdpa); - struct vduse_virtqueue *vq =3D &dev->vqs[idx]; + struct vduse_virtqueue *vq =3D dev->vqs[idx]; =20 if (dev->driver_features & BIT_ULL(VIRTIO_F_RING_PACKED)) return vduse_dev_get_vq_state_packed(dev, vq, &state->packed); @@ -624,8 +624,8 @@ static u16 vduse_vdpa_get_vq_num_max(struct vdpa_device= *vdpa) int i; =20 for (i =3D 0; i < dev->vq_num; i++) - if (num_max < dev->vqs[i].num_max) - num_max =3D dev->vqs[i].num_max; + if (num_max < dev->vqs[i]->num_max) + num_max =3D dev->vqs[i]->num_max; =20 return num_max; } @@ -863,7 +863,7 @@ static int vduse_kickfd_setup(struct vduse_dev *dev, return -EINVAL; =20 index =3D array_index_nospec(eventfd->index, dev->vq_num); - vq =3D &dev->vqs[index]; + vq =3D dev->vqs[index]; if (eventfd->fd >=3D 0) { ctx =3D eventfd_ctx_fdget(eventfd->fd); if (IS_ERR(ctx)) @@ -889,7 +889,7 @@ static bool vduse_dev_is_ready(struct vduse_dev *dev) int i; =20 for (i =3D 0; i < dev->vq_num; i++) - if (!dev->vqs[i].num_max) + if (!dev->vqs[i]->num_max) return false; =20 return true; @@ -1130,7 +1130,7 @@ static long vduse_dev_ioctl(struct file *file, unsign= ed int cmd, break; =20 index =3D array_index_nospec(config.index, dev->vq_num); - dev->vqs[index].num_max =3D config.max_size; + dev->vqs[index]->num_max =3D config.max_size; ret =3D 0; break; } @@ -1148,7 +1148,7 @@ static long vduse_dev_ioctl(struct file *file, unsign= ed int cmd, break; =20 index =3D array_index_nospec(vq_info.index, dev->vq_num); - vq =3D &dev->vqs[index]; + vq =3D dev->vqs[index]; vq_info.desc_addr =3D vq->desc_addr; vq_info.driver_addr =3D vq->driver_addr; vq_info.device_addr =3D vq->device_addr; @@ -1198,7 +1198,7 @@ static long vduse_dev_ioctl(struct file *file, unsign= ed int cmd, break; =20 index =3D array_index_nospec(index, dev->vq_num); - ret =3D vduse_dev_queue_irq_work(dev, &dev->vqs[index].inject); + ret =3D vduse_dev_queue_irq_work(dev, &dev->vqs[index]->inject); break; } case VDUSE_IOTLB_REG_UMEM: { @@ -1339,6 +1339,49 @@ static const struct file_operations vduse_dev_fops = =3D { .llseek =3D noop_llseek, }; =20 +static void vduse_dev_deinit_vqs(struct vduse_dev *dev) +{ + int i; + + if (!dev->vqs) + return; + + for (i =3D 0; i < dev->vq_num; i++) + kfree(dev->vqs[i]); + kfree(dev->vqs); +} + +static int vduse_dev_init_vqs(struct vduse_dev *dev, u32 vq_align, u32 vq_= num) +{ + int i; + + dev->vq_align =3D vq_align; + dev->vq_num =3D vq_num; + dev->vqs =3D kcalloc(dev->vq_num, sizeof(*dev->vqs), GFP_KERNEL); + if (!dev->vqs) + return -ENOMEM; + + for (i =3D 0; i < vq_num; i++) { + dev->vqs[i] =3D kzalloc(sizeof(*dev->vqs[i]), GFP_KERNEL); + if (!dev->vqs[i]) + goto err; + + dev->vqs[i]->index =3D i; + INIT_WORK(&dev->vqs[i]->inject, vduse_vq_irq_inject); + INIT_WORK(&dev->vqs[i]->kick, vduse_vq_kick_work); + spin_lock_init(&dev->vqs[i]->kick_lock); + spin_lock_init(&dev->vqs[i]->irq_lock); + } + + return 0; +err: + while (i--) + kfree(dev->vqs[i]); + kfree(dev->vqs); + dev->vqs =3D NULL; + return -ENOMEM; +} + static struct vduse_dev *vduse_dev_create(void) { struct vduse_dev *dev =3D kzalloc(sizeof(*dev), GFP_KERNEL); @@ -1396,7 +1439,7 @@ static int vduse_destroy_dev(char *name) device_destroy(vduse_class, MKDEV(MAJOR(vduse_major), dev->minor)); idr_remove(&vduse_idr, dev->minor); kvfree(dev->config); - kfree(dev->vqs); + vduse_dev_deinit_vqs(dev); vduse_domain_destroy(dev->domain); kfree(dev->name); vduse_dev_destroy(dev); @@ -1486,7 +1529,7 @@ ATTRIBUTE_GROUPS(vduse_dev); static int vduse_create_dev(struct vduse_dev_config *config, void *config_buf, u64 api_version) { - int i, ret; + int ret; struct vduse_dev *dev; =20 ret =3D -EEXIST; @@ -1513,19 +1556,10 @@ static int vduse_create_dev(struct vduse_dev_config= *config, =20 dev->config =3D config_buf; dev->config_size =3D config->config_size; - dev->vq_align =3D config->vq_align; - dev->vq_num =3D config->vq_num; - dev->vqs =3D kcalloc(dev->vq_num, sizeof(*dev->vqs), GFP_KERNEL); - if (!dev->vqs) - goto err_vqs; =20 - for (i =3D 0; i < dev->vq_num; i++) { - dev->vqs[i].index =3D i; - INIT_WORK(&dev->vqs[i].inject, vduse_vq_irq_inject); - INIT_WORK(&dev->vqs[i].kick, vduse_vq_kick_work); - spin_lock_init(&dev->vqs[i].kick_lock); - spin_lock_init(&dev->vqs[i].irq_lock); - } + ret =3D vduse_dev_init_vqs(dev, config->vq_align, config->vq_num); + if (ret) + goto err_vqs; =20 ret =3D idr_alloc(&vduse_idr, dev, 1, VDUSE_DEV_MAX, GFP_KERNEL); if (ret < 0) @@ -1546,7 +1580,7 @@ static int vduse_create_dev(struct vduse_dev_config *= config, err_dev: idr_remove(&vduse_idr, dev->minor); err_idr: - kfree(dev->vqs); + vduse_dev_deinit_vqs(dev); err_vqs: vduse_domain_destroy(dev->domain); err_domain: --=20 2.20.1 From nobody Tue Sep 9 16:08:53 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C594CC64EC7 for ; Tue, 28 Feb 2023 09:42:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230314AbjB1Jmf (ORCPT ); Tue, 28 Feb 2023 04:42:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230421AbjB1Jmb (ORCPT ); Tue, 28 Feb 2023 04:42:31 -0500 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46DDC2D159 for ; Tue, 28 Feb 2023 01:42:27 -0800 (PST) Received: by mail-pl1-x633.google.com with SMTP id i10so9754952plr.9 for ; Tue, 28 Feb 2023 01:42:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2hJ42EGznmmKdit+DkkUJ/Qt7N29KFOU8Z1Bu00RmiQ=; b=aNO16KpcRslYIothZ5ipnOru7C83teAcH/Izpsl5+VmpmgqL6Aro+jC4UBMPVIl8gg H8VOlspRP99FW5HRqlnu2KR/cRKQ2bu6V9NWQrUPmHDHf4W+swM9rDCCBxq4a6yfVp7y gexE/286EPl6w8XSysh5hwFI6x/ddWDN6UEUg1LgAOqMAnbo6JuUJr4lTgNy89NaxFt+ JbD0sjqk6DOAMecWDqpqK8fGL27nbwpF4vHpArxc1h5oQ36e8sizg9Ik+FHzmCzW7h7Y CCnAm4KnbAosnfpz0YrdB5vc8o1i4M8D/tCbvxQZIjcqNL6gJ5tkuw9lGquVDDtjzYwT +tOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2hJ42EGznmmKdit+DkkUJ/Qt7N29KFOU8Z1Bu00RmiQ=; b=GHhZ8y6P7wbqpyxNHSfnoA5gOk/N3GmjqwcaTEhbbuGZv4jYn4Q3XsE7wgDq0nwirY WOR8fziVyLjYiO/nDGhj7I+aYvj4lQ5utivrYTSLoMdFyF4tA+zsSgxGWjbrtIDrJyLO l1vpgTq6TGnJqlIojJ4LIc05t5NSnXrUkr2pquSi1Ngfo/c+MIyp7UMNjkcCLlVESDuM h+TKdfZvDHS9PSfoIjGx1qfi3NdtGWzwCWtIUXabGtEx4D/MkNP233g4/9k0grrHfgwh 7t928kepQi9xnp5d7jz3Fr4MgfhXXAIioEhLEYSRQA+LYQuA5f3aoxQt5nm9tiWv6v6i rfOw== X-Gm-Message-State: AO0yUKVOZryQlUSq9OrdS9w9ZAluoXDPVlRqetwuBOY3vKWRI+K20Vst ajeTwyE3z1R5ipytboAzPtlj X-Google-Smtp-Source: AK7set+TPNFC/xncKPCXhfN++ABlTZBeiZtGV3YKs6GK4wnSj6THLwqFKjksu7//B6tGl/kOzqVSJA== X-Received: by 2002:a17:903:2290:b0:19b:33c0:4097 with SMTP id b16-20020a170903229000b0019b33c04097mr2524721plh.27.1677577346715; Tue, 28 Feb 2023 01:42:26 -0800 (PST) Received: from localhost ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id jj13-20020a170903048d00b001967692d6f5sm6054858plb.227.2023.02.28.01.42.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 01:42:26 -0800 (PST) From: Xie Yongji To: mst@redhat.com, jasowang@redhat.com, tglx@linutronix.de, hch@lst.de Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 05/11] vduse: Support automatic irq callback affinity Date: Tue, 28 Feb 2023 17:41:04 +0800 Message-Id: <20230228094110.37-6-xieyongji@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230228094110.37-1-xieyongji@bytedance.com> References: <20230228094110.37-1-xieyongji@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This brings current interrupt affinity spreading mechanism to vduse device. We will make use of group_cpus_evenly() to create an irq callback affinity mask for each virtqueue of vduse device. Then we will spread IRQs between CPUs in the affinity mask, in a round-robin manner, to run the irq callback. Signed-off-by: Xie Yongji --- drivers/vdpa/vdpa_user/vduse_dev.c | 130 +++++++++++++++++++++++++++-- 1 file changed, 123 insertions(+), 7 deletions(-) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vd= use_dev.c index 98359d87a06f..bde28a8692d5 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -23,6 +23,8 @@ #include #include #include +#include +#include #include #include #include @@ -41,6 +43,8 @@ #define VDUSE_IOVA_SIZE (128 * 1024 * 1024) #define VDUSE_MSG_DEFAULT_TIMEOUT 30 =20 +#define IRQ_UNBOUND -1 + struct vduse_virtqueue { u16 index; u16 num_max; @@ -57,6 +61,8 @@ struct vduse_virtqueue { struct vdpa_callback cb; struct work_struct inject; struct work_struct kick; + int irq_effective_cpu; + struct cpumask irq_affinity; }; =20 struct vduse_dev; @@ -128,6 +134,7 @@ static struct class *vduse_class; static struct cdev vduse_ctrl_cdev; static struct cdev vduse_cdev; static struct workqueue_struct *vduse_irq_wq; +static struct workqueue_struct *vduse_irq_bound_wq; =20 static u32 allowed_device_id[] =3D { VIRTIO_ID_BLOCK, @@ -708,6 +715,82 @@ static u32 vduse_vdpa_get_generation(struct vdpa_devic= e *vdpa) return dev->generation; } =20 +static void default_calc_sets(struct irq_affinity *affd, unsigned int affv= ecs) +{ + affd->nr_sets =3D 1; + affd->set_size[0] =3D affvecs; +} + +struct cpumask * +create_affinity_masks(unsigned int nvecs, struct irq_affinity *affd) +{ + unsigned int affvecs =3D 0, curvec, usedvecs, i; + struct cpumask *masks =3D NULL; + + if (nvecs > affd->pre_vectors + affd->post_vectors) + affvecs =3D nvecs - affd->pre_vectors - affd->post_vectors; + + if (!affd->calc_sets) + affd->calc_sets =3D default_calc_sets; + + affd->calc_sets(affd, affvecs); + + if (!affvecs) + return NULL; + + masks =3D kcalloc(nvecs, sizeof(*masks), GFP_KERNEL); + if (!masks) + return NULL; + + /* Fill out vectors at the beginning that don't need affinity */ + for (curvec =3D 0; curvec < affd->pre_vectors; curvec++) + cpumask_setall(&masks[curvec]); + + for (i =3D 0, usedvecs =3D 0; i < affd->nr_sets; i++) { + unsigned int this_vecs =3D affd->set_size[i]; + int j; + struct cpumask *result =3D group_cpus_evenly(this_vecs); + + if (!result) { + kfree(masks); + return NULL; + } + + for (j =3D 0; j < this_vecs; j++) + cpumask_copy(&masks[curvec + j], &result[j]); + kfree(result); + + curvec +=3D this_vecs; + usedvecs +=3D this_vecs; + } + + /* Fill out vectors at the end that don't need affinity */ + if (usedvecs >=3D affvecs) + curvec =3D affd->pre_vectors + affvecs; + else + curvec =3D affd->pre_vectors + usedvecs; + for (; curvec < nvecs; curvec++) + cpumask_setall(&masks[curvec]); + + return masks; +} + +static void vduse_vdpa_set_irq_affinity(struct vdpa_device *vdpa, + struct irq_affinity *desc) +{ + struct vduse_dev *dev =3D vdpa_to_vduse(vdpa); + struct cpumask *masks; + int i; + + masks =3D create_affinity_masks(dev->vq_num, desc); + if (!masks) + return; + + for (i =3D 0; i < dev->vq_num; i++) + cpumask_copy(&dev->vqs[i]->irq_affinity, &masks[i]); + kfree(masks); +} + static int vduse_vdpa_set_map(struct vdpa_device *vdpa, unsigned int asid, struct vhost_iotlb *iotlb) @@ -758,6 +841,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_o= ps =3D { .get_config =3D vduse_vdpa_get_config, .set_config =3D vduse_vdpa_set_config, .get_generation =3D vduse_vdpa_get_generation, + .set_irq_affinity =3D vduse_vdpa_set_irq_affinity, .reset =3D vduse_vdpa_reset, .set_map =3D vduse_vdpa_set_map, .free =3D vduse_vdpa_free, @@ -917,7 +1001,8 @@ static void vduse_vq_irq_inject(struct work_struct *wo= rk) } =20 static int vduse_dev_queue_irq_work(struct vduse_dev *dev, - struct work_struct *irq_work) + struct work_struct *irq_work, + int irq_effective_cpu) { int ret =3D -EINVAL; =20 @@ -926,7 +1011,11 @@ static int vduse_dev_queue_irq_work(struct vduse_dev = *dev, goto unlock; =20 ret =3D 0; - queue_work(vduse_irq_wq, irq_work); + if (irq_effective_cpu =3D=3D IRQ_UNBOUND) + queue_work(vduse_irq_wq, irq_work); + else + queue_work_on(irq_effective_cpu, + vduse_irq_bound_wq, irq_work); unlock: up_read(&dev->rwsem); =20 @@ -1029,6 +1118,22 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev, return ret; } =20 +static void vduse_vq_update_effective_cpu(struct vduse_virtqueue *vq) +{ + int curr_cpu =3D vq->irq_effective_cpu; + + while (true) { + curr_cpu =3D cpumask_next(curr_cpu, &vq->irq_affinity); + if (cpu_online(curr_cpu)) + break; + + if (curr_cpu >=3D nr_cpu_ids) + curr_cpu =3D -1; + } + + vq->irq_effective_cpu =3D curr_cpu; +} + static long vduse_dev_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -1111,7 +1216,7 @@ static long vduse_dev_ioctl(struct file *file, unsign= ed int cmd, break; } case VDUSE_DEV_INJECT_CONFIG_IRQ: - ret =3D vduse_dev_queue_irq_work(dev, &dev->inject); + ret =3D vduse_dev_queue_irq_work(dev, &dev->inject, IRQ_UNBOUND); break; case VDUSE_VQ_SETUP: { struct vduse_vq_config config; @@ -1198,7 +1303,10 @@ static long vduse_dev_ioctl(struct file *file, unsig= ned int cmd, break; =20 index =3D array_index_nospec(index, dev->vq_num); - ret =3D vduse_dev_queue_irq_work(dev, &dev->vqs[index]->inject); + + vduse_vq_update_effective_cpu(dev->vqs[index]); + ret =3D vduse_dev_queue_irq_work(dev, &dev->vqs[index]->inject, + dev->vqs[index]->irq_effective_cpu); break; } case VDUSE_IOTLB_REG_UMEM: { @@ -1367,10 +1475,12 @@ static int vduse_dev_init_vqs(struct vduse_dev *dev= , u32 vq_align, u32 vq_num) goto err; =20 dev->vqs[i]->index =3D i; + dev->vqs[i]->irq_effective_cpu =3D -1; INIT_WORK(&dev->vqs[i]->inject, vduse_vq_irq_inject); INIT_WORK(&dev->vqs[i]->kick, vduse_vq_kick_work); spin_lock_init(&dev->vqs[i]->kick_lock); spin_lock_init(&dev->vqs[i]->irq_lock); + cpumask_setall(&dev->vqs[i]->irq_affinity); } =20 return 0; @@ -1858,12 +1968,15 @@ static int vduse_init(void) if (ret) goto err_cdev; =20 + ret =3D -ENOMEM; vduse_irq_wq =3D alloc_workqueue("vduse-irq", WQ_HIGHPRI | WQ_SYSFS | WQ_UNBOUND, 0); - if (!vduse_irq_wq) { - ret =3D -ENOMEM; + if (!vduse_irq_wq) goto err_wq; - } + + vduse_irq_bound_wq =3D alloc_workqueue("vduse-irq-bound", WQ_HIGHPRI, 0); + if (!vduse_irq_bound_wq) + goto err_bound_wq; =20 ret =3D vduse_domain_init(); if (ret) @@ -1877,6 +1990,8 @@ static int vduse_init(void) err_mgmtdev: vduse_domain_exit(); err_domain: + destroy_workqueue(vduse_irq_bound_wq); +err_bound_wq: destroy_workqueue(vduse_irq_wq); err_wq: cdev_del(&vduse_cdev); @@ -1896,6 +2011,7 @@ static void vduse_exit(void) { vduse_mgmtdev_exit(); vduse_domain_exit(); + destroy_workqueue(vduse_irq_bound_wq); destroy_workqueue(vduse_irq_wq); cdev_del(&vduse_cdev); device_destroy(vduse_class, vduse_major); --=20 2.20.1 From nobody Tue Sep 9 16:08:53 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A773FC64EC7 for ; Tue, 28 Feb 2023 09:43:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230324AbjB1JnR (ORCPT ); Tue, 28 Feb 2023 04:43:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230280AbjB1JnB (ORCPT ); Tue, 28 Feb 2023 04:43:01 -0500 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C55CB2CC6A for ; Tue, 28 Feb 2023 01:42:30 -0800 (PST) Received: by mail-pl1-x62f.google.com with SMTP id v11so6244207plz.8 for ; Tue, 28 Feb 2023 01:42:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FUYWrtxJPMWqrVBSuRM8/ObKVuGrxJBlEB/bWz01tiQ=; b=Es0tXIpQ6ZSjbv3k9aP5Z4OWz5PL5qzw/4xKb63RJQIhAR3Hb+kg1h1q05xImDI0Of fZSwPTiIEGLjaX60iBkE37bvjMeKy6DVbt0v1TK2PD8gNbsFu8Qnz250ywc/Oa9RIYRA FgkrIPXUQspAN26+vu+ahfP1HcUuc5ezqIQRyq1rEVHvOoa9y8Q46eHT0iBFhWIRyxju uNrUPXc+19EZzB5TfIaHc0uurBmzt48n8ak6qs8Ypux0TZpshO/GVaOFk1t5daMBNno3 3oHQjtvfHT57BPD9QGivd/fppis29De1CHA/4wJ+yQlSNPDF+TW6fQbBWWnEO4csR9dF F/Jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FUYWrtxJPMWqrVBSuRM8/ObKVuGrxJBlEB/bWz01tiQ=; b=I+BP7v64nTD6GE6N8oWqflL0lvaWBlrx78mFBPEzt8FykaSb/FOL70ZbNGpG1/hTof Q0gJC9WJb84Kc+HvYxp5m0DhJ8zOtQfwlhUruLmAh+Rgdvjt2bbR0lPQ7KZYtGFJgP16 v66AT22AQ6oEQ+MmHdqFtjefzNvX+xbTJYloOZ9ud9fFmb0QbIrqJ3A5QX52s5u3oGGw cYvSpWMUhDYmZRDVn64AGFb9nbDMr06foM14N8u2wfNtu34yvlHqs8967Fb/dsAuCWkz HOznWM8Pvfp5q6VtIG6zd28oZVUOvv/ePWwgSEtFbYTdxjprySzw/Ak9awrAeFa4Jt82 PRFw== X-Gm-Message-State: AO0yUKW0ZIliKIlyn2Kfc3uohuPAVUAgrFFEUXh3/qJ73+x8qnUfwU3o /x7y53G79Yq05zAkDHU9FiNE X-Google-Smtp-Source: AK7set+CsMXxwRfnpjHDrrbMQmI4HxT0UuXcehhgf1a3fIyXnztPgZ0KrxjQncEHzry5tMsB8BjytQ== X-Received: by 2002:a17:903:124d:b0:19c:a5dd:fadb with SMTP id u13-20020a170903124d00b0019ca5ddfadbmr2190908plh.54.1677577350310; Tue, 28 Feb 2023 01:42:30 -0800 (PST) Received: from localhost ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id ji10-20020a170903324a00b00194caf3e975sm6086982plb.208.2023.02.28.01.42.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 01:42:29 -0800 (PST) From: Xie Yongji To: mst@redhat.com, jasowang@redhat.com, tglx@linutronix.de, hch@lst.de Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 06/11] vduse: Support set/get_vq_affinity callbacks Date: Tue, 28 Feb 2023 17:41:05 +0800 Message-Id: <20230228094110.37-7-xieyongji@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230228094110.37-1-xieyongji@bytedance.com> References: <20230228094110.37-1-xieyongji@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Since we already support irq callback affinity management, let's implement the set/get_vq_affinity callbacks to make it possible for the virtio device driver to change or be aware of the affinity. This would make it possible for the virtio-blk driver to build the blk-mq queues based on the irq callback affinity. Signed-off-by: Xie Yongji --- drivers/vdpa/vdpa_user/vduse_dev.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vd= use_dev.c index bde28a8692d5..e2988a1476e4 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -775,6 +775,23 @@ create_affinity_masks(unsigned int nvecs, struct irq_a= ffinity *affd) return masks; } =20 +static int vduse_vdpa_set_vq_affinity(struct vdpa_device *vdpa, u16 idx, + const struct cpumask *cpu_mask) +{ + struct vduse_dev *dev =3D vdpa_to_vduse(vdpa); + + cpumask_copy(&dev->vqs[idx]->irq_affinity, cpu_mask); + return 0; +} + +static const struct cpumask * +vduse_vdpa_get_vq_affinity(struct vdpa_device *vdpa, u16 idx) +{ + struct vduse_dev *dev =3D vdpa_to_vduse(vdpa); + + return &dev->vqs[idx]->irq_affinity; +} + static void vduse_vdpa_set_irq_affinity(struct vdpa_device *vdpa, struct irq_affinity *desc) { @@ -841,6 +858,8 @@ static const struct vdpa_config_ops vduse_vdpa_config_o= ps =3D { .get_config =3D vduse_vdpa_get_config, .set_config =3D vduse_vdpa_set_config, .get_generation =3D vduse_vdpa_get_generation, + .set_vq_affinity =3D vduse_vdpa_set_vq_affinity, + .get_vq_affinity =3D vduse_vdpa_get_vq_affinity, .set_irq_affinity =3D vduse_vdpa_set_irq_affinity, .reset =3D vduse_vdpa_reset, .set_map =3D vduse_vdpa_set_map, --=20 2.20.1 From nobody Tue Sep 9 16:08:53 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 225B4C64EC7 for ; Tue, 28 Feb 2023 09:42:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231200AbjB1Jmq (ORCPT ); Tue, 28 Feb 2023 04:42:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231167AbjB1Jmk (ORCPT ); Tue, 28 Feb 2023 04:42:40 -0500 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7DB562CFF7 for ; Tue, 28 Feb 2023 01:42:34 -0800 (PST) Received: by mail-pj1-x102b.google.com with SMTP id m8-20020a17090a4d8800b002377bced051so13101202pjh.0 for ; Tue, 28 Feb 2023 01:42:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gXM72I6oSJKufWn/quSJk+pwmhX/n98osWkCo8kxouU=; b=PgzIbtzFGF22EBlwqzvX8aiRohLZBqC02pR3pKa8B4D5vk0aOOrLo8IGekJ5gaNjYZ 9T7OLh6JAD2wnf4tYM5nKSnLhTfltDCh/erNsMtGeecXrqibixUV/ZeFP7WBRTxwu4AI FbIVeAJKiEzHw7Tk68abwdOltACASmXEKOBNz11bZe+ZAZ8e8NvxUj/hrPAJtvQ42dEt hwb4GcL4VQHsgcpIbXX7zoiw80WUzrPYvfOtxtRdMJxJ6esToDovjXof8yTlnOesGwlt 4744Oxt0Ev+xo4QN5J25IKiav+UpBNZ+vAbwvLEibRMRpuv8sTux7V5tePkecKACsynt lxEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gXM72I6oSJKufWn/quSJk+pwmhX/n98osWkCo8kxouU=; b=kScSBv65sH1CEi5VaAEcrDK5IWpo5F5kGtJyLvauS/0mwhYBG0tmaE6sCtR/uH/pum PPmxb9op1vHFYHyYcVh2pzP17T53w9iTvQ6bQ8Im/mX9kKsNWoIUoc7kDggyRrAckJ48 sXPhLxw3saoNHcbViAfiFjH2jrbdgliFu2elhLbWOEvpqlCG8gs9A61+n/ovpBjWYC0k 9V6Ue/I8hHTsWMQcTKVFn1DYzOKFbukcPyaExUky7BkmKth9Z7a5hTVEX8ViTHQ9Om2c UMy+VnI/+DD1QNpnDj0JUf07zioXH0YZa2eoitXDwSzhniS5UfW8Ez9JxK3SsNPbYoeN n6yw== X-Gm-Message-State: AO0yUKWSV8AiG9SgMdTd3S19sxmv1O9jXskT2W7Zjb2YfQSA7Yy2MtyA TSGo4BN3T5NOclkGSKcKuzbd X-Google-Smtp-Source: AK7set+4l+2/JtNFVaFrQaY796soQpS/3Zz0be4R5uLqq2Oyfmu9qUXepWUwLA70p5FFU4+qqws91g== X-Received: by 2002:a17:902:ea0e:b0:19d:1fe3:4941 with SMTP id s14-20020a170902ea0e00b0019d1fe34941mr2586722plg.2.1677577353921; Tue, 28 Feb 2023 01:42:33 -0800 (PST) Received: from localhost ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id u13-20020a170902714d00b00194ac38bc86sm6059429plm.131.2023.02.28.01.42.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 01:42:33 -0800 (PST) From: Xie Yongji To: mst@redhat.com, jasowang@redhat.com, tglx@linutronix.de, hch@lst.de Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 07/11] vduse: Add sysfs interface for irq callback affinity Date: Tue, 28 Feb 2023 17:41:06 +0800 Message-Id: <20230228094110.37-8-xieyongji@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230228094110.37-1-xieyongji@bytedance.com> References: <20230228094110.37-1-xieyongji@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add sysfs interface for each vduse virtqueue to get/set the affinity for irq callback. This might be useful for performance tuning when the irq callback affinity mask contains more than one CPU. Signed-off-by: Xie Yongji --- drivers/vdpa/vdpa_user/vduse_dev.c | 124 ++++++++++++++++++++++++++--- 1 file changed, 113 insertions(+), 11 deletions(-) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vd= use_dev.c index e2988a1476e4..869cc7860d82 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -63,6 +63,7 @@ struct vduse_virtqueue { struct work_struct kick; int irq_effective_cpu; struct cpumask irq_affinity; + struct kobject kobj; }; =20 struct vduse_dev; @@ -1466,6 +1467,96 @@ static const struct file_operations vduse_dev_fops = =3D { .llseek =3D noop_llseek, }; =20 +static ssize_t irq_cb_affinity_show(struct vduse_virtqueue *vq, char *buf) +{ + return sprintf(buf, "%*pb\n", cpumask_pr_args(&vq->irq_affinity)); +} + +static ssize_t irq_cb_affinity_store(struct vduse_virtqueue *vq, + const char *buf, size_t count) +{ + cpumask_var_t new_value; + int ret; + + if (!zalloc_cpumask_var(&new_value, GFP_KERNEL)) + return -ENOMEM; + + ret =3D cpumask_parse(buf, new_value); + if (ret) + goto free_mask; + + ret =3D -EINVAL; + if (!cpumask_intersects(new_value, cpu_online_mask)) + goto free_mask; + + cpumask_copy(&vq->irq_affinity, new_value); + ret =3D count; +free_mask: + free_cpumask_var(new_value); + return ret; +} + +struct vq_sysfs_entry { + struct attribute attr; + ssize_t (*show)(struct vduse_virtqueue *vq, char *buf); + ssize_t (*store)(struct vduse_virtqueue *vq, const char *buf, + size_t count); +}; + +static struct vq_sysfs_entry irq_cb_affinity_attr =3D __ATTR_RW(irq_cb_aff= inity); + +static struct attribute *vq_attrs[] =3D { + &irq_cb_affinity_attr.attr, + NULL, +}; +ATTRIBUTE_GROUPS(vq); + +static ssize_t vq_attr_show(struct kobject *kobj, struct attribute *attr, + char *buf) +{ + struct vduse_virtqueue *vq =3D container_of(kobj, + struct vduse_virtqueue, kobj); + struct vq_sysfs_entry *entry =3D container_of(attr, + struct vq_sysfs_entry, attr); + + if (!entry->show) + return -EIO; + + return entry->show(vq, buf); +} + +static ssize_t vq_attr_store(struct kobject *kobj, struct attribute *attr, + const char *buf, size_t count) +{ + struct vduse_virtqueue *vq =3D container_of(kobj, + struct vduse_virtqueue, kobj); + struct vq_sysfs_entry *entry =3D container_of(attr, + struct vq_sysfs_entry, attr); + + if (!entry->store) + return -EIO; + + return entry->store(vq, buf, count); +} + +static const struct sysfs_ops vq_sysfs_ops =3D { + .show =3D vq_attr_show, + .store =3D vq_attr_store, +}; + +static void vq_release(struct kobject *kobj) +{ + struct vduse_virtqueue *vq =3D container_of(kobj, + struct vduse_virtqueue, kobj); + kfree(vq); +} + +static const struct kobj_type vq_type =3D { + .release =3D vq_release, + .sysfs_ops =3D &vq_sysfs_ops, + .default_groups =3D vq_groups, +}; + static void vduse_dev_deinit_vqs(struct vduse_dev *dev) { int i; @@ -1474,13 +1565,13 @@ static void vduse_dev_deinit_vqs(struct vduse_dev *= dev) return; =20 for (i =3D 0; i < dev->vq_num; i++) - kfree(dev->vqs[i]); + kobject_put(&dev->vqs[i]->kobj); kfree(dev->vqs); } =20 static int vduse_dev_init_vqs(struct vduse_dev *dev, u32 vq_align, u32 vq_= num) { - int i; + int ret, i; =20 dev->vq_align =3D vq_align; dev->vq_num =3D vq_num; @@ -1490,8 +1581,10 @@ static int vduse_dev_init_vqs(struct vduse_dev *dev,= u32 vq_align, u32 vq_num) =20 for (i =3D 0; i < vq_num; i++) { dev->vqs[i] =3D kzalloc(sizeof(*dev->vqs[i]), GFP_KERNEL); - if (!dev->vqs[i]) + if (!dev->vqs[i]) { + ret =3D -ENOMEM; goto err; + } =20 dev->vqs[i]->index =3D i; dev->vqs[i]->irq_effective_cpu =3D -1; @@ -1500,15 +1593,23 @@ static int vduse_dev_init_vqs(struct vduse_dev *dev= , u32 vq_align, u32 vq_num) spin_lock_init(&dev->vqs[i]->kick_lock); spin_lock_init(&dev->vqs[i]->irq_lock); cpumask_setall(&dev->vqs[i]->irq_affinity); + + kobject_init(&dev->vqs[i]->kobj, &vq_type); + ret =3D kobject_add(&dev->vqs[i]->kobj, + &dev->dev->kobj, "vq%d", i); + if (ret) { + kfree(dev->vqs[i]); + goto err; + } } =20 return 0; err: while (i--) - kfree(dev->vqs[i]); + kobject_put(&dev->vqs[i]->kobj); kfree(dev->vqs); dev->vqs =3D NULL; - return -ENOMEM; + return ret; } =20 static struct vduse_dev *vduse_dev_create(void) @@ -1686,10 +1787,6 @@ static int vduse_create_dev(struct vduse_dev_config = *config, dev->config =3D config_buf; dev->config_size =3D config->config_size; =20 - ret =3D vduse_dev_init_vqs(dev, config->vq_align, config->vq_num); - if (ret) - goto err_vqs; - ret =3D idr_alloc(&vduse_idr, dev, 1, VDUSE_DEV_MAX, GFP_KERNEL); if (ret < 0) goto err_idr; @@ -1703,14 +1800,19 @@ static int vduse_create_dev(struct vduse_dev_config= *config, ret =3D PTR_ERR(dev->dev); goto err_dev; } + + ret =3D vduse_dev_init_vqs(dev, config->vq_align, config->vq_num); + if (ret) + goto err_vqs; + __module_get(THIS_MODULE); =20 return 0; +err_vqs: + device_destroy(vduse_class, MKDEV(MAJOR(vduse_major), dev->minor)); err_dev: idr_remove(&vduse_idr, dev->minor); err_idr: - vduse_dev_deinit_vqs(dev); -err_vqs: vduse_domain_destroy(dev->domain); err_domain: kfree(dev->name); --=20 2.20.1 From nobody Tue Sep 9 16:08:53 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9721AC64EC7 for ; Tue, 28 Feb 2023 09:43:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231396AbjB1JnV (ORCPT ); Tue, 28 Feb 2023 04:43:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59634 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231191AbjB1JnH (ORCPT ); Tue, 28 Feb 2023 04:43:07 -0500 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 15BE62BF3F for ; Tue, 28 Feb 2023 01:42:38 -0800 (PST) Received: by mail-pj1-x102f.google.com with SMTP id u3-20020a17090a450300b00239db6d7d47so1896806pjg.4 for ; Tue, 28 Feb 2023 01:42:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Fwy2X+T9chZ+5E4FxXuZFD79e/GDnXtVfCKv0Zjao+M=; b=jPaZx6FGDOnufngtQJs1CiF5uSwt3y/7YA6kvxHdmJgkmE/5wMiCxi0w8w5IPdsqju Dr5Gyw2DOq1Ju5u/eEX3lfLM8AmYZeLkCmsh/CLI+NgOMtrJtTvfKACUtsRf7Twr+N8n DZTBXIRcVgseIXNnmxfknJ5hBHTvrugOkiG5yeHDY54a4u3cNLSR8P3m/WoB2NYLb28z jJkvMTTI8x3YIKBFw822javuDEecXShcaINxzGoNWsJ77Cjyhxlu40U0a8g9UfNk9Fc9 X8vNY2K3GL2Yr/JR9HDBRcMwuV7zzSoPMFiNy5bB5bnbjjUSgymJ9xYMKuqrRwGk0dFX SwJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Fwy2X+T9chZ+5E4FxXuZFD79e/GDnXtVfCKv0Zjao+M=; b=UcSFOeTYDbWuBnAzK5MKoYfQ+1WJDIjd8ywqHMRZDeGQl2cpbD604HkzydyLArnst+ 6ym4pE/8hz+gib+vKfCNbcj7+xI4OnHqzCJvFaYsEHWER5kfq6ON5O9JERNKHB/6m+0E 5ctJT4dIqw9DWNtU3+JIYrXmz19fyefXUrpKl9HB4V7LA+rZs9rxGNbsGmzri0hH5gHG kEe4XjeuLXnsXdkpXmPj1zj6KHSxRBwHQfFyRTkYMFvsIKxndx5qnN1DNEuw2vddO4ei X2DRrz6FIW3xhmUHf7zvPZWfFd7t7eVbTzWjG0pB9bgGz7maXiAXxxl1BMfC2wsIP7j4 1cPw== X-Gm-Message-State: AO0yUKXfq3zRiS5lVWASxXBC2ER1H3Ljsg1697S22KgXvRr0K5sxY4HI GzhxxXcdUMR1l7bVVMVwd0Tn X-Google-Smtp-Source: AK7set8ktea2NbLFir6Qw+4FmjTJlMfZQqklYWaMW6TN5etaTWF3fxk2zflJeZCnsfJ2hNn0wPQHKQ== X-Received: by 2002:a17:902:e543:b0:19d:62b:f040 with SMTP id n3-20020a170902e54300b0019d062bf040mr2660180plf.37.1677577357568; Tue, 28 Feb 2023 01:42:37 -0800 (PST) Received: from localhost ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id a23-20020a170902b59700b001993a1fce7bsm6042978pls.196.2023.02.28.01.42.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 01:42:37 -0800 (PST) From: Xie Yongji To: mst@redhat.com, jasowang@redhat.com, tglx@linutronix.de, hch@lst.de Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 08/11] vdpa: Add eventfd for the vdpa callback Date: Tue, 28 Feb 2023 17:41:07 +0800 Message-Id: <20230228094110.37-9-xieyongji@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230228094110.37-1-xieyongji@bytedance.com> References: <20230228094110.37-1-xieyongji@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add eventfd for the vdpa callback so that user can signal it directly instead of running the callback. It will be used for vhost-vdpa case. Signed-off-by: Xie Yongji --- drivers/vhost/vdpa.c | 2 ++ drivers/virtio/virtio_vdpa.c | 1 + include/linux/vdpa.h | 3 +++ 3 files changed, 6 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index dc12dbd5b43b..ae89c0ccc2bb 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -599,9 +599,11 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *= v, unsigned int cmd, if (vq->call_ctx.ctx) { cb.callback =3D vhost_vdpa_virtqueue_cb; cb.private =3D vq; + cb.irq_ctx =3D vq->call_ctx.ctx; } else { cb.callback =3D NULL; cb.private =3D NULL; + cb.irq_ctx =3D NULL; } ops->set_vq_cb(vdpa, idx, &cb); vhost_vdpa_setup_vq_irq(v, idx); diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c index 9eee8afabda8..a5cecafbc2d1 100644 --- a/drivers/virtio/virtio_vdpa.c +++ b/drivers/virtio/virtio_vdpa.c @@ -195,6 +195,7 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsign= ed int index, /* Setup virtqueue callback */ cb.callback =3D callback ? virtio_vdpa_virtqueue_cb : NULL; cb.private =3D info; + cb.irq_ctx =3D NULL; ops->set_vq_cb(vdpa, index, &cb); ops->set_vq_num(vdpa, index, virtqueue_get_vring_size(vq)); =20 diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 10bd22387276..94a7ec49583a 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -13,10 +13,13 @@ * struct vdpa_calllback - vDPA callback definition. * @callback: interrupt callback function * @private: the data passed to the callback function + * @irq_ctx: the eventfd for the callback, user can signal + * it directly instead of running the callback */ struct vdpa_callback { irqreturn_t (*callback)(void *data); void *private; + struct eventfd_ctx *irq_ctx; }; =20 /** --=20 2.20.1 From nobody Tue Sep 9 16:08:53 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC3A6C64ED6 for ; Tue, 28 Feb 2023 09:43:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231391AbjB1Jnc (ORCPT ); Tue, 28 Feb 2023 04:43:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60026 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230511AbjB1JnN (ORCPT ); Tue, 28 Feb 2023 04:43:13 -0500 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA41B2CFC0 for ; Tue, 28 Feb 2023 01:42:41 -0800 (PST) Received: by mail-pj1-x1034.google.com with SMTP id y15-20020a17090aa40f00b00237ad8ee3a0so8976656pjp.2 for ; Tue, 28 Feb 2023 01:42:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ly2xpCqkDa3VUqVVIz1pS+ZMFDIGwE7D0BD8n/b68po=; b=XN3F1rzbLnKWrDsYwKtpbWWtg91kAAGOspI+8XNv22TRjz680IynDquIOKQyxB7Hi+ O3EEWcSJN7wC82vL6JNMzDjk9K6lDOoXR0kbhg3xfXi4ibUOxSKFBdMYjIgh7rUGyi1A TmHiiYrtINUv5fJtPVdXIEa08w0Fgh1Dzsnp5mXRsFFnorRPUtJ5DYUiaHxQfYZsN4Gx Q/8WXAllv/BftZL36sTc8LtzSyVLuOtwJlt/5c/y5knAK/+jFOqu8IRfds+VHbeEbAJr cGSlOZ1Kazfx60IIMbddIVvwn7G+swqkJMn3TJo4m3lkwaR6FRb1HISRau7Z/CxEhOB6 Gi0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ly2xpCqkDa3VUqVVIz1pS+ZMFDIGwE7D0BD8n/b68po=; b=Lq5vTtE83D/cMUSGSKtunGjeCIo+qVAGwLDEVt5aNihRGl5NyEUIN0629ewmZAFZ6x DPwVhkup+rf+CxWofY7UHuRYET3yD7cOOVJrwOPR0m5X7PtLXCdBt/gTZzUJLc86cotl mpDrNlLJxMgu9w2mJQcT4ImNTrLMMsCTdnE5spfkNgghZTd1bGMc3WKUq5THXTy4LpLy We97et1P0Iy6Xl15Qof9Q36bVzRs8R0w0EoXjEv+L3qLNjBxS7M+drZ3CrwcTqO3weua NT1hf5QIXZy2V8Atc7QiYAWpZENQ9ynz1N+Im6mjsKc83O/k6Ggwe8kPm7xvjP1579QT Lq2g== X-Gm-Message-State: AO0yUKWJ1Kl+3CxSmXTWBlai2TCwnFEK2HRwpLoyIJZlJrKYLbMnK6P+ in23zpARjcbIJGD1SKVaksw8Ol0Bm0MnFMQ= X-Google-Smtp-Source: AK7set9ZncnIuyhPuckSPhZTvvuTfcJ63cX5OC6JvKKkEaxvOek2mn1zFkW2AAsa6GC9xz4FKGaBTg== X-Received: by 2002:a17:902:d483:b0:19c:1904:4490 with SMTP id c3-20020a170902d48300b0019c19044490mr2622502plg.63.1677577361211; Tue, 28 Feb 2023 01:42:41 -0800 (PST) Received: from localhost ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id x18-20020a170902ea9200b00198fde9178csm6085550plb.197.2023.02.28.01.42.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 01:42:40 -0800 (PST) From: Xie Yongji To: mst@redhat.com, jasowang@redhat.com, tglx@linutronix.de, hch@lst.de Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 09/11] vduse: Signal interrupt's eventfd directly in vhost-vdpa case Date: Tue, 28 Feb 2023 17:41:08 +0800 Message-Id: <20230228094110.37-10-xieyongji@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230228094110.37-1-xieyongji@bytedance.com> References: <20230228094110.37-1-xieyongji@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Now the vdpa callback will associate an eventfd in vhost-vdpa case. For performance reasons, VDUSE can signal it directly during irq injection. Signed-off-by: Xie Yongji Acked-by: Jason Wang --- drivers/vdpa/vdpa_user/vduse_dev.c | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vd= use_dev.c index 869cc7860d82..56f3c2480c2a 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -461,6 +461,7 @@ static void vduse_dev_reset(struct vduse_dev *dev) spin_lock(&vq->irq_lock); vq->cb.callback =3D NULL; vq->cb.private =3D NULL; + vq->cb.irq_ctx =3D NULL; spin_unlock(&vq->irq_lock); flush_work(&vq->inject); flush_work(&vq->kick); @@ -526,6 +527,7 @@ static void vduse_vdpa_set_vq_cb(struct vdpa_device *vd= pa, u16 idx, spin_lock(&vq->irq_lock); vq->cb.callback =3D cb->callback; vq->cb.private =3D cb->private; + vq->cb.irq_ctx =3D cb->irq_ctx; spin_unlock(&vq->irq_lock); } =20 @@ -1020,6 +1022,20 @@ static void vduse_vq_irq_inject(struct work_struct *= work) spin_unlock_irq(&vq->irq_lock); } =20 +static bool vduse_vq_signal_irqfd(struct vduse_virtqueue *vq) +{ + bool signal =3D false; + + spin_lock_irq(&vq->irq_lock); + if (vq->ready && vq->cb.irq_ctx) { + eventfd_signal(vq->cb.irq_ctx, 1); + signal =3D true; + } + spin_unlock_irq(&vq->irq_lock); + + return signal; +} + static int vduse_dev_queue_irq_work(struct vduse_dev *dev, struct work_struct *irq_work, int irq_effective_cpu) @@ -1322,11 +1338,14 @@ static long vduse_dev_ioctl(struct file *file, unsi= gned int cmd, if (index >=3D dev->vq_num) break; =20 + ret =3D 0; index =3D array_index_nospec(index, dev->vq_num); - - vduse_vq_update_effective_cpu(dev->vqs[index]); - ret =3D vduse_dev_queue_irq_work(dev, &dev->vqs[index]->inject, - dev->vqs[index]->irq_effective_cpu); + if (!vduse_vq_signal_irqfd(dev->vqs[index])) { + vduse_vq_update_effective_cpu(dev->vqs[index]); + ret =3D vduse_dev_queue_irq_work(dev, + &dev->vqs[index]->inject, + dev->vqs[index]->irq_effective_cpu); + } break; } case VDUSE_IOTLB_REG_UMEM: { --=20 2.20.1 From nobody Tue Sep 9 16:08:53 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26043C64ED6 for ; Tue, 28 Feb 2023 09:43:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231387AbjB1JnE (ORCPT ); Tue, 28 Feb 2023 04:43:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231244AbjB1Jm5 (ORCPT ); Tue, 28 Feb 2023 04:42:57 -0500 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E57E2DE48 for ; Tue, 28 Feb 2023 01:42:45 -0800 (PST) Received: by mail-pj1-x102f.google.com with SMTP id cp7-20020a17090afb8700b0023756229427so13073368pjb.1 for ; Tue, 28 Feb 2023 01:42:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JacAVd/i9tXSpaabrsRjnKulqjP+fEUdOpIlt3tr++8=; b=LuAfBPCvlasAcVen2ZhsGCkdyZErAX9iEnoSEYLj2O7XbQmhaibZzLocjTV4/KxuRd lwpeNFGgQSY+V4149nU4yia/uSu6rW6b0MpnHByhBL//kU4gnEXNhvDTbEQtrJbL2Z/X 0FUhl0V6433Ta7+3GLz+FZvtGgFkt5vK2oGGpdBem8tiQ2FvVsq9Fx8ykhyQdDiuROsr cCz/E7gCtmw59PqcLtRxwxoZLevwZJ65iOqGlt0cEvz5wSDTFuOC5jgliWE9nPUykX2E P/W3wQU7vvLtP78mo/omTs6X0veopqGjniWCbocG3M2dah4fZnQESo0ompkwyxYmaRCo yH7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JacAVd/i9tXSpaabrsRjnKulqjP+fEUdOpIlt3tr++8=; b=K43vl4JLuJDUaPzbErSaix8szuTbAKJ5lsIO39wVP592p6ZTYeJa9MsYAII44ORjf8 x4jnMe1d+9Xx3dF5OdFmFdndEPuQgqQ3K2MY4FWSDgkukfHrJ1iEPfsFKCwGuynJ9ljN 7cvApYu1+bMBlwr0slrLCTfaaIeKy/CTMJxNGq2VfG90WRUWHl8L8/FGepnG7g1QJhIj MdykPC94NgrOeznmVcnehepv5HajokuMJcZgA6Sa5UXI/5OtOVHVe1+WE+6/Vz0U6tsY 0oqH33kHhX2hZYty7niTqFwGt+1o5gO9ka+dk1AogqQix4TZdwnenDaU05S5lbJ+YVIi IVug== X-Gm-Message-State: AO0yUKWFn0DJf46fAxgZTgPyCh+IE0Gp+9sortYJNzLbpnMQkv/F1zgS 2hx32QgJeF1WzxDEJpVBKoQL X-Google-Smtp-Source: AK7set/thexCOt6TuD/Pzrnk/gebCdkrp4+xSuFQb8I+DPEs6jtN/Q8s+1UB01rtLTc5LVzlfxrMyA== X-Received: by 2002:a05:6a20:a692:b0:c7:5cb6:2ff7 with SMTP id ba18-20020a056a20a69200b000c75cb62ff7mr2310016pzb.22.1677577364850; Tue, 28 Feb 2023 01:42:44 -0800 (PST) Received: from localhost ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id x25-20020a63b219000000b00502ecc282e2sm5314344pge.5.2023.02.28.01.42.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 01:42:44 -0800 (PST) From: Xie Yongji To: mst@redhat.com, jasowang@redhat.com, tglx@linutronix.de, hch@lst.de Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 10/11] vduse: Delay iova domain creation Date: Tue, 28 Feb 2023 17:41:09 +0800 Message-Id: <20230228094110.37-11-xieyongji@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230228094110.37-1-xieyongji@bytedance.com> References: <20230228094110.37-1-xieyongji@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Delay creating iova domain until the vduse device is registered to vdpa bus. This is a preparation for adding sysfs interface to support specifying bounce buffer size for the iova domain. Signed-off-by: Xie Yongji Acked-by: Jason Wang --- drivers/vdpa/vdpa_user/vduse_dev.c | 75 +++++++++++++++++++++--------- 1 file changed, 53 insertions(+), 22 deletions(-) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vd= use_dev.c index 56f3c2480c2a..1702565efc82 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -113,6 +113,8 @@ struct vduse_dev { u32 vq_align; struct vduse_umem *umem; struct mutex mem_lock; + unsigned int bounce_size; + struct mutex domain_lock; }; =20 struct vduse_dev_msg { @@ -427,7 +429,7 @@ static void vduse_dev_reset(struct vduse_dev *dev) struct vduse_iova_domain *domain =3D dev->domain; =20 /* The coherent mappings are handled in vduse_dev_free_coherent() */ - if (domain->bounce_map) + if (domain && domain->bounce_map) vduse_domain_reset_bounce_map(domain); =20 down_write(&dev->rwsem); @@ -1069,6 +1071,9 @@ static int vduse_dev_dereg_umem(struct vduse_dev *dev, goto unlock; =20 ret =3D -EINVAL; + if (!dev->domain) + goto unlock; + if (dev->umem->iova !=3D iova || size !=3D dev->domain->bounce_size) goto unlock; =20 @@ -1095,7 +1100,7 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev, unsigned long npages, lock_limit; int ret; =20 - if (!dev->domain->bounce_map || + if (!dev->domain || !dev->domain->bounce_map || size !=3D dev->domain->bounce_size || iova !=3D 0 || uaddr & ~PAGE_MASK) return -EINVAL; @@ -1185,7 +1190,6 @@ static long vduse_dev_ioctl(struct file *file, unsign= ed int cmd, struct vduse_iotlb_entry entry; struct vhost_iotlb_map *map; struct vdpa_map_file *map_file; - struct vduse_iova_domain *domain =3D dev->domain; struct file *f =3D NULL; =20 ret =3D -EFAULT; @@ -1196,8 +1200,13 @@ static long vduse_dev_ioctl(struct file *file, unsig= ned int cmd, if (entry.start > entry.last) break; =20 - spin_lock(&domain->iotlb_lock); - map =3D vhost_iotlb_itree_first(domain->iotlb, + mutex_lock(&dev->domain_lock); + if (!dev->domain) { + mutex_unlock(&dev->domain_lock); + break; + } + spin_lock(&dev->domain->iotlb_lock); + map =3D vhost_iotlb_itree_first(dev->domain->iotlb, entry.start, entry.last); if (map) { map_file =3D (struct vdpa_map_file *)map->opaque; @@ -1207,7 +1216,8 @@ static long vduse_dev_ioctl(struct file *file, unsign= ed int cmd, entry.last =3D map->last; entry.perm =3D map->perm; } - spin_unlock(&domain->iotlb_lock); + spin_unlock(&dev->domain->iotlb_lock); + mutex_unlock(&dev->domain_lock); ret =3D -EINVAL; if (!f) break; @@ -1360,8 +1370,10 @@ static long vduse_dev_ioctl(struct file *file, unsig= ned int cmd, sizeof(umem.reserved))) break; =20 + mutex_lock(&dev->domain_lock); ret =3D vduse_dev_reg_umem(dev, umem.iova, umem.uaddr, umem.size); + mutex_unlock(&dev->domain_lock); break; } case VDUSE_IOTLB_DEREG_UMEM: { @@ -1375,15 +1387,15 @@ static long vduse_dev_ioctl(struct file *file, unsi= gned int cmd, if (!is_mem_zero((const char *)umem.reserved, sizeof(umem.reserved))) break; - + mutex_lock(&dev->domain_lock); ret =3D vduse_dev_dereg_umem(dev, umem.iova, umem.size); + mutex_unlock(&dev->domain_lock); break; } case VDUSE_IOTLB_GET_INFO: { struct vduse_iova_info info; struct vhost_iotlb_map *map; - struct vduse_iova_domain *domain =3D dev->domain; =20 ret =3D -EFAULT; if (copy_from_user(&info, argp, sizeof(info))) @@ -1397,18 +1409,24 @@ static long vduse_dev_ioctl(struct file *file, unsi= gned int cmd, sizeof(info.reserved))) break; =20 - spin_lock(&domain->iotlb_lock); - map =3D vhost_iotlb_itree_first(domain->iotlb, + mutex_lock(&dev->domain_lock); + if (!dev->domain) { + mutex_unlock(&dev->domain_lock); + break; + } + spin_lock(&dev->domain->iotlb_lock); + map =3D vhost_iotlb_itree_first(dev->domain->iotlb, info.start, info.last); if (map) { info.start =3D map->start; info.last =3D map->last; info.capability =3D 0; - if (domain->bounce_map && map->start =3D=3D 0 && - map->last =3D=3D domain->bounce_size - 1) + if (dev->domain->bounce_map && map->start =3D=3D 0 && + map->last =3D=3D dev->domain->bounce_size - 1) info.capability |=3D VDUSE_IOVA_CAP_UMEM; } - spin_unlock(&domain->iotlb_lock); + spin_unlock(&dev->domain->iotlb_lock); + mutex_unlock(&dev->domain_lock); if (!map) break; =20 @@ -1431,7 +1449,10 @@ static int vduse_dev_release(struct inode *inode, st= ruct file *file) { struct vduse_dev *dev =3D file->private_data; =20 - vduse_dev_dereg_umem(dev, 0, dev->domain->bounce_size); + mutex_lock(&dev->domain_lock); + if (dev->domain) + vduse_dev_dereg_umem(dev, 0, dev->domain->bounce_size); + mutex_unlock(&dev->domain_lock); spin_lock(&dev->msg_lock); /* Make sure the inflight messages can processed after reconncection */ list_splice_init(&dev->recv_list, &dev->send_list); @@ -1640,6 +1661,7 @@ static struct vduse_dev *vduse_dev_create(void) =20 mutex_init(&dev->lock); mutex_init(&dev->mem_lock); + mutex_init(&dev->domain_lock); spin_lock_init(&dev->msg_lock); INIT_LIST_HEAD(&dev->send_list); INIT_LIST_HEAD(&dev->recv_list); @@ -1689,7 +1711,8 @@ static int vduse_destroy_dev(char *name) idr_remove(&vduse_idr, dev->minor); kvfree(dev->config); vduse_dev_deinit_vqs(dev); - vduse_domain_destroy(dev->domain); + if (dev->domain) + vduse_domain_destroy(dev->domain); kfree(dev->name); vduse_dev_destroy(dev); module_put(THIS_MODULE); @@ -1798,11 +1821,7 @@ static int vduse_create_dev(struct vduse_dev_config = *config, if (!dev->name) goto err_str; =20 - dev->domain =3D vduse_domain_create(VDUSE_IOVA_SIZE - 1, - VDUSE_BOUNCE_SIZE); - if (!dev->domain) - goto err_domain; - + dev->bounce_size =3D VDUSE_BOUNCE_SIZE; dev->config =3D config_buf; dev->config_size =3D config->config_size; =20 @@ -1832,8 +1851,6 @@ static int vduse_create_dev(struct vduse_dev_config *= config, err_dev: idr_remove(&vduse_idr, dev->minor); err_idr: - vduse_domain_destroy(dev->domain); -err_domain: kfree(dev->name); err_str: vduse_dev_destroy(dev); @@ -2000,9 +2017,23 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, = const char *name, if (ret) return ret; =20 + mutex_lock(&dev->domain_lock); + if (!dev->domain) + dev->domain =3D vduse_domain_create(VDUSE_IOVA_SIZE - 1, + dev->bounce_size); + mutex_unlock(&dev->domain_lock); + if (!dev->domain) { + put_device(&dev->vdev->vdpa.dev); + return -ENOMEM; + } + ret =3D _vdpa_register_device(&dev->vdev->vdpa, dev->vq_num); if (ret) { put_device(&dev->vdev->vdpa.dev); + mutex_lock(&dev->domain_lock); + vduse_domain_destroy(dev->domain); + dev->domain =3D NULL; + mutex_unlock(&dev->domain_lock); return ret; } =20 --=20 2.20.1 From nobody Tue Sep 9 16:08:53 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35F56C64ED6 for ; Tue, 28 Feb 2023 09:43:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231398AbjB1Jni (ORCPT ); Tue, 28 Feb 2023 04:43:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231424AbjB1Jn0 (ORCPT ); Tue, 28 Feb 2023 04:43:26 -0500 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19B192DE6C for ; Tue, 28 Feb 2023 01:42:48 -0800 (PST) Received: by mail-pj1-x102c.google.com with SMTP id qa18-20020a17090b4fd200b0023750b675f5so13019588pjb.3 for ; Tue, 28 Feb 2023 01:42:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r5CfE7ygRl6QRb/Gj2o2fNw8BK8ceWBf4KOBuolXf2U=; b=gDiWqIT5V8RTGrcADustI0XUwWLOvjOln/IxsavTPX63G5oaPqruNbvgGr1Fzkf+QY cfB9dXM2m0LYm7FkTRgnK6wclpwo2qN9xTbsfOHEGvdHLOVp5/vvfSPcvd44OsOkWWxD VijXLwBgikzfncnFgaNd1fWcTqPEhpe/IjGvXYFrbr6J/utz5mW3oKaVk//p9hHPbMyV qwoSJ7htVj2F8TnWk7wm8knwdwKVUCzTks1WVjwPqqQGi7YbxwGuIxKOnNlimUE+Uf/m 6FEbUkvHhUVQern56WSQ8hKfTDNPK25iAD6cbWdOKDi/Q2DUc+W0lvEnlQe0GLujrriO 8zCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r5CfE7ygRl6QRb/Gj2o2fNw8BK8ceWBf4KOBuolXf2U=; b=oxv/ZbZrdc6DWkVjliI/5Me0AYCZVFm/tqEQf5JwLUrDlsZClzWt5zT2aG8igf5+3W F1rzhf46sPxOSl1FutDJ2S3U/XaInzfoO6Taa65O6fCtceP1JiZM1krw9PGZz1K+gM/R mvN2wbH8Kz83+3ws7FooTBksoMNdJ7X9KMaA1wNHESlG9wh3Amf7aNzA9lVkVSD/85r/ aBZA4780y/CPJd/LY/KBoOiQWztgMfBYxTTYYchERy//ZiNXNY2B7yHMpqts4uPyo+I+ sdaf6mmlw1XUWMLuKEfgXr2ipZoA51UwoOjCo5vR+Iv4VqMXGgDvbC2ra+vE2syeO7TL x7UA== X-Gm-Message-State: AO0yUKVMfNNFzhMhXKJ9vc6VYZpDGLYjyc7QA3GOC4wBAgcUUHaQ6zlP YLZtGNxsnuKhWSb3IrvUNo6U X-Google-Smtp-Source: AK7set8IYgfcowU9TcF8iddoj24XB5SMXaL5najujz2hKD0+S8OtnwUSu3cKGpZZxMCLcCInfwAZ4A== X-Received: by 2002:a17:902:7e86:b0:19a:973b:b615 with SMTP id z6-20020a1709027e8600b0019a973bb615mr1735905pla.36.1677577368460; Tue, 28 Feb 2023 01:42:48 -0800 (PST) Received: from localhost ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id s6-20020a17090302c600b00196519d8647sm6178324plk.4.2023.02.28.01.42.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 01:42:48 -0800 (PST) From: Xie Yongji To: mst@redhat.com, jasowang@redhat.com, tglx@linutronix.de, hch@lst.de Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 11/11] vduse: Support specifying bounce buffer size via sysfs Date: Tue, 28 Feb 2023 17:41:10 +0800 Message-Id: <20230228094110.37-12-xieyongji@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230228094110.37-1-xieyongji@bytedance.com> References: <20230228094110.37-1-xieyongji@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" As discussed in [1], this adds sysfs interface to support specifying bounce buffer size in virtio-vdpa case. It would be a performance tuning parameter for high throughput workloads. [1] https://lore.kernel.org/netdev/e8f25a35-9d45-69f9-795d-bdbbb90337a3@red= hat.com/ Signed-off-by: Xie Yongji Acked-by: Jason Wang --- drivers/vdpa/vdpa_user/vduse_dev.c | 45 +++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vd= use_dev.c index 1702565efc82..a0f796d20027 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -39,8 +39,11 @@ #define DRV_LICENSE "GPL v2" =20 #define VDUSE_DEV_MAX (1U << MINORBITS) +#define VDUSE_MAX_BOUNCE_SIZE (1024 * 1024 * 1024) +#define VDUSE_MIN_BOUNCE_SIZE (1024 * 1024) #define VDUSE_BOUNCE_SIZE (64 * 1024 * 1024) -#define VDUSE_IOVA_SIZE (128 * 1024 * 1024) +/* 128 MB reserved for virtqueue creation */ +#define VDUSE_IOVA_SIZE (VDUSE_MAX_BOUNCE_SIZE + 128 * 1024 * 1024) #define VDUSE_MSG_DEFAULT_TIMEOUT 30 =20 #define IRQ_UNBOUND -1 @@ -1791,8 +1794,48 @@ static ssize_t msg_timeout_store(struct device *devi= ce, =20 static DEVICE_ATTR_RW(msg_timeout); =20 +static ssize_t bounce_size_show(struct device *device, + struct device_attribute *attr, char *buf) +{ + struct vduse_dev *dev =3D dev_get_drvdata(device); + + return sysfs_emit(buf, "%u\n", dev->bounce_size); +} + +static ssize_t bounce_size_store(struct device *device, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct vduse_dev *dev =3D dev_get_drvdata(device); + unsigned int bounce_size; + int ret; + + ret =3D -EPERM; + mutex_lock(&dev->domain_lock); + if (dev->domain) + goto unlock; + + ret =3D kstrtouint(buf, 10, &bounce_size); + if (ret < 0) + goto unlock; + + ret =3D -EINVAL; + if (bounce_size > VDUSE_MAX_BOUNCE_SIZE || + bounce_size < VDUSE_MIN_BOUNCE_SIZE) + goto unlock; + + dev->bounce_size =3D bounce_size & PAGE_MASK; + ret =3D count; +unlock: + mutex_unlock(&dev->domain_lock); + return ret; +} + +static DEVICE_ATTR_RW(bounce_size); + static struct attribute *vduse_dev_attrs[] =3D { &dev_attr_msg_timeout.attr, + &dev_attr_bounce_size.attr, NULL }; =20 --=20 2.20.1