From nobody Tue Jun 30 13:55:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 857F9C433EF for ; Fri, 14 Jan 2022 21:57:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230006AbiANV5c (ORCPT ); Fri, 14 Jan 2022 16:57:32 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:46675 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230001AbiANV5b (ORCPT ); Fri, 14 Jan 2022 16:57:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642197449; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=L16b4DGvxyPb5nATXEM8DIgZjgXdywlPz0ryK6DeeCM=; b=BkjCaRmIUO636ZPtGKt18sGd8GVMVbAzKVbqf7Np5vcQsamgX8keP+NDi/ZeNgcbzefeTU 83hCa5f+AY0J2v/Az1N3USl2pNTLjfXs/Gmy9oPNOHLhxzj73UBpyNtrSoD81ANesWfVbR /swPwKlYZUyxfZRv7cDK1oARChBzkMg= Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-439-cSLt-ZQ_M_SZfTI4Lc569g-1; Fri, 14 Jan 2022 16:57:28 -0500 X-MC-Unique: cSLt-ZQ_M_SZfTI4Lc569g-1 Received: by mail-ed1-f70.google.com with SMTP id z10-20020a05640235ca00b003f8efab3342so9313144edc.2 for ; Fri, 14 Jan 2022 13:57:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=L16b4DGvxyPb5nATXEM8DIgZjgXdywlPz0ryK6DeeCM=; b=opk8HRqnuhBthmgndoDiuJ0o1KNVkWd++mdXoQFfxGOAomibbBZM5fD3NfJnL15oKg /O2i+CISI3A4rvHe3pr37bHaRjy1WgqTX8655a4DMXGCZPjx5yRWh69whTxsTpvmGxfW gtJeHxcxK2P2jFCeOpFJvS7GXxm6ByNNvOl8HuRmAyn5ToAJPXHrjbaPkasg3BFRvEOU t2GvVWemHjne9Th/cuO9EXWiGL51eug57ojy/vf/F09QBK78fw/9804H8ddExSmY6LGH r8ozoyrPEXhHWOs2g+ugVspuvBWan/OiLMZykrLS3ZfGZtDyNHhlMYbyUDOPgz2hcH7d lHPQ== X-Gm-Message-State: AOAM530iiWrA8qvX4zkDfcWLnK34HMIH5WVc32yRufQtCT2zsJkdRI5J emMAaSK7OyeEEUb8DZ8BpHlUag0syb4ALPWo3lXn1jsuu1g4kT8Inqw8rS83K8BduN2vl7ZvMvj MtMJFKFiRSfjBw1o94IAi3eoLFFhR/tpDggksI4QxsLtLaSnyVeiMNnqElJY4dP/ynLq9jA== X-Received: by 2002:aa7:ce14:: with SMTP id d20mr8882139edv.234.1642197447022; Fri, 14 Jan 2022 13:57:27 -0800 (PST) X-Google-Smtp-Source: ABdhPJzRf2UPE1rQiu4oDrmGOKJ5fch7iHEHP3P6A+VJ1Pk64gSLJydxmVYQaT6KOfiVjxBzt05taw== X-Received: by 2002:aa7:ce14:: with SMTP id d20mr8882118edv.234.1642197446731; Fri, 14 Jan 2022 13:57:26 -0800 (PST) Received: from redhat.com ([2.55.154.210]) by smtp.gmail.com with ESMTPSA id c12sm2869794edx.80.2022.01.14.13.57.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jan 2022 13:57:25 -0800 (PST) Date: Fri, 14 Jan 2022 16:57:24 -0500 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Jason Wang , virtualization@lists.linux-foundation.org Subject: [PATCH v2 1/3] virtio: document virtio_reset_device Message-ID: <20220114215641.239984-2-mst@redhat.com> References: <20220114215641.239984-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20220114215641.239984-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Looks like most callers get driver/device removal wrong. Document what's expected of callers. Signed-off-by: Michael S. Tsirkin --- drivers/virtio/virtio.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/drivers/virtio/virtio.c b/drivers/virtio/virtio.c index 2ed6e2451fd8..631a346a3aa6 100644 --- a/drivers/virtio/virtio.c +++ b/drivers/virtio/virtio.c @@ -201,6 +201,22 @@ static int virtio_finalize_features(struct virtio_devi= ce *dev) return 0; } =20 +/** + * virtio_reset_device - quiesce device for removal + * @dev: the device to reset + * + * Prevents device from sending interrupts and accessing memory. + * + * Generally used for cleanup during driver / device removal. + * + * Once this has been invoked, caller must ensure that + * virtqueue_notify / virtqueue_kick are not in progress. + * + * Note: this guarantees that vq callbacks are not in progress, however ca= ller + * is responsible for preventing access from other contexts, such as a sys= tem + * call/workqueue/bh. Invoking virtio_break_device then flushing any such + * contexts is one way to handle that. + * */ void virtio_reset_device(struct virtio_device *dev) { dev->config->reset(dev); --=20 MST From nobody Tue Jun 30 13:55:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5E0FC433FE for ; Fri, 14 Jan 2022 21:57:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230037AbiANV5f (ORCPT ); Fri, 14 Jan 2022 16:57:35 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:27639 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230001AbiANV5d (ORCPT ); Fri, 14 Jan 2022 16:57:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642197452; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=0wOvJxanjv8dJh5us8ooGzpb/lM+VsWX3ZsU7uSOCqY=; b=BbbzC1QWvLFkLm0przHgaiEssyvFXWbObc/7qcWgMpRtKwL7z8YJYZjaPN0xyx3qjnBuoI ELFfrMGl2Gr0GP/u9oGi9IX/mFnK7r+/ZELRJvPvQMh+rPxWLMe16xRDV+BlF4VfqaHmwB rwX/fAalxBh1gEGJ0pZbDmtAAov2tcQ= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-300-44Pu1O30N3KqjY7B0gl_Fg-1; Fri, 14 Jan 2022 16:57:31 -0500 X-MC-Unique: 44Pu1O30N3KqjY7B0gl_Fg-1 Received: by mail-ed1-f71.google.com with SMTP id l10-20020a056402124a00b0040186fbe40dso717157edw.7 for ; Fri, 14 Jan 2022 13:57:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=0wOvJxanjv8dJh5us8ooGzpb/lM+VsWX3ZsU7uSOCqY=; b=FL0wExhWXZtsFUnn+JBSu+8cZ/9B3BPUGHozkDHU25Z2LFSN4quw19z5Tq1JsVtsTG 2tMtRiKedyxVWAIv81qZntMT6P7BNwtXfgOPj/rmIS9M2BpSITyj9Us+e/asa1FF24A2 vizxVZ2caHUKgBWZ+hap/Th/ZB9vjSsUD75tUgDGYWOgVea9qbY4sAM++nJvc/JONXC9 kOwoW+QiPAu4zKLvYcxWTxAaMFF0EViGnLsDRUrhiQe2TN2qvI/7hftOLPF7AC9bE8AC BAGtoGv9/ewqAt6nXCG8cqD5ZCClH+AeXZm9RXjFMUDjW4MDJH5AujABOSlwH66ag+3P TNgA== X-Gm-Message-State: AOAM531XbjuOY+FdFW7MeaYCGm6v82TwHeF1j68M2wUxyN4gzH0k6dHd k9reVwPqYrCWHrRQsldbcYuWzl9oXTmFs5v7e+a7Q/QAhCXsZjMUC03mjpUedRoK0lAZ9XcIj4t MDTEqnnX6XWyWTMN8pRoy3Ic6hMrw3cgt1zN13uIMWXqh/vBR/h7jUoC/ctzbPxwGzRzB8A== X-Received: by 2002:aa7:db41:: with SMTP id n1mr10643172edt.307.1642197449930; Fri, 14 Jan 2022 13:57:29 -0800 (PST) X-Google-Smtp-Source: ABdhPJxPcMkPHRedToYHc/SYw7TuOGuXM5aNzWoT0JmhpwS9rK3yCFyfEvKXtgc7v8dDKX/rq3pNmQ== X-Received: by 2002:aa7:db41:: with SMTP id n1mr10643153edt.307.1642197449573; Fri, 14 Jan 2022 13:57:29 -0800 (PST) Received: from redhat.com ([2.55.154.210]) by smtp.gmail.com with ESMTPSA id hd16sm2084204ejc.62.2022.01.14.13.57.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jan 2022 13:57:29 -0800 (PST) Date: Fri, 14 Jan 2022 16:57:27 -0500 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Amit Shah , Arnd Bergmann , Greg Kroah-Hartman , virtualization@lists.linux-foundation.org Subject: [PATCH v2 2/3] virtio_console: break out of buf poll on remove Message-ID: <20220114215641.239984-3-mst@redhat.com> References: <20220114215641.239984-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20220114215641.239984-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A common pattern for device reset is currently: vdev->config->reset(vdev); .. cleanup .. reset prevents new interrupts from arriving and waits for interrupt handlers to finish. However if - as is common - the handler queues a work request which is flushed during the cleanup stage, we have code adding buffers / trying to get buffers while device is reset. Not good. This was reproduced by running modprobe virtio_console modprobe -r virtio_console in a loop. Fix this up by calling virtio_break_device + flush before reset. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=3D1786239 Signed-off-by: Michael S. Tsirkin --- drivers/char/virtio_console.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c index 2359889a35a0..e3c430539a17 100644 --- a/drivers/char/virtio_console.c +++ b/drivers/char/virtio_console.c @@ -1957,6 +1957,13 @@ static void virtcons_remove(struct virtio_device *vd= ev) list_del(&portdev->list); spin_unlock_irq(&pdrvdata_lock); =20 + /* Device is going away, exit any polling for buffers */ + virtio_break_device(vdev); + if (use_multiport(portdev)) + flush_work(&portdev->control_work); + else + flush_work(&portdev->config_work); + /* Disable interrupts for vqs */ virtio_reset_device(vdev); /* Finish up work that's lined up */ --=20 MST From nobody Tue Jun 30 13:55:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7B69C43219 for ; Fri, 14 Jan 2022 21:57:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230040AbiANV5i (ORCPT ); Fri, 14 Jan 2022 16:57:38 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:42562 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230035AbiANV5f (ORCPT ); Fri, 14 Jan 2022 16:57:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642197455; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=H65UNYpk1YgIkr6+F7jbPWFp6tYbCp821iY0zXAi0/8=; b=SeSurwWyHP9c/HCum+/op3Sorzkx4GjZUQiUrzlR3sPxULSeNG7VbcVou0l6dqYYd5pVEf /o4hRGh/2CHvqUu54h6AtsMmWvuAukd/bETeu4DqLEGsHETf6kHCHRX50bqwq8UYtXnpsj GxvFaJnOIkFGWd1hmWLTIWc8/l6wdX4= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-411-EVS3s5H1PHOGJ-DqQtEShQ-1; Fri, 14 Jan 2022 16:57:34 -0500 X-MC-Unique: EVS3s5H1PHOGJ-DqQtEShQ-1 Received: by mail-ed1-f69.google.com with SMTP id r8-20020a05640251c800b003f9a52daa3fso9264217edd.22 for ; Fri, 14 Jan 2022 13:57:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=H65UNYpk1YgIkr6+F7jbPWFp6tYbCp821iY0zXAi0/8=; b=lU1+EHiTBg+Z4puZQCi7mqm56ZHUEx+UboIAb3Ep9kxuVbSmB+BdYVdKlqBnDXGW/m +dXiWuRZNenR4I+5I8F2LNQIUw0H1u3nDHhGgT590Ka+Xx1fem1+2QtcacJTKUD5htbZ te6i/KNoiOLYmI8dIBkvwrk47F+mxqNBwptJJehCJlRqprNAFZbo00XvOGpvK0YgNw7S Epfw0dnjksHxi0UeLuCuNUxnzaptpEhwOI3r3tzd1jhuNZ9dLovWy+jiIHwlOHr7ZmY2 tKSavIjbHbCpHd0bkiRk4Sfbpx84wZwiQ8LChT6k0l+Ydf3fGuVnBmEgLcCExuXpvYJe XpHQ== X-Gm-Message-State: AOAM532VmCR4jDXuHe9aqTNuDsaZDlLCzymBuXdDslnEVXlLzdgA0QD1 qF1T2TB4sNNLrpE+v76ZOo855jXWyS83l2RiICqBf98rR7aAY7/0wJQQmGEQp2QXC6INTq939sR HBdX4g96kf89eqRDDqMdQswUV0TNg62CWxmCP/NT9SbXIWJYupHkniHBT7EnfvFlMqrLK7w== X-Received: by 2002:a17:907:1689:: with SMTP id hc9mr8651759ejc.228.1642197452489; Fri, 14 Jan 2022 13:57:32 -0800 (PST) X-Google-Smtp-Source: ABdhPJzsTiLtSAGxhk+CZk+P/jiuyGZo11tboZXauUuIvMgtr2CQW2woyxWUOXD9q5He4VsWBSjrIg== X-Received: by 2002:a17:907:1689:: with SMTP id hc9mr8651746ejc.228.1642197452229; Fri, 14 Jan 2022 13:57:32 -0800 (PST) Received: from redhat.com ([2.55.154.210]) by smtp.gmail.com with ESMTPSA id c15sm2779480edv.77.2022.01.14.13.57.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jan 2022 13:57:31 -0800 (PST) Date: Fri, 14 Jan 2022 16:57:29 -0500 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: David Hildenbrand , Jason Wang , virtualization@lists.linux-foundation.org Subject: [PATCH v2 3/3] virtio_mem: break device on remove Message-ID: <20220114215641.239984-4-mst@redhat.com> References: <20220114215641.239984-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20220114215641.239984-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A common pattern for device reset is currently: vdev->config->reset(vdev); .. cleanup .. reset prevents new interrupts from arriving and waits for interrupt handlers to finish. However if - as is common - the handler queues a work request which is flushed during the cleanup stage, we have code adding buffers / trying to get buffers while device is reset. Not good. This was reproduced by running modprobe virtio_console modprobe -r virtio_console in a loop, and this reasoning seems to apply to virtio mem though I could not reproduce it there. Fix this up by calling virtio_break_device + flush before reset. Signed-off-by: Michael S. Tsirkin --- drivers/virtio/virtio_mem.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 38becd8d578c..33b8a118a3ae 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -2888,6 +2888,8 @@ static void virtio_mem_remove(struct virtio_device *v= dev) virtio_mem_deinit_hotplug(vm); =20 /* reset the device and cleanup the queues */ + virtio_break_device(vdev); + flush_work(&vm->wq); virtio_reset_device(vdev); vdev->config->del_vqs(vdev); =20 --=20 MST