From nobody Sat Apr 11 00:51:21 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62535C28B2B for ; Thu, 18 Aug 2022 13:53:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245362AbiHRNxQ (ORCPT ); Thu, 18 Aug 2022 09:53:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245342AbiHRNw4 (ORCPT ); Thu, 18 Aug 2022 09:52:56 -0400 Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7192F63F3E for ; Thu, 18 Aug 2022 06:52:52 -0700 (PDT) Received: by mail-pf1-x432.google.com with SMTP id f28so1614105pfk.1 for ; Thu, 18 Aug 2022 06:52:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=gOTOSimzRInzIF9tRyR7wLEvlpNDEpWviLzVhGKpvgs=; b=WCbM+4/NKJJDt0kRk6rJwkWEkflQ6N1dGlE9bMBj4GfRk/vaWuvMGhpZ7qok9H59pt FzzGmSb879NTnucQQeVU2UviTYhO+ZKbhhupO0Y89TpwWa/xKqK/siYSPnCnXJ/EdyXS FRZ5ybT3D3C+M+v98Ea92xLRhT4/F1gDLrMEHGfonb7b0fdSnAvKvV1u2iknqlkvja3S dQJeFQMeCcruokhXJ93Sp0E6DKFLZjx7B+Uge8Gbu8E2IU09CxIhP7T9uIHXpmdoRiew NObXzsPdjMN2a9qzmbHj4kyETeF/0e0wQHG6N5jmu5HtGjTpmJS3egxkpn39mclrjO5s 0GIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=gOTOSimzRInzIF9tRyR7wLEvlpNDEpWviLzVhGKpvgs=; b=IxBDouLNndU9IIxtDGBOhaMOGk6IlU462eawakOe7/dIc80QKiHSJn1f1V79l4W/5I umHfst+ycSCs5pQsq6HbzuYq0jLZk+lqhBkU1QEQc1eYqkoC1B3dJuwbFQ4/H7/muiBA 1HoN51vopu+bW7y4DmUurc43fhUNABj3jGPim94ZIF8pIGPx2oFV60lv9iE8P5ZWDpWE ycS4oFMlz37IjBULAk9gvThz1CJwfld0Ezf7lQRxb6mX2xQgT/aKRLcQ5rdGdYE//TgS YF4TQEVrB6VzJ8fHD99V9v2E7GuK7MrxvaFKtwiSEXJa+5W4E0/DhHOX4SS3tBusoYLV rWuQ== X-Gm-Message-State: ACgBeo1ySwvbNxIPshGDIm6CFYaR1LrKyJ0C0L1jSe/HqJlhOMMnOOUO +i/oz6HXmZMx4b74M3K0thhOYgsRVICuLA== X-Google-Smtp-Source: AA6agR6P7ElV4nXCTIaBHrH8KRLneoCsGKc20E9YjNNkU13UpJdntmJ71RwZSeWCFQtAb7zJsZNqMA== X-Received: by 2002:a63:1624:0:b0:41a:9dea:1c80 with SMTP id w36-20020a631624000000b0041a9dea1c80mr2531645pgl.400.1660830772110; Thu, 18 Aug 2022 06:52:52 -0700 (PDT) Received: from C02G705SMD6V.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id k17-20020a170902ce1100b0016db0d877e4sm1385697plg.221.2022.08.18.06.52.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Aug 2022 06:52:51 -0700 (PDT) From: Jia Zhu To: dhowells@redhat.com, xiang@kernel.org, jefflexu@linux.alibaba.com Cc: linux-cachefs@redhat.com, linux-erofs@lists.ozlabs.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, yinxin.x@bytedance.com, Jia Zhu Subject: [RFC PATCH 3/5] cachefiles: resend an open request if the read request's object is closed Date: Thu, 18 Aug 2022 21:52:02 +0800 Message-Id: <20220818135204.49878-4-zhujia.zj@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220818135204.49878-1-zhujia.zj@bytedance.com> References: <20220818135204.49878-1-zhujia.zj@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" When an anonymous fd is closed by user daemon, if there is a new read request for this file comes up, the anonymous fd should be re-opened to handle that read request rather than fail it directly. 1. Introduce reopening state for objects that are closed but have inflight/subsequent read requests. 2. No longer flush READ requests but only CLOSE requests when anonymous fd is closed. 3. Enqueue the reopen work to workqueue, thus user daemon could get rid of daemon_read context and handle that request smoothly. Otherwise, the user daemon will send a reopen request and wait for itself to process the request. Signed-off-by: Jia Zhu Reviewed-by: Xin Yin --- fs/cachefiles/internal.h | 3 ++ fs/cachefiles/ondemand.c | 79 +++++++++++++++++++++++++++------------- 2 files changed, 56 insertions(+), 26 deletions(-) diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h index cdf4ec781933..66bbd4f1d22a 100644 --- a/fs/cachefiles/internal.h +++ b/fs/cachefiles/internal.h @@ -48,9 +48,11 @@ struct cachefiles_volume { enum cachefiles_object_state { CACHEFILES_ONDEMAND_OBJSTATE_close, /* Anonymous fd closed by daemon or i= nitial state */ CACHEFILES_ONDEMAND_OBJSTATE_open, /* Anonymous fd associated with object= is available */ + CACHEFILES_ONDEMAND_OBJSTATE_reopening, /* Object that was closed and is = being reopened. */ }; =20 struct cachefiles_ondemand_info { + struct work_struct work; int ondemand_id; enum cachefiles_object_state state; struct cachefiles_object *object; @@ -341,6 +343,7 @@ cachefiles_ondemand_set_object_##_state(struct cachefil= es_object *object) \ =20 CACHEFILES_OBJECT_STATE_FUNCS(open); CACHEFILES_OBJECT_STATE_FUNCS(close); +CACHEFILES_OBJECT_STATE_FUNCS(reopening); #else #define CACHEFILES_ONDEMAND_OBJINFO(object) NULL =20 diff --git a/fs/cachefiles/ondemand.c b/fs/cachefiles/ondemand.c index f51266554e4d..79ffb19380cd 100644 --- a/fs/cachefiles/ondemand.c +++ b/fs/cachefiles/ondemand.c @@ -18,14 +18,10 @@ static int cachefiles_ondemand_fd_release(struct inode = *inode, info->ondemand_id =3D CACHEFILES_ONDEMAND_ID_CLOSED; cachefiles_ondemand_set_object_close(object); =20 - /* - * Flush all pending READ requests since their completion depends on - * anon_fd. - */ - xas_for_each(&xas, req, ULONG_MAX) { + /* Only flush CACHEFILES_REQ_NEW marked req to avoid race with daemon_rea= d */ + xas_for_each_marked(&xas, req, ULONG_MAX, CACHEFILES_REQ_NEW) { if (req->msg.object_id =3D=3D object_id && - req->msg.opcode =3D=3D CACHEFILES_OP_READ) { - req->error =3D -EIO; + req->msg.opcode =3D=3D CACHEFILES_OP_CLOSE) { complete(&req->done); xas_store(&xas, NULL); } @@ -175,6 +171,7 @@ int cachefiles_ondemand_copen(struct cachefiles_cache *= cache, char *args) trace_cachefiles_ondemand_copen(req->object, id, size); =20 cachefiles_ondemand_set_object_open(req->object); + wake_up_all(&cache->daemon_pollwq); =20 out: complete(&req->done); @@ -234,6 +231,14 @@ static int cachefiles_ondemand_get_fd(struct cachefile= s_req *req) return ret; } =20 +static void ondemand_object_worker(struct work_struct *work) +{ + struct cachefiles_object *object =3D + ((struct cachefiles_ondemand_info *)work)->object; + + cachefiles_ondemand_init_object(object); +} + ssize_t cachefiles_ondemand_daemon_read(struct cachefiles_cache *cache, char __user *_buffer, size_t buflen) { @@ -249,7 +254,27 @@ ssize_t cachefiles_ondemand_daemon_read(struct cachefi= les_cache *cache, * requests from being processed repeatedly. */ xa_lock(&cache->reqs); - req =3D xas_find_marked(&xas, UINT_MAX, CACHEFILES_REQ_NEW); + xas_for_each_marked(&xas, req, UINT_MAX, CACHEFILES_REQ_NEW) { + /* + * Reopen the closed object with associated read request. + * Skip read requests whose related object are reopening. + */ + if (req->msg.opcode =3D=3D CACHEFILES_OP_READ) { + ret =3D cmpxchg(&CACHEFILES_ONDEMAND_OBJINFO(req->object)->state, + CACHEFILES_ONDEMAND_OBJSTATE_close, + CACHEFILES_ONDEMAND_OBJSTATE_reopening); + if (ret =3D=3D CACHEFILES_ONDEMAND_OBJSTATE_close) { + INIT_WORK(&CACHEFILES_ONDEMAND_OBJINFO(req->object)->work, + ondemand_object_worker); + queue_work(fscache_wq, + &CACHEFILES_ONDEMAND_OBJINFO(req->object)->work); + continue; + } else if (ret =3D=3D CACHEFILES_ONDEMAND_OBJSTATE_reopening) { + continue; + } + } + break; + } if (!req) { xa_unlock(&cache->reqs); return 0; @@ -267,14 +292,18 @@ ssize_t cachefiles_ondemand_daemon_read(struct cachef= iles_cache *cache, xa_unlock(&cache->reqs); =20 id =3D xas.xa_index; - msg->msg_id =3D id; =20 if (msg->opcode =3D=3D CACHEFILES_OP_OPEN) { ret =3D cachefiles_ondemand_get_fd(req); - if (ret) + if (ret) { + cachefiles_ondemand_set_object_close(req->object); goto error; + } } =20 + msg->msg_id =3D id; + msg->object_id =3D CACHEFILES_ONDEMAND_OBJINFO(req->object)->ondemand_id; + if (copy_to_user(_buffer, msg, n) !=3D 0) { ret =3D -EFAULT; goto err_put_fd; @@ -307,19 +336,23 @@ static int cachefiles_ondemand_send_req(struct cachef= iles_object *object, void *private) { struct cachefiles_cache *cache =3D object->volume->cache; - struct cachefiles_req *req; + struct cachefiles_req *req =3D NULL; XA_STATE(xas, &cache->reqs, 0); int ret; =20 if (!test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags)) return 0; =20 - if (test_bit(CACHEFILES_DEAD, &cache->flags)) - return -EIO; + if (test_bit(CACHEFILES_DEAD, &cache->flags)) { + ret =3D -EIO; + goto out; + } =20 req =3D kzalloc(sizeof(*req) + data_len, GFP_KERNEL); - if (!req) - return -ENOMEM; + if (!req) { + ret =3D -ENOMEM; + goto out; + } =20 req->object =3D object; init_completion(&req->done); @@ -357,7 +390,7 @@ static int cachefiles_ondemand_send_req(struct cachefil= es_object *object, /* coupled with the barrier in cachefiles_flush_reqs() */ smp_mb(); =20 - if (opcode !=3D CACHEFILES_OP_OPEN && + if (opcode =3D=3D CACHEFILES_OP_CLOSE && !cachefiles_ondemand_object_is_open(object)) { WARN_ON_ONCE(CACHEFILES_ONDEMAND_OBJINFO(object)->ondemand_id =3D=3D 0); xas_unlock(&xas); @@ -382,8 +415,12 @@ static int cachefiles_ondemand_send_req(struct cachefi= les_object *object, wake_up_all(&cache->daemon_pollwq); wait_for_completion(&req->done); ret =3D req->error; + kfree(req); + return ret; out: kfree(req); + if (opcode =3D=3D CACHEFILES_OP_OPEN) + cachefiles_ondemand_set_object_close(req->object); return ret; } =20 @@ -435,7 +472,6 @@ static int cachefiles_ondemand_init_close_req(struct ca= chefiles_req *req, if (!cachefiles_ondemand_object_is_open(object)) return -ENOENT; =20 - req->msg.object_id =3D CACHEFILES_ONDEMAND_OBJINFO(object)->ondemand_id; trace_cachefiles_ondemand_close(object, &req->msg); return 0; } @@ -451,16 +487,7 @@ static int cachefiles_ondemand_init_read_req(struct ca= chefiles_req *req, struct cachefiles_object *object =3D req->object; struct cachefiles_read *load =3D (void *)req->msg.data; struct cachefiles_read_ctx *read_ctx =3D private; - int object_id =3D CACHEFILES_ONDEMAND_OBJINFO(object)->ondemand_id; =20 - /* Stop enqueuing requests when daemon has closed anon_fd. */ - if (!cachefiles_ondemand_object_is_open(object)) { - WARN_ON_ONCE(object_id =3D=3D 0); - pr_info_once("READ: anonymous fd closed prematurely.\n"); - return -EIO; - } - - req->msg.object_id =3D object_id; load->off =3D read_ctx->off; load->len =3D read_ctx->len; trace_cachefiles_ondemand_read(object, &req->msg, load); --=20 2.20.1