From nobody Thu Apr 9 11:49:32 2026 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8DB5A410D1A for ; Mon, 9 Mar 2026 19:31:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773084677; cv=none; b=GWNoe31/LSwIez1Fy1gfe1gp1hHXseOkCVl7Pil0S74dvtaRvnLzLLF05I6k4znN/xPLNnlwJpNP5z9QwSMIiOAMfpI4lgGkuA3FwsG3k1xWFodW09HXdT1bmH0zbH2hHTJk5IZidONe3S+DS/Rrzr2uKlCJhQy1+PEsyCZvXh8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773084677; c=relaxed/simple; bh=mUKw0kl0f4KxNAzKhr8UyqO0R7jCXXC379eGBhra2Q8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=psQY8NaO5/VKAWL2RQ0Gt+AaLaP3knZML5hu+VSXbLSlFzack+jynMMgVMti/2HPuH3D54XlAQAkySfB4uvW5HwY9+s2BD/bgWwVOTS2GxgtMhp6CkiZ7iS+mS/9GEEgNKnLEPcdqL3TAyISg38QfyV3PxHbQE/Qcvp5vFYR4sw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=zYVzPMLx; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="zYVzPMLx" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-82988b04c5cso11742191b3a.3 for ; Mon, 09 Mar 2026 12:31:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1773084676; x=1773689476; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=1G09RtyABu4hfI9DMnNr2bllkac+dmhEIEcvARTdDJk=; b=zYVzPMLxyfgWNWIGVx+FhZzYjdW6KlRPal5FWU5nVKX9Ekz020jycy/N8L4pWnNwDv tIr8HIwIpIO4PDWBU9CESIhnABTGwOpuVAKEGjcBiO1rlqIOHn0tauHjBU8TShrppBV0 6Vj/daXm88s5hP4OJkotY7xxO6TH75dDN4Xl+7R/pFnCx1STvW+t1kKUI5DAtNpFfVDZ pNZ5xKZgudojvAYZWIuK9bNn4utQO6v/5Uz0saTNC5dwlTzdhaMS+RzudcSybAlMEgPN wzMy0JC3b3V4l7eQqF4rxbCJSQAGiQxRjtmUPxt1h1+U1RIu0+TxKTL/9q0FUB8tHUz7 awRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1773084676; x=1773689476; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1G09RtyABu4hfI9DMnNr2bllkac+dmhEIEcvARTdDJk=; b=MUUL/NISRCipl0gTTpvOe81DiGiP+Sa1hQkz8F8zN1KoeI9zyg+8qzwua5KGchbbPD V32w8m4eZ8lVXbSPUASntERWZNxP63v1/VVg1AA8Dxk6hHfvlcqU0OmKtWnNuQB9uEb9 ibJtr7fjNTqzZHRCxmLsIBURtFsgASZizGXjpAcqdUpiHzTXwRCpOx8vPGaXH1SVYaOb 8m42a137gIdj94uSqp7pxXMhLXCKX+v9C/gbWD3ST+WfbYx7TJYNCoqeIg3qaqNsag9d 7C/k2pyGmTxwZlqXZv5BkODmumnS7XAmXJtSEdBv3PrRFUt/db3DaC4jjmwMU0gjGrmq Iwow== X-Forwarded-Encrypted: i=1; AJvYcCWce310XfCwVjHXt4UKs5T+U0vm2j8fDO9BwSn3x7nkBFROb0Rj7EDRjnUvY54A8c8aGmOBWYnDFrAi4F8=@vger.kernel.org X-Gm-Message-State: AOJu0YyI4yhUFVThSSwrO9j+2QeoreErm7UlxSjt1lAdgLz6RN5PmXo5 K4yb2tpB74nxZkYxbanFF25NzVZ1xb1PbvECMckEa5fQWRMg+NfcD8aY/H8KfXCJT3V4Zr5/Tj8 pVmgjKQ== X-Received: from pfog16.prod.google.com ([2002:aa7:8750:0:b0:829:707f:6f3e]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:1827:b0:827:444a:58e0 with SMTP id d2e1a72fcca58-829a2ea9fd8mr11118741b3a.32.1773084675798; Mon, 09 Mar 2026 12:31:15 -0700 (PDT) Reply-To: Sean Christopherson Date: Mon, 9 Mar 2026 12:30:58 -0700 In-Reply-To: <20260309193059.2244645-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260309193059.2244645-1-seanjc@google.com> X-Mailer: git-send-email 2.53.0.473.g4a7958ca14-goog Message-ID: <20260309193059.2244645-3-seanjc@google.com> Subject: [RFC PATCH 2/3] srcu: Add and export call_srcu_expedited() to avoid transferring grace periods From: Sean Christopherson To: Lai Jiangshan , "Paul E. McKenney" , Josh Triplett , Paolo Bonzini Cc: rcu@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Sean Christopherson , Nikita Kalyazin , Keir Fraser Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add and export an expedited version of call_srcu() so that users of synchronize_srcu_expedited() can avoid "transferring" non-expedited grace periods. In response to userspace changes to guest devices and memory, KVM uses call_srcu() when freeing an object to avoid having to wait for readers to go away, but then often emits a synchronize_srcu_expedited() in a largely unrelated path shortly thereafter (on the same SRCU object). Due to differences in how VMMs manage guest devices, and in the architecture being emulated by userspace, some updates trigger call_srcu() with concurrent readers (i.e. while the VM is active), while others occur without readers, e.g. when configuring devices during a pre-boot setup. For the later case (no concurrent readers), using the vanilla call_srcu() is problematic, as it can kick off a normal grace period (totally fine for freeing the object) and effectively transfer the non-expedited grace period to the upcoming synchronize_srcu_expedited(). For micro-VM use cases with CONFIG_HZ=3D100 kernels, the resulting ~20ms delay on the would-be-expedited sync can increase the boot time of the VM by 15% or more. Link: https://lore.kernel.org/all/a84ddba8-12da-489a-9dd1-ccdf7451a1ba@amaz= on.com Signed-off-by: Sean Christopherson --- include/linux/srcutiny.h | 6 ++++++ include/linux/srcutree.h | 2 ++ kernel/rcu/srcutree.c | 7 +++++++ 3 files changed, 15 insertions(+) diff --git a/include/linux/srcutiny.h b/include/linux/srcutiny.h index 4976536e8b28..e2fc8c138e6a 100644 --- a/include/linux/srcutiny.h +++ b/include/linux/srcutiny.h @@ -130,6 +130,12 @@ static inline void srcu_barrier(struct srcu_struct *ss= p) synchronize_srcu(ssp); } =20 +static inline void call_srcu_expedited(struct srcu_struct *ssp, struct rcu= _head *rhp, + rcu_callback_t func) +{ + call_srcu(ssp, rhp, func); +} + static inline void srcu_expedite_current(struct srcu_struct *ssp) { } #define srcu_check_read_flavor(ssp, read_flavor) do { } while (0) =20 diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h index 958cb7ef41cb..ed3cbbe7f5ce 100644 --- a/include/linux/srcutree.h +++ b/include/linux/srcutree.h @@ -234,6 +234,8 @@ struct srcu_struct { __DEFINE_SRCU(name, SRCU_READ_FLAVOR_FAST_UPDOWN, static) =20 int __srcu_read_lock(struct srcu_struct *ssp) __acquires_shared(ssp); +void call_srcu_expedited(struct srcu_struct *ssp, struct rcu_head *head, + rcu_callback_t func); void synchronize_srcu_expedited(struct srcu_struct *ssp); void srcu_barrier(struct srcu_struct *ssp); void srcu_expedite_current(struct srcu_struct *ssp); diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index aef8e91ad33e..77076d2a1c57 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -1495,6 +1495,13 @@ void call_srcu(struct srcu_struct *ssp, struct rcu_h= ead *rhp, } EXPORT_SYMBOL_GPL(call_srcu); =20 +void call_srcu_expedited(struct srcu_struct *ssp, struct rcu_head *rhp, + rcu_callback_t func) +{ + __call_srcu(ssp, rhp, func, rcu_gp_is_normal()); +} +EXPORT_SYMBOL_GPL(call_srcu_expedited); + /* * Helper function for synchronize_srcu() and synchronize_srcu_expedited(). */ --=20 2.53.0.473.g4a7958ca14-goog