From nobody Sun Feb 8 09:26:54 2026 Received: from mail-yb1-f176.google.com (mail-yb1-f176.google.com [209.85.219.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 497D720011F for ; Mon, 7 Apr 2025 15:39:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744040342; cv=none; b=iL/ZJETBjqqOaCHO0GqV0hWyBw7vmFllDEE9sSlusq0Y3X8lkyRfrkMzPz+qtbNyXojjtJjxQqJc3DyXv8e1+Dz9RMba9T3jRaHc3joScG4oBcaUXK0/EYMQRIwFRHn0oS7fKylMRQf9GircjxdeENHEjyqZ2wIGD8IwC5Nn3s0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744040342; c=relaxed/simple; bh=PHv3c3vhIJln97xnwqLJo53XteMXlVsowkn247j2FXY=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PcWeIvgzHyo1NMLUkZcdtBfOKRg5h6MN1WEbcec75zUUPsLLPe1IhOtka2oAOGwlOLxLXme/nEP8sY+QERn5glU7wUPI1MSIZT3JW0Mt7VSAHYZ2ZuKCSQhGWb2RtAlHLKE2JkSyHiKwn9MDf7LjV4PKBTmgDGeWpiUZl2R4TGM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SrQxGP+p; arc=none smtp.client-ip=209.85.219.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SrQxGP+p" Received: by mail-yb1-f176.google.com with SMTP id 3f1490d57ef6-e53a91756e5so4163066276.1 for ; Mon, 07 Apr 2025 08:39:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1744040340; x=1744645140; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=pKcvgfvOu/nIKfn1Z/lLcoSsPpuWPl28y0Pjgymqn9Q=; b=SrQxGP+p2rowOgn0FxsmSI5ArdU5fD6DvxLpcgZKGhZ/37jqTyEoLZ1Zp1vFk0U5Je KWcvLZjZdQkVPn0X58BW3w7Kl9m4RNny8p2T4qUWvxf/Jw935HsxnqjPLnirYldb21o8 bO/uXRkSdF1nbxE17usIwtvDEX0LxdG9OkG18Wt3XLP0k2C2t2LHAFcAC7AniXRc5/P7 l3OOqWy9o2KbNpdRrcDtR6cyZiKFc2q+HNJThnR9i2mrw/qvNa32xYwYnLx8baQlVpfD voO1tAAc3ZjcKbQ3lNWuw6fFfJtNrizI22ZsV/ZU5lerYZE4Z1+LO3TupGueYF0oYZem uwkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744040340; x=1744645140; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pKcvgfvOu/nIKfn1Z/lLcoSsPpuWPl28y0Pjgymqn9Q=; b=gx4q2Thw1nh/SbGG1fsaEZf0IM0uBG/0nXlvxTifn4cE5+84zsVcqCpD0x6ZsO1A1b M0IbaT1CC9MqHPD/tPRuNYmQ5+X4vFiF8R61NZYPoKdFXY2sw1FmqTE/cjdh+3yPy8xF m04LxqLzYroOZpyIWxJFg6WDh1qBCwZZgsWFb5tWRdrLM533lVuosPpuZ+9kmuCiuqCb +7yV7opaOFKE0nBmZD7GjT7y60xKCCSj974tLotVGaqEYYmCo5Hne7KyIVERaPeLUQfR wBfgr+Pd9dmNg9ypqmpnF/A8BVD3RZ+lhlUXG88G0YGbgvWhcC3lBHgQzTqC8/IUGqmb Vffw== X-Forwarded-Encrypted: i=1; AJvYcCVRFXwGiso4WWBWlA8HoebDAUI7p+CriFKEJtgXsSbUgUgkHU0lNW4McZoZ9eRfmFMVRDe50FxYMlkdX6s=@vger.kernel.org X-Gm-Message-State: AOJu0YyVgYXkbwyizmrPM9HsbAMR6+LFGY3xXhTJAnFojfbdV/j5SMIC hFHhKLS9z2dUyr4XdZkiASxN7sJFnRbpJQHqoOD5HpNeTulXpHzO X-Gm-Gg: ASbGncufth3NV57QLultZh/NYusITOCh2gwozs0fIXmbQoWjPbY79JnZOp/pV6SVrTI e8675ijWKOUJo7cfi/nRjJqDU4xoi3zeF2POTi4ZqoMpwpdJ1VPmCo/4I40HTVgA6I609rsUECk Cvq0oly8CiCYfYmaHqLy5pA1V62xuu0Rm6V+AYSYbH8WKxS2gU591em9A7eU9fBbb4hzoSqKniY MQ769Tl5g0tWwc6t7t4/EWrqu0gYHnZkg0YJQuiQZFKRxk35uFIk14dsY4VuqxLE7YKq2+R/MBY fpJOn41rcGXFF22VFhSakusCE1XkPi6ySV9pjaIbDdh0qQ+QjxSEtDYfzdCkp9gBbXqE22Wb3mu 6IjWD X-Google-Smtp-Source: AGHT+IFBngPkA24tKRFHHGc3DSInjf3c16iyEDRDgmoVfT2mUXMgOabZJFmPqbdFAsDYmmRrYdQ+KQ== X-Received: by 2002:a05:6902:2782:b0:e60:9f50:5d0 with SMTP id 3f1490d57ef6-e6e1c2e4279mr21290835276.29.1744040340121; Mon, 07 Apr 2025 08:39:00 -0700 (PDT) Received: from localhost (c-73-224-175-84.hsd1.fl.comcast.net. [73.224.175.84]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e6e0c8b9b44sm2433580276.1.2025.04.07.08.38.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Apr 2025 08:38:59 -0700 (PDT) From: Yury Norov To: Tony Luck , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , "Yury Norov [NVIDIA]" , Rasmus Villemoes , x86@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/4] cpumask: relax cpumask_any_but() Date: Mon, 7 Apr 2025 11:38:52 -0400 Message-ID: <20250407153856.133093-2-yury.norov@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250407153856.133093-1-yury.norov@gmail.com> References: <20250407153856.133093-1-yury.norov@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Yury Norov [NVIDIA] Similarly to other cpumask search functions, accept -1, and consider it as 'any cpu' hint. This helps users to avoid coding special cases. Signed-off-by: Yury Norov [NVIDIA] Reviewed-by: James Morse Tested-by: James Morse --- include/linux/cpumask.h | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h index beff4d26e605..0f816092c891 100644 --- a/include/linux/cpumask.h +++ b/include/linux/cpumask.h @@ -413,6 +413,7 @@ unsigned int cpumask_next_wrap(int n, const struct cpum= ask *src) * @cpu: the cpu to ignore. * * Often used to find any cpu but smp_processor_id() in a mask. + * If @cpu =3D=3D -1, the function is equivalent to cpumask_any(). * Return: >=3D nr_cpu_ids if no cpus set. */ static __always_inline @@ -420,7 +421,10 @@ unsigned int cpumask_any_but(const struct cpumask *mas= k, unsigned int cpu) { unsigned int i; =20 - cpumask_check(cpu); + /* -1 is a legal arg here. */ + if (cpu !=3D -1) + cpumask_check(cpu); + for_each_cpu(i, mask) if (i !=3D cpu) break; @@ -433,6 +437,7 @@ unsigned int cpumask_any_but(const struct cpumask *mask= , unsigned int cpu) * @mask2: the second input cpumask * @cpu: the cpu to ignore * + * If @cpu =3D=3D -1, the function is equivalent to cpumask_any_and(). * Returns >=3D nr_cpu_ids if no cpus set. */ static __always_inline @@ -442,7 +447,10 @@ unsigned int cpumask_any_and_but(const struct cpumask = *mask1, { unsigned int i; =20 - cpumask_check(cpu); + /* -1 is a legal arg here. */ + if (cpu !=3D -1) + cpumask_check(cpu); + i =3D cpumask_first_and(mask1, mask2); if (i !=3D cpu) return i; --=20 2.43.0 From nobody Sun Feb 8 09:26:54 2026 Received: from mail-yw1-f171.google.com (mail-yw1-f171.google.com [209.85.128.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 70A68201024 for ; Mon, 7 Apr 2025 15:39:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744040344; cv=none; b=IiUsGOYA1ehNiM7gRpydvtTzLeAaQduBYPX0UI4wxt2qsNNnSEpAHJbQNKtJ0P6vNdi9q5VI0iU0TTISin6RiMLirgqxy2vt1joZfR70+TefuvYxXxJaDOfgiupsokVf2t/7SiBtX/Nx/tWZZouy8x0L9Uy+lnDoLpVjSdnUyrQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744040344; c=relaxed/simple; bh=a8JHMzhzYG758Tnl/DZgrwwRUDHVUlCO333Gr7EpXFg=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PdbSGU7S52D9Jt7IRrHrhwVUTUxKd44QAVh69wpSD1oY5rb73G3wv7ZBpWIBpFaVXuI5zEzY1gkpFj9FjoeZDk5Rv3KjKDxylNXQ6dVjd9a5Q26XoLZkkK0yY/2e82MR9x/YQN6VQR8S5Sq5TZS3KkynLGwct0V/D6+SKjR5vw8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=VL0brsXa; arc=none smtp.client-ip=209.85.128.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="VL0brsXa" Received: by mail-yw1-f171.google.com with SMTP id 00721157ae682-700b17551cdso43111287b3.0 for ; Mon, 07 Apr 2025 08:39:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1744040341; x=1744645141; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=YobkztMKnvk249lFXQRoFaShwGuxuoRz0jtkrT6QWQE=; b=VL0brsXafL7oNTiiXGC/yhPUcDJoNZLJpZ/6yaSWwPQCPThV2rD27+F5y4i4xm1ifA rW3uxGlQBLYOjeT3H8w6A8cfEaV14ChGj99oL/kH4Lf0JOVfbV6nms1SK+TLtjfXY7w4 FoZ7OuE96//9JpYNrxNFqnRHx5mtkOZMCmaQwZqzvGgnCNduIFt3WYHh1OzKrZvScIGK Z1ZUMNjURZ+J6pjQEDz8TqE1R0iWX3H4az5gKfioAGZEGgeaG5fzhPG3SdTAl5+jzaQG dVUPiNBY09UVh0yRJdsigDqBRW8Cgf+RJ2uYRR+pcAvxMsvUowqwaAgxOnhZpr9bicqO hJFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744040341; x=1744645141; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YobkztMKnvk249lFXQRoFaShwGuxuoRz0jtkrT6QWQE=; b=ETbPc3SKq6Gxzo57J68X582PvZgjziBoe8NcpNcut1nQAVXLm+eyOmLsPg5NQ1MZRX yNGPCvEGPucFOr4ePiElbcCPf6XO1LWdoLlFhz1vzjZ84f6ZeI1tpDFez0HDoFTrQ9du fCeKTNCM1GHsx61qCR1z6Bfy8w4u6s6ZezNW4cXi2NL/wK2bilTQiS66gnhvxe8FS+B6 92Nn6El8Gsn9zRJkE6bwjxKdQS/1ZqNeUhq/OiZVGDOVMJ7uMEFFgAIg41cZ8jmpYX4u cBKtFQcH4SGnbF91sH4bPiclCakXM0c5Le1nMj5Pk9mO3N0LtXYvUnWczq6qeIxl4d6w nNsA== X-Forwarded-Encrypted: i=1; AJvYcCVIdG+BGcVk5VZe9qbaa7bV8i6a6ej6kMsqAilmFVc/l8SoDsbjiyhagX9s/0JtesOuAIg2FCRS30uxYOU=@vger.kernel.org X-Gm-Message-State: AOJu0YwxUSI0FchgsF5c903mJIpmpp1xUawz7NKJy2vApbIpwRpCyfbp G9+jfLoKxl9JWXIaiPRFFfskVSANurXNePgGNjahTboLCV/nwpH2HSeaaBlm X-Gm-Gg: ASbGncvPioHFHXBi5EOiB2xJzigsd0kL3DmJaAxuQF0P2EcEP6S5+zxEvT9fyfd5uDx rgE+JaLzp+eJl0xpYiSbfiqD8DZ+2Lt48HncUQIKbovoKQAK/aN5WWNT3CafV/5a94vZgxm7Itk PgrZdEB61/cQ7oqAAn5BHGUiHnuQu3AW1vIk84w6G1Lkakhhy/RZj5dWa5dbRAui/F3O0Gyj8Zg QLsoiq0mvxmIUypeOidED65Hg+ETdkKh/ZhAZqWSW4Jv+IFJ+e2+6C5dZ6/i8T7EPTjv8EF6EEI UHuz1d00jfHUgJ1uoLvBAdrOfANKBor9ix9lbhtee9PXhZmHR1uFCDcJKJiVXlzzouKBIFPxj5/ wLq/D X-Google-Smtp-Source: AGHT+IHmz5WoV9i7L7rT8MD50liCBf6aVMxiEzKBiZURGiemYCV9je1fD7No+Y7mq6erBRWmPhrplQ== X-Received: by 2002:a05:690c:f8b:b0:703:c3b4:45d7 with SMTP id 00721157ae682-703e16337d5mr227561357b3.28.1744040341328; Mon, 07 Apr 2025 08:39:01 -0700 (PDT) Received: from localhost (c-73-224-175-84.hsd1.fl.comcast.net. [73.224.175.84]) by smtp.gmail.com with ESMTPSA id 00721157ae682-703d1f70626sm26143867b3.69.2025.04.07.08.39.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Apr 2025 08:39:00 -0700 (PDT) From: Yury Norov To: Tony Luck , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , "Yury Norov [NVIDIA]" , Rasmus Villemoes , x86@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/4] find: add find_first_andnot_bit() Date: Mon, 7 Apr 2025 11:38:53 -0400 Message-ID: <20250407153856.133093-3-yury.norov@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250407153856.133093-1-yury.norov@gmail.com> References: <20250407153856.133093-1-yury.norov@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Yury Norov [NVIDIA] The function helps to implement cpumask_andnot() APIs. Signed-off-by: Yury Norov [NVIDIA] Reviewed-by: James Morse Tested-by: James Morse --- include/linux/find.h | 25 +++++++++++++++++++++++++ lib/find_bit.c | 11 +++++++++++ 2 files changed, 36 insertions(+) diff --git a/include/linux/find.h b/include/linux/find.h index 68685714bc18..d1578cfb667c 100644 --- a/include/linux/find.h +++ b/include/linux/find.h @@ -29,6 +29,8 @@ unsigned long __find_nth_and_andnot_bit(const unsigned lo= ng *addr1, const unsign unsigned long n); extern unsigned long _find_first_and_bit(const unsigned long *addr1, const unsigned long *addr2, unsigned long size); +unsigned long _find_first_andnot_bit(const unsigned long *addr1, const uns= igned long *addr2, + unsigned long size); unsigned long _find_first_and_and_bit(const unsigned long *addr1, const un= signed long *addr2, const unsigned long *addr3, unsigned long size); extern unsigned long _find_first_zero_bit(const unsigned long *addr, unsig= ned long size); @@ -347,6 +349,29 @@ unsigned long find_first_and_bit(const unsigned long *= addr1, } #endif =20 +/** + * find_first_andnot_bit - find the first bit set in 1st memory region and= unset in 2nd + * @addr1: The first address to base the search on + * @addr2: The second address to base the search on + * @size: The bitmap size in bits + * + * Returns the bit number for the matched bit + * If no bits are set, returns >=3D @size. + */ +static __always_inline +unsigned long find_first_andnot_bit(const unsigned long *addr1, + const unsigned long *addr2, + unsigned long size) +{ + if (small_const_nbits(size)) { + unsigned long val =3D *addr1 & (~*addr2) & GENMASK(size - 1, 0); + + return val ? __ffs(val) : size; + } + + return _find_first_andnot_bit(addr1, addr2, size); +} + /** * find_first_and_and_bit - find the first set bit in 3 memory regions * @addr1: The first address to base the search on diff --git a/lib/find_bit.c b/lib/find_bit.c index 0836bb3d76c5..06b6342aa3ae 100644 --- a/lib/find_bit.c +++ b/lib/find_bit.c @@ -116,6 +116,17 @@ unsigned long _find_first_and_bit(const unsigned long = *addr1, EXPORT_SYMBOL(_find_first_and_bit); #endif =20 +/* + * Find the first bit set in 1st memory region and unset in 2nd. + */ +unsigned long _find_first_andnot_bit(const unsigned long *addr1, + const unsigned long *addr2, + unsigned long size) +{ + return FIND_FIRST_BIT(addr1[idx] & ~addr2[idx], /* nop */, size); +} +EXPORT_SYMBOL(_find_first_andnot_bit); + /* * Find the first set bit in three memory regions. */ --=20 2.43.0 From nobody Sun Feb 8 09:26:54 2026 Received: from mail-yw1-f179.google.com (mail-yw1-f179.google.com [209.85.128.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C550D201258 for ; Mon, 7 Apr 2025 15:39:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744040345; cv=none; b=T6pIwj3xrVPTqOdwDZ2myNqFrnfx4uDA74k57pGlxWaB8CK4COiTDmb1dBNjjXMTNf1b1We2iU/PIRZxx6V6bekUN89zoll9ydvhfWV67AukgVkSuseCTaYAb3ceAcLTvmXxipmWAAXOxprgJLrquM60ltEm+93vSEPTwb6UHOA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744040345; c=relaxed/simple; bh=/j3M1uu9C+F77GujjQypyAKgz544kzX0sqIEJgki2CU=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jQlgBTPh7s2ipJzBZjh+uI0jRqmOXMhkQ+KQ0KLqZ4OdHnw5RlH7k2xqlaxIXONGiqUhbvCx8eI5d1dEJQBNVnaxsCKkwvrjZlfgG1pfAzx04qd7Z5TTqb9zRpEkK0HFOLk9zQ3Bp5l5oX1PJ/f/FfAoTimgSmxtwjT9/slzfAw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=HTwqHIc5; arc=none smtp.client-ip=209.85.128.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HTwqHIc5" Received: by mail-yw1-f179.google.com with SMTP id 00721157ae682-6feab7c5f96so42556277b3.3 for ; Mon, 07 Apr 2025 08:39:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1744040343; x=1744645143; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=NUg9GEd9vWrZ/rj86A40MHUOre17ooeFUfErMtu4AI4=; b=HTwqHIc5xPY5Lh5zVMZJoQMUzRwgyyxmB4Dxk+IMwKoE98g+L9TOUEjo7mTj6grEOw 1PYYjZVWavIphxNL7uevzVsbwRVIVZYX6X34ACBgtd2p2I+yf7KAKwh0aYUhDz5IuvfV a8BVhb69XMAeGHTf/w0L3aFl4R1tu0UvENml0IZdxjA0Rs3hmk+ya/I7yS5Tc8RNLpxn Q2eQBOd+a5AtoXA1m3DELzdNvCWnshqsiPrUH0MLckhM8468fp9hscw0MHatYsssBhVC rL6YpnP3KZvHxDDU/PIjUuLz9k6muDZg7G8zNJBcqjbQaNh9KZYfzrguh0DqC519ftVL pHxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744040343; x=1744645143; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NUg9GEd9vWrZ/rj86A40MHUOre17ooeFUfErMtu4AI4=; b=E16ckdm7LIXI3QskcJHFU+XqKRdkmJgEUcuopNiJwUb7cDmcW6K6YIAUh9zMsJMUZs djC5q+QZyti0t7p1ary/totv4PyMYHhpb7oBLq+tGJ5J9IaGt0nBC6cvrD8CtHouKGAY +aLphyAdLuc9BLdZ8zk0LkyEjM8RwboDHJMP5dlh6aLEyT3lS5WKWBVCIbDcckMbD+4h z0rLCm+ZdsNwmzCE51drC7MgNNqCKdIDxAe6hhc0in0hP1vx4+L93F0TY+bahZYQJUOK yv2lMrcPGBWFkBfYIpynA7dqx52H/qO32L6pjzFoslT2NRwpzdDH63FlfA8LfE3+hoZD T6gw== X-Forwarded-Encrypted: i=1; AJvYcCXKU71qll+Silh9526zQ8HBcU8VHFNo8YwcQmVx7kkT7ENnEi7ra2l5yY2NS34P6q/mHNg3HthF46UxrK4=@vger.kernel.org X-Gm-Message-State: AOJu0YwFUDt2s3yTBPeB0TcTw3hT+xEUpKqlmfE3hSb5K8xtfLVZo8QH vjqRP8VWvhxPwJcSEuFuS3DUF2RyHpqqc/IB+6fCBBrNYYAN/Wnk X-Gm-Gg: ASbGncsPLJ3ecaC1IjH0ifP7MujK1a/hUJu24XvgBfKbjt9ByJgyKrHsbB2PDr+YOJo zul4s+bMzV5amB4c+ggzLBFE/Lzs+ZG2hoAXMmFRWrEObVo2MglCkDXGyiODifcsLEXen3QIDlt io3IzmR78e8VkN165Hb0tAYXA9aYCHnPMVPdk7XhNho5Hf6vyOzS+trtQL8BoYlaGMK+Fc767h7 DlaqdVs269ASnKZbO7N56pRAozDY/KB1Oxtn9V0sp9iUOf8mYODWkSdtYaxRq6m6YwYnJf1BacH vpO1fz1FQOQIOVnMAk8LLkN2ixlAS1XAFKvGyDFsZkbSkopyL+2ECT2dYtUwEks1qO1wuZep4Pt 6qkch X-Google-Smtp-Source: AGHT+IFwIENfljndIrpGt9Cf8l7TIOozxBCW4Fb9t7PBibicOkS4GOtf842XC6W7UgHmvMuB4aWZkg== X-Received: by 2002:a05:690c:998e:b0:6f9:af1f:fdcd with SMTP id 00721157ae682-703e1545832mr254558847b3.11.1744040342591; Mon, 07 Apr 2025 08:39:02 -0700 (PDT) Received: from localhost (c-73-224-175-84.hsd1.fl.comcast.net. [73.224.175.84]) by smtp.gmail.com with ESMTPSA id 00721157ae682-703d1e4dedasm25946297b3.27.2025.04.07.08.39.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Apr 2025 08:39:02 -0700 (PDT) From: Yury Norov To: Tony Luck , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , "Yury Norov [NVIDIA]" , Rasmus Villemoes , x86@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/4] cpumask: add cpumask_{first,next}_andnot() API Date: Mon, 7 Apr 2025 11:38:54 -0400 Message-ID: <20250407153856.133093-4-yury.norov@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250407153856.133093-1-yury.norov@gmail.com> References: <20250407153856.133093-1-yury.norov@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Yury Norov [NVIDIA] With the lack of the functions, client code has to abuse less efficient cpumask_nth(). Signed-off-by: Yury Norov [NVIDIA] Reviewed-by: James Morse Tested-by: James Morse --- include/linux/cpumask.h | 59 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h index 0f816092c891..9067c3411cd0 100644 --- a/include/linux/cpumask.h +++ b/include/linux/cpumask.h @@ -178,6 +178,19 @@ unsigned int cpumask_first_and(const struct cpumask *s= rcp1, const struct cpumask return find_first_and_bit(cpumask_bits(srcp1), cpumask_bits(srcp2), small= _cpumask_bits); } =20 +/** + * cpumask_first_andnot - return the first cpu from *srcp1 & ~*srcp2 + * @srcp1: the first input + * @srcp2: the second input + * + * Return: >=3D nr_cpu_ids if no such cpu found. + */ +static __always_inline +unsigned int cpumask_first_andnot(const struct cpumask *srcp1, const struc= t cpumask *srcp2) +{ + return find_first_andnot_bit(cpumask_bits(srcp1), cpumask_bits(srcp2), sm= all_cpumask_bits); +} + /** * cpumask_first_and_and - return the first cpu from *srcp1 & *srcp2 & *sr= cp3 * @srcp1: the first input @@ -284,6 +297,25 @@ unsigned int cpumask_next_and(int n, const struct cpum= ask *src1p, small_cpumask_bits, n + 1); } =20 +/** + * cpumask_next_andnot - get the next cpu in *src1p & ~*src2p + * @n: the cpu prior to the place to search (i.e. return will be > @n) + * @src1p: the first cpumask pointer + * @src2p: the second cpumask pointer + * + * Return: >=3D nr_cpu_ids if no further cpus set in both. + */ +static __always_inline +unsigned int cpumask_next_andnot(int n, const struct cpumask *src1p, + const struct cpumask *src2p) +{ + /* -1 is a legal arg here. */ + if (n !=3D -1) + cpumask_check(n); + return find_next_andnot_bit(cpumask_bits(src1p), cpumask_bits(src2p), + small_cpumask_bits, n + 1); +} + /** * cpumask_next_and_wrap - get the next cpu in *src1p & *src2p, starting f= rom * @n+1. If nothing found, wrap around and start from @@ -458,6 +490,33 @@ unsigned int cpumask_any_and_but(const struct cpumask = *mask1, return cpumask_next_and(cpu, mask1, mask2); } =20 +/** + * cpumask_andnot_any_but - pick an arbitrary cpu from *mask1 & ~*mask2, b= ut not this one. + * @mask1: the first input cpumask + * @mask2: the second input cpumask + * @cpu: the cpu to ignore + * + * If @cpu =3D=3D -1, the function returns the first matching cpu. + * Returns >=3D nr_cpu_ids if no cpus set. + */ +static __always_inline +unsigned int cpumask_andnot_any_but(const struct cpumask *mask1, + const struct cpumask *mask2, + unsigned int cpu) +{ + unsigned int i; + + /* -1 is a legal arg here. */ + if (cpu !=3D -1) + cpumask_check(cpu); + + i =3D cpumask_first_andnot(mask1, mask2); + if (i !=3D cpu) + return i; + + return cpumask_next_andnot(cpu, mask1, mask2); +} + /** * cpumask_nth - get the Nth cpu in a cpumask * @srcp: the cpumask pointer --=20 2.43.0 From nobody Sun Feb 8 09:26:54 2026 Received: from mail-yb1-f180.google.com (mail-yb1-f180.google.com [209.85.219.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D182202996 for ; Mon, 7 Apr 2025 15:39:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744040346; cv=none; b=JiPHgvyPPdlFTz3+Id4psxIw4hoqT4IgSZWBz1j7POvjADErrUyCOnL4SYHsnAsiO9njTa+Dt7tQslLT2guCHAmrUHgSHZrPiy1BMoD8N2HKs9b9rrwXNS7WSqRBp5NH0Z9PUMgvtiwhrO1PLBHoGWZKBOYwDEWrhIkvkJ+AKOw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744040346; c=relaxed/simple; bh=YccRhJzXOVUtPTATWsrLUBAvdjjA7nJr0RfSoXtT7x0=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IzdmcIjt+qBCpRbi2jMJygetLX0uBoTpbZ/Hhl+TcXCJccMrGD8/cibp7lvXUPjf70n/N53+136E1Onk2fdknmJMs/YUg5o6aX7Ms5cUdyf4cb8eNgtZT5hy9U/2X3pY8BvRNwaeIiysA77M91SErs+fsXM/BuBm0cKnnV6b93o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MLJjhlTL; arc=none smtp.client-ip=209.85.219.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MLJjhlTL" Received: by mail-yb1-f180.google.com with SMTP id 3f1490d57ef6-e6e09fb66c7so3653595276.2 for ; Mon, 07 Apr 2025 08:39:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1744040344; x=1744645144; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=uu42nQ2bSmAwDWGBzjWYiwCDhQBT41z5F8NL5MUeB9A=; b=MLJjhlTLwX+K8313I1EqLCfPgbk2UqT1zIhZ2EAKVApYr1cENBI5iuXANKgrTy54pA Y3Ks5qPkpm/wQSCV2GQocx0FVinQvq3iyHbl9MaobBitL2agdMIDmgaxqbrC7ILvoHXe VHYp4A66tisAWiyG/0KL7wwQLuEHIL7bGvGt2lXAOOhfRPrKDm98xzto+5yy2+054TTx xHkU+NlTT1PLUlNONydkm2u1d7lBCKYcFJ/a8AIVh25ZUD/XbqAh0QWUrWATiBrlqG9F /qMboEVxe0VordbHF7Zb+vGZ+XD0vb2ZOephPu2rbLRQ1TnxhI6aRxnscLN3R6NicZe9 c9Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744040344; x=1744645144; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uu42nQ2bSmAwDWGBzjWYiwCDhQBT41z5F8NL5MUeB9A=; b=IB1U1NFD3pXu2RLkNP5hz9EPZMAozUbuYNsdy+c+cfz7JHzoQUctJICW8N1fbOt9mR s9g12/EW0KjlLB6O4NRHcshVJEYtYt2ufPBRGI8lErkk/TiicXWlRZNCE1GRhFtNq62C RzdmZcmbNh08R7FSapTMBTjcC29RkL08EHfn8IERHlvkUsnCLh1mU4lH1AgYnsnU+hpH bicgLTK/RIzeNHFlDv2Nlywg5XkODi5HlwhyDtAWFOjKBAiBE6CqoUCHQMcRCUhWkdso ffvSJYGlX9xITiHcN3XO1dXMO1u0ta/P4ncHRil6j+PnWWtrliTTYn3Qywk3Oduh6hYx 1OQw== X-Forwarded-Encrypted: i=1; AJvYcCVr5VX4f4Vx2ZUmV9Fp4A1pXspf3gLUyui9WTpvU/pCVLUdwXZ+NYZEMeM4ohLlpKPCE8RIB4mCLfqf9so=@vger.kernel.org X-Gm-Message-State: AOJu0YzaKv+wEPcO53xUG6CfUmdsvvmp5/X/T4QCjggl23WFI5IOjP0D mrnXWNkzQvQRh0Nxm412ndJD6sOHPlPFtlSb0GeILZ2PLPu3+ZcF X-Gm-Gg: ASbGncuIyIt4djG4CMlChpDhyNPgxLauduboEk9wmHVIKAcKiRL87fJ2xZR+AsCj932 d+fuJbq4eFRY43Bfv2z1/252VIOlB9PH6z0E4KUFN4g3AqaxOCg7KCqy59+5xPHd3qwiNMNi+6z wPx4LgQUQOhU4P2SU26X6oRs+x9bER0Wu28jwqpAQ3fMuDhf+LXIbXioQq0t/BCFZqNFDH2cFSz LlStIPcjjXx+qjHobBwrE9KTkb8zsJgxp1XR/3tZl5p+fMOv6eqDs0OxzMzxnLGvMSLMGqr7lMe iQqVVSjGBXEPuhi0GzNTa1dzLbuHR0Qz8xnxMGIHyam1fVwTSSwUcLvF8Ucu85hk9/a/wvdGGbL LfyestysnMV+dxuA= X-Google-Smtp-Source: AGHT+IGH8gkmMulMeXKGXIZTf8+s7ZupbfV3/p6oz6Ro5ygbXr9jdCeHXgH4ndUknwBwdh9y2n+VLw== X-Received: by 2002:a05:690c:6f85:b0:6ff:1c8c:b670 with SMTP id 00721157ae682-703e152e632mr216891757b3.14.1744040344063; Mon, 07 Apr 2025 08:39:04 -0700 (PDT) Received: from localhost (c-73-224-175-84.hsd1.fl.comcast.net. [73.224.175.84]) by smtp.gmail.com with ESMTPSA id 00721157ae682-703d1f778b4sm25735757b3.91.2025.04.07.08.39.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Apr 2025 08:39:03 -0700 (PDT) From: Yury Norov To: Tony Luck , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , "Yury Norov [NVIDIA]" , Rasmus Villemoes , x86@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/4] x86/resctrl: optimize cpumask_any_housekeeping() Date: Mon, 7 Apr 2025 11:38:55 -0400 Message-ID: <20250407153856.133093-5-yury.norov@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250407153856.133093-1-yury.norov@gmail.com> References: <20250407153856.133093-1-yury.norov@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Yury Norov [NVIDIA] With the lack of cpumask_andnot_any_but(), users have to abuse cpumask_nth() functions which are O(N*log(N)), comparing to O(N) for cpumask_any(). This series adds missing cpumask_andnot_any_but() and makes cpumask_any_but() understanding the RESCTRL_PICK_ANY_CPU hint. This simplifies cpumask_any_housekeeping() significantly. Signed-off-by: Yury Norov [NVIDIA] Reviewed-by: James Morse Tested-by: James Morse --- arch/x86/kernel/cpu/resctrl/internal.h | 28 +++++++------------------- 1 file changed, 7 insertions(+), 21 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 20c898f09b7e..1db02bab9743 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -71,30 +71,16 @@ static inline unsigned int cpumask_any_housekeeping(const struct cpumask *mask, int exclude_cpu) { - unsigned int cpu, hk_cpu; - - if (exclude_cpu =3D=3D RESCTRL_PICK_ANY_CPU) - cpu =3D cpumask_any(mask); - else - cpu =3D cpumask_any_but(mask, exclude_cpu); - - /* Only continue if tick_nohz_full_mask has been initialized. */ - if (!tick_nohz_full_enabled()) - return cpu; - - /* If the CPU picked isn't marked nohz_full nothing more needs doing. */ - if (cpu < nr_cpu_ids && !tick_nohz_full_cpu(cpu)) - return cpu; + unsigned int cpu; =20 /* Try to find a CPU that isn't nohz_full to use in preference */ - hk_cpu =3D cpumask_nth_andnot(0, mask, tick_nohz_full_mask); - if (hk_cpu =3D=3D exclude_cpu) - hk_cpu =3D cpumask_nth_andnot(1, mask, tick_nohz_full_mask); - - if (hk_cpu < nr_cpu_ids) - cpu =3D hk_cpu; + if (tick_nohz_full_enabled()) { + cpu =3D cpumask_andnot_any_but(mask, tick_nohz_full_mask, exclude_cpu); + if (cpu < nr_cpu_ids) + return cpu; + } =20 - return cpu; + return cpumask_any_but(mask, exclude_cpu); } =20 struct rdt_fs_context { --=20 2.43.0