From nobody Tue Jun 23 09:12:03 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FD45C433F5 for ; Tue, 8 Mar 2022 10:59:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346089AbiCHLAa (ORCPT ); Tue, 8 Mar 2022 06:00:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346062AbiCHLAZ (ORCPT ); Tue, 8 Mar 2022 06:00:25 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C551742ED4 for ; Tue, 8 Mar 2022 02:59:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646737167; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iRvTRLpjbDLOx0VqbXyM4lpVNCGi5++MSF5TUnleufc=; b=DOcl9bZm5EhfHmdo4tQY4zFzxthwiKw0KY9laG3VWEYYQ1rUVw5b2ndKZ87mD3mv7LJKoI E9Oh2CRT9jkScdQuAeXw9iitRKLAF8gkvZ2dxUCX1TkeeVFoAUdgWXGwmxvnsKBmAIuZdg zeJIIuzJhU0/Ju7l3lYxfsYnQ23zKi0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-407-O4Nm8WHmOeShJnwzttdIGg-1; Tue, 08 Mar 2022 05:59:24 -0500 X-MC-Unique: O4Nm8WHmOeShJnwzttdIGg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 67B9F1006AA7; Tue, 8 Mar 2022 10:59:22 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7884E752D3; Tue, 8 Mar 2022 10:59:21 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Michael Ellerman , Andrew Morton , Dennis Zhou , Tejun Heo , Christoph Lameter , Johannes Weiner , Michal Hocko , Vladimir Davydov , linux-mm@kvack.org, cgroups@vger.kernel.org, stable@vger.kernel.org Subject: [PATCH 1/3] mm: vmalloc: introduce array allocation functions Date: Tue, 8 Mar 2022 05:59:16 -0500 Message-Id: <20220308105918.615575-2-pbonzini@redhat.com> In-Reply-To: <20220308105918.615575-1-pbonzini@redhat.com> References: <20220308105918.615575-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Linux has dozens of occurrences of vmalloc(array_size()) and vzalloc(array_size()). Allow to simplify the code by providing vmalloc_array and vcalloc, as well as the underscored variants that let the caller specify the GFP flags. Cc: stable@vger.kernel.org Cc: linux-mm@kvack.org Signed-off-by: Paolo Bonzini Reviewed-by: David Hildenbrand --- include/linux/vmalloc.h | 5 +++++ mm/util.c | 50 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 880227b9f044..d1bbd4fd50c5 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -159,6 +159,11 @@ void *__vmalloc_node(unsigned long size, unsigned long= align, gfp_t gfp_mask, int node, const void *caller) __alloc_size(1); void *vmalloc_no_huge(unsigned long size) __alloc_size(1); =20 +extern void *__vmalloc_array(size_t n, size_t size, gfp_t flags) __alloc_s= ize(1, 2); +extern void *vmalloc_array(size_t n, size_t size) __alloc_size(1, 2); +extern void *__vcalloc(size_t n, size_t size, gfp_t flags) __alloc_size(1,= 2); +extern void *vcalloc(size_t n, size_t size) __alloc_size(1, 2); + extern void vfree(const void *addr); extern void vfree_atomic(const void *addr); =20 diff --git a/mm/util.c b/mm/util.c index 7e43369064c8..94475abe54a0 100644 --- a/mm/util.c +++ b/mm/util.c @@ -647,6 +647,56 @@ void *kvrealloc(const void *p, size_t oldsize, size_t = newsize, gfp_t flags) } EXPORT_SYMBOL(kvrealloc); =20 +/** + * __vmalloc_array - allocate memory for a virtually contiguous array. + * @n: number of elements. + * @size: element size. + * @flags: the type of memory to allocate (see kmalloc). + */ +void *__vmalloc_array(size_t n, size_t size, gfp_t flags) +{ + size_t bytes; + + if (unlikely(check_mul_overflow(n, size, &bytes))) + return NULL; + return __vmalloc(bytes, flags); +} +EXPORT_SYMBOL(__vmalloc_array); + +/** + * vmalloc_array - allocate memory for a virtually contiguous array. + * @n: number of elements. + * @size: element size. + */ +void *vmalloc_array(size_t n, size_t size) +{ + return __vmalloc_array(n, size, GFP_KERNEL); +} +EXPORT_SYMBOL(vmalloc_array); + +/** + * __vcalloc - allocate and zero memory for a virtually contiguous array. + * @n: number of elements. + * @size: element size. + * @flags: the type of memory to allocate (see kmalloc). + */ +void *__vcalloc(size_t n, size_t size, gfp_t flags) +{ + return __vmalloc_array(n, size, flags | __GFP_ZERO); +} +EXPORT_SYMBOL(__vcalloc); + +/** + * vcalloc - allocate and zero memory for a virtually contiguous array. + * @n: number of elements. + * @size: element size. + */ +void *vcalloc(size_t n, size_t size) +{ + return __vmalloc_array(n, size, GFP_KERNEL | __GFP_ZERO); +} +EXPORT_SYMBOL(vcalloc); + /* Neutral page->mapping pointer to address_space or anon_vma or other */ void *page_rmapping(struct page *page) { --=20 2.31.1 From nobody Tue Jun 23 09:12:03 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 411B5C433FE for ; Tue, 8 Mar 2022 10:59:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346098AbiCHLAd (ORCPT ); Tue, 8 Mar 2022 06:00:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53134 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346063AbiCHLA2 (ORCPT ); Tue, 8 Mar 2022 06:00:28 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 2534741F84 for ; Tue, 8 Mar 2022 02:59:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646737171; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aqPCGU3/YxaZYETOn7Lp2yXhW5De7uTDNtK+spwoHhQ=; b=ILNCdPQ9Z/6pHqNXUYjQjLDgN+M3IFRgsIwWG6kZUi3ZCDWCtf0ter8mdLacRhLtAoM/2N ibe05CjrsxbOzUtvDPk10EOBKcq05wvPvgstTDO0MN+zX0Ysm+Vu6K+FgyhgPl4wA2R9B9 j/i7Euubu6N74fEmvBzLDgxa1nOToPI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-391-rc6sWkT3NxS1VHxqq63bzA-1; Tue, 08 Mar 2022 05:59:25 -0500 X-MC-Unique: rc6sWkT3NxS1VHxqq63bzA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6412C800050; Tue, 8 Mar 2022 10:59:23 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 82C976FB03; Tue, 8 Mar 2022 10:59:22 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Michael Ellerman , Andrew Morton , Dennis Zhou , Tejun Heo , Christoph Lameter , Johannes Weiner , Michal Hocko , Vladimir Davydov , linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [PATCH 2/3] mm: use vmalloc_array and vcalloc for array allocations Date: Tue, 8 Mar 2022 05:59:17 -0500 Message-Id: <20220308105918.615575-3-pbonzini@redhat.com> In-Reply-To: <20220308105918.615575-1-pbonzini@redhat.com> References: <20220308105918.615575-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Instead of using array_size or just a multiply, use a function that takes care of both the multiplication and the overflow checks. Cc: linux-mm@kvack.org Signed-off-by: Paolo Bonzini Acked-by: Dennis Zhou Acked-by: Michal Hocko Reviewed-by: David Hildenbrand --- mm/percpu-stats.c | 2 +- mm/swap_cgroup.c | 4 +--- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/mm/percpu-stats.c b/mm/percpu-stats.c index c6bd092ff7a3..e71651cda2de 100644 --- a/mm/percpu-stats.c +++ b/mm/percpu-stats.c @@ -144,7 +144,7 @@ static int percpu_stats_show(struct seq_file *m, void *= v) spin_unlock_irq(&pcpu_lock); =20 /* there can be at most this many free and allocated fragments */ - buffer =3D vmalloc(array_size(sizeof(int), (2 * max_nr_alloc + 1))); + buffer =3D vmalloc_array(2 * max_nr_alloc + 1, sizeof(int)); if (!buffer) return -ENOMEM; =20 diff --git a/mm/swap_cgroup.c b/mm/swap_cgroup.c index 7f34343c075a..5a9442979a18 100644 --- a/mm/swap_cgroup.c +++ b/mm/swap_cgroup.c @@ -167,14 +167,12 @@ unsigned short lookup_swap_cgroup_id(swp_entry_t ent) int swap_cgroup_swapon(int type, unsigned long max_pages) { void *array; - unsigned long array_size; unsigned long length; struct swap_cgroup_ctrl *ctrl; =20 length =3D DIV_ROUND_UP(max_pages, SC_PER_PAGE); - array_size =3D length * sizeof(void *); =20 - array =3D vzalloc(array_size); + array =3D vcalloc(length, sizeof(void *)); if (!array) goto nomem; =20 --=20 2.31.1 From nobody Tue Jun 23 09:12:03 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A26A5C433F5 for ; Tue, 8 Mar 2022 10:59:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346103AbiCHLAh (ORCPT ); Tue, 8 Mar 2022 06:00:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346076AbiCHLA1 (ORCPT ); Tue, 8 Mar 2022 06:00:27 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id EDD3342A3B for ; Tue, 8 Mar 2022 02:59:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646737170; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u7L8HPCDt5lQJUN1tDnCrctmaA+f57JR/0xy1mw5PmA=; b=KYQhFmY7UEoMFb2ynyQH8Xb+tc1z8t8QIVTxQmsjVtWmfNOsAl35qwtCRfsqoQfydFPf8e AeHF1T9osBmrkJCnc/3DXu70bIun+9cvq2J8V0OXv5cO160xjkR+CZd7UEh67InImR2BAH teLb+rTNZabvj15th83frL4SV+EJTiY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-496-chAL4ODlPyqgTlVgKLSBag-1; Tue, 08 Mar 2022 05:59:26 -0500 X-MC-Unique: chAL4ODlPyqgTlVgKLSBag-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7067C1091DA1; Tue, 8 Mar 2022 10:59:24 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7F35F6FB03; Tue, 8 Mar 2022 10:59:23 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Michael Ellerman , Andrew Morton , Dennis Zhou , Tejun Heo , Christoph Lameter , Johannes Weiner , Michal Hocko , Vladimir Davydov , linux-mm@kvack.org, cgroups@vger.kernel.org, stable@vger.kernel.org Subject: [PATCH 3/3] KVM: use vcalloc/__vcalloc for very large allocations Date: Tue, 8 Mar 2022 05:59:18 -0500 Message-Id: <20220308105918.615575-4-pbonzini@redhat.com> In-Reply-To: <20220308105918.615575-1-pbonzini@redhat.com> References: <20220308105918.615575-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Allocations whose size is related to the memslot size can be arbitrarily large. Do not use kvzalloc/kvcalloc, as those are limited to "not crazy" sizes that fit in 32 bits. Now that it is available, they can use either vcalloc or __vcalloc, the latter if accounting is required. Cc: stable@vger.kernel.org Cc: kvm@vger.kernel.org Signed-off-by: Paolo Bonzini Reviewed-by: David Hildenbrand --- arch/powerpc/kvm/book3s_hv_uvmem.c | 2 +- arch/x86/kvm/mmu/page_track.c | 7 ++++--- arch/x86/kvm/x86.c | 4 ++-- virt/kvm/kvm_main.c | 4 ++-- 4 files changed, 9 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_uvmem.c b/arch/powerpc/kvm/book3s_h= v_uvmem.c index e414ca44839f..be441403925b 100644 --- a/arch/powerpc/kvm/book3s_hv_uvmem.c +++ b/arch/powerpc/kvm/book3s_hv_uvmem.c @@ -251,7 +251,7 @@ int kvmppc_uvmem_slot_init(struct kvm *kvm, const struc= t kvm_memory_slot *slot) p =3D kzalloc(sizeof(*p), GFP_KERNEL); if (!p) return -ENOMEM; - p->pfns =3D vzalloc(array_size(slot->npages, sizeof(*p->pfns))); + p->pfns =3D vcalloc(slot->npages, sizeof(*p->pfns)); if (!p->pfns) { kfree(p); return -ENOMEM; diff --git a/arch/x86/kvm/mmu/page_track.c b/arch/x86/kvm/mmu/page_track.c index 68eb1fb548b6..2e09d1b6249f 100644 --- a/arch/x86/kvm/mmu/page_track.c +++ b/arch/x86/kvm/mmu/page_track.c @@ -47,8 +47,8 @@ int kvm_page_track_create_memslot(struct kvm *kvm, continue; =20 slot->arch.gfn_track[i] =3D - kvcalloc(npages, sizeof(*slot->arch.gfn_track[i]), - GFP_KERNEL_ACCOUNT); + __vcalloc(npages, sizeof(*slot->arch.gfn_track[i]), + GFP_KERNEL_ACCOUNT); if (!slot->arch.gfn_track[i]) goto track_free; } @@ -75,7 +75,8 @@ int kvm_page_track_write_tracking_alloc(struct kvm_memory= _slot *slot) if (slot->arch.gfn_track[KVM_PAGE_TRACK_WRITE]) return 0; =20 - gfn_track =3D kvcalloc(slot->npages, sizeof(*gfn_track), GFP_KERNEL_ACCOU= NT); + gfn_track =3D __vcalloc(slot->npages, sizeof(*gfn_track), + GFP_KERNEL_ACCOUNT); if (gfn_track =3D=3D NULL) return -ENOMEM; =20 diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index f79bf4552082..4fa4d8269e5b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11838,7 +11838,7 @@ int memslot_rmap_alloc(struct kvm_memory_slot *slot= , unsigned long npages) if (slot->arch.rmap[i]) continue; =20 - slot->arch.rmap[i] =3D kvcalloc(lpages, sz, GFP_KERNEL_ACCOUNT); + slot->arch.rmap[i] =3D __vcalloc(lpages, sz, GFP_KERNEL_ACCOUNT); if (!slot->arch.rmap[i]) { memslot_rmap_free(slot); return -ENOMEM; @@ -11875,7 +11875,7 @@ static int kvm_alloc_memslot_metadata(struct kvm *k= vm, =20 lpages =3D __kvm_mmu_slot_lpages(slot, npages, level); =20 - linfo =3D kvcalloc(lpages, sizeof(*linfo), GFP_KERNEL_ACCOUNT); + linfo =3D __vcalloc(lpages, sizeof(*linfo), GFP_KERNEL_ACCOUNT); if (!linfo) goto out_free; =20 diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index c941b97fa133..a5726099df67 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1274,9 +1274,9 @@ static int kvm_vm_release(struct inode *inode, struct= file *filp) */ static int kvm_alloc_dirty_bitmap(struct kvm_memory_slot *memslot) { - unsigned long dirty_bytes =3D 2 * kvm_dirty_bitmap_bytes(memslot); + unsigned long dirty_bytes =3D kvm_dirty_bitmap_bytes(memslot); =20 - memslot->dirty_bitmap =3D kvzalloc(dirty_bytes, GFP_KERNEL_ACCOUNT); + memslot->dirty_bitmap =3D __vcalloc(2, dirty_bytes, GFP_KERNEL_ACCOUNT); if (!memslot->dirty_bitmap) return -ENOMEM; =20 --=20 2.31.1