From nobody Thu May 9 06:45:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1633349211828252.7006060889952; Mon, 4 Oct 2021 05:06:51 -0700 (PDT) Received: from localhost ([::1]:56990 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXMk6-00060p-OW for importer@patchew.org; Mon, 04 Oct 2021 08:06:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44100) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMgE-0001hg-37 for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:02:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:47976) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMgA-0006BW-NL for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:02:49 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-46-cDSeACuYPTiHbdW7CNNb4A-1; Mon, 04 Oct 2021 08:02:42 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0D9AD802947; Mon, 4 Oct 2021 12:02:42 +0000 (UTC) Received: from t480s.redhat.com (unknown [10.39.194.159]) by smtp.corp.redhat.com (Postfix) with ESMTP id EE27F19C59; Mon, 4 Oct 2021 12:02:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633348966; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NMOA9kQbOGwFqcPMc+ilA6kpME97Hs3N/G+0b3k1Sx0=; b=B3a4p7EbMY9MoGCqVYn6/8IMjbgiSnpyIOc6aqm8hoWP1utnljUrKF5VgO96LQWx2joUDl w4+U2Rz21MuhTNfAPtp5fThjVQtISqZjFm2H6PML8UVj1NL8NUSS7FXtNtDzKhm88wzLd9 eUzBDZAuTuC3TLgEYV2i3gWoYwEzme0= X-MC-Unique: cDSeACuYPTiHbdW7CNNb4A-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 1/7] util/oslib-posix: Let touch_all_pages() return an error Date: Mon, 4 Oct 2021 14:02:02 +0200 Message-Id: <20211004120208.7409-2-david@redhat.com> In-Reply-To: <20211004120208.7409-1-david@redhat.com> References: <20211004120208.7409-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Pankaj Gupta , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Igor Mammedov , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1633349212517100001 Let's prepare touch_all_pages() for returning differing errors. Return an error from the thread and report the last processed error. Translate SIGBUS to -EFAULT, as a SIGBUS can mean all different kind of things (memory error, read error, out of memory). When allocating memory fails via the current SIGBUS-based mechanism, we'll get: os_mem_prealloc: preallocating memory failed: Bad address Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: David Hildenbrand --- util/oslib-posix.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/util/oslib-posix.c b/util/oslib-posix.c index e8bdb02e1d..b146beef78 100644 --- a/util/oslib-posix.c +++ b/util/oslib-posix.c @@ -84,7 +84,6 @@ typedef struct MemsetThread MemsetThread; =20 static MemsetThread *memset_thread; static int memset_num_threads; -static bool memset_thread_failed; =20 static QemuMutex page_mutex; static QemuCond page_cond; @@ -452,6 +451,7 @@ static void *do_touch_pages(void *arg) { MemsetThread *memset_args =3D (MemsetThread *)arg; sigset_t set, oldset; + int ret =3D 0; =20 /* * On Linux, the page faults from the loop below can cause mmap_sem @@ -470,7 +470,7 @@ static void *do_touch_pages(void *arg) pthread_sigmask(SIG_UNBLOCK, &set, &oldset); =20 if (sigsetjmp(memset_args->env, 1)) { - memset_thread_failed =3D true; + ret =3D -EFAULT; } else { char *addr =3D memset_args->addr; size_t numpages =3D memset_args->numpages; @@ -494,7 +494,7 @@ static void *do_touch_pages(void *arg) } } pthread_sigmask(SIG_SETMASK, &oldset, NULL); - return NULL; + return (void *)(uintptr_t)ret; } =20 static inline int get_memset_num_threads(int smp_cpus) @@ -509,13 +509,13 @@ static inline int get_memset_num_threads(int smp_cpus) return ret; } =20 -static bool touch_all_pages(char *area, size_t hpagesize, size_t numpages, - int smp_cpus) +static int touch_all_pages(char *area, size_t hpagesize, size_t numpages, + int smp_cpus) { static gsize initialized =3D 0; size_t numpages_per_thread, leftover; + int ret =3D 0, i =3D 0; char *addr =3D area; - int i =3D 0; =20 if (g_once_init_enter(&initialized)) { qemu_mutex_init(&page_mutex); @@ -523,7 +523,6 @@ static bool touch_all_pages(char *area, size_t hpagesiz= e, size_t numpages, g_once_init_leave(&initialized, 1); } =20 - memset_thread_failed =3D false; threads_created_flag =3D false; memset_num_threads =3D get_memset_num_threads(smp_cpus); memset_thread =3D g_new0(MemsetThread, memset_num_threads); @@ -545,12 +544,16 @@ static bool touch_all_pages(char *area, size_t hpages= ize, size_t numpages, qemu_mutex_unlock(&page_mutex); =20 for (i =3D 0; i < memset_num_threads; i++) { - qemu_thread_join(&memset_thread[i].pgthread); + int tmp =3D (uintptr_t)qemu_thread_join(&memset_thread[i].pgthread= ); + + if (tmp) { + ret =3D tmp; + } } g_free(memset_thread); memset_thread =3D NULL; =20 - return memset_thread_failed; + return ret; } =20 void os_mem_prealloc(int fd, char *area, size_t memory, int smp_cpus, @@ -573,9 +576,10 @@ void os_mem_prealloc(int fd, char *area, size_t memory= , int smp_cpus, } =20 /* touch pages simultaneously */ - if (touch_all_pages(area, hpagesize, numpages, smp_cpus)) { - error_setg(errp, "os_mem_prealloc: Insufficient free host memory " - "pages available to allocate guest RAM"); + ret =3D touch_all_pages(area, hpagesize, numpages, smp_cpus); + if (ret) { + error_setg_errno(errp, -ret, + "os_mem_prealloc: preallocating memory failed"); } =20 ret =3D sigaction(SIGBUS, &oldact, NULL); --=20 2.31.1 From nobody Thu May 9 06:45:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1633349149781771.4863552292818; Mon, 4 Oct 2021 05:05:49 -0700 (PDT) Received: from localhost ([::1]:55098 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXMj6-0004a8-CP for importer@patchew.org; Mon, 04 Oct 2021 08:05:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44110) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMgF-0001j8-OZ for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:02:52 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:35116) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMgD-0006C9-Pm for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:02:51 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-317-PaM-FWZBO2-vi4SL1QECIg-1; Mon, 04 Oct 2021 08:02:46 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4D54356B47; Mon, 4 Oct 2021 12:02:45 +0000 (UTC) Received: from t480s.redhat.com (unknown [10.39.194.159]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6B7DB19C59; Mon, 4 Oct 2021 12:02:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633348967; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FuqDVk/VmU0BGNDYQ+fzVu3fCHn52LEbJwMF35lPkRA=; b=fmCy6vuybfWXSJ2wuQgxcflSKXBSbSHb3qyChEH01UyR6WebTfTr67viqpuEKSIarNGOtQ gJ5DTVXC7jOLW83cIRZ2VaUUI54P7gZ+LgUWfmqcZDR4EmX2O/qF0rD0qzAFc/ccb1h58Y e8ppD+bRsvR5C7IDAIwSGqX3pAQtEEs= X-MC-Unique: PaM-FWZBO2-vi4SL1QECIg-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 2/7] util/oslib-posix: Support MADV_POPULATE_WRITE for os_mem_prealloc() Date: Mon, 4 Oct 2021 14:02:03 +0200 Message-Id: <20211004120208.7409-3-david@redhat.com> In-Reply-To: <20211004120208.7409-1-david@redhat.com> References: <20211004120208.7409-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Pankaj Gupta , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Pankaj Gupta , Igor Mammedov , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1633349151742100003 Let's sense support and use it for preallocation. MADV_POPULATE_WRITE does not require a SIGBUS handler, doesn't actually touch page content, and avoids context switches; it is, therefore, faster and easier to handle than our current approach. While MADV_POPULATE_WRITE is, in general, faster than manual prefaulting, and especially faster with 4k pages, there is still value in prefaulting using multiple threads to speed up preallocation. More details on MADV_POPULATE_WRITE can be found in the Linux commits 4ca9b3859dac ("mm/madvise: introduce MADV_POPULATE_(READ|WRITE) to prefault page tables") and eb2faa513c24 ("mm/madvise: report SIGBUS as -EFAULT for MADV_POPULATE_(READ|WRITE)"), and in the man page proposal [1]. This resolves the TODO in do_touch_pages(). In the future, we might want to look into using fallocate(), eventually combined with MADV_POPULATE_READ, when dealing with shared file/fd mappings and not caring about memory bindings. [1] https://lkml.kernel.org/r/20210816081922.5155-1-david@redhat.com Reviewed-by: Pankaj Gupta Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: David Hildenbrand --- include/qemu/osdep.h | 7 ++++ util/oslib-posix.c | 83 +++++++++++++++++++++++++++++++++----------- 2 files changed, 69 insertions(+), 21 deletions(-) diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h index 60718fc342..d1660d67fa 100644 --- a/include/qemu/osdep.h +++ b/include/qemu/osdep.h @@ -471,6 +471,11 @@ static inline void qemu_cleanup_generic_vfree(void *p) #else #define QEMU_MADV_REMOVE QEMU_MADV_DONTNEED #endif +#ifdef MADV_POPULATE_WRITE +#define QEMU_MADV_POPULATE_WRITE MADV_POPULATE_WRITE +#else +#define QEMU_MADV_POPULATE_WRITE QEMU_MADV_INVALID +#endif =20 #elif defined(CONFIG_POSIX_MADVISE) =20 @@ -484,6 +489,7 @@ static inline void qemu_cleanup_generic_vfree(void *p) #define QEMU_MADV_HUGEPAGE QEMU_MADV_INVALID #define QEMU_MADV_NOHUGEPAGE QEMU_MADV_INVALID #define QEMU_MADV_REMOVE QEMU_MADV_DONTNEED +#define QEMU_MADV_POPULATE_WRITE QEMU_MADV_INVALID =20 #else /* no-op */ =20 @@ -497,6 +503,7 @@ static inline void qemu_cleanup_generic_vfree(void *p) #define QEMU_MADV_HUGEPAGE QEMU_MADV_INVALID #define QEMU_MADV_NOHUGEPAGE QEMU_MADV_INVALID #define QEMU_MADV_REMOVE QEMU_MADV_INVALID +#define QEMU_MADV_POPULATE_WRITE QEMU_MADV_INVALID =20 #endif =20 diff --git a/util/oslib-posix.c b/util/oslib-posix.c index b146beef78..cb89e07770 100644 --- a/util/oslib-posix.c +++ b/util/oslib-posix.c @@ -484,10 +484,6 @@ static void *do_touch_pages(void *arg) * * 'volatile' to stop compiler optimizing this away * to a no-op - * - * TODO: get a better solution from kernel so we - * don't need to write at all so we don't cause - * wear on the storage backing the region... */ *(volatile char *)addr =3D *addr; addr +=3D hpagesize; @@ -497,6 +493,26 @@ static void *do_touch_pages(void *arg) return (void *)(uintptr_t)ret; } =20 +static void *do_madv_populate_write_pages(void *arg) +{ + MemsetThread *memset_args =3D (MemsetThread *)arg; + const size_t size =3D memset_args->numpages * memset_args->hpagesize; + char * const addr =3D memset_args->addr; + int ret =3D 0; + + /* See do_touch_pages(). */ + qemu_mutex_lock(&page_mutex); + while (!threads_created_flag) { + qemu_cond_wait(&page_cond, &page_mutex); + } + qemu_mutex_unlock(&page_mutex); + + if (size && qemu_madvise(addr, size, QEMU_MADV_POPULATE_WRITE)) { + ret =3D -errno; + } + return (void *)(uintptr_t)ret; +} + static inline int get_memset_num_threads(int smp_cpus) { long host_procs =3D sysconf(_SC_NPROCESSORS_ONLN); @@ -510,10 +526,11 @@ static inline int get_memset_num_threads(int smp_cpus) } =20 static int touch_all_pages(char *area, size_t hpagesize, size_t numpages, - int smp_cpus) + int smp_cpus, bool use_madv_populate_write) { static gsize initialized =3D 0; size_t numpages_per_thread, leftover; + void *(*touch_fn)(void *); int ret =3D 0, i =3D 0; char *addr =3D area; =20 @@ -523,6 +540,12 @@ static int touch_all_pages(char *area, size_t hpagesiz= e, size_t numpages, g_once_init_leave(&initialized, 1); } =20 + if (use_madv_populate_write) { + touch_fn =3D do_madv_populate_write_pages; + } else { + touch_fn =3D do_touch_pages; + } + threads_created_flag =3D false; memset_num_threads =3D get_memset_num_threads(smp_cpus); memset_thread =3D g_new0(MemsetThread, memset_num_threads); @@ -533,7 +556,7 @@ static int touch_all_pages(char *area, size_t hpagesize= , size_t numpages, memset_thread[i].numpages =3D numpages_per_thread + (i < leftover); memset_thread[i].hpagesize =3D hpagesize; qemu_thread_create(&memset_thread[i].pgthread, "touch_pages", - do_touch_pages, &memset_thread[i], + touch_fn, &memset_thread[i], QEMU_THREAD_JOINABLE); addr +=3D memset_thread[i].numpages * hpagesize; } @@ -556,6 +579,12 @@ static int touch_all_pages(char *area, size_t hpagesiz= e, size_t numpages, return ret; } =20 +static bool madv_populate_write_possible(char *area, size_t pagesize) +{ + return !qemu_madvise(area, pagesize, QEMU_MADV_POPULATE_WRITE) || + errno !=3D EINVAL; +} + void os_mem_prealloc(int fd, char *area, size_t memory, int smp_cpus, Error **errp) { @@ -563,30 +592,42 @@ void os_mem_prealloc(int fd, char *area, size_t memor= y, int smp_cpus, struct sigaction act, oldact; size_t hpagesize =3D qemu_fd_getpagesize(fd); size_t numpages =3D DIV_ROUND_UP(memory, hpagesize); + bool use_madv_populate_write; =20 - memset(&act, 0, sizeof(act)); - act.sa_handler =3D &sigbus_handler; - act.sa_flags =3D 0; - - ret =3D sigaction(SIGBUS, &act, &oldact); - if (ret) { - error_setg_errno(errp, errno, - "os_mem_prealloc: failed to install signal handler"); - return; + /* + * Sense on every invocation, as MADV_POPULATE_WRITE cannot be used for + * some special mappings, such as mapping /dev/mem. + */ + use_madv_populate_write =3D madv_populate_write_possible(area, hpagesi= ze); + + if (!use_madv_populate_write) { + memset(&act, 0, sizeof(act)); + act.sa_handler =3D &sigbus_handler; + act.sa_flags =3D 0; + + ret =3D sigaction(SIGBUS, &act, &oldact); + if (ret) { + error_setg_errno(errp, errno, + "os_mem_prealloc: failed to install signal handler"); + return; + } } =20 /* touch pages simultaneously */ - ret =3D touch_all_pages(area, hpagesize, numpages, smp_cpus); + ret =3D touch_all_pages(area, hpagesize, numpages, smp_cpus, + use_madv_populate_write); if (ret) { error_setg_errno(errp, -ret, "os_mem_prealloc: preallocating memory failed"); } =20 - ret =3D sigaction(SIGBUS, &oldact, NULL); - if (ret) { - /* Terminate QEMU since it can't recover from error */ - perror("os_mem_prealloc: failed to reinstall signal handler"); - exit(1); + if (!use_madv_populate_write) { + ret =3D sigaction(SIGBUS, &oldact, NULL); + if (ret) { + /* Terminate QEMU since it can't recover from error */ + perror("os_mem_prealloc: failed to reinstall signal handler"); + exit(1); + } } } =20 --=20 2.31.1 From nobody Thu May 9 06:45:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1633349394548163.3915832860887; Mon, 4 Oct 2021 05:09:54 -0700 (PDT) Received: from localhost ([::1]:37600 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXMn3-0003gM-9M for importer@patchew.org; Mon, 04 Oct 2021 08:09:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44166) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMgV-0001uz-Ma for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:03:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22016) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMgT-0006Lc-EP for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:03:07 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-506-YRRcrq5gOA-2P_JRkIlSEA-1; Mon, 04 Oct 2021 08:02:49 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 68D625721D; Mon, 4 Oct 2021 12:02:48 +0000 (UTC) Received: from t480s.redhat.com (unknown [10.39.194.159]) by smtp.corp.redhat.com (Postfix) with ESMTP id B4CEE19D9B; Mon, 4 Oct 2021 12:02:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633348984; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E2hPa3OeU2VShLBIWGRBI4Pl+wvAn0MdoP0MtSPhYqc=; b=GmeQgkJXw6QjOtzKOWSJ6sq7ZAwlbe8r1jZA6c/JmdJbHcb2Nm5NOuIH3xzzfM5vj7zEcH oB1XAzG3exf9P18QBKb+CgIj7FA6APQUPQfXiig7c9Odf4JLc+0eZVP2vXTAh8z3/wBR+L fNDZwxmsufQAlNCdvSJZQHbn25Kj2/g= X-MC-Unique: YRRcrq5gOA-2P_JRkIlSEA-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 3/7] util/oslib-posix: Introduce and use MemsetContext for touch_all_pages() Date: Mon, 4 Oct 2021 14:02:04 +0200 Message-Id: <20211004120208.7409-4-david@redhat.com> In-Reply-To: <20211004120208.7409-1-david@redhat.com> References: <20211004120208.7409-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Pankaj Gupta , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Igor Mammedov , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1633349396244100001 Let's minimize the number of global variables to prepare for os_mem_prealloc() getting called concurrently and make the code a bit easier to read. The only consumer that really needs a global variable is the sigbus handler, which will require protection via a mutex in the future either way as we cannot concurrently mess with the SIGBUS handler. Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: David Hildenbrand --- util/oslib-posix.c | 73 +++++++++++++++++++++++++++++----------------- 1 file changed, 47 insertions(+), 26 deletions(-) diff --git a/util/oslib-posix.c b/util/oslib-posix.c index cb89e07770..cf2ead54ad 100644 --- a/util/oslib-posix.c +++ b/util/oslib-posix.c @@ -73,21 +73,30 @@ =20 #define MAX_MEM_PREALLOC_THREAD_COUNT 16 =20 +struct MemsetThread; + +typedef struct MemsetContext { + bool all_threads_created; + bool any_thread_failed; + struct MemsetThread *threads; + int num_threads; +} MemsetContext; + struct MemsetThread { char *addr; size_t numpages; size_t hpagesize; QemuThread pgthread; sigjmp_buf env; + MemsetContext *context; }; typedef struct MemsetThread MemsetThread; =20 -static MemsetThread *memset_thread; -static int memset_num_threads; +/* used by sigbus_handler() */ +static MemsetContext *sigbus_memset_context; =20 static QemuMutex page_mutex; static QemuCond page_cond; -static bool threads_created_flag; =20 int qemu_get_thread_id(void) { @@ -438,10 +447,13 @@ const char *qemu_get_exec_dir(void) static void sigbus_handler(int signal) { int i; - if (memset_thread) { - for (i =3D 0; i < memset_num_threads; i++) { - if (qemu_thread_is_self(&memset_thread[i].pgthread)) { - siglongjmp(memset_thread[i].env, 1); + + if (sigbus_memset_context) { + for (i =3D 0; i < sigbus_memset_context->num_threads; i++) { + MemsetThread *thread =3D &sigbus_memset_context->threads[i]; + + if (qemu_thread_is_self(&thread->pgthread)) { + siglongjmp(thread->env, 1); } } } @@ -459,7 +471,7 @@ static void *do_touch_pages(void *arg) * clearing until all threads have been created. */ qemu_mutex_lock(&page_mutex); - while(!threads_created_flag){ + while (!memset_args->context->all_threads_created) { qemu_cond_wait(&page_cond, &page_mutex); } qemu_mutex_unlock(&page_mutex); @@ -502,7 +514,7 @@ static void *do_madv_populate_write_pages(void *arg) =20 /* See do_touch_pages(). */ qemu_mutex_lock(&page_mutex); - while (!threads_created_flag) { + while (!memset_args->context->all_threads_created) { qemu_cond_wait(&page_cond, &page_mutex); } qemu_mutex_unlock(&page_mutex); @@ -529,6 +541,9 @@ static int touch_all_pages(char *area, size_t hpagesize= , size_t numpages, int smp_cpus, bool use_madv_populate_write) { static gsize initialized =3D 0; + MemsetContext context =3D { + .num_threads =3D get_memset_num_threads(smp_cpus), + }; size_t numpages_per_thread, leftover; void *(*touch_fn)(void *); int ret =3D 0, i =3D 0; @@ -546,35 +561,41 @@ static int touch_all_pages(char *area, size_t hpagesi= ze, size_t numpages, touch_fn =3D do_touch_pages; } =20 - threads_created_flag =3D false; - memset_num_threads =3D get_memset_num_threads(smp_cpus); - memset_thread =3D g_new0(MemsetThread, memset_num_threads); - numpages_per_thread =3D numpages / memset_num_threads; - leftover =3D numpages % memset_num_threads; - for (i =3D 0; i < memset_num_threads; i++) { - memset_thread[i].addr =3D addr; - memset_thread[i].numpages =3D numpages_per_thread + (i < leftover); - memset_thread[i].hpagesize =3D hpagesize; - qemu_thread_create(&memset_thread[i].pgthread, "touch_pages", - touch_fn, &memset_thread[i], + context.threads =3D g_new0(MemsetThread, context.num_threads); + numpages_per_thread =3D numpages / context.num_threads; + leftover =3D numpages % context.num_threads; + for (i =3D 0; i < context.num_threads; i++) { + context.threads[i].addr =3D addr; + context.threads[i].numpages =3D numpages_per_thread + (i < leftove= r); + context.threads[i].hpagesize =3D hpagesize; + context.threads[i].context =3D &context; + qemu_thread_create(&context.threads[i].pgthread, "touch_pages", + touch_fn, &context.threads[i], QEMU_THREAD_JOINABLE); - addr +=3D memset_thread[i].numpages * hpagesize; + addr +=3D context.threads[i].numpages * hpagesize; + } + + if (!use_madv_populate_write) { + sigbus_memset_context =3D &context; } =20 qemu_mutex_lock(&page_mutex); - threads_created_flag =3D true; + context.all_threads_created =3D true; qemu_cond_broadcast(&page_cond); qemu_mutex_unlock(&page_mutex); =20 - for (i =3D 0; i < memset_num_threads; i++) { - int tmp =3D (uintptr_t)qemu_thread_join(&memset_thread[i].pgthread= ); + for (i =3D 0; i < context.num_threads; i++) { + int tmp =3D (uintptr_t)qemu_thread_join(&context.threads[i].pgthre= ad); =20 if (tmp) { ret =3D tmp; } } - g_free(memset_thread); - memset_thread =3D NULL; + + if (!use_madv_populate_write) { + sigbus_memset_context =3D NULL; + } + g_free(context.threads); =20 return ret; } --=20 2.31.1 From nobody Thu May 9 06:45:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1633349553280775.9785693571649; Mon, 4 Oct 2021 05:12:33 -0700 (PDT) Received: from localhost ([::1]:42824 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXMpc-0007HQ-5F for importer@patchew.org; Mon, 04 Oct 2021 08:12:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44130) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMgP-0001oH-Er for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:03:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:34149) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMgM-0006Gm-P8 for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:03:00 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-117-yFc8H_kyO4yqHltN6fPOcA-1; Mon, 04 Oct 2021 08:02:52 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 90D8C5721E; Mon, 4 Oct 2021 12:02:51 +0000 (UTC) Received: from t480s.redhat.com (unknown [10.39.194.159]) by smtp.corp.redhat.com (Postfix) with ESMTP id DD0CA19C59; Mon, 4 Oct 2021 12:02:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633348976; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6PRu0OPJVIqoF1U6fV+wR5ie0tm5wZlfI3mKkoAKXfo=; b=O1k0hAXqyneOyaYUjvhi5c/Xig9AR7TLJLfnTY1exSTT3ZylzhH18ufhwGSSqJfumBRNyP Aqeok7qw/epdGE9SUKBMOmFeLzb43TU/H+9wZE/3060QeDUyeh/qcMag/jtfjskw4bf0H7 YsEqeGAQbigiTZIwu3ZFWkSerNdwt0A= X-MC-Unique: yFc8H_kyO4yqHltN6fPOcA-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 4/7] util/oslib-posix: Don't create too many threads with small memory or little pages Date: Mon, 4 Oct 2021 14:02:05 +0200 Message-Id: <20211004120208.7409-5-david@redhat.com> In-Reply-To: <20211004120208.7409-1-david@redhat.com> References: <20211004120208.7409-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Pankaj Gupta , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Pankaj Gupta , Igor Mammedov , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1633349555505100001 Let's limit the number of threads to something sane, especially that - We don't have more threads than the number of pages we have - We don't have threads that initialize small (< 64 MiB) memory Reviewed-by: Pankaj Gupta Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: David Hildenbrand --- util/oslib-posix.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/util/oslib-posix.c b/util/oslib-posix.c index cf2ead54ad..67c08a425e 100644 --- a/util/oslib-posix.c +++ b/util/oslib-posix.c @@ -40,6 +40,7 @@ #include #include "qemu/cutils.h" #include "qemu/compiler.h" +#include "qemu/units.h" =20 #ifdef CONFIG_LINUX #include @@ -525,7 +526,8 @@ static void *do_madv_populate_write_pages(void *arg) return (void *)(uintptr_t)ret; } =20 -static inline int get_memset_num_threads(int smp_cpus) +static inline int get_memset_num_threads(size_t hpagesize, size_t numpages, + int smp_cpus) { long host_procs =3D sysconf(_SC_NPROCESSORS_ONLN); int ret =3D 1; @@ -533,6 +535,12 @@ static inline int get_memset_num_threads(int smp_cpus) if (host_procs > 0) { ret =3D MIN(MIN(host_procs, MAX_MEM_PREALLOC_THREAD_COUNT), smp_cp= us); } + + /* Especially with gigantic pages, don't create more threads than page= s. */ + ret =3D MIN(ret, numpages); + /* Don't start threads to prealloc comparatively little memory. */ + ret =3D MIN(ret, MAX(1, hpagesize * numpages / (64 * MiB))); + /* In case sysconf() fails, we fall back to single threaded */ return ret; } @@ -542,7 +550,7 @@ static int touch_all_pages(char *area, size_t hpagesize= , size_t numpages, { static gsize initialized =3D 0; MemsetContext context =3D { - .num_threads =3D get_memset_num_threads(smp_cpus), + .num_threads =3D get_memset_num_threads(hpagesize, numpages, smp_c= pus), }; size_t numpages_per_thread, leftover; void *(*touch_fn)(void *); --=20 2.31.1 From nobody Thu May 9 06:45:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1633349645527697.6627319065939; Mon, 4 Oct 2021 05:14:05 -0700 (PDT) Received: from localhost ([::1]:46386 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXMr6-0001G9-Ev for importer@patchew.org; Mon, 04 Oct 2021 08:14:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44650) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMia-0005Ly-Av for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:05:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:32382) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMiU-0003eo-Ek for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:05:15 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-162-r2EoFnMbMnaShP0FJpMUPQ-1; Mon, 04 Oct 2021 08:02:55 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A53101B18BCB; Mon, 4 Oct 2021 12:02:54 +0000 (UTC) Received: from t480s.redhat.com (unknown [10.39.194.159]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0A2A819C59; Mon, 4 Oct 2021 12:02:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633349109; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Wlt40QBQkDU6CaDYw9NJ1KuuNgpytHW2nxi1LfiTHfw=; b=YTlclzjMVa8FxCR8m/ZrTtMkPjbXRulQaZ6o5roo8ywid2pVpornY+5EMaMQCBCqaJZbRr FwBFK/b8+pOgUXCXhanRweRgJQgITwZ4aw9ygM9A/JO2ROYyFG0sVJcQj1Xv6mIcrLKPuL oXGxsQYvpuLwXFtSdcqICBvIhqMYiqM= X-MC-Unique: r2EoFnMbMnaShP0FJpMUPQ-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 5/7] util/oslib-posix: Avoid creating a single thread with MADV_POPULATE_WRITE Date: Mon, 4 Oct 2021 14:02:06 +0200 Message-Id: <20211004120208.7409-6-david@redhat.com> In-Reply-To: <20211004120208.7409-1-david@redhat.com> References: <20211004120208.7409-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -9 X-Spam_score: -1.0 X-Spam_bar: - X-Spam_report: (-1.0 / 5.0 requ) DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Pankaj Gupta , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Pankaj Gupta , Igor Mammedov , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1633349645983100001 Let's simplify the case when we only want a single thread and don't have to mess with signal handlers. Reviewed-by: Pankaj Gupta Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: David Hildenbrand --- util/oslib-posix.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/util/oslib-posix.c b/util/oslib-posix.c index 67c08a425e..efa4f96d56 100644 --- a/util/oslib-posix.c +++ b/util/oslib-posix.c @@ -564,6 +564,14 @@ static int touch_all_pages(char *area, size_t hpagesiz= e, size_t numpages, } =20 if (use_madv_populate_write) { + /* Avoid creating a single thread for MADV_POPULATE_WRITE */ + if (context.num_threads =3D=3D 1) { + if (qemu_madvise(area, hpagesize * numpages, + QEMU_MADV_POPULATE_WRITE)) { + return -errno; + } + return 0; + } touch_fn =3D do_madv_populate_write_pages; } else { touch_fn =3D do_touch_pages; --=20 2.31.1 From nobody Thu May 9 06:45:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1633349365012504.3197413642134; Mon, 4 Oct 2021 05:09:25 -0700 (PDT) Received: from localhost ([::1]:35428 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXMmZ-0002DO-Om for importer@patchew.org; Mon, 04 Oct 2021 08:09:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44164) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMgV-0001uy-4P for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:03:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:45097) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMgT-0006LZ-7M for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:03:06 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-479-jhAdaFcbOjSOoCYE-7GDHQ-1; Mon, 04 Oct 2021 08:02:59 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0A2B15721F; Mon, 4 Oct 2021 12:02:58 +0000 (UTC) Received: from t480s.redhat.com (unknown [10.39.194.159]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4C06019C59; Mon, 4 Oct 2021 12:02:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633348984; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e29Gggx2QJ++Sey6LRiwKPCe4BhS91CgAJ0+aKHqpRE=; b=I9mr4XJyaIU6uo+eD3ggphbv329T8JQETiZsQeBmukmJbZcB3IwnpzQZJtBiYs/QhPRKbw LXUIwdqw0sjk0EclWSWXG7QBFfWxfhBpGx5EIQishDvUx6wjvoxYIKDzy1GgJ4D1zhwlmN 0SYn/KMtIYuiaMCPytUurc9wcjTEMJo= X-MC-Unique: jhAdaFcbOjSOoCYE-7GDHQ-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 6/7] util/oslib-posix: Support concurrent os_mem_prealloc() invocation Date: Mon, 4 Oct 2021 14:02:07 +0200 Message-Id: <20211004120208.7409-7-david@redhat.com> In-Reply-To: <20211004120208.7409-1-david@redhat.com> References: <20211004120208.7409-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Pankaj Gupta , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Pankaj Gupta , Igor Mammedov , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1633349366454100001 Add a mutex to protect the SIGBUS case, as we cannot mess concurrently with the sigbus handler and we have to manage the global variable sigbus_memset_context. The MADV_POPULATE_WRITE path can run concurrently. Note that page_mutex and page_cond are shared between concurrent invocations, which shouldn't be a problem. This is a preparation for future virtio-mem prealloc code, which will call os_mem_prealloc() asynchronously from an iothread when handling guest requests. Reviewed-by: Pankaj Gupta Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: David Hildenbrand --- util/oslib-posix.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/util/oslib-posix.c b/util/oslib-posix.c index efa4f96d56..9829149e4b 100644 --- a/util/oslib-posix.c +++ b/util/oslib-posix.c @@ -95,6 +95,7 @@ typedef struct MemsetThread MemsetThread; =20 /* used by sigbus_handler() */ static MemsetContext *sigbus_memset_context; +static QemuMutex sigbus_mutex; =20 static QemuMutex page_mutex; static QemuCond page_cond; @@ -625,6 +626,7 @@ static bool madv_populate_write_possible(char *area, si= ze_t pagesize) void os_mem_prealloc(int fd, char *area, size_t memory, int smp_cpus, Error **errp) { + static gsize initialized; int ret; struct sigaction act, oldact; size_t hpagesize =3D qemu_fd_getpagesize(fd); @@ -638,6 +640,12 @@ void os_mem_prealloc(int fd, char *area, size_t memory= , int smp_cpus, use_madv_populate_write =3D madv_populate_write_possible(area, hpagesi= ze); =20 if (!use_madv_populate_write) { + if (g_once_init_enter(&initialized)) { + qemu_mutex_init(&sigbus_mutex); + g_once_init_leave(&initialized, 1); + } + + qemu_mutex_lock(&sigbus_mutex); memset(&act, 0, sizeof(act)); act.sa_handler =3D &sigbus_handler; act.sa_flags =3D 0; @@ -665,6 +673,7 @@ void os_mem_prealloc(int fd, char *area, size_t memory,= int smp_cpus, perror("os_mem_prealloc: failed to reinstall signal handler"); exit(1); } + qemu_mutex_unlock(&sigbus_mutex); } } =20 --=20 2.31.1 From nobody Thu May 9 06:45:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1633349571969202.19117753748242; Mon, 4 Oct 2021 05:12:51 -0700 (PDT) Received: from localhost ([::1]:44142 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXMpu-00089l-Se for importer@patchew.org; Mon, 04 Oct 2021 08:12:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44182) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMgY-0001w3-F6 for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:03:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:50135) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXMgV-0006Nh-Jk for qemu-devel@nongnu.org; Mon, 04 Oct 2021 08:03:10 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-297-OHvYf5mqMtanAssbt2L5Ow-1; Mon, 04 Oct 2021 08:03:06 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9C0D381426F; Mon, 4 Oct 2021 12:03:01 +0000 (UTC) Received: from t480s.redhat.com (unknown [10.39.194.159]) by smtp.corp.redhat.com (Postfix) with ESMTP id 692CF19C59; Mon, 4 Oct 2021 12:02:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633348987; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5OgDf6BrqOXrYlJwSdM6+2Enyt2bTUEUbFrjj4pWhyM=; b=Wl3S6OtSBbKoWbIsN0NgTPSE4jFpsryPwdkjYCaaRSjkD625/SYOctcB3qqLu5WKys8djV EMoDHLmFGq4ANWC3ayPLsV/5+osfT9WR8J46bajKlATO/NQH37N/4eiyBFCjKI6aQjr0YO eaHy5W7qVVHkM/dcF3Zk4tOhwa8n3W4= X-MC-Unique: OHvYf5mqMtanAssbt2L5Ow-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 7/7] util/oslib-posix: Forward SIGBUS to MCE handler under Linux Date: Mon, 4 Oct 2021 14:02:08 +0200 Message-Id: <20211004120208.7409-8-david@redhat.com> In-Reply-To: <20211004120208.7409-1-david@redhat.com> References: <20211004120208.7409-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Pankaj Gupta , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Igor Mammedov , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1633349573660100001 Temporarily modifying the SIGBUS handler is really nasty, as we might be unlucky and receive an MCE SIGBUS while having our handler registered. Unfortunately, there is no way around messing with SIGBUS when MADV_POPULATE_WRITE is not applicable or not around. Let's forward SIGBUS that don't belong to us to the already registered handler and document the situation. Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: David Hildenbrand --- softmmu/cpus.c | 4 ++++ util/oslib-posix.c | 36 +++++++++++++++++++++++++++++++++--- 2 files changed, 37 insertions(+), 3 deletions(-) diff --git a/softmmu/cpus.c b/softmmu/cpus.c index 071085f840..23bca46b07 100644 --- a/softmmu/cpus.c +++ b/softmmu/cpus.c @@ -352,6 +352,10 @@ static void qemu_init_sigbus(void) { struct sigaction action; =20 + /* + * ALERT: when modifying this, take care that SIGBUS forwarding in + * os_mem_prealloc() will continue working as expected. + */ memset(&action, 0, sizeof(action)); action.sa_flags =3D SA_SIGINFO; action.sa_sigaction =3D sigbus_handler; diff --git a/util/oslib-posix.c b/util/oslib-posix.c index 9829149e4b..5c47aa9cb7 100644 --- a/util/oslib-posix.c +++ b/util/oslib-posix.c @@ -95,6 +95,7 @@ typedef struct MemsetThread MemsetThread; =20 /* used by sigbus_handler() */ static MemsetContext *sigbus_memset_context; +struct sigaction sigbus_oldact; static QemuMutex sigbus_mutex; =20 static QemuMutex page_mutex; @@ -446,7 +447,11 @@ const char *qemu_get_exec_dir(void) return exec_dir; } =20 +#ifdef CONFIG_LINUX +static void sigbus_handler(int signal, siginfo_t *siginfo, void *ctx) +#else /* CONFIG_LINUX */ static void sigbus_handler(int signal) +#endif /* CONFIG_LINUX */ { int i; =20 @@ -459,6 +464,26 @@ static void sigbus_handler(int signal) } } } + +#ifdef CONFIG_LINUX + /* + * We assume that the MCE SIGBUS handler could have been registered. We + * should never receive BUS_MCEERR_AO on any of our threads, but only = on + * the main thread registered for PR_MCE_KILL_EARLY. Further, we shoul= d not + * receive BUS_MCEERR_AR triggered by action of other threads on one of + * our threads. So, no need to check for unrelated SIGBUS when seeing = one + * for our threads. + * + * We will forward to the MCE handler, which will either handle the SI= GBUS + * or reinstall the default SIGBUS handler and reraise the SIGBUS. The + * default SIGBUS handler will crash the process, so we don't care. + */ + if (sigbus_oldact.sa_flags & SA_SIGINFO) { + sigbus_oldact.sa_sigaction(signal, siginfo, ctx); + return; + } +#endif /* CONFIG_LINUX */ + warn_report("os_mem_prealloc: unrelated SIGBUS detected and ignored"); } =20 static void *do_touch_pages(void *arg) @@ -628,10 +653,10 @@ void os_mem_prealloc(int fd, char *area, size_t memor= y, int smp_cpus, { static gsize initialized; int ret; - struct sigaction act, oldact; size_t hpagesize =3D qemu_fd_getpagesize(fd); size_t numpages =3D DIV_ROUND_UP(memory, hpagesize); bool use_madv_populate_write; + struct sigaction act; =20 /* * Sense on every invocation, as MADV_POPULATE_WRITE cannot be used for @@ -647,10 +672,15 @@ void os_mem_prealloc(int fd, char *area, size_t memor= y, int smp_cpus, =20 qemu_mutex_lock(&sigbus_mutex); memset(&act, 0, sizeof(act)); +#ifdef CONFIG_LINUX + act.sa_sigaction =3D &sigbus_handler; + act.sa_flags =3D SA_SIGINFO; +#else /* CONFIG_LINUX */ act.sa_handler =3D &sigbus_handler; act.sa_flags =3D 0; +#endif /* CONFIG_LINUX */ =20 - ret =3D sigaction(SIGBUS, &act, &oldact); + ret =3D sigaction(SIGBUS, &act, &sigbus_oldact); if (ret) { error_setg_errno(errp, errno, "os_mem_prealloc: failed to install signal handler"); @@ -667,7 +697,7 @@ void os_mem_prealloc(int fd, char *area, size_t memory,= int smp_cpus, } =20 if (!use_madv_populate_write) { - ret =3D sigaction(SIGBUS, &oldact, NULL); + ret =3D sigaction(SIGBUS, &sigbus_oldact, NULL); if (ret) { /* Terminate QEMU since it can't recover from error */ perror("os_mem_prealloc: failed to reinstall signal handler"); --=20 2.31.1