From: Tiwei Bie <tiwei.btw@antgroup.com>
Turn signals_enabled, signals_pending and signals_active into
thread-local variables. This enables us to control and track
signals independently on each CPU thread. This is a preparation
for adding SMP support.
Signed-off-by: Tiwei Bie <tiwei.btw@antgroup.com>
---
arch/um/include/asm/irqflags.h | 4 ++--
arch/um/include/shared/longjmp.h | 3 +--
arch/um/include/shared/os.h | 1 +
arch/um/kernel/ksyms.c | 2 +-
arch/um/os-Linux/signal.c | 11 ++++++++---
5 files changed, 13 insertions(+), 8 deletions(-)
diff --git a/arch/um/include/asm/irqflags.h b/arch/um/include/asm/irqflags.h
index 1e69ef5bc35e..31e49e0894c5 100644
--- a/arch/um/include/asm/irqflags.h
+++ b/arch/um/include/asm/irqflags.h
@@ -2,7 +2,7 @@
#ifndef __UM_IRQFLAGS_H
#define __UM_IRQFLAGS_H
-extern int signals_enabled;
+int um_get_signals(void);
int um_set_signals(int enable);
void block_signals(void);
void unblock_signals(void);
@@ -10,7 +10,7 @@ void unblock_signals(void);
#define arch_local_save_flags arch_local_save_flags
static inline unsigned long arch_local_save_flags(void)
{
- return signals_enabled;
+ return um_get_signals();
}
#define arch_local_irq_restore arch_local_irq_restore
diff --git a/arch/um/include/shared/longjmp.h b/arch/um/include/shared/longjmp.h
index 8863319039f3..c53e43d980c8 100644
--- a/arch/um/include/shared/longjmp.h
+++ b/arch/um/include/shared/longjmp.h
@@ -5,7 +5,6 @@
#include <sysdep/archsetjmp.h>
#include <os.h>
-extern int signals_enabled;
extern int setjmp(jmp_buf);
extern void longjmp(jmp_buf, int);
@@ -15,7 +14,7 @@ extern void longjmp(jmp_buf, int);
#define UML_SETJMP(buf) ({ \
int n, enable; \
- enable = *(volatile int *)&signals_enabled; \
+ enable = um_get_signals(); \
n = setjmp(*buf); \
if(n != 0) \
um_set_signals_trace(enable); \
diff --git a/arch/um/include/shared/os.h b/arch/um/include/shared/os.h
index b35cc8ce333b..324d4eed3385 100644
--- a/arch/um/include/shared/os.h
+++ b/arch/um/include/shared/os.h
@@ -243,6 +243,7 @@ extern void send_sigio_to_self(void);
extern int change_sig(int signal, int on);
extern void block_signals(void);
extern void unblock_signals(void);
+extern int um_get_signals(void);
extern int um_set_signals(int enable);
extern int um_set_signals_trace(int enable);
extern void deliver_alarm(void);
diff --git a/arch/um/kernel/ksyms.c b/arch/um/kernel/ksyms.c
index f2fb77da08cf..96314c31e61c 100644
--- a/arch/um/kernel/ksyms.c
+++ b/arch/um/kernel/ksyms.c
@@ -6,8 +6,8 @@
#include <linux/module.h>
#include <os.h>
+EXPORT_SYMBOL(um_get_signals);
EXPORT_SYMBOL(um_set_signals);
-EXPORT_SYMBOL(signals_enabled);
EXPORT_SYMBOL(os_stat_fd);
EXPORT_SYMBOL(os_stat_file);
diff --git a/arch/um/os-Linux/signal.c b/arch/um/os-Linux/signal.c
index 11f07f498270..58da8c6ece98 100644
--- a/arch/um/os-Linux/signal.c
+++ b/arch/um/os-Linux/signal.c
@@ -68,12 +68,12 @@ static void sig_handler_common(int sig, struct siginfo *si, mcontext_t *mc)
#define SIGCHLD_BIT 2
#define SIGCHLD_MASK (1 << SIGCHLD_BIT)
-int signals_enabled;
+static __thread int signals_enabled;
#if IS_ENABLED(CONFIG_UML_TIME_TRAVEL_SUPPORT)
static int signals_blocked, signals_blocked_pending;
#endif
-static unsigned int signals_pending;
-static unsigned int signals_active = 0;
+static __thread unsigned int signals_pending;
+static __thread unsigned int signals_active;
static void sig_handler(int sig, struct siginfo *si, mcontext_t *mc)
{
@@ -342,6 +342,11 @@ void unblock_signals(void)
}
}
+int um_get_signals(void)
+{
+ return signals_enabled;
+}
+
int um_set_signals(int enable)
{
int ret;
--
2.34.1On Sun, 2025-08-10 at 13:51 +0800, Tiwei Bie wrote: > From: Tiwei Bie <tiwei.btw@antgroup.com> > > Turn signals_enabled, signals_pending and signals_active into > thread-local variables. This enables us to control and track > signals independently on each CPU thread. This is a preparation > for adding SMP support. [...] > +static __thread int signals_enabled; How much glibc infrastructure does __thread rely on? More specifically: Some time ago we had a discussion about building UML as a nolibc binary, what would that mean for the __thread usage here? johannes
On Wed, 10 Sep 2025 14:15:28 +0200, Johannes Berg wrote: > On Sun, 2025-08-10 at 13:51 +0800, Tiwei Bie wrote: > > From: Tiwei Bie <tiwei.btw@antgroup.com> > > > > Turn signals_enabled, signals_pending and signals_active into > > thread-local variables. This enables us to control and track > > signals independently on each CPU thread. This is a preparation > > for adding SMP support. > > [...] > > > +static __thread int signals_enabled; > > How much glibc infrastructure does __thread rely on? More specifically: > Some time ago we had a discussion about building UML as a nolibc binary, > what would that mean for the __thread usage here? We would need to parse TLS data (PT_TLS) from the ELF file ourselves and properly set up TLS when creating threads using clone(). Regards, Tiwei
Hi, On Thu, 2025-09-11 at 12:34 +0800, Tiwei Bie wrote: > On Wed, 10 Sep 2025 14:15:28 +0200, Johannes Berg wrote: > > On Sun, 2025-08-10 at 13:51 +0800, Tiwei Bie wrote: > > > From: Tiwei Bie <tiwei.btw@antgroup.com> > > > > > > Turn signals_enabled, signals_pending and signals_active into > > > thread-local variables. This enables us to control and track > > > signals independently on each CPU thread. This is a preparation > > > for adding SMP support. > > > > [...] > > > > > +static __thread int signals_enabled; > > > > How much glibc infrastructure does __thread rely on? More > > specifically: > > Some time ago we had a discussion about building UML as a nolibc > > binary, > > what would that mean for the __thread usage here? > > We would need to parse TLS data (PT_TLS) from the ELF file ourselves > and properly set up TLS when creating threads using clone(). I guess right now we cannot use PER_CPU variables in these files. However, my expectation that this is possible when using nolibc, and then it should be simple enough to replace the __thread. Benjamin
On Thu, 2025-09-11 at 09:37 +0200, Benjamin Berg wrote: > Hi, > > On Thu, 2025-09-11 at 12:34 +0800, Tiwei Bie wrote: > > On Wed, 10 Sep 2025 14:15:28 +0200, Johannes Berg wrote: > > > On Sun, 2025-08-10 at 13:51 +0800, Tiwei Bie wrote: > > > > From: Tiwei Bie <tiwei.btw@antgroup.com> > > > > > > > > Turn signals_enabled, signals_pending and signals_active into > > > > thread-local variables. This enables us to control and track > > > > signals independently on each CPU thread. This is a preparation > > > > for adding SMP support. > > > > > > [...] > > > > > > > +static __thread int signals_enabled; > > > > > > How much glibc infrastructure does __thread rely on? More > > > specifically: > > > Some time ago we had a discussion about building UML as a nolibc > > > binary, > > > what would that mean for the __thread usage here? > > > > We would need to parse TLS data (PT_TLS) from the ELF file ourselves > > and properly set up TLS when creating threads using clone(). > > I guess right now we cannot use PER_CPU variables in these files. Maybe? The only thing would be to know which "CPU" we're executing on? getpid() is async signal safe (i.e. you can call it), but there could be better ways of doing this such as setting different signal handler functions in different CPUs. johannes
On Thu, 2025-09-11 at 11:44 +0200, Johannes Berg wrote: > On Thu, 2025-09-11 at 09:37 +0200, Benjamin Berg wrote: > > Hi, > > > > On Thu, 2025-09-11 at 12:34 +0800, Tiwei Bie wrote: > > > On Wed, 10 Sep 2025 14:15:28 +0200, Johannes Berg wrote: > > > > On Sun, 2025-08-10 at 13:51 +0800, Tiwei Bie wrote: > > > > > From: Tiwei Bie <tiwei.btw@antgroup.com> > > > > > > > > > > Turn signals_enabled, signals_pending and signals_active into > > > > > thread-local variables. This enables us to control and track > > > > > signals independently on each CPU thread. This is a preparation > > > > > for adding SMP support. > > > > > > > > [...] > > > > > > > > > +static __thread int signals_enabled; > > > > > > > > How much glibc infrastructure does __thread rely on? More > > > > specifically: > > > > Some time ago we had a discussion about building UML as a nolibc > > > > binary, > > > > what would that mean for the __thread usage here? > > > > > > We would need to parse TLS data (PT_TLS) from the ELF file ourselves > > > and properly set up TLS when creating threads using clone(). > > > > I guess right now we cannot use PER_CPU variables in these files. > > Maybe? The only thing would be to know which "CPU" we're executing on? > getpid() is async signal safe (i.e. you can call it), but there could be > better ways of doing this such as setting different signal handler > functions in different CPUs. There are a number of ways to solve that problem. One way would be to use a signal stack and calculating it from the stack pointer. Though I am still considering that we should use the tasks stack as the signal stack, in which case we may need to track the CPU that a task is executing on. On 64bit, one could also use the FS/GS registers for per-CPU data. I believe the libc uses FS only on 64bit, so we could probably already use the GS register to for per-CPU data. So, I am not really worried about this, we probably need a nice solution for per-CPU data anyway. Benjamin
Hi, On Thu, 2025-09-11 at 09:37 +0200, Benjamin Berg wrote: > Hi, > > On Thu, 2025-09-11 at 12:34 +0800, Tiwei Bie wrote: > > On Wed, 10 Sep 2025 14:15:28 +0200, Johannes Berg wrote: > > > On Sun, 2025-08-10 at 13:51 +0800, Tiwei Bie wrote: > > > > From: Tiwei Bie <tiwei.btw@antgroup.com> > > > > > > > > Turn signals_enabled, signals_pending and signals_active into > > > > thread-local variables. This enables us to control and track > > > > signals independently on each CPU thread. This is a preparation > > > > for adding SMP support. > > > > > > [...] > > > > > > > +static __thread int signals_enabled; > > > > > > How much glibc infrastructure does __thread rely on? More > > > specifically: > > > Some time ago we had a discussion about building UML as a nolibc > > > binary, > > > what would that mean for the __thread usage here? > > > > We would need to parse TLS data (PT_TLS) from the ELF file > > ourselves > > and properly set up TLS when creating threads using clone(). > > I guess right now we cannot use PER_CPU variables in these files. > However, my expectation that this is possible when using nolibc, and > then it should be simple enough to replace the __thread. That said, I do believe that the allocations from the libc itself are problematic. A lot of the mappings from UML are there already (i.e. the physical memory is mapped). However, I believe the vmalloc area for example is not guarded. So when pthread allocates the thread specific memory (stack, TLS, ...), we really do not know where this will be mapped into the address space. If it happens to be in an area that UML wants to use later, then UML could map e.g. vmalloc data over it. Now, it could be that (currently) the addresses picked by pthread (or the host kernel) do not actually clash with anything. However, I do not think there is any guarantee for that. Benjamin
Hi, On Thu, 11 Sep 2025 10:06:53 +0200, Benjamin Berg wrote: > On Thu, 2025-09-11 at 09:37 +0200, Benjamin Berg wrote: > > On Thu, 2025-09-11 at 12:34 +0800, Tiwei Bie wrote: > > > On Wed, 10 Sep 2025 14:15:28 +0200, Johannes Berg wrote: > > > > On Sun, 2025-08-10 at 13:51 +0800, Tiwei Bie wrote: > > > > > From: Tiwei Bie <tiwei.btw@antgroup.com> > > > > > > > > > > Turn signals_enabled, signals_pending and signals_active into > > > > > thread-local variables. This enables us to control and track > > > > > signals independently on each CPU thread. This is a preparation > > > > > for adding SMP support. > > > > > > > > [...] > > > > > > > > > +static __thread int signals_enabled; > > > > > > > > How much glibc infrastructure does __thread rely on? More > > > > specifically: > > > > Some time ago we had a discussion about building UML as a nolibc > > > > binary, > > > > what would that mean for the __thread usage here? > > > > > > We would need to parse TLS data (PT_TLS) from the ELF file > > > ourselves > > > and properly set up TLS when creating threads using clone(). > > > > I guess right now we cannot use PER_CPU variables in these files. > > However, my expectation that this is possible when using nolibc, and > > then it should be simple enough to replace the __thread. Good idea! > > That said, I do believe that the allocations from the libc itself are > problematic. A lot of the mappings from UML are there already (i.e. the > physical memory is mapped). However, I believe the vmalloc area for > example is not guarded. > > So when pthread allocates the thread specific memory (stack, TLS, ...), > we really do not know where this will be mapped into the address space. > If it happens to be in an area that UML wants to use later, then UML > could map e.g. vmalloc data over it. > > Now, it could be that (currently) the addresses picked by pthread (or > the host kernel) do not actually clash with anything. However, I do not > think there is any guarantee for that. Indeed. The mmap from libc (pthread, shared libs, ...) can potentially conflict with UML. The reason it has been working on x86_64 so far might be that we did this in linux_main(): task_size = task_size & PGDIR_MASK; The current layout is: shared libs and pthreads are located at 7ffxxxxxxxxx TASK_SIZE = 7f8000000000 VMALLOC_END = 7f7fffffe000 (which is TASK_SIZE-2*PAGE_SIZE) However, on i386, the risk of conflicts looks much higher: TASK_SIZE = ffc00000 VMALLOC_END = ffbfe000 ...... f7c00000-f7c20000 r--p 00000000 08:01 9114 /usr/lib32/libc.so.6 f7c20000-f7d9e000 r-xp 00020000 08:01 9114 /usr/lib32/libc.so.6 f7d9e000-f7e23000 r--p 0019e000 08:01 9114 /usr/lib32/libc.so.6 f7e23000-f7e24000 ---p 00223000 08:01 9114 /usr/lib32/libc.so.6 f7e24000-f7e26000 r--p 00223000 08:01 9114 /usr/lib32/libc.so.6 f7e26000-f7e27000 rw-p 00225000 08:01 9114 /usr/lib32/libc.so.6 f7e27000-f7e31000 rw-p 00000000 00:00 0 f7fbe000-f7fc0000 rw-p 00000000 00:00 0 f7fc0000-f7fc4000 r--p 00000000 00:00 0 [vvar] f7fc4000-f7fc6000 r-xp 00000000 00:00 0 [vdso] f7fc6000-f7fc7000 r--p 00000000 08:01 9107 /usr/lib32/ld-linux.so.2 f7fc7000-f7fec000 r-xp 00001000 08:01 9107 /usr/lib32/ld-linux.so.2 f7fec000-f7ffb000 r--p 00026000 08:01 9107 /usr/lib32/ld-linux.so.2 f7ffb000-f7ffd000 r--p 00034000 08:01 9107 /usr/lib32/ld-linux.so.2 f7ffd000-f7ffe000 rw-p 00036000 08:01 9107 /usr/lib32/ld-linux.so.2 fffdd000-ffffe000 rw-p 00000000 00:00 0 [stack] Ideally, we could completely eliminate the dependency on libc. Before that, perhaps we could reserve a region of address space for UML with mmap(PROT_NONE). Regards, Tiwei
On Fri, 2025-09-12 at 08:30 +0800, Tiwei Bie wrote: > On Thu, 11 Sep 2025 10:06:53 +0200, Benjamin Berg wrote: > > > [SNIP] > > That said, I do believe that the allocations from the libc itself are > > problematic. A lot of the mappings from UML are there already (i.e. the > > physical memory is mapped). However, I believe the vmalloc area for > > example is not guarded. > > > > So when pthread allocates the thread specific memory (stack, TLS, ...), > > we really do not know where this will be mapped into the address space. > > If it happens to be in an area that UML wants to use later, then UML > > could map e.g. vmalloc data over it. > > > > Now, it could be that (currently) the addresses picked by pthread (or > > the host kernel) do not actually clash with anything. However, I do not > > think there is any guarantee for that. > > Indeed. The mmap from libc (pthread, shared libs, ...) can potentially > conflict with UML. The reason it has been working on x86_64 so far might > be that we did this in linux_main(): > > task_size = task_size & PGDIR_MASK; > > The current layout is: > > shared libs and pthreads are located at 7ffxxxxxxxxx > TASK_SIZE = 7f8000000000 > VMALLOC_END = 7f7fffffe000 (which is TASK_SIZE-2*PAGE_SIZE) Uh, right, yes. Because of the masking we are capping ourselves to 0x7f8000000000. And then all of the interesting bits (vdso, ...) happen to be mapped above that address and are effectively protected. And, there is also plenty of space for other allocations technically. That is kind of horrible, as it only works because all of this happens to be mapped into the top of the address space. But, maybe something to just wilfully ignore and only fix as part of a nolibc port? > However, on i386, the risk of conflicts looks much higher: > > TASK_SIZE = ffc00000 > VMALLOC_END = ffbfe000 > > ...... > f7c00000-f7c20000 r--p 00000000 08:01 9114 /usr/lib32/libc.so.6 > f7c20000-f7d9e000 r-xp 00020000 08:01 9114 /usr/lib32/libc.so.6 > f7d9e000-f7e23000 r--p 0019e000 08:01 9114 /usr/lib32/libc.so.6 > f7e23000-f7e24000 ---p 00223000 08:01 9114 /usr/lib32/libc.so.6 > f7e24000-f7e26000 r--p 00223000 08:01 9114 /usr/lib32/libc.so.6 > f7e26000-f7e27000 rw-p 00225000 08:01 9114 /usr/lib32/libc.so.6 > f7e27000-f7e31000 rw-p 00000000 00:00 0 > f7fbe000-f7fc0000 rw-p 00000000 00:00 0 > f7fc0000-f7fc4000 r--p 00000000 00:00 0 [vvar] > f7fc4000-f7fc6000 r-xp 00000000 00:00 0 [vdso] > f7fc6000-f7fc7000 r--p 00000000 08:01 9107 /usr/lib32/ld-linux.so.2 > f7fc7000-f7fec000 r-xp 00001000 08:01 9107 /usr/lib32/ld-linux.so.2 > f7fec000-f7ffb000 r--p 00026000 08:01 9107 /usr/lib32/ld-linux.so.2 > f7ffb000-f7ffd000 r--p 00034000 08:01 9107 /usr/lib32/ld-linux.so.2 > f7ffd000-f7ffe000 rw-p 00036000 08:01 9107 /usr/lib32/ld-linux.so.2 > fffdd000-ffffe000 rw-p 00000000 00:00 0 [stack] > > Ideally, we could completely eliminate the dependency on libc. Before that, > perhaps we could reserve a region of address space for UML with mmap(PROT_NONE). Yeah, that does seem reasonable. That should at least protect us from libc using our vmalloc area. And it is easy to do, as it just needs an initial mmap and changing the kern_unmap implementation in tlb.c Benjamin
On Fri, 12 Sep 2025 09:58:49 +0200, Benjamin Berg wrote: > On Fri, 2025-09-12 at 08:30 +0800, Tiwei Bie wrote: > > On Thu, 11 Sep 2025 10:06:53 +0200, Benjamin Berg wrote: > > > > > [SNIP] > > > That said, I do believe that the allocations from the libc itself are > > > problematic. A lot of the mappings from UML are there already (i.e. the > > > physical memory is mapped). However, I believe the vmalloc area for > > > example is not guarded. > > > > > > So when pthread allocates the thread specific memory (stack, TLS, ...), > > > we really do not know where this will be mapped into the address space. > > > If it happens to be in an area that UML wants to use later, then UML > > > could map e.g. vmalloc data over it. > > > > > > Now, it could be that (currently) the addresses picked by pthread (or > > > the host kernel) do not actually clash with anything. However, I do not > > > think there is any guarantee for that. > > > > Indeed. The mmap from libc (pthread, shared libs, ...) can potentially > > conflict with UML. The reason it has been working on x86_64 so far might > > be that we did this in linux_main(): > > > > task_size = task_size & PGDIR_MASK; > > > > The current layout is: > > > > shared libs and pthreads are located at 7ffxxxxxxxxx > > TASK_SIZE = 7f8000000000 > > VMALLOC_END = 7f7fffffe000 (which is TASK_SIZE-2*PAGE_SIZE) > > Uh, right, yes. Because of the masking we are capping ourselves to > 0x7f8000000000. And then all of the interesting bits (vdso, ...) happen > to be mapped above that address and are effectively protected. And, > there is also plenty of space for other allocations technically. > > That is kind of horrible, as it only works because all of this happens > to be mapped into the top of the address space. +1. > But, maybe something to > just wilfully ignore and only fix as part of a nolibc port? Sure. Thanks! Regards, Tiwei
© 2016 - 2025 Red Hat, Inc.