When O_TMPFILE is set, the open mode needs to be passed to the kernel as
per the documentation.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
Thomas Weißschuh (4):
      tools/nolibc: split implicit open flags into a macro
      tools/nolibc: split open mode handling into a macro
      tools/nolibc: pass mode to open syscall if O_TMPFILE is set
      selftests/nolibc: test open mode handling

 tools/include/nolibc/fcntl.h                 | 46 ++++++++++++----------------
 tools/testing/selftests/nolibc/nolibc-test.c | 23 ++++++++++++++
 2 files changed, 43 insertions(+), 26 deletions(-)
---
base-commit: 266df86c4893fed1a7f027e767fe1c7f6456b100
change-id: 20260429-nolibc-open-tmpfile-19e677e33c8f

Best regards,
--  
Thomas Weißschuh <linux@weissschuh.net>

Hi Thomas,

On Wed, Apr 29, 2026 at 04:45:08PM +0200, Thomas Weißschuh wrote:
> When O_TMPFILE is set, the open mode needs to be passed to the kernel as
> per the documentation.
> 
> Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
> ---
> Thomas Weißschuh (4):
>       tools/nolibc: split implicit open flags into a macro
>       tools/nolibc: split open mode handling into a macro
>       tools/nolibc: pass mode to open syscall if O_TMPFILE is set
>       selftests/nolibc: test open mode handling
> 
>  tools/include/nolibc/fcntl.h                 | 46 ++++++++++++----------------
>  tools/testing/selftests/nolibc/nolibc-test.c | 23 ++++++++++++++
>  2 files changed, 43 insertions(+), 26 deletions(-)

This is a clever approach, and it can definitely work.

Acked-by: Willy Tarreau <w@1wt.eu>

I'm wondering why we're looking at the mode only for certain flags
though. After all, the kernel is supposed to look at it when it needs.
Why wouldn't we just always pass the va_args and let the syscall refer
to it when it needs ? If it's a register, it's implicitly passed anyway,
and if it's in the stack, it's only one entry, it's the same as if someone
calls open() with one of these flags without passing the argument. Any
garbage it contains should be ignored by the kernel when not needed, so
I don't see why it would be a problem.

Cheers,
Willy

On 2026-05-01 10:08:45+0200, Willy Tarreau wrote:
> Hi Thomas,
> 
> On Wed, Apr 29, 2026 at 04:45:08PM +0200, Thomas Weißschuh wrote:
> > When O_TMPFILE is set, the open mode needs to be passed to the kernel as
> > per the documentation.
> > 
> > Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
> > ---
> > Thomas Weißschuh (4):
> >       tools/nolibc: split implicit open flags into a macro
> >       tools/nolibc: split open mode handling into a macro
> >       tools/nolibc: pass mode to open syscall if O_TMPFILE is set
> >       selftests/nolibc: test open mode handling
> > 
> >  tools/include/nolibc/fcntl.h                 | 46 ++++++++++++----------------
> >  tools/testing/selftests/nolibc/nolibc-test.c | 23 ++++++++++++++
> >  2 files changed, 43 insertions(+), 26 deletions(-)
> 
> This is a clever approach, and it can definitely work.
> 
> Acked-by: Willy Tarreau <w@1wt.eu>

Thanks!

> I'm wondering why we're looking at the mode only for certain flags
> though. After all, the kernel is supposed to look at it when it needs.
> Why wouldn't we just always pass the va_args and let the syscall refer
> to it when it needs ? If it's a register, it's implicitly passed anyway,
> and if it's in the stack, it's only one entry, it's the same as if someone
> calls open() with one of these flags without passing the argument. Any
> garbage it contains should be ignored by the kernel when not needed, so
> I don't see why it would be a problem.

I think there are some architectures with trap representations for
register values. There that aproach could fail. However these should be
irrelevant for nolibc. I'll try your suggestion for the next revision.


Thomas