Problem: Before starting the enumeration of the supported formats the
default return value is set to no entity which is misleading since if
the kernel module of the binary format is not found, it would return no
entity to user mode which is misleading since it is signaling that a
file was not found, but in this case the more suitable error is that the
executable has an unsupported format

Solution: Refactor to return no-exec error instead

Signed-off-by: Nir Lichtman <nir@lichtman.org>
---
 fs/exec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/exec.c b/fs/exec.c
index 3394de5882af..6324f9546b09 100644
--- a/fs/exec.c
+++ b/fs/exec.c
@@ -1740,7 +1740,7 @@ static int search_binary_handler(struct linux_binprm *bprm)
 	if (retval)
 		return retval;
 
-	retval = -ENOENT;
+	retval = -ENOEXEC;
  retry:
 	read_lock(&binfmt_lock);
 	list_for_each_entry(fmt, &formats, lh) {
-- 
2.39.2

On Fri, Nov 15, 2024 at 04:55:41PM +0000, Nir Lichtman wrote:
> Problem: Before starting the enumeration of the supported formats the
> default return value is set to no entity which is misleading since if
> the kernel module of the binary format is not found, it would return no
> entity to user mode which is misleading since it is signaling that a
> file was not found, but in this case the more suitable error is that the
> executable has an unsupported format

Disregard this, I have come to the conclusion that it is unnecessary,
since at least one built-in binary format supported directly in the kernel 
is required for loading kernel modules, this is because the kernel executes
a user mode modprobe to load the module.

Interestingly, placing a "modprobe" in /sbin/modprobe and adding 0
supported formats in the kernel configuration, but adding module support,
results in a loop of calls to the search_binary_handler function,
since it keeps trying to load a module to load "modprobe", it does give up
though at some point and just panics as expected, but it does result in some
visible hold before hand.

Thanks and sorry for the confusion,
Nir

> 
> Solution: Refactor to return no-exec error instead
> 
> Signed-off-by: Nir Lichtman <nir@lichtman.org>
> ---
>  fs/exec.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/fs/exec.c b/fs/exec.c
> index 3394de5882af..6324f9546b09 100644
> --- a/fs/exec.c
> +++ b/fs/exec.c
> @@ -1740,7 +1740,7 @@ static int search_binary_handler(struct linux_binprm *bprm)
>  	if (retval)
>  		return retval;
>  
> -	retval = -ENOENT;
> +	retval = -ENOEXEC;
>   retry:
>  	read_lock(&binfmt_lock);
>  	list_for_each_entry(fmt, &formats, lh) {
> -- 
> 2.39.2
>