From: Bobby Eshleman <bobbyeshleman@meta.com>

When running vmtest.sh inside a nested VM, there occurs a problem
with stacking two sets of virtiofs/overlay layers (the first set from
the outer VM and the second set from the inner VM). The virtme init
scripts (sshd, udhcpd, etc...) fail to execute basic programs (e.g.,
/bin/cat) and load library dependencies (e.g., libpam) due to ESTALE.
This only occurs when both layers (outer and inner) use virtiofs. Work
around this by using 9p in the inner VM via --force-9p.

Additionally, when the outer VM is read-only, the inner VM's attempt at
populating SSH keys to the root filesystem fails:

virtme-ng-init: mkdir: cannot create directory '/root/.cache': Read-only file system

Work around this by creating a temporary home directory with generated
SSH keys and passing it through to the guest as /root via --rwdir.
Disable strict host key checking in vm_ssh() since the VM will be seen
as a new host each run.

The --rw arg had to be removed to prevent a vng complaint about overlay
(in combination with the other parameters). The guest doesn't really
need write access anyway, so this was probably overly permissive to
begin with.

Signed-off-by: Bobby Eshleman <bobbyeshleman@meta.com>
---
 tools/testing/selftests/vsock/vmtest.sh | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/vsock/vmtest.sh b/tools/testing/selftests/vsock/vmtest.sh
index 86e338886b33..c2cfcdf05d99 100755
--- a/tools/testing/selftests/vsock/vmtest.sh
+++ b/tools/testing/selftests/vsock/vmtest.sh
@@ -42,6 +42,8 @@ readonly KERNEL_CMDLINE="\
 	virtme.ssh virtme_ssh_channel=tcp virtme_ssh_user=$USER \
 "
 readonly LOG=$(mktemp /tmp/vsock_vmtest_XXXX.log)
+readonly TEST_HOME=$(mktemp -d /tmp/vmtest_home_XXXX)
+readonly SSH_KEY_PATH="${TEST_HOME}"/.ssh/id_ed25519
 
 # Namespace tests must use the ns_ prefix. This is checked in check_netns() and
 # is used to determine if a test needs namespace setup before test execution.
@@ -257,7 +259,12 @@ vm_ssh() {
 
 	shift
 
-	${ns_exec} ssh -q -o UserKnownHostsFile=/dev/null -p "${SSH_HOST_PORT}" localhost "$@"
+	${ns_exec} ssh -q \
+		-i "${SSH_KEY_PATH}" \
+		-o UserKnownHostsFile=/dev/null \
+		-o StrictHostKeyChecking=no \
+		-p "${SSH_HOST_PORT}" \
+		localhost "$@"
 
 	return $?
 }
@@ -265,6 +272,7 @@ vm_ssh() {
 cleanup() {
 	terminate_pidfiles "${!PIDFILES[@]}"
 	del_namespaces
+	rm -rf "${TEST_HOME}"
 }
 
 check_args() {
@@ -382,6 +390,11 @@ handle_build() {
 	popd &>/dev/null
 }
 
+setup_home() {
+	mkdir -p "$(dirname "${SSH_KEY_PATH}")"
+	ssh-keygen -t ed25519 -f "${SSH_KEY_PATH}" -N "" -q
+}
+
 create_pidfile() {
 	local pidfile
 
@@ -451,11 +464,14 @@ vm_start() {
 		--run \
 		${kernel_opt} \
 		${verbose_opt} \
+		--rwdir=/root=${TEST_HOME} \
+		--force-9p \
+		--cwd /root \
 		--qemu-opts="${qemu_opts}" \
 		--qemu="${qemu}" \
 		--user root \
 		--append "${KERNEL_CMDLINE}" \
-		--rw  &> ${logfile} &
+		&> ${logfile} &
 
 	timeout "${WAIT_QEMU}" \
 		bash -c 'while [[ ! -s '"${pidfile}"' ]]; do sleep 1; done; exit 0'
@@ -1532,6 +1548,7 @@ check_deps
 check_vng
 check_socat
 handle_build
+setup_home
 
 echo "1..${#ARGS[@]}"
 

-- 
2.52.0

On 3/14/26 1:00 AM, Bobby Eshleman wrote:
>  tools/testing/selftests/vsock/vmtest.sh | 21 +++++++++++++++++++--
>  1 file changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/testing/selftests/vsock/vmtest.sh b/tools/testing/selftests/vsock/vmtest.sh
> index 86e338886b33..c2cfcdf05d99 100755
> --- a/tools/testing/selftests/vsock/vmtest.sh
> +++ b/tools/testing/selftests/vsock/vmtest.sh
> @@ -42,6 +42,8 @@ readonly KERNEL_CMDLINE="\
>  	virtme.ssh virtme_ssh_channel=tcp virtme_ssh_user=$USER \
>  "
>  readonly LOG=$(mktemp /tmp/vsock_vmtest_XXXX.log)
> +readonly TEST_HOME=$(mktemp -d /tmp/vmtest_home_XXXX)

shellcheck complains:

In vmtest.sh line 45:
readonly TEST_HOME=$(mktemp -d /tmp/vmtest_home_XXXX)
         ^-------^ SC2155 (warning): Declare and assign separately to
avoid masking return values.

> @@ -451,11 +464,14 @@ vm_start() {
>  		--run \
>  		${kernel_opt} \
>  		${verbose_opt} \
> +		--rwdir=/root=${TEST_HOME} \

And here, too:

In vmtest.sh line 467:
		--rwdir=/root=${TEST_HOME} \
                              ^----------^ SC2086 (info): Double quote
to prevent globbing and word splitting.

Did you mean:
		--rwdir=/root="${TEST_HOME}" \

The first occurrence could possibly be explicitly silenced explicitly,
but it's probably better to really address the latter.

/P

On Tue, Mar 17, 2026 at 03:46:35PM +0100, Paolo Abeni wrote:
> On 3/14/26 1:00 AM, Bobby Eshleman wrote:
> >  tools/testing/selftests/vsock/vmtest.sh | 21 +++++++++++++++++++--
> >  1 file changed, 19 insertions(+), 2 deletions(-)
> > 
> > diff --git a/tools/testing/selftests/vsock/vmtest.sh b/tools/testing/selftests/vsock/vmtest.sh
> > index 86e338886b33..c2cfcdf05d99 100755
> > --- a/tools/testing/selftests/vsock/vmtest.sh
> > +++ b/tools/testing/selftests/vsock/vmtest.sh
> > @@ -42,6 +42,8 @@ readonly KERNEL_CMDLINE="\
> >  	virtme.ssh virtme_ssh_channel=tcp virtme_ssh_user=$USER \
> >  "
> >  readonly LOG=$(mktemp /tmp/vsock_vmtest_XXXX.log)
> > +readonly TEST_HOME=$(mktemp -d /tmp/vmtest_home_XXXX)
> 
> shellcheck complains:
> 
> In vmtest.sh line 45:
> readonly TEST_HOME=$(mktemp -d /tmp/vmtest_home_XXXX)
>          ^-------^ SC2155 (warning): Declare and assign separately to
> avoid masking return values.
> 
> > @@ -451,11 +464,14 @@ vm_start() {
> >  		--run \
> >  		${kernel_opt} \
> >  		${verbose_opt} \
> > +		--rwdir=/root=${TEST_HOME} \
> 
> And here, too:
> 
> In vmtest.sh line 467:
> 		--rwdir=/root=${TEST_HOME} \
>                               ^----------^ SC2086 (info): Double quote
> to prevent globbing and word splitting.
> 
> Did you mean:
> 		--rwdir=/root="${TEST_HOME}" \
> 
> The first occurrence could possibly be explicitly silenced explicitly,
> but it's probably better to really address the latter.
> 
> /P
> 
> 

That makes sense, fixing both and resending now.

Thanks,
Bobby