> BTW: I can see in QEMU sources /dev/sgx_vepc and /dev/sgx_provision being
> opened, but not sgx_enclave. And I see the former two on my system but not
> the last one. Can you Yang, share more info on this please?

True, QEMU only need read and write access to /dev/sgx_vepc and /dev/sgx_provision. 

/dev/sgx_vepc allows userspace to allocate "raw" EPC without an associated enclave.
The only known use case for raw EPC allocation is to expose EPC to a KVM guest, 
hence call it 'vepc'.

/dev/sgx_enclave allows creating host enclave. It is not suitable for allocating EPC for
KVM guest. Having separate device nodes, /dev/sgx_vepc and /dev/sgx_enclave,
allows separate permission control for creating host SGX enclaves and KVM SGX guests.

/dev/sgx_provision allows creating provisioning enclaves, which typically have more 
strict permissions than the plain enclave device /dev/sgx_enclave.

Usually /dev/sgx_enclave and /dev/sgx_provision should exist together on your system. 
Set "CONFIG_X86_SGX=y" in Kconfig and enable SGX in bios should enable SGX driver
and create /dev/sgx_enclave and /dev/sgx_provision device nodes.
"CONFIG_X86_SGX_KVM=y" will create /dev/sgx_vepc device node.

Regrading to permission control, one suggested way is making /dev/sgx_enclave is
accessible to all userspace applications to create its enclave. Having strict permissions
on /dev/sgx_vepc and /dev/sgx_provision only for user in specific group "XYZ".

# ls -l /dev/sgx*
crw-rw-rw- 1 root root 10, 125 Nov 16  2021 /dev/sgx_enclave
crw-rw---- 1 root XYZ  10, 126 Nov 16  2021 /dev/sgx_provision
crw-rw---- 1 root XYZ  10, 124 Nov 16  2021 /dev/sgx_vepc

Instead of running QEMU by root, one straightforward way is admin create a
dedicated "qemu" user and add it to "XYZ" group. In /etc/libvirt/qemu.conf,

user = "qemu"

Any concerns about this solution?

Thanks,
Lin.

On 7/22/22 08:57, Yang, Lin A wrote:
>> BTW: I can see in QEMU sources /dev/sgx_vepc and /dev/sgx_provision being
>> opened, but not sgx_enclave. And I see the former two on my system but not
>> the last one. Can you Yang, share more info on this please?
> 
> True, QEMU only need read and write access to /dev/sgx_vepc and /dev/sgx_provision. 
> 
> /dev/sgx_vepc allows userspace to allocate "raw" EPC without an associated enclave.
> The only known use case for raw EPC allocation is to expose EPC to a KVM guest, 
> hence call it 'vepc'.
> 
> /dev/sgx_enclave allows creating host enclave. It is not suitable for allocating EPC for
> KVM guest. Having separate device nodes, /dev/sgx_vepc and /dev/sgx_enclave,
> allows separate permission control for creating host SGX enclaves and KVM SGX guests.

I see where the problem is now. My processor lacks X86_FEATURE_SGX_LC
therefore, when SGX driver wants to register /dev/sgx_enclave inside of
sgx_drv_init() (arch/x86/kernel/cpu/sgx/driver.c) the function exits early.

> 
> /dev/sgx_provision allows creating provisioning enclaves, which typically have more 
> strict permissions than the plain enclave device /dev/sgx_enclave.
> 
> Usually /dev/sgx_enclave and /dev/sgx_provision should exist together on your system. 
> Set "CONFIG_X86_SGX=y" in Kconfig and enable SGX in bios should enable SGX driver
> and create /dev/sgx_enclave and /dev/sgx_provision device nodes.
> "CONFIG_X86_SGX_KVM=y" will create /dev/sgx_vepc device node.
> 
> Regrading to permission control, one suggested way is making /dev/sgx_enclave is
> accessible to all userspace applications to create its enclave. Having strict permissions
> on /dev/sgx_vepc and /dev/sgx_provision only for user in specific group "XYZ".
> 
> # ls -l /dev/sgx*
> crw-rw-rw- 1 root root 10, 125 Nov 16  2021 /dev/sgx_enclave
> crw-rw---- 1 root XYZ  10, 126 Nov 16  2021 /dev/sgx_provision
> crw-rw---- 1 root XYZ  10, 124 Nov 16  2021 /dev/sgx_vepc
> 
> Instead of running QEMU by root, one straightforward way is admin create a
> dedicated "qemu" user and add it to "XYZ" group. In /etc/libvirt/qemu.conf,
> 
> user = "qemu"
> 
> Any concerns about this solution?

Well, as discussed with Daniel earlier, libvirt creates a separate mount
namespace for each QEMU and inside it creates a very thin /dev with only
a handful of nodes (per guest config). And what my patch does (and what
we already do for /dev/sev) is mknod() /dev/sgx_provision and
/dev/sgx_vepc inside that thin /dev and chown() it to the user under
which QEMU is about to run.

This namespace feature can be turned off though, in which case libvirt
won't chown() those files (well, my patch is written that way). I think
this is acceptable trade off between security and usability. Namespaces
are enabled by default (if kernel supports them).

Alright, so here's what we'll do. I'll polish my patches, fix up yours
and send for review. Does this work for you?

Michal

> Well, as discussed with Daniel earlier, libvirt creates a separate mount
> namespace for each QEMU and inside it creates a very thin /dev with only a
> handful of nodes (per guest config). And what my patch does (and what we
> already do for /dev/sev) is mknod() /dev/sgx_provision and /dev/sgx_vepc inside
> that thin /dev and chown() it to the user under which QEMU is about to run.
> 
> This namespace feature can be turned off though, in which case libvirt won't
> chown() those files (well, my patch is written that way). I think this is acceptable
> trade off between security and usability. Namespaces are enabled by default (if
> kernel supports them).
> 
> Alright, so here's what we'll do. I'll polish my patches, fix up yours and send for
> review. Does this work for you?

Definitely Yes! This is awesome!
Really appreciated your help.

Good to know libvirt creates separate mount namespace and thin /dev for each
QEMU.

Lin.