Note that I am sending this during the merge window due to the high
level of interest. My current expectation, barring major review
concerns, is that this intercepts Linux-next soon after v6.6-rc1 for a
v6.7 merge. Given the switch to configfs I did not carry forward
Reviewed-by's from v2.

Changes since v2 [1]:
- Switch from sysfs to configfs to scale the interface for containers
  (Jeremi)
- Fix locking in outblob_read() to avoid racing freeing and generation
  of ->outblob (Jeremi)
- Add missing mutex to sev_report_new() (Jeremi)
- Fix incorrect usage of no_free_ptr(), switch to return_ptr() (Peter)
- Drop hex input parsing (configfs bin attributes are not seekable which
  eliminates the concern) (Greg)
- Note why DEFINE_FREE() for kvfree() includes a NULL check (Greg)
- Document the permissible values of privlevel in the ABI documentation
  (Greg)
- Bump column limit to 100 for sev-guest changes, since that's existing
  code, for tsm.c use the .clang_format default. (Tom)
- Drop report buffer size to 4K (Tom)
- Fix uninitialized variable @rc in register_tsm() (Tom)
- Fix collision detection confusion, always increment write_generation
  on successful write regardless if old data is being re-written (Tom)
- Switch to sockptr_t for sharing {get,get_ext}_report() between the
  ioctl and configfs paths (Andy)

[1]: http://lore.kernel.org/r/169199898909.1782217.10899362240465838600.stgit@dwillia2-xfh.jf.intel.com

An attestation report is signed evidence of how a Trusted Virtual
Machine (TVM) was launched and its current state. A verifying party uses
the report to make judgements of the confidentiality and integrity of
that execution environment. Upon successful attestation the verifying
party may, for example, proceed to deploy secrets to the TVM to carry
out a workload. Multiple confidential computing platforms share this
similar flow.

The approach of adding adding new char devs and new ioctls, for what
amounts to the same logical functionality with minor formatting
differences across vendors [2], is untenable. Common concepts and the
community benefit from common infrastructure. 
 
Use configfs for this facility for maintainability compared to ioctl(),
and for its scalability compared to sysfs. Atomicity can be enforced at
item creation time, and a conflict detection mechanism is included for
scenarios where multiple threads may share a single configuration
instance.
 
[2]: http://lore.kernel.org/r/cover.1684048511.git.sathyanarayanan.kuppuswamy@linux.intel.com

---

Dan Williams (5):
      virt: coco: Add a coco/Makefile and coco/Kconfig
      configfs-tsm: Introduce a shared ABI for attestation reports
      virt: sevguest: Prep for kernel internal {get,get_ext}_report()
      mm/slab: Add __free() support for kvfree
      virt: sevguest: Add TSM_REPORTS support for SNP_{GET,GET_EXT}_REPORT


 Documentation/ABI/testing/configfs-tsm  |   68 +++++
 MAINTAINERS                             |    8 +
 drivers/virt/Kconfig                    |    6
 drivers/virt/Makefile                   |    4
 drivers/virt/coco/Kconfig               |   14 +
 drivers/virt/coco/Makefile              |    8 +
 drivers/virt/coco/sev-guest/Kconfig     |    1
 drivers/virt/coco/sev-guest/sev-guest.c |  133 +++++++++--
 drivers/virt/coco/tdx-guest/Kconfig     |    1
 drivers/virt/coco/tsm.c                 |  391 +++++++++++++++++++++++++++++++
 include/linux/slab.h                    |    2
 include/linux/tsm.h                     |   54 ++++
 12 files changed, 665 insertions(+), 25 deletions(-)
 create mode 100644 Documentation/ABI/testing/configfs-tsm
 create mode 100644 drivers/virt/coco/Kconfig
 create mode 100644 drivers/virt/coco/Makefile
 create mode 100644 drivers/virt/coco/tsm.c
 create mode 100644 include/linux/tsm.h

base-commit: 2dde18cd1d8fac735875f2e4987f11817cc0bc2c

On 8/30/2023 9:33 PM, Dan Williams wrote:
> Note that I am sending this during the merge window due to the high
> level of interest. My current expectation, barring major review
> concerns, is that this intercepts Linux-next soon after v6.6-rc1 for a
> v6.7 merge. Given the switch to configfs I did not carry forward
> Reviewed-by's from v2.
> 
> Changes since v2 [1]:
> - Switch from sysfs to configfs to scale the interface for containers
>   (Jeremi)
> - Fix locking in outblob_read() to avoid racing freeing and generation
>   of ->outblob (Jeremi)
> - Add missing mutex to sev_report_new() (Jeremi)
> - Fix incorrect usage of no_free_ptr(), switch to return_ptr() (Peter)
> - Drop hex input parsing (configfs bin attributes are not seekable which
>   eliminates the concern) (Greg)
> - Note why DEFINE_FREE() for kvfree() includes a NULL check (Greg)
> - Document the permissible values of privlevel in the ABI documentation
>   (Greg)
> - Bump column limit to 100 for sev-guest changes, since that's existing
>   code, for tsm.c use the .clang_format default. (Tom)
> - Drop report buffer size to 4K (Tom)
> - Fix uninitialized variable @rc in register_tsm() (Tom)
> - Fix collision detection confusion, always increment write_generation
>   on successful write regardless if old data is being re-written (Tom)
> - Switch to sockptr_t for sharing {get,get_ext}_report() between the
>   ioctl and configfs paths (Andy)
> 
> [1]: http://lore.kernel.org/r/169199898909.1782217.10899362240465838600.stgit@dwillia2-xfh.jf.intel.com
> 
> An attestation report is signed evidence of how a Trusted Virtual
> Machine (TVM) was launched and its current state. A verifying party uses
> the report to make judgements of the confidentiality and integrity of
> that execution environment. Upon successful attestation the verifying
> party may, for example, proceed to deploy secrets to the TVM to carry
> out a workload. Multiple confidential computing platforms share this
> similar flow.

Besides the platform (cpu) attestation report, there are also attestation
reports from individual secure PCIe devices that we'd want to fetch. This
uses the SPDM protocol[1]. There is a CHALLENGE command which (too me)
roughly maps to an attestation request, but also separate interfaces to
fetch individual measurements and certificates (like the SNP extended
report interface allows).

If this is to become the one attestation interface then we'll need to
consider that. That will probably require adding a second level
directory: /sys/kernel/config/tsm/<device path>.

[1]: https://www.dmtf.org/sites/default/files/standards/documents/DSP2058_1.0.0_1.pdf

[ Add Lukas since 'SPDM' was mentioned ]

Jeremi Piotrowski wrote:
[..]
> > An attestation report is signed evidence of how a Trusted Virtual
> > Machine (TVM) was launched and its current state. A verifying party uses
> > the report to make judgements of the confidentiality and integrity of
> > that execution environment. Upon successful attestation the verifying
> > party may, for example, proceed to deploy secrets to the TVM to carry
> > out a workload. Multiple confidential computing platforms share this
> > similar flow.
> 
> Besides the platform (cpu) attestation report, there are also attestation
> reports from individual secure PCIe devices that we'd want to fetch. This
> uses the SPDM protocol[1]. There is a CHALLENGE command which (too me)
> roughly maps to an attestation request, but also separate interfaces to
> fetch individual measurements and certificates (like the SNP extended
> report interface allows).

Yes, but I am not yet convinced this configfs-tsm interface would get
involved there.

> 
> If this is to become the one attestation interface then we'll need to
> consider that. That will probably require adding a second level
> directory: /sys/kernel/config/tsm/<device path>.

The SPDM situation is different in my mind in that the kernel has an
interest in being able to attest a device itself. Think of cases like
power management where userspace is frozen, but the kernel needs to
validate the device in the resume flow.

For TVMs the kernel would validate devices and the verifying party would
validate the kernel as part of the guest measurement.

The main difficulty again here is evidence format differentiation. My
hope is that there is some standardization or otherwise a way to update
the kernel's verification logic for per-device evidence-formats.  Maybe
eBPF has a role to play in that story, but that's a converstation for a
different patch set.

> 
> [1]: https://www.dmtf.org/sites/default/files/standards/documents/DSP2058_1.0.0_1.pdf

On Fri, Sep 01, 2023 at 09:51:42AM -0700, Dan Williams wrote:
> [ Add Lukas since 'SPDM' was mentioned ]
> 
> Jeremi Piotrowski wrote:
> [..]
> > > An attestation report is signed evidence of how a Trusted Virtual
> > > Machine (TVM) was launched and its current state. A verifying party uses
> > > the report to make judgements of the confidentiality and integrity of
> > > that execution environment. Upon successful attestation the verifying
> > > party may, for example, proceed to deploy secrets to the TVM to carry
> > > out a workload. Multiple confidential computing platforms share this
> > > similar flow.
> > 
> > Besides the platform (cpu) attestation report, there are also attestation
> > reports from individual secure PCIe devices that we'd want to fetch. This
> > uses the SPDM protocol[1]. There is a CHALLENGE command which (too me)
> > roughly maps to an attestation request, but also separate interfaces to
> > fetch individual measurements and certificates (like the SNP extended
> > report interface allows).
> 
> Yes, but I am not yet convinced this configfs-tsm interface would get
> involved there.

User space will want to get those devices attestation reports, and those
would be carried through the TSM. It would be nice to be able to use a
common ABI for this. A CPU/platform attestation report is not that
different from a device one.

> > 
> > If this is to become the one attestation interface then we'll need to
> > consider that. That will probably require adding a second level
> > directory: /sys/kernel/config/tsm/<device path>.
> 
> The SPDM situation is different in my mind in that the kernel has an
> interest in being able to attest a device itself. Think of cases like
> power management where userspace is frozen, but the kernel needs to
> validate the device in the resume flow.
> 
> For TVMs the kernel would validate devices

That means the TVM kernel would be provisioned with reference values and
policies that are likely to be tenant specific. The same TVM kernel,
running the same user space stack, getting the same PCIe device
attached, could either accept or reject such device, depending on the
tenant/workload owner policies and acceptable reference values. That
means each tenant would have to build its own guest images, and maintain
and update them with potentially each new device or new device stack it
wants to support.
Keeping most of the device attestation stack (similar to where the
platform attestation stack lives today) in user space seems more
flexible to me, and allows for tenant to use single guest images.

> and the verifying party would
> validate the kernel as part of the guest measurement.
> 
> The main difficulty again here is evidence format differentiation. My
> hope is that there is some standardization

FWIW there are IETF driven standardization efforts like e.g. EAT [1]
that go into the right directions imho. The latest CC implementations
(CCA, CoVE) follow those specs (EAT, CWT/JWT).

DMTF (driving the SPDM spec) defines also its own format, through its
measurement format spec. Device vendors may choose to implement that or
to e.g. add their EAT formatted attestation report in the reported
certificate chain. Realistically, we'll have to support all of those
flows.

> or otherwise a way to update
> the kernel's verification logic for per-device evidence-formats.  Maybe
> eBPF has a role to play in that story, but that's a converstation for a
> different patch set.

This conversation will hopefully include a user space architecture.
Potentially something we could talk about at the CC LPC microconference?

Cheers,
Samuel.

[1] https://datatracker.ietf.org/doc/draft-ietf-rats-eat/

Samuel Ortiz wrote:
> On Fri, Sep 01, 2023 at 09:51:42AM -0700, Dan Williams wrote:
> > [ Add Lukas since 'SPDM' was mentioned ]
> > 
> > Jeremi Piotrowski wrote:
> > [..]
> > > > An attestation report is signed evidence of how a Trusted Virtual
> > > > Machine (TVM) was launched and its current state. A verifying party uses
> > > > the report to make judgements of the confidentiality and integrity of
> > > > that execution environment. Upon successful attestation the verifying
> > > > party may, for example, proceed to deploy secrets to the TVM to carry
> > > > out a workload. Multiple confidential computing platforms share this
> > > > similar flow.
> > > 
> > > Besides the platform (cpu) attestation report, there are also attestation
> > > reports from individual secure PCIe devices that we'd want to fetch. This
> > > uses the SPDM protocol[1]. There is a CHALLENGE command which (too me)
> > > roughly maps to an attestation request, but also separate interfaces to
> > > fetch individual measurements and certificates (like the SNP extended
> > > report interface allows).
> > 
> > Yes, but I am not yet convinced this configfs-tsm interface would get
> > involved there.
> 
> User space will want to get those devices attestation reports, and those
> would be carried through the TSM. It would be nice to be able to use a
> common ABI for this. A CPU/platform attestation report is not that
> different from a device one.
> 
> > > 
> > > If this is to become the one attestation interface then we'll need to
> > > consider that. That will probably require adding a second level
> > > directory: /sys/kernel/config/tsm/<device path>.
> > 
> > The SPDM situation is different in my mind in that the kernel has an
> > interest in being able to attest a device itself. Think of cases like
> > power management where userspace is frozen, but the kernel needs to
> > validate the device in the resume flow.
> > 
> > For TVMs the kernel would validate devices
> 
> That means the TVM kernel would be provisioned with reference values and
> policies that are likely to be tenant specific. The same TVM kernel,
> running the same user space stack, getting the same PCIe device
> attached, could either accept or reject such device, depending on the
> tenant/workload owner policies and acceptable reference values. That
> means each tenant would have to build its own guest images, and maintain
> and update them with potentially each new device or new device stack it
> wants to support.
> Keeping most of the device attestation stack (similar to where the
> platform attestation stack lives today) in user space seems more
> flexible to me, and allows for tenant to use single guest images.

I am not seeing how moving the verification mechanism into the kernel
makes this less flexible. For an analogy the kernel supports firewall
policy without requiring userspace network stacks.

Userspace still has a role play in dynamically updating device
validation policy, but the design is such that the kernel can be
independent of userspace in scenarios like power management where it is
awkward to round trip to userspace.

> 
> > and the verifying party would
> > validate the kernel as part of the guest measurement.
> > 
> > The main difficulty again here is evidence format differentiation. My
> > hope is that there is some standardization
> 
> FWIW there are IETF driven standardization efforts like e.g. EAT [1]
> that go into the right directions imho. The latest CC implementations
> (CCA, CoVE) follow those specs (EAT, CWT/JWT).
> 
> DMTF (driving the SPDM spec) defines also its own format, through its
> measurement format spec. Device vendors may choose to implement that or
> to e.g. add their EAT formatted attestation report in the reported
> certificate chain. Realistically, we'll have to support all of those
> flows.

Yes, but hopefully the differentiation here converges to a static or
slowly moving set of evidence types. If Linux makes it painful for new
evidence types to be invented that seems like a *good* thing from a
maintenance perspective.

> 
> > or otherwise a way to update
> > the kernel's verification logic for per-device evidence-formats.  Maybe
> > eBPF has a role to play in that story, but that's a converstation for a
> > different patch set.
> 
> This conversation will hopefully include a user space architecture.
> Potentially something we could talk about at the CC LPC microconference?

Definitely. I have been distracted from submitting a formal topic
proposal here, but I definitely think LPC is a chance to converge on a
direction here.