meson.build | 7 + meson_options.txt | 2 + scripts/meson-buildoptions.sh | 3 + target/i386/meson.build | 2 +- target/i386/sev.c | 311 ++++++++++++---------------------- target/i386/sev.h | 4 +- target/i386/trace-events | 1 + 7 files changed, 123 insertions(+), 207 deletions(-)
These patches are submitted as an RFC mainly because I'm a relative newcomer to QEMU with no knowledge of the community's views on including Rust code, nor it's preference of using library APIs for ioctls that were previously implemented in QEMU directly. Recently, the Rust sev library [0] has introduced a C API to take advantage of the library outside of Rust. Should the inclusion of the library as a dependency be desired, it can be extended further to include the firmware/platform ioctls, the attestation report fetching, and more. This would result in much of the AMD-SEV portion of QEMU being offloaded to the library. This series looks to explore the possibility of using the library and show a bit of what it would look like. I'm looking for comments regarding if this feature is desired. [0] https://github.com/virtee/sev Tyler Fanelli (8): Add SEV Rust library as dependency with CONFIG_SEV i386/sev: Replace INIT and ES_INIT ioctls with sev library equivalents i386/sev: Replace LAUNCH_START ioctl with sev library equivalent i386/sev: Replace UPDATE_DATA ioctl with sev library equivalent i386/sev: Replace LAUNCH_UPDATE_VMSA ioctl with sev library equivalent i386/sev: Replace LAUNCH_MEASURE ioctl with sev library equivalent i386/sev: Replace LAUNCH_SECRET ioctl with sev library equivalent i386/sev: Replace LAUNCH_FINISH ioctl with sev library equivalent meson.build | 7 + meson_options.txt | 2 + scripts/meson-buildoptions.sh | 3 + target/i386/meson.build | 2 +- target/i386/sev.c | 311 ++++++++++++---------------------- target/i386/sev.h | 4 +- target/i386/trace-events | 1 + 7 files changed, 123 insertions(+), 207 deletions(-) -- 2.40.1
On Thu, Sep 14, 2023 at 01:58:27PM -0400, Tyler Fanelli wrote: > These patches are submitted as an RFC mainly because I'm a relative > newcomer to QEMU with no knowledge of the community's views on > including Rust code, nor it's preference of using library APIs for > ioctls that were previously implemented in QEMU directly. We've talked about Rust alot, but thus far most focus has been on areas peripheral to QEMU. Projects that might have been part of QEMU in the past, and now being done as separate efforts, and have bene taking advantage of Rust. eg virtiofsd Rust replacing QEMU's in -tree C impl. eg passt providing an alternative to slirp. eg the dbus display in QEMU allowing a remote display frontend to be provided, written in rust. eg libblkio providing a block backend in Rust. The libblkio work is likely closest to what you've proposed here, in that it is a Rust create exposed as a C shared library for apps to consume. In theory apps don't need to care that it is written in Rust, as it is opaque. The one key difference though is that it was not replacing existing functionality, it was adding a new feature. So users who didn't have libblkio or whom want to avoid Rust dependancies didn't loose anything they were already using. If we use the libsev.so we create a hard dependancy on the Rust sev crate, otherwise users loose the SEV feature in QEMU. Right now the sev crate C library is not present in *any* distro that I can see. If we treat 'sev' as just another opaque 3rd party library to be provided by the distro, this creates a problem. Our support policy is that we usually won't drop features in existing distros, but that is what would happen if we applied this patchset today. We did bend that rule slightly with virtiofsd, but that was already a separate binary and we followed our deprecation path before deleting it, giving distros time to adapt. If we rollback the curtain, however, and decide to expose Rust directly to QEMU we could address this problem. We could bundle the dependant Rust crates directly with QEMU tarballs, and generate the FFI C library as part of QEMU build and static link the library. Distros would not have todo anything, though they could have the choice of dyn linking if they really wanted to. If we directly exposed the notion of Rust to QEMU, then we are also not limited by whether a Rust crate provides a C FFI itself. QEMU could provide C FFI glue for any Rust crate it sees as useful to its code. This all forces us, however, to have the difficult discussion about whether we're willing to make Rust a mandatory dependancy of QEMU and permit (or even welcome) its use /anywhere/ in the QEMU tree that looks relevant. We've already queried whether Rust will actually benefit the core QEMU codebase, or whether we'll end up punching too many holes in its safety net to make it worthwhile. My opinion is that we probably shouldn't obsess over that as I think it is hard to predict the future, it has a habit of surprising us. Your patch series here doesn't demonstrate an obvious safety benefit, since we have existing working code and that code is not especially complex. Once we open the doors to Rust code in QEMU though, we will probably surprise ourselves with the range of benefits we'll see 2, 3, 5 years down the road. IOW, we shouldn't judge future benefits based on this patch series. It is great that this series is actually quite simple, because it lets us focus on how we might integrate Rust more directly into QEMU, without worrying much about the actual code being replaced. > This series looks to explore the possibility of using the library and > show a bit of what it would look like. I'm looking for comments > regarding if this feature is desired. My summary, is that I'd personally be in favour of opening the door to Rust code as a mandatory pre-requisite for QEMU, at the very least for system emulators. Not because this particular series is compelling, but because I think Rust could be more beneficial to QEMU over the long term than we expect. In terms of consuming it though, if we're going to replace existing QEMU functionality, then I think we need to bundle the Rust code and natively integrate it into the build system, as we have recently started doing with our python deps, to detach ourselves from the limits of what distros ship. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Fri, 15 Sept 2023 at 10:54, Daniel P. Berrangé <berrange@redhat.com> wrote: > My summary, is that I'd personally be in favour of opening the door > to Rust code as a mandatory pre-requisite for QEMU, at the very least > for system emulators. Not because this particular series is compelling, > but because I think Rust could be more beneficial to QEMU over the long > term than we expect. In terms of consuming it though, if we're going > to replace existing QEMU functionality, then I think we need to bundle > the Rust code and natively integrate it into the build system, as we > have recently started doing with our python deps, to detach ourselves > from the limits of what distros ship. I'm not against this, but there is a fair amount of work here in figuring out how exactly to integrate Rust components into the build system, questions like what our minimum required rust version would be, liasing with downstream distros to check that what we're proposing isn't a nightmare for them to package, etc. -- PMM
On Fri, 15 Sept 2023 at 09:50, Peter Maydell <peter.maydell@linaro.org> wrote: > > On Fri, 15 Sept 2023 at 10:54, Daniel P. Berrangé <berrange@redhat.com> wrote: > > My summary, is that I'd personally be in favour of opening the door > > to Rust code as a mandatory pre-requisite for QEMU, at the very least > > for system emulators. Not because this particular series is compelling, > > but because I think Rust could be more beneficial to QEMU over the long > > term than we expect. In terms of consuming it though, if we're going > > to replace existing QEMU functionality, then I think we need to bundle > > the Rust code and natively integrate it into the build system, as we > > have recently started doing with our python deps, to detach ourselves > > from the limits of what distros ship. > > I'm not against this, but there is a fair amount of work here > in figuring out how exactly to integrate Rust components > into the build system, questions like what our minimum required > rust version would be, liasing with downstream distros to > check that what we're proposing isn't a nightmare for them > to package, etc. Those details are similar to what librsvg2, libblkio, and other libraries (like the sev crate in this patch series) have had to solve. libblkio uses meson as the build system and has C tests that cover the C API. Cargo is still used to build the Rust code. It is possible to integrate the two and I think QEMU could take that approach. It's a little ugly to glue together the two build systems, but it has been shown to work. Finding the minimum Rust version across QEMU's support matrix is doable. Stefan
Hi On Fri, Sep 15, 2023 at 9:10 PM Stefan Hajnoczi <stefanha@gmail.com> wrote: > > On Fri, 15 Sept 2023 at 09:50, Peter Maydell <peter.maydell@linaro.org> wrote: > > > > On Fri, 15 Sept 2023 at 10:54, Daniel P. Berrangé <berrange@redhat.com> wrote: > > > My summary, is that I'd personally be in favour of opening the door > > > to Rust code as a mandatory pre-requisite for QEMU, at the very least > > > for system emulators. Not because this particular series is compelling, > > > but because I think Rust could be more beneficial to QEMU over the long > > > term than we expect. In terms of consuming it though, if we're going > > > to replace existing QEMU functionality, then I think we need to bundle > > > the Rust code and natively integrate it into the build system, as we > > > have recently started doing with our python deps, to detach ourselves > > > from the limits of what distros ship. > > > > I'm not against this, but there is a fair amount of work here > > in figuring out how exactly to integrate Rust components > > into the build system, questions like what our minimum required > > rust version would be, liasing with downstream distros to > > check that what we're proposing isn't a nightmare for them > > to package, etc. > > Those details are similar to what librsvg2, libblkio, and other > libraries (like the sev crate in this patch series) have had to solve. > > libblkio uses meson as the build system and has C tests that cover the > C API. Cargo is still used to build the Rust code. It is possible to > integrate the two and I think QEMU could take that approach. It's a > little ugly to glue together the two build systems, but it has been > shown to work. That's also what I did ~2y ago, when I added QAPI rust generator: https://patchew.org/QEMU/20210907121943.3498701-1-marcandre.lureau@redhat.com/ But meson has learned to support rust better during that time, and there is some upcoming cargo support in 1.3 (https://github.com/mesonbuild/meson/pull/11856). (I don't know the details) -- Marc-André Lureau
On Fri, 15 Sept 2023 at 05:54, Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Thu, Sep 14, 2023 at 01:58:27PM -0400, Tyler Fanelli wrote: > > These patches are submitted as an RFC mainly because I'm a relative > > newcomer to QEMU with no knowledge of the community's views on > > including Rust code, nor it's preference of using library APIs for > > ioctls that were previously implemented in QEMU directly. > > We've talked about Rust alot, but thus far most focus has been on > areas peripheral to QEMU. Projects that might have been part of > QEMU in the past, and now being done as separate efforts, and > have bene taking advantage of Rust. eg virtiofsd Rust replacing > QEMU's in -tree C impl. eg passt providing an alternative to > slirp. eg the dbus display in QEMU allowing a remote display > frontend to be provided, written in rust. eg libblkio providing > a block backend in Rust. > > The libblkio work is likely closest to what you've proposed > here, in that it is a Rust create exposed as a C shared library > for apps to consume. In theory apps don't need to care that it > is written in Rust, as it is opaque. > > The one key difference though is that it was not replacing > existing functionality, it was adding a new feature. So users > who didn't have libblkio or whom want to avoid Rust dependancies > didn't loose anything they were already using. > > If we use the libsev.so we create a hard dependancy on the Rust > sev crate, otherwise users loose the SEV feature in QEMU. Right > now the sev crate C library is not present in *any* distro that > I can see. > > If we treat 'sev' as just another opaque 3rd party library to be > provided by the distro, this creates a problem. Our support > policy is that we usually won't drop features in existing distros, > but that is what would happen if we applied this patchset today. > We did bend that rule slightly with virtiofsd, but that was already > a separate binary and we followed our deprecation path before > deleting it, giving distros time to adapt. > > > If we rollback the curtain, however, and decide to expose Rust > directly to QEMU we could address this problem. We could bundle > the dependant Rust crates directly with QEMU tarballs, and > generate the FFI C library as part of QEMU build and static > link the library. Distros would not have todo anything, though > they could have the choice of dyn linking if they really wanted > to. > > If we directly exposed the notion of Rust to QEMU, then we are > also not limited by whether a Rust crate provides a C FFI itself. > QEMU could provide C FFI glue for any Rust crate it sees as > useful to its code. > > This all forces us, however, to have the difficult discussion > about whether we're willing to make Rust a mandatory dependancy > of QEMU and permit (or even welcome) its use /anywhere/ in the > QEMU tree that looks relevant. > > We've already queried whether Rust will actually benefit the > core QEMU codebase, or whether we'll end up punching too many > holes in its safety net to make it worthwhile. My opinion is > that we probably shouldn't obsess over that as I think it is > hard to predict the future, it has a habit of surprising us. > Your patch series here doesn't demonstrate an obvious safety > benefit, since we have existing working code and that code is > not especially complex. Once we open the doors to Rust code > in QEMU though, we will probably surprise ourselves with the > range of benefits we'll see 2, 3, 5 years down the road. > > IOW, we shouldn't judge future benefits based on this patch > series. It is great that this series is actually quite simple, > because it lets us focus on how we might integrate Rust more > directly into QEMU, without worrying much about the actual > code being replaced. > > > This series looks to explore the possibility of using the library and > > show a bit of what it would look like. I'm looking for comments > > regarding if this feature is desired. > > My summary, is that I'd personally be in favour of opening the door > to Rust code as a mandatory pre-requisite for QEMU, at the very least > for system emulators. Not because this particular series is compelling, > but because I think Rust could be more beneficial to QEMU over the long > term than we expect. In terms of consuming it though, if we're going > to replace existing QEMU functionality, then I think we need to bundle > the Rust code and natively integrate it into the build system, as we > have recently started doing with our python deps, to detach ourselves > from the limits of what distros ship. I support using Rust directly within QEMU. David Gibson looked at Rust's operating system and CPU architecture coverage a few years ago: https://wiki.qemu.org/RustInQemu Please update that support matrix to check that depending on Rust in core QEMU code really works everywhere where QEMU is supported today. This is probably just a formality at this stage since Rust has become widely used over the past few years. The library approach worked well for libblkio but the overhead of creating a separate shared library and shipping it is significant. When QEMU is the only user of some code, then it should definitely be part of QEMU. Also, when QEMU needs early access to code that isn't widely available yet, then bundling it inside QEMU until packages are available also seems reasonable to me (I think we already do that for libvfio-user and maybe other libraries). I would prefer it if we minimize Rust wrappers for C APIs and instead focus on using Rust to build new subsystems. Writing and maintaing two sets of the same API is expensive and I hope we don't get bogged down keeping C and Rust APIs in sync. That said, I think there's an argument for wrapping core QEMU APIs needed for device emulation (e.g. DeviceState, PCIDevice) because of the security benefits of writing new device emulation code in Rust. Stefan
On 9/15/23 7:33 AM, Stefan Hajnoczi wrote: > On Fri, 15 Sept 2023 at 05:54, Daniel P. Berrangé <berrange@redhat.com> wrote: >> On Thu, Sep 14, 2023 at 01:58:27PM -0400, Tyler Fanelli wrote: >>> These patches are submitted as an RFC mainly because I'm a relative >>> newcomer to QEMU with no knowledge of the community's views on >>> including Rust code, nor it's preference of using library APIs for >>> ioctls that were previously implemented in QEMU directly. >> We've talked about Rust alot, but thus far most focus has been on >> areas peripheral to QEMU. Projects that might have been part of >> QEMU in the past, and now being done as separate efforts, and >> have bene taking advantage of Rust. eg virtiofsd Rust replacing >> QEMU's in -tree C impl. eg passt providing an alternative to >> slirp. eg the dbus display in QEMU allowing a remote display >> frontend to be provided, written in rust. eg libblkio providing >> a block backend in Rust. >> >> The libblkio work is likely closest to what you've proposed >> here, in that it is a Rust create exposed as a C shared library >> for apps to consume. In theory apps don't need to care that it >> is written in Rust, as it is opaque. >> >> The one key difference though is that it was not replacing >> existing functionality, it was adding a new feature. So users >> who didn't have libblkio or whom want to avoid Rust dependancies >> didn't loose anything they were already using. >> >> If we use the libsev.so we create a hard dependancy on the Rust >> sev crate, otherwise users loose the SEV feature in QEMU. Right >> now the sev crate C library is not present in *any* distro that >> I can see. Yes, the C API is very new and not packaged in any distro at the moment. >> >> If we treat 'sev' as just another opaque 3rd party library to be >> provided by the distro, this creates a problem. Our support >> policy is that we usually won't drop features in existing distros, >> but that is what would happen if we applied this patchset today. >> We did bend that rule slightly with virtiofsd, but that was already >> a separate binary and we followed our deprecation path before >> deleting it, giving distros time to adapt. >> >> >> If we rollback the curtain, however, and decide to expose Rust >> directly to QEMU we could address this problem. We could bundle >> the dependant Rust crates directly with QEMU tarballs, and >> generate the FFI C library as part of QEMU build and static >> link the library. Distros would not have todo anything, though >> they could have the choice of dyn linking if they really wanted >> to. >> >> If we directly exposed the notion of Rust to QEMU, then we are >> also not limited by whether a Rust crate provides a C FFI itself. >> QEMU could provide C FFI glue for any Rust crate it sees as >> useful to its code. >> >> This all forces us, however, to have the difficult discussion >> about whether we're willing to make Rust a mandatory dependancy >> of QEMU and permit (or even welcome) its use /anywhere/ in the >> QEMU tree that looks relevant. >> >> We've already queried whether Rust will actually benefit the >> core QEMU codebase, or whether we'll end up punching too many >> holes in its safety net to make it worthwhile. My opinion is >> that we probably shouldn't obsess over that as I think it is >> hard to predict the future, it has a habit of surprising us. >> Your patch series here doesn't demonstrate an obvious safety >> benefit, since we have existing working code and that code is >> not especially complex. Correct, there isn't any new features being added here. SEV on QEMU should work _exactly_ how it did before these patches. >> Once we open the doors to Rust code >> in QEMU though, we will probably surprise ourselves with the >> range of benefits we'll see 2, 3, 5 years down the road. >> >> IOW, we shouldn't judge future benefits based on this patch >> series. It is great that this series is actually quite simple, >> because it lets us focus on how we might integrate Rust more >> directly into QEMU, without worrying much about the actual >> code being replaced. It also shows that much of Rust's security benefits in QEMU would depend on Rust being integrated more directly, rather than just using C FFI. Most of the code in the libsev C API is unsafe Rust anyway (it must be in order to interact with C). Therefore there is not much of an added security benefit here. However, if Rust can be further expanded in QEMU, much of the unsafe bits can be removed entirely (i.e. by bypassing the C API). >> >>> This series looks to explore the possibility of using the library and >>> show a bit of what it would look like. I'm looking for comments >>> regarding if this feature is desired. >> My summary, is that I'd personally be in favour of opening the door >> to Rust code as a mandatory pre-requisite for QEMU, at the very least >> for system emulators. Not because this particular series is compelling, >> but because I think Rust could be more beneficial to QEMU over the long >> term than we expect. In terms of consuming it though, if we're going >> to replace existing QEMU functionality, then I think we need to bundle >> the Rust code and natively integrate it into the build system, as we >> have recently started doing with our python deps, to detach ourselves >> from the limits of what distros ship. > I support using Rust directly within QEMU. > > David Gibson looked at Rust's operating system and CPU architecture > coverage a few years ago: > https://wiki.qemu.org/RustInQemu > > Please update that support matrix to check that depending on Rust in > core QEMU code really works everywhere where QEMU is supported today. > This is probably just a formality at this stage since Rust has become > widely used over the past few years. > > The library approach worked well for libblkio but the overhead of > creating a separate shared library and shipping it is significant. > When QEMU is the only user of some code, then it should definitely be > part of QEMU. Also, when QEMU needs early access to code that isn't > widely available yet, then bundling it inside QEMU until packages are > available also seems reasonable to me (I think we already do that for > libvfio-user and maybe other libraries). > > I would prefer it if we minimize Rust wrappers for C APIs and instead > focus on using Rust to build new subsystems. Writing and maintaing two > sets of the same API is expensive and I hope we don't get bogged down > keeping C and Rust APIs in sync. It would also allow QEMU to take advantage of other Rust crates without having to add some type of shim layer to facilitate the interaction. Tyler > That said, I think there's an > argument for wrapping core QEMU APIs needed for device emulation (e.g. > DeviceState, PCIDevice) because of the security benefits of writing > new device emulation code in Rust. > > Stefan >
On Fri, Sep 15, 2023 at 07:33:32AM -0400, Stefan Hajnoczi wrote: > The library approach worked well for libblkio but the overhead of > creating a separate shared library and shipping it is significant. > When QEMU is the only user of some code, then it should definitely be > part of QEMU. Also, when QEMU needs early access to code that isn't > widely available yet, then bundling it inside QEMU until packages are > available also seems reasonable to me (I think we already do that for > libvfio-user and maybe other libraries). Yep, avoiding the public shared library significantly cuts down the maint burden, as you can freely adapt the exposed C FFI API to suit QEMU's needs and not worry about ABI compatibility. > I would prefer it if we minimize Rust wrappers for C APIs and instead > focus on using Rust to build new subsystems. Writing and maintaing two > sets of the same API is expensive and I hope we don't get bogged down > keeping C and Rust APIs in sync. That said, I think there's an > argument for wrapping core QEMU APIs needed for device emulation (e.g. > DeviceState, PCIDevice) because of the security benefits of writing > new device emulation code in Rust. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Hi Tyler, On 14/9/23 19:58, Tyler Fanelli wrote: > These patches are submitted as an RFC mainly because I'm a relative > newcomer to QEMU with no knowledge of the community's views on > including Rust code, nor it's preference of using library APIs for > ioctls that were previously implemented in QEMU directly. > > Recently, the Rust sev library [0] has introduced a C API to take > advantage of the library outside of Rust. > > Should the inclusion of the library as a dependency be desired, it can > be extended further to include the firmware/platform ioctls, the > attestation report fetching, and more. This would result in much of > the AMD-SEV portion of QEMU being offloaded to the library. > > This series looks to explore the possibility of using the library and > show a bit of what it would look like. I'm looking for comments > regarding if this feature is desired. > > [0] https://github.com/virtee/sev > > Tyler Fanelli (8): > Add SEV Rust library as dependency with CONFIG_SEV > i386/sev: Replace INIT and ES_INIT ioctls with sev library equivalents > i386/sev: Replace LAUNCH_START ioctl with sev library equivalent > i386/sev: Replace UPDATE_DATA ioctl with sev library equivalent > i386/sev: Replace LAUNCH_UPDATE_VMSA ioctl with sev library equivalent > i386/sev: Replace LAUNCH_MEASURE ioctl with sev library equivalent > i386/sev: Replace LAUNCH_SECRET ioctl with sev library equivalent > i386/sev: Replace LAUNCH_FINISH ioctl with sev library equivalent There is still one ioctl use, GET_ATTESTATION_REPORT. No libsev equivalent for this one yet?
On 9/14/23 3:04 PM, Philippe Mathieu-Daudé wrote: > Hi Tyler, > > On 14/9/23 19:58, Tyler Fanelli wrote: >> These patches are submitted as an RFC mainly because I'm a relative >> newcomer to QEMU with no knowledge of the community's views on >> including Rust code, nor it's preference of using library APIs for >> ioctls that were previously implemented in QEMU directly. >> >> Recently, the Rust sev library [0] has introduced a C API to take >> advantage of the library outside of Rust. >> >> Should the inclusion of the library as a dependency be desired, it can >> be extended further to include the firmware/platform ioctls, the >> attestation report fetching, and more. This would result in much of >> the AMD-SEV portion of QEMU being offloaded to the library. >> >> This series looks to explore the possibility of using the library and >> show a bit of what it would look like. I'm looking for comments >> regarding if this feature is desired. >> >> [0] https://github.com/virtee/sev >> >> Tyler Fanelli (8): >> Add SEV Rust library as dependency with CONFIG_SEV >> i386/sev: Replace INIT and ES_INIT ioctls with sev library >> equivalents >> i386/sev: Replace LAUNCH_START ioctl with sev library equivalent >> i386/sev: Replace UPDATE_DATA ioctl with sev library equivalent >> i386/sev: Replace LAUNCH_UPDATE_VMSA ioctl with sev library >> equivalent >> i386/sev: Replace LAUNCH_MEASURE ioctl with sev library equivalent >> i386/sev: Replace LAUNCH_SECRET ioctl with sev library equivalent >> i386/sev: Replace LAUNCH_FINISH ioctl with sev library equivalent > > There is still one ioctl use, GET_ATTESTATION_REPORT. No libsev > equivalent for this one yet? > There is an equivalent, however the machine that I'm using currently hangs when trying to fetch an attestation report (not a libsev issue, as it hangs when I try with latest qemu release as well). When I can either update its firmware or get access to another SEV machine, I can test and confirm it behaves as intended with the libsev API. Once this is done, I can add that API to the patch series. Tyler
© 2016 - 2024 Red Hat, Inc.