Hi, qemu maintainers and community members, The following 6 patches implemented a new x86_64 CPU accelerator called the Android Emulator hypervisor driver (AEHD). The Android Emulator hypervisor driver is a hypervisor for Windows (7 or later), made by porting the KVM from the linux kernel 4.9-rc7. Its initial purpose was to support the Android Emulator on the AMD platforms as the old name "Android Emulator Hypervisor Driver for AMD Processors" suggested. Despite the name, Intel processors have been supported ever since its first release. Since Intel dropped HAXM support, the android emulator is switching from HAXM to AEHD. Refer to: https://github.com/google/android-emulator-hypervisor-driver. This patchset implements the user space support for the new hypervisor. It was initially made by cloning and modifying the KVM source codes. In order to support users who wanted to use the new hypervisor to run generic OSes such as Windows and Ubuntu, it was periodically rebased to latest QEMU releases ever since QEMU v4.2.0. Refer to: https://github.com/qemu-gvm/qemu-gvm. Given that both the Windows driver and patched QEMU were out and tested for a few years, we would like to see if we could submit our work to the QEMU community. Users can have another choice of hypervisor on Windows. The patchset is tested on Windows 11 (64bit) with the AEHD 2.1 by installing Ubuntu 22.04 and Windows 10 guests and playing with savevm/loadvm. It is also tested on Linux (Ubuntu 22.04) and MacOS (Intel and Apple Silicon), making sure it does not break others. Any comments are welcome. Thanks! Note: The Android Emulator hypervisor driver will be maintained by Google. However, there is no plan to rebase it to the latest KVM. -- Haitao @Google
On Thu, Mar 02, 2023 at 06:25:59PM -0800, Haitao Shan wrote: > The Android Emulator hypervisor driver is a hypervisor for Windows (7 > or later), made by porting the KVM from the linux kernel 4.9-rc7. Its > initial purpose was to support the Android Emulator on the AMD > platforms as the old name "Android Emulator Hypervisor Driver for AMD > Processors" suggested. Despite the name, Intel processors have been > supported ever since its first release. Since Intel dropped HAXM support, > the android emulator is switching from HAXM to AEHD. When HAXM was proposed for deprecation & removal from QEMU, the suggestion was that users should switch to Windows' native replacement WHPX, which QEMU already has support for. What is the rationale for wanting to introduce a 3rd party hypervisor solution like AEHD, for the Android emulator, rather than just sticking with the standard WHPX hypervisor available for Windows ? IIUC, the Android emulator can already support WHPX according to these pages: https://developer.android.com/studio/run/emulator-acceleration https://learn.microsoft.com/en-us/dotnet/maui/android/emulator/hardware-acceleration?view=net-maui-7.0 With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Fri, Mar 3, 2023 at 2:34 AM Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Thu, Mar 02, 2023 at 06:25:59PM -0800, Haitao Shan wrote: > > The Android Emulator hypervisor driver is a hypervisor for Windows (7 > > or later), made by porting the KVM from the linux kernel 4.9-rc7. Its > > initial purpose was to support the Android Emulator on the AMD > > platforms as the old name "Android Emulator Hypervisor Driver for AMD > > Processors" suggested. Despite the name, Intel processors have been > > supported ever since its first release. Since Intel dropped HAXM support, > > the android emulator is switching from HAXM to AEHD. > > When HAXM was proposed for deprecation & removal from QEMU, the suggestion > was that users should switch to Windows' native replacement WHPX, which > QEMU already has support for. Sorry I was not aware of Intel's suggestion when HAXM was deprecated. Is this a decision already, which shuts the door for any other 3rd party hypervisors? > > What is the rationale for wanting to introduce a 3rd party hypervisor > solution like AEHD, for the Android emulator, rather than just sticking > with the standard WHPX hypervisor available for Windows ? IIUC, the > Android emulator can already support WHPX according to these pages: > > https://developer.android.com/studio/run/emulator-acceleration > https://learn.microsoft.com/en-us/dotnet/maui/android/emulator/hardware-acceleration?view=net-maui-7.0 You are right. WHPX is supported by the android emulator. The reason for supporting AEHD: 1. HyperV is a type-1 hypervisor, which does not coexist with other hypervisors. According to our data, there are both users who have to live with HyperV on or with HyperV off. Those users depend on certain Windows features (or 3rd party programs) that have to turn on or turn off HyperV. Offering AEHD allows us to serve both types of users. This is the major benefit. I think this argument is true for upstream QEMU users as well. 2. Actually, AEHD started development before WHPX was added to the android emulator. But porting/writing a new hypervisor just took lots of time. In the middle, Microsoft offered us WHPX. It could be the case that AEHD was never started if WHPX had been offered one or two years earlier. However, we decided to continue. First, see reason 1. Second, at least at that time, WHPX was noticeably slower than both HAXM and AEHD. Third, Microsoft clearly stated there would not be any backporting of WHPX to older versions of Windows. And those using old versions of Windows need a solution in addition to HAXM. 3. Compared with HAXM, which looks like the default solution when HyperV must be off, AEHD supports both Intel and AMD. And according to user feedback and our own tests, AEHD can support Windows 10. This was the reason why I maintained a patched QEMU ever since 4.2.0 specifically for them. 4. Although it is called Android Emulator hypervisor driver, it has nothing that is android specific. And I've seen the upstream QEMU successfully refactored the accelerator logic. This made adding a new hypervisor support less intrusive to the main code base. I have a good will and intention to maintain what I submitted if it could be approved by the community. I hope this does not place (or at least place very minimum) burden on the maintainers, should it be accepted. Hope this can answer your questions. Thanks! > > With regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > -- Haitao @Google
On Fri, Mar 03, 2023 at 08:50:18AM -0800, Haitao Shan wrote: > On Fri, Mar 3, 2023 at 2:34 AM Daniel P. Berrangé <berrange@redhat.com> wrote: > > > > On Thu, Mar 02, 2023 at 06:25:59PM -0800, Haitao Shan wrote: > > > The Android Emulator hypervisor driver is a hypervisor for Windows (7 > > > or later), made by porting the KVM from the linux kernel 4.9-rc7. Its > > > initial purpose was to support the Android Emulator on the AMD > > > platforms as the old name "Android Emulator Hypervisor Driver for AMD > > > Processors" suggested. Despite the name, Intel processors have been > > > supported ever since its first release. Since Intel dropped HAXM support, > > > the android emulator is switching from HAXM to AEHD. > > > > When HAXM was proposed for deprecation & removal from QEMU, the suggestion > > was that users should switch to Windows' native replacement WHPX, which > > QEMU already has support for. > > Sorry I was not aware of Intel's suggestion when HAXM was deprecated. > Is this a decision already, which shuts the door for any other 3rd party > hypervisors? No, we're always open to new proposals. It merely means that it might be harder to justify why the new hypervisor is a net benefit for QEMU, when there is a competing solution supported by the OS vendor. > > What is the rationale for wanting to introduce a 3rd party hypervisor > > solution like AEHD, for the Android emulator, rather than just sticking > > with the standard WHPX hypervisor available for Windows ? IIUC, the > > Android emulator can already support WHPX according to these pages: > > > > https://developer.android.com/studio/run/emulator-acceleration > > https://learn.microsoft.com/en-us/dotnet/maui/android/emulator/hardware-acceleration?view=net-maui-7.0 > You are right. WHPX is supported by the android emulator. The reason for > supporting AEHD: > > 1. HyperV is a type-1 hypervisor, which does not coexist with other hypervisors. > According to our data, there are both users who have to live with HyperV on or > with HyperV off. Those users depend on certain Windows features (or 3rd party > programs) that have to turn on or turn off HyperV. Offering AEHD allows us to > serve both types of users. This is the major benefit. I think this > argument is true > for upstream QEMU users as well. Yes, it is an akward situation if users cannot enable use of WHPX. I don't have any better suggestion in that scenario. > 2. Actually, AEHD started development before WHPX was added to the android > emulator. But porting/writing a new hypervisor just took lots of time. > In the middle, > Microsoft offered us WHPX. It could be the case that AEHD was never started if > WHPX had been offered one or two years earlier. However, we decided to continue. > First, see reason 1. Second, at least at that time, WHPX was noticeably slower > than both HAXM and AEHD. Third, Microsoft clearly stated there would not be > any backporting of WHPX to older versions of Windows. And those using old > versions of Windows need a solution in addition to HAXM. FYI, from the QEMU POV, we only aim to support the two most recent major releases of Windows. At a technical level we've set the compilation baseline to Windows 8, but per the support policy we only really support Windows 10 and 11 right now. > 3. Compared with HAXM, which looks like the default solution when HyperV > must be off, AEHD supports both Intel and AMD. And according to user feedback > and our own tests, AEHD can support Windows 10. This was the reason > why I maintained a patched QEMU ever since 4.2.0 specifically for them. Yep, no disagreement that AEHD looks more useful for HAXM given the cross vendor CPU support. > 4. Although it is called Android Emulator hypervisor driver, it has nothing that > is android specific. And I've seen the upstream QEMU successfully refactored the > accelerator logic. This made adding a new hypervisor support less intrusive to > the main code base. I have a good will and intention to maintain what I > submitted if it could be approved by the community. I hope this does > not place (or > at least place very minimum) burden on the maintainers, should it be accepted. Ok, thank you. I don't have any specific requests / suggestion myself, but I see others replied wondering whether there's any sense is sharing code with the KVM driver in QEMU. Hopefully the QEMU KVM maintainers will comment with their views on the matter. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Fri, Mar 3, 2023 at 9:12 AM Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Fri, Mar 03, 2023 at 08:50:18AM -0800, Haitao Shan wrote: > > On Fri, Mar 3, 2023 at 2:34 AM Daniel P. Berrangé <berrange@redhat.com> wrote: > > > > > > On Thu, Mar 02, 2023 at 06:25:59PM -0800, Haitao Shan wrote: > > > > The Android Emulator hypervisor driver is a hypervisor for Windows (7 > > > > or later), made by porting the KVM from the linux kernel 4.9-rc7. Its > > > > initial purpose was to support the Android Emulator on the AMD > > > > platforms as the old name "Android Emulator Hypervisor Driver for AMD > > > > Processors" suggested. Despite the name, Intel processors have been > > > > supported ever since its first release. Since Intel dropped HAXM support, > > > > the android emulator is switching from HAXM to AEHD. > > > > > > When HAXM was proposed for deprecation & removal from QEMU, the suggestion > > > was that users should switch to Windows' native replacement WHPX, which > > > QEMU already has support for. > > > > Sorry I was not aware of Intel's suggestion when HAXM was deprecated. > > Is this a decision already, which shuts the door for any other 3rd party > > hypervisors? > > No, we're always open to new proposals. It merely means that it > might be harder to justify why the new hypervisor is a net benefit > for QEMU, when there is a competing solution supported by the OS > vendor. Thanks for the clarification. It is great that the door is not shut completely. > > > > What is the rationale for wanting to introduce a 3rd party hypervisor > > > solution like AEHD, for the Android emulator, rather than just sticking > > > with the standard WHPX hypervisor available for Windows ? IIUC, the > > > Android emulator can already support WHPX according to these pages: > > > > > > https://developer.android.com/studio/run/emulator-acceleration > > > https://learn.microsoft.com/en-us/dotnet/maui/android/emulator/hardware-acceleration?view=net-maui-7.0 > > You are right. WHPX is supported by the android emulator. The reason for > > supporting AEHD: > > > > 1. HyperV is a type-1 hypervisor, which does not coexist with other hypervisors. > > According to our data, there are both users who have to live with HyperV on or > > with HyperV off. Those users depend on certain Windows features (or 3rd party > > programs) that have to turn on or turn off HyperV. Offering AEHD allows us to > > serve both types of users. This is the major benefit. I think this > > argument is true > > for upstream QEMU users as well. > > Yes, it is an akward situation if users cannot enable use of WHPX. > I don't have any better suggestion in that scenario. > > > 2. Actually, AEHD started development before WHPX was added to the android > > emulator. But porting/writing a new hypervisor just took lots of time. > > In the middle, > > Microsoft offered us WHPX. It could be the case that AEHD was never started if > > WHPX had been offered one or two years earlier. However, we decided to continue. > > First, see reason 1. Second, at least at that time, WHPX was noticeably slower > > than both HAXM and AEHD. Third, Microsoft clearly stated there would not be > > any backporting of WHPX to older versions of Windows. And those using old > > versions of Windows need a solution in addition to HAXM. > > FYI, from the QEMU POV, we only aim to support the two most recent major > releases of Windows. Just want to point out that only Windows 10 1809+ (or perhaps 1803+) supports WHPX. And how to enable WHPX varies among versions of Windows 11/10. For example, on Windows 11, there is no need to explicitly enable it using the Windows Features dialog, which is good according to my taste. But I do agree that earlier Windows 10 versions should not be considered as recent major releases any more nowadays. > > At a technical level we've set the compilation baseline to Windows 8, but > per the support policy we only really support Windows 10 and 11 right now. > > > 3. Compared with HAXM, which looks like the default solution when HyperV > > must be off, AEHD supports both Intel and AMD. And according to user feedback > > and our own tests, AEHD can support Windows 10. This was the reason > > why I maintained a patched QEMU ever since 4.2.0 specifically for them. > > Yep, no disagreement that AEHD looks more useful for HAXM given the > cross vendor CPU support. Oops. By saying supporting Windows 10, I really mean Windows 10 guests. > > > 4. Although it is called Android Emulator hypervisor driver, it has nothing that > > is android specific. And I've seen the upstream QEMU successfully refactored the > > accelerator logic. This made adding a new hypervisor support less intrusive to > > the main code base. I have a good will and intention to maintain what I > > submitted if it could be approved by the community. I hope this does > > not place (or > > at least place very minimum) burden on the maintainers, should it be accepted. > > Ok, thank you. > > I don't have any specific requests / suggestion myself, but I see others > replied wondering whether there's any sense is sharing code with the > KVM driver in QEMU. Hopefully the QEMU KVM maintainers will comment with > their views on the matter. > > With regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > -- Haitao @Google
On 3/3/23 18:39, Haitao Shan wrote: >> No, we're always open to new proposals. It merely means that it >> might be harder to justify why the new hypervisor is a net benefit >> for QEMU, when there is a competing solution supported by the OS >> vendor. > > Thanks for the clarification. It is great that the door is not shut completely. Hi, sorry for not answering before. I think in general QEMU should be open to merging work from the Android Emulator. If AEHD is useful to the Android emulator, I would consider it interesting for QEMU as well. However, I would rather have it as an extension to KVM if possible rather than a completely new emulator. One possibility is to introduce a new file that encapsulates all KVM ioctls, with a struct that encapsulates the Unix file descriptor/Windows HANDLE. For example int kvm_ioctl_get_supported_cpuid(KVMState *s, struct kvm_cpuid *cpuid, int max) { cpuid->nent = max; #ifdef CONFIG_POSIX return ioctl(s, KVM_GET_SUPPORTED_VCPUID, cpuid); #else size_t size = sizeof(*cpuid) + max * sizeof(*cpuid->entries); return aehd_ioctl(s, AEHD_GET_SUPPORTED_CPUID, cpuid, size, cpuid, size); #endif } int kvm_ioctl_create_vcpu(KVMState *s, int vcpu_id, CPUState *out) { #ifdef CONFIG_POSIX out.kvm_fd = kvm_vm_ioctl(KVM_CREATE_VCPU, vcpu_id); return out.kvm_fd; #else return aehd_vm_ioctl(s, AEHD_CREATE_VCPU, &vcpu_id, sizeof(vcpu_id), &out.kvm_fd, sizeof(out.kvm_fd)); #endif } etc. These are just general examples, the actual level of abstraction is up to you. Paolo
On Tue, Apr 4, 2023 at 4:55 AM Paolo Bonzini <pbonzini@redhat.com> wrote: > > On 3/3/23 18:39, Haitao Shan wrote: > >> No, we're always open to new proposals. It merely means that it > >> might be harder to justify why the new hypervisor is a net benefit > >> for QEMU, when there is a competing solution supported by the OS > >> vendor. > > > > Thanks for the clarification. It is great that the door is not shut completely. > > Hi, > > sorry for not answering before. Thanks for your reply. I was taking a long vacation and did not see your reply earlier. > > I think in general QEMU should be open to merging work from the Android > Emulator. If AEHD is useful to the Android emulator, I would consider > it interesting for QEMU as well. Thanks for being open to us. For patchset V1, the most important feedback we can have is that our work can be useful to the community (not just the android emulator). > > However, I would rather have it as an extension to KVM if possible > rather than a completely new emulator. One possibility is to introduce > a new file that encapsulates all KVM ioctls, with a struct that > encapsulates the Unix file descriptor/Windows HANDLE. For example > > int kvm_ioctl_get_supported_cpuid(KVMState *s, struct kvm_cpuid *cpuid, > int max) > { > cpuid->nent = max; > #ifdef CONFIG_POSIX > return ioctl(s, KVM_GET_SUPPORTED_VCPUID, cpuid); > #else > size_t size = sizeof(*cpuid) + max * sizeof(*cpuid->entries); > return aehd_ioctl(s, AEHD_GET_SUPPORTED_CPUID, cpuid, size, cpuid, > size); > #endif > } > > int kvm_ioctl_create_vcpu(KVMState *s, int vcpu_id, CPUState *out) > { > #ifdef CONFIG_POSIX > out.kvm_fd = kvm_vm_ioctl(KVM_CREATE_VCPU, vcpu_id); > return out.kvm_fd; > #else > return aehd_vm_ioctl(s, AEHD_CREATE_VCPU, &vcpu_id, sizeof(vcpu_id), > &out.kvm_fd, sizeof(out.kvm_fd)); > #endif > } > > etc. > > These are just general examples, the actual level of abstraction is up > to you. I will work on the new patchset. And most likely it will take some time. > > Paolo > -- Haitao @Google
© 2016 - 2024 Red Hat, Inc.