semantic: 0.902
assembly: 0.899
other: 0.892
graphic: 0.888
device: 0.871
KVM: 0.865
vnc: 0.863
mistranslation: 0.853
network: 0.846
instruction: 0.834
boot: 0.808
socket: 0.796

QEMU installer with WHPX support

People often ask the community to add WHPX support to the QEMU installer for Windows,
but it is impossible due to the license limitations of the WHPX SDK.

The WinHvEmulation.h and WinHvPlatform.h header files needed are "All
rights reserved".

However these headers only contain struct definitions and integer constants,
no functional code in macros or inline functions. See:
https://<email address hidden>/msg645815.html
It is questionable whether the headers alone can be considered copyrightable material.

Has anyone raised an RFE with the mingw64 project to provide these headers / APIs ? That's what provides the interfaces we usually rely on for Windows builds, and they're likely familiar with what they can & can't do from a legal POV. I don't see this as something QEMU needs to solve itself.

+launchpad ticket

On 9/19/19 1:26 PM, Philippe Mathieu-Daudé wrote:
> On 9/19/19 1:18 PM, Stefan Weil wrote:
>> Am 19.09.2019 um 12:59 schrieb Philippe Mathieu-Daudé:
>>> Add a job to cross-build QEMU with WHPX enabled.
>>>
>>> Use the Win10SDK headers from the Android Project, as commented
>>> in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
>>>
>>> Based-on: <email address hidden>
>>> https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.html
>>>
>>> Philippe Mathieu-Daudé (2):
>>>    tests/docker: Add fedora-win10sdk-cross image
>>>    .shippable.yml: Build WHPX enabled binaries
>>>
>>>   .shippable.yml                                |  2 ++
>>>   tests/docker/Makefile.include                 |  1 +
>>>   .../dockerfiles/fedora-win10sdk-cross.docker  | 21 +++++++++++++++++++
>>>   3 files changed, 24 insertions(+)
>>>   create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
>>>
>>
>> Please note that the required header files are part of the Win10SDK
>> which is not published under a free license, so I am afraid that they
>> cannot be used with QEMU code to produce free binaries.
> 
> Yes :S
> 
>> I have addressed that some time ago, and Justin Terry is still looking
>> for a solution on the Microsoft side.
> 
> Oh this is a good news, thanks for caring about this issue,
> and thanks Justin for looking for a solution!
> 
> Trying to understand how WHPX is used, I noticed there are much many
> Windows QEMU users than I thought, and it would be nice if we can have
> some upstream CI testing to not break the various projects using it.
> 
> Regards,
> 
> Phil.
> 



+launchpad ticket

On 9/20/19 6:53 PM, Justin Terry (VM) wrote:
> Hey Phil,
> 
> I have contacted our legal department for guidance on this specific use case and will update you when I hear back. Thank you for your patience.
> 
> Justin Terry
> 
>> -----Original Message-----
>> From: Philippe Mathieu-Daudé <email address hidden>
>> Sent: Friday, September 20, 2019 8:18 AM
>> To: <email address hidden>; Justin Terry (VM) <email address hidden>
>> Cc: Daniel P . Berrangé <email address hidden>; Fam Zheng
>> <email address hidden>; Thomas Huth <email address hidden>; Paolo Bonzini
>> <email address hidden>; Alex Bennée <email address hidden>; Richard
>> Henderson <email address hidden>; Eduardo Habkost <email address hidden>;
>> Stefan Weil <email address hidden>
>> Subject: Re: [PATCH v2 0/3] testing: Build WHPX enabled binaries
>>
>> On 9/20/19 1:33 PM, Philippe Mathieu-Daudé wrote:
>>> Add a job to cross-build QEMU with WHPX enabled.
>>>
>>> Since the WHPX is currently broken, include the patch required to have
>>> successful Shippable build.
>>>
>>> I previously included the WHPX headers shared by the Android project,
>>> and Daniel asked me to check the EULA. While trying to manually
>>> install the Windows SDK, I noticed the installer fetches archives
>>> directly, kindly asking where they are stored via the /fwlink API.
>>> Do the same, fetch the required archives and extract them. No need to
>>> accept EULA...
>>>
>>> Docker build the image first, then build QEMU in a instance of this
>>> image. The image is internal to Shippable, the instances are not
>>> reachable and are thrown once the build is finished. What we collect
>>> from Shippable is the console output of QEMU build process, and if the
>>> build process succeed or failed. So far we do not redistribute the
>>> image or built binaries.
>>>
>>> Philippe Mathieu-Daudé (3):
>>>    target/i386: Fix broken build with WHPX enabled
>>>    tests/docker: Add fedora-win10sdk-cross image
>>>    .shippable.yml: Build WHPX enabled binaries
>>
>> FWIW here is the result of this series:
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapp.
>> shippable.com%2Fgithub%2Fphilmd%2Fqemu%2Fruns%2F516%2F11%2Fcon
>> sole&amp;data=02%7C01%7Cjuterry%40microsoft.com%7C733a566f3233427
>> 8ae6f08d73dddb39f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6
>> 37045894733463150&amp;sdata=55URgDII5r74QMUpLOD%2FWT5%2B5jbzyv
>> nfCSdv%2FNaWDAw%3D&amp;reserved=0
>> Duration 17 minutes (1076 seconds)
>>
>> 4m49s building the qemu:fedora-win10sdk-cross docker image, 11m10s
>> building WHPX QEMU.



+launchpad ticket

On 11/7/19 11:52 PM, Sunil Muthuswamy wrote:
>>> You will need the Windows 10 SDK for RS5 (build 17763) or above to
>>> to be able to compile this patch because of the definition of the
>>> XCR0 register.
>>>
>>> Changes since v1:
>>> - Added a sign-off line in the patch.
>>
>>
>> I am not very happy with the current situation which suggests using non
>> free header files from the Microsoft Windows SDK, thus making it
>> impossible to produce QEMU executables for Windows with WHPX support
>> without having legal complications.
>>
>> Could you please add the required headers with a suitable license to the
>> QEMU source code? That would clarify the license issue and make builds
>> with WHPX much easier because those files would not have to be extracted
>> from a very large SDK installation.
>>
>> It would also be acceptable if Microsoft could update the license
>> comments in those files and use a QEMU compatible license.
>>
> I agree in principle that there should be an easier way to consume the Windows
> SDK headers without having to play around with the licenses. I also agree that
> that will make life lot easier for many developers. I am reaching out
> internally here to see what can be done about this, but, that might take some
> time. Meanwhile, is it possible to make some progress on this patch?
> 
>> Kind regards
>> Stefan Weil
>>
>>
> 



+Mike Battista & lanchpad ticket

On 2/24/20 8:43 PM, Sunil Muthuswamy wrote:
>> -----Original Message-----
>> From: Stefan Weil <email address hidden>
>> Sent: Thursday, February 20, 2020 11:54 PM
>> To: Justin Terry (SF) <email address hidden>; Philippe Mathieu-Daudé <email address hidden>; Sunil Muthuswamy
>> <email address hidden>; Eduardo Habkost <email address hidden>; Paolo Bonzini <email address hidden>; Richard Henderson
>> <email address hidden>
>> Cc: <email address hidden>
>> Subject: Re: [EXTERNAL] Re: [PATCH] WHPX: Assigning maintainer for Windows Hypervisor Platform
>>
>> Am 19.02.20 um 16:50 schrieb Justin Terry (SF):
>>
>>
>> Hello Justin, hello Sunil,
>>
>> just a reminder: we still have the problem with the proprietary license
>> for the required Microsoft header files.
>>
>> Can you estimate when this will be solved?
>>
> 
> Thanks for the reminder, Stefan. Yes, agreed this problem still exists. We followed up with
> the SDK team and the legal team end of last year. I will nudge them again for an update
> here.
> 
>> Regards,
>> Stefan
>>
> 



Hi Sunil,

On 5/19/20 11:59 PM, Sunil Muthuswamy wrote:
>> -----Original Message-----
>> From: Stefan Weil <email address hidden>
>> Sent: Thursday, February 20, 2020 11:54 PM
>> To: Justin Terry (SF) <email address hidden>; Philippe Mathieu-Daudé <email address hidden>; Sunil Muthuswamy
>> <email address hidden>; Eduardo Habkost <email address hidden>; Paolo Bonzini <email address hidden>; Richard
>> Henderson <email address hidden>
>> Cc: <email address hidden>
>> Subject: Re: [EXTERNAL] Re: [PATCH] WHPX: Assigning maintainer for Windows Hypervisor Platform
>>
>> Am 19.02.20 um 16:50 schrieb Justin Terry (SF):
>>
>> Hello Justin, hello Sunil,
>>
>> just a reminder: we still have the problem with the proprietary license
>> for the required Microsoft header files.
>>
>> Can you estimate when this will be solved?
>>
>> Regards,
>> Stefan
>>
> 
> Adding Mike Battista, who is on the SDK team and can help provide some clarity around the questions about SDK licensing.
> 

To ease communication and track the changes over time regarding this 
problem, I opened a ticket on Launchpad:
https://bugs.launchpad.net/qemu/+bug/1879672

Last time (Sept 2019) Justin Terry contacted Microsoft legal department 
for guidance but no update since.
This is unfortunate as we can not let the community use this feature, 
neither can we keep testing WHPX to avoid code bitrot.

Can you meanwhile provide Azure CI builds using WHPX enabled?

Regards,

Phil.



> Has anyone raised an RFE with the mingw64 project to provide these headers / APIs?

I had asked a long time ago on IRC (#mingw-w64 IRC channel on irc.oftc.net), but got no answer.

Regards,
Stefan

Hi Justin, Sunil,

On 5/20/20 12:26 PM, Philippe Mathieu-Daudé wrote:
> +launchpad ticket
> 
> On 9/20/19 6:53 PM, Justin Terry (VM) wrote:
>> Hey Phil,
>>
>> I have contacted our legal department for guidance on this specific
>> use case and will update you when I hear back. Thank you for your
>> patience.

I recently understood legal changes can be very complex, thus it is
implicit it can take years before getting updates.

Since the project is still actively developed, maybe you could provide
a Azure CI job to build a WHPX binary. We don't need to have access to
the binary, just to the exit status (success/fail) and build logs.

Do you think it is doable?

Thanks,

Phil.

>>
>> Justin Terry
>>
>>> -----Original Message-----
>>> From: Philippe Mathieu-Daudé <email address hidden>
>>> Sent: Friday, September 20, 2019 8:18 AM
>>> To: <email address hidden>; Justin Terry (VM) <email address hidden>
>>> Cc: Daniel P . Berrangé <email address hidden>; Fam Zheng
>>> <email address hidden>; Thomas Huth <email address hidden>; Paolo Bonzini
>>> <email address hidden>; Alex Bennée <email address hidden>; Richard
>>> Henderson <email address hidden>; Eduardo Habkost <email address hidden>;
>>> Stefan Weil <email address hidden>
>>> Subject: Re: [PATCH v2 0/3] testing: Build WHPX enabled binaries
>>>
>>> On 9/20/19 1:33 PM, Philippe Mathieu-Daudé wrote:
>>>> Add a job to cross-build QEMU with WHPX enabled.
>>>>
>>>> Since the WHPX is currently broken, include the patch required to have
>>>> successful Shippable build.
>>>>
>>>> I previously included the WHPX headers shared by the Android project,
>>>> and Daniel asked me to check the EULA. While trying to manually
>>>> install the Windows SDK, I noticed the installer fetches archives
>>>> directly, kindly asking where they are stored via the /fwlink API.
>>>> Do the same, fetch the required archives and extract them. No need to
>>>> accept EULA...
>>>>
>>>> Docker build the image first, then build QEMU in a instance of this
>>>> image. The image is internal to Shippable, the instances are not
>>>> reachable and are thrown once the build is finished. What we collect
>>>> from Shippable is the console output of QEMU build process, and if the
>>>> build process succeed or failed. So far we do not redistribute the
>>>> image or built binaries.
>>>>
>>>> Philippe Mathieu-Daudé (3):
>>>>    target/i386: Fix broken build with WHPX enabled
>>>>    tests/docker: Add fedora-win10sdk-cross image
>>>>    .shippable.yml: Build WHPX enabled binaries
>>>
>>> FWIW here is the result of this series:
>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapp.
>>> shippable.com%2Fgithub%2Fphilmd%2Fqemu%2Fruns%2F516%2F11%2Fcon
>>> sole&amp;data=02%7C01%7Cjuterry%40microsoft.com%7C733a566f3233427
>>> 8ae6f08d73dddb39f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6
>>> 37045894733463150&amp;sdata=55URgDII5r74QMUpLOD%2FWT5%2B5jbzyv
>>> nfCSdv%2FNaWDAw%3D&amp;reserved=0
>>> Duration 17 minutes (1076 seconds)
>>>
>>> 4m49s building the qemu:fedora-win10sdk-cross docker image, 11m10s
>>> building WHPX QEMU.
> 



Hi Sunil,

On 8/1/20 1:31 AM, Sunil Muthuswamy wrote:
>> Hi Justin, Sunil,
> 
> Justin has moved to a different team is no longer working with WHPX. Moving him
> to bcc.

OK. Does that mean you are the new responsible of updating the ticket
regarding the WHPX headers and their license?

> 
>>
>> On 5/20/20 12:26 PM, Philippe Mathieu-Daudé wrote:
>>> +launchpad ticket
>>>
>>> On 9/20/19 6:53 PM, Justin Terry (VM) wrote:
>>>> Hey Phil,
>>>>
>>>> I have contacted our legal department for guidance on this specific
>>>> use case and will update you when I hear back. Thank you for your
>>>> patience.
>>
>> I recently understood legal changes can be very complex, thus it is
>> implicit it can take years before getting updates.
>>
>> Since the project is still actively developed, maybe you could provide
>> a Azure CI job to build a WHPX binary. We don't need to have access to
>> the binary, just to the exit status (success/fail) and build logs.
>>
>> Do you think it is doable?
>>
>> Thanks,
>>
>> Phil.
>>
> The ask generally sounds reasonable. But, can you help me understand the full
> scope of the ask. Few questions:
> 1. Stefan has a CI pipeline to build WHPX.

Great! I didn't know Stefan already did it :)
Can you share the URL please, so we can integrate it with mainstream CI?

> What's the benefit of having another CI
> job, that doesn't export the binary, but, just the status?

As usual, we do not want to circumvent the license. IANAL but IIUC we
can not force a CI job to accept the EULA when installing it, even to
test it. So the best we can do is check if the build succeeded (exit
status).

> 2. Which branch is the CI pipeline expected to build?

'master', to be sure no regressions are introduced.

> 3. Is the expectation also that it will build WHPX patches that are submitted to the
> WHPX branch?

You describe a "downstream CI" testing, which is out of scope of the
community public CI.

Regards,

Phil.



On 8/4/20 8:55 AM, Stefan Weil wrote:
> Am 04.08.20 um 08:43 schrieb Thomas Huth:
> 
>> On 03/08/2020 22.25, Stefan Weil wrote:
>>> We can add a CI pipeline on Microsoft infrastructure by using a GitHub
>>> action.
>> Sorry for being ignorant, but how does that solve the legal questions
>> just because it is running on GitHub instead of a different CI?
>>
>>  Thomas
>>
> 
> Sorry, I though that would be clear by looking at the included shell script.
> 
> The build does not use the Microsoft SDK. It gets the required header
> files from Mingw-w64. They added them in git master.

Oh, so we can do that with GitLab too now, we don't need to rely on the
GitHub 'Actions' CI in particular, right?

> 
> See
> https://github.com/stweil/qemu/blob/master/.github/workflows/build.sh#L50
> for code details.
> 
> It's still shameful that MS is forcing developers to waste time
> rewriting API headers, just because the MS legal departments are not
> able to understand the needs of Open Source development.

There has be a big switch from Microsoft toward Open Source, I attended
some of there talk at the Open Source Summit in 2018. Maybe we simply
haven't contacted the right persons to make the changes...?

> 
> Stefan
> 
> 
> 



On 8/4/20 9:42 AM, Stefan Weil wrote:
> Am 04.08.20 um 09:23 schrieb Philippe Mathieu-Daudé:
> 
>> On 8/4/20 8:55 AM, Stefan Weil wrote:
>>> Am 04.08.20 um 08:43 schrieb Thomas Huth:
>>>
>>>> On 03/08/2020 22.25, Stefan Weil wrote:
>>>>> We can add a CI pipeline on Microsoft infrastructure by using a GitHub
>>>>> action.
>>>> Sorry for being ignorant, but how does that solve the legal questions
>>>> just because it is running on GitHub instead of a different CI?
>>>>
>>>>  Thomas
>>>>
>>> Sorry, I though that would be clear by looking at the included shell script.
>>>
>>> The build does not use the Microsoft SDK. It gets the required header
>>> files from Mingw-w64. They added them in git master.
>> Oh, so we can do that with GitLab too now, we don't need to rely on the
>> GitHub 'Actions' CI in particular, right?
> 
> 
> That's right. The build script was written for Ubuntu, so depending on
> the distribution used for GitLab CI it will need some modifications. If
> GitLab already has a recent Mingw-w64, it might be sufficient to fix the
> case of the header file names. Mingw-w64 uses winhvplatform.h while QEMU
> expects WinHvPlatform.h and so on. I used symbolic links to add the
> camel case filenames.
> 
> 
>>> See
>>> https://github.com/stweil/qemu/blob/master/.github/workflows/build.sh#L50
>>> for code details.
>>>
>>> It's still shameful that MS is forcing developers to waste time
>>> rewriting API headers, just because the MS legal departments are not
>>> able to understand the needs of Open Source development.
>> There has be a big switch from Microsoft toward Open Source, I attended
>> some of there talk at the Open Source Summit in 2018. Maybe we simply
>> haven't contacted the right persons to make the changes...?
> 
> 
> Maybe, but it is difficult to find the right person in a large company
> like MS, and legal departments are often somehow special.

Sunil seems quite active with the WHPX development, and the section is
listed as "Supported [my Microsoft]" in MAINTAINERS. I'm confident we
have someone else able to help use finding the right contacts in the
company :)

> 
> And yes, they learned that Open Source can help them for their business,
> too.
> 
> Stefan
> 
> 
> 



On Tue, Aug 04, 2020 at 10:10:31AM +0200, Thomas Huth wrote:
> On 04/08/2020 09.42, Stefan Weil wrote:
> > Am 04.08.20 um 09:23 schrieb Philippe Mathieu-Daudé:
> > 
> >> On 8/4/20 8:55 AM, Stefan Weil wrote:
> >>> Am 04.08.20 um 08:43 schrieb Thomas Huth:
> >>>
> >>>> On 03/08/2020 22.25, Stefan Weil wrote:
> >>>>> We can add a CI pipeline on Microsoft infrastructure by using a GitHub
> >>>>> action.
> >>>> Sorry for being ignorant, but how does that solve the legal questions
> >>>> just because it is running on GitHub instead of a different CI?
> >>>>
> >>>>  Thomas
> >>>>
> >>> Sorry, I though that would be clear by looking at the included shell script.
> >>>
> >>> The build does not use the Microsoft SDK. It gets the required header
> >>> files from Mingw-w64. They added them in git master.
> 
> Great, thanks for the clarification!
> 
> >> Oh, so we can do that with GitLab too now, we don't need to rely on the
> >> GitHub 'Actions' CI in particular, right?
> > 
> > That's right. The build script was written for Ubuntu, so depending on
> > the distribution used for GitLab CI it will need some modifications. If
> > GitLab already has a recent Mingw-w64, it might be sufficient to fix the
> > case of the header file names. Mingw-w64 uses winhvplatform.h while QEMU
> > expects WinHvPlatform.h and so on. I used symbolic links to add the
> > camel case filenames.
> 
> I'm currently working on a patch series for our gitlab-CI that uses our
> containers to all possible kinds of cross-compiler builds (basically the
> ones that we are doing on shippable.com so far), including the 32-bit
> and 64-bit MinGW cross-compilation jobs. I can have a look whether I can
> integrate these headers there!

Fedora rawhide carries mingw64 v7.0.0, which was released in Nov 2019

The WHPX headers were added to mingw64 git a week later, so they're
not available in any distro yet. 

The mingw64 release schedule looks "sporadic" so maybe we can just
request a new release to make WPHX stuff available. It'll thus be
available for our CI in rawhide/sid shortly thereafter, which will
be the best solution to let us do this in GitLab.

We certainly don't want to add yet another separate CI system just
for WHPX.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



On 8/18/20 11:20 PM, Sunil Muthuswamy wrote:
>>>> It's still shameful that MS is forcing developers to waste time
>>>> rewriting API headers, just because the MS legal departments are not
>>>> able to understand the needs of Open Source development.
>>> There has be a big switch from Microsoft toward Open Source, I attended
>>> some of there talk at the Open Source Summit in 2018. Maybe we simply
>>> haven't contacted the right persons to make the changes...?
>>
>>
>> Maybe, but it is difficult to find the right person in a large company
>> like MS, and legal departments are often somehow special.
>>
>> And yes, they learned that Open Source can help them for their business,
>> too.
>>
>> Stefan
> 
> Mike Battista is the program manager owner of the SDK license and should be
> able to take/respond to any feedback about the SDK licensing for open source
> projects (I have added him here). He has also been added to previous threads
> about the licensing and is also included in this conversation:
> https://bugs.launchpad.net/qemu/+bug/1879672

Hi Mike, thanks for helping us with this issue!

And thanks a lot Sunil to bring Mike here :)

> 
> - Sunil
>  
> 



Removing 'Opinion' and moving back to 'New'; as 'Opinion' is essentially the same as "WONTFIX" but allows discussion to continue. I believe you want a Feature Request tag instead.

If there is still work for us to do, let's move this to Confirmed/Triaged and add the feature request tag.

Thanks!

Hi John,

On 11/4/20 9:01 PM, John Snow wrote:
> Removing 'Opinion' and moving back to 'New'; as 'Opinion' is essentially
> the same as "WONTFIX" but allows discussion to continue. I believe you
> want a Feature Request tag instead.
> 
> If there is still work for us to do, let's move this to
> Confirmed/Triaged and add the feature request tag.

It seems Launchpad didn't Cc'ed the interested parties.

Our contact is Mike Battista (see [*]) but so far he never
made any comment regarding this issue.

Cc'ing Sunil who is the active developer (of WHPX in QEMU)
at Microsoft, and Stefan, who does the Open Source packaging.

Regards,

Phil.

[*] https://<email address hidden>/msg731227.html




This is an automated cleanup. This bug report has been moved to QEMU's
new bug tracker on gitlab.com and thus gets marked as 'invalid' now.
Please continue with the discussion here:

 https://gitlab.com/qemu-project/qemu/-/issues/233


Meanwhile QEMU builds for CI and also my inofficial QEMU installers for Windows use free WHPX headers instead of the copyrighted MS ones, so this issue is fixed. 

But looking at the latest pipeline:
https://gitlab.com/qemu-project/qemu/-/pipelines/310113928
in particular the cross-win64-system job:
https://gitlab.com/qemu-project/qemu/-/jobs/1296341064

WHPX isn't built anymore:

  Targets and accelerators
                      KVM support: NO
                      HAX support: YES
                      HVF support: NO
                     WHPX support: NO
                     NVMM support: NO
                      Xen support: NO
                      TCG support: YES
                      TCG backend: native (x86_64)

We likely lost the coverage with commit 93cc0506f6c
("tests/docker: Use Fedora containers for MinGW cross-builds in the gitlab-CI")

Should we open a new issue?