[RFC 00/10] R300 QEMU device V2

aaron.zakhrov@gmail.com posted 10 patches 4 years, 5 months ago
Only 5 patches received!
[RFC 00/10] R300 QEMU device V2
Posted by aaron.zakhrov@gmail.com 4 years, 5 months ago
From: Aaron Dominick <aaron.zakhrov@gmail.com>

I have removed the botched patches and have got the code working upto the GART initialization.
I am not sure how to implement the GART. I am guessing it should be an IOMMU device but I think that is a bit much for an emulated card. 
The earlier problem of display probing seems to be resolved by using an R300 bios I got from TechPowerUP's GPU database:

	https://www.techpowerup.com/vgabios/14509/14509
I am NOT sure if we can distribute it in the QEMU source tree. If it does cause problems I can send a patch to remove it.

Aaron Dominick (10):
  Add Radeon kernel headers. Will clean up later
  Fix MC STATUS resgister
  R300 fixes
  Got GPU init working. Stops at probing display
  Clean up Radeon Header files
  Add a new R300 VBIOS courtesy of TechPowerUP

-- 
2.24.0


Re: [RFC 00/10] R300 QEMU device V2
Posted by Daniel P. Berrangé 4 years, 5 months ago
On Tue, Nov 26, 2019 at 06:14:27PM +0530, aaron.zakhrov@gmail.com wrote:
> From: Aaron Dominick <aaron.zakhrov@gmail.com>
> 
> I have removed the botched patches and have got the code working upto the GART initialization.
> I am not sure how to implement the GART. I am guessing it should be an IOMMU device but I think that is a bit much for an emulated card. 
> The earlier problem of display probing seems to be resolved by using an R300 bios I got from TechPowerUP's GPU database:
> 
> 	https://www.techpowerup.com/vgabios/14509/14509
> I am NOT sure if we can distribute it in the QEMU source tree. If it
> does cause problems I can send a patch to remove it.

That site seems to be a repository of BIOS uploaded by arbitrary users,
with no information on what license terms might apply to the uploads.

We have to therefore assume the worst and treat the BIOS images on that
site as proprietary and not re-distributable, despite the fact that the
site itself is acting as a 3rd party distributor.

IOW, we can't have this in QEMU git I'm afraid, unless someone can find
a trustworthy vendor source for the original image with accompanying
license information.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|


Re: [RFC 00/10] R300 QEMU device V2
Posted by Philippe Mathieu-Daudé 4 years, 5 months ago
Hi Daniel, Aaron.

On 11/26/19 3:19 PM, Daniel P. Berrangé wrote:
> On Tue, Nov 26, 2019 at 06:14:27PM +0530, aaron.zakhrov@gmail.com wrote:
>> From: Aaron Dominick <aaron.zakhrov@gmail.com>
>>
>> I have removed the botched patches and have got the code working upto the GART initialization.
>> I am not sure how to implement the GART. I am guessing it should be an IOMMU device but I think that is a bit much for an emulated card.
>> The earlier problem of display probing seems to be resolved by using an R300 bios I got from TechPowerUP's GPU database:
>>
>> 	https://www.techpowerup.com/vgabios/14509/14509
>> I am NOT sure if we can distribute it in the QEMU source tree. If it
>> does cause problems I can send a patch to remove it.
> 
> That site seems to be a repository of BIOS uploaded by arbitrary users,
> with no information on what license terms might apply to the uploads.
> 
> We have to therefore assume the worst and treat the BIOS images on that
> site as proprietary and not re-distributable, despite the fact that the
> site itself is acting as a 3rd party distributor.

We can not redistribute this BIOS.

> IOW, we can't have this in QEMU git I'm afraid, unless someone can find
> a trustworthy vendor source for the original image with accompanying
> license information.

Daniel, I think there is no problem if Aaron contributes a model of the 
R300 device to QEMU, right? This doesn't involve redistributing any BIOS.


Aaron: I never received patches 1-4 and 10 of your series, not sure why, 
maybe they are too big.

I recommend to rebase your series, squashing patch 9 over patch 5.

If you don't know about git-rebase, this blog is quite easy to understand:
https://thoughtbot.com/blog/git-interactive-rebase-squash-amend-rewriting-history

Regards,

Phil.


Re: [RFC 00/10] R300 QEMU device V2
Posted by Daniel P. Berrangé 4 years, 5 months ago
On Wed, Nov 27, 2019 at 04:00:01PM +0100, Philippe Mathieu-Daudé wrote:
> Hi Daniel, Aaron.
> 
> On 11/26/19 3:19 PM, Daniel P. Berrangé wrote:
> > On Tue, Nov 26, 2019 at 06:14:27PM +0530, aaron.zakhrov@gmail.com wrote:
> > > From: Aaron Dominick <aaron.zakhrov@gmail.com>
> > > 
> > > I have removed the botched patches and have got the code working upto the GART initialization.
> > > I am not sure how to implement the GART. I am guessing it should be an IOMMU device but I think that is a bit much for an emulated card.
> > > The earlier problem of display probing seems to be resolved by using an R300 bios I got from TechPowerUP's GPU database:
> > > 
> > > 	https://www.techpowerup.com/vgabios/14509/14509
> > > I am NOT sure if we can distribute it in the QEMU source tree. If it
> > > does cause problems I can send a patch to remove it.
> > 
> > That site seems to be a repository of BIOS uploaded by arbitrary users,
> > with no information on what license terms might apply to the uploads.
> > 
> > We have to therefore assume the worst and treat the BIOS images on that
> > site as proprietary and not re-distributable, despite the fact that the
> > site itself is acting as a 3rd party distributor.
> 
> We can not redistribute this BIOS.
> 
> > IOW, we can't have this in QEMU git I'm afraid, unless someone can find
> > a trustworthy vendor source for the original image with accompanying
> > license information.
> 
> Daniel, I think there is no problem if Aaron contributes a model of the R300
> device to QEMU, right? This doesn't involve redistributing any BIOS.

Having just the device impl doesn't cause any legal problems.

It does become a slight usability issue, as any users need to go and find
the suitable BIOS in order to use the device. No downstream OS vendors are
going to be able to distribute this BIOS either

I don't know if we have hit this problem before & if we have any
general policies about it ?


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|


Re: [RFC 00/10] R300 QEMU device V2
Posted by Philippe Mathieu-Daudé 4 years, 5 months ago
Cc'ing more.

On 11/27/19 4:05 PM, Daniel P. Berrangé wrote:
> On Wed, Nov 27, 2019 at 04:00:01PM +0100, Philippe Mathieu-Daudé wrote:
>> Hi Daniel, Aaron.
>>
>> On 11/26/19 3:19 PM, Daniel P. Berrangé wrote:
>>> On Tue, Nov 26, 2019 at 06:14:27PM +0530, aaron.zakhrov@gmail.com wrote:
>>>> From: Aaron Dominick <aaron.zakhrov@gmail.com>
>>>>
>>>> I have removed the botched patches and have got the code working upto the GART initialization.
>>>> I am not sure how to implement the GART. I am guessing it should be an IOMMU device but I think that is a bit much for an emulated card.
>>>> The earlier problem of display probing seems to be resolved by using an R300 bios I got from TechPowerUP's GPU database:
>>>>
>>>> 	https://www.techpowerup.com/vgabios/14509/14509
>>>> I am NOT sure if we can distribute it in the QEMU source tree. If it
>>>> does cause problems I can send a patch to remove it.
>>>
>>> That site seems to be a repository of BIOS uploaded by arbitrary users,
>>> with no information on what license terms might apply to the uploads.
>>>
>>> We have to therefore assume the worst and treat the BIOS images on that
>>> site as proprietary and not re-distributable, despite the fact that the
>>> site itself is acting as a 3rd party distributor.
>>
>> We can not redistribute this BIOS.
>>
>>> IOW, we can't have this in QEMU git I'm afraid, unless someone can find
>>> a trustworthy vendor source for the original image with accompanying
>>> license information.
>>
>> Daniel, I think there is no problem if Aaron contributes a model of the R300
>> device to QEMU, right? This doesn't involve redistributing any BIOS.
> 
> Having just the device impl doesn't cause any legal problems.
> 
> It does become a slight usability issue, as any users need to go and find
> the suitable BIOS in order to use the device. No downstream OS vendors are
> going to be able to distribute this BIOS either
> 
> I don't know if we have hit this problem before & if we have any
> general policies about it ?

This appears to me the same case as a user downloading a proprietary 
driver/firmware at some vendor website, accepting the EULA, then using 
the driver within a QEMU VM.

No close source blob are redistributed, but users can still run them, 
and both party (the QEMU project, and the users) are legally correct, 
right? But IANAL.


Re: [RFC 00/10] R300 QEMU device V2
Posted by Daniel P. Berrangé 4 years, 5 months ago
On Wed, Nov 27, 2019 at 04:13:18PM +0100, Philippe Mathieu-Daudé wrote:
> Cc'ing more.
> 
> On 11/27/19 4:05 PM, Daniel P. Berrangé wrote:
> > On Wed, Nov 27, 2019 at 04:00:01PM +0100, Philippe Mathieu-Daudé wrote:
> > > Hi Daniel, Aaron.
> > > 
> > > On 11/26/19 3:19 PM, Daniel P. Berrangé wrote:
> > > > On Tue, Nov 26, 2019 at 06:14:27PM +0530, aaron.zakhrov@gmail.com wrote:
> > > > > From: Aaron Dominick <aaron.zakhrov@gmail.com>
> > > > > 
> > > > > I have removed the botched patches and have got the code working upto the GART initialization.
> > > > > I am not sure how to implement the GART. I am guessing it should be an IOMMU device but I think that is a bit much for an emulated card.
> > > > > The earlier problem of display probing seems to be resolved by using an R300 bios I got from TechPowerUP's GPU database:
> > > > > 
> > > > > 	https://www.techpowerup.com/vgabios/14509/14509
> > > > > I am NOT sure if we can distribute it in the QEMU source tree. If it
> > > > > does cause problems I can send a patch to remove it.
> > > > 
> > > > That site seems to be a repository of BIOS uploaded by arbitrary users,
> > > > with no information on what license terms might apply to the uploads.
> > > > 
> > > > We have to therefore assume the worst and treat the BIOS images on that
> > > > site as proprietary and not re-distributable, despite the fact that the
> > > > site itself is acting as a 3rd party distributor.
> > > 
> > > We can not redistribute this BIOS.
> > > 
> > > > IOW, we can't have this in QEMU git I'm afraid, unless someone can find
> > > > a trustworthy vendor source for the original image with accompanying
> > > > license information.
> > > 
> > > Daniel, I think there is no problem if Aaron contributes a model of the R300
> > > device to QEMU, right? This doesn't involve redistributing any BIOS.
> > 
> > Having just the device impl doesn't cause any legal problems.
> > 
> > It does become a slight usability issue, as any users need to go and find
> > the suitable BIOS in order to use the device. No downstream OS vendors are
> > going to be able to distribute this BIOS either
> > 
> > I don't know if we have hit this problem before & if we have any
> > general policies about it ?
> 
> This appears to me the same case as a user downloading a proprietary
> driver/firmware at some vendor website, accepting the EULA, then using the
> driver within a QEMU VM.
> 
> No close source blob are redistributed, but users can still run them, and
> both party (the QEMU project, and the users) are legally correct, right? But
> IANAL.

Well in this case the user wouldn't be reading or accepting any EULA,
because this 3rd party site doesn't provide any such things with the
firmware it is re-distributing.

IOW, the only way the user can use this device is to download the
firmware from this 3rd site & ignore the lack of legal information
they provide about license/distribution rules.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|


Re: [RFC 00/10] R300 QEMU device V2
Posted by Gerd Hoffmann 4 years, 5 months ago
  Hi,

> It does become a slight usability issue, as any users need to go and find
> the suitable BIOS in order to use the device. No downstream OS vendors are
> going to be able to distribute this BIOS either
> 
> I don't know if we have hit this problem before & if we have any
> general policies about it ?

Booting from lsi scsi adapter used to work with a vendor bios only
loooooong ago.  Fixed by adding an lsi driver to seabios.

Building a r300 vgabios shouldn't be too hard, we already have
support in seavgabios for the other ati variants emulated by qemu.

cheers,
  Gerd


Re: [RFC 00/10] R300 QEMU device V2
Posted by Daniel P. Berrangé 4 years, 5 months ago
On Wed, Nov 27, 2019 at 05:12:10PM +0100, Gerd Hoffmann wrote:
>   Hi,
> 
> > It does become a slight usability issue, as any users need to go and find
> > the suitable BIOS in order to use the device. No downstream OS vendors are
> > going to be able to distribute this BIOS either
> > 
> > I don't know if we have hit this problem before & if we have any
> > general policies about it ?
> 
> Booting from lsi scsi adapter used to work with a vendor bios only
> loooooong ago.  Fixed by adding an lsi driver to seabios.
> 
> Building a r300 vgabios shouldn't be too hard, we already have
> support in seavgabios for the other ati variants emulated by qemu.

That sounds reasonable, so it is fine to add r300 to QEMU without the BIOS.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|


Re: [RFC 00/10] R300 QEMU device V2
Posted by Aaron Zakhrov 4 years, 5 months ago
I tested my code with the vgabios-ati.bin rom file and it seems to get
passed the earlier issue I had.
I have cleaned up my code and have sent a new patch series. The new one is
pretty big but it contains only the necessary header files and it should be
a little easier to review

On Wed, Nov 27, 2019 at 10:02 PM Daniel P. Berrangé <berrange@redhat.com>
wrote:

> On Wed, Nov 27, 2019 at 05:12:10PM +0100, Gerd Hoffmann wrote:
> >   Hi,
> >
> > > It does become a slight usability issue, as any users need to go and
> find
> > > the suitable BIOS in order to use the device. No downstream OS vendors
> are
> > > going to be able to distribute this BIOS either
> > >
> > > I don't know if we have hit this problem before & if we have any
> > > general policies about it ?
> >
> > Booting from lsi scsi adapter used to work with a vendor bios only
> > loooooong ago.  Fixed by adding an lsi driver to seabios.
> >
> > Building a r300 vgabios shouldn't be too hard, we already have
> > support in seavgabios for the other ati variants emulated by qemu.
>
> That sounds reasonable, so it is fine to add r300 to QEMU without the BIOS.
>
> Regards,
> Daniel
> --
> |: https://berrange.com      -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-
> https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-
> https://www.instagram.com/dberrange :|
>
>
Re: [RFC 00/10] R300 QEMU device V2
Posted by BALATON Zoltan 4 years, 5 months ago
Hello,

On Thu, 28 Nov 2019, Aaron Zakhrov wrote:
> I tested my code with the vgabios-ati.bin rom file and it seems to get
> passed the earlier issue I had.

Good, so then the BIOS problem seems to be sorted for now. (Maybe it 
needed the tables or EDID support that some drivers use from the VGA 
BIOS.)

> I have cleaned up my code and have sent a new patch series. The new one is

I still got pathces twice so rebase may not have worked completely and 
changes are still not separate patches but hopefully there are less 
unnecessary files in the last series. If you have problem with rebase and 
using git to rework patches you could also start from a clean git tree 
then apply patches with patch command and make new separate commits to 
clean them up. This may be easier to do until you get familiar with git's 
more obscure commands.

> pretty big but it contains only the necessary header files and it should be
> a little easier to review

I don't have time now to check all the added registers but is there a 
reason you're targetting r300 instead of trying to make rv100 working 
first? RV100 is a simpler chip with less features so you probably will 
have less to implement and it's also clearer what might be needed than 
having to implement a lot of new features the newer r300 may have. Once 
rv100 works it may be easier to update that to r300 than going for that 
right away, at least that's what I thought. If you think you can do all of 
r300 features or you need that for some reason I'm fine with trying to 
target r300 just saying that a more incremental approach may be easier to 
do.

I haven't checked this but I think what you get now is that the driver is 
trying to set up shared memory buffers via GART that it will use to send 
command packets to the GPU that the emulated chip will need to parse and 
convert to register access. This is the microengine/command 
processor/CCE/PM4 I've referred to before. Unless we implement this in 
some way it won't work as communication between the driver and card is 
done using this facility so this should be the next step before adding 
more registers and emulation. If you search for "ati microengine" it may 
turn up some documentation on this where the buffers and command packets 
are described but the actual microengine and it's microcode appears to be 
undocumented. I've also said before that Xenia emulator has some code to 
parse command packets of the XBox 360 GPU which is similar to some late 
r5xx GPUs so some of these might be useful for emulating previous Radeons 
as well.

Regards,
BALATON Zoltan

Re: [RFC 00/10] R300 QEMU device V2
Posted by BALATON Zoltan 4 years, 5 months ago
On Wed, 27 Nov 2019, Daniel P. Berrangé wrote:
> On Wed, Nov 27, 2019 at 04:00:01PM +0100, Philippe Mathieu-Daudé wrote:
>> Hi Daniel, Aaron.
>>
>> On 11/26/19 3:19 PM, Daniel P. Berrangé wrote:
>>> On Tue, Nov 26, 2019 at 06:14:27PM +0530, aaron.zakhrov@gmail.com wrote:
>>>> From: Aaron Dominick <aaron.zakhrov@gmail.com>
>>>>
>>>> I have removed the botched patches and have got the code working upto the GART initialization.
>>>> I am not sure how to implement the GART. I am guessing it should be an IOMMU device but I think that is a bit much for an emulated card.
>>>> The earlier problem of display probing seems to be resolved by using an R300 bios I got from TechPowerUP's GPU database:
>>>>
>>>> 	https://www.techpowerup.com/vgabios/14509/14509
>>>> I am NOT sure if we can distribute it in the QEMU source tree. If it
>>>> does cause problems I can send a patch to remove it.
>>>
>>> That site seems to be a repository of BIOS uploaded by arbitrary users,
>>> with no information on what license terms might apply to the uploads.
>>>
>>> We have to therefore assume the worst and treat the BIOS images on that
>>> site as proprietary and not re-distributable, despite the fact that the
>>> site itself is acting as a 3rd party distributor.
>>
>> We can not redistribute this BIOS.
>>
>>> IOW, we can't have this in QEMU git I'm afraid, unless someone can find
>>> a trustworthy vendor source for the original image with accompanying
>>> license information.
>>
>> Daniel, I think there is no problem if Aaron contributes a model of the R300
>> device to QEMU, right? This doesn't involve redistributing any BIOS.
>
> Having just the device impl doesn't cause any legal problems.
>
> It does become a slight usability issue, as any users need to go and find
> the suitable BIOS in order to use the device. No downstream OS vendors are
> going to be able to distribute this BIOS either

We may be able to avoid this problem if we identify what the driver needs 
from the BIOS and implement that in our vgabios. Gerd has already added 
some tables that some drivers need in the latest vgabios version that's 
currently in git master (but those were for Rage128 and RV100 that ati-vga 
currently implements). Aaron, did you try with latest git master and does 
that still need a BIOS from a real card or if so do you happen to know 
what the driver needs from the BIOS? (It may be some tables/structure or 
BIOS calls that the QEMU vgabios-ati does not implement yet.) If that's 
not too difficult to add it may be implemented in QEMU's vgabios to avoid 
needing proprietary blobs. It could also be EDID access that may use 
different registers on R300 but I think that may be simple to fix if more 
details are known.

> I don't know if we have hit this problem before & if we have any
> general policies about it ?

I don't know but this may be similar to boards needing firmware ROMs or 
the firmware blobs needed by some Linux kernel drivers. How are those
handled? Distros usually put them in a non-free repo I think.

Regrads,
BALATON Zoltan