[PATCH V4 0/6] xen: Add support of extended regions (safe ranges) on Arm

Oleksandr Tyshchenko posted 6 patches 2 years, 4 months ago
Failed in applying to current master (apply log)
Documentation/devicetree/bindings/arm/xen.txt |  14 +--
arch/arm/xen/enlighten.c                      | 132 ++++++++++++++++++++++++--
drivers/xen/Kconfig                           |   2 +-
drivers/xen/balloon.c                         |  20 ++--
drivers/xen/unpopulated-alloc.c               |  87 ++++++++++++++++-
include/xen/balloon.h                         |   3 +
include/xen/xen.h                             |  16 ++++
7 files changed, 245 insertions(+), 29 deletions(-)
[PATCH V4 0/6] xen: Add support of extended regions (safe ranges) on Arm
Posted by Oleksandr Tyshchenko 2 years, 4 months ago
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Hello all.

You can find the RFC-V3 patch series at [1],[2] and [3].

The corresponding Xen support (for both Dom0 and DomU) is already committed and
is available in mainline Xen since the following commit:
57f87857dc2de452a796d6bad4f476510efd2aba libxl/arm: Add handling of extended regions for DomU

The extended region (safe range) is a region of guest physical address space
which is unused and could be safely used to create grant/foreign mappings instead
of ballooning out real RAM pages to obtain a physical address space for creating
these mappings (which simply results in wasting domain memory and shattering super
pages in P2M table).

The problem is that we cannot follow Linux advise which memory ranges are unused
on Arm as there might be some identity mappings in P2M table (stage 2) the guest is not
aware of or not all device I/O regions might be known (registered) by the time the guest
starts creating grant/foreign mappings. This is why we need some hints from the hypervisor
which knows all details in advance to be able to choose extended regions (which won't
clash with other resources).

The extended regions are chosen at the domain creation time and advertised to it via
"reg" property under hypervisor node in the guest device-tree [4]. As region 0 is reserved
for grant table space (always present), the indexes for extended regions are 1...N.
No device tree bindings update is needed, guest infers the presence of extended regions
from the number of regions in "reg" property.

Please note the following:
- The ACPI case is not covered for now
- patch series was created in a way to retain existing behavior on x86

The patch series is based on v5.16-rc3 and also available at [5], it was fully
tested on Arm64 and only compile tested on x86.

[1] https://lore.kernel.org/all/1627490656-1267-1-git-send-email-olekstysh@gmail.com/
    https://lore.kernel.org/all/1627490656-1267-2-git-send-email-olekstysh@gmail.com/
[2] https://lore.kernel.org/all/1635264312-3796-1-git-send-email-olekstysh@gmail.com/
[3] https://lore.kernel.org/all/1637787223-21129-1-git-send-email-olekstysh@gmail.com/
[4] https://xenbits.xen.org/gitweb/?p=xen.git;a=blob_plain;f=docs/misc/arm/device-tree/guest.txt;hb=refs/heads/master
[5] https://github.com/otyshchenko1/linux/commits/map_opt_ml7

Oleksandr Tyshchenko (6):
  xen/unpopulated-alloc: Drop check for virt_addr_valid() in fill_list()
  arm/xen: Switch to use gnttab_setup_auto_xlat_frames() for DT
  xen/balloon: Bring alloc(free)_xenballooned_pages helpers back
  xen/unpopulated-alloc: Add mechanism to use Xen resource
  arm/xen: Read extended regions from DT and init Xen resource
  dt-bindings: xen: Clarify "reg" purpose

 Documentation/devicetree/bindings/arm/xen.txt |  14 +--
 arch/arm/xen/enlighten.c                      | 132 ++++++++++++++++++++++++--
 drivers/xen/Kconfig                           |   2 +-
 drivers/xen/balloon.c                         |  20 ++--
 drivers/xen/unpopulated-alloc.c               |  87 ++++++++++++++++-
 include/xen/balloon.h                         |   3 +
 include/xen/xen.h                             |  16 ++++
 7 files changed, 245 insertions(+), 29 deletions(-)

-- 
2.7.4


Re: [PATCH V4 0/6] xen: Add support of extended regions (safe ranges) on Arm
Posted by Oleksandr 2 years, 4 months ago
On 09.12.21 22:05, Oleksandr Tyshchenko wrote:


Hello Juergen, Boris


May I please ask, are you happy (or otherwise) with current patch series 
(I assume, especially with commits #3-4)?

For the convenience:

   1. xen/unpopulated-alloc: Drop check for virt_addr_valid() in fill_list()
- Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>

   2. arm/xen: Switch to use gnttab_setup_auto_xlat_frames() for DT
- Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

   3. xen/balloon: Bring alloc(free)_xenballooned_pages helpers back
- Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

   4. xen/unpopulated-alloc: Add mechanism to use Xen resource
- Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

   5. arm/xen: Read extended regions from DT and init Xen resource
- Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

   6. dt-bindings: xen: Clarify "reg" purpose
- Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
- Acked-by: Rob Herring <robh@kernel.org>
- Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> Hello all.
>
> You can find the RFC-V3 patch series at [1],[2] and [3].
>
> The corresponding Xen support (for both Dom0 and DomU) is already committed and
> is available in mainline Xen since the following commit:
> 57f87857dc2de452a796d6bad4f476510efd2aba libxl/arm: Add handling of extended regions for DomU
>
> The extended region (safe range) is a region of guest physical address space
> which is unused and could be safely used to create grant/foreign mappings instead
> of ballooning out real RAM pages to obtain a physical address space for creating
> these mappings (which simply results in wasting domain memory and shattering super
> pages in P2M table).
>
> The problem is that we cannot follow Linux advise which memory ranges are unused
> on Arm as there might be some identity mappings in P2M table (stage 2) the guest is not
> aware of or not all device I/O regions might be known (registered) by the time the guest
> starts creating grant/foreign mappings. This is why we need some hints from the hypervisor
> which knows all details in advance to be able to choose extended regions (which won't
> clash with other resources).
>
> The extended regions are chosen at the domain creation time and advertised to it via
> "reg" property under hypervisor node in the guest device-tree [4]. As region 0 is reserved
> for grant table space (always present), the indexes for extended regions are 1...N.
> No device tree bindings update is needed, guest infers the presence of extended regions
> from the number of regions in "reg" property.
>
> Please note the following:
> - The ACPI case is not covered for now
> - patch series was created in a way to retain existing behavior on x86
>
> The patch series is based on v5.16-rc3 and also available at [5], it was fully
> tested on Arm64 and only compile tested on x86.
>
> [1] https://lore.kernel.org/all/1627490656-1267-1-git-send-email-olekstysh@gmail.com/
>      https://lore.kernel.org/all/1627490656-1267-2-git-send-email-olekstysh@gmail.com/
> [2] https://lore.kernel.org/all/1635264312-3796-1-git-send-email-olekstysh@gmail.com/
> [3] https://lore.kernel.org/all/1637787223-21129-1-git-send-email-olekstysh@gmail.com/
> [4] https://xenbits.xen.org/gitweb/?p=xen.git;a=blob_plain;f=docs/misc/arm/device-tree/guest.txt;hb=refs/heads/master
> [5] https://github.com/otyshchenko1/linux/commits/map_opt_ml7
>
> Oleksandr Tyshchenko (6):
>    xen/unpopulated-alloc: Drop check for virt_addr_valid() in fill_list()
>    arm/xen: Switch to use gnttab_setup_auto_xlat_frames() for DT
>    xen/balloon: Bring alloc(free)_xenballooned_pages helpers back
>    xen/unpopulated-alloc: Add mechanism to use Xen resource
>    arm/xen: Read extended regions from DT and init Xen resource
>    dt-bindings: xen: Clarify "reg" purpose
>
>   Documentation/devicetree/bindings/arm/xen.txt |  14 +--
>   arch/arm/xen/enlighten.c                      | 132 ++++++++++++++++++++++++--
>   drivers/xen/Kconfig                           |   2 +-
>   drivers/xen/balloon.c                         |  20 ++--
>   drivers/xen/unpopulated-alloc.c               |  87 ++++++++++++++++-
>   include/xen/balloon.h                         |   3 +
>   include/xen/xen.h                             |  16 ++++
>   7 files changed, 245 insertions(+), 29 deletions(-)
>
-- 
Regards,

Oleksandr Tyshchenko

Re: [PATCH V4 0/6] xen: Add support of extended regions (safe ranges) on Arm
Posted by Boris Ostrovsky 2 years, 4 months ago
On 12/16/21 5:02 PM, Oleksandr wrote:
>
> On 09.12.21 22:05, Oleksandr Tyshchenko wrote:
>
>
> Hello Juergen, Boris
>
>
> May I please ask, are you happy (or otherwise) with current patch series (I assume, especially with commits #3-4)?


I think I mentioned last time, in patch 4:


+	if (target_resource != &iomem_resource) {
+		tmp_res = kzalloc(sizeof(*tmp_res), GFP_KERNEL);
+		if (!res) {


Other than that --- LGTM.


-boris


Re: [PATCH V4 0/6] xen: Add support of extended regions (safe ranges) on Arm
Posted by Oleksandr 2 years, 4 months ago
On 17.12.21 21:13, Boris Ostrovsky wrote:

Hi Boris

>
> On 12/16/21 5:02 PM, Oleksandr wrote:
>>
>> On 09.12.21 22:05, Oleksandr Tyshchenko wrote:
>>
>>
>> Hello Juergen, Boris
>>
>>
>> May I please ask, are you happy (or otherwise) with current patch 
>> series (I assume, especially with commits #3-4)?
>
>
> I think I mentioned last time, in patch 4:
>
>
> +    if (target_resource != &iomem_resource) {
> +        tmp_res = kzalloc(sizeof(*tmp_res), GFP_KERNEL);
> +        if (!res) {

Yes, this is already fixed in V4 of patch [1]



>
>
> Other than that --- LGTM.
Thank you!


[1] 
https://lore.kernel.org/xen-devel/1639080336-26573-5-git-send-email-olekstysh@gmail.com/


>
>
> -boris
>
-- 
Regards,

Oleksandr Tyshchenko


Re: [PATCH V4 0/6] xen: Add support of extended regions (safe ranges) on Arm
Posted by Boris Ostrovsky 2 years, 4 months ago
On 12/17/21 2:19 PM, Oleksandr wrote:
>
> On 17.12.21 21:13, Boris Ostrovsky wrote:
>
> Hi Boris
>
>>
>> On 12/16/21 5:02 PM, Oleksandr wrote:
>>>
>>> On 09.12.21 22:05, Oleksandr Tyshchenko wrote:
>>>
>>>
>>> Hello Juergen, Boris
>>>
>>>
>>> May I please ask, are you happy (or otherwise) with current patch series (I assume, especially with commits #3-4)?
>>
>>
>> I think I mentioned last time, in patch 4:
>>
>>
>> +    if (target_resource != &iomem_resource) {
>> +        tmp_res = kzalloc(sizeof(*tmp_res), GFP_KERNEL);
>> +        if (!res) {
>
> Yes, this is already fixed in V4 of patch [1]



So this email did escape. Yes, I realized I was looking at V3 but apparently didn't hit ^C quickly enough ;-)



-boris





Re: [PATCH V4 0/6] xen: Add support of extended regions (safe ranges) on Arm
Posted by Boris Ostrovsky 2 years, 4 months ago
On 12/16/21 5:02 PM, Oleksandr wrote:
>
> On 09.12.21 22:05, Oleksandr Tyshchenko wrote:
>
>
> Hello Juergen, Boris
>
>
> May I please ask, are you happy (or otherwise) with current patch series (I assume, especially with commits #3-4)?


LGTM, I have no more comments.


-boris




Re: [PATCH V4 0/6] xen: Add support of extended regions (safe ranges) on Arm
Posted by Juergen Gross 2 years, 3 months ago
On 09.12.21 21:05, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> Hello all.
> 
> You can find the RFC-V3 patch series at [1],[2] and [3].
> 
> The corresponding Xen support (for both Dom0 and DomU) is already committed and
> is available in mainline Xen since the following commit:
> 57f87857dc2de452a796d6bad4f476510efd2aba libxl/arm: Add handling of extended regions for DomU
> 
> The extended region (safe range) is a region of guest physical address space
> which is unused and could be safely used to create grant/foreign mappings instead
> of ballooning out real RAM pages to obtain a physical address space for creating
> these mappings (which simply results in wasting domain memory and shattering super
> pages in P2M table).
> 
> The problem is that we cannot follow Linux advise which memory ranges are unused
> on Arm as there might be some identity mappings in P2M table (stage 2) the guest is not
> aware of or not all device I/O regions might be known (registered) by the time the guest
> starts creating grant/foreign mappings. This is why we need some hints from the hypervisor
> which knows all details in advance to be able to choose extended regions (which won't
> clash with other resources).
> 
> The extended regions are chosen at the domain creation time and advertised to it via
> "reg" property under hypervisor node in the guest device-tree [4]. As region 0 is reserved
> for grant table space (always present), the indexes for extended regions are 1...N.
> No device tree bindings update is needed, guest infers the presence of extended regions
> from the number of regions in "reg" property.
> 
> Please note the following:
> - The ACPI case is not covered for now
> - patch series was created in a way to retain existing behavior on x86
> 
> The patch series is based on v5.16-rc3 and also available at [5], it was fully
> tested on Arm64 and only compile tested on x86.
> 
> [1] https://lore.kernel.org/all/1627490656-1267-1-git-send-email-olekstysh@gmail.com/
>      https://lore.kernel.org/all/1627490656-1267-2-git-send-email-olekstysh@gmail.com/
> [2] https://lore.kernel.org/all/1635264312-3796-1-git-send-email-olekstysh@gmail.com/
> [3] https://lore.kernel.org/all/1637787223-21129-1-git-send-email-olekstysh@gmail.com/
> [4] https://xenbits.xen.org/gitweb/?p=xen.git;a=blob_plain;f=docs/misc/arm/device-tree/guest.txt;hb=refs/heads/master
> [5] https://github.com/otyshchenko1/linux/commits/map_opt_ml7
> 
> Oleksandr Tyshchenko (6):
>    xen/unpopulated-alloc: Drop check for virt_addr_valid() in fill_list()
>    arm/xen: Switch to use gnttab_setup_auto_xlat_frames() for DT
>    xen/balloon: Bring alloc(free)_xenballooned_pages helpers back
>    xen/unpopulated-alloc: Add mechanism to use Xen resource
>    arm/xen: Read extended regions from DT and init Xen resource
>    dt-bindings: xen: Clarify "reg" purpose
> 
>   Documentation/devicetree/bindings/arm/xen.txt |  14 +--
>   arch/arm/xen/enlighten.c                      | 132 ++++++++++++++++++++++++--
>   drivers/xen/Kconfig                           |   2 +-
>   drivers/xen/balloon.c                         |  20 ++--
>   drivers/xen/unpopulated-alloc.c               |  87 ++++++++++++++++-
>   include/xen/balloon.h                         |   3 +
>   include/xen/xen.h                             |  16 ++++
>   7 files changed, 245 insertions(+), 29 deletions(-)
> 

Series pushed to xen/tip.git for-linus-5.17


Juergen