Drivers: hv: balloon: Disable balloon and hot-add accordingly

[RFC v1.1] Drivers: hv: balloon: Disable balloon and hot-add accordingly

Posted by Boqun Feng 4 years, 3 months ago

Currently there are known potential issues for balloon and hot-add on
ARM64:

*	Unballoon requests from Hyper-V should only unballoon ranges
	that are guest page size aligned, otherwise guests cannot handle
	because it's impossible to partially free a page.

*	Memory hot-add requests from Hyper-V should provide the NUMA
	node id of the added ranges or ARM64 should have a functional
	memory_add_physaddr_to_nid(), otherwise the node id is missing
	for add_memory().

These issues require discussions on design and implementation. In the
meanwhile, post_status() is working and essiential to guest monitoring.
Therefore instead of the entire hv_balloon driver, the balloon and
hot-add are disabled accordingly for now. Once the issues are fixed,
they can be re-enable in these cases.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
v1 --> v1.1:

*	Use HV_HYP_PAGE_SIZE instead of hard coding 4096 as suggested by
	Michael.

*	Explicitly print out the disable message if a function is
	disabled as suggested by Michael.

 drivers/hv/hv_balloon.c | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)

diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c
index 062156b88a87..eee7402cfc02 100644
--- a/drivers/hv/hv_balloon.c
+++ b/drivers/hv/hv_balloon.c
@@ -1660,6 +1660,38 @@ static void disable_page_reporting(void)
 	}
 }
 
+static int ballooning_enabled(void)
+{
+	/*
+	 * Disable ballooning if the page size is not 4k (HV_HYP_PAGE_SIZE),
+	 * since currently it's unclear to us whether an unballoon request can
+	 * make sure all page ranges are guest page size aligned.
+	 */
+	if (PAGE_SIZE != HV_HYP_PAGE_SIZE) {
+		pr_info("Ballooning disabled because page size is not 4096 bytes\n");
+		return 0;
+	}
+
+	return 1;
+}
+
+static int hot_add_enabled(void)
+{
+	/*
+	 * Disable hot add on ARM64, because we currently rely on
+	 * memory_add_physaddr_to_nid() to get a node id of a hot add range,
+	 * however ARM64's memory_add_physaddr_to_nid() always return 0 and
+	 * DM_MEM_HOT_ADD_REQUEST doesn't have the NUMA node information for
+	 * add_memory().
+	 */
+	if (IS_ENABLED(CONFIG_ARM64)) {
+		pr_info("Memory hot add disabled on ARM64\n");
+		return 0;
+	}
+
+	return 1;
+}
+
 static int balloon_connect_vsp(struct hv_device *dev)
 {
 	struct dm_version_request version_req;
@@ -1731,8 +1763,8 @@ static int balloon_connect_vsp(struct hv_device *dev)
 	 * currently still requires the bits to be set, so we have to add code
 	 * to fail the host's hot-add and balloon up/down requests, if any.
 	 */
-	cap_msg.caps.cap_bits.balloon = 1;
-	cap_msg.caps.cap_bits.hot_add = 1;
+	cap_msg.caps.cap_bits.balloon = ballooning_enabled();
+	cap_msg.caps.cap_bits.hot_add = hot_add_enabled();
 
 	/*
 	 * Specify our alignment requirements as it relates
-- 
2.33.0

RE: [RFC v1.1] Drivers: hv: balloon: Disable balloon and hot-add accordingly

Posted by Michael Kelley (LINUX) 4 years, 3 months ago

From: Boqun Feng <boqun.feng@gmail.com> Sent: Thursday, February 24, 2022 6:17 PM
> 
> Currently there are known potential issues for balloon and hot-add on
> ARM64:
> 
> *	Unballoon requests from Hyper-V should only unballoon ranges
> 	that are guest page size aligned, otherwise guests cannot handle
> 	because it's impossible to partially free a page.

The above problem occurs only when the guest page size is > 4 Kbytes.

> 
> *	Memory hot-add requests from Hyper-V should provide the NUMA
> 	node id of the added ranges or ARM64 should have a functional
> 	memory_add_physaddr_to_nid(), otherwise the node id is missing
> 	for add_memory().
> 
> These issues require discussions on design and implementation. In the
> meanwhile, post_status() is working and essiential to guest monitoring.

s/essiential/essential/

> Therefore instead of the entire hv_balloon driver, the balloon and
> hot-add are disabled accordingly for now. Once the issues are fixed,
> they can be re-enable in these cases.

Missing the word "disabling" in the first line?  Also the balloon
function is disabled only if the page size is > 4 Kbytes.

> 
> Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> ---
> v1 --> v1.1:
> 
> *	Use HV_HYP_PAGE_SIZE instead of hard coding 4096 as suggested by
> 	Michael.
> 
> *	Explicitly print out the disable message if a function is
> 	disabled as suggested by Michael.
> 
>  drivers/hv/hv_balloon.c | 36 ++++++++++++++++++++++++++++++++++--
>  1 file changed, 34 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c
> index 062156b88a87..eee7402cfc02 100644
> --- a/drivers/hv/hv_balloon.c
> +++ b/drivers/hv/hv_balloon.c
> @@ -1660,6 +1660,38 @@ static void disable_page_reporting(void)
>  	}
>  }
> 
> +static int ballooning_enabled(void)
> +{
> +	/*
> +	 * Disable ballooning if the page size is not 4k (HV_HYP_PAGE_SIZE),
> +	 * since currently it's unclear to us whether an unballoon request can
> +	 * make sure all page ranges are guest page size aligned.

My interpretation of the conversations with Hyper-V is that that they clearly
don't guarantee page ranges are guest page aligned.

> +	 */
> +	if (PAGE_SIZE != HV_HYP_PAGE_SIZE) {
> +		pr_info("Ballooning disabled because page size is not 4096 bytes\n");
> +		return 0;
> +	}
> +
> +	return 1;
> +}
> +
> +static int hot_add_enabled(void)
> +{
> +	/*
> +	 * Disable hot add on ARM64, because we currently rely on
> +	 * memory_add_physaddr_to_nid() to get a node id of a hot add range,
> +	 * however ARM64's memory_add_physaddr_to_nid() always return 0 and
> +	 * DM_MEM_HOT_ADD_REQUEST doesn't have the NUMA node information for
> +	 * add_memory().
> +	 */
> +	if (IS_ENABLED(CONFIG_ARM64)) {
> +		pr_info("Memory hot add disabled on ARM64\n");
> +		return 0;
> +	}
> +
> +	return 1;
> +}
> +
>  static int balloon_connect_vsp(struct hv_device *dev)
>  {
>  	struct dm_version_request version_req;
> @@ -1731,8 +1763,8 @@ static int balloon_connect_vsp(struct hv_device *dev)
>  	 * currently still requires the bits to be set, so we have to add code
>  	 * to fail the host's hot-add and balloon up/down requests, if any.
>  	 */
> -	cap_msg.caps.cap_bits.balloon = 1;
> -	cap_msg.caps.cap_bits.hot_add = 1;
> +	cap_msg.caps.cap_bits.balloon = ballooning_enabled();
> +	cap_msg.caps.cap_bits.hot_add = hot_add_enabled();
> 
>  	/*
>  	 * Specify our alignment requirements as it relates
> --
> 2.33.0

The code looks good to me.

Michael

Re: [RFC v1.1] Drivers: hv: balloon: Disable balloon and hot-add accordingly

Posted by Boqun Feng 4 years, 3 months ago

On Fri, Feb 25, 2022 at 05:06:45PM +0000, Michael Kelley (LINUX) wrote:
> From: Boqun Feng <boqun.feng@gmail.com> Sent: Thursday, February 24, 2022 6:17 PM
> > 
> > Currently there are known potential issues for balloon and hot-add on
> > ARM64:
> > 
> > *	Unballoon requests from Hyper-V should only unballoon ranges
> > 	that are guest page size aligned, otherwise guests cannot handle
> > 	because it's impossible to partially free a page.
> 
> The above problem occurs only when the guest page size is > 4 Kbytes.
> 

Ok, I wil call it out in next version.

> > 
> > *	Memory hot-add requests from Hyper-V should provide the NUMA
> > 	node id of the added ranges or ARM64 should have a functional
> > 	memory_add_physaddr_to_nid(), otherwise the node id is missing
> > 	for add_memory().
> > 
> > These issues require discussions on design and implementation. In the
> > meanwhile, post_status() is working and essiential to guest monitoring.
> 
> s/essiential/essential/
> 
> > Therefore instead of the entire hv_balloon driver, the balloon and
> > hot-add are disabled accordingly for now. Once the issues are fixed,
> > they can be re-enable in these cases.
> 
> Missing the word "disabling" in the first line?  Also the balloon

The phrasing that I was trying to use here is "Instead of A, B and C are
disabled" or "B and C are disabled instead of A". Looks like I'm
inventing my own English? Any I will add the "disabling" in the next
version ;-)

Regards,
Boqun

> function is disabled only if the page size is > 4 Kbytes.
> 
> > 
> > Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> > ---
> > v1 --> v1.1:
> > 
> > *	Use HV_HYP_PAGE_SIZE instead of hard coding 4096 as suggested by
> > 	Michael.
> > 
> > *	Explicitly print out the disable message if a function is
> > 	disabled as suggested by Michael.
> > 
> >  drivers/hv/hv_balloon.c | 36 ++++++++++++++++++++++++++++++++++--
> >  1 file changed, 34 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c
> > index 062156b88a87..eee7402cfc02 100644
> > --- a/drivers/hv/hv_balloon.c
> > +++ b/drivers/hv/hv_balloon.c
> > @@ -1660,6 +1660,38 @@ static void disable_page_reporting(void)
> >  	}
> >  }
> > 
> > +static int ballooning_enabled(void)
> > +{
> > +	/*
> > +	 * Disable ballooning if the page size is not 4k (HV_HYP_PAGE_SIZE),
> > +	 * since currently it's unclear to us whether an unballoon request can
> > +	 * make sure all page ranges are guest page size aligned.
> 
> My interpretation of the conversations with Hyper-V is that that they clearly
> don't guarantee page ranges are guest page aligned.
> 
> > +	 */
> > +	if (PAGE_SIZE != HV_HYP_PAGE_SIZE) {
> > +		pr_info("Ballooning disabled because page size is not 4096 bytes\n");
> > +		return 0;
> > +	}
> > +
> > +	return 1;
> > +}
> > +
> > +static int hot_add_enabled(void)
> > +{
> > +	/*
> > +	 * Disable hot add on ARM64, because we currently rely on
> > +	 * memory_add_physaddr_to_nid() to get a node id of a hot add range,
> > +	 * however ARM64's memory_add_physaddr_to_nid() always return 0 and
> > +	 * DM_MEM_HOT_ADD_REQUEST doesn't have the NUMA node information for
> > +	 * add_memory().
> > +	 */
> > +	if (IS_ENABLED(CONFIG_ARM64)) {
> > +		pr_info("Memory hot add disabled on ARM64\n");
> > +		return 0;
> > +	}
> > +
> > +	return 1;
> > +}
> > +
> >  static int balloon_connect_vsp(struct hv_device *dev)
> >  {
> >  	struct dm_version_request version_req;
> > @@ -1731,8 +1763,8 @@ static int balloon_connect_vsp(struct hv_device *dev)
> >  	 * currently still requires the bits to be set, so we have to add code
> >  	 * to fail the host's hot-add and balloon up/down requests, if any.
> >  	 */
> > -	cap_msg.caps.cap_bits.balloon = 1;
> > -	cap_msg.caps.cap_bits.hot_add = 1;
> > +	cap_msg.caps.cap_bits.balloon = ballooning_enabled();
> > +	cap_msg.caps.cap_bits.hot_add = hot_add_enabled();
> > 
> >  	/*
> >  	 * Specify our alignment requirements as it relates
> > --
> > 2.33.0
> 
> The code looks good to me.
> 
> Michael