drm/vmwgfx: Fix hrtimer interrupt storm due to 0-period vblank

[PATCH] drm/vmwgfx: Fix hrtimer interrupt storm due to 0-period vblank

Posted by w15303746062@163.com 6 days, 21 hours ago

From: Mingyu Wang <25181214217@stu.xidian.edu.cn>

When vmwgfx is configured to use VKMS for vblank simulation, it relies
on drm_calc_timestamping_constants() to calculate the frame duration
(vblank->framedur_ns).

However, Fuzzers (like Syzkaller) can submit extremely malicious
display modes through DRM_IOCTL_MODE_SETCRTC. If the user-space passes
a mode with a massive pixel clock (crtc_clock) and small resolution
(htotal/vtotal), the integer division in drm_calc_timestamping_constants()
truncates the result to 0.

Consequently, vmw_vkms_enable_vblank() blindly sets the hrtimer period
to 0. When the timer is started, it fires instantly and continuously.
Because hrtimer_forward_now() cannot advance time for a 0-period,
the overrun value skyrockets, locking the CPU in an infinite hard-IRQ
loop (vkms_vblank_simulate() -> HRTIMER_RESTART).

This completely starves the CPU, leading to massive RCU stalls and
blocking other essential tasks (like jbd2 and writeback workers)
indefinitely:

  [ C1] vkms_vblank_simulate: vblank timer overrun
  ...
  INFO: task kworker/u18:2:50 blocked for more than 143 seconds.
  Workqueue: writeback wb_workfn (flush-8:0)
  Call Trace:
   <TASK>
   __schedule+0x1044/0x5bb0
   wbt_wait+0x1c8/0x3b0
   blk_mq_submit_bio+0x29fa/0x31f0
   submit_bio_noacct+0xca7/0x1f90
   ext4_bio_write_folio+0x95a/0x1d10
   ...

  NMI backtrace for cpu 1
  Call Trace:
   <IRQ>
   vkms_vblank_simulate+0x8f/0x390
   __hrtimer_run_queues+0x1f5/0xb30
   hrtimer_interrupt+0x39a/0x880

Fix this DoS vulnerability by adding a defensive sanity check in
vmw_vkms_enable_vblank() to reject a 0-ns frame duration, allowing
DRM core to gracefully fallback/reject the mode without crashing.

Fixes: cd2eb57df1b8 ("drm/vmwgfx: Implement virtual kms")
Cc: stable@vger.kernel.org
Signed-off-by: Mingyu Wang <25181214217@stu.xidian.edu.cn>
---
 drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c b/drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c
index 5abd7f5ad2db..b3950ae424f3 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c
@@ -288,6 +288,16 @@ vmw_vkms_enable_vblank(struct drm_crtc *crtc)
 
 	drm_calc_timestamping_constants(crtc, &crtc->mode);
 
+	/*
+	 * DEFENSIVE CHECK:
+	 * drm_calc_timestamping_constants() can calculate a framedur_ns
+	 * of 0 if user-space provides a malicious mode with a huge
+	 * crtc_clock and small htotal/vtotal due to integer division
+	 * truncation. Prevent hrtimer interrupt storms by refusing such modes.
+	 */
+	if (WARN_ON_ONCE(vblank->framedur_ns == 0))
+		return -EINVAL;
+
 	hrtimer_setup(&du->vkms.timer, &vmw_vkms_vblank_simulate, CLOCK_MONOTONIC,
 		      HRTIMER_MODE_REL);
 	du->vkms.period_ns = ktime_set(0, vblank->framedur_ns);
-- 
2.34.1

Re: [PATCH] drm/vmwgfx: Fix hrtimer interrupt storm due to 0-period vblank

Posted by Thomas Zimmermann 2 days, 20 hours ago

Hi

Am 18.05.26 um 09:17 schrieb w15303746062@163.com:
> From: Mingyu Wang <25181214217@stu.xidian.edu.cn>
>
> When vmwgfx is configured to use VKMS for vblank simulation, it relies
> on drm_calc_timestamping_constants() to calculate the frame duration
> (vblank->framedur_ns).
>
> However, Fuzzers (like Syzkaller) can submit extremely malicious
> display modes through DRM_IOCTL_MODE_SETCRTC. If the user-space passes
> a mode with a massive pixel clock (crtc_clock) and small resolution
> (htotal/vtotal), the integer division in drm_calc_timestamping_constants()
> truncates the result to 0.
>
> Consequently, vmw_vkms_enable_vblank() blindly sets the hrtimer period
> to 0. When the timer is started, it fires instantly and continuously.
> Because hrtimer_forward_now() cannot advance time for a 0-period,
> the overrun value skyrockets, locking the CPU in an infinite hard-IRQ
> loop (vkms_vblank_simulate() -> HRTIMER_RESTART).
>
> This completely starves the CPU, leading to massive RCU stalls and
> blocking other essential tasks (like jbd2 and writeback workers)
> indefinitely:
>
>    [ C1] vkms_vblank_simulate: vblank timer overrun
>    ...
>    INFO: task kworker/u18:2:50 blocked for more than 143 seconds.
>    Workqueue: writeback wb_workfn (flush-8:0)
>    Call Trace:
>     <TASK>
>     __schedule+0x1044/0x5bb0
>     wbt_wait+0x1c8/0x3b0
>     blk_mq_submit_bio+0x29fa/0x31f0
>     submit_bio_noacct+0xca7/0x1f90
>     ext4_bio_write_folio+0x95a/0x1d10
>     ...
>
>    NMI backtrace for cpu 1
>    Call Trace:
>     <IRQ>
>     vkms_vblank_simulate+0x8f/0x390
>     __hrtimer_run_queues+0x1f5/0xb30
>     hrtimer_interrupt+0x39a/0x880
>
> Fix this DoS vulnerability by adding a defensive sanity check in
> vmw_vkms_enable_vblank() to reject a 0-ns frame duration, allowing
> DRM core to gracefully fallback/reject the mode without crashing.
>
> Fixes: cd2eb57df1b8 ("drm/vmwgfx: Implement virtual kms")
> Cc: stable@vger.kernel.org
> Signed-off-by: Mingyu Wang <25181214217@stu.xidian.edu.cn>
> ---
>   drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c | 10 ++++++++++
>   1 file changed, 10 insertions(+)
>
> diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c b/drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c
> index 5abd7f5ad2db..b3950ae424f3 100644
> --- a/drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c
> +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c
> @@ -288,6 +288,16 @@ vmw_vkms_enable_vblank(struct drm_crtc *crtc)
>   
>   	drm_calc_timestamping_constants(crtc, &crtc->mode);
>   
> +	/*
> +	 * DEFENSIVE CHECK:
> +	 * drm_calc_timestamping_constants() can calculate a framedur_ns
> +	 * of 0 if user-space provides a malicious mode with a huge
> +	 * crtc_clock and small htotal/vtotal due to integer division
> +	 * truncation. Prevent hrtimer interrupt storms by refusing such modes.
> +	 */
> +	if (WARN_ON_ONCE(vblank->framedur_ns == 0))
> +		return -EINVAL;

This code does no longer exist in the development tree (i.e., drm-misc). 
Although the new implementation might have a similar issue.

Best regards
Thomas

> +
>   	hrtimer_setup(&du->vkms.timer, &vmw_vkms_vblank_simulate, CLOCK_MONOTONIC,
>   		      HRTIMER_MODE_REL);
>   	du->vkms.period_ns = ktime_set(0, vblank->framedur_ns);

-- 
--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com
GF: Jochen Jaser, Andrew McDonald, Werner Knoblich, (HRB 36809, AG Nürnberg)

[PATCH v2] drm/vblank: Reject 0-period timers to prevent hrtimer storm

Posted by w15303746062@163.com 2 days, 2 hours ago

From: Mingyu Wang <25181214217@stu.xidian.edu.cn>

Fuzzers like Syzkaller can submit extremely malicious display modes
through DRM_IOCTL_MODE_SETCRTC. If userspace passes a mode with a
massive pixel clock (crtc_clock) and small resolution (htotal/vtotal),
the integer division in drm_calc_timestamping_constants() truncates
the resulting frame duration (vblank->framedur_ns) to 0.

When virtual display drivers (such as vmwgfx or vkms) rely on the DRM
core's software vblank simulation, drm_crtc_vblank_start_timer() is
called. It blindly converts this 0-ns framedur_ns into a ktime interval
and starts the hrtimer. An hrtimer with a 0-period fires instantly and
continuously. Since hrtimer_forward_now() cannot advance time for a
0-period, the CPU gets locked in an infinite hard-IRQ loop, starving
the system and causing massive RCU stalls.

Fix this DoS vulnerability by adding a defensive sanity check in
drm_crtc_vblank_start_timer() to reject a 0-ns frame duration, allowing
the DRM core to gracefully reject the malicious mode.

Signed-off-by: Mingyu Wang <25181214217@stu.xidian.edu.cn>
---
Changes in v2:
- Moved the defensive check from vmwgfx to drm_vblank.c. The timer
  logic was refactored into the DRM core, so placing the check here
  protects all drivers relying on the core software vblank timer.
- Dropped WARN_ON_ONCE() to prevent unprivileged userspace from easily
  triggering kernel panics on systems with panic_on_warn enabled.

 drivers/gpu/drm/drm_vblank.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/gpu/drm/drm_vblank.c b/drivers/gpu/drm/drm_vblank.c
index f90fb2d13e42..b38d0b30a651 100644
--- a/drivers/gpu/drm/drm_vblank.c
+++ b/drivers/gpu/drm/drm_vblank.c
@@ -2241,6 +2241,16 @@ int drm_crtc_vblank_start_timer(struct drm_crtc *crtc)
 
 	drm_calc_timestamping_constants(crtc, &crtc->mode);
 
+	/*
+	 * DEFENSIVE CHECK:
+	 * drm_calc_timestamping_constants() truncates framedur_ns to 0 if
+	 * userspace provides a malicious mode with a huge crtc_clock and
+	 * small htotal/vtotal. Prevent an infinite hard-IRQ loop from a
+	 * 0-period hrtimer by rejecting such modes.
+	 */
+	if (unlikely(vblank->framedur_ns == 0))
+		return -EINVAL;
+
 	spin_lock_irqsave(&vtimer->interval_lock, flags);
 	vtimer->interval = ns_to_ktime(vblank->framedur_ns);
 	spin_unlock_irqrestore(&vtimer->interval_lock, flags);
-- 
2.34.1