ui/sdl2: add grab-on-tablet option for absolute input devices

[PATCH qemu] ui/sdl2: add grab-on-tablet option for absolute input devices

Posted by ~rongyichang 2 days, 11 hours ago

From: rongyichang <rongyichang@xiaomi.com>

When using absolute coordinate input devices (e.g. virtio-tablet-device),
the SDL display backend implements a "soft grab" where the mouse is
automatically grabbed when it enters the window interior and ungrabbed
when it hits the window edge. This edge-ungrab behavior causes problems
in embedded emulation scenarios:

1. Mouse escapes the SDL window at edges
2. SDL does not deliver BUTTONUP events for the escaped mouse
3. The guest gets stuck in a pressed/touch-down state
4. First click back into the window is silently dropped

This issue is confirmed as a known SDL limitation (SDL issue #5301).

Add a new -display sdl,grab-on-tablet=on option that makes absolute
coordinate devices use the same grab behavior as relative (mouse)
devices: user must click to grab, Ctrl+Alt+G to release, and no
automatic grab on window enter or edge-based ungrab/regrab.

Signed-off-by: Yichang Rong <rongyichang@xiaomi.com>
---
 qapi/ui.json    |  8 +++++++-
 qemu-options.hx |  9 +++++++--
 ui/sdl2.c       | 16 +++++++++++++---
 3 files changed, 27 insertions(+), 6 deletions(-)

diff --git a/qapi/ui.json b/qapi/ui.json
index b2c42a7f57..454d9041bf 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -1477,10 +1477,16 @@
 # @grab-mod: Modifier keys that should be pressed together with the
 #     "G" key to release the mouse grab.
 #
+# @grab-on-tablet: When enabled, the mouse grab is required even for
+#     tablet (absolute) input devices.  This is useful when the guest
+#     OS uses a tablet device but you still want click-to-grab
+#     semantics (e.g. NuttX touchscreen emulation).
+#
 # Since: 7.1
 ##
 { 'struct'  : 'DisplaySDL',
-  'data'    : { '*grab-mod'   : 'HotKeyMod' } }
+  'data'    : { '*grab-mod'   : 'HotKeyMod',
+                '*grab-on-tablet' : 'bool' } }
 
 ##
 # @DisplayType:
diff --git a/qemu-options.hx b/qemu-options.hx
index 96ae41f787..1e1836312c 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -2205,8 +2205,8 @@ DEF("display", HAS_ARG, QEMU_OPTION_display,
     "-display spice-app[,gl=on|off]\n"
 #endif
 #if defined(CONFIG_SDL)
-    "-display sdl[,gl=on|core|es|off][,grab-mod=<mod>][,show-cursor=on|off]\n"
-    "            [,window-close=on|off]\n"
+    "-display sdl[,gl=on|core|es|off][,grab-mod=<mod>][,grab-on-tablet=on|off]\n"
+    "            [,show-cursor=on|off][,window-close=on|off]\n"
 #endif
 #if defined(CONFIG_GTK)
     "-display gtk[,clipboard=on|off][,full-screen=on|off][,gl=on|off]\n"
@@ -2284,6 +2284,11 @@ SRST
         the mouse grabbing in conjunction with the "g" key. ``<mods>`` can be
         either ``lshift-lctrl-lalt`` or ``rctrl``.
 
+        ``grab-on-tablet=on|off`` : When enabled, the mouse grab is required
+        even for tablet (absolute) input devices.  This is useful when the
+        guest OS uses a tablet device but you still want click-to-grab
+        semantics.
+
         ``gl=on|off|core|es`` : Use OpenGL for displaying
 
         ``show-cursor=on|off`` :  Force showing the mouse cursor
diff --git a/ui/sdl2.c b/ui/sdl2.c
index 4fcdbd79d3..af94ae8500 100644
--- a/ui/sdl2.c
+++ b/ui/sdl2.c
@@ -46,6 +46,7 @@ static SDL_Surface *guest_sprite_surface;
 static int gui_grab; /* if true, all keyboard/mouse events are grabbed */
 static bool alt_grab;
 static bool ctrl_grab;
+static bool grab_on_tablet;
 
 static int gui_saved_grab;
 static int gui_fullscreen;
@@ -504,7 +505,8 @@ static void handle_mousemotion(SDL_Event *ev)
     }
 
     SDL_GetWindowSize(scon->real_window, &scr_w, &scr_h);
-    if (qemu_input_is_absolute(scon->dcl.con) || absolute_enabled) {
+    if ((qemu_input_is_absolute(scon->dcl.con) || absolute_enabled) &&
+        !grab_on_tablet) {
         max_x = scr_w - 1;
         max_y = scr_h - 1;
         if (gui_grab && !gui_fullscreen
@@ -545,7 +547,8 @@ static void handle_mousebutton(SDL_Event *ev)
     x = (int64_t)bev->x * surface_width(scon->surface) / scr_w;
     y = (int64_t)bev->y * surface_height(scon->surface) / scr_h;
 
-    if (!gui_grab && !qemu_input_is_absolute(scon->dcl.con)) {
+    if (!gui_grab &&
+        (!qemu_input_is_absolute(scon->dcl.con) || grab_on_tablet)) {
         if (ev->type == SDL_MOUSEBUTTONUP && bev->button == SDL_BUTTON_LEFT) {
             /* start grabbing all events */
             sdl_grab_start(scon);
@@ -614,7 +617,10 @@ static void handle_windowevent(SDL_Event *ev)
     case SDL_WINDOWEVENT_FOCUS_GAINED:
         /* fall through */
     case SDL_WINDOWEVENT_ENTER:
-        if (!gui_grab && (qemu_input_is_absolute(scon->dcl.con) || absolute_enabled)) {
+        if (!gui_grab &&
+            (qemu_input_is_absolute(scon->dcl.con) ||
+             absolute_enabled) &&
+            !grab_on_tablet) {
             absolute_mouse_grab(scon);
         }
         /* If a new console window opened using a hotkey receives the
@@ -921,6 +927,10 @@ static void sdl2_display_init(DisplayState *ds, DisplayOptions *o)
         }
     }
 
+    if (o->u.sdl.has_grab_on_tablet) {
+        grab_on_tablet = o->u.sdl.grab_on_tablet;
+    }
+
     for (i = 0;; i++) {
         QemuConsole *con = qemu_console_lookup_by_index(i);
         if (!con) {
-- 
2.49.1

Re: [PATCH qemu] ui/sdl2: add grab-on-tablet option for absolute input devices

Posted by BALATON Zoltan 2 days, 4 hours ago

On Fri, 5 Jun 2026, ~rongyichang wrote:
> From: rongyichang <rongyichang@xiaomi.com>
>
> When using absolute coordinate input devices (e.g. virtio-tablet-device),
> the SDL display backend implements a "soft grab" where the mouse is
> automatically grabbed when it enters the window interior and ungrabbed
> when it hits the window edge. This edge-ungrab behavior causes problems
> in embedded emulation scenarios:
>
> 1. Mouse escapes the SDL window at edges
> 2. SDL does not deliver BUTTONUP events for the escaped mouse
> 3. The guest gets stuck in a pressed/touch-down state
> 4. First click back into the window is silently dropped
>
> This issue is confirmed as a known SDL limitation (SDL issue #5301).
>
> Add a new -display sdl,grab-on-tablet=on option that makes absolute
> coordinate devices use the same grab behavior as relative (mouse)
> devices: user must click to grab, Ctrl+Alt+G to release, and no
> automatic grab on window enter or edge-based ungrab/regrab.

Maybe it's better to name the option click-to-grab or grab-on-click or 
something similar as that's what it changes and not specific to tablets.

Regards.
BALATON Zoltan

答复: [External Mail]Re: [PATCH qemu] ui/sdl2: add grab-on-tablet option for absolute input devices

Posted by 荣义昌 via qemu development 1 day, 18 hours ago

Hi,

Thanks for the review.

If we use a generic name like "click-to-grab" (not tablet-specific),
should we also consider the reverse direction ― i.e., allowing relative
(mouse) devices to use the tablet-style edge-ungrab/regrab behavior
via something like "click-to-grab=off"?

Currently relative devices always use click-to-grab, and absolute
devices always use edge-based auto-grab. If we make this a generic
toggle, the semantics would be:

  -display sdl,click-to-grab=on   → both device types use click-to-grab
  -display sdl,click-to-grab=off  → both device types use edge-ungrab

However, edge-ungrab doesn't really work for relative (mouse) devices,
because once the mouse escapes the window, SDL stops delivering motion
events and the guest cursor becomes uncontrollable.

So in practice, this option can only meaningfully change behavior for
absolute devices. That's why I chose a tablet-specific name ― to make
it clear that it only affects absolute coordinate devices.

That said, I'm open to "click-to-grab=on|off" if you prefer a more
generic name. The implementation would remain the same (only affecting
absolute devices), just with a more general name.

What do you think?

Best regards,
Yichang

________________________________
发件人: BALATON Zoltan <balaton@eik.bme.hu>
发送时间: 2026年6月6日 0:47:00
收件人: 荣义昌
抄送: qemu-devel@nongnu.org; marcandre.lureau@redhat.com; armbru@redhat.com; eblake@redhat.com
主题: [External Mail]Re: [PATCH qemu] ui/sdl2: add grab-on-tablet option for absolute input devices

[外部邮件] 此邮件来源于小米公司外部，请谨慎处理。若对邮件安全性存疑，请将邮件转发给misec@xiaomi.com进行反馈

On Fri, 5 Jun 2026, ~rongyichang wrote:
> From: rongyichang <rongyichang@xiaomi.com>
>
> When using absolute coordinate input devices (e.g. virtio-tablet-device),
> the SDL display backend implements a "soft grab" where the mouse is
> automatically grabbed when it enters the window interior and ungrabbed
> when it hits the window edge. This edge-ungrab behavior causes problems
> in embedded emulation scenarios:
>
> 1. Mouse escapes the SDL window at edges
> 2. SDL does not deliver BUTTONUP events for the escaped mouse
> 3. The guest gets stuck in a pressed/touch-down state
> 4. First click back into the window is silently dropped
>
> This issue is confirmed as a known SDL limitation (SDL issue #5301).
>
> Add a new -display sdl,grab-on-tablet=on option that makes absolute
> coordinate devices use the same grab behavior as relative (mouse)
> devices: user must click to grab, Ctrl+Alt+G to release, and no
> automatic grab on window enter or edge-based ungrab/regrab.

Maybe it's better to name the option click-to-grab or grab-on-click or
something similar as that's what it changes and not specific to tablets.

Regards.
BALATON Zoltan
#/******本邮件及其附件含有小米公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!******/#