From nobody Mon Nov 25 05:42:20 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1717878091; cv=none; d=zohomail.com; s=zohoarc; b=LWa4Vcdm7rLw9Ts1BrZyxYy65OGiWKe7TbWbesAJoZ2Dt14pVhxCsmETHcwXsFB4CZHzFPxynvIM4EC04pZrlYVlwnzK8+v9kMyIqqZGm2yCU0l+g5nAgzImel0xUkgTnDx4/pp+/8dqShF1NrELLXUasp9JO+1qIADNzzsEfhw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1717878091; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Dxl23C1D5XzzTzk0m+i0wmEqonXMUbdqpGsXRiYEAFg=; b=IEoaHQpHntBREScOyZFrV4BUICl/1AEGbv9++a07BCyT9JVcjCEI50B2WyBxqXIeY8NXc42KcSzhLT03CL1izNeFLT5F2XePWzZEWMFANL138gAVdAyR/zv41YNgNB+QzF5T4eJ3g8+EROFDJV2NaiIWHAUDf+6UgRd7Em37nsI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1717878091124870.3959334662019; Sat, 8 Jun 2024 13:21:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sG2Z3-0000sJ-Vf; Sat, 08 Jun 2024 16:21:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sG2Z2-0000qh-KH for qemu-devel@nongnu.org; Sat, 08 Jun 2024 16:21:24 -0400 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sG2Z0-0006b7-FB for qemu-devel@nongnu.org; Sat, 08 Jun 2024 16:21:24 -0400 Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-421798185f0so8560435e9.1 for ; Sat, 08 Jun 2024 13:21:22 -0700 (PDT) Received: from localhost.localdomain (89-104-8-17.customer.bnet.at. [89.104.8.17]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-35f0ccc5f03sm3803845f8f.88.2024.06.08.13.21.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Jun 2024 13:21:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=philjordan-eu.20230601.gappssmtp.com; s=20230601; t=1717878080; x=1718482880; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Dxl23C1D5XzzTzk0m+i0wmEqonXMUbdqpGsXRiYEAFg=; b=YHCn/zd1nhjc6ypUPqdcfVbCjrmFvS5F4HiYzsy52V3RgbT0jmqZnQVZ5nmWTOJ53o FJOAmWaL9N7+ZVaHdpffGoQybqwm9hzkwiIVj/OMqtnn5TFyvT3M7c5mw1V9Tge73PTa AYyG1WqoNnVeoPgzGa0+8peSLKeXBBSpFn58d0OMonfW4j+k7m2dSAd6FAUU1UeLmzNt +y7GlVRf4jdJpvMl841gTGAbtogkEZz+B5gkmwkmoq1svpd4F+7i+rBOyTVY7iL5oOEH znSGkRKfpGuX7kvsmW+Q5UoNIQT4M2JLqzNfB6R0/LMdvP0uhYhaovug/4dC7O0tV/r0 zRrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717878080; x=1718482880; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Dxl23C1D5XzzTzk0m+i0wmEqonXMUbdqpGsXRiYEAFg=; b=ZM9Dg5MpSKzWG7MoLc9yU8N/hRtojmfYZ3nmmn4Vyz4k8ljlwekclnwEfx6nbEJDN4 sPqkujwd465k+C0rji/A2VevkXxOInz2Vh2KuVBCGFOXXgtqIo4Xk1eKblFDd8oDcGaM fc7dNE3FbG13FV9LDFc18Yz9LD3NpOk63Br7mWww3u/P8C+CI/rAjqT2ic3alaO9xboj AViyFoJE11GTMuQ5L+62YBytsyVzUW6quGzpFQAlkoc5p01Z4VAjykPkCkKxfqiW8qkH C52NocPleDSno8dCFYwEGE/G+alanVjbjTcOHGd27uFvzD+rdeNpZp7DpxJb7HlUy+Fl PY9w== X-Gm-Message-State: AOJu0YyQzSHOkI8UllcRXmbFJ4vjg+nf+a6fDXhuT9202yrTtVTulHOB n4z0jUNAzshbk/Pm9zhiNBm7UaLKPmPippwPWhuRVdoueFHYvwJN2FnySSQeBWVsOpJNB9qslAg = X-Google-Smtp-Source: AGHT+IHoNjyzlUraKjTkqG0iB21jvsxx/ijmNIf4UkvN28ua/dQ+XNXoqG5lfy29LG/wKzYmmCEFAw== X-Received: by 2002:adf:fc85:0:b0:35b:5a14:984a with SMTP id ffacd0b85a97d-35efee03d4fmr3691525f8f.56.1717878080206; Sat, 08 Jun 2024 13:21:20 -0700 (PDT) From: Phil Dennis-Jordan To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, philmd@linaro.org, akihiko.odaki@daynix.com, marcandre.lureau@redhat.com, Phil Dennis-Jordan Subject: [PATCH 3/3] ui/cocoa: Adds support for mouse cursors Date: Sat, 8 Jun 2024 22:20:45 +0200 Message-Id: <20240608202045.2815-4-phil@philjordan.eu> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20240608202045.2815-1-phil@philjordan.eu> References: <20240608202045.2815-1-phil@philjordan.eu> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: neutral client-ip=2a00:1450:4864:20::330; envelope-from=phil@philjordan.eu; helo=mail-wm1-x330.google.com X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NEUTRAL=0.779, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @philjordan-eu.20230601.gappssmtp.com) X-ZM-MESSAGEID: 1717878092422100003 Content-Type: text/plain; charset="utf-8" This change implements the callbacks dpy_cursor_define and dpy_mouse_set for the Cocoa UI. The incoming mouse cursor image is converted into an NSCursor object, allowing the guest mouse cursor to be rendered as the host's native OS cursor on macOS. This is straightforward in absolute pointing mode, but rather trickier with a relative pointing device: 1. The cursor position in Qemu's coordinate system must be translated and converted into macOS's Core Graphics/Quartz coordinates when positioning the cursor. Additionally, the position already includes the hotspot offset; we'd prefer to use the host OS's hotspot support so we need subtract the hotspot vector off again. 2. Setting the cursor position programmatically on macOS biases the next mouse movement event by the amount the cursor was shifted. If we didn't reverse that bias when forwarding the next event back into Qemu's input stack, this would create a feedback loop. (The behaviour of affecting mouse events makes sense for e.g. setting the cursor position in a remote access system.) This change slightly improves the user experience when using virtual display adapter implementations which check for UI back-end cursor support, and fixes the issue of no visible mouse cursor when using one which does not. (Such as virtio-vga) Signed-off-by: Phil Dennis-Jordan --- ui/cocoa.m | 167 +++++++++++++++++++++++++++++++++++++++++++++++- ui/trace-events | 7 ++ 2 files changed, 171 insertions(+), 3 deletions(-) diff --git a/ui/cocoa.m b/ui/cocoa.m index 981615a8b9..0c71533835 100644 --- a/ui/cocoa.m +++ b/ui/cocoa.m @@ -49,6 +49,7 @@ #include "qemu/error-report.h" #include #include "hw/core/cpu.h" +#include "trace.h" =20 #ifndef MAC_OS_VERSION_14_0 #define MAC_OS_VERSION_14_0 140000 @@ -80,11 +81,17 @@ static void cocoa_switch(DisplayChangeListener *dcl, =20 static void cocoa_refresh(DisplayChangeListener *dcl); =20 +static void cocoa_cursor_define(DisplayChangeListener *dcl, QEMUCursor *cu= rsor); + +static void cocoa_mouse_set(DisplayChangeListener *dcl, int x, int y, int = on); + static const DisplayChangeListenerOps dcl_ops =3D { .dpy_name =3D "cocoa", .dpy_gfx_update =3D cocoa_update, .dpy_gfx_switch =3D cocoa_switch, .dpy_refresh =3D cocoa_refresh, + .dpy_cursor_define =3D cocoa_cursor_define, + .dpy_mouse_set =3D cocoa_mouse_set, }; static DisplayChangeListener dcl =3D { .ops =3D &dcl_ops, @@ -299,6 +306,11 @@ @interface QemuCocoaView : NSView BOOL isMouseGrabbed; BOOL isAbsoluteEnabled; CFMachPortRef eventsTap; + NSCursor *current_cursor; + int cursor_hot_x; + int cursor_hot_y; + int offset_delta_x; + int offset_delta_y; } - (void) switchSurface:(pixman_image_t *)image; - (void) grabMouse; @@ -320,6 +332,9 @@ - (BOOL) isMouseGrabbed; - (BOOL) isAbsoluteEnabled; - (QEMUScreen) gscreen; - (void) raiseAllKeys; +- (void) setCursor:(NSCursor*)newCursor hotspotX:(int)hotX y:(int)hotY; +- (void) setMouseX:(int)x y:(int)y showCursor:(BOOL)showCursor; + @end =20 QemuCocoaView *cocoaView; @@ -376,6 +391,9 @@ - (void) dealloc pixman_image_unref(pixman_image); } =20 + [self->current_cursor release]; + self->current_cursor =3D nil; + if (eventsTap) { CFRelease(eventsTap); } @@ -407,6 +425,68 @@ - (void) selectConsoleLocked:(unsigned int)index [self updateUIInfo]; } =20 +- (void) setCursor:(NSCursor*)newCursor hotspotX:(int)hotX y:(int)hotY +{ + [self->current_cursor release]; + [newCursor retain]; + self->current_cursor =3D newCursor; + + cocoaView->cursor_hot_x =3D hotX; + cocoaView->cursor_hot_y =3D hotY; + + [self.window invalidateCursorRectsForView:self]; +} + +- (void) resetCursorRects +{ + if (self->current_cursor =3D=3D nil) { + [super resetCursorRects]; + } else { + [self addCursorRect:NSMakeRect(0.0, 0.0, self->screen.width, self-= >screen.height) cursor:self->current_cursor]; + } +} + +- (void) setMouseX:(int)x y:(int)y showCursor:(BOOL)showCursor +{ + if (isAbsoluteEnabled) { + offset_delta_x =3D 0; + offset_delta_y =3D 0; + return; + } else if (!isMouseGrabbed) { + return; + } + + NSWindow* window =3D [self window]; + + /* Coordinates seem to come in already offset by hotspot; undo that. A= lso + * avoid out-of-window coordinates. */ + x +=3D cursor_hot_x; + y +=3D cursor_hot_y; + x =3D int_clamp(x, 0, screen.width); + y =3D int_clamp(y, 0, screen.height); + /* Flip coordinates so origin is bottom left (Cocoa), not top left (Qe= mu), + * before translating into window and then desktop coordinate systems.= */ + y =3D screen.height - y; + + NSPoint new_pos_window =3D [self convertPoint:NSMakePoint(x, y) toView= :nil]; + NSPoint prev_pos_window =3D window.mouseLocationOutsideOfEventStream; + + CGPoint screen_pos =3D [window convertPointToScreen:new_pos_window]; + + /* Translate from Cocoa (origin: main screen bottom left, +y up) + * to Quartz (origin: top left, +y down) coordinate system. */ + screen_pos.y =3D NSScreen.mainScreen.frame.size.height - screen_pos.y; + + /* Warp moves the host cursor to the new position. This doesn't genera= te a + * spurious move event, but it does offset the delta for next genuine = event, + * which we need to take into account when that event comes in. */ + CGWarpMouseCursorPosition(screen_pos); + + offset_delta_x +=3D (prev_pos_window.x - new_pos_window.x); + /* -ve due to Cocoa -> Qemu/Quartz Y axis conversion: */ + offset_delta_y -=3D (prev_pos_window.y - new_pos_window.y); +} + - (void) hideCursor { if (!cursor_hide) { @@ -1005,9 +1085,21 @@ - (void) handleMouseEvent:(NSEvent *)event /* Note that the origin for Cocoa mouse coords is bottom left,= not top left. */ qemu_input_queue_abs(dcl.con, INPUT_AXIS_X, p.x * d, 0, screen= .width); qemu_input_queue_abs(dcl.con, INPUT_AXIS_Y, screen.height - p.= y * d, 0, screen.height); + trace_cocoa_handle_mouse_event_absolute(p.x, p.y, screen.width= , screen.height, p.x * d, screen.height - p.y * d); } else { - qemu_input_queue_rel(dcl.con, INPUT_AXIS_X, [event deltaX]); - qemu_input_queue_rel(dcl.con, INPUT_AXIS_Y, [event deltaY]); + /* Programmatically moving the Cocoa mouse cursor also offsets= the + * next mouse event, so we offset by the amount we moved the c= ursor + * to avoid a feedback loop. */ + int delta_x =3D [event deltaX] + offset_delta_x; + int delta_y =3D [event deltaY] + offset_delta_y; + trace_cocoa_handle_mouse_event_relative( + [event deltaX], [event deltaY], + offset_delta_x, offset_delta_y, + delta_x, delta_y); + qemu_input_queue_rel(dcl.con, INPUT_AXIS_X, /*[event deltaX]/*= /delta_x/**/); + qemu_input_queue_rel(dcl.con, INPUT_AXIS_Y, /*[event deltaY]/*= /delta_y/**/); + offset_delta_x =3D 0; + offset_delta_y =3D 0; } =20 qemu_input_event_sync(); @@ -1084,19 +1176,26 @@ - (void) otherMouseUp:(NSEvent *)event =20 - (void) grabMouse { + trace_cocoa_grab_mouse(); COCOA_DEBUG("QemuCocoaView: grabMouse\n"); =20 if (qemu_name) [[self window] setTitle:[NSString stringWithFormat:@"QEMU %s - (Pr= ess " UC_CTRL_KEY " " UC_ALT_KEY " G to release Mouse)", qemu_name]]; else [[self window] setTitle:@"QEMU - (Press " UC_CTRL_KEY " " UC_ALT_= KEY " G to release Mouse)"]; - [self hideCursor]; + + if (current_cursor =3D=3D nil) { + [self hideCursor]; + } CGAssociateMouseAndMouseCursorPosition(isAbsoluteEnabled); isMouseGrabbed =3D TRUE; // while isMouseGrabbed =3D TRUE, QemuCocoaAp= p sends all events to [cocoaView handleEvent:] + offset_delta_x =3D 0; + offset_delta_y =3D 0; } =20 - (void) ungrabMouse { + trace_cocoa_ungrab_mouse(); COCOA_DEBUG("QemuCocoaView: ungrabMouse\n"); =20 if (qemu_name) @@ -1104,6 +1203,11 @@ - (void) ungrabMouse else [[self window] setTitle:@"QEMU"]; [self unhideCursor]; + + if (current_cursor =3D=3D nil) { + [self unhideCursor]; + } + CGAssociateMouseAndMouseCursorPosition(TRUE); isMouseGrabbed =3D FALSE; [self raiseAllButtons]; @@ -2000,6 +2104,63 @@ static void cocoa_refresh(DisplayChangeListener *dcl) [pool release]; } =20 +static NSImage *cocoa_create_image_argb32(size_t width, size_t height, con= st void *pixel_data) +{ + size_t buffer_size =3D width * height * 4lu; + CGDataProviderRef provider =3D CGDataProviderCreateWithData( + NULL, pixel_data, buffer_size, NULL); + size_t bpc =3D 8; + size_t bpp =3D 32; + size_t bytes_per_row =3D 4u * width; + CGColorSpaceRef color_space =3D CGColorSpaceCreateDeviceRGB(); + CGBitmapInfo bitmap_info =3D + kCGBitmapByteOrder32Little | kCGImageAlphaFirst; + CGColorRenderingIntent intent =3D kCGRenderingIntentDefault; + + CGImageRef cg_image =3D CGImageCreate( + width, + height, + bpc, + bpp, + bytes_per_row, + color_space, + bitmap_info, + provider, + NULL, // decode + YES, // should interpolate + intent); + + NSImage *image =3D [[NSImage alloc] initWithCGImage:cg_image size:NSMa= keSize(width, height)]; + CGImageRelease(cg_image); + CGColorSpaceRelease(color_space); + CGDataProviderRelease(provider); + return image; +} + +static void cocoa_cursor_define(DisplayChangeListener *dcl, QEMUCursor *cu= rsor) +{ + NSImage *cursor_image =3D nil; + NSPoint hotspot =3D { cursor->hot_x, cursor->hot_y }; + trace_cocoa_cursor_define(cursor->hot_x, cursor->hot_y, + cursor->width, cursor->height); + if (cursor =3D=3D NULL || cursor->width <=3D 0 || cursor->height <=3D = 0) { + cursor_image =3D [[NSImage alloc] initWithSize:NSMakeSize(1.0, 1.0= )]; + } else { + cursor_image =3D cocoa_create_image_argb32(cursor->width, cursor->= height, + cursor->data); + } + NSCursor *cocoa_cursor =3D + [[NSCursor alloc] initWithImage:cursor_image hotSpot:hotspot]; + [cursor_image release]; + [cocoaView setCursor:cocoa_cursor hotspotX:cursor->hot_x y:cursor->hot= _y]; + [cocoa_cursor release]; +} + +static void cocoa_mouse_set(DisplayChangeListener *dcl, int x, int y, int = on) +{ + [cocoaView setMouseX:x y:y showCursor:(on !=3D 0)]; +} + static void cocoa_display_init(DisplayState *ds, DisplayOptions *opts) { NSAutoreleasePool * pool =3D [[NSAutoreleasePool alloc] init]; diff --git a/ui/trace-events b/ui/trace-events index 69ff22955d..e53601693d 100644 --- a/ui/trace-events +++ b/ui/trace-events @@ -166,3 +166,10 @@ dbus_filter(unsigned int serial, unsigned int filter) = "serial=3D%u (<=3D %u)" =20 # egl-helpers.c egl_init_d3d11_device(void *p) "d3d device: %p" + +# cocoa.m +cocoa_cursor_define(int hot_x, int hot_y, uint16_t width, uint16_t height)= "Cursor hotspot: (%d, %d), size: %" PRIu16 "x%" PRIu16 +cocoa_grab_mouse(void) "" +cocoa_ungrab_mouse(void) "" +cocoa_handle_mouse_event_relative(int raw_dx, int raw_dy, int offset_dx, i= nt offset_dy, int net_dx, int net_dy) "raw delta: %d, %d; offset for delta:= %d, %d; net delta: %d, %d" +cocoa_handle_mouse_event_absolute(int event_x, int event_y, int screen_wid= th, int screen_height, int window_x, int window_y) "event: %d, %d; screen: = %d x %d; position: %d, %d" --=20 2.36.1