From nobody Tue Apr 23 19:12:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16595948931081005.8755629539279; Wed, 3 Aug 2022 23:34:53 -0700 (PDT) Received: from localhost ([::1]:40370 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oJURW-0007O1-Fl for importer@patchew.org; Thu, 04 Aug 2022 02:34:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58162) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oJUL8-0004cZ-PW for qemu-devel@nongnu.org; Thu, 04 Aug 2022 02:28:16 -0400 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:38943) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oJUL5-0002nA-Mu for qemu-devel@nongnu.org; Thu, 04 Aug 2022 02:28:14 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id DA2483200934; Thu, 4 Aug 2022 02:28:06 -0400 (EDT) Received: from imap50 ([10.202.2.100]) by compute2.internal (MEProxy); Thu, 04 Aug 2022 02:28:07 -0400 Received: by mailuser.nyi.internal (Postfix, from userid 501) id 1AB80170007E; Thu, 4 Aug 2022 02:28:06 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nunn.io; h=cc:cc :content-type:date:date:from:from:in-reply-to:message-id :mime-version:reply-to:sender:subject:subject:to:to; s=fm2; t= 1659594486; x=1659680886; bh=haI64DsnZTvk8P6+/erRTkxoijVQaJDot4+ fywobWVA=; b=lMq+6rSZFwVU9m0lyQyT/EKqWmggLgdE6IRvEv5qWsw/Ym3Tkvr yE1T3dN0uEgwCR/kD8bMmRDuUeC1F9541klSIZNR/6x4eDTHWqwb3Z2svl7MPCMx FKlIhfjkf8Rqmn4vfl9XtzPwOASsCCAs3uMXt30/64GVg5wmadbHwrZStP6uymto /GvSymd5UT2zyCXn4wNWcqnlbb/b2BYx7vYq/rRP+ZA2holTD3zoO1KjAYuwTnFy YEjHHlVOhljwpbYgqJ3d9ksd/HSGJooeiueS5ZuCNp+JxJ+EOPsMkJlC8Ws10AFM UZlgNCcqiBzvzLIXoSdyp2+/SVD3tkD+skQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:message-id:mime-version :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1659594486; x= 1659680886; bh=haI64DsnZTvk8P6+/erRTkxoijVQaJDot4+fywobWVA=; b=f 9xxYYhc35P/0amjHH3g+Kdyj1Fl6DFIzrn9NYkfLEO7Bw/v1fEX2rwkqit/EsPa3 RUFnm3Cgs/oRhKtX6flML5Mh2nNGsjtuXLBS62+m3yUy6NzD/A2ud6Pa+4EsPFh6 nFnsrwKTCCaf7+7vsvnG/MMXSap4orNG3tHfrsuYvbww3cbRwmXJ8rLBeVr43JlI h9gUv5p4JazzaaWKbuffy22BPUG9tEsnBbeJZfN8JesDyUVs5iTjqGv6qsZ6uzF0 Hog17rkEgp6L7ouSkKLKTd5ftPvoyg+U7aATACdmpckozfLrvAps+puTeA6uK/Sw 7QK7jyy65q8owsG4C1X3Q== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvddvkedguddutdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepofgfggfkfffhvfevufgtsehttd ertderredtnecuhfhrohhmpedfgfhllhhiohhtucfpuhhnnhdfuceovghllhhiohhtsehn uhhnnhdrihhoqeenucggtffrrghtthgvrhhnpeeuuddukeejvddvgfevffeijefhkeffge ejffelvddtieeftdeukefgheduvedvudenucevlhhushhtvghrufhiiigvpedtnecurfgr rhgrmhepmhgrihhlfhhrohhmpegvlhhlihhothesnhhunhhnrdhioh X-ME-Proxy: Feedback-ID: i6a78429f:Fastmail X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.7.0-alpha0-758-ge0d20a54e1-fm-20220729.001-ge0d20a54 Mime-Version: 1.0 Message-Id: <54930451-d85f-4ce0-9a45-b3478c5a6468@www.fastmail.com> Date: Thu, 04 Aug 2022 14:27:45 +0800 From: "Elliot Nunn" To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, f4bug@amsat.org Subject: [PATCH] ui/cocoa: Support hardware cursor interface Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=64.147.123.24; envelope-from=elliot@nunn.io; helo=wout1-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1659594894414100001 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Implement dpy_cursor_define() and dpy_mouse_set() on macOS. The main benefit is from dpy_cursor_define: in absolute pointing mode, the host can redraw the cursor on the guest's behalf much faster than the guest can itself. To provide the programmatic movement expected from a hardware cursor, dpy_mouse_set is also implemented. Tricky cases are handled: - dpy_mouse_set() avoids rounded window corners. - The sometimes-delay between warping the cursor and an affected mouse-move event is accounted for. - Cursor bitmaps are nearest-neighbor scaled to Retina size. Signed-off-by: Elliot Nunn --- ui/cocoa.m | 263 ++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 240 insertions(+), 23 deletions(-) diff --git a/ui/cocoa.m b/ui/cocoa.m index 5a8bd5dd84..f9d54448e4 100644 --- a/ui/cocoa.m +++ b/ui/cocoa.m @@ -85,12 +85,20 @@ static void cocoa_switch(DisplayChangeListener *dcl, =20 static void cocoa_refresh(DisplayChangeListener *dcl); =20 +static void cocoa_mouse_set(DisplayChangeListener *dcl, + int x, int y, int on); + +static void cocoa_cursor_define(DisplayChangeListener *dcl, + QEMUCursor *c); + static NSWindow *normalWindow; static const DisplayChangeListenerOps dcl_ops =3D { .dpy_name =3D "cocoa", .dpy_gfx_update =3D cocoa_update, .dpy_gfx_switch =3D cocoa_switch, .dpy_refresh =3D cocoa_refresh, + .dpy_mouse_set =3D cocoa_mouse_set, + .dpy_cursor_define =3D cocoa_cursor_define, }; static DisplayChangeListener dcl =3D { .ops =3D &dcl_ops, @@ -313,6 +321,13 @@ @interface QemuCocoaView : NSView BOOL isFullscreen; BOOL isAbsoluteEnabled; CFMachPortRef eventsTap; + NSCursor *guestCursor; + BOOL cursorHiddenByMe; + BOOL guestCursorVis; + int guestCursorX, guestCursorY; + int lastWarpX, lastWarpY; + int warpDeltaX, warpDeltaY; + BOOL ignoreNextMouseMove; } - (void) switchSurface:(pixman_image_t *)image; - (void) grabMouse; @@ -323,6 +338,10 @@ - (void) handleMonitorInput:(NSEvent *)event; - (bool) handleEvent:(NSEvent *)event; - (bool) handleEventLocked:(NSEvent *)event; - (void) setAbsoluteEnabled:(BOOL)tIsAbsoluteEnabled; +- (void) cursorDefine:(NSCursor *)cursor; +- (void) mouseSetX:(int)x Y:(int)y on:(int)on; +- (void) setCursorAppearance; +- (void) setCursorPosition; /* The state surrounding mouse grabbing is potentially confusing. * isAbsoluteEnabled tracks qemu_input_is_absolute() [ie "is the emulated * pointing device an absolute-position one?"], but is only updated on @@ -432,22 +451,6 @@ - (CGPoint) screenLocationOfEvent:(NSEvent *)ev } } =20 -- (void) hideCursor -{ - if (!cursor_hide) { - return; - } - [NSCursor hide]; -} - -- (void) unhideCursor -{ - if (!cursor_hide) { - return; - } - [NSCursor unhide]; -} - - (void) drawRect:(NSRect) rect { COCOA_DEBUG("QemuCocoaView: drawRect\n"); @@ -635,6 +638,8 @@ - (void) switchSurface:(pixman_image_t *)image screen.height =3D h; [self setContentDimensions]; [self setFrame:NSMakeRect(cx, cy, cw, ch)]; + [self setCursorAppearance]; + [self setCursorPosition]; } =20 // update screenBuffer @@ -681,6 +686,7 @@ - (void) toggleFullScreen:(id)sender styleMask:NSWindowStyleMaskBorderless backing:NSBackingStoreBuffered defer:NO]; + [fullScreenWindow disableCursorRects]; [fullScreenWindow setAcceptsMouseMovedEvents: YES]; [fullScreenWindow setHasShadow:NO]; [fullScreenWindow setBackgroundColor: [NSColor blackColor]]; @@ -812,6 +818,7 @@ - (bool) handleEventLocked:(NSEvent *)event int buttons =3D 0; int keycode =3D 0; bool mouse_event =3D false; + bool mousemoved_event =3D false; // Location of event in virtual screen coordinates NSPoint p =3D [self screenLocationOfEvent:event]; NSUInteger modifiers =3D [event modifierFlags]; @@ -1023,6 +1030,7 @@ - (bool) handleEventLocked:(NSEvent *)event } } mouse_event =3D true; + mousemoved_event =3D true; break; case NSEventTypeLeftMouseDown: buttons |=3D MOUSE_EVENT_LBUTTON; @@ -1039,14 +1047,17 @@ - (bool) handleEventLocked:(NSEvent *)event case NSEventTypeLeftMouseDragged: buttons |=3D MOUSE_EVENT_LBUTTON; mouse_event =3D true; + mousemoved_event =3D true; break; case NSEventTypeRightMouseDragged: buttons |=3D MOUSE_EVENT_RBUTTON; mouse_event =3D true; + mousemoved_event =3D true; break; case NSEventTypeOtherMouseDragged: buttons |=3D MOUSE_EVENT_MBUTTON; mouse_event =3D true; + mousemoved_event =3D true; break; case NSEventTypeLeftMouseUp: mouse_event =3D true; @@ -1121,7 +1132,12 @@ - (bool) handleEventLocked:(NSEvent *)event qemu_input_update_buttons(dcl.con, bmap, last_buttons, buttons= ); last_buttons =3D buttons; } - if (isMouseGrabbed) { + + if (!isMouseGrabbed) { + return false; + } + + if (mousemoved_event) { if (isAbsoluteEnabled) { /* Note that the origin for Cocoa mouse coords is bottom l= eft, not top left. * The check on screenContainsPoint is to avoid sending ou= t of range values for @@ -1132,11 +1148,38 @@ - (bool) handleEventLocked:(NSEvent *)event qemu_input_queue_abs(dcl.con, INPUT_AXIS_Y, screen.hei= ght - p.y, 0, screen.height); } } else { - qemu_input_queue_rel(dcl.con, INPUT_AXIS_X, (int)[event de= ltaX]); - qemu_input_queue_rel(dcl.con, INPUT_AXIS_Y, (int)[event de= ltaY]); + if (ignoreNextMouseMove) { + // Discard the first mouse-move event after a grab, be= cause + // it includes the warp delta from an unknown initial = position. + ignoreNextMouseMove =3D NO; + warpDeltaX =3D warpDeltaY =3D 0; + } else { + // Correct subsequent events to remove the known warp = delta. + // The warp delta is sometimes late to be reported, so= never + // allow the delta compensation to alter the direction. + int dX =3D (int)[event deltaX]; + int dY =3D (int)[event deltaY]; + + if (dX =3D=3D 0 || (dX ^ (dX - warpDeltaX)) < 0) { // = Flipped sign? + warpDeltaX -=3D dX; // Save excess correction for = later + dX =3D 0; + } else { + dX -=3D warpDeltaX; // Apply entire correction + warpDeltaX =3D 0; + } + + if (dY =3D=3D 0 || (dY ^ (dY - warpDeltaY)) < 0) { + warpDeltaY -=3D dY; + dY =3D 0; + } else { + dY -=3D warpDeltaY; + warpDeltaY =3D 0; + } + + qemu_input_queue_rel(dcl.con, INPUT_AXIS_X, dX); + qemu_input_queue_rel(dcl.con, INPUT_AXIS_Y, dY); + } } - } else { - return false; } qemu_input_event_sync(); } @@ -1153,9 +1196,15 @@ - (void) grabMouse else [normalWindow setTitle:@"QEMU - (Press ctrl + alt + g to relea= se Mouse)"]; } - [self hideCursor]; CGAssociateMouseAndMouseCursorPosition(isAbsoluteEnabled); isMouseGrabbed =3D TRUE; // while isMouseGrabbed =3D TRUE, QemuCocoaAp= p sends all events to [cocoaView handleEvent:] + [self setCursorAppearance]; + [self setCursorPosition]; + + // We took over and warped the mouse, so ignore the next mouse-move + if (!isAbsoluteEnabled) { + ignoreNextMouseMove =3D YES; + } } =20 - (void) ungrabMouse @@ -1168,9 +1217,14 @@ - (void) ungrabMouse else [normalWindow setTitle:@"QEMU"]; } - [self unhideCursor]; CGAssociateMouseAndMouseCursorPosition(TRUE); isMouseGrabbed =3D FALSE; + [self setCursorAppearance]; + + if (!isAbsoluteEnabled) { + ignoreNextMouseMove =3D NO; + warpDeltaX =3D warpDeltaY =3D 0; + } } =20 - (void) setAbsoluteEnabled:(BOOL)tIsAbsoluteEnabled { @@ -1179,6 +1233,116 @@ - (void) setAbsoluteEnabled:(BOOL)tIsAbsoluteEnable= d { CGAssociateMouseAndMouseCursorPosition(isAbsoluteEnabled); } } + +// Indirectly called by dpy_cursor_define() in the virtual GPU +- (void) cursorDefine:(NSCursor *)cursor { + guestCursor =3D cursor; + [self setCursorAppearance]; +} + +// Indirectly called by dpy_mouse_set() in the virtual GPU +- (void) mouseSetX:(int)x Y:(int)y on:(int)on { + if (!on !=3D !guestCursorVis) { + guestCursorVis =3D on; + [self setCursorAppearance]; + } + + if (on && (x !=3D guestCursorX || y !=3D guestCursorY)) { + guestCursorX =3D x; + guestCursorY =3D y; + [self setCursorPosition]; + } +} + +// Change the cursor image to the default, the guest cursor bitmap or hidd= en. +// Said to be an expensive operation on macOS Monterey, so use sparingly. +- (void) setCursorAppearance { + NSCursor *cursor =3D NULL; // NULL means hidden + + if (!isMouseGrabbed) { + cursor =3D [NSCursor arrowCursor]; + } else if (!guestCursor && !cursor_hide) { + cursor =3D [NSCursor arrowCursor]; + } else if (guestCursorVis && guestCursor) { + cursor =3D guestCursor; + } else { + cursor =3D NULL; + } + + if (cursor !=3D NULL) { + [cursor set]; + + if (cursorHiddenByMe) { + [NSCursor unhide]; + cursorHiddenByMe =3D NO; + } + } else { + if (!cursorHiddenByMe) { + [NSCursor hide]; + cursorHiddenByMe =3D YES; + } + } +} + +// Move the cursor within the virtual screen +- (void) setCursorPosition { + // Ignore the guest's request if the cursor belongs to Cocoa + if (!isMouseGrabbed || isAbsoluteEnabled) { + return; + } + + // Get guest screen rect in Cocoa coordinates (bottom-left origin). + NSRect virtualScreen =3D [[self window] convertRectToScreen:[self fram= e]]; + + // Convert to top-left origin. + NSInteger hostScreenH =3D [NSScreen screens][0].frame.size.height; + int scrX =3D virtualScreen.origin.x; + int scrY =3D hostScreenH - virtualScreen.origin.y - virtualScreen.size= .height; + int scrW =3D virtualScreen.size.width; + int scrH =3D virtualScreen.size.height; + + int cursX =3D scrX + guestCursorX; + int cursY =3D scrY + guestCursorY; + + // Clip to edges + cursX =3D MIN(MAX(scrX, cursX), scrX + scrW - 1); + cursY =3D MIN(MAX(scrY, cursY), scrY + scrH - 1); + + // Move diagonally towards the center to avoid rounded window corners. + // Limit the number of hit-tests and discard failed attempts. + int betterX =3D cursX, betterY =3D cursY; + for (int i=3D0; i<16; i++) { + if ([NSWindow windowNumberAtPoint:NSMakePoint(betterX, hostScreenH= - betterY) + belowWindowWithWindowNumber:0] =3D=3D self.window.windowNumber= ) { + cursX =3D betterX; + cursY =3D betterY; + break; + }; + + if (betterX < scrX + scrW/2) { + betterX++; + } else { + betterX--; + } + + if (betterY < scrY + scrH/2) { + betterY++; + } else { + betterY--; + } + } + + // Subtract this warp delta from the next NSEventTypeMouseMoved. + // These are in down-is-positive coords, same as NSEvent deltaX/deltaY. + warpDeltaX +=3D cursX - lastWarpX; + warpDeltaY +=3D cursY - lastWarpY; + + CGWarpMouseCursorPosition(NSMakePoint(cursX, cursY)); + + lastWarpX =3D cursX; + lastWarpY =3D cursY; +} + - (BOOL) isMouseGrabbed {return isMouseGrabbed;} - (BOOL) isAbsoluteEnabled {return isAbsoluteEnabled;} - (float) cdx {return cdx;} @@ -1251,6 +1415,7 @@ - (id) init error_report("(cocoa) can't create window"); exit(1); } + [normalWindow disableCursorRects]; [normalWindow setAcceptsMouseMovedEvents:YES]; [normalWindow setTitle:@"QEMU"]; [normalWindow setContentView:cocoaView]; @@ -2123,6 +2288,58 @@ static void cocoa_display_init(DisplayState *ds, Dis= playOptions *opts) qemu_clipboard_peer_register(&cbpeer); } =20 +static void cocoa_mouse_set(DisplayChangeListener *dcl, int x, int y, int = on) { + dispatch_async(dispatch_get_main_queue(), ^{ + [cocoaView mouseSetX:x Y:y on:on]; + }); +} + +// Convert QEMUCursor to NSCursor, then call cursorDefine +static void cocoa_cursor_define(DisplayChangeListener *dcl, QEMUCursor *cu= rsor) { + CFDataRef cfdata =3D CFDataCreate( + /*allocator*/ NULL, + /*bytes*/ (void *)cursor->data, + /*length*/ sizeof(uint32_t) * cursor->width * cursor->height); + + CGDataProviderRef dataprovider =3D CGDataProviderCreateWithCFData(cfda= ta); + + CGImageRef cgimage =3D CGImageCreate( + cursor->width, cursor->height, + /*bitsPerComponent*/ 8, + /*bitsPerPixel*/ 32, + /*bytesPerRow*/ sizeof(uint32_t) * cursor->width, + /*colorspace*/ CGColorSpaceCreateWithName(kCGColorSpaceSRGB), + /*bitmapInfo*/ kCGBitmapByteOrder32Host | kCGImageAlphaLast, + /*provider*/ dataprovider, + /*decode*/ NULL, + /*shouldInterpolate*/ FALSE, + /*intent*/ kCGRenderingIntentDefault); + + NSImage *unscaled =3D [[NSImage alloc] initWithCGImage:cgimage size:NS= ZeroSize]; + + CFRelease(cfdata); + CGDataProviderRelease(dataprovider); + CGImageRelease(cgimage); + + // Nearest-neighbor scale to the possibly "Retina" cursor size + NSImage *scaled =3D [NSImage + imageWithSize:NSMakeSize(cursor->width, cursor->height) + flipped:NO + drawingHandler:^BOOL(NSRect dest) { + [NSGraphicsContext currentContext].imageInterpolation =3D NSIm= ageInterpolationNone; + [unscaled drawInRect:dest]; + return YES; + }]; + + NSCursor *nscursor =3D [[NSCursor alloc] + initWithImage:scaled + hotSpot:NSMakePoint(cursor->hot_x, cursor->hot_y)]; + + dispatch_async(dispatch_get_main_queue(), ^{ + [cocoaView cursorDefine:nscursor]; + }); +} + static QemuDisplay qemu_display_cocoa =3D { .type =3D DISPLAY_TYPE_COCOA, .init =3D cocoa_display_init,