Input Events in Qt 6

The delivery of mouse and touch events in Qt Quick is complex, and it became clear a few years ago that we needed to refactor the event inheritance hierarchy, to have some common API for various event types, so that more of the delivery code could be shared. In Qt 5.8 we added QQuickPointerEvent and associated types, as a way of prototyping what that could look like. They are QObjects; and since then, QQuickWindow has been delivering these wrapper events that carry the original events inside. Now finally in Qt 6 we have been able to complete the QEvent refactoring, so that QQuickWindow no longer needs the wrappers. Along with that, we were able to add a few features and fix a few bugs. Many of the remaining bugs that seemed intractable in Qt 5 should at least be possible to fix later on.

Say hello to QPointerEvent and QEventPoint

The inheritance hierarchy now looks like this:

QPointerEvent is the new abstract type for all events that come from pointing devices (mouse, touchscreen, tablet stylus). It has the common API to be able to handle all these in a device-agnostic way. Since QTouchEvent can carry multiple touchpoints in one event, we standardized this concept: every QPointerEvent represents potentially a cluster of QEventPoint instances (even though most events only carry one point), and therefore has appropriate API: points(), point(i) and pointCount().
Every QInputEvent (including QPointerEvent) carries a pointer to the QInputDevice that it came from. This allows event-handling code to respond in device-specific ways, even when handling synthetic mouse events.

Every QEventPoint has velocity. The Kalman filter that Qt Quick was using in Qt 5 has been moved to QtGui, so that the average velocity from the last few movements is available wherever the event is delivered. This enables velocity-sensitive behavior (such as distinguishing a slow drag from a quick flick, or responding to a particular direction of movement) regardless which device it comes from. Instantaneous velocity is generally too volatile for such purposes, but if you need it you can calculate it as (globalPosition() - globalLastPosition()) / (timestamp() - lastTimestamp()).

QSinglePointEvent is another abstract type that standardizes the position accessors that used to be separately and inconsistently implemented in QMouseEvent, QTabletEvent, QHoverEvent, QWheelEvent and a few more. They are all floating-point now. position() replaces pos() and posF(), scenePosition() replaces windowPos(), and globalPosition() replaces screenPos(). The old accessors are still there for now, but deprecated: a Qt 5 application will not encounter an SC break just because of handling QMouseEvent, for example. QEventPoint replaces QTouchEvent::TouchPoint, but there is a "using" declaration for the sake of source compatibililty.

I have forked clazy and added a new qevent-accessors check, which might save you some trouble: it can automatically apply "fixits" to get rid of the deprecation warnings stemming from the event accessor renaming. (Yes we should upstream that change.)

Device-agnostic event handling in C++

In various places in Qt we can now respond to mouse, touch and tablet events (such as detecting a click or a drag), either by iterating the QEventPoints, or just responding to the first point. Here's a contrived example of how a QQuickItem subclass could do that:

bool MyItem::event(QEvent *ev) override {
  if (ev->isPointerEvent()) {
    QPointerEvent *pev = static_cast<QPointerEvent *>(ev);
    for (QEventPoint &point : pev->points()) {
      switch (point.state()) {
      case QEventPoint::State::Pressed:
        if (reactToPress(point.position()))
          pev->setExclusiveGrabber(point, this);
        break;
      case QEventPoint::State::Updated:
        ...
      }
    }
    return true;
  }
  return QQuickItem::event(ev);
}

QQuickFlickable::childMouseEventFilter() works this way, for example. And that has an interesting result:

Flickable handles touch now!

There were many open bugs related to the fact that Qt 5's Flickable could only handle actual mouse events and synthetic mouse events. Qt only supports one mouse, one mouse position, one cursor (so far, but we're working on that...) and therefore you could not flick two Flickables with two fingers. If you touch some component inside the Flickable that is able to handle touch events, but then you drag your finger across the Flickable in the allowed direction, it uses childMouseEventFilter() to steal the grab from that component; but that involved switching from the actual touch event to a synthetic mouse event, and also remembering to deliver the following updates as synth-mouse events to Flickable. Various things went wrong. Well... those days are over, because Flickable::childMouseEventFilter() no longer cares which device the QPointerEvents come from. If you set pressDelay, it's able to withhold the actual touch press and then replay it to the items inside when the timer expires. Yes, you can now drag multiple Flickables with multiple fingers too.

Multi-touch still doesn't work with the remaining mouse-only items like MouseArea though, because those still rely on synth-mouse events. But it can be avoided. In general: try to use Event Handlers rather than MouseArea, because (as its name indicates) it's not really intended to support anything more than mouse interaction.

QTabletEvents (from your Wacom stylus, Samsung S-pen, Apple Pencil etc.) are also just pointer events that carry a few more properties, and can be handled by any device-agnostic code that handles mouse and touch events. But we will keep working on improving the experience with those. We didn't add any new virtual functions in QQuickItem for them, but they will get delivered to QQuickItem::event() soon.

Another thing we're still working on is making Flickable behave better with laptop touchpads. A fix is coming soon.

Left to your own devices

Platform plugins now have responsibility to discover and register input devices, in order to fulfill the promise that every QInputEvent tells you exactly where it came from. Some of this work is already done on some of the platforms. (On others it's proving to be problematic.) You can get a list of all registered devices from QInputDevice::devices(). The qtdiag utility also shows you this list.

QInputDevice is a QObject, and its parent() can be another device in case there is a natural hierarchy: for example, X11 has master and slave devices, and a tablet stylus "belongs to" a particular tablet device. In other cases, the parent is simply an object within the platform plugin that owns the device for memory management purposes.

On the platforms where the device discovery work is not done, QInputEvent::device() is never null, but may be a generic instance taken from QInputDevice::primaryKeyboard() or QPointingDevice::primaryMouse(). Touchscreen devices are unique though; we already did it that way in Qt 5.

QInputDevice::seatName() correponds to the Wayland notion of a "seat": a set of devices that one user is using. There is little multi-seat support so far, but it will be improved over time. If you configure a multi-pointer X server, you can see different seat names on different devices, but those names are automatically generated from xinput IDs in the xcb plugin. On Wayland compositors such as Sway, it's possible to give the seats arbitrary names; and we plan that Qt will eventually work with that.

$ xinput list
Virtual core pointer id=2 [master pointer (3)]
  Virtual core XTEST pointer id=4 [slave pointer (2)]
  ZSA Technology Labs ErgoDox EZ Mouse id=11 [slave pointer (2)]
  ZSA Technology Labs ErgoDox EZ Consumer Control id=13 [slave pointer (2)]
  Logitech MX Master 2S id=15 [slave pointer (2)]
Virtual core keyboard id=3 [master keyboard (2)]
  Virtual core XTEST keyboard id=5 [slave keyboard (3)]
  Power Button id=6 [slave keyboard (3)]
  Power Button id=7 [slave keyboard (3)]
  Sleep Button id=8 [slave keyboard (3)]
  UVC Camera (046d:0992) id=9 [slave keyboard (3)]
  ZSA Technology Labs ErgoDox EZ id=10 [slave keyboard (3)]
  ZSA Technology Labs ErgoDox EZ System Control id=12 [slave keyboard (3)]
  ZSA Technology Labs ErgoDox EZ Keyboard id=14 [slave keyboard (3)]
  ZSA Technology Labs ErgoDox EZ Consumer Control id=16 [slave keyboard (3)]
  Logitech MX Master 2S id=17 [slave keyboard (3)]
aux pointer id=22 [master pointer (23)]
  Microsoft Microsoft Optical Mouse by Starck id=19 [slave pointer (22)]
  aux XTEST pointer id=24 [slave pointer (22)]
aux keyboard id=23 [master keyboard (22)]
  Apple, Inc Apple Keyboard id=20 [slave keyboard (23)]
  Apple, Inc Apple Keyboard id=21 [slave keyboard (23)]
  aux XTEST keyboard id=25 [slave keyboard (23)]

$ qtdiag
Qt 6.0.0 (x86_64-little_endian-lp64 shared (dynamic) debug build; by GCC 10.2.0) on "xcb" 
OS: Arch Linux [linux version 5.9.11-arch2-1]
...
Input devices: 23
QInputDevice::DeviceType::Mouse "Virtual core pointer", seat: "30002" capabilities: Position Scroll Hover
QInputDevice::DeviceType::Keyboard "Virtual core keyboard", seat: "30002" capabilities:
QInputDevice::DeviceType::Mouse "aux pointer", seat: "170016" capabilities: Position Scroll Hover
QInputDevice::DeviceType::Keyboard "aux keyboard", seat: "170016" capabilities:
QInputDevice::DeviceType::Mouse "Virtual core XTEST pointer", seat: "30002" capabilities: Position Scroll Hover
QInputDevice::DeviceType::Keyboard "Virtual core XTEST keyboard", seat: "30002" capabilities:
QInputDevice::DeviceType::Keyboard "Power Button", seat: "30002" capabilities:
QInputDevice::DeviceType::Keyboard "Power Button", seat: "30002" capabilities:
QInputDevice::DeviceType::Keyboard "Sleep Button", seat: "30002" capabilities:
QInputDevice::DeviceType::Keyboard "UVC Camera (046d:0992)", seat: "30002" capabilities:
QInputDevice::DeviceType::Keyboard "ZSA Technology Labs ErgoDox EZ", seat: "30002" capabilities:
QInputDevice::DeviceType::Mouse "ZSA Technology Labs ErgoDox EZ Mouse", seat: "30002" capabilities: Position Scroll Hover
QInputDevice::DeviceType::Keyboard "ZSA Technology Labs ErgoDox EZ System Control", seat: "30002" capabilities:
QInputDevice::DeviceType::Mouse "ZSA Technology Labs ErgoDox EZ Consumer Control", seat: "30002" capabilities: Position Scroll Hover
QInputDevice::DeviceType::Keyboard "ZSA Technology Labs ErgoDox EZ Keyboard", seat: "30002" capabilities:
QInputDevice::DeviceType::Keyboard "ZSA Technology Labs ErgoDox EZ Consumer Control", seat: "30002" capabilities:
QInputDevice::DeviceType::Mouse "Logitech MX Master 2S", seat: "30002" capabilities: Position Scroll Hover
QInputDevice::DeviceType::Keyboard "Logitech MX Master 2S", seat: "30002" capabilities:
QInputDevice::DeviceType::Mouse "Microsoft Microsoft Optical Mouse by Starck", seat: "170016" capabilities: Position Scroll Hover
QInputDevice::DeviceType::Keyboard "Apple, Inc Apple Keyboard", seat: "170016" capabilities:
QInputDevice::DeviceType::Keyboard "Apple, Inc Apple Keyboard", seat: "170016" capabilities:
QInputDevice::DeviceType::Mouse "aux XTEST pointer", seat: "170016" capabilities: Position Scroll Hover
QInputDevice::DeviceType::Keyboard "aux XTEST keyboard", seat: "170016" capabilities:

QInputDevice::availableVirtualGeometry() is intended to tell you which area of the virtual desktop this device can access. For example you might be using a touchscreen laptop with an external monitor: then the touchscreen's QPointingDevice::availableVirtualGeometry() should be the same as the screen's QScreen::geometry(). A Wacom tablet could be mapped to an area smaller than the whole screen, to improve drawing precision (using xinput or an OS-specific control panel). But again, this work is not yet done on all supported platforms.

Synthetic mouse events

Synthetic mouse events still exist, even though we are trying to rely less on them now.

QEvent::spontaneous() is the oldest method we have to distinguish OS-generated events from synthetic events, but it wasn't suitable for distinguishing synth-mouse events. In Qt 5, QMouseEvent::source() was added to help you distinguish mouse events that were synthesized from some other device, by the operating system, by Qt or by the application; but then we found it to be both tempting and wrong to assume that such an event was synthesized from a touchpoint. (It could be synthesized from a QTabletEvent, for example.) So we recommend using event->device()->type() and/or pointerDevice()->pointerType() to distinguish these now. When a QMouseEvent is synthesized from some other type of event, the device instance remains the same, so that you can tell where it really came from.

Categorized logging

Since the addition of categorized logging a few years ago, we have been adding more and more internal logging in Qt. (git grep Q_LOGGING_CATEGORY in qtbase and qtdeclarative will find many useful ones for debugging Qt Quick applications.) The most useful category for troubleshooting mouse and touch interaction problems is qt.pointer.grab, because grab transitions are either a symptom or a cause of most such problems. But there are many more: you can log the events at the QPA level, at the QtGui level, in QQuickWindow, during delivery to items and/or to handlers; and you can log various aspects of interaction in various items and handlers.

I might write about more technical details concerning grabbing and accepting events in a future post.

Good luck and have fun with Qt 6!