Smooth mouse movements, and a library to intercept mouse events

Smoothing mouse movements for a demo video

The problem statement: I was trying to capture a demo video of my Mac app. But I wanted the mouse movements in that video to look smooth and precise, not an easy job for a human.

I found cliclick, a command-line program which can smoothly move the mouse around and inject clicks and keystrokes, and installed it from Homebrew. You can write a single cliclick command which seqences mouse movements, clicks of different types, keypresses, and pauses, or you can build a small shell script to sequence multiple cliclick actions with other things. (It works great to start the script with a pause, so you can kick it off and then hide everything but the app you’re demoing.)

This meant that I needed a sequence of click coordinates on the screen. You can get these a few different ways: position the mouse and run cliclick p, for example, or run the Digital Color Meter app and enable Show Mouse Location (under the View menu).

Writing code to intercept mouse input

But this gave me an excuse to play around with Quartz Event Services, an API built into macOS which lets you monitor, filter, transform, or synthesize user input events—mouse actions, keyboard actions, and more. All I cared about was intercepting mouse actions and printing them out so I could compose my demo script.

The core API of interest, defined in Core Graphics, is:

CGEvent.tapCreate(tap:place:options:eventsOfInterest:callback:userInfo:)

This accepts a callback (C-style plain function pointer plus optional untyped user pointer) and a set of event types (defined by the enumeration CGEventType), and returns a Mach port object which you must add to a run loop. That run loop will then dispatch callbacks for every intercepted event.

(This tap will only be enabled if the user allowed it in System Preferences, under the Input Monitoring settings. The first time your program runs, macOS will prompt the user; if those settings change, your program must restart.)

EventMonitor is a new Swift wrapper library I wrote to encapsulate this API safely and set up a passive event tap. It’s published as an SPM package but it’s really just a single, small file with embedded documentation comments.

To use it, create an EventMonitor object, register one or more handler functions or closures by their desired event types, and call start(). It will set up the tap, validate that the tap is active (allowed by the user), route messages to your handlers, and clean up when deallocated.

Here’s a little command-line tool I made, based on that library. It sets up an EventMonitor, reporting each mouse-up’s coordinates as c:X,Y, which is how cliclick expects click events. I also detect a press of a number key and print out a little divider, so as I navigated the screen, I could delimit the output. (As a command-line tool, I needed to end it with a call to RunLoop.run, or nothing would happen; GUI apps would of course already have an active run loop).

import Foundation
import EventMonitor

var tap = EventMonitor()

tap.handle(.leftMouseUp) { event in
    print("c:\(Int(event.location.x)),\(Int(event.location.y))")
}
tap.handleKeyDown { str, _ in
    if let first = str.first, first.isNumber {
        print("--- \(str) ---")
    }
}

try! tap.start()
print("event tap installed")

RunLoop.current.run()

Pretty simple. Event taps are much more powerful than this, of course, but this was a nice foray into an aspect of the Mac API I hadn’t touched before.