AppKit notes: Customizing text view drawing

I just implemented a small customization to text-view drawing which I thought was pretty neat. I thought I’d walk through it, describing some of the steps to customize text drawing in AppKit and TextKit 1, for anyone thinking about doing something similar in their own code.

My app lets you type into an unbordered, transparent text view, over a background whose colors can vary. The code previously tried to pick the one best-contrasting text color, but due to a bug report I finally bit the bullet and made it adapt within the view as you type:

To apply colors to character ranges, you just need to configure your view’s attributed string content. But I wanted to apply colors to rectangular regions, even when those regions split a single glyph; this requires customizing the text drawing code.

NSTextView is the main text view class in AppKit. In TextKit 1, the classic text system, it uses NSTextContainer to define the geometry of its available text space and NSLayoutManager to lay out and draw glyphs into that space. Customizing text drawing therefore starts with subclassing NSLayoutManager and installing our custom instance by calling replaceLayoutManager(_:) on our text view’s text container.

Note: Apple introduced TextKit 2 (link to WWDC video) in macOS Monterey, making it the default for text views in Ventura. TextKit 2 uses an updated architecture with a new layout manager, but as soon as you access NSLayoutManager, your view reconfigures back to the older architecture. More info in a useful article by Daniel Jalkut, from which I got this clarification. Since my app still targets back to Catalina, I haven’t even tried TextKit 2.

This is the key method to override in our NSLayoutManager subclass:

override func drawGlyphs(forGlyphRange glyphsToShow: NSRange, at textContainerOrigin: NSPoint) {

This little bit of code, at the top of that method, gets us access to our text view and text container objects. (These objects are normally 1:1:1, but don’t have to be.)

guard
    let textView = firstTextView,
    let textContainer = textContainer(forGlyphAt: glyphsToShow.location, effectiveRange: nil)
else {
    super.drawGlyphs(forGlyphRange: glyphsToShow, at: textContainerOrigin)
    return
}

We can ask the layout manager to compute a bounding rectangle for any range of glyphs; that rectangle comes back in the text container’s coordinate space, so we next normally need to apply the container’s relative origin (passed into this method) to map it into the text view’s coordinate space. Finally, for my use case, I needed to map that rectangle into one in my background view’s coordinate space, so I could query my own model for the correct colors.

let boundsInContainer = boundingRect(forGlyphRange: glyphsToShow, in: textContainer)
let boundsInView = boundsInContainer.offsetBy(dx: textContainerOrigin.x, dy: textContainerOrigin.y)
let boundsInBackground = backgroundView.convert(boundsInView, from: textView)

I’ll skip the app-specific logic here; what it returns is a collection of “regions” defined by rectangles (in background-view space) and chosen foreground colors. Each region needs to be mapped back to text-view and text-container spaces, and then to both glyph and character ranges, since we’ll need all of those before we’re done.

// for each "rectInBackground" and "color":
let rectInTextView = textView.convert(rectInBackground, from: backgroundView)
let rectInContainer = rectInTextView.offsetBy(dx: -textContainerOrigin.x, dy: -textContainerOrigin.y)
let glyphsInRegion = glyphRange(forBoundingRect: rectInContainer, in: textContainer)
let charsInRegion = characterRange(forGlyphRange: glyphsInRegion, actualGlyphRange: nil)

Using the character range, we can apply a temporary attribute (of the NSAttributedString kind). Temporary attributes can affect drawing, including color, but cannot affect layout.

addTemporaryAttribute(.foregroundColor, value: color, forCharacterRange: charsInRegion)
defer { removeTemporaryAttribute(.foregroundColor, forCharacterRange: charsInRegion) }

Using the view-relative rectangle, I set the current clip, since I’m going to be drawing entire glyph ranges, but I want this color to end at exactly the rectangle’s bounds:

NSGraphicsContext.current?.saveGraphicsState()
defer { NSGraphicsContext.current?.restoreGraphicsState() }
rectInTextView.clip()

Finally, using the glyph range, I just ask the superclass to draw the the glyphs with my clip and colors in effect:

super.drawGlyphs(forGlyphRange: glyphsInRegion, at: textContainerOrigin)

That takes care of the text drawing, but not the insertion point (cursor) color. The layout manager does not draw the cursor; that’s directly handled by NSTextView. In my NSTextView subclass, I overrode drawInsertionPoint(in:color:turnedOn:), looked up the correct color, and just called super with that color instead of the original one. Note that the passed-in rect is in the view’s coordinate space, not the container’s.

(Users of TimeStory will see the result in 3.5, whenever I get it released.)