The GNU/Linux counterpart of the Metal backend: the cross-platform
drawing policy in gfxterm.c now renders through a second gfx_driver
implementation, src/glterm.c, built on EGL + OpenGL ES 3 with glyphs
rasterized by FreeType/cairo into a GPU atlas.
Highlights:
- Pixel parity with the stock GTK/cairo renderer, verified with a
capture harness across text (ASCII/Latin/CJK/emoji/BiDi), faces,
decorations (underline/overline/box/relief), fringes, margins,
line numbers, mode/header/tab lines, images (PNG/JPEG/SVG/GIF),
scroll runs and the four cursor types.
- On-screen present: the frame FBO is blitted to the X window through
an EGL window surface with deferred presents (no flicker); headless
runs render to the FBO alone (GL_FORCE_SURFACELESS for Xvfb).
- Glyph quads are batched into one draw call per state change, and the
present can blit only the dirty region (GL_PARTIAL_PRESENT, opt-in).
- Color glyphs (emoji) rasterized through the cairo scaled font into
RGBA textures; HiDPI supersampling via GL_SCALE.
- Inline video through GStreamer (playbin + appsink): decoded RGBA
frames are uploaded and composited over the frame, following scroll
and clipped to the window (gpu-video-insert / gpu-video-mode).
- Buffer-switch cross-fade and the animated cursor effects (spring,
torpedo, sonicboom, ripple, pixiedust, hollow, beam), composited in
the present pass; sonicboom is the default on this backend.
- --with-gpu configure flag selects the platform backend (Metal on
macOS, OpenGL on X11) and auto-enables it at startup, with
EMACS_GPU_DISABLE as the opt-out.
- gpu-status / gpu-enable / gpu-set-cursor and friends work on both
backends; Metal-only primitives are guarded.
Shared-policy fixes that also benefit the Metal driver: relief edge
order and outermost shadow line now match the native renderers,
fringe bitmaps are clipped to the row's visible band, and the
mouse-face highlight no longer goes stale after a deferred present
(mouse_face_defer is cleared at update_end).
The character drawing path resolved glyphs through face->font, but
char2b holds glyph IDs encoded for s->font. These usually match, so
the bug stayed hidden until a mouse-face highlight over text in a
non-default font (e.g. shr / variable-pitch links in elfeed): Emacs
swaps s->face to the highlight face while leaving s->font and char2b
as the original font, so the highlight redraw looked the proportional
glyph IDs up in the highlight face's monospace atlas and rendered
garbage. Use s->font, matching the NS and X backends and the
composition glyph string path in this file.
Add declare-function forms for the primitives implemented in
src/mtlfns.m so byte-compiling the file without the backend is
warning-free, and capitalize two error messages to satisfy checkdoc.
Add gpu-video-mode: visiting a video file (mp4/mov/m4v/3gp) opens it in
a buffer that autoplays and loops, fit to the window, with a play/pause
button and a clickable, draggable timeline. Registered in
auto-mode-alist so RET in Dired opens it; animated GIFs keep using
image-mode.
Expose the AVPlayer state the controls need: gpu-video-duration,
gpu-video-position, gpu-video-seek, gpu-video-playing-p and
gpu-video-size.
- Auto-load the gpu module and enable the backend on the initial frame at
startup when built --with-mtl; opt out with EMACS_GPU_DISABLE.
- Add autoload cookies to gpu-enable/status/set-cursor/toggle-animations/
set-scroll so they are available via M-x without requiring gpu first.
- Suppress the motion cursor effects (sonicboom/ripple/pixiedust bursts and
the torpedo trail) during typing/editing commands; only cursor movement
and jumps trigger them. Configurable via gpu-cursor-effects-while-typing
and gpu-cursor-typing-commands.
- README: document startup auto-enable, the typing behavior, and the
Ventura+ requirement.
-[NSScreen displayLinkWithTarget:selector:] and CADisplayLink are macOS
14+ APIs. Guard startAnimating/stopAnimating behind @available so the
backend builds and runs with a 13.0 deployment target; continuous
animation is driven by the Lisp 30fps timer anyway. Bump version to 0.1.2.
The gpu-cursor-animation defcustom used :set #'set-default, so changing
the cursor mode at runtime (via gpu-set-cursor / setopt) only updated the
Lisp variable and never reached the live Metal driver. Wire :set to call
gpu-cursor-mode so the change propagates immediately.
Also set the default cursor to 'block (static, no effect); cursor effects
are now opt-in via gpu-set-cursor. Bump version to 0.1.1.
The project is emacs-gpu and the drawing core is platform-neutral; the
user API should not be Metal-branded (a future GNU/Linux user calling
mtl-enable would be absurd). gpu.el carries the whole API under the
gpu- prefix; the mtl- entry points live on as obsolete aliases and
(require 'mtl) still works through a compatibility shim.
C internals keep their accurate names: gfx_* is the neutral policy and
mtl_* is the Metal driver.
Buffer switches now cross-fade: just before redisplay paints the new
buffer (pre-redisplay-functions), the driver snapshots the static
texture and the compositor fades it out over the new content
(smoothstep, GPU-composited). Configurable with mtl-buffer-transitions
(default t) and mtl-buffer-transition-duration (default 0.15s), and
exposed as mtl-transition-start for custom effects.
Cursor animations are enabled by default now that they behave: the
burst modes (sonicboom/ripple/pixiedust) keep the proper static cursor
-- inverted glyph, exact NS parity -- and only draw their effects in
the overlay; a solid overlay body would have hidden the character
under the cursor. The body-animated modes (spring/torpedo/hollow/
beam) still draw the overlay body.
The animation pump helpers moved above the defcustoms: with the
default flipped to t, custom-initialize-reset runs the :set at load
time, which called mtl--anim-pump-start before it was defined.
README documents the user commands (status, draw-stats, cursor modes,
vsync, video, transitions).
mtl--maybe-enable-on-startup enabled the SELECTED frame instead of the
frame being created (mid-creation the new frame is not selected yet),
and an error there aborted frame creation: tooltip, child and TTY
frames must simply be skipped. Use the hook's frame argument, guard on
display-graphic-p and swallow enable errors.
The application bundle now ships a site-start.el that turns the GPU
backend on by default (EMACS_GPU_DISABLE=1 reverts to the stock
backend); the bundle-build recipe adds it after make install.
Bottom dividers arrive outside the update cycle, where a bare fill is
silently dropped (vertical dividers happened to come in-cycle, which is
why they worked): wrap both divider/border paths in the self-contained
immediate cycle used by the other outside-the-cycle draws.
Also stop the Metal resize observer from writing FRAME_PIXEL_* behind
change_frame_size's back; the view's own resize path already drives the
layout machinery.
The draw params carry the CLIPPED display width while the bit rows stay
MSB-aligned within the bitmap's own width, so a fringe narrower than
the bitmap (e.g. fringe-mode 4 with the 8px buffer-boundary angles)
rendered the bitmap's right half; the native backends draw the full
bitmap and clip, showing the left half. Export the bitmap's true width
from fringe.c (fringe_bitmap_width) and extract bits against it.
draw_bitmap in the gfx driver vtable gains the true-width parameter.
A redisplay pass runs several update cycles back-to-back (buffer window
plus echo area), and with display sync every present blocked on a
drawable: two blocking presents per keystroke halved typing throughput,
and the present itself paid for a second command buffer.
Presents now coalesce: one landing within ~8 ms of the previous commits
to the static texture and is flushed by a one-shot main-queue block
(or absorbed by the next cycle's present), and the composite pass is
encoded on the same command buffer as the cycle's draws. The policy's
flush presents through the same coalescing path.
Benchmarks (M1 Pro, font-locked xdisp.c, 120x45): machine-paced typing
goes from 30 to 106 chars/s and sustained scroll from 116 to 475
redisplays/s with vsync off, matching the stock backend's throughput
and total CPU; with vsync on (default) the same visible 60 fps now
costs ~35% less CPU than stock. README gains the numbers.
Pixel parity, GIF/video playback, the first-tab-switch deferral and the
echo-area tests are all unchanged.
With vsync on (default) presents wait for the panel refresh: redisplay
self-caps at the display rate, which is also a power win (the stock
backend happily renders hundreds of invisible frames per second under
sustained scroll). Disabling it trades that cap for NS-like latency on
machine-paced workloads.
The patched rif and terminal hooks are terminal-wide, but only frames
the user enabled the GPU backend on have driver data: tooltips, child
frames (posframe and friends) and frames created without mtl-enable
came up blank. Capture the original NS implementations before patching
(gfx_fallback) and delegate every policy entry point to them for frames
the driver is not ready on. Verified: tooltip, posframe-style child
frame and a second top-level frame all render identically to a pure NS
session while the main frame renders through Metal.
Also resolved while auditing the display feature matrix:
- Underline styles dots and dashes (FACE_UNDERLINE_DOTS/DASHES) were
drawn as a solid line; port ns_draw_dash ([segment, segment] pattern
anchored at the absolute x so it joins across glyph strings).
- :underline (:position N) larger than the descent crossed the glyphs:
NS computes the descent as unsigned, so the position underflows and
clamps to the row bottom; mirror that clamp.
- The internal border was cleared with the frame background; port
ns_clear_under_internal_border (internal-border/child-frame-border
face, face remapping, top/bottom margins).
- With cursor animations enabled the GPU cursor never blinked
(blink toggles visibility through erase_phys_cursor, which an overlay
never sees): the animation tick now mirrors the window's cursor_off_p
and hides the overlay cursor and trail during the off phase.
- Before the first cursor draw the overlay used a hardcoded pale tint;
fall back to the frame cursor color.
No regressions: baseline 141 px and all capture batteries unchanged.
Cursor-only motion takes redisplay's fast path: no render cycle opens,
and the CADisplayLink stops firing while Emacs idles, so with
animations enabled the moved cursor overlay was never presented (it
appeared frozen until the next input event) and rings/trails never
animated between events.
Three fixes: note_cursor composites immediately when no cycle is open;
the animation step is factored into tickWithDt: and exposed as
mtl-anim-tick, driven by a 30fps Lisp timer while animations are
enabled (the same mechanism that drives video playback); and burst
particles use the real cursor color instead of a hardcoded pale tint
that was invisible on light backgrounds.
Full GPL header blocks everywhere (the abbreviated form is gone), drop
the dead mtl_shaders.metal (shaders are embedded in mtlterm.m), remove
internal phase names and TODO.org references from comments, fix the
package URL in mtl.el to the emacs-gpu repository, and refresh stale
file titles.
Chosen as the default for tests and demos after trying all eight modes
side by side. Animations themselves remain opt-in
(g_mtl_animations_enabled / mtl-animations).
The redisplay-facing drawing policy -- glyph strings with backgrounds,
boxes, reliefs, underlines (incl. wave) and strike-through; composition
strings (static and shaped); images with slices and button reliefs;
cursors; fringe bitmaps; window borders and dividers; scroll blits; and
the render-cycle policy (clear-only present deferral, deferred immediate
presents, pending scroll-bar gutter clears, the minibuffer expose
substitute) -- moves to the platform-neutral gfxterm.c, written in pure C
against a small driver vtable (gfxdrv.h, ~25 ops: cycle control, fill,
clip, glyph atlas via struct font *, textures, masks, region copies,
relief/frame colors, a cursor-animation hint).
mtlterm.m shrinks by ~1000 lines to the Metal driver (pipelines, atlas
and CoreText rasterization, MtlFrameData, animator, video) plus the
mtl_gfx_driver vtable; the patched rif and terminal hooks now point at
the gfx_* policy functions.
OpenGL is NOT implemented, by design: glterm.c is a documented skeleton
of the vtable to fill in, and configure --with-gl is reserved (errors
out pointing at the integration point), so a future GL driver for
GNU/Linux and Windows plugs in under the already-pixel-perfect policy.
Verified zero regressions: baseline 141 px (0.055%), all nine capture
batteries byte-identical (text/disp/bars/bidi/wrap/img/fontlock/echo/
hscroll), GIF animation, inline video, dynamic text-scale, draw-stats
counters, and the clear-only deferral trace shows the same single
present on the first tab switch.
AVFoundation decodes into BGRA pixel buffers (AVPlayerItemVideoOutput)
that CVMetalTextureCache wraps as Metal textures with zero copies; the
compositor draws the newest frame as a textured quad over the static
texture on every present, scissor-clipped to the window interior so a
half-scrolled video does not bleed over the mode line.
Presents are driven by a 30fps Lisp timer (mtl-video-tick): Emacs's
event loop starves the CADisplayLink while idle (it stalled after two
ticks), and Lisp timers are what fire reliably; this also matches how
image-animate drives GIFs. The timer re-anchors the video to a buffer
marker each tick (mtl-video-move), so it follows scrolling and window
changes, and parks it off-screen when not visible.
Lisp API: mtl-video-open/close/pause/move/tick (mtlfns.m) and the
user-facing mtl-video-insert/mtl-video-stop in mtl.el, which insert a
pixel-sized placeholder space and track it.
MtlVideoPlayer goes through property setters everywhere: this file is
compiled without ARC, and raw ivar assignment autoreleased the AVPlayer
graph under us (crash: unrecognized selector on a reused dictionary).
Links AVFoundation/CoreMedia/CoreVideo (configure.ac libs_nsgui and
MTL_LIBS).
Switching to a tab whose buffer faces are not realized yet garbages the
frame: redisplay first runs an update cycle that only calls clear_frame,
and Metal presented that blank frame, flashing white for the ~30 ms the
follow-up cycle needs to realize faces and repaint. NS does not flash
because AppKit coalesces backing-store flushes within the event cycle.
Track per-cycle whether clear_frame ran and whether any glyph string was
drawn; a clear-only cycle commits to the static texture but defers the
present (needsPresent), which the follow-up content cycle or
flush_display then picks up. Also adds MTL_LOG_SEQ=1 tracing of the
begin/clear/draw/end/present sequence, which is what exposed the blank
present.
The IMAGE_GLYPH branch returned before any decoration: tab-bar and
tool-bar buttons (and :relief images) lost their relief, and boxed
image faces lost their box, leaving stale pixels in the box rows that
the inset background fill deliberately skips (seen as a white gap in
the tab-bar bottom edge under the new-tab button).
Port ns_draw_image_relief (button relief with tab/tool-bar margins),
draw the face box after the image like ns_draw_glyph_string, fill the
image background at full cell height (NS skips the box inset there),
honor the left box line and slice origin for the image position, and
factor the relief colors/edges into mtl_relief_colors/mtl_draw_relief.
Also give mouse-face runs box edges at their boundaries and stop
reading past first_glyph for image/composition strings in the box
code, both mirroring NS.
NS repaints uncovered regions through its drawRect: expose path; the
Metal overlay has none, so when a multi-line echo message shrank back,
stale pixels (e.g. a wrap-arrow shard in the echo fringe) survived.
Detect minibuffer pixel-height changes in update_begin and clear the
affected bottom strip inside the same update cycle, so the redraw and
the clear land in one present without flashing.
mtl_texture_for_image rasterized the NSImage at its natural size with no
transform, so :rotation came out with the wrong orientation and :scale
ignored the engine's display size. Mirror ns_dumpglyphs_image: rasterize
at the engine display size (img->width/height), concat the EmacsImage's
NSAffineTransform inside a flipped context (drawing with
respectFlipped:YES, the two flips cancel for the memory layout) and
honor the smoothing flag for interpolation. The texture cache
re-rasterizes when the display size changes.
Image glyph strings now also sample only their slice subrect of the
texture (mtl_draw_image_texture_uv), fixing insert-sliced-image, which
previously squeezed the whole image into every slice.
The NS backend clips every glyph string draw to
get_glyph_string_clip_rect via ns_focus; Metal drew unclipped. The most
visible casualty: a filled-box cursor on a tall row (e.g. a line with an
inline image) painted the cursor background over the full row height
(60px bar) instead of the character cell, because draw_phys_cursor_glyph
relies on the clip rect, narrowed to the physical cursor box, to bound
the DRAW_CURSOR background fill.
Apply the same rect as an MTLScissorRect around each glyph string draw
(applyClipRect:/clearClipRect on the static-texture encoder, mapped to
physical pixels and clamped), in both the in-cycle and immediate paths.
Also bounds overhang bleed into neighboring windows for free.
Verified: cursor on an image row now renders as a character-sized box
with the glyph inverted, identical to NS; baseline AE unchanged (0.068%).
Wrap mtl_draw_fringe_bitmap and mtl_clear_frame in the same
self-contained immediate frame used by draw_glyph_string (begin, draw,
commit with deferred present) so they are not silently dropped when
invoked outside update_begin/end (clear_garbaged_frames runs before the
update cycle; fringe updates can arrive immediately too). Also make the
fringe clear run even when the bitmap is empty (a clear-only call).
Note: a tiny stale shard can still remain in the echo-area fringe corner
after a multi-line message shrinks the minibuffer back. Root cause is
that NS repaints uncovered regions through its expose path (drawRect:
re-renders from the glyph matrices), which the Metal layer does not have
yet; tracked in TODO.org.
The glyph atlas is R8 grayscale coverage, so emoji rendered as dark
monochrome silhouettes (and emoji compositions like heart+VS16 drew
nothing before the composite fix). Detect color fonts via
kCTFontTraitColorGlyphs and rasterize their glyphs into small BGRA
textures with CTFontDrawGlyphs (which renders sbix color), drawn through
the image pipeline. Cached per (font, glyph) with the same metric
conventions as the grayscale atlas (physical-resolution baking, bearing_y
= bh - raster_oy, logical advance); the cache is cleared when the backing
scale changes. Wired into both the plain char loop and composition runs.
Verified: smileys, party popper, red heart (ZWJ/VS16 composition) and
thumbs-up now render in color, matching NS.
COMPOSITE_GLYPH had no branch in mtl_draw_glyph_string, so combining
sequences (e + COMBINING ACUTE, etc.), ligatures and shaped scripts drew
nothing (blank cells). Port ns_draw_composite_glyph_string_foreground to
the Metal atlas: char2b[] holds glyph IDs indexed by the composition /
gstring index and s->font is the composition's font; static compositions
draw each glyph at its table offset, automatic ones walk the LGSTRING
applying XOFF/YOFF/WADJUST adjustments, with sequential runs advanced by
the natural font advance.
Verified: "e+acute a+grave n+tilde o+diaeresis" now renders identically
to NS (previously empty).
The first big redraw with new content (e.g. the first tab switch)
rasterized a burst of new glyphs into the atlas, which showed as a
one-time whole-frame blink; subsequent switches were fluid because the
cache was hot. Pre-rasterize printable ASCII for the frame's default
face in mtl-enable-for-frame so that cost is paid up front.
The wave fell back to a straight line, and the fallback branch was dead
anyway: FACE_UNDERLINE_WAVE is above FACE_UNDERLINE_SINGLE in the enum,
so the >= SINGLE branch swallowed it. Check wave first (like NS) and
draw the ns_draw_underwave zigzag: wave height 3, period 4, triangle
pattern 0,1,2,1 at ybase..ybase+2, one 1px cell per column, phase
anchored to the absolute column so the wave continues seamlessly across
adjacent glyph strings. Verified visually against NS (red and default).
ns_setup_relief_colors derives the relief edges via NSColor
highlightWithLevel:/shadowWithLevel:, which are appearance-dynamic (in
dark mode the system highlight is not a blend toward pure white), so the
manual mtl_shade_color blend did not match what NS renders: the
mode-line top edge came out 211 vs NS's 177. Use the same NSColor APIs,
falling back to the manual shade if they return nil.
Mode-line relief now matches NS exactly; sparse-baseline AE drops from
0.33% to 0.068%.
bearing_y used bh - 1 - raster_oy, which placed every glyph one pixel
lower than the NS backend across the whole frame. The bitmap row at
top-down index r covers CG y in [bh-1-r, bh-r), so the baseline at CG
y = raster_oy sits bh - raster_oy rows below the top edge, without the
extra -1.
Found by cross-correlating wrapped-text captures: shifting Metal text up
one pixel dropped the per-row mean error from ~40 to ~3 gray levels.
This was the dominant share of the residual diff attributed to
antialiasing: AE vs NS drops from 0.88% to 0.33% on the sparse baseline
and from 3.32% to 0.56% on dense text.
Replace the empty stub with a real implementation mirroring
ns_shift_glyphs_for_insert: move the rest of the line right by SHIFT
pixels inside the static texture, bouncing through the scratch texture
(overlapping src/dst) with the same backing-scale mapping and encoder
suspend/reopen as scrollRunFrom:.
Note: like the X backend documents (xterm.c, emacs-devel 2015-05), this
RIF hook is effectively never called on GUI frames, so this is for
completeness/correctness rather than a visible fix.
The NS (CoreGraphics) backend renders on-screen text slightly heavier
(more ink at every coverage level) due to its coverage gamma / stem
darkening, leaving plain linear-coverage Metal text a touch lighter.
Apply coverage = pow(coverage, 0.82) in the glyph fragment shader to lift
coverage uniformly and match NS's weight.
Dense-text ink ratios vs NS moved from 0.88-0.92 to 0.96-0.99, mean gray
247.6 -> 247.25 (NS 246.95), and AE improved 3.40% -> 3.32%; the "Hola
Mundo" baseline AE stays at 0.88% and is visually identical (no
over-bolding).
Match ns_maybe_dumpglyphs_background: fill the background from
s->y + box_line_width with height s->height - 2*box_line_width instead of
the full cell height. For a boxed face (e.g. the selected tab-bar tab,
or a boxed mode-line) this leaves the box edge rows untouched, so the
relief drawn afterwards is not first overwritten with the background and
then redrawn on top. For unboxed faces box_line_width is 0, so it is
identical to the previous full-height fill (AE vs NS unchanged at 0.88%).
The real draw path used Nord theme colors (0x2E3440 bg, 0xECEFF4 fg) as
fallbacks for when the frame or face is null. Those paths are effectively
unreachable, but baking a dark theme into a backend that mirrors the NS
default light theme is wrong if ever hit. Use neutral white bg / black fg
instead. The MtlView test window and the offscreen PNG render keep their
Nord palette (out of the real redisplay path), per TODO F4.
Mouse-face highlight (note_mouse_highlight -> show_mouse_face) draws
outside the update_begin/end cycle. Presenting on every immediate
draw_glyph_string flickered: one mouse move runs clear_mouse_face +
show_mouse_face, several draws, each flashing the intermediate
(un-highlighted) state.
Each immediate draw now commits to the static texture but defers the
on-screen present (endFramePresent:NO sets needsPresent); flush_display
and update_begin present once, showing the whole clear+redraw sequence
in a single composite. This keeps the pipeline clean (no encoder left
open between draws) while removing the flicker.
Verified: hover highlight still applies (gamma 168 vs 179 inactive),
AE vs NS unchanged at 0.88%.
Strike-through used s->ybase - s->height/3, which sits too high and
mis-centers when the row is taller than the string. Match NS: center a
1px line on the first glyph's body (glyph_y + (glyph_height-1)/2). Add
the overline (1px at the top of the string), which was missing.
Verified vs NS: both lines now land at the same place relative to the
glyphs.
The underline was drawn at s->ybase + s->underline_position, but
underline_position/underline_thickness are filled in by each backend
(the NS backend computes them in ns_draw_text_decoration) and were left
at 0 here, so the line landed on the baseline and cut through the bottom
of the glyphs.
Add mtl_underline_metrics, mirroring the NS computation: position from
font->underline_position (or descent/2, or the descent line per the
face), thickness from the font (min 1), clamped to stay inside the cell,
and matching a previous underlined run for continuity. Handle single,
double (second line above), and colored underlines; wave falls back to a
straight line at the same position for now.
Verified vs NS: the underline is now a full-width bar just below the
glyph body instead of through it; AE unchanged at ~0.88%.
note_mouse_highlight -> show_mouse_face draws the mouse-face highlight by
calling draw_glyph_string directly, outside the update_begin/end cycle, so
no Metal render encoder was active and mtl_draw_glyph_string returned early:
the highlight never appeared (the NS backend draws immediately via lockFocus).
Split the drawing body into mtl_draw_glyph_string_impl and make the public
mtl_draw_glyph_string open a self-contained frame (beginFrame uses LOAD, so
the static texture is preserved), draw, then endFrame to present, whenever it
is called with no encoder active. The same path also restores normal text
when the mouse leaves (clear_mouse_face).
Verified by hovering a mouse-face region with cliclick: the region row
background switches to the highlight face (209,210,210) vs (244,244,244) for
a normal row, with +6 draw_glyph_string calls and no-fd == 0; AE vs NS
unchanged at ~0.88%.
