termio-sh / app-screenshot-debug

Screenshot-driven UI debugging (AppleScript + screencapture)

The loop for seeing a termio UI change instead of guessing from code:

(rebuild if code changed)  →  drive app into the target state  →  capture the window  →  Read the PNG  →  judge / iterate

You cannot evaluate spacing, alignment, overlap, ghosting, or "this looks ugly"
from source. Get a real screenshot, Read it, then decide.

Prerequisites

• The app must be running and built. If you just changed code, run the
  macos-rebuild-dev skill first. The process name for System Events is
  termio and the main window's title is termio.

• These calls need macOS Accessibility + Screen Recording permission for
  the terminal / Claude Code host. If System Events errors with -25211 or the
  capture is black, that permission is missing — tell the user to grant it in
  System Settings → Privacy & Security.

• termio is a single-window app (sidebar + terminal panes); there is **no Settings
  window**. Everything you need is in window 1.

The two helper scripts (in scripts/)

Run them with bash; they take the app name from $TERMIO_APP_NAME (default
termio).

1. capture-window.sh [out.png] — screenshot just the frontmost termio
   window (-o no shadow, -x silent), reading its live bounds first. Prints the
   window origin + size. Captures the region 1:1 in points, so **a point you read
   off the resulting image is window-relative**: image (dx,dy) == screen
   (originX+dx, originY+dy).

2. click-in-window.sh <dx> <dy> — click at a window-relative point: it
   re-reads the live window origin and adds your offset. Use the exact (dx,dy) you
   measured on a capture-window.sh image — this survives the window moving or
   reopening on a different display.

Canonical recipe

SK=skills/app-screenshot-debug/scripts

osascript -e 'tell application "System Events" to tell process "termio" to set frontmost to true'
bash $SK/capture-window.sh /tmp/shot.png
# → Read /tmp/shot.png with the Read tool, find the control you want at (dx,dy)
bash $SK/click-in-window.sh 120 180   # e.g. click a sidebar project row
bash $SK/capture-window.sh /tmp/shot2.png   # Read again to see the result

Then Read each PNG to analyze. Capture before and after a change for an
honest before/after.

Driving the UI — patterns that actually work

• Focus the window: tell application "System Events" to tell process "termio" to set frontmost to true
  (or tell application "termio" to activate).

• Click a control: prefer click-in-window.sh with coords read off the
  screenshot. click at {x,y} (global points) conveniently **returns the AX
  element path it hit** — use that to confirm you hit the right thing.

• Get window bounds live: get position of window 1 / get size of window 1.

• Type into the focused terminal: keystroke "ls\n" after the window is
  frontmost — useful to put the terminal surface into a known state before
  capturing.

Gotchas (learned the hard way)

• Never hardcode window coordinates across calls. The window can reopen at a
  different origin or on another display. Always re-read bounds (the scripts do) —
  that's the whole reason click-in-window.sh exists.

• screencapture -R x,y,w,h uses the same global point coords as System
  Events, mapping points→pixels 1:1 for the width you ask for, so window-relative
  math is trivial: dx = screenX - originX.

• Always -o (no shadow) and -x (silent) so the crop is exact and there's
  no shutter sound.

• Add a sleep (~0.8–1.2s) after navigation/clicks before capturing so the
  animation settles — otherwise you screenshot a mid-transition frame.

• The terminal surface is a libghostty-rendered view, mostly opaque to AX — drive
  it by focusing + keystrokes, and judge it from the screenshot, not the AX
  tree.

• Save shots under /tmp/<task>/…png and Read them; don't leave them in the repo.