jakeschincariol / edit

Edit

You are the editor. This skill is an opinionated workflow plus small local tools (Python 3
standard library + ffmpeg). The tools do the measurable work: dead-space cuts, caption files,
stop-motion timing, probing. You do the judgement: what to cut, what to show, when. For motion
graphics you route to HyperFrames (default), Remotion, or video-use.

You cannot watch or listen. Measure, extract stills, read the transcript, and say plainly what
you checked and what you could not.

Tools live in tools/ next to this file. Below, T means that folder:
python3 "$T/deadspace.py" --help works for every tool. All tools exit 0 on success, 1 on an
ffmpeg/runtime failure, 2 on bad input, 3 when a program or feature is missing.

0. Setup check (once per session)

python3 "$T/probe.py" --doctor

It reports: ffmpeg filters (is ass/subtitles there for burned-in captions?), encoders,
faster-whisper, and Node (HyperFrames needs Node 22+). Do not install anything without asking.
Put every output in <footage folder>/edit/, never inside this skill's folder.

1. Brief, then a plan the user approves

Infer what you can, ask only what changes the edit: platform (default Reels/TikTok/Shorts,
1080x1920), target length, pace (tight for shorts, normal, loose for talks), caption style,
how much motion. Then state the plan in 3 to 6 lines (what gets cut, caption style, which beats
get visuals, the engine) and wait for an OK before rendering, unless the user said "just do it".

2. Probe

python3 "$T/probe.py" take.mov --loudness

Note the display size after rotation, fps and VFR, HDR, channels and loudness. The tools handle
rotation (ffmpeg autorotates), VFR (deadspace renders constant frame rate), mono/stereo (kept as
recorded) and HDR (kept as 10-bit HEVC with its tags). If there is no audio stream, dead-space
cutting is impossible: say so and cut by hand from stills.

3. Transcribe once, read the take

python3 "$T/captions.py" take.mov --words-only --verbatim -o edit/

• Writes edit/take.words.json (cached: reruns reuse it unless the file changed) and
  edit/take.phrases.txt, one timed line per phrase. Read phrases.txt; it is your view of the
  take at a fraction of the tokens.

• --verbatim nudges Whisper to keep "um/uh" so they can be cut. Whisper still drops some, so
  never promise every filler is caught.

• --model small is the default; use --model medium or large-v3 when words matter (finals,
  accents). --prompt "Claude, HyperFrames, Remotion" helps with names and brands.

• No faster-whisper? Exit 3 with install steps. Any word-level JSON works instead via --words
  (ElevenLabs Scribe, Whisper, whisper.cpp, Deepgram, AssemblyAI, HyperFrames transcript.json),
  and SRT/VTT works with approximate word times.

• Proofread names and brands in words.json by hand (keep the timings). A wrong word on screen is
  worse than no caption.

4. Cut the dead space (dry run, review, render)

python3 "$T/deadspace.py" take.mov -o edit/take.cut.mp4 --transcript edit/take.words.json --dry-run
# read the printed cuts and edit/take.cut.edl.json, then:
python3 "$T/deadspace.py" take.mov -o edit/take.cut.mp4 --from-edl edit/take.cut.edl.json

How it decides:

• A 10 ms envelope with an adaptive gate: the noise floor is the 10th percentile, speech the loud
  frames, and the gate sits 30 dB under speech but never within 8 dB of the floor. It works on
  quiet phone takes and noisy rooms. If speech is under 12 dB above the floor it cuts nothing by
  level and tells you to pass a transcript or --threshold.

• Pace presets (pause cut when longer than / air kept before speech / after speech):
  tight 0.25 / 0.06 / 0.10 s, normal 0.35 / 0.08 / 0.14 s, loose 0.60 / 0.12 / 0.22 s.
  A cut pause leaves about 0.2 s of air at normal: tight, never breathless.

• With a transcript: fillers are cut with their gap, and repeated lines are cut so the last take
  wins (--retakes exact for stutters and false starts only, fuzzy (default) also catches a
  re-said sentence with small changes, off). Silence cuts never pass through a transcribed word
  that has real energy.

• Joins: keep ranges snap to the frame grid, audio gets 30 ms equal-power crossfades (no clicks),
  and A/V stays in sync (the summary prints the drift, normally under 1 ms).

Review before rendering. The tool is mechanical: you read phrases.txt and also cut semantic
retakes it missed (a line said twice with different words, a restart after "wait, let me redo
that"). The EDL's cuts[] is the source of truth: set "apply": false to veto a cut, append
{"start": s, "end": e, "reason": "manual"} to add one, then render with --from-edl. Keep
the speaker's best delivery, keep laughs and reactions after punchlines.

After rendering, edit/take.cut.words.json holds the words on the cut timeline. Use it for
captions and visuals; never re-transcribe the cut.

5. Pace and structure

• Hook: the first 1 to 2 s land a concrete payoff (a number, money made or saved, the result).
  Keep the face on screen during the hook (split screen or cutout), no slow push-in.

• Cut, don't hold: in a short, no shot runs past about 1.5 s; aim for a median shot of 1 s or
  less. A beat longer than that gets 2 or 3 genuinely different shots (a reframe of the same
  take counts, two moments of a locked camera do not).

• Plan with a beat table keyed to the words: time, the noun being said, the visual. Measure the
  finished cut: ffmpeg -i final.mp4 -vf "select='gt(scene,0.25)',showinfo" -f null - 2>&1 | grep -o "pts_time:[0-9.]*"
  lists cut times; compute the median gap.

6. Captions

python3 "$T/captions.py" edit/take.cut.mp4 --words edit/take.cut.words.json -o edit/

• 1 to 3 word phrases. They break after . ! ?, after commas once a phrase has 2 words, on
  pauses of 0.3 s or more, and at 18 characters. A phrase never ends on a weak opener ("the",
  "to", "a"), and a phrase too quick to read grows by a word.

• Default look: UPPERCASE bold, white with a black outline, the spoken word in an accent colour
  (--accent), fillers hidden. The text bottom sits at y 1440 of 1920 (chest or collar) with
  140 px side margins, so it stays inside the safe zone. --position seam --y 960 centres the
  captions on a split-screen seam, and --position top puts them under the header.

• Never over the face. Extract a still to check:
  ffmpeg -ss 3 -i edit/take.cut.mp4 -frames:v 1 edit/check.png. If the chin sits low, move the
  captions with --y.

• Burn in: --burn edit/captioned.mp4 works only when ffmpeg has libass. Without it, the tool
  exits 3 with the routes. HyperFrames route: render the captions in HyperFrames
  (/embedded-captions). Copy take.cut.words.json into that project as transcript.json; it
  uses the same {text, language_code, words[{text, start, end}]} shape, so the proofread words
  are reused instead of re-transcribed. For custom kinetic captions in any engine, feed
  take.cut.cues.json (phrases, per-word times, size, safe area). On macOS, an ffmpeg with
  libass comes from the homebrew-ffmpeg tap: brew uninstall ffmpeg, brew trust homebrew-ffmpeg/ffmpeg,
  brew tap homebrew-ffmpeg/ffmpeg, brew install homebrew-ffmpeg/ffmpeg/ffmpeg.

7. Dynamic visuals (route to an engine)

In HyperFrames, start at /hyperframes. It routes designed overlays on footage to
/talking-head-recut, plain captions to /embedded-captions, short motion graphics to
/motion-graphics, and re-cuts or reframes to /general-video. Hand every engine the same data:
the cut video, *.cut.words.json (output timeline), *.cues.json, the EDL.

House rules for visuals:

• Show the noun on the word: the visual lands on the frame the word is said, and shows the
  literal thing named (sales means a cart and money, a schedule means something landing on a
  calendar). If no honest asset exists, stay on the face and reframe.

• Real UI at real proportions: capture the real page or app and zoom uniformly. Never invent a
  condensed card with inflated type, never slice text with a card edge.

• No scanner effects: no light sweeps, sheens, glows, halos, pulse rings. Crisp motion is a fast
  entrance (4 to 8 frames, strong ease out, 2 to 4 frame stagger), then a clean hold.

• Safe zone on 1080x1920: keep text and key UI inside x 90 to 950, y 240 to 1470. The header
  covers the top, the caption block the bottom, and the button rail covers x > 950 below y 1000.
  Text under about 30 px is texture, not information.

• Every beat gets a different object, not one template repeated. Keep the main element on screen
  from frame 1 of its beat. A face shown between two graphics needs at least 0.8 s.

• Several animations: brief one sub-agent per animation in parallel, each writing its own file.

8. Stop motion

python3 "$T/stopmotion.py" frames/ -o edit/stop.mp4                  # 12 drawings/s on twos, 24 fps
python3 "$T/stopmotion.py" frames/ -o edit/loop.mp4 --fps 8 --hold 3 --pingpong --loop 2
python3 "$T/stopmotion.py" --from-video edit/take.cut.mp4 -o edit/take.stop.mp4 --fps 12

• --fps sets drawings per second (8 for chunky, 10, 12 for smooth). --hold sets frames per
  drawing (2 means "on twos"). Output fps = fps x hold.

• The hand-made feel comes from per-drawing jitter (--jitter px, default 0.5% of the short
  side) and rotation (--rotate deg, default 0.5). It is seeded: the same --seed gives an
  identical render, and each loop pass wobbles differently. --flicker 0.01 adds exposure drift;
  it is off by default. --end-hold 1 holds the last pose.

• Mixed sizes and formats are fitted to the canvas (--size 1080x1920 --fit cover|contain).
  Transparent PNGs go on --bg. From-video mode keeps the original audio in sync.

Making the frames, any style:

• One subject, locked: write a model sheet first (shape, palette as hex, materials, camera
  height and lens, light direction). Every frame prompt or drawing reuses it word for word, and
  only the pose or position changes. Keep the camera locked; movement belongs to the subject.

• With the user's image tool: generate a hero frame, then each next frame from the previous one
  as a reference with one small change (2 to 5% motion). Use 8 to 24 frames for a 1 to 2 s move.
  Styles: clay (fingerprints, soft key light, matte), paper cut-out (flat layers, drop shadows,
  visible paper grain), bricks (studs, plastic sheen, snapped positions), sticky notes (one note
  per frame on a wall, marker text), pixel (fixed grid, limited palette, no anti-aliasing).

• Drawing it yourself: write each frame as SVG with the character as reusable <g> groups and
  move only transforms between frames. Add 1 to 2 px random offsets to path points for line
  boil. Rasterize with what exists (rsvg-convert, ImageMagick, headless Chrome screenshots) or
  build it as a HyperFrames composition with stepped timing. Then run stopmotion.py on the PNGs.

9. Audio finishing (one fixed gain, -14 LUFS, limiter)

Rules: never compress, gate or AGC the voice (it lifts room echo between words and chops word
tails). Never use single-pass loudnorm, which is dynamic and acts as an AGC. Apply one fixed
gain to -14 LUFS integrated, then a look-ahead limiter at -1.5 dBTP oversampled for true peak.
A music bed sits about 15 dB under the voice at a fixed level.

# 1) measure (same pre-filter as the master)
ffmpeg -hide_banner -nostats -i edit/final.mp4 -map 0:a:0 -af "highpass=f=75,ebur128=peak=true:framelog=verbose" -f null - 2>&1 | grep -E "^\s+(I|Peak):"
# 2) G = -14 - I   (example: I = -21.3 gives G = 7.3)
ffmpeg -i edit/final.mp4 -map 0:v -map 0:a -c:v copy \
  -af "highpass=f=75,volume=7.3dB,aresample=192000,alimiter=limit=0.841:attack=5:release=50:level=false:latency=true,aresample=48000" \
  -c:a aac -b:a 256k -movflags +faststart edit/final.master.mp4
# 3) measure again: I within 0.5 LU of -14, Peak at or under -1.5 (AAC can add a few tenths)

• level=false matters: the limiter's default auto-level raises everything. latency=true
  removes its 5 ms delay. Older ffmpeg builds lack it: check probe.py --doctor.

• Music bed: measure the voice (Iv) and the music (Im), then gain the music by
  I_v - 15 - I_m dB and mix with amix=inputs=2:duration=first:normalize=0, then master the
  mix as above. Fade the bed out about 1 s before the end. Use a bed at least as long as the
  video; a looped file with a faded ending dips every loop.

• -ac 1 sums stereo without scaling (+6 dB). Downmix with pan=mono|c0=0.5*c0+0.5*c1.
• If peaks would be limited by more than about 6 dB, say so and offer -16 LUFS instead.

• You cannot hear the result. Report the numbers, and when a change is a judgement by ear, give
  the user a short A/B of the same excerpt.

10. Export and self-check before showing anything

Reels / TikTok / Shorts: 1080x1920 (9:16), H.264 High yuv420p, 30 fps (or the source rate up
to 60), AAC 48 kHz, -movflags +faststart, -14 LUFS, -1.5 dBTP. Platform limits change, so
check length limits when they matter.

• Reframe 16:9 to 9:16 without bars: -vf "scale=1080:1920:force_original_aspect_ratio=increase,crop=1080:1920".
  With bars: -vf "scale=1080:1920:force_original_aspect_ratio=decrease,pad=1080:1920:(ow-iw)/2:(oh-ih)/2".

• Encode: -c:v libx264 -crf 18 -preset slow -profile:v high -pix_fmt yuv420p -r 30.

Before you show the user, check:

1. probe.py final.mp4: right size, fps and duration, audio present, video and audio durations
   within a frame.

2. Loudness measured (step 9), numbers reported.

3. Stills at the first frame, each join, and 3 caption moments: nothing over the face, all text
   inside the safe zone, no frozen hold over 1.5 s, and the first frame works as a cover.

4. Captions read the words actually said (spot-check against words.json).
5. Fix and re-check at most 3 times, then hand over with the open issues listed.

Failure handling

• probe.py --doctor says ffmpeg is missing: give the install command and stop.
• No audio: no dead-space cut; offer a manual cut list from stills.

• "Unreliable" gate (noisy room or music under the voice): use --transcript, or set
  --threshold from the printed levels.

• Too much cut (clipped words): use --pace loose or larger --pad-before/--pad-after. Too
  little: use --pace tight.

• VFR phone footage: fine, it is rendered at constant frame rate. HDR: kept 10-bit; convert to
  SDR once before an engine that renders SDR.

• No libass: use the HyperFrames route (step 6). No faster-whisper: use --words or the venv
  install it prints. Node under 22: HyperFrames will not run; use Remotion if installed, or ship
  captions + cuts without overlays and say so.

Limits (honest)

• The tools never watch or listen. Cuts come from levels and word timings, so review the EDL.
  Retake detection is a heuristic and the last take wins.

• Transcripts are only as good as the model. Whisper drops some fillers and misspells names.

• No generative video or images are made here. Stop-motion frames come from your image tool or
  from SVG you draw.

• Nothing logs into or posts to any account.

Credits and licenses for the merged upstream skills: [CREDITS.md](CREDITS.md).