# Scenario Kinetic Music Video

> Use when turning a song into a kinetic-typography or motion-design music video, lyric video, visu…

Use when turning a song into a kinetic-typography or motion-design music video, lyric video, visualizer or K-pop style MV with Scenario, landscape or vertical 9:16 for TikTok, Reels and Shorts: a generated subject sheet, style frames and footage, plus a deterministic JavaScript/three.js engine for lyric type, mattes and tracked graphics timed to sung onsets. Also for revising, recutting or cutting teasers from such a video.

- Page: https://ss33.ai/skills/scenario-labs/scenario-kinetic-music-video-9480993269
- Publisher: scenario-labs
- Category: [Creative & Media](https://ss33.ai/skills/category/creative-and-media-0j3x01) › [Video](https://ss33.ai/skills/category/creative-and-media-0j3x01/video-yerjjv)
- License: MIT
- Source: https://github.com/scenario-labs/skills/blob/ccc6c5c623a4f96423df0d157b8ef174f14edfd8/skills/scenario-kinetic-music-video/SKILL.md

## Skill

SKILL.md: https://ss33.ai/api/v1/ss33/download/scenario-labs/scenario-kinetic-music-video-9480993269

# Kinetic Music Video

A song goes in. A 1080p60 music video comes out, landscape for YouTube and X or vertical for TikTok,
Reels and Shorts: a motion designer's showreel cut to the track. Kinetic type, 2D and 3D graphics
and transitions land on the beats and sung syllables. Generated footage of the subject is woven
through it where the concept wants a performer. The approved master is the only soundtrack. Preserve
the original; an authorized audio edit produces a new, versioned master.

The work splits by what each tool does best:

- **Scenario** supplies what code can't: a consistent subject with real physics, choreography, hair
  and lip-sync, in whatever art style each section needs.
- **The JS engine** supplies what video models can't: exact typography, frame-exact timing to onsets
  and beats, infographics, compositing around the subject, and endless re-renders at zero generation
  cost.
- **Sub-agents** give each section a dedicated designer. A shared style bible and a verification
  loop keep them coherent.

Connection and the core generation loop: see the `scenario` skill in this repo. For a song-to-video
cut without the engine, see `scenario-seedance-music-video`. If a sibling skill named here is
missing from your available skills, ask the user to install it
(`npx skills add scenario-labs/skills --skill <name>`); unattended, proceed from tool schemas and
flag the gap.

Read `references/lessons.md` before starting. Every item in it cost real time or money once.
`references/motion_library.md` is the design menu.

## Before you start (ask once, then run)

- **Workspace:** the Scenario team and project (`teams_list`; ask if there is more than one).
- **Subject:**
  - Who or what performs: a character from reference images, a mascot, a product, the artist's own
    likeness (their own photos and consent only), or nobody (a pure motion-graphics or lyric video).
  - Footage share: performer-led, a mix (often about 60/40), or graphics only.
- **Look:**
  - Ask for references: images, videos or links for the mood, the motion and the **color**. Vague
    color asks without references drift to safe palettes.
  - Decide the 2-4 looks and which sections own which.
  - Ask what must never appear.
  - Defaults: generated footage shown clean (no filters on it), graphics layered on and around it, a
    fast and upbeat showreel energy.
- **Brand:** any logo, wordmark or credit. Use official files in their official colors, only where
  the director wants them.
- **Budget:** lean (6-10 video generations), moderate (12-20) or big. Price each paid step with
  `dry_run` (one run per distinct payload, then total) before spending. Relative anchors at 720p: a
  6 s clip costs about twenty style frames, a 10 s clip about thirty-five, and a full mixed video
  about five hundred. A 1080p clip measured about 2.5 times its 720p price, so settle the footage
  resolution before quoting.
- **Format:** one canvas per project, asked once: portrait 9:16 for TikTok, Reels or Shorts,
  landscape 16:9 for YouTube or X (unattended with no destination: landscape). It is `CANVAS` in
  `engine/timeline.js`, and style frames and footage are generated at that ratio, never cropped from
  the other. On portrait, check the destination's current length cap and plan the teaser as the
  platform cut when the song runs longer, or when you cannot check. Both ratios only on request: the
  code re-renders free with `--canvas` and most shots are windowed, but each shot that must fill the
  second frame (hook, lip-sync close-ups, finale) is paid again (`references/pipeline.md` §9). X
  caps standard accounts at 2:20, so plan a teaser.
- **Lyrics:** use them if supplied (they beat any transcript). Otherwise transcribe and flag the
  uncertain words at the end.

With no one around to answer, choose sensible defaults for the creative choices and write them into
TREATMENT.md. Do not default the money: with no stated budget or no explicit team and project, stop
before the first paid step and say what is missing.

## Workflow

### 1. Scaffold and analyze the song

Run `bash <skill>/scripts/scaffold.sh <song>` inside a new project folder, then follow
`references/pipeline.md` §2:

- `audio_analysis.py` writes stems and `engine/data/audio.json`: beats, downbeats, kick/snare/vocal
  onsets, and 60 fps envelopes. **Measure the tempo**; labels lie (a "130 BPM" song was 98 with
  double-time drums).
- `lyrics_align.py transcribe`, then hand-correct `analysis/lyrics.txt`, then
  `lyrics_align.py align` writes `engine/data/lyrics.json` with word onsets.
- Check timing by eye with `plot_lyrics.py`. Audition when playback is available; a spectrogram
  locates onsets but cannot prove pronunciation or musical continuity.

### 1a. Edit the soundtrack when requested

For pronunciation repairs, lyric changes, section replacements or structural cuts, read
[references/audio_editing.md](references/audio_editing.md). Use `scenario-ace-step` for its
Cover/Repaint contracts. Keep the current cut and master until a candidate is selected; preserve
audio outside the agreed window and rebuild affected timing data after applying it. A picture-only
revision reuses the approved soundtrack.

### 2. Research (parallel, background)

Launch two sub-agents with web access while you work on other things:

- **Culture:** for the song's subject, find recognizable events, memes and visual tropes, each with
  a verified fact, its iconic visual form, an insert idea and a short on-screen string.
- **Craft:** study the user's motion references (download them, contact-sheet them, read the
  frames), plus genre direction techniques. Extract timing numbers, color hex values and
  implementable JS moves.

Save the condensed results as `analysis/research_*.md`. Scene agents read them alongside
`references/motion_library.md`.

### 3. Concept, style bible and treatment

- **Give the video a spine.** The strongest concepts make the edit itself carry the song's meaning:
  a device that progresses through the whole video, for example a counter, clock or countdown that
  moves with the song while the cut rate or the looks escalate with it. The spine usually lives in
  the HUD counters.
- **STYLE.md** (the law):
  - a one-line concept and the spine
  - the footage rules (clean plate and the back/mid/front layers, or another policy)
  - the looks, each with ground, ink, accent, rendering rule and matching footage style
  - the palette with roles
  - 4-5 type roles with named fonts
  - lyric display rules
  - a motion grammar mapping beats to effects, with the cut density per section
  - the subject (identity, design details usable as graphic motifs)
  - brand rules, references and bans
- **TREATMENT.md:**
  - song facts and the energy map
  - the arc
  - the **hook for the first 3 seconds** (the strongest idea in the video, and it must work as a
    muted autoplay)
  - one row per section: time range on downbeats, lyrics, look, must-have ideas
  - a footage map (filled in as clips land)
- **PLAN.md:** a one-page version for the director to approve before you spend on footage
  (unattended, this is the stop point if the budget is unstated).

### 4. Subject, style frames and footage on Scenario

Skip this for a graphics-only video, apart from any stills or 3D props. Otherwise follow
`references/pipeline.md` §3-5:

- **Subject sheet:** a turnaround, expressions and callouts from the user's references. Look at it,
  then crop identity references from it.
- **Style frames:** one still per shot at the canvas ratio (an image model that takes reference
  images), in its section's look. Compose with negative space for type (on portrait, the face and
  the type space inside the safe band), on a set that composites (black void with rim light, flat
  color cyc, or white high-key). No text. In a lip-sync frame nothing covers the mouth (no mic, hand
  or prop): a covered mouth fails the sync check and is paid again. Contact-sheet them and get the
  look approved cheaply (unattended: proceed only inside a stated budget).
- **Footage routes:**
  - Image-to-video from the frame for actions.
  - A video model with reference images and reference audio, given a **synthesized beat track** for
    dances that lock to the song, then beat-warp.
  - **Audio-to-video** from a frame plus a vocal-stem slice for lip-sync. The alternative is
    reference-audio video plus a lip-sync correction model.
  - Code instead of generation for networks, charts, UI and particles.
- Never send the song to a video model. Turn the model's native audio off when its schema offers it
  (the render takes sound only from the master either way), and launch in waves within the team's
  concurrency ceiling (`jobs_wait` re-called with `pending_job_ids`, never `job_get`). Relaunch only
  a launch refused with 429 `parallel-custom-jobs`: it created no job, and dropping it leaves a hole
  in the cut.
- **Prep** every clip with `tools/prep_clip.sh`: frames, a subject matte (Apple Vision, with
  Scenario background removal as the fallback; see `references/pipeline.md` §5) and tracking
  (MediaPipe pose plus the matte bbox). Inspect the mattes. Write each clip's content, key moments,
  slot and matte quality into the footage map.

### 5. Engine and scene agents

- **Smoke-test first:**
  - Set `PAL` in `core.js`, and in `engine/timeline.js` set `CANVAS`, the sections on downbeats,
    `HUD` (title, mark, section codes, spine counters) and `POST` (neutral by default).
  - Write a tiny `_smoke.js`: plate → type behind the subject → matte → tracked label.
  - Render one still and a 5 s clip, then confirm audio and flashes line up with `av_sync_check.py`.
- **Brief the agents.** Fill in `AGENTS_BRIEF.md` from the template, then launch one background
  agent per section in a single message. Each gets its file, time range, lyrics, must-have ideas and
  clips, and reads AGENTS_BRIEF, STYLE, TREATMENT, ENGINE_API and the research first. Log the agent
  ids. For a teaser or anything under about 30 s, the lead may write the scenes itself and skip the
  research agents, PLAN.md and the per-section agents; keep the smoke test.
- **Deliver clips as they land.** Message the agent that needs each one ("clip ready: content, key
  moments, slot").
- **Review every report.** Contact-sheet the best stills and Read them. Fix engine-level problems
  yourself (they affect everyone) and tell the agents what changed.

### 6. Assemble, watch, iterate

- **Render the full cut** (3 workers, about 5 min for 2:00 on an M-series Mac). Wait for it in the
  foreground (or poll until it prints `wrote`) and carry on to verification and delivery: never end
  the run while a render or encode is still going, because a headless session that stops kills it.
- **Verify:**
  - `av_sync_check.py`: expect 0.0 ms audio and 0 to +1 frame visual. Below a correlation of about
    0.15 the visual lag means nothing (type leading by `E.LEAD` reads as up to -3 frames): check
    onset stills instead.
  - `freezedetect`: no frozen footage.
  - Contact sheets at 0.5 s steps, 20 s per sheet.
  - Every section boundary, frame by frame.
  - On portrait, `still.mjs --safe` sheets, 20 s each: no word, logo or face outside the outlined
    band.
- **Global look pass:** a still sheet across all sections, then tune `POST`, the palette and the HUD
  theme.
- **Revisions:** map each director note (they arrive as timestamps) to its section and send it back
  to the owning agent via SendMessage. Resuming keeps its context. Fix a weak shot by replacing it
  with a new generation in a different look, or with code, never by adding effects. Offer
  alternatives as timeline `variant`s (render with `--variant b`) instead of guessing.

### 7. Deliver

- **Videos:** a master, an upload file (landscape: the master for YouTube, an X-ready 2-pass 21 Mb/s
  file under 512 MB for X; portrait: the vertical upload in `references/pipeline.md` §8), a teaser
  (verse plus first chorus), plus one file per variant, and one set per canvas, suffixed `_16x9` and
  `_9x16`, when both were asked for. Use distinct filenames, because macOS is case-insensitive.
- **Stills:** full-res PNGs rendered by the engine, never grabbed from the compressed video.
- **Style sheet / making-of:** publish it as an artifact or doc: palette, type, subject sheet,
  source-vs-final footage pairs, a frame index, the pipeline and iteration counts.
- **Report:** tell the user what was verified, which lyrics you guessed, and the CU spent, summed
  from this run's successful `jobs_wait` rows (`cuCost`, logged in `analysis/jobs.md`; a failed job
  is refunded but its row keeps a `cuCost`); `usage` is project-wide and includes other work.

## Quality bar

- **Every word shows on time.** Each sung word is visible at its onset (as a hero or in the subtitle
  track), and hero words peak on the onset frame.
- **Integration, not decoration.** Every footage moment has a graphic that reacts to it (behind via
  the matte, tracked to it, timed to its move, or continuing its shapes). The footage pixels are
  untouched.
- **No static frames.** Footage always plays live, and something always moves.
- **One focal point per frame.** Type frames the subject and never covers the face.
  Behind-the-subject type still reads. On portrait, every word, logo and face sits inside the safe
  band.
- **Section variance.** Mix the looks, 2D and 3D, poster slams and quiet subtitles. Never use the
  same type treatment on two consecutive lines. Cut density climbs with the song's energy, and
  showcase moments (logo, end card) get time.
- **Photosensitivity.** At most 3 full-frame luminance flips per second.
- **Watch it yourself** before calling it done, then take a second look at the first 3 seconds.

## Files in this skill

- Setup and analysis: [scaffold.sh](scripts/scaffold.sh),
  [audio_analysis.py](scripts/audio_analysis.py), [lyrics_align.py](scripts/lyrics_align.py),
  [plot_lyrics.py](scripts/plot_lyrics.py)
- Approved-master stem lookup for transcription, plots and vocal slices:
  [audio_paths.py](scripts/audio_paths.py)
- Footage generation helpers: [synth_beat.py](scripts/synth_beat.py),
  [beatwarp.py](scripts/beatwarp.py)
- Footage prep: [prep_clip.sh](scripts/prep_clip.sh), [prep_video.py](scripts/prep_video.py),
  [matte.swift](scripts/matte.swift), [fixmatte.py](scripts/fixmatte.py),
  [matte_keyed.py](scripts/matte_keyed.py), [matte_fallback.py](scripts/matte_fallback.py),
  [track.py](scripts/track.py)
- Sync checks: [lipsync_check.py](scripts/lipsync_check.py),
  [dance_sync_check.py](scripts/dance_sync_check.py), [av_sync_check.py](scripts/av_sync_check.py)
- Rendering and review: [render.mjs](scripts/render.mjs), [still.mjs](scripts/still.mjs),
  [serve.mjs](scripts/serve.mjs), [sheet.py](scripts/sheet.py)
- Engine, copied into the project by scaffold.sh: [index.html](assets/engine/index.html),
  [main.js](assets/engine/main.js), [canvas.js](assets/engine/canvas.js),
  [core.js](assets/engine/core.js), [plate.js](assets/engine/plate.js),
  [roto.js](assets/engine/roto.js), [typekit.js](assets/engine/typekit.js),
  [hud.js](assets/engine/scenes/hud.js), plus [timeline.template.js](assets/timeline.template.js)
  and [scene.template.js](assets/scene.template.js)
- [references/pipeline.md](references/pipeline.md): every command and Scenario call, step by step
- [references/engine_api.md](references/engine_api.md): the scene contract and the full `E` API
  (canvas and safe band, plates, mattes, tracking, post, HUD, variants)
- [references/motion_library.md](references/motion_library.md): looks, type system, a 40-move
  technique library, integration ideas, transitions, timing, easing and springs, sync and cut
  density
- [references/agent_brief_template.md](references/agent_brief_template.md): the brief each scene
  agent reads
- [references/lessons.md](references/lessons.md): production gotchas and direction defaults that
  held