Adding Voice and Music¶
Skill version 2026.11.03 · 17,856 chars · 2,825 words · 4,780 estimated tokens
Narration, captions, music and sound effects for a demo or explainer video built on the HTML contract. Feeds autoplay muted, so add audio only when the user asks, deliver the silent cut first, and let no sound carry a claim the picture doesn't.
Contents¶
Every script here caches or reproduces its output from a spec, so commit the spec and the script call, not the audio. You can't hear any of it: hand the user the file and change the spec from their words.
Narration¶
Write a narration spec and render it with
scripts/make_voice.py, which runs the local
Qwen3-TTS models through faster-qwen3-tts. Every speaker has one reference
take, and the Base model clones it for every line. Start from the voice this
skill ships, "measured": a calm, quietly confident developer presenting at
a conference, the same clip on every machine. "casual" is the shipped
female voice (a senior developer showing a colleague a trick, about 189 Hz);
it reads slower, and the same script ran about 8% longer. When the user
wants to compare voices, render the spec once with each and keep both cuts.
{
"voice": {"lang": "en",
"speakers": {"narrator": {"voice": "measured"}}},
"lead": 0.4, "tail": 0.5, "loudness": -16,
"timeline": {"bar": 0.1, "gap": 0.3, "pad": 0.7},
"lexicon": {"hvPlot": "aitch vee plot", "HoloViz": "holo viz"},
"lines": [
{"id": "open", "scene": "open", "text": "This diff does a lot, so let's look at what we can trim."},
{"id": "keep", "scene": "keep", "text": "The docs already prefer speakers, so let's keep that one and let the other four go."}
]
}
python scripts/make_voice.py narration.json --out voice.wav --srt voice.srt --json voice.json
python scripts/make_voice.py narration.json --line keep --takes 4 # audition one line
python scripts/make_voice.py narration.json --pick 3 # steadiest take of every line
- One speaker per voice, also for one narrator. A speaker is
{"voice": name}(a clip inassets/voices, with its transcript and the description it was designed from beside it),{"line": id, "design": {"instruct": ...}, "take": n}(that line rendered from a description) or{"audio": path, "text": ...}(a saved clip and its exact transcript). A line without"speaker"belongs to the first one. The clone step gets the reference's transcript as well as its audio, which carries the reference's delivery into every line along with its timbre.voice.designholds defaults shared by every speaker's design. - A description doesn't reproduce a voice; the clip does. VoiceDesign
has no seed, and the same description and take give a different voice on
another machine or toolchain version. Once the user likes a designed voice,
copy its reference take to
voices/<name>.wavand switch the speaker to{"audio": ..., "text": ...}; before that, rewriting the reference line changes the voice. Commit the WAV when others must get the same voice. A voice the whole project should share goes inassets/voices/as<name>.wavplus<name>.json(text,instruct, how it was made), and keep the clip under the 500 KB large-file check: about 5 s of 48 kHz mono. - Pick the reference for its delivery. Every clone inherits it, and a flat statement as the reference gives a flat speaker. Use the speaker's liveliest line (a reveal, a question, a reaction).
- Describe a person, not a timbre. Say who is speaking, to whom and where: "a man in his late thirties co-hosting a relaxed explainer podcast, natural American accent, warm mid-low voice, lively intonation", and say what to avoid ("never corporate", "not an announcer", "never squeaky or cartoonish"). "Bright", "bubbly" or "reacts with surprise" drifts toward a high, cartoonish voice. A brand voice (Anaconda's is a curious, encouraging guide) becomes a person in a scene: "showing a teammate something neat she found", not "professional and confident", which reads as a corporate announcer. Don't describe or name a real person.
- Pick the voice by measurement, then by ear. You can't hear the takes,
but you can measure each take's median pitch and how much it moves (the
standard deviation in semitones around the median). Natural speaking pitch
is about 165–255 Hz for a woman and 85–180 Hz for a man; a take far above
that sounds cartoonish. Movement is liveliness: under about 2.5 semitones
sounds flat or robotic, 3–4 calm and even, and 5–6 an enthusiastic
presenter, which reviewers of a technical walkthrough found too lively.
--line <reference> --takes 4renders candidate voices; keep the ones in range nearest the energy the user wants, and let them choose by ear. - For a calmer voice, change three things together: describe the
delivery ("measured and even, speaks steadily without hype, never
excited"), make a plain statement the reference line, and pick for
moderate movement. Lowering
design.tempto 0.5 flattens it further, toward robotic. Calm and experienced descriptions drift low and male; say "woman" or "mid-range voice" when that matters. - Let the narration set the timeline whenever the screens can move.
Fixed scene times leave each screen silent for however long its line falls
short, and viewers hear those waits as the video stalling. Give lines a
"scene"and no"t": a scene's first line startsleads into it, its lines followtimeline.gaps apart, and it endspads after its last line, rounded up totimeline.bar.make_voice.pywrites the scene times to--timeline timeline.jsonfor the page and the score to read. Without music, use abarof 0.1 and apadof about 0.7 (about 1.1 s from one line to the next); with music, keepbarequal to the music's bar (3 s at 80 BPM). Use"t"only when the picture can't move, such as footage cut to a fixed length. - Write one walkthrough, not a line per screen. Each line picks up from
the last ("Start with…", "Then there are two spots…", "After that…",
"Lastly…", "So that's…"); lines that each open fresh ("First up", "Now",
"Oh, and") sound like separate recordings however steady the voice, and a
bare "Last," reads clipped where "Lastly," signals the final step. Keep
lines to one thought of similar length. A two-part quip ("Nine entries?
It's really three.") puts half a second of silence mid-line, and a run of them chops
the narration up. Open with a hook the viewer recognizes ("Ever plotted a
big dataset and just gotten a blob? Let's fix that."), and give the hook
its own line: two statements in one opening line ("Every account says
they already have that. Here's what we keep hearing it leaves open.")
came out run-on and robotic, while a question line plus a short second
line in the same scene ("Ever pitched something and heard, we already
have that?" / "Fair enough. So let's look at what it leaves open.")
landed with a real beat between them. For a two-host
conversation, short reactions ("Right.", "Exactly.") as their own lines,
with
timeline.gap0.25, make it sound like talk. - Pick each line's take with
--pick. Clones of one reference still vary from line to line, in pace (2.4–3.6 words a second in one narration) and in pitch (one line 3 semitones above the rest sounds like another speaker), and that swing breaks the flow.--pick 6renders takes 1–6 of every line in one model load per speaker, keeps the take nearest the median pace and the reference's pitch with no pause over 0.45 s, and writes the choices into the spec as"take". On the shipped voice it kept all eight lines within 114–124 Hz. - Cut the reference from the narration the user chose. The user judges a voice by its lines, which are clones and drift from a designed reference take. Once they pick a narration, cut whole lines of it (about 10 s, at 24 kHz to stay under the 500 KB check) from its voice track, not the compressed MP4, with their exact spoken transcript, and make that the reference.
- Visuals follow the speech, not the other way around.
voice.jsonholds each line's phrases, split at sentence ends, colons and ellipses and timed from the pauses in the take (by character count when a pause is too short to find). Time each visual event to the word that names it: a helper likeat(scene, word, fallback)finds the phrase containing the word and estimates the word's time from its position. Expose the event times aswindow.EVENTSso the build can place sound effects on them. - The voice repeats the screen. A line says about what the headline says, in one sentence; don't read code aloud, and run the lines through deslop.
- Say it positively. A listener can't re-read a line, and a double negative costs them the sentence. A reviewer stumbled on "MCPs that touch company data don't ship without governance"; "…need governance before they ship" fixed it. Change the on-screen label with it ("No governance, no go-live" became "Governance before go-live").
- With fixed times, fit the line to the screen, not the voice to the
line. Each line starts
leads after its screen. When a line runs pasttails before the next screen, the script prints how many words fit at this voice's pace; cut words, because sped-up speech sounds wrong. - Pauses come from punctuation inside a line and from
timeline.gapbetween lines. The prompt is just the line: each ellipsis or colon adds 0.3–0.5 s and slows the words around it, a full stop adds a short beat, and dropping commas removes pauses and speeds the read up. For a pause that must land (a beat between a problem and its fix, before the product's name appears), split the line into two lines in the same scene:timeline.gap(0.5 s) then sets the silence exactly, and each half gets its own take. - Put every acronym and mixed-case name in
lexicon(GPU, AI, PyTorch, hvPlot), plus anything else the model might read wrong. The lexicon changes only what the model hears; captions keep the on-screen spelling. Before it renders,make_voice.pywarns about each word with two or more capitals, or camelCase from a lowercase start, that no entry covers, and a plural needs its own entry (GPUdoesn't coverGPUs). Map a word to itself ("API": "API") once you've heard the model read it right, so the warning records that someone checked it. Spell acronyms the way they're said, not letter by letter with spaces:"CVEs": "see vee eez","CISO": "see-so"("C V Es"came out wrong),"AI": "ay-eye"("A I"sounded robotic). A product name of two or more words gets its own entry ("Enkrypt AI": "Encrypt ay-eye") even when each word already has one. Entries apply longest first, and the phrase entry keeps the model from reading a verb followed by an acronym. Write out dates and symbols:"12 Nov": "November twelfth","500+": "five hundred plus". Listen to every respelled line once. - Every line is levelled to the target before mixing, because clones of the same reference can differ by several dB.
- Check the words. TTS models sometimes drop or repeat words. If whisper.cpp is installed, transcribe the voice track and compare it with the spec.
- Install and locate the engine.
uv tool install faster-qwen3-tts(it shares the Hugging Face cache, so weights aren't downloaded twice). The script looks in$FASTER_QWEN3_TTS, then PATH, then beside the running Python and in conda envs, and runs it with--backend ggml --quant Q8_0: on Apple silicon GGML's Metal library has nokernel_get_rows_bf16, so--quant BF16segfaults, and--backend torchneeds CUDA graphs. Each speaker's uncached lines go through onefaster-qwen3-tts serveprocess, because model loads, not speech, dominate short lines. - bf16 is optional.
"engine": "qwen-tts"invoicerenders the same spec in the models' native bf16 through the officialqwen-ttspackage (scripts/_qwen_tts_bf16.py, on Apple's GPU, CUDA or CPU), which avoids Q8_0's quantization but is slower. It needs its own venv (pip install qwen-tts soundfile, since it pins its own transformers) at~/.venvs/qwen-ttsor$QWEN_TTS_PYTHON. Takes are cached per engine, so switching re-renders every line; a saved"audio"reference keeps the voice, and only the clones change. - Check the models' licenses.
Captions¶
make_voice.py writes an SRT and a JSON of cue times measured from the takes.
- Soft track and sidecar.
record_video.py --srtembeds the SRT as amov_texttrack; also upload the SRT beside the video, since LinkedIn and X ignore embedded tracks. - Burned in, for feeds that show no captions: draw the JSON cues in the
HTML as a function of
t(shown with?captions=1), and letrecord_video.py --burn-captionsoverlay them on the plain cut, so there's no second recording. - Captions sit below the picture. Let the browser wrap them
(
text-wrap: balance) rather than keeping the SRT's line breaks, which leave one-word lines.
Music¶
Render a score with scripts/make_music.py. Start
from the lofi-jazz preset with only the scene times:
{"preset": "lofi-jazz", "duration": 54, "seed": "my-video",
"scenes": [0, 6, 12, 18, 24, 30, 36, 42, 48]}
python scripts/make_music.py score.json --out music.wav
python scripts/make_music.py --preset lofi-jazz --scenes 0,6,12 --duration 18
The preset fits the tempo so every scene starts on a bar line (80 BPM for 6 s
scenes), plays ii9–V9–Imaj9–vi9 with off-beat comping, swung hats and vinyl
texture, starts with pad, keys and bass, adds drums from the second scene and
resolves on the last. A preset render checks itself for 6 LU jumps and
lowers the offending section's gain until none is left (--fix does the
same for a hand-written spec), so don't tune gains unless the user asks.
Change seed for another take with the same structure, and cache the WAV by
a hash of the spec so later builds skip it.
For a calmer, documentary feel use "preset": "cinematic": D lydian at
80 BPM half-time, I–V–vi–IV one bar per chord, pad, keys and bass from the
start, a rising motif from the second scene, a swell on the scene before the
close, and no drums. Two bars per chord stretched the opening fade-in into a
jump warning; the jump check ignores the deliberate two-bar fade-in at the
start of the track.
Write sections by hand (--example prints a full spec) only when the user
wants a specific arrangement:
- Tempo and sections from the beat sheet. One section per scene change,
each on a bar line; fix a bar-line warning by changing
bpm, not the picture's times. Scenes 10–14 s apart on even seconds fit 120 BPM with"feel": "half". - A motif per concept in
motifs(scale degrees, or"pulse"), added tolayerswhere that concept first appears on screen. - Build from a full start. A lone pad followed by a new layer trips the 6 LU jump warning; start with two or three layers instead.
- The payoff gets
"layers": "all","swell": trueand a littlegain; the close gets"resolve": true, which holds the tonic while the music fades over the last 3 s. - Mood picks
key,modeandtexture: major orlydianaround 0.5 for an explainer,majorormixolydianwith less texture for a launch.
If the user would rather have a produced track, a platform library track (LinkedIn, YouTube) avoids licensing questions.
Sound effects¶
scripts/make_sfx.py generates original effects, so
they carry the repository's license and need no credit.
python scripts/make_sfx.py --list # names and what each is for
python scripts/make_sfx.py --out sfx/ # every sound + all-sounds.wav
python scripts/make_sfx.py bubble --pitch -2 # retune after feedback
- Each effect marks an event already on screen, at that frame. When the
page exposes
window.EVENTS, read the times from it in the build rather than typing them, so effects stay on their events when the narration moves them. Put them in the score'shitsas[sound, t, LU](LU above the music just before, default 3, capped at 6) so the music ducks under each one. Keep hits out of narration. - Don't imitate a recognizable app, phone or game sound. Sounds from elsewhere
need a commercial-use license (CC0; never NonCommercial or NoDerivatives)
and a row in a
SOURCES.mdbeside the HTML.
Mixing and loudness¶
The scripts master through scripts/_mastering.py and scripts/_ffmpeg.py,
so nothing clips or startles; a sudden loud moment is the fastest way to lose
a viewer.
- Duck the music under the voice with a sidechain from the voice track,
then level the mix to −16 LUFS integrated with true peaks under −1.5 dBFS.
_ffmpeg.loudnorm(out, -16, graph, [music, voice])takes a graph ending in[pre]:
[1]aformat=channel_layouts=stereo,asplit[vsc][vmix];
[0][vsc]sidechaincompress=threshold=0.015:ratio=6:attack=40:release=600[bed];
[bed][vmix]amix=inputs=2:normalize=0:duration=first[pre]
- Levels: voice at −16 LUFS; music alone at −20 to −22 under a voice, −16 to −20 without one, and lower for a video watched at a desk or on a call.
- Don't hand over a score that still prints a warning (a 6 LU jump within 3 s, or a peak 10 LU over the average); fix the spec first.