From Whisper Transcript to Broadcast-Quality SRT and VTT Subtitles

Short answer: don’t turn Whisper’s segments into subtitle cues one to one. Build the cues yourself from word timestamps, under explicit rules: at most 42 characters per line and two lines per cue, a duration between about 5/6 of a second and 7 seconds, a reading speed limit (17 to 20 characters per second), a small gap between cues, and a new cue at every long pause, sentence end and speaker change. Then break each cue into balanced lines, preferably after punctuation. When the speech is simply too fast to meet the rules, flag the cue instead of stretching it over the next one.

The numbers above come from Netflix’s English timed text guidelines, which are a sensible default when nobody has given you house rules. Many broadcasters use shorter lines, so make every rule a setting.

Why Whisper segments make bad subtitles

Whisper’s segments are units of decoding, not units of reading. In practice:

• A segment can hold several sentences, or stop halfway through one.
• Nothing limits its length. One segment can be 120 characters on one line.
• Its duration has nothing to do with how long a person needs to read it.
• It doesn’t care about speaker changes, so two people end up in one cue.

whisper --output_format srt writes segments straight to SRT, which is fine for a rough draft and not fine for anything published. The good news is that the information you need is already there: with word timestamps turned on, every word has a start and an end.

The rules

Here are the rules worth enforcing, with typical values:

Rule	Typical value	Why
Max characters per line	42 (some broadcasters: 37)	Fits the screen at a readable size
Max lines per cue	2	More covers the picture
Min duration	5/6 s to 1 s	Shorter cues flash past before the eye lands on them
Max duration	7 s	Longer cues are usually re-read or ignored
Reading speed	17 cps (children), 20 cps (adults)	Characters per second the viewer can comfortably read
Min gap between cues	about 2 frames	Lets the eye register that the text changed
New cue on pause	after about 1 s of silence	Subtitles should follow the rhythm of speech

Netflix’s English guide sets 42 characters per line, a maximum of two lines, a minimum duration of 5/6 of a second, a maximum of 7 seconds, and reading speeds of up to 20 characters per second for adult programs and 17 for children’s.

Step 1: build cues from words

Walk the words in order and close the current cue whenever adding the next word would break a rule, or when the speech gives you a natural boundary:

type Word = { text: string; start: number; end: number; speaker?: string };
type Cue = { start: number; end: number; lines: string[]; speaker?: string };

const rules = { maxChars: 42, maxLines: 2, minDuration: 0.833, maxDuration: 7, maxCps: 17, minGap: 0.08, pauseSplit: 1.0 };

const join = (ws: Word[]) => ws.map((w) => w.text.trim()).join(" ");

function fits(ws: Word[]): boolean {
    const lines = breakLines(join(ws), rules.maxChars);
    return lines.length <= rules.maxLines && lines.every((l) => l.length <= rules.maxChars);
}

function buildCues(words: Word[]): Cue[] {
    const cues: Cue[] = [];
    let current: Word[] = [];
    const close = () => {
        if (current.length) {
            const first = current[0];
            const last = current[current.length - 1];
            cues.push({ start: first.start, end: last.end, lines: breakLines(join(current), rules.maxChars), speaker: first.speaker });
            current = [];
        }
    };

    for (const word of words) {
        const prev = current[current.length - 1];
        if (prev) {
            const pause = word.start - prev.end;
            const speakerChange = word.speaker !== prev.speaker;
            const overflow = !fits([...current, word]) || word.end - current[0].start > rules.maxDuration;
            // End at a sentence boundary once the cue is reasonably full.
            const sentenceEnd = /[.?!]["')\]]?$/.test(prev.text) && join(current).length > rules.maxChars * 0.6;
            if (pause >= rules.pauseSplit || speakerChange || overflow || sentenceEnd) {
                close();
            }
        }
        current.push(word);
    }
    close();
    return cues;
}

A refinement worth adding: when a cue overflows in the middle of a clause, back up to the last comma or clause boundary inside the cue and start the new cue there. Subtitles that break at “and” or after a comma read much better than ones that break wherever the character count ran out.

Step 2: break lines so they look balanced

The simple approach fills the top line as far as it will go and puts the rest underneath. You get cues like this:

I think the most important thing we
learned was this

A balanced break tries every split point and picks the one where both lines are closest in length, with a bonus for splitting after punctuation:

function breakLines(text: string, maxChars: number): string[] {
    if (text.length <= maxChars) {
        return [text];
    }
    const words = text.split(" ");
    let best: string[] = [text];
    let bestScore = Number.POSITIVE_INFINITY;
    for (let k = 1; k < words.length; k++) {
        const top = words.slice(0, k).join(" ");
        const bottom = words.slice(k).join(" ");
        if (top.length > maxChars || bottom.length > maxChars) {
            continue;
        }
        let score = Math.abs(top.length - bottom.length);
        if (/[,.;:?!]$/.test(top)) {
            score -= 8; // prefer breaking after punctuation
        }
        if (top.length > bottom.length) {
            score += 1; // on ties, keep the bottom line longer
        }
        if (score < bestScore) {
            bestScore = score;
            best = [top, bottom];
        }
    }
    return best;
}

I think the most important
thing we learned was this

Keeping the bottom line longer on ties is a common subtitling convention: the eye starts lower on the screen and has less distance to travel. Count characters as Unicode code points ([...text].length) if you handle anything beyond plain English, and remember that speaker prefixes and dashes count toward the limit.

Step 3: fix the timing

Word timestamps give you when the words were spoken. The cue also needs to stay on screen long enough to read, without touching the next cue:

function fixTiming(cues: Cue[]) {
    const warnings: string[] = [];
    for (let i = 0; i < cues.length; i++) {
        const cue = cues[i];
        const next = cues[i + 1];
        const latestEnd = next ? next.start - rules.minGap : Number.POSITIVE_INFINITY;
        const chars = cue.lines.join("").length;

        // How long the cue should stay up: at least the minimum, and long enough to read.
        const wantedEnd = Math.max(cue.end, cue.start + rules.minDuration, cue.start + chars / rules.maxCps);
        cue.end = Math.min(wantedEnd, latestEnd, cue.start + rules.maxDuration);

        const duration = cue.end - cue.start;
        if (duration < rules.minDuration) {
            warnings.push(`cue ${i + 1}: shorter than minimum duration`);
        } else if (chars / duration > rules.maxCps) {
            warnings.push(`cue ${i + 1}: reading speed ${(chars / duration).toFixed(1)} cps`);
        }
    }
    return warnings;
}

The important design choice is in the last few lines. When someone talks faster than the reading-speed rule allows, there is no timing that satisfies every rule. Stretching the cue over the next one makes everything after it drift. Report it instead, so a person can shorten the text or accept the cue as it is.

Step 4: write SRT and VTT

The two formats are nearly identical. The differences that trip people up are the millisecond separator and the header:

const pad = (n: number, w = 2) => String(n).padStart(w, "0");

function stamp(seconds: number, sep: "," | "."): string {
    const ms = Math.round(seconds * 1000);
    const h = Math.floor(ms / 3_600_000);
    const m = Math.floor((ms % 3_600_000) / 60_000);
    const s = Math.floor((ms % 60_000) / 1000);
    return `${pad(h)}:${pad(m)}:${pad(s)}${sep}${pad(ms % 1000, 3)}`;
}

const toSrt = (cues: Cue[]) =>
    cues.map((c, i) => `${i + 1}\n${stamp(c.start, ",")} --> ${stamp(c.end, ",")}\n${c.lines.join("\n")}\n`).join("\n");

const toVtt = (cues: Cue[]) =>
    `WEBVTT\n\n${cues.map((c) => `${stamp(c.start, ".")} --> ${stamp(c.end, ".")}\n${c.lines.join("\n")}\n`).join("\n")}`;

SRT:

1
00:00:01,200 --> 00:00:04,050
I think the most important
thing we learned was this

WebVTT:

WEBVTT

00:00:01.200 --> 00:00:04.050
I think the most important
thing we learned was this

VTT also supports cue settings after the timing (line:85% align:center) and voice tags for speakers, which SRT does not have.

Speaker labels

If your transcript is diarized, there are three common ways to show who is talking:

• Prefix on the first cue after a speaker change: HOST: Welcome back.

• Dashes, when two short lines from different speakers share one cue:

  - Did it ship?
  - Last night.

• VTT voice tags, which players can style and screen readers can announce: <v Host>Welcome back.

Whichever you choose, start a new cue on every speaker change unless you are deliberately using dashes. A cue that starts with one voice and ends with another is the most confusing thing a subtitle can do.

A checklist before you publish

• Built from word timestamps, not segments.
• Every line within the character limit, counted in code points, speaker labels included.
• No cue longer than two lines or seven seconds.
• Reading speed checked, with fast cues reported rather than stretched.
• A gap between every pair of cues, and no overlaps.
• New cues at pauses, sentence ends and speaker changes.
• Lines balanced, with breaks after punctuation where possible.
• Every rule a setting, because the next client will have different house rules.

Where this fits

All of this assumes you already have one clean, continuous word list. If your recording was transcribed in chunks, join them properly first: How to merge chunked Whisper transcripts without duplicate or lost words.

This is the subtitle engine inside Splice, our service that takes chunked Whisper or WhisperX output from Google Cloud Storage or S3 and renders SRT, VTT, DOCX, PDF and CSV, with every rule above configurable per request. If you need this in your own pipeline, get in touch.