--- title: Use Whisper timings with invideo description: Turn a Whisper transcript, SRT or word JSON into an editable caption track on an invideo timeline, with each word timed as it is spoken. --- It depends on where your transcript comes from. Whisper, and any tool that writes SRT or word-level JSON, gives you captions and subtitles as timed text; invideo turns that text into an editable caption track on a real timeline, with each word highlighted as it is spoken. invideo also makes its own transcript with `media` `audio.transcript`. Bring Whisper's when you already have one, when a person corrected it, or when someone checked the language by hand. ## When to use both Use Whisper's timings in invideo when the transcript is already done and trusted. Use `audio.transcript` when you have no transcript at all. - **You already ran Whisper.** The transcript exists on your machine. Your agent reads it and places the captions, so no second transcript runs. - **A person corrected the words.** Names, product terms and jargon are fixed in the file. A new transcript would bring the old mistakes back. - **The language was checked.** Someone who speaks the language read the transcript. Keep their version. - **You want speakers or a fresh pass.** Run `audio.transcript`. It gives every word with its time and its speaker. ## Hand-off: from Whisper into invideo Whisper writes its transcript as a file. The `whisper` command takes `--output_format` with `txt`, `vtt`, `srt`, `tsv`, `json`, `jsonl` or `all`. The `--word_timestamps True` option adds a time for each word, which Whisper's own help marks as experimental. ```bash whisper interview.mp4 --model turbo --language English \ --word_timestamps True --output_format json ``` SRT gives one time per line, so each line becomes a caption slide. Word JSON gives a time per word, so each word gets its own highlight inside its slide. Your agent brings the video into the project: - **A public link.** `add_media` with `media_url`. - **A file on your computer.** `start_upload` with the `file_name`, a PUT of the bytes to the `upload_url`, then `add_media` with the `upload_id`. In a terminal, `invideo asset upload `. - **A file in a chat app.** You drag the file into the chat. It arrives through `file` on `add_media`. Each file becomes a sequence in the project, with its own sequence URL. On a timeline's URL, `add_media` also puts it in that timeline's media bin. The transcript file does not go into the project. The agent reads it on your machine, or you paste it or attach it in a chat app. The agent then writes one `add_captions` op. Each slide has `text`, `start_f` and `end_f`, and an optional `words` list with the same three fields. Times are in frames at the timeline's rate, so the agent multiplies Whisper's seconds by the timeline's fps. Words must sit inside their slide, and slide text cannot be empty. The fence is the bare list of ops that `--operations @ops.json` takes: ```json [ { "op": "add_captions", "track": "V2", "slides": [ { "text": "Welcome back to the show", "start_f": 0, "end_f": 60, "words": [ { "text": "Welcome", "start_f": 0, "end_f": 18 }, { "text": "back", "start_f": 18, "end_f": 28 }, { "text": "to", "start_f": 28, "end_f": 34 }, { "text": "the", "start_f": 34, "end_f": 42 }, { "text": "show", "start_f": 42, "end_f": 60 } ] } ], "style": { "highlight_color": "#ffd400", "stroke": true } } ] ``` The frame numbers are Whisper's seconds times the timeline's fps, from `get_timeline`. The `text` values come from the transcript file. After that the captions are clips like any other. The agent fixes one line with `set_caption_slide_text`, splits a long slide with `split_caption_fragment`, and turns captions off for one export with `set_clip_enabled`. ## Hand-off: from invideo to Whisper Two things go the other way. For a caption file, the agent reads the caption slides with `get_timeline` mode `collection` and writes the SRT or VTT file itself. For a new Whisper run on the edited cut, the agent exports the sound with `export` mode `audio` and `audio_format` `wav`, then runs Whisper on that file. ## Steps Your agent sets up with [invideo for AI agents](/agents) first. Then it runs these steps on a branch. The agent calls `add_media` on the timeline URL, or `start_upload` then `add_media` for a local file. Expected result: a sequence URL and a media bin id. The agent reads the SRT or word JSON and lists each line with its start and end in seconds. Expected result: lines in time order with no empty text. If a line is empty, the agent drops it. The agent runs `media` `audio.content` on the video and compares the speech spans with the first and last lines. Expected result: the first word starts where speech starts. The agent creates a branch with `timeline_history`, reads the fps with `get_timeline`, and checks the fields with `timeline_catalog`. Then it sends `add_captions` in one `edit_timeline` call. Expected result: a new `state_token` and a caption clip. The agent calls `capture` at a few caption times and runs `check_timeline`. Expected result: words on screen match the sound and sit inside the frame. The agent calls `export` with `format` `video` and an `idempotency_key`. Expected result: an export card with status `completed`. Placing captions from your own transcript costs nothing. Edits and exports are free. Generation and transcripts cost credits. Paid stock costs a small credit charge, or one of your workspace's iStock licenses. Your agent runs a dry run first and tells you the price before it spends anything. ## When to stay in Whisper Stay with Whisper alone when you need only a transcript or a caption file and no video. Stay with it too when the audio must not leave your machine, because Whisper runs locally. For translation of the speech into English text, Whisper has `--task translate`. ## If something goes wrong | Symptom | Fix | | --- | --- | | `add_media` with `media_url` is refused | The link must be public. Download the file, then upload it with `start_upload` and `add_media` with the `upload_id`. | | Every caption starts early or late by the same amount | The transcript came from a trimmed file. Shift every time by the gap that `audio.content` shows. | | One caption line is wrong or off time | Fix it with `set_caption_slide_text` or `trim_caption_slide`. | | `add_captions` is refused | Each word must sit inside its slide, and slide text cannot be empty. Drop empty lines and check the frame numbers. | | Avoid: resend after a timeout | Instead: retry the same request with the same token. | ## Related - [Word-by-word captions](/how-to/creators/word-by-word-captions) - [SRT for an LMS](/how-to/training/srt-for-an-lms) - [Cut ums and retakes](/how-to/creators/cut-ums-and-retakes) - [Use ElevenLabs with invideo](/integrations/elevenlabs) - [Edit operations](/reference/edit-operations) - [Check your video before you say it is done](/how-to/check-your-video)