|Docs

Localize a winning ad with the same voice and lip-sync

Localize an ad that already sells: translate it, dub it, and keep the speaker's own voice and lips in sync. Your agent translates the script, you fix the words in text, and only then does it voice each line and re-sync the mouth. You end up with one branch and one export per language, with on-screen text and captions in that language.

This is ordinary ad work for a real product. The agent makes the files. You upload them and switch the ads on in the ad platform. The agent never publishes an ad or sends an email.

Use invideo for this when you want to check the translation before anyone pays for a voice, and keep every language as an editable branch of the same timeline.

Quickest path

Set up your agent first with invideo for AI agents. Then give it this prompt with the timeline URL of the winning ad.

Localize this ad into German and Spanish (Mexico): <edit-url>.Transcribe it, translate the script, and show me the translation as textbefore you make any voice. Keep the brand name in English. After I approve,clone the speaker's voice, re-sync the lips, change the on-screen text andcaptions, one branch per language, and show me frames before you export.

Replace <edit-url> with the ad's timeline URL, from the editor's address bar. In a chat app, you paste the timeline URL and you fix the translation in the chat. In a terminal, the agent writes the translation to a file, you edit the file, and the agent reads it back.

What you need

The agent needs the ad and the rules for each language.

  • The timeline URL of the ad, or the project name.
  • The target languages, with the region where it matters (Spanish for Spain or for Mexico).
  • Words that never change: the brand name, the product name, a slogan.
  • A native speaker who reads the translation before the voice is made. This is you or someone you trust.
  • The right to clone the speaker's voice. Get the actor's consent in writing before you start.

Steps

The agent translates and voices first, then builds one branch per language.

Translate and voice

The agent fixes the words in text before it spends anything on a voice.

  1. Transcribe the ad

    The agent prices audio.transcript on the speaker's clip by its sequence URL with dry_run, tells you the price, starts it with max_credits set to that price, and calls again after retry_after_s for the words. It sees every word with its time and speaker.

  2. Translate in text, and wait for your fixes

    The agent translates each line itself and writes the lines into a brief note with new_brief and edit, one note per language. You read the note in invideo and fix it. The agent sees your changes when it reads the brief again with read.

  3. Cut clean reference clips of the voice

    The agent pulls the speaker's sound with media audio.extract on the clip's sequence URL, then cuts short clean pieces of speech with audio.trim (start_time, end_time) on the extracted audio's sequence URL. Each makes a new file, so each call carries an idempotency_key. It sees one audio sequence per clip.

  4. Voice the approved lines in the same voice

    The agent runs find_models with the job in words, picks one, and reads its inputs with get_model. A good pick is seed_audio_1_0.ta2a. Each input takes the sequence URL of one short clip: reference_audio_1 is required, and reference_audio_2 and reference_audio_3 are optional. The prompt tags each voice as @Audio1 to @Audio3, matching the inputs, before its line in quotes. The model detects the language by itself. It calls generate with dry_run: true and tells you the price. Then it calls generate with max_credits set to that price and an idempotency_key. It sees a sequence URL for each new voice and reads it with read; it never calls generate again to check.

Build each language

The agent works on one branch per language and never edits main.

  1. Branch and lay in the new voice

    The agent branches with timeline_history action create_branch, named after the language. It puts the new voice under the picture with media video.replaceAudio, or places it on a new track and mutes the original with set_audio_muted. It sees the branch URL and a new state_token.

  2. Re-sync the lips

    The agent runs find_models with the job in words, picks one, and reads its inputs with get_model. A good pick is sync_lipsync_v3.video (inputs video, audio, sync_mode), and kling_lipsync (inputs video, audio) is another. It calls generate with dry_run: true and tells you the price, then runs with max_credits and an idempotency_key. It sees a new video sequence URL, reads it with read, and swaps it in with place_media mode replace.

  3. Change the on-screen text and captions

    The agent changes each text clip with set_text. It prices audio.transcript on the new voice the same way and calls again after retry_after_s for the word times. Then it sends add_captions. It sees the clips with the new words and a new state_token.

  4. Check each language, then export

    The agent calls capture at each line and each text, and check_timeline. It looks for a mouth out of sync and text that runs off the frame, because German and Spanish lines are often longer. Then it runs export on each branch. It sees one export card per branch with status completed.

The terminal version, for one language:

invideo agent register <project-url> --name <name>export INVIDEO_AS=<name>invideo media audio.transcript <sequence-url> --dry-runinvideo media audio.transcript <sequence-url> --max-credits <price>invideo context new-brief <project-url> --name "Ad 12 localization" \  --description "German and Spanish (Mexico) scripts for review"invideo media audio.extract <sequence-url> \  --idempotency-key ad12-extractinvideo media audio.trim <audio-sequence-url> --start-time 2 --end-time 9 \  --idempotency-key ad12-ref-1invideo gen models <project-url> --query "seed audio"invideo gen model <project-url> --model seed_audio_1_0.ta2ainvideo gen run <page-url> --model seed_audio_1_0.ta2a --dry-run \  -i prompt=@de-line-01.txt -i reference_audio_1=<reference-clip-url>invideo gen run <page-url> --model seed_audio_1_0.ta2a \  --max-credits <price> --idempotency-key ad12-de-voice \  -i prompt=@de-line-01.txt -i reference_audio_1=<reference-clip-url>invideo edit show <edit-url> --jsoninvideo branch new <edit-url> --name de --token <state_token>invideo gen models <project-url> --query "lip sync"invideo gen model <project-url> --model sync_lipsync_v3.videoinvideo gen run <page-url> --model sync_lipsync_v3.video --dry-run \  -i video=<sequence-url> -i audio=<de-voice-url>invideo gen run <page-url> --model sync_lipsync_v3.video \  --max-credits <price> --idempotency-key ad12-de-sync \  -i video=<sequence-url> -i audio=<de-voice-url>invideo edit show <branch-url> --jsoninvideo edit change <branch-url> --timeline T1 --token <state_token> \  --operations @de.jsoninvideo edit capture <branch-url> --at 1,4,8,12 --out ./framesinvideo edit check <branch-url>invideo edit export <branch-url> --format video --quality 1080 --jsoninvideo status <card-url> --waitinvideo notebook show <card-url> --out ./demv ./de/export.mp4 ./de/ad12_de_9x16_v1.mp4

Where the values come from:

  • <project-url> is the ad's project, and <edit-url> is the ad's timeline URL, the one you give the agent. <name> is the name you choose in invideo agent register.
  • <page-url> is a notebook page; invideo notebook show <project-url> lists them.
  • <sequence-url> is the speaker clip's sequence URL, from invideo notebook show <project-url> or what the upload returns.
  • <audio-sequence-url> is the URL audio.extract returns, and <reference-clip-url> the URL audio.trim returns. <de-voice-url> is the sequence URL the paid voice run returns.
  • Each <price> is the dry run right before it.
  • <state_token> comes from invideo edit show <edit-url>, and on the branch from invideo edit show <branch-url>; the main token is stale there. <branch-url> is the .branch_url that invideo branch new returns.
  • <card-url> is .sequences[0] of what the export returns. invideo status waits and does not download; invideo notebook show saves ./de/export.mp4.
  • The file ./de.json holds a bare list of operations: the place_media swap, the set_text changes and the captions. The file de-line-01.txt holds the prompt with its @Audio1 tag.

The voice call over MCP, as a dry run. The clip in reference_audio_1 is @Audio1 in the prompt. <page-url> comes from read on the project, and <reference-clip-url> is the URL audio.trim returns. The paid call is the same with dry_run removed, max_credits set to the price and an idempotency_key.

{  "url": "<page-url>",  "model": "seed_audio_1_0.ta2a",  "dry_run": true,  "inputs": {    "prompt": "@Audio1, warm and upbeat: \"Schluss mit kaltem Kaffee.\"",    "reference_audio_1": "<reference-clip-url>"  }}

invideo has no one-shot dubbing model. For a one-shot draft, ElevenLabs Dubbing or HeyGen video translation makes one outside invideo, with no step where you fix the words first.

Cost and time

Edits and exports are free. Generation and transcripts cost credits. Paid stock costs a small credit charge, or one of your workspace's iStock licenses. Your agent runs a dry run first and tells you the price before it spends anything.

  • Costs credits: the transcripts, the voice with seed_audio_1_0.ta2a, and the lip sync.
  • Free: the translation the agent writes, your fixes, the branches, every edit, capture and export.
  • audio.extract and audio.trim: the dry run shows the price before anything starts.

One language takes under an hour, most of it the wait for lip sync and your review. Lip sync charges by the length of the audio. See Credits for prices.

If something goes wrong

Most problems come from a weak reference clip or a line that grew in translation.

SymptomFix
get_model or generate refuses the model as not foundModels retire. Run find_models with the job in words, use the best match, and tell the person which model you used.
Exit code 6, or status is heldA safety review, not a refusal. Wait, then run invideo status <url> --wait again. Do not resend. If it ends failed, change the input the review flagged.
You have no consent to clone the voiceUse elevenlabs_text_to_speech.elevenlabs_tts_v4 with a voice from elevenlabs_text_to_speech_voices instead, priced the same way.
The new voice sounds like another personUse cleaner reference clips: speech only, no music under it. Add a second clip as reference_audio_2.
The mouth drifts out of sync at the endThe new audio is longer than the video. Set sync_mode to remap, or shorten the line in text and voice it again.
German text runs off the frameMake the text smaller with set_text_style, or split the line into two text clips.
The brand name is translatedFix it in the brief note before the voice, and list it as a word that never changes.
CLI exit 3 on the branchThe token is from main. Run invideo edit show <branch-url> and use its state_token.
CLI exit 5A limit: credits, the --max-credits cap or a quota. Read the Fix: line, then tell the person and stop.
Avoid: voicing before the text is approvedInstead: wait for the person's fixes in the brief. A voice costs credits; text does not.
Avoid: editing main for a languageInstead: one branch per language with timeline_history.

Prompts to try

Each prompt is a real variation of this job.

My best ad is in English. Make French, German and Italian versions. Keep"Brewly" in English. Put each translation in the brief, wait for my OK,then voice, lip-sync and export each one.
Make a quick Spanish draft of this ad so I can show the team. Voice thetranslation, no lip sync. One export, named as a draft.
The Japanese version is approved. Change only the on-screen price text tothe yen price I give you, add Japanese captions, and export.

When to use something else

invideo does not check a translation for you. A native speaker or a translation service should read every line before the voice is made. For regulated claims in a market, ask your legal or compliance team. For a full studio dub with several actors, a dubbing studio or ElevenLabs Dubbing is a better fit.