Localize a winning ad with the same voice and lip-sync
Localize an ad that already sells: translate it, dub it, and keep the speaker's own voice and lips in sync. Your agent translates the script, you fix the words in text, and only then does it voice each line and re-sync the mouth. You end up with one branch and one export per language, with on-screen text and captions in that language.
This is ordinary ad work for a real product. The agent makes the files. You upload them and switch the ads on in the ad platform. The agent never publishes an ad or sends an email.
Use invideo for this when you want to check the translation before anyone pays for a voice, and keep every language as an editable branch of the same timeline.
Quickest path
Set up your agent first with invideo for AI agents. Then give it this prompt with the timeline URL of the winning ad.
Localize this ad into German and Spanish (Mexico): <edit-url>.Transcribe it, translate the script, and show me the translation as textbefore you make any voice. Keep the brand name in English. After I approve,clone the speaker's voice, re-sync the lips, change the on-screen text andcaptions, one branch per language, and show me frames before you export.Replace <edit-url> with the ad's timeline URL, from the editor's address bar. In a chat app, you paste the timeline URL and you fix the translation in the chat. In a terminal, the agent writes the translation to a file, you edit the file, and the agent reads it back.
What you need
The agent needs the ad and the rules for each language.
- The timeline URL of the ad, or the project name.
- The target languages, with the region where it matters (Spanish for Spain or for Mexico).
- Words that never change: the brand name, the product name, a slogan.
- A native speaker who reads the translation before the voice is made. This is you or someone you trust.
- The right to clone the speaker's voice. Get the actor's consent in writing before you start.
Steps
The agent translates and voices first, then builds one branch per language.
Translate and voice
The agent fixes the words in text before it spends anything on a voice.
Transcribe the ad
The agent prices
audio.transcripton the speaker's clip by its sequence URL withdry_run, tells you the price, starts it withmax_creditsset to that price, and calls again afterretry_after_sfor the words. It sees every word with its time and speaker.Translate in text, and wait for your fixes
The agent translates each line itself and writes the lines into a brief note with
new_briefandedit, one note per language. You read the note in invideo and fix it. The agent sees your changes when it reads the brief again withread.Cut clean reference clips of the voice
The agent pulls the speaker's sound with
mediaaudio.extracton the clip's sequence URL, then cuts short clean pieces of speech withaudio.trim(start_time,end_time) on the extracted audio's sequence URL. Each makes a new file, so each call carries anidempotency_key. It sees one audio sequence per clip.Voice the approved lines in the same voice
The agent runs
find_modelswith the job in words, picks one, and reads its inputs withget_model. A good pick isseed_audio_1_0.ta2a. Each input takes the sequence URL of one short clip:reference_audio_1is required, andreference_audio_2andreference_audio_3are optional. The prompt tags each voice as@Audio1to@Audio3, matching the inputs, before its line in quotes. The model detects the language by itself. It callsgeneratewithdry_run: trueand tells you the price. Then it callsgeneratewithmax_creditsset to that price and anidempotency_key. It sees a sequence URL for each new voice and reads it withread; it never callsgenerateagain to check.
Build each language
The agent works on one branch per language and never edits main.
Branch and lay in the new voice
The agent branches with
timeline_historyactioncreate_branch, named after the language. It puts the new voice under the picture withmediavideo.replaceAudio, or places it on a new track and mutes the original withset_audio_muted. It sees the branch URL and a newstate_token.Re-sync the lips
The agent runs
find_modelswith the job in words, picks one, and reads its inputs withget_model. A good pick issync_lipsync_v3.video(inputsvideo,audio,sync_mode), andkling_lipsync(inputsvideo,audio) is another. It callsgeneratewithdry_run: trueand tells you the price, then runs withmax_creditsand anidempotency_key. It sees a new video sequence URL, reads it withread, and swaps it in withplace_mediamodereplace.Change the on-screen text and captions
The agent changes each text clip with
set_text. It pricesaudio.transcripton the new voice the same way and calls again afterretry_after_sfor the word times. Then it sendsadd_captions. It sees the clips with the new words and a newstate_token.Check each language, then export
The agent calls
captureat each line and each text, andcheck_timeline. It looks for a mouth out of sync and text that runs off the frame, because German and Spanish lines are often longer. Then it runsexporton each branch. It sees one export card per branch with statuscompleted.
The terminal version, for one language:
invideo agent register <project-url> --name <name>export INVIDEO_AS=<name>invideo media audio.transcript <sequence-url> --dry-runinvideo media audio.transcript <sequence-url> --max-credits <price>invideo context new-brief <project-url> --name "Ad 12 localization" \ --description "German and Spanish (Mexico) scripts for review"invideo media audio.extract <sequence-url> \ --idempotency-key ad12-extractinvideo media audio.trim <audio-sequence-url> --start-time 2 --end-time 9 \ --idempotency-key ad12-ref-1invideo gen models <project-url> --query "seed audio"invideo gen model <project-url> --model seed_audio_1_0.ta2ainvideo gen run <page-url> --model seed_audio_1_0.ta2a --dry-run \ -i prompt=@de-line-01.txt -i reference_audio_1=<reference-clip-url>invideo gen run <page-url> --model seed_audio_1_0.ta2a \ --max-credits <price> --idempotency-key ad12-de-voice \ -i prompt=@de-line-01.txt -i reference_audio_1=<reference-clip-url>invideo edit show <edit-url> --jsoninvideo branch new <edit-url> --name de --token <state_token>invideo gen models <project-url> --query "lip sync"invideo gen model <project-url> --model sync_lipsync_v3.videoinvideo gen run <page-url> --model sync_lipsync_v3.video --dry-run \ -i video=<sequence-url> -i audio=<de-voice-url>invideo gen run <page-url> --model sync_lipsync_v3.video \ --max-credits <price> --idempotency-key ad12-de-sync \ -i video=<sequence-url> -i audio=<de-voice-url>invideo edit show <branch-url> --jsoninvideo edit change <branch-url> --timeline T1 --token <state_token> \ --operations @de.jsoninvideo edit capture <branch-url> --at 1,4,8,12 --out ./framesinvideo edit check <branch-url>invideo edit export <branch-url> --format video --quality 1080 --jsoninvideo status <card-url> --waitinvideo notebook show <card-url> --out ./demv ./de/export.mp4 ./de/ad12_de_9x16_v1.mp4Where the values come from:
<project-url>is the ad's project, and<edit-url>is the ad's timeline URL, the one you give the agent.<name>is the name you choose ininvideo agent register.<page-url>is a notebook page;invideo notebook show <project-url>lists them.<sequence-url>is the speaker clip's sequence URL, frominvideo notebook show <project-url>or what the upload returns.<audio-sequence-url>is the URLaudio.extractreturns, and<reference-clip-url>the URLaudio.trimreturns.<de-voice-url>is the sequence URL the paid voice run returns.- Each
<price>is the dry run right before it. <state_token>comes frominvideo edit show <edit-url>, and on the branch frominvideo edit show <branch-url>; the main token is stale there.<branch-url>is the.branch_urlthatinvideo branch newreturns.<card-url>is.sequences[0]of what the export returns.invideo statuswaits and does not download;invideo notebook showsaves./de/export.mp4.- The file
./de.jsonholds a bare list of operations: theplace_mediaswap, theset_textchanges and the captions. The filede-line-01.txtholds the prompt with its@Audio1tag.
The voice call over MCP, as a dry run. The clip in reference_audio_1 is @Audio1 in the prompt. <page-url> comes from read on the project, and <reference-clip-url> is the URL audio.trim returns. The paid call is the same with dry_run removed, max_credits set to the price and an idempotency_key.
{ "url": "<page-url>", "model": "seed_audio_1_0.ta2a", "dry_run": true, "inputs": { "prompt": "@Audio1, warm and upbeat: \"Schluss mit kaltem Kaffee.\"", "reference_audio_1": "<reference-clip-url>" }}invideo has no one-shot dubbing model. For a one-shot draft, ElevenLabs Dubbing or HeyGen video translation makes one outside invideo, with no step where you fix the words first.
Cost and time
Edits and exports are free. Generation and transcripts cost credits. Paid stock costs a small credit charge, or one of your workspace's iStock licenses. Your agent runs a dry run first and tells you the price before it spends anything.
- Costs credits: the transcripts, the voice with
seed_audio_1_0.ta2a, and the lip sync. - Free: the translation the agent writes, your fixes, the branches, every edit,
captureandexport. audio.extractandaudio.trim: the dry run shows the price before anything starts.
One language takes under an hour, most of it the wait for lip sync and your review. Lip sync charges by the length of the audio. See Credits for prices.
If something goes wrong
Most problems come from a weak reference clip or a line that grew in translation.
| Symptom | Fix |
|---|---|
get_model or generate refuses the model as not found | Models retire. Run find_models with the job in words, use the best match, and tell the person which model you used. |
Exit code 6, or status is held | A safety review, not a refusal. Wait, then run invideo status <url> --wait again. Do not resend. If it ends failed, change the input the review flagged. |
| You have no consent to clone the voice | Use elevenlabs_text_to_speech.elevenlabs_tts_v4 with a voice from elevenlabs_text_to_speech_voices instead, priced the same way. |
| The new voice sounds like another person | Use cleaner reference clips: speech only, no music under it. Add a second clip as reference_audio_2. |
| The mouth drifts out of sync at the end | The new audio is longer than the video. Set sync_mode to remap, or shorten the line in text and voice it again. |
| German text runs off the frame | Make the text smaller with set_text_style, or split the line into two text clips. |
| The brand name is translated | Fix it in the brief note before the voice, and list it as a word that never changes. |
| CLI exit 3 on the branch | The token is from main. Run invideo edit show <branch-url> and use its state_token. |
| CLI exit 5 | A limit: credits, the --max-credits cap or a quota. Read the Fix: line, then tell the person and stop. |
| Avoid: voicing before the text is approved | Instead: wait for the person's fixes in the brief. A voice costs credits; text does not. |
| Avoid: editing main for a language | Instead: one branch per language with timeline_history. |
Prompts to try
Each prompt is a real variation of this job.
My best ad is in English. Make French, German and Italian versions. Keep"Brewly" in English. Put each translation in the brief, wait for my OK,then voice, lip-sync and export each one.Make a quick Spanish draft of this ad so I can show the team. Voice thetranslation, no lip sync. One export, named as a draft.The Japanese version is approved. Change only the on-screen price text tothe yen price I give you, add Japanese captions, and export.When to use something else
invideo does not check a translation for you. A native speaker or a translation service should read every line before the voice is made. For regulated claims in a market, ask your legal or compliance team. For a full studio dub with several actors, a dubbing studio or ElevenLabs Dubbing is a better fit.