|Docs

Use Runway with invideo

It depends on where the work is. Runway is strong at making and changing shots: Gen-4.5 for video generation, Aleph for video transformation, Act-Two for performances, and Workflows for custom pipelines. invideo adds the timeline where those shots become a video, with captions, music, cuts, and an agent that edits in the same project as you.

Use Runway to make and transform the shots. Use invideo to cut, caption, score and deliver them.

When to use both

Use both when you have Runway output and need a finished piece. Common cases:

  • A short film or ad from many shots. The shots live in Runway. The cut, the titles and the sound happen on a timeline.
  • A team review. Your agent edits on a branch, and the person reviews the branch in invideo.
  • Captions and music. The shots need captions in the brand style and music that clears rights.
  • Many versions. One cut needs a 9:16, a 1:1 and a 16:9 version, one branch each.

invideo also runs two Runway models inside a project: runway_gen45 (Runway Gen-4.5) and runway_act_two (Runway Act-Two). Your agent calls them with generate, so a missing shot lands in the same project as the rest.

Hand-off: from Runway into invideo

Download each shot from Runway as an MP4. Then your agent brings it in:

  • A public link. add_media with media_url.
  • A file on your computer. start_upload with the file_name, a PUT of the bytes to the upload_url, then add_media with the upload_id. In a terminal, invideo asset upload <project-url> <file>.
  • A file in a chat app. You drag the file into the chat. It arrives through file on add_media.

Each file becomes a sequence in the project, with its own sequence URL. On a timeline's URL, add_media also puts it in that timeline's media bin.

Then the agent finishes the video:

  • Cuts. place_media, split_clip, trim_clip and move_clip, with transitions from add_transition.
  • Captions. media audio.transcript, then add_captions with slides and words.
  • Music and sound. search_library in stock_music, or a model such as elevenlabs_sfx for a hit.
  • Upscale. media video.upscale with a model from its list, such as topaz_astra_2.

Hand-off: from invideo to Runway

When you want to change a finished cut in Runway, your agent runs export with format video. The result is a 1080p MP4. You upload it to Runway, for example as the input to a video transformation.

To send one shot back, the agent exports only that part with start and end in seconds.

Steps

Edits and exports are free. Generation and transcripts cost credits. Paid stock costs a small credit charge, or one of your workspace's iStock licenses. Your agent runs a dry run first and tells you the price before it spends anything.

Your agent runs a dry run first. For any call that costs 1 credit or more, it tells you the price and waits for your yes. A call under 1 credit runs without asking, and your agent tells you what it cost.

  1. Bring the shots in

    The agent calls add_media for each Runway shot. Expected result: one sequence URL per shot.

  2. Make a timeline on a branch

    The agent calls new_timeline, then makes a branch with timeline_history. Expected result: a timeline URL on a branch.

  3. Cut the shots in order

    The agent calls edit_timeline with place_media and add_transition. Expected result: a new state_token.

  4. Price the missing shot

    The agent reads runway_gen45 with get_model. It calls generate with dry_run: true and the real inputs, and tells you the price. Expected result: a price and no charge.

  5. Make the missing shot

    The agent calls generate with max_credits set to that price and an idempotency_key. Expected result: a thread URL, then a sequence URL when the missing shot lands. The agent reads that URL with read, and never calls generate again to check.

  6. Caption the speech

    The agent prices audio.transcript with dry_run, starts it, and calls again after retry_after_s for the words. Then it sends add_captions. Expected result: a caption clip.

  7. Check, then export

    The agent runs check_timeline and capture, then export with format video. Expected result: an export card with status completed.

The key MCP call, the full generate call that prices a missing shot before it runs:

{  "url": "<page-url>",  "model": "runway_gen45",  "inputs": { "prompt": "slow push in on the lighthouse at dusk" },  "dry_run": true}

<page-url> is a notebook page from read on the project URL. The agent reads the model's real inputs with get_model before this call.

The same shot from a terminal:

invideo agent register <project-url> --name <name>export INVIDEO_AS=<name>invideo gen model <project-url> --model runway_gen45invideo gen run <page-url> --model runway_gen45 --dry-run \  -i prompt="slow push in on the lighthouse at dusk"invideo gen run <page-url> --model runway_gen45 \  -i prompt="slow push in on the lighthouse at dusk" \  --max-credits <price> --idempotency-key shot-1

<project-url> comes from invideo project new or list_projects. <name> is the name you choose in invideo agent register. <page-url> is a page that invideo notebook show <project-url> lists. <price> is the price the dry run returns.

When to stay in Runway

Stay in Runway when the work is on the shot. Runway's own tools are the place to generate, restyle, extend and transform a clip, and to animate a performance with Act-Two. Its video editor and Workflows suit a pipeline that stays inside Runway from start to end.

invideo is the better place once the job is a sequence of shots with words and sound, edited by an agent and reviewed by a person.

If something goes wrong

SymptomFix
add_media with media_url is refusedThe link must be public. Download the file, then upload it with start_upload and add_media with the upload_id.
edit_timeline is refused as a conflict, or CLI exit code 3Someone's edit landed first. Read the timeline again with get_timeline, then send the change with the new state_token.
get_model or generate refuses the model as not foundModels retire. Run find_models with the job in words, use the best match, and tell the person which model you used.
Avoid: call generate again to check on a runInstead: read the thread or sequence URL with read. A second generate starts a second paid run.