vedit is a video editor you talk to. Drop a clip into a folder, describe the edit in plain language to an AI coding agent, and a local engine handles the cutting, captioning, reframing, graphics and final render.
The Idea
Editing a talking-head clip is mostly the same mechanical passes every time: trim the pauses and filler words, burn in captions, punch in on key lines, reframe for vertical, add a title. Only a few decisions need real judgment, like which take to drop, how to phrase a translation, or where a graphic should land.
So the work is split. A deterministic Python engine does everything mechanical. The agent only handles judgment and reads as little as possible. The engine reduces a video to a short file of numbered, timestamped lines, and the whole edit is described in a small plan.json that the agent changes with small, targeted edits.
What We Built
-
✦
Word-level transcription faster-whisper (large-v3-turbo by default) transcribes the clip with word timestamps and voice-activity filtering. It runs on the GPU and falls back to the CPU automatically. The slower large-v3 model is available for better Hindi and Hinglish accuracy.
-
✦
Silence and filler cuts Pauses longer than 0.45 s are cut using the word timings. English and Hindi filler words are dropped, and any bad take can be removed by its line number. Every cut is snapped to a whole frame.
-
✦
Captions in any script There are four caption styles: pop, karaoke, single word and boxed line. Translated captions are a plain text file mapping line numbers to text. The engine detects the script and picks a matching Noto font, covering Devanagari and other Indic scripts as well as Arabic, Hebrew, Thai and CJK.
-
✦
Face-aware reframing and zooms OpenCV's YuNet detector samples the speaker's face twice a second. Vertical, square and 4:5 crops follow the face, but the crop only moves at cut points, so viewers don't notice the reframing. Auto-zooms land on alternate long lines and push in harder on lines that end in a question or exclamation.
-
✦
Motion graphics as code Titles, lower thirds, pop text and calls to action are React components rendered by Remotion into transparent ProRes 4444 clips. The clips are cached by their properties and overlaid on the output timeline. The agent can also write and register new components.
-
✦
Voice enhancement An ffmpeg audio chain applies a high-pass filter, RNNoise and FFT denoising, a two-band EQ and compression, then normalises loudness to −14 LUFS.
-
✦
One filter graph, one render Cuts, crop, zooms, captions, graphics and an optional progress bar compile into a single ffmpeg filter script. It encodes with NVENC and falls back to libx264. A preview flag renders just the first few seconds.
The Stack
The engine is a Python 3.12 package with a small CLI: prep, render, auto and info. It is built on faster-whisper with CUDA libraries, OpenCV, NumPy and ffmpeg. Graphics use Remotion 4 with React 19 on Node. A one-time setup script installs ffmpeg, Node and uv, then downloads the speech model, the face and denoise models, and the fonts. The Python environment lives outside the synced project folder because the GPU libraries alone are about 1.5 GB. On the agent side, a Claude Code skill sets strict token rules. The agent never opens the transcript, face or metadata files, and reads the line list only when an edit needs judgment. Plain requests take a one-command fast path with no reading at all.
What We Learned
The pipeline runs end to end. A 37-second 1080p test clip went through prep and render and came out as a 9:16 vertical edit with silence cuts, face-tracked crop, pop captions, auto-zooms, a title card, a pop graphic and a progress bar. Putting all the mechanics in the engine keeps the agent's share of the work small. Translation, take selection and graphic timing are the only edits that need it to read the transcript lines. It is still a version 0.1 prototype with no automated tests. Hindi words can be misheard on the default model, so those edits are re-prepped with large-v3. Graphics depend on the Node install being present.
Need something similar? Reach out. We build AI tools that pair an agent with a deterministic engine underneath.