AI Video Composer: Simplify Video Editing with Natural Language

Calculating... Comments

X user @cocktailpeanut has shared a fascinating tool originally brought to life by @victormustar: AI Video Composer. This tool lets you use simple natural language to combine your video, image, and audio assets into new creations, powered by Qwen2.5-Coder from Alibaba Cloud and FFMPEG.

What AI Video Composer Does

With AI Video Composer, you can drag and drop your assets and write plain prompts like:

"Make the image the background with an audio waveform in the center."
Converts to:
ffmpeg -i ai_talk.wav -i bg-image.png -filter_complex [0:a]channelsplit=channel_layout=stereo[left][right];[left][right]amerge=inputs=2,showwaves=s=1080x200:mode=line:colors=white[left_wave];[1][left_wave]overlay=(main_w-overlay_w)/2:(main_h-overlay_h)/2 -c:v libx264 -c:a aac -y output.mp4
"Make an MP4 slideshow with 2-second photo intervals and add the audio as background."
Converts to:
ffmpeg -framerate 0.5 -pattern_type glob -i cat*.jpeg -i heat-wave.mp3 -c:v libx264 -c:a aac -shortest -y output.mp4
"Add overlay to the video."
Converts to:
ffmpeg -i waterfall.mp4 -i waterfall-overlay.png -filter_complex overlay=0:0 -c:a copy -y output.mp4

Essentially, you describe what you want, and the tool creates the matching FFMPEG commands for you.

Why It’s Cool

FFMPEG is incredibly powerful but hard for most people to use directly. This tool simplifies everything by letting you "talk" to FFMPEG to handle almost any editing task. It’s like having a command-line video wizard at your fingertips. You can check out many usage examples in the original tweet https://x.com/cocktailpeanut/status/1861518841886511126

AI Video Composer can:

Combine two videos side-by-side.
Make videos black-and-white.
Add overlay text.
Turn photos into a video (without complex animations).
Flip or crop videos.
Speed them up or slow them down.
Add soundtracks.
Reverse clips.
Perform most of FFMPEG's advanced features, like color grading and audio visualizations.

At this point the tool would probably benefit from a large list of sample prompts since most people might not know what FFMPEG's capabilities are.

Downsides

It’s not perfect since it relies on AI to generate FFMPEG commands. Sometimes, commands might fail or need tweaking. If that happens, switching to OpenAI’s API might help.

Some X users commented that their commands failed with error message "Error FFMPEG generated command is not valid. Please try something else." I've actually ran into the same error when I simply changed the picture from the demo for my own without changing the prompt. It seems to be struggling with odd resolutions, but 1080 x 1080 is working fine. Square videos also do well.

Things like these are to be expected though, the software will likely get improved over time.

Where to Get AI Video Composer

Try the demo on Hugging Face. It’s fully open-source and works across platforms. You’ll need Ollama installed, but everything is streamlined for a one-click setup via Pinokio.