Motion Inversion AI Video Tool

Motion Inversion is a new free, open-source AI tool that transfers the movement patterns from one video to another. It uses motion embedding techniques to capture the motion in a source video and then maps it onto a new video, maintaining smooth and natural motion.

• Online Demo: Try it on Hugging Face
• Source Code: Get it on GitHub
• License: Free and open-source under the Apache 2.0 license.

Here are some examples of how Motion Inversion works (view the video here):

1. Orbit Shot Applied to Rabbit Video

   • Original Video: A camera moves around a subject in an orbit.
   • Output Video: A rabbit designed in a low poly art style.
   • Result: The camera's circling motion from the original video is applied to the rabbit, creating the effect of the rabbit being encircled by the camera.

2. Crane Up Shot Mapped onto Island Scene

   • Original Video: An upward camera movement.
   • Output Video: An island view near the ocean.
   • Result: The upward movement reveals the island as if the camera is lifting upward.

3. Santa's Dance Mapped to a Robot

   • Original Video: Santa dancing.
   • Output Video: A robot replicates the dance.
   • Result: The robot mimics Santa's dance moves, matching the rhythm and style.

4. Kittens Playing Transferred to Monkeys with a Coconut

   • Original Video: Two kittens at play.
   • Output Video: Monkeys interacting with a coconut.
   • Result: The monkeys imitate the playful movements of the kittens in the new video.

5. Orbit Shot Mapped onto a 3D House

   • Original Video: An orbit shot.
   • Output Video: A 3D-style house.
   • Result: The camera's circular movement is applied to the house, making it look as though it's being circled.

6. Santa's Dance Transferred to a Skeleton in a Suit in Autumn

   • Original Video: Santa dancing.
   • Output Video: A skeleton in a suit dances in a fall setting.
   • Result: The skeleton follows Santa's dance moves, creating a similar performance.

How Motion Inversion Works

The Motion Inversion approach introduces a new way to handle motion customization in video creation. It focuses on enhancing how motion is represented in generative models. The key is using "Motion Embeddings," which are specific motion representations derived from a video. These embeddings connect with the video generation model's temporal processing, allowing the AI to adjust motion while keeping the image quality intact. By separating motion details from visual elements, this technique enables a more accurate and efficient application of motion.

So Motion Inversion uses a system of "motion embeddings" to capture movements from a reference video. These embeddings are fed into a video generation model, which uses them to apply the captured movements naturally across frames. The tool allows filmmakers and creatives to control video motion using reference clips.

You can read more technical details in the paper published by the developers.