Tencent's Hunyuan Video: New AI for Text-to-Video Generation

Tencent has unveiled Hunyuan Video, an open-source AI text-to-video generator that’s raising the bar for video creation. With 13 billion parameters and state-of-the-art technology, this model produces top-tier video content from simple text prompts, rivaling even the best closed-source solutions.

What is Hunyuan Video?

HunyuanVideo is a cutting-edge video generator powered by advanced AI technologies. Trained using unique strategies like image-video joint learning, efficient infrastructure, and large-scale data curation, it’s designed to deliver unmatched text alignment, motion quality, and visual results. Evaluations show it outshines leading models like Runway Gen-3 and Luma 1.6.

Key Features That Set HunyuanVideo Apart

1. Unified Image and Video Architecture

   • Utilizes a hybrid "dual-stream to single-stream" Transformer design for seamless text-video integration.
   • Captures complex relationships between visuals and text for improved video quality.

2. Advanced Text Encoding with MLLM

   • Features a Multimodal Large Language Model (MLLM) with a decoder-only structure for better text-to-video translation.
   • MLLM enables zero-shot learning, precise detail capture, and enhanced reasoning over alternatives like CLIP and T5.

3. Efficient Compression with 3D VAE

   • Reduces data size using CausalConv3D, allowing training at full resolution without compromising speed or quality.

4. Prompt Rewrite Options

   • Offers Normal and Master modes to optimize user prompts for varying levels of video detail and creativity.

Performance Benchmark

HunyuanVideo underwent rigorous tests, generating over 1,500 videos for comparison against top models. Results highlight superior motion quality, text alignment, and overall video clarity.

What You Need to Run It

• GPU Requirements: 60GB minimum memory (recommended 80GB).
• Tested on NVIDIA GPUs like H800/H20.  Operating System: Linux.

An NVIDIA GPU with CUDA support is required.

Redditors Reactions

Here's the concise gist of Reddit buzz about Tencent HunyuanVideo:

• High GPU Demands: Requires 60–80GB VRAM, sparking jokes about GPU wishlists.
• NVIDIA Criticism: Users blame NVIDIA for limited VRAM, monopolizing CUDA, and forcing upgrades.
• Optimization Hope: Anticipation for optimized versions to fit consumer GPUs like the RTX 4090.
• Impressive Quality: Praised for video generation and potential filmmaking uses, though VRAM needs limit accessibility.
• Strategic Open Release: Seen as a move by Tencent to outpace competitors with free models while reserving premium options.
• Mixed Feelings: Awe at the innovation tempered by frustration over hardware constraints.

Tencent Hunyuan Community License Agreement

On Github, you can find the full text of the license, I've just processed it through GPT-4o to extract the gist of terms:

• Territorial Restrictions: The license is expressly limited to territories outside the European Union, United Kingdom, and South Korea.

• Usage Rights: It grants a non-exclusive, non-transferable, royalty-free limited license to use, reproduce, distribute, and create derivative works of the Tencent Hunyuan materials, but only in accordance with the terms specified in the agreement and the Acceptable Use Policy.

• Distribution Conditions: Users can distribute the Tencent Hunyuan Works under certain conditions, such as providing a copy of the agreement to recipients and including specific notices.

• Commercial Limitations: Entities with more than 100 million monthly active users must request a separate license from Tencent.

• Use Restrictions: The license includes specific prohibitions, such as using the works to improve other AI models, using them outside the defined territory, or violating the Acceptable Use Policy.

• Intellectual Property: Tencent retains ownership of the Tencent Hunyuan Works and their intellectual property rights.

• Termination Clause: Tencent reserves the right to terminate the agreement if the user breaches any terms.