Mercury: Diffusion Large Language Models Are Here

Mercury by Inception Labs is changing how AI generates text. Unlike traditional models that build text word by word Mercury produces full responses at once then refines them step by step. This makes it 10x faster and cheaper than older systems with speeds over 1000 tokens per second on standard NVIDIA H100 GPUs.

Regular AI models build text from left to right. Mercury flips the script—working like image AI tools that start with noise and refine it into clear output. Instead of running 75+ steps per response it needs just 14 to deliver polished answers.

This means instant responses making AI-powered tools like coding assistants way more useful.

Key Features

• Parallel Text Generation. Creates full responses instantly instead of token-by-token.
• Iterative Refinement. Boosts quality with step-by-step improvements.
• Super Fast. Over 1000 tokens per second on standard hardware.
• Better Reasoning. More precise answers thanks to holistic corrections.
• Error Fixing. Adjusts mistakes while refining text.
• Easy Deployment. Available via API or on-premise for businesses.
• Code Generation. Mercury Coder specializes in fast efficient coding AI.

Why It Matters

Speed isn’t just a bonus—it’s a game changer. Faster AI means:

• Smarter AI agents. They react quicker and handle complex tasks better.
• Real-time applications. Tools like coding assistants become actually useful.
• Stronger reasoning. AI gets more time to refine and improve responses.
• More accessible AI. No need for expensive custom hardware.

A Shift in AI Thinking?

AI expert Andrej Karpathy sees this as a big shift. On X (Twitter) he shared:

"This is interesting as a first large diffusion-based LLM... Most of the LLMs you've been seeing are ~clones as far as the core modeling approach goes... Diffusion is different—it doesn’t go left to right but all at once... If you look close enough a lot of interesting connections emerge."
— @karpathy

For years diffusion worked best for images and videos while text stuck with the old autoregressive method. Mercury challenges that showing a new way to process language—one that might change AI text forever.