Google DeepMind Converts Gemma 4 into DiffusionGemma

Google DeepMind has released a technical report detailing DiffusionGemma, an experimental text diffusion model built by retrofitting the existing Gemma-4-26B-A4B model. Instead of generating text one token at a time like traditional autoregressive models, DiffusionGemma refines blocks of 256 tokens in parallel. Running on an Nvidia H100 accelerator, the model achieves an output speed of approximately 1,500 tokens per second. The engineering team accomplished this conversion using less than 10 percent of the original model's training token budget.

The conversion process relies on two distinct training stages to balance quality and speed. First, the model learns to reconstruct noisy text blocks. Second, Google employs a combined phase of reinforcement learning and sampler distillation, which they call SD-RL. This combined approach raised the model's scores on reasoning benchmarks by an average of 10 points while nearly quadrupling the number of tokens processed per compute step. Additionally, DiffusionGemma's responses are about 50 percent shorter, further accelerating output.

Because it uses diffusion, the model benefits from bidirectional reasoning, allowing it to correct its own mistakes during the denoising process before finalizing output. For example, after minimal fine-tuning, DiffusionGemma correctly solves close to 85 percent of Sudoku puzzles, a task where the base model fails entirely. It also completes structured outputs like JSON or code repairs in just two to three refinement steps. Users can also switch the model back to standard word-by-word generation when needed.

Despite these advantages, DiffusionGemma still trails the autoregressive Gemma 4 on absolute quality benchmarks. It occasionally gets stuck in repetition loops and struggles on multimodal tasks. Furthermore, its speed advantage over standard models diminishes once concurrent requests reach about 32 users. Released under an Apache 2.0 license on Hugging Face, the model follows Google's May 2025 Gemini Diffusion demo. It is already being used by the startup Interfaze for multilingual speech recognition and in a research project for interactive radiology report generation.