AI startup Baseten raises $13B in Series F round

AI infrastructure firm Baseten has secured a $13 billion Series F funding round, cementing its position as a decacorn in the rapidly growing field of inference engineering. As highlighted by Baseten's Philip Kiely and Ali Taha, the discipline has evolved from a niche category into a critical pillar of AI deployment. Instead of focusing on model training, inference engineering addresses how to turn trained weights into fast, reliable, and affordable production APIs. This shift is driving massive optimization efforts, with practitioners regularly achieving performance gains of 20%, 100%, or even 200%, and sometimes making frontier models up to 10x faster.

To achieve these speeds, engineers are moving beyond standard setups. In a recent GLM-5.2 experiment, Baseten utilized a state-of-the-art quantization method based on Fourier Analysis on Groups. By quantizing more of the model, they achieved 20% higher throughput compared to existing configurations while matching downstream quality. This counterintuitive result occurred because quantization errors introduced in different layers actually canceled each other out. Additionally, the team has experimented with model retrofits, such as grafting Kimi’s vision encoder onto GLM-5.2 without altering the underlying language model, and managing massive models like Kimi K3, which require GB300-class hardware.

For practitioners, these advancements completely change how high-volume workloads are managed. When handling a 200,000-token request, systems now employ cache-aware routing to reuse previously computed KV cache, alongside disaggregating prefill and decode processes across different GPUs. Speculative decoding further accelerates generation by using a smaller draft model to predict tokens. While shared APIs are common, high-volume users pushing millions of tokens per hour find it more cost-effective to transition to dedicated deployments. This allows them to run custom NVFP4 quantization for Blackwell compatibility, optimize batch sizes, and avoid noisy-neighbor issues on shared endpoints.