wasl.tech

SGLang Consortium: Nvidia, AMD, Intel and Google Unite Behind Open-Source AI Inference

The Unlikeliest Alliance in AI

Picture the stage at Fort Mason in San Francisco. The lights come up on the opening keynote of a brand-new conference, and the names on the screen — Nvidia, AMD, Intel, Google, Meta — read like a list of companies that are supposed to hate each other. They were there to launch the SGLang Consortium.

For years, they have spent billions trying to out-build, out-price and out-market one another. On November 12, they all signed up to the same team.

The team is the SGLang Consortium, launched by LMSYS Org, the nonprofit behind the open-source inference engine SGLang. Eight founding members — Nvidia, AMD, Intel, Google, Meta, Cloudflare, Modal and RadixArk — will now help steer the roadmap of the software that serves AI models to the world.

It is the most unlikely alliance in AI infrastructure this year. And it says something important about where the real battle has moved: no longer only about who trains the biggest model, but about who can afford to run one.

Eight Rivals, One Table: Who Joined the SGLang Consortium

The announcement came in the opening keynote of the first SGLang Summit, held November 12–13 at the Fort Mason Center in San Francisco. More than 100 speakers from across AI hardware, models, infrastructure and applications took part.

The eight founding members span the whole AI stack. Nvidia, AMD and Intel are direct rivals in accelerators. Google and Meta build frontier models and run them at enormous scale. Cloudflare and Modal operate the infrastructure that serves models to users.

RadixArk, founded by SGLang creators Ying Sheng and Banghua Zhu, employs many of the engine’s core maintainers. Each founding member will designate a technical representative and join two roadmap meetings a year.

Members contribute through engineering expertise, deployment feedback, testing resources or joint development, and working groups can collaborate between meetings. LMSYS says it will publish public updates on the consortium’s direction and progress.

Crucially, the maintainers keep the final say. LMSYS retains responsibility for technical decisions and code acceptance, so the consortium guides the roadmap rather than taking over the project. The project stays open to the entire community.

What SGLang Actually Does — and Why Inference Is the New Battleground

SGLang is a full-stack inference engine for frontier AI, and part of the PyTorch ecosystem. Inference is the unglamorous half of AI: the work of actually running a trained model and answering questions, millions of times a day, for real users.

Training gets the headlines and the giant data-centre announcements. Inference gets the bills. According to Deloitte, inference is expected to account for roughly two-thirds of all AI compute in 2026, up from about half in 2025.

The inference engine determines how fast and cost-effectively that compute runs on each company’s hardware. SGLang already runs on hundreds of thousands of GPUs, according to LMSYS, powering large production deployments for Meta, Microsoft, Google, Amazon and others.

The engine runs across Nvidia GPUs, AMD Instinct accelerators, Intel Xeon processors, Google TPUs, Huawei Ascend chips and Moore Threads hardware. That cross-hardware reach is exactly why the consortium matters.

A model that only runs well on one company’s chips locks its owner into that company’s prices. An engine that runs well everywhere gives buyers leverage — and gives every hardware maker a reason to make sure it runs well on theirs.

The same logic applies to the open-source world beyond SGLang. The llama.cpp 1.0 release made a similar promise for local AI: a stable, versioned engine that developers can rely on instead of chasing nightly builds. Inference software is quietly becoming the layer where AI’s economics are decided.

What the SGLang Consortium Will Work On

The consortium named five focus areas: support for frontier models, reasoning workloads, multimodal inference, distributed serving and hardware optimisation. In plain language, that means making new models work on day one, making multi-step reasoning cheaper, and handling text, image and video together.

It also means spreading work across many machines and squeezing more performance out of every chip. The launch arrived alongside a version 1.0 milestone: more than 330 adopting organisations and day-zero support for 217 models.

A concrete release landed the same day: SGLang-Neuron v0.1.0, co-developed by RadixArk and AWS Annapurna Labs, brings the serving stack to AWS Trainium through a native PyTorch plugin in public preview.

Intel and RadixArk separately announced they will optimise SGLang for Intel CPUs, GPUs and accelerators, including support for Intel Advanced Matrix Extensions. AMD had already said in August that its Hyperloom GPU optimisation tool would support SGLang.

“The teams running SGLang at the largest scale see what inference needs next before anyone else does,” said Ying Sheng, co-author of the original SGLang paper and CEO of RadixArk. “Software approaches to improve AI efficiency will be incredibly important.”

Ian Buck, vice president of hyperscale and high-performance computing at Nvidia, said the company was proud to be a founding member. “Better software helps developers get more performance from NVIDIA compute,” he said. “Lower inference costs make AI more accessible to organizations everywhere.”

The Catch: A Security Scare Days Before the Launch

The consortium did not arrive in a vacuum. Days before the announcement, a security flaw in SGLang was exploited at NEAR AI, according to reports — a reminder that the software serving the world’s AI models is itself a target, and that openness cuts both ways.

That context makes the consortium’s governance choices look deliberate rather than decorative. Keeping technical decisions with the maintainers, while giving the biggest deployers a formal channel for feedback, is a way to move fast on security without letting any single vendor steer the project toward its own hardware.

It also sets a test the consortium cannot dodge. A shared roadmap is only worth something if it produces merged code, reproducible cross-platform benchmarks and cheaper serving in practice — not just joint press releases.

The announcement itself contains no financial commitments, no formal voting bylaws and no standardised performance metrics. Nobody promised a breakthrough. They promised to work together on the plumbing, twice a year, in public.

Why This Matters for Anyone Running AI

For developers and companies that run models in production, the consortium is good news with a long fuse. If it works, the same engine will run faster on more hardware, new models will be supported on release day, and the cost of answering each user query will keep falling.

Those savings decide which AI products survive and which quietly shut down. For the hardware makers, the stakes are just as high. Google’s TPU 8tx and TPU 8i Max chips, announced days earlier, are a direct challenge to Nvidia’s dominance — yet Google joined the same consortium as Nvidia in the same week.

AMD’s EPYC 6 “Venice” server CPUs, with up to 256 cores per socket, are built for the same AI-agent era. Every one of these chips is only as useful as the software that runs on it. That is the quiet shift the SGLang Consortium captures.

The chip war is not going away, but the winners may increasingly be decided by software — by who makes the serving layer faster, cheaper and more open. That is why eight rivals agreed to sit at the same table.

What Happens Next

The real test of the SGLang Consortium is not this launch but the next one. Twice a year, the founding members will meet with the maintainers to set roadmap priorities. LMSYS has promised public updates on direction and progress.

Watch for three things. First, whether the consortium ships measurable improvements — faster serving, lower cost per query, day-one support for new models — rather than announcements. Second, whether it produces public cross-vendor benchmarks, so buyers can compare hardware honestly.

And third, whether the security lessons from the NEAR AI incident turn into stronger defaults for everyone running SGLang. If those three things happen, that Fort Mason photograph will look less like a curiosity and more like a turning point.

According to Business Wire, LMSYS Org announced the consortium in the opening keynote of the inaugural SGLang Summit in San Francisco.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top