Buoyant Announces Kinference: An Open Source Inference Serving Platform for Kubernetes

Kinference [kay-inference] helps organizations reduce token spend and deliver policy-driven inference to every developer, application, and agent by turning open weight models and GPUs into a self-hosted inference platform.

SAN FRANCISCO, Wednesday, November 4, 2026 — Buoyant, the creators of Linkerd, today announced the general availability of Kinference [kay-inference], an open source inference control plane for Kubernetes. Building on top of the open source vLLM inference engine, Kinference enables organizations to deliver safe, reliable, and cost-effective inference across multiple models, GPUs, and machines, while providing the governance and policies necessary for enterprise use at scale.

As adoption of generative AI matures, organizations are increasingly moving inference in-house, running LLMs, often with open weight models, on GPUs that they control. This approach allows these organizations to decrease token spend, build custom AI capabilities, handle regulatory concerns, and “own their intelligence” by having full operational control of this critical resource. The use cases for Kinference include:

• Software engineering: generative AI used to produce and review code 
• Application intelligence: generative AI for specific application features
• Agentic tasks: generative AI for autonomous or semi-autonomous tasks

Central to self-hosted inference is Kubernetes: 66% of organizations hosting generative AI models use Kubernetes for some or all of their inference (CNCF, 2026). To date, inference infrastructure for Kubernetes has largely focused on LLM performance and support for new models. To this existing foundation, Kinference now adds a unified layer of routing, load balancing, and policy-driven control. Kinference’s rich feature set allows organizations to provide an true, internal “inference platform” capable of:

• Inference guardrails and controls. Kinference allows platform owners to define boundaries around performance, cost, and compliance; developers are then given maximum flexibility for consuming inference within those constraints.
• Lowered cost of inference. Kinference improves GPU utilization and inference efficiency by selecting endpoints that minimize GPU work, by taking into account KV cache data on the GPU, queueing status, and other real-time operational characteristics.
• Intent-based routing. Kinference provides semantic classification to enable policies such as “send simple prompts to the faster, smaller model”.
• Burst to cloud. Kinference can optionally send inference requests to cloud providers (including frontier models) under conditions defined by the platform owner, which can include GPU pool saturation, usage by specific tenants, or characteristics of the inference request itself.
• Per-tenant usage and cost attribution. Kinference can track GPU usage per tenant, group, or team, and provide the resulting cost attribution, allowing organizations to have a full picture of inference spend.

Kinference builds on Buoyant’s long history of providing mission-critical infrastructure to organizations around the world. Following the same design philosophy as Linkerd, Kinference is lightweight, easy to deploy, and can be run without a team of specialists. Kinference is agnostic to LLM model, cloud provider, and GPU type. 

In public Kinference benchmarks, when compared to standard Kubernetes load balancing, Kinference was able to cut median time-to-first-token (TTFT) by 95% and median end-to-end latency by 45% of requests on one workload type.

"Enterprises need a plug-and-play solution to private AI that allows them to handle the wide variety of use cases and tenants within their organization in a unified way,” said William Morgan, CEO and founder of Buoyant. “We’re excited to launch Kinference as a reliable, high performance, secure option that platform engineers can easily adopt, no matter what the model or compute environment."

To see Kinference in action, join Buoyant for a livestream on Thursday, November 19, 2026, at 9 AM PT. Register here: https://www.buoyant.io/register/free-tokens-for-everyone 

Availability. Kinference is available today under the Apache 2.0 license. See kinference.ai for more information. 

About Buoyant
Buoyant is the creator of Linkerd, the category-defining service mesh, and Kinference, the open source self-hosted inference control plane for Kubernetes. Buoyant is dedicated to ensuring that platform teams can run, manage and observe mission-critical workloads without compromising performance or security. For more information, visit buoyant.io.