S02 E07 - The State of Cloud Native AI and What’s Still Evolving

Cloud native AI still doesn't have an agreed-upon definition, but Ron Petty helped write the CNCF's attempt at one as part of the TCG AI (Technical Community Group for AI). Outside that work, Petty consults on cloud native technology at rx-m, one of the companies involved in founding the CNCF. Its team also built the Certified Kubernetes Administrator (CKA) exam, and (fun fact) when they took it themselves, everyone failed the first time. Ron joined William Morgan on the latest episode of the AI Kubernetes Show to talk through what's actually settled in this space and what isn't.

What does cloud native AI mean? 

Petty contributed to the CNCF AI Working Group's white paper attempting to define cloud native AI, published a few years ago. While most people collapse it into one definition, Petty differentiates between cloud native AI (building and training models using cloud native infrastructure) and AI for cloud native (using AI to operate cloud native systems). It's genuinely bidirectional.

He also pushes back on tying the cloud native AI definition too tightly to Kubernetes. If you are using containers and they are scaling somehow, possibly on infrastructure you don't own, that's cloud native. Kubernetes is the most popular way to achieve that, but certainly not the only one. Petty cited an industry estimate that roughly 60% of major AI-building operations run on Kubernetes today.

The problem AI hasn't fixed yet: certificate management

Even before AI, certificate management and rotation have been operational challenges. Root certs, intermediate certs, leaf certs, organization units, and common names. Every team is solving the identical problem with a different tool, and teams still get it wrong.

"If certs were so important and if encryption was so important, why isn't it automated out of the box in your operating system? Why do humans need to be involved at all?" asked Petty.

According to Morgan, that's exactly why Linkerd defaults to mutual TLS, a decision the project made as early as 2018 or 2019. Even with that default in place, Linkerd is still shipping infrastructure to automate things like trust anchor certificate rotation across clusters. If you make the secure path hard, people route around it. Make it the path of least resistance, and they take it. That's the logic behind why Linkerd defaults to mTLS instead of asking operators to opt in.

Three buckets of AI workload, and the one nobody's built tooling for

Morgan proposed a simple split for AI workloads on Kubernetes: training, inference, and agents (what you build on top of inference). Petty agreed with it with one addition. Hosting a model does nothing until something calls it, so the real center of gravity right now is the layer that consumes inference, not the layer that produces it.

More important than classification is verification, though. Ron isn't worried about getting one answer wrong. He's worried about a workflow that makes 50 corrections, each of which needs its own verification, and no tooling exists yet to check the combination. "That whole game is completely underdeveloped," he said. Whether or not AI is involved, most organizations still can't hit a single button to fully reproduce their own environment in 2026.

AI agents must be treated like a new hire. Trust but verify. Don't extend more trust to an LLM on day one than you would to a new colleague. The difference, per Petty, is that the LLM is faster than the human, which raises the volume of output that now needs checking.

MCP got adopted first, but nobody said it was well designed

Model Context Protocol (MCP) is the clearest example of a fast-moving standard landing before the ecosystem had time to agree it was good. MCP specifies how tools get proxied to agents. At a glance, it looks like it's reinventing REST: Hit an endpoint, get a list of resources, and execute them. First-mover advantage explains most of its traction over prior, less popular specs.

Morgan was even blunter. As someone who's spent years building proxies, his view is that MCP's design skipped security considerations at the protocol level and left every implementer to parse and handle the fallout.  

MCP's spec added "sampling," a reverse use of an LLM inside the protocol, but Anthropic's own Claude hasn't implemented it. According to Petty, that's a signal that the feature might not be as load-bearing as the spec implies, though Morgan suggested a more cynical intepretation: a hard-to-meet spec that a first mover doesn't have to implement itself while everyone else spends cycles building to it. 

MCP now sits under the Agentic AI Foundation (AAIF), a Linux Foundation project parallel to the CNCF, the same way Kubernetes sits under the CNCF for governance. 

What does CNCF AI conformance on Kubernetes actually require?

The CNCF does have a conformance program for AI on Kubernetes, but according to Petty the bar is pretty low by design and is based on self-reporting. You check yourself against roughly 15 criteria, and someone evaluates the submission.

Most of those criteria are about GPU management: dynamic resource allocation, GPU isolation, and GPU monitoring. The rest assumes your platform already clears standard Kubernetes conformance like autoscaling, metrics gathering, and the baseline practices any production cluster should have.  

The bar is intentionally high-level, because if you get too specific and you lock the industry into one implementation, it cuts against the "survival of the fittest" approach the CNCF wants for this space.

Long-running agents get more attention than they deserve

Many people in the agent space claim that long-running agents are a fundamentally different and harder problem than short-running ones. Petty disagrees about where the value actually is. Most agent tasks deliver their value in one short execution window regardless of how long the agent nominally runs. Booking a plane ticket might involve rules set a year in advance, but the actual work happens in a single short slice of time when the booking executes.

An agent is a workflow at any timescale. Elevating long-running to its own category of concern, instead of treating duration as one property among many, is giving it "an unnecessary position of power."

Agent-to-agent commerce is the use case with real money behind it

Petty pointed to Project NANDA, an MIT Media Lab effort led by Professor Ramesh Raskar, as the most concrete answer he has seen to the question of why would anyone build the infrastructure to support a trillion agents? NANDA's working answer leans on a reverse auction model: an agent with a budget and a set of constraints (a travel budget and a set of guardrails) negotiates directly with other agents to execute a task, without a human in the loop until the transaction needs to happen.

This isn't NANDA's invention. Google has used reverse-auction-style mechanics before. What Petty finds compelling is that commerce gives agent-to-agent interaction an economic reason to exist, rather than staying a thought experiment. But none of this works without fairness guarantees between agents, and those don't exist yet. 

The practical starting point: K8sGPT and a whiteboard

If you want to learn about Kubernetes and AI, you need to learn Kubernetes first. If you can't draw the system on a whiteboard, you don't understand it yet, and no amount of AI tooling changes that.

Once you have that foundation, Petty recommends K8sGPT as an entry point. It's stripped down to its actual mechanism and takes existing, pre-AI Kubernetes checks (the same hard-coded checks that catch things like a missing image), packages the matching events, and sends them to an LLM to explain in plain language. Some people dismiss it as too simple, just text going to an LLM with a prompt. Petty believes that simplicity is exactly why it works.  

His broader point about where AI actually helps today isn't about building autonomous systems. It's about using AI to interrogate what you already have. Does the flow of your logs match what you'd expect by hand? That kind of exploration got a lot cheaper, and Petty argues most people are skipping the free, easy version of that in favor of chasing something more ambitious.

Where this leaves platform teams today

Nothing in this conversation is a finished standard. The cloud native AI definition is one contributor's framing inside a still-evolving white paper. The AI conformance program is self-reported and deliberately loose. MCP is the dominant protocol by adoption, not by design consensus, and even its authors haven't fully implemented their own spec. According to Petty, verification and validation tooling for AI-driven changes doesn't exist yet at the level the industry needs, and that's where he expects the real engineering work over the next several years, whether or not it gets branded as AI.

‍

FAQ 

What is cloud native AI?

Cloud native AI can be understood in two ways: cloud native AI (building models on cloud native infrastructure) and AI for cloud native (using AI to operate cloud native systems). 

What does CNCF's cloud native AI conformance program require?

It's self-reported against roughly 15 criteria. About 10 cover GPU management (dynamic resource allocation, isolation, monitoring). The rest assume standard Kubernetes conformance: autoscaling, metrics gathering, and an AI router on the cluster.  

What is K8sGPT and how does it use AI?

K8sGPT takes existing hard-coded Kubernetes checks (like a missing image) and sends the matching events to an LLM for a plain-language explanation. It's a single binary that helps junior engineers diagnose issues faster.  

Do long-running AI agents need different guardrails than short-running ones?

Not really. Most agent tasks deliver their value in one short execution window regardless of nominal duration. Duration is one property of a workflow, not a separate risk category.  

‍