S02 E06 - Treat Testing as a Platform Service on Kubernetes

If there was one central piece of advice from the conversation between Ole Lensmar, CTO of Testkube, and William Morgan on the AI Kubernetes Show, it's that testing should become a platform service on Kubernetes. 

Lensmar has been building software since the late 1990s, mostly in testing and APIs. He was a steward of Swagger before it became the OpenAPI Specification, created SoapUI, and worked at DataWire before it became Ambassador Labs. About 5 years ago, he and co-founder Dimitri Fonarev started Testkube, built around the idea to use Kubernetes as the execution engine for test workloads.

Why decouple test execution from CI/CD

Testkube's origin story starts with a familiar complaint. As CI/CD tooling multiplied (Jenkins, then GitHub Actions, GitLab, and Azure DevOps), tests got buried inside pipelines and QA teams became second-class citizens in their own operational space.

To address that, Lensmar pulls test execution out of CI/CD and runs tests as Kubernetes workloads instead. Testkube runs your existing tests, Selenium, Playwright, K6, JMeter, Postman, rather than asking teams to adopt a new framework. Containerize the test, and Kubernetes runs it as a job.

That decoupling also means tests can be triggered by more than a pipeline step. GitOps tools like Argo and Flux, progressive delivery rollouts, or a production signal can all kick off a run. Shift-left and shift-right testing both depend on that flexibility: some tests need to run before code merges, while others need to run against production traffic.

Who owns testing: developers vs. the platform team

According to Lensmar, functional tests, unit tests and most integration tests are the developer's job. The platform team should not have to review your test assertions.

Non-functional testing like performance, security, or chaos works differently. The platform or ops team usually makes sure infrastructure can handle a given load. But when a load test fails, engineering is typically on the hook to fix it: a memory leak, a regression, or code that can't scale the way it's supposed to. The platform team runs the test, but the dev team owns the consequence of failing it.

Testkube aims to change that. They give platform teams a framework and guardrails defining where tests run, where results land, and which tools are sanctioned, while letting application teams pick their own tools inside that boundary. Whether a platform team mandates a single tool or leaves it open depends on how much governance the organization wants.

The advice: treat testing as a platform capability, like CI and CD

Lensmar recommends treating testing and quality as a platform service the same way you already treat CI and CD. Continuous integration, continuous deployment, and continuous testing should get equal billing when you design a platform.

While delegate testing entirely to individual teams might work ok early on, as application complexity grows, the lack of guardrails becomes a real cost. And retrofitting governance onto testing that's already scattered across a dozen teams is a much harder problem than building it in from the start.

3 ways AI is changing testing on Kubernetes

While the conversation covers a lot of AI-and-testing ground (you should definitely check out the full conversation), these are the three concrete points you can act on:

1. AI-generated code needs more test coverage, not less

Code that compiles isn't the same as code with full system context. AI writes reasonable unit tests but doesn't have full system or application context. A human still needs to check that AI-generated tests validate the actual business requirement, not just whatever behavior the generated code happens to produce, bugs included.

2. More tests mean slower pipelines but there are 2 solutions 

One solution is more Kubernetes infrastructure. You can just run more tests in parallel. The other is what Lensmar calls intelligent test selection, using AI to map dependencies across a change and figure out which tests need to run. While that's manageable for a monolith, it gets hard fast across 10, 20, or 50 microservices with external API dependencies. Lensmar's recommendation? Never rely only on an AI-selected subset, and keep a recurring full run in the mix, because the selection will sometimes be wrong, …but so will humans.

3. Reading failed test results is where AI can start helping you today

Sifting logs and traces to explain a failure, using a Kubernetes MCP server or a Grafana MCP server to pull context, is where AI is almost best positioned to immediately bring value in QA. It doesn't require trusting an LLM's judgment about what to test, only its ability to correlate signals a human would otherwise chase by hand.

One note on cost. Log analysis at volume doesn't need a frontier model. Lensmar's team uses smaller or open source models for that work, sometimes asking a frontier model to write the prompt that a cheaper model then runs.

Testing AI agents is becoming its own discipline

Beyond testing code that AI helped write, there's a separate and growing need to test the AI components themselves: agents, chat interfaces, anything non-deterministic that's part of the application. That means evals for accuracy, relevance, and toxicity as prompts and models change over time. Lensmar points to Kubernetes-native agent frameworks, like Kagent, as infrastructure that will need its own test coverage.

Testkube is also adding a capability to generate tests with AI, using context from a customer's existing infrastructure, tests, and results. He invites the audience to sign up for the Testkube AI EAP early access program. Treat that as one data point on where testing tooling is headed.

The takeaway: quality is a platform service, with or without AI

Host William Morgan predicts that a meaningful share of Kubernetes workloads will have an inference component, and that pushes testing even earlier. Not just at runtime but at deploy time, with quality gates before code ships.

Whether or not you've adopted AI, treat quality and testing as a platform capability provisioned the same way you provision CI and CD. AI changes what's inside that capability (e.g., generated tests, smarter test selection, and faster log triage), but it doesn't change who should own building it.

‍

FAQ

Should platform teams or developers own testing in Kubernetes?

It depends on the test type. Functional tests are the developer's job. Performance, security, and chaos testing usually sit with the platform team, though the fix for a failed load test (a memory leak, a regression) still lands back on engineering.  

What does it mean to treat testing as a platform service?

Treating testing like CI and CD, a capability the platform team provisions with guardrails (where tests run, where results land, which tools are sanctioned) instead of leaving each team to build its own testing setup from scratch. 

How does AI decide which tests to run for a given change?

It maps dependencies across a change and suggests which tests to run, called intelligent test selection. It helps more as microservice count grows, but you should still run the full suite regularly since AI selection can miss things, just like a human can.  

Can AI-generated tests be trusted without a human review?

Not yet. AI-generated tests often check what the code does rather than what it's supposed to do, so a human still needs to confirm they match the actual requirement.  

Where is AI already useful for analyzing test failures?

Reading failed test logs and traces is where AI is almost best positioned to immediately bring value in QA today, using a Kubernetes MCP server or Grafana MCP server to pull context.