In this AI Kubernetes Show episode, we talked with Rob Koch, Senior Principal at Slalom Build and Co-Chair of the CNCF Deaf and Hard of Hearing Working Group (a Merge Forward community group), about AI, Kubernetes, platform engineering, and all things software development. He shared his thoughts on the current state of AI adoption, how engineers can adapt to non-deterministic systems, and practical applications for automation and accessibility.
This blog post was generated by AI from the interview transcript, with some editing.
AI is more than just a buzzword; it's a promising technology actively changing how we work. For Koch, AI is definitely a positive force with real potential.
The shift toward AI adoption is happening fast, and many are already moving from seeing it as a simple tool to relying on it. Our dependency on AI is growing rapidly. This pace of evolution strongly suggests that any organization not embracing AI will likely struggle to keep up.
When adopting AI, organizations often struggle with where to start. Compounding this are employees who may lack the skills or simply don't have the time to dedicate to new AI initiatives. A "move fast, fail fast" approach can be risky, leading to potential legal and regulatory issues or public embarrassment.
To build a solid AI strategy, start with a clear picture of the desired result and work backwards. Focus on what you expect to get out of AI and where the project is ultimately headed. This approach, often described as the engineering mindset, provides the necessary focus for the entire initiative.
To mitigate the risk of an AI application "going off the rails" and causing large-scale failures—like famous cases of AI over-ordering—we need to implement strong safeguards.
These safeguards should include guardrails and checks. We essentially need to put checks in place, perhaps even using AI to validate the output from other parts of the AI system. Not unlike the American governmental system of checks and balances, where different branches keep the system balanced.
To get more accurate and predictable results, the AI needs constraints. A critical guardrail is ensuring the AI operates within the right context. If you want a more deterministic result, you need to provide more context, the right prompts, and clear specifications.
Engineers are traditionally trained to think in terms of math and input/output. You put something into a function, and you get a predictable result. With AI, however, the output is non-deterministic. For engineers moving into the probabilistic world of AI, the strategy centers on establishing controls and maintaining human oversight.
A key strategy is setting contextual limits. This means carefully limiting the information fed to the AI to ensure it stays focused on the task at hand.
Despite the trend toward automation, maintaining subject matter expertise is still crucial. For AI to work effectively, you really have to know the subject matter. That's why opportunities for learning remain vital. This expertise is essential for verification. Engineers must be able to verify the AI's output, perhaps by reviewing generated YAML to check for accurate structure and formatting. This is the only way engineers can detect when AI is hallucinating.
The core principle is to automate the mundane and repetitive tasks to free up engineering time, which aligns perfectly with the Don't Repeat Yourself (DRY) principle in software development.
Koch shared a recent win involving a database upgrade project that used AI with sub-agents to streamline the work. The system was configured to analyze the database logs and diagnose their contents. It also determined if the database was correctly sized—checking for both over- and under-provisioning. Finally, it drafted the necessary documentation, which could then be automatically pushed out to each product team.
Koch, who is deaf, uses AI to improve his written communication. His first language is American Sign Language (ASL), making him a non-native English speaker even though he was born and raised in the US (a very typical case for deaf individuals relying on sign language). He uses AI to polish communication, specifically to ensure the English word order is correct, which differs from ASL. He also highlighted the promising application of LLMs in sign language recognition.
You can connect with Rob Koch on LinkedIn or on the CNCF Slack under his name.
Start with the desired result and work backward, focusing on expected outcomes and the project's ultimate goal. This approach, often described as the engineering mindset, provides the necessary focus.
Strong safeguards include guardrails, checks (perhaps using AI for validation), and constraints. A critical guardrail is ensuring the AI operates within the right context, along with providing more context, correct prompts, and clear specifications for more deterministic results.
The core principle is automating mundane, repetitive tasks to free up engineering time, aligning with the DRY principle.
Copyright © 2025 Buoyant Inc.
