S02 E11 - Is It Easier to Write a Vulnerability Than to Find One?

In this episode of The AI Kubernetes Show, Kathleen Goeschel, Ph.D. and Principal Product Security Engineer at Red Hat, breaks down how machine learning, and now large language models, have reshaped the hunt for AI-generated code vulnerabilities, and why writing one is still a lot easier than finding one.

From software developer to vulnerability hunter

Goeschel's career didn't start in security. She began in telecommunications, took a programming class, and loved software development. Years into that career, a continuing education course on secure coding practices changed her direction. She realized how much insecure code was already running in production, went back for a master's and then a doctorate, and moved into security organizations inside IT. That path led to Red Hat, where she first worked on supply chain security before moving to the company's AI team. Today she's a Principal Product Security Engineer focused on securing Red Hat's AI products, and she also sits on the company's product security research guild.

Teaching machines to spot real vulnerabilities

Before generative AI, Goeschel's job was improving static analysis tools. The scanners of that era combined regex matching, pattern detection, abstract syntax tree parsing, and taint analysis. Tracing an input all the way to its output to see if it got sanitized along the way. The accuracy, she said, "was no better than flipping a coin." The tools were also memory and GPU intensive, and they had no context for what counted as valid code versus dead code or test code that should have been ignored.

To address that, Goeschel ran multiple open source scanner tools against known codebases and aggregated their alerts. She merged those alerts with software metrics, including how stale a file was, how many developers were actively touching it, how much code churned in a given commit, and the codebase's history of CVEs. Prior CVEs became her training data's true positives, on the theory that developers tend to repeat the same mistakes. She then used a genetic algorithm to figure out which of those features actually predicted an actionable finding.

The most surprising result was that code churn alone, without any of the alert data, was a strong predictor of a true positive. Files that changed hands often and got edited frequently were more likely to contain something worth flagging than the static analysis alerts by themselves.

What changes when the code writes itself

LLMs changed two things at once for Goeschel's team. How fast code gets written, and how fast it gets reviewed. Code generation, she said, is "functional, and it works," even when it isn't the best possible code, and it gets there fast. On the review side, LLMs are efficient at finding security issues once you tell them what to look for, and newer models are starting to surface problems without being told what to look for at all.

None of that means teams can hand the keys over yet. Goeschel pointed to two gaps, hallucination and consistency. Ask an LLM to write something one day and it writes it one way. Ask again 5 minutes later and it writes the same thing differently. That inconsistency matters, because teams spend real time building coding standards, and they want agentic systems to follow those standards the same way a human engineer would.

She believes this will change soon through specialized agents, not one general-purpose agent. One agent focused on security, another on writing the code, another on planning it, each checking the others' work. Not unlike the human division of labor in engineering teams today.

The real asymmetry: why vulnerabilities are easier to write than find

When asked whether it's harder to write a vulnerability than to find one, Goeschel didn't hesitate: writing is definitely easier. "Vulnerabilities are all over the place. They're very easy to write," she said. Programming languages give developers the freedom to execute all kinds of operations, and that freedom is exactly what creates the opening.

The difference comes down to paths. Writing a vulnerability only requires exploiting one path through the code. Detecting one means checking every path that could be exploited. A single vulnerable loop can be written 10 different ways, maybe more, and a detection system has to catch all 10, not just the version it was trained on.

LLMs inherit that same asymmetry. They don't write secure code by default. You have to tell them to. And when you point one at a codebase to hunt for vulnerabilities, you still have to tell it, today, what to look for. Whatever system is accepting code, meaning the one doing the review, ends up doing more work than the system proposing it.

When the attacker doesn't need to be an expert

Malicious actors aren't new, but AI has certainly changed who's capable of being one. Attacks used to require real technical skill and real resources. Now someone with no technical background could launch an attack on a corporate organization with an LLM doing the work. She pointed to an example from Anthropic, which disclosed late last year that it had detected an orchestrated attack on its own network.

The same shift cuts against a common assumption about where vulnerabilities come from. Proprietary software used to be relatively insulated, since only the owning company could commit code to it. That protection doesn't hold once a proprietary system starts pulling in external data through an LLM. If that data source can be manipulated, the software is exposed to prompt injection, and a codebase nobody outside the company could ever touch directly becomes exploitable anyway.

Open source's reputation problem

The assumption that open source is the riskier target, because anyone can commit to it, doesn't match what Goeschel has seen. Maintainers review incoming code, and there's usually somebody double-checking before anything merges. Actively malicious commits, she said, would mostly get caught. "It's been shown time and time again, open source is pretty darn secure."

What's straining that system now isn't malicious commits, but volume. Everyone is running LLM-based scanners against open source projects and forwarding whatever the model finds, without validating the results first. Maintainers end up fielding reports in different formats with different priorities from different parties, most of which turn out to be false positives, and even the true positives sometimes don't need fixing because the risk is already mitigated somewhere else in the system. Someone still has to do that triage. Goeschel doesn't think it's only street cred and curiosity driving it, though both play a role. Some of it is just people testing what these tools are now capable of.

What 18 months of agentic development looks like

Asked to project 18 months out, Goeschel said that most code will be agentically generated by then. Systems already exist, in testing, that poll JIRA tickets, plan the work, generate the code, review it, secure it, and push it to a repository, with agents editing each other's output along the way, all without a human in the loop. None of that is fully in production yet. Teams are still testing better ways to run it and figuring out how to isolate these systems, which she called the real concern right now.

The role that stays human, in her view, is architecture and prioritization. Humans set the guidelines for how something should be built, and a separate agent checks that those guidelines got followed. Humans also keep riding the JIRA tickets, deciding what gets built at all. That's a piece of control she said she isn't ready to give up, even as she pointed to a counterexample already in the wild. Some product teams are letting user feature requests flow straight to agents that decide, without a human filter, which features actually ship.

Her advice for keeping up

Goeschel's advice for anyone trying to track this field isn't a reading list. The domain moves too fast for that and what mattered 2 months ago might not matter now. Her advice instead is not to be afraid to say you don't know something. Look up unfamiliar terms as they come up rather than trying to master the whole field in advance. Ask an LLM to explain something in plain language if you need to, and then check its answer against another source before you trust it.

FAQ

Is it harder to write a vulnerability or find one?

Writing is easier. A vulnerability only needs one exploitable path through the code, while detecting it means checking every possible path. That asymmetry applies to both human review and LLM-based scanning. 

Can machine learning reduce false positives in static analysis?

Yes. One approach trains models on aggregated scanner alerts plus software metrics like code churn, file staleness, and prior CVEs treated as true positives, to separate actionable findings from noise. 

Is open source code less secure than proprietary code?

Not by default. Maintainer review typically catches malicious commits in open source projects. The bigger strain today is volume. Unvalidated, AI-generated vulnerability reports are flooding maintainers, not malicious code getting merged.

Are LLMs good at finding security vulnerabilities in code?

They're efficient once you tell them what to look for, and newer models can surface issues without explicit direction. They still hallucinate and produce inconsistent results on identical prompts, so human review still matters. 

Is proprietary software vulnerable to prompt injection?

Yes, once it pulls in external data through an LLM. If that data source can be manipulated, the software is exposed to prompt injection, even though nobody outside the company can commit code to it directly.

‍