Get Service Mesh Certified with Buoyant.

Enroll now!
close

Production-Ready Infrastructure for MCP Traffic

Introduction

Traditional LLMs are powerful, but they are fundamentally limited in what they can do through their static training data and their context window. As enterprises accelerate AI adoption, they’re going beyond using AI for text-based response and moving to implementing agentic AI. Instead of the prompt and response model through a traditional LLM, agentic AI can autonomously perform tasks and take action. In order to take action, these AI agents need access to tools and resources. Model Context Protocol (MCP) is designed to be this translator between external data sources and tools and connects AI applications to your most critical infrastructure.

What is MCP and how does it work?

Linkerd is the only service mesh that uses a ultra-secure "micro-proxy" written in the Rust programming language, achieving critical memory safety guarantees. Other service mesh such as Istio and Cilium use Envoy, a complex proxy written C++, a language known for CVEs, buffer overflow exploits, and endemic security vulnerabilities.

Linkerd uses industry standards such as mutual TLS and the Gateway API to provide a comprehensive suite of network security controls, including cryptographic workload identity, fine-grained micro-segmentation, deny-by-default, and true zero trust networking. With Buoyant Enterprise for Linkerd, not only is your service mesh ultrafast, small, and simple—it's built for security from the ground up.

Diagram showing an agent with an LLM interacting bidirectionally with three MCP servers, each connected to a tools icon.
What is MCP and how does it work?

Challenges with MCP Servers and How They Differ from REST API Servers

To effectively manage MCP servers and agentic communication, it is important to understand how they are unique compared to the traditional REST API servers.

  1. Stateful: Agentic traffic pulls from past requests and uses those interactions to build context for future communications. This means that rather than a one-off interaction, they take place over a persistent session and require complex workflows.
  2. Unpredictable: Because of the need for rich context and often large datasets, agentic communication has unpredictable traffic patterns that can create massive traffic spikes without warning. Compared to traditional workloads, where each request has a simpler result, agentic workloads are not always so linear and depend largely on the previous request.
  3. Specialized Resource Access: Because each MCP server runs as a separate process, each server needs its own configuration, credentials, and resource access.

Because of how this agentic communication is unique, operating MCP servers is a new challenge for platform teams. Traditional networking security tools don’t have the capabilities to give this type of visibility into what AI agents are doing, which tools they are accessing, what data in those tools is being accessed, and how often they are accessing the data.

Benefits of MCP Support in Linkerd

Traditional networking security tools don’t have the capabilities of managing complex traffic without visibility into what AI agents are doing, which tools they are accessing, what data in those tools is being accessed, and how often they are accessing the data. Managing complex network traffic is Linkerd's proven area of expertise and AI traffic is no exception. With the introduction of MCP support, Linkerd will provide the same visibility, access control, and traffic shaping capabilities that users rely on today for their existing set of protocols. By managing traffic at the platform level, a service mesh like Linkerd can transparently add critical capabilities without changing application code. Additionally, integrating MCP support at the service mesh layer eliminates the need for additional, specialized tooling and provides a single pane of glass for all traffic. Linkerd is the only service mesh that offers support for MCP integrated into the core functionality, meaning that you can use the same trusted workflows and consistent policies in Linkerd to manage both traditional and AI workloads. With Linkerd you get:

  1. Zero-Trust Security for AI Agents: Linkerd applies policy-based access control to all MCP traffic so you can control access to specific tools and encrypt MCP traffic automatically with zero code changes.
  2. Observability into AI Agent Behavior: Linkerd provides visibility into MCP traffic so organizations can understand resource usage, performance bottlenecks, and active connection status.
  3. Reliability for Mission-Critical Workloads: Linkerd prevents disruptions to AI connections through intelligent load balancing and automatic retries without manual intervention.

Use Cases

‍Multi-Tenant MCP Deployments

If you’re building an AI platform where multiple teams are running agents that can access shared MCP servers, Linkerd can help isolate sensitive data, set rate limits, and see resource usage per tenant.

Example: A SaaS company deploys different AI agents for different customer tiers (free and lower tiers getting basic features and premium tiers getting more robust features). With Linkerd, free tier agents can be rate-limited to and flagged if they attempt to abuse the system by creating multiple accounts. Higher tier customers can get priority access and each tier has isolated metrics for capacity planning and monitoring. 

‍Hybrid and Multi-Cluster Infrastructure

‍If you’re running MCP servers across multiple Kubernetes clusters and cloud regions, Linkerd can help set unified security policies across clusters and secure communication regardless of where it is deployed.

Example: A federal agency operates Kubernetes clusters across multiple security zones—an unclassified cloud environment, a classified on-premises environment, and edge clusters at field offices. AI agents helping process data need to access MCP servers in all three environments. Linkerd provides secure, mTLS-encrypted communication across all zones. When the field office connection is unreliable, Linkerd automatically fails over to MCP servers in the main datacenter, ensuring continuity even with degraded connectivity.

Compliance and Auditing for AI Workloads

‍ If you’re implementing AI at an enterprise level, those AI workloads are no exception to regulatory requirements. Linkerd can help with consistent policy enforcement and tell you which agents have been accessing which resources.

Example:  A large bank deploys AI agents to assist in daily operations. These agents have access to sensitive data like account balances, translation histories, credit reports and investments through MCP servers. Under specific regulations, they are required to implement strong data protection controls. They can use Linkerd for audit logs to demonstrate only authorized information was accessed and policy enforcement to block unwanted access requests.

Cost Optimization for AI Workloads

AI workloads can be costly, and you need visibility into which resources are being consumed in order to cut spend. 

Example: An online retailer is experiencing abnormally high AI infrastructure costs but without proper observability, they are unable to see the source of the spend. Linkerd allows them to see if a specific agent is making redundant calls to an MCP server due to poor caching. With Linkerd’s ability to see metrics per tool, they also see specific tools on the website (like a chatbot or a virtual try-on) might be running 24/7 but only used during specific times. From Linkerd’s visibility they are able to take actionable insights and scale down the tools to reduce spend.

Getting Started

MCP support will be fully available soon in both open source Linkerd and the enterprise distribution from Buoyant. If you are interested in partnering with us, please complete the early access program form.

What is FIPS?

The FIPS standards cover a broad range of topics, including encryption algorithms, access control mechanisms, physical security, and network protocols. These standards are used by government agencies, contractors, and other organizations that handle sensitive information to ensure that their systems meet the highest levels of security and compliance. Compliance with FIPS standards is often a requirement for government contracts and is a critical component of cybersecurity best practices.
‍
FIPS (Federal Information Processing Standard) is a set of guidelines and requirements, including for cryptographic modules, that is used by federal agencies and other regulated industries. FIPS compliance ensures that cryptographic algorithms are implemented correctly and securely, providing strong protection for sensitive data.

FIPS 140-2 vs 800-204A vs 800-207

FIPS 140-2, 800-204A, and 800-207 are different standards published by the National Institute of Standards and Technology (NIST) that address different aspects of information security.

FIPS 140-2 is a standard that specifies the security requirements for cryptographic modules used in protecting sensitive information. It outlines the requirements for the design, implementation, and testing of cryptographic modules, including encryption algorithms, key management, and random number generation.

‍NIST 800-204A is a guideline that provides recommendations for the security and privacy requirements of controlled unclassified information (CUI) in nonfederal systems and organizations. It outlines the security controls and best practices that should be implemented to protect CUI, including access control, audit and accountability, and incident response.

‍NIST 800-207 is a standard that outlines the security considerations and best practices for the use of Zero Trust Architecture (ZTA) in securing modern information systems. It provides guidance on the design, implementation, and management of ZTA, which is a security model that assumes no trust in any user or device and requires strict authentication and authorization for all system interactions.

In short, FIPS 140-2 specifies the security requirements for cryptographic modules, NIST 800-204A provides guidance on securing controlled unclassified information, and NIST 800-207 outlines the best practices for implementing Zero Trust Architecture in information security.