We Replaced Our OPA Sidecars with a Single eBPF Program

We Replaced Our OPA Sidecars with a Single eBPF Program

The sidecar pattern has become a cornerstone of cloud-native architecture. For years, it was the default solution for injecting cross-cutting concerns like service discovery, observability, and security into our applications. But as our Kubernetes clusters grew, the cracks in this model began to show. We found ourselves managing thousands of proxy and agent sidecars, each consuming precious CPU and memory. That's when we decided to make a change: we replaced our OPA sidecars with a single eBPF program, and it fundamentally transformed our approach to policy enforcement.

For anyone running Open Policy Agent (OPA) in a sidecar model, the story is likely familiar. You face a constant battle between robust security and resource efficiency. The operational overhead of managing, updating, and monitoring thousands of identical sidecar containers becomes a significant tax on the platform team. This article details our journey away from that complexity and toward a more efficient, powerful, and scalable solution for cloud-native security.

The Hidden Costs of the OPA Sidecar Pattern

OPA is an incredible tool. It has rightfully become the de-facto standard for policy-as-code, providing a unified way to declare and enforce policies across the stack. The sidecar deployment model, where an OPA agent runs in its own container alongside every application pod, seems logical at first. It isolates the policy decision-making process from the application. However, at scale, this pattern introduces several critical challenges.

Resource Overhead at Scale

The most immediate pain point is resource consumption. Each OPA sidecar requires its own CPU and memory allocation. While a single sidecar might seem negligible, the cost multiplies rapidly.

• Before: In a cluster with 1,000 application pods, we were running 1,000 OPA sidecar instances. If each sidecar conservatively consumed 50 millicores of CPU and 100MB of memory, that's and dedicated solely to policy agents.