1 Introduction↩︎

Modern data center applications are no longer monolithic: they are assembled from swarms of microservices [@benchara_efficient; @jha_study; @gan_deathstarbench; @kakivaya_servicefabric] or “pods”, stitched together by service meshes [@saleh_an; @li_service]. Service meshes simplify development by abstracting away the networking layer, making applications easier to deploy and manage. This approach has fueled the widespread adoption of platforms such as Istio  [@online_istio] and Linkerd [@online_linkerd].

A key component of these service meshes is the service proxy, also known as “sidecar”. A service proxy acts as the data plane of the service mesh: it processes all the inter-pod traffic and enforces policies defined at the transport (L4) and/or application layer (L7). For example, Istio relies upon Envoy [@online_envoy]—a popular service proxy—to enforce L4 load balancing or L7 request authorization.

While convenient, service proxies can slow down a service mesh significantly: previous studies have shown that they can increase request latency by up to 185% [@zhu_dissecting], and increase CPU utilization by 41%–92%. This overhead can be traced back to two main sources: (1) Service proxies execute highly general code and can enforce any conceivable policy. This generality simplifies deployment but carries substantial performance overheads. As we show (c.f. 2), these bloated processing times can contribute to more than half of the overhead of proxies like Envoy. (2) Service proxies increase the amount of inter-process communication (IPC), which in turn increases both CPU requirements and processing time. We visualize this extra cost in 1 (see the blue line): each message that the web server receives is routed through the loopback device, traversing the network stack and crossing the user-kernel-boundary threetimes.

To reduce this overhead, state-of-the-art service meshes such as Cilium [@online_cilium] and Calico [@online_calico] have mainly relied upon two techniques. The first technique aims at minimizing the IPC cost by bypassing the network stack using eBPF [@online_ebpf] (see the orange line in 1). In practice, though, such bypasses tend to yield limited performance improvements because the bottleneck tends to be—as we show in this paper—the message processing time in the service proxy. Acknowledging this limitation, the second technique (illustrated in green) aims at removing the service proxy entirely from the critical path by offloading L4 policies to the kernel [@online_cilium; @online_calico]. While extremely effective, this technique only applies to L4 policies, meaning the user space service proxy still needs to enforce L7 policies. Unfortunately, L7 policies also constitute the bulk of the policies in modern deployments. As an illustration, Alibaba reports that the vast majority of their customers (between 80% to 95%) use L7 policies in their service mesh [@song_canal]. A natural, yet still open, question is therefore: Is it practical to offload L7 policies to the kernel?

We answer this question in the affirmative and show that offloading L7 policies is both practical and dramatically improves the performance of realistic workloads. More specifically, we describe a kernel-based fast path—named —that can transparently accelerate the most common L7 policies (e.g., HTTP-based). It achieves this by jointly addressing both main sources of overhead: generality and IPC cost.

We implement our fast path in eBPF. Despite eBPF-based facilities to process application-layer traffic in the kernel [@online_kcm; @online_strparser], and previous work demonstrating eBPF’s potential to do so [@kumar_feasibility], this approach has remained impractical for HTTP-based policies due to eBPF’s stringent limitations. In this paper, we present eBPF-compliant techniques that enable the enforcement of complex L7 policies in the kernel.

Given a description of an L7 policy, first automatically synthesizes a highly optimized and policy-specific data plane in eBPF and loads it into the kernel. As we show, -synthesized code is able to process in the kernel the vast majority (89%) of the use cases found in 2417 open-source projects. In doing so, completely eliminates all IPC overhead that would typically arise, similarly to L4 offloads. In the unlikely case a policy cannot be enforced in the kernel, transparently falls back to the user space service proxy. We stress that existing service proxies do notneed to be modified to benefit from .

We implemented and demonstrate its practicality by speeding up state-of-the-art service proxies such as Envoy, and show that significantly improves performance. Under realistic conditions, reduces the median request latency of state-of-the-art service meshes by up to \(6\times\), and the \(99^{th}\)-percentile by \(4\times\), while serving up to \(3\times\) more traffic. To sum up, we make the following contributions:

1. We make the case for processing L7 policies in kernel space and demonstrate that eBPF is mature enough to implement the most commonly used L7 policies observed in the wild.

2. We introduce , an eBPF-based fast path for any service proxy that processes L7 policies in user space. It overcomes the drawbacks of state-of-the-art solutions by significantly improving request latency and throughput.

3. We implement with support for , , TLS and six popular policies. We evaluate its performance on synthetic and realistic workloads and share the code to foster reproducibility.

2 The Case for an L7 Fast Path↩︎

In this section, we make the case for enforcing L7 policies directly in the kernel. We begin with a brief background on how popular service proxies such as Envoy [@online_envoy] operate before quantifying their main sources of overhead. We show that, with or without IPC acceleration, Envoy imposes significant performance overheads. We then discuss what it would take to reduce these overheads and use those insights to motivate ’s design.

2.1 Background: Service Proxies↩︎

Services proxies act as the data plane of service meshes and are responsible for handling all the inter-pod traffic. Proxies are typically deployed either per pod (e.g., a sidecar in each pod) or per node (one proxy handling traffic for multiple applications on the same host). A service policy specifies how the proxy should handle the traffic. Typical service policies relate to inter-pod routing, protocol bridging, and security (e.g., authorization). As such, they tend to operate either at L4 (e.g., forwarding a message from one port to another) or at L7 (e.g., routing an HTTP request to a particular pod based on method, path, or headers).

2.2 Identifying the Overhead Sources↩︎

To measure the overhead of services proxies, we deploy the Social Network application from DeathStarBench [@gan_deathstarbench] while enforcing a minimal L7 policy that routes traffic based on the HTTP path. We measure the average request latency at 1,500 req/s in four different configurations.¹

In the first configuration, Envoy, we measure the request latency of the application with a single (per-node) instance of Envoy (the blue line in 1). In the second configuration, L4 fast path, we also use Envoy, but accelerate it with an L4 fast path that reduces IPC cost by short-circuiting traffic at the socket layer (the orange line in 1). In the third configuration, , we enforce the L7 policy in the kernel, removing Envoy from the request path (the green line in 1). We compare these three configurations against the lower bound, which represents the application without Envoy, but its traffic accelerated with the L4 fast path.

We decompose the per-request latency into the three major sources of overhead identified by MeshInsight [@zhu_dissecting]: protocol parsing, policy enforcement, and IPC.² We label the remaining time that consists mostly of the application’s processing time as Other (c.f. 1). 2 shows the result.

Envoy adds 3.4 ms per request, which is 47% of the overall 7.2 ms average request latency at this load (lower bound without Envoy: 3.7 ms, on our testbed described in 5). Adding the L4 fast path reduces per-request latency by 17%. Across these configurations, we see that Envoy’s overhead is dominated by protocol parsing, followed by IPC. Protocol parsing accounts for 1.8 ms (54%) of Envoy’s total overhead. Moreover, even with a minimal policy, enforcement makes up another 7%. On the other hand, by employing a specialized data plane in kernel space, reduces the total overhead to 0.12 ms—a 96% reduction.

Similarly, IPC accounts for 0.5 ms (15%) of Envoy’s overhead. The L4 fast path reduces this cost by bypassing the network stack—also shrinking the Other remainder—for an overall reduction of 17%. Again, by enforcing the policy in the kernel, eliminates any proxy-induced IPC.

2.3 Addressing the Major Overheads↩︎

The experiment above illustrates that the performance of service proxies, even for simple L7 policies, is shaped by multiple factors. Protocol parsing dominates overall cost, while IPC overheads make a secondary but measurable contribution. Maximizing performance, therefore, requires jointly optimizing both aspects.

In this work, we propose , a fast path for L7 policies. It specializes the data plane to the policy and thus avoids unnecessary work when parsing the protocol and enforcing the policy. Moreover, it offloads the data plane to kernel space, allowing it to eliminate any IPC overhead that would arise from enforcing the policy outside of the pod’s process. Together, these optimizations reduce the Social Network’s request latency by 46% (c.f. 2).

The following paragraphs discuss why ’s data plane can profit from specialization and what the benefits of offloading that data plane to the kernel space using eBPF are.

2.3.0.1 Specializing the data plane

Service proxies like Envoy employ one single, complex data plane that is designed to support every policy on every protocol. This results in complexity that is unnecessary in most cases, which in turn leads to expensive policy enforcement and protocol parsing.

takes the opposite approach: It synthesizes a specialized data plane that is specific to the service policy. Program specialization is a well-known approach that simplifies code by fixing variables to known values [@bhatia_automatic; @mcnamee_specialization]. This makes otherwise necessary complexity redundant, allowing to eliminate dead code, use more efficient data structures, and compile policy-specific invariants directly into the binary. Exploiting these optimization opportunities renders the data plane simpler and ultimately, more efficient.

2.3.0.2 Offloading the data plane

As previously noted, IPC overhead arises because the policy is enforced outside of the pod’s process. This results in a number of additional context switches that grows linearly with the call graph size of each request. By offloading the data plane to the kernel, these rising IPC costs can be avoided. Despite this, the experiment above suggests that a specialized data plane in user space might be useful, too.

Kernel offloading is commonly done with two different techniques: Kernel modules or eBPF [@online_ebpf]. In recent years, eBPF has become increasingly popular as it facilitates kernel offloads significantly. With the Compile Once-Run Everywhere (CO-RE) feature, eBPF achieves portability across different architectures and kernel versions. This simplifies development and lowers maintenance costs. In the context of service proxies, eBPF has another advantage: Its interface makes it easy to dynamically deploy new policies.

However, using eBPF to offload the data plane comes with its own set of limitations. Most importantly, the Linux kernel verifies the safety of an eBPF program by performing an exhaustive execution path traversal [@vishwanathan_verifying; @sun_validating] before it is loaded into its address space. This process becomes intractable for programs with complex control flows, hence limiting the complexity that can be offloaded with eBPF. Moreover, the eBPF runtime is event-based, making it impossible for an eBPF-based data plane to send messages on its own accord. This renders the offload of some L7 policies impractical.

A common practice to circumvent these limitations is the modification of the kernel with a kernel module. It is not verified and can execute arbitrarily complex programs. However, as we will show, the most popular L7 policies are simple enough to offload with eBPF. We show this by analyzing the L7 policies used in 2417 open-source projects on GitHub. More specifically, we analyse 4699 distinct Envoy configuration files, and classify all containing L7 policies in one of the categories listed in 2. We manually inspect each policy to determine if it can be implemented in eBPF without kernel changes. The results are shown in 3.

Our analysis shows that 89% of all deployed L7 policies can be implemented without any kernel changes. However, there are some exceptions. 83% of customization policies extend Envoy’s functionality with a custom runtime like WASM, Lua, or Go. Such policies must be transpiled to the eBPF instruction set architecture (ISA) and are unlikely to pass verification without careful consideration. 8% of the protocol-related policies apply (de-)compression on the HTTP body. 2% of the routing policies are not event-based. For example, health checking requires the host to send messages to the upstream pods on a regular basis. Finally, 13% of security policies make use of cryptographic hash functions that are currently not supported by eBPF³. eBPF is a continuously evolving platform with more features added every day. As such, we expect the number of policies that require kernel changes to shrink in the future.

We conclude that eBPF is sufficient to offload the most popular L7 policies to kernel space. In the following section, we discuss how accelerates applications in detail.

3 Design↩︎

enforces the vast majority of the L7 policies found in the wild (3) directly in the kernel using a streamlined, eBPF-based data plane. For the (few) unsupported policies, automatically and transparently redirects the corresponding requests to existing service proxies, such as Envoy.

In many ways, ’s design follows a classical SDN split [@mckeown_openflow; @pfaff_the] composed of a data plane and a control plane, alongside with an interface between the two. The key difference is the level of abstraction at which functions: ’s data plane forwards incoming L7 messages onto sockets, rather than packets onto ports.

In this section we provide an overview of ’s data and control plane using a running example (4) in which accelerates an application composed of two pods, A and B, and a service proxy. We assume that the application serves two endpoints, /feed and /admin, and that the service mesh is configured with two L7 policies:

The first policy adds an accept header to /feed requests before routing them to pod B. The second policy mandates the requests to /admin to be authorized using a custom script running on the service proxy and that cannot execute in the kernel, e.g. because it would require an in-kernel interpreter. This is a typical deployment where maintenance endpoints are secured with a company-wide authorization script.

As 4 shows, eventually ends up enforcing /feed requests entirely in the kernel. Conversely, it ends up routing /admin requests onto an open socket to the service proxy, as these requests cannot be authorized in the kernel.

3.1 Data Plane↩︎

Given a high-level policy, automatically synthesizes an eBPF-based data plane and loads it into the kernel. This data plane adopts the Parse-Match-Action paradigm commonly seen in network functions. It intercepts all (ingress, egress, or local) requests to the application, enforces the corresponding policy, and redirects the traffic to the appropriate socket. 5 visualizes the integration into the kernel for the ingress data path. First, at the TCP layer, the kernel manages the TCP state machine and reassembles segments to provide a reliable byte stream. Next, kTLS [@online_ktls] leverages stream parser (strparser) [@online_strparser] to delineate TLS records before decrypting them. The decrypted stream then reaches ‘s data plane at the socket level (SK_SKB for ingress or egress traffic, SK_MSG for local traffic). It integrates the Parse stage with strparser to delineate application-layer messages. This extracts only the necessary headers to enforce the policies, and stores them into a header vector \(H\). Next, the Match stage compares the extracted headers with the policies’ match criteria. This yields the policy that should be enforced, along with the forwarding target for the request. Finally, the Action stage runs a sequence of actions; simple, generic functions that collectively execute the policy. At the end of each action sequence, the data plane performs a connection pool lookup for a previously established socket to the forwarding target. Conceptually, this connection pool is akin to the forwarding table of an SDN switch. The following paragraphs discuss for each stage first their functionality at runtime, and then how synthesizes them.

3.1.0.1 Parse

This stage extracts policy-relevant information from each message. It consists of multiple, protocol-specific deterministic finite automaton (DFA). The data plane keeps track of the protocol each connection is currently using, and selects the DFA accordingly. It uses this DFA to parse the data in the receive queue and identify individual messages. It is possible that this data does not yet contain an entire message. In that case, the data plane waits for more data to arrive. If a message can be identified, it extracts all policy-relevant information from the message. To this end, the data plane feeds the message byte-by-byte to the DFA. The DFA then indicates the relevant byte ranges to the data plane, which it stores into the header vector \(H\). This data structure holds the extracted information and other metadata of one L7 message.

constructs each DFA on startup for the given policy. It encodes them as a matrix of integers and loads them into the data plane. Additionally, it defines a header vector that can hold all policy-relevant information. The procedures that execute the DFA remain the same across service policies.

In the example above, the data plane in 5 only needs to know the HTTP path to enforce either policy. Thus, the header vector consists of a single pointer to the located path, along with metadata like the length of the header block.

3.1.0.2 Match

This stage selects the appropriate policy to enforce. It consists of a sequence of comparisons between the header vector \(H\) and the policies’ match criteria. The service policy determines the order of this sequence, which encodes the priority for each endpoint’s policy. synthesizes this stage as a sequence of if clauses.

In the example above, the Match stage in 5 employs two if clauses that compare the extracted HTTP path in the header vector with “/feed”, and “/admin”, respectively.

3.1.0.3 Action

This stage executes a sequence of actions required by the policy. Each action is a simple function, as listed in 3. They are generic and do not change across policies. Every action sequence terminates with a drop or forward action. The latter action queries the connection pool for an open connection that it can use to forward the message.

synthesizes this stage by first identifying which policies it can offload to kernel space. It replaces unsupported policies with a route policy to fall back to the service proxy. Next, it generates code for each policy using a template that defines its action control flow with policy-specific arguments as placeholders. During synthesis, replaces these placeholders with the actual data from the policy. It inserts the resulting code into the data plane code, along with the implementation of each action. This yields the final eBPF code, that can be compiled and loaded into the kernel.

In the example above, for the /feed request, the data plane adds the accept header with get and write before routing it with get and forward. Likewise, for the /admin request, it performs get and forward to route it to the service proxy.

6 visualizes the synthesis of the /feed policy. It shows that the service policy requires to add an accept header before routing the request to 172.18.0.3, the IP address of pod B. uses the header mutation policy to append the new header. This policy first calls the get action to retrieve the insert location of the new header, and then calls write to insert the string into the message. Similarly, it uses the route policy to forward the message to pod B. This policy also first calls get to query the connection pool, and then calls forward to perform the redirection. replaces the accept header with the HDR placeholder in the header mutation policy, and 172.18.0.3 with the IP placeholder in the route policy. This results in code that is almost ready to compile. In a final step, concats the specialized templates into a single function, and inserts that function into the data plane code. The data plane code contains the implementation of the actions and the execution engine for the parsing stage.

The synthesis of the /admin policy looks very similar. Because cannot execute custom scripts in eBPF, it replaces the policy with a route policy, into which it inserts the IP address of the service proxy.

Note that policy templates may be L7 protocol specific. In the case of , it is also necessary for to be able to upgrade connections from , i.e. switch between policy implementations. To account for this, generates the header mutation code with two templates, keeps state at runtime on the currently used protocol, and calls the appropriate action sequence accordingly. Note that header mutation template shown in 6 is specific. This can be seen by the plain text hdr variable that terminates with “\r\n”. is extensible, and new policies can be implemented by providing with new templates.

3.2 Control Plane↩︎

Given that connection establishment is not possible in eBPF, relies on a control plane to maintain the connection pool. It is empty on startup and lazily replenished during runtime. When a connection pool miss occurs, the data plane automatically forwards the request to the control plane. The control plane in turn establishes a new connection to the forwarding target, inserts it into the connection pool, before letting the data plane forward the request. These connections remain open and are reused until the peer closes them. In this regard, the control plane manages connections to the service proxy like for any other forwarding target.

4 Implementation↩︎

This section describes the implementation of . We implement the control plane in approximately 4K lines of Rust code, the data plane in 2K lines of C code. In the following paragraphs, we outline the details of the Parse-Match-Action architecture, and some challenges of implementing it in eBPF.

4.1 The Parse Stage↩︎

We implement the Parse stage for and . is extensible and new protocols can be added by designing a DFA construction for the respective protocol.

4.1.0.1 Parsing TLS-encrypted traffic

A common use case of service proxies is TLS. Service meshes rely on the proxy to authenticate the application to the client, or authenticate two communicating pods with mutual TLS (mTLS).

terminates TLS and parses encrypted traffic directly in the kernel. We highlight two key implementation details required to achieve this. First, offloads TLS to the kernel with kTLS [@online_ktls]. To this end, the control plane performs the TLS handshake upon connection establishment, before passing the cryptographic connection state to the kernel. Second, attaches the data plane to the SK_MSG or SK_SKB hook on each socket. It uses the former to process and redirect local traffic before the kernel encrypts it. It uses the latter to process ingress traffic after the kernel decrypts it, or egress traffic before the kernel encrypts it, respectively.

4.2 The Match Stage↩︎

’s data plane matches HTTP headers with basic string comparison functions. Future versions of can extend this with non-backtracking regex patterns, for example.

4.3 The Action Stage↩︎

Guided by our policy survey (see 3), GitHub repositories [@online_cilium_github; @online_istio_github] and previous work [@saxena_copper; @song_canal], we implement six policies (c.f. 4): two routing policies, two security policies, telemetry, and one traffic mutation policy. ’s architecture is extensible, allowing users to add new policies with minimal effort.

4.3.0.1 Encoding and Hashing

Actions like hash, en-/decode, and en-/decrypt are inherently impractical to implement directly in eBPF. While Linux kernel version 6.10 provides kfuncs [@online_kfunc] to en- and decrypt data, similar functions to hash or encode arbitrary data are missing. However, popular policies like ring load balancers rely on non-cryptographic hashes functions to distribute traffic evenly across servers, and JSON Web Token (JWT) [@online_jwt] depend on base64url.

Fortunately, the kernel already implements many commonly used hashing and encoding schemes [@online_cryptoapi; @online_xxhash], the eBPF runtime just cannot access them. employs a minimal kernel module (172 lines of C code) which exposes this functionality to the eBPF runtime.

4.3.0.2 Routing

The protocol compresses headers [@online_hpacks] using a Huffman encoding and a cache that allows the sender to transmit references to previously used headers. While the former is stateless, the latter renders the header compression stateful. This poses a problem, as cannot forward header references to a different receiver without dereferencing them first. To avoid this, requires all pods to disable header caching. This is a common feature of libraries [@online_http2_rust; @online_http2_go]. We will show in 5 that despite this, accelerates significantly. Note that Huffman-encoded headers are fully supported and can remain enabled.

Moreover, routes stream-wise: The header frame at the start of the stream dictates the forwarding target of the following frames. However, not all frame types have a clear destination. For example, a push promise notifies the peer of a stream prior to sending the header frame. Likewise, the flow control frame can be connection-wide. If the forwarding target is unclear, the data plane forwards the frames to the control plane. Future versions of cache push promises and split up flow control frames.

4.3.0.3 Managing Connections

manages open connections with the connection pool. We implement it with an eBPF hash map that maps IP addresses to queues of sockets. This design gives the data plane the flexibility to dynamically adapt the multiplexing behavior for each connection. For example, for connections, the data plane only pops the socket out of the queue if the maximum number of concurrent streams is reached, and does not append it back to the queue until a stream closes. In the meantime, forwards requests onto the next socket in the queue.

5 Evaluation↩︎

In this section, we assess the performance of along three dimensions. First, in 5.2, we analyze the performance benefits that provides. We show that for a realistic workload, can improve the median request latency of state-of-the-art service meshes by up to \(6\times\) while sustaining up to \(3\times\) more traffic. Second, in 5.3, we evaluate ’s overhead in the worst case, where all traffic must be processed by the service proxy and show that it still outperforms Envoy for requests with headers smaller than 6.8 kB. For requests that are larger than this, the parsing overhead outgrows the IPC acceleration. Finally, in 5.4, we examine how scales with increasingly complex policies. We find that while ’s performance degrades faster than Envoy’s, its throughput remains at least 39% higher, even when Envoy is accelerated with an L4 fast path.

5.1 Methodology↩︎

5.1.0.1 Applications

We evaluate with three realistic and one synthetic workload. We use Docker Compose to spawn and configure each application.

Social Network:

The Social Network is a realistic application from DeathStarBench [@gan_deathstarbench] that uses Thrift RPC [@slee_thrift], which we configure to use . We generate load with TLS-encrypted /wrk2-api/post/compose requests.

Media Service:

The Media Service is realistic application from DeathStarBench [@gan_deathstarbench] that uses Thrift RPC, which we configure to use . We generate load with TLS-encrypted /wrk2-api/review/compose requests.

Hotel Reservation:

The Hotel Reservation is realistic application from DeathStarBench [@gan_deathstarbench] that uses gRPC [@online_grpc] over . We disable header caching when benchmarking , but leave it otherwise enabled. We tested both configurations and found that in our setup, the difference is negligible. We generate load with TLS-encrypted /recommendations requests.

Echo Service:

The Echo Service is a synthetic application consisting of two pods, one frontend, and a single echo pod. For this workload, we send a 100 B long request to the frontend which forwards it to the echo pod before responding. This emulates small applications with minor IPC overhead.

Note that DeathStarBench’s frontend endpoints only support . We perform the experiments accordingly but emphasize that fully supports + TLS, too.

5.1.0.2 Service Proxies

For all experiments, we deploy Envoy [@online_envoy], a state-of-the-art service proxy typically used with Istio [@online_istio]. However, unlike Istio, we deploy Envoy on a per-node basis, i.e., there is only one service proxy for the entire service mesh. For the evaluation presented in this section, we expect a per-node deployment to always be faster than a per-pod deployment. We did not compare against other approaches [@saokar_servicerouter; @chen_remote] that break the service proxy abstraction and enforce policies within the application’s process because they are closed-source. also runs on a per-node basis. Its data plane operates from the socket level and bypasses Envoy completely for the supported policies. For an unsupported policy, routes the traffic to Envoy. We compare against two Envoy configurations:

Envoy:

This deployment uses Envoy without any acceleration. It routes traffic through the loopback device.

L4 Fast Path:

This deployment reduces the IPC costs of Envoy with an eBPF program at the socket level. The eBPF program reroutes the traffic so that the network stack is bypassed. This configuration replicates the data path of state-of-the-art service meshes [@online_cilium; @online_calico].

5.1.0.3 Policies

Based on the policy survey (see 3), GitHub repositories [@online_cilium_github; @online_istio_github] and previous work [@saxena_copper; @song_canal], we specify a set of policies to benchmark in a diverse environment. It is designed to exercise all components of ’s data plane and includes application-layer policies like RBAC, JWT, routing, and traffic telemetry.

In 5.4, we will also evaluate ’s performance as a function of how complex a policy is. We define a policy as being more complex if it parses and processes a large amount of data, and/or requires more instructions to enforce.

5.1.0.4 Testbed

We evaluate on two nodes, one that generates the traffic using k6 v1.1.0 [@online_k6] and the other that hosts the application. The first is equipped with a 24-Core Intel Xeon CPU E5-2670 v3 (2.3GHz), while the latter has a 20-Core Intel Xeon CPU E5-2670 v2 (2.5GHz). They both have 270GB RAM and run Ubuntu 22.04.5 LTS, Envoy v1.34.0, and Docker v28.2.2. The machine that hosts the service is running Linux kernel v6.16.12. All experiments are performed on bare metal, with TurboBoost and dynamic CPU frequency scaling disabled to reduce measurement variance.

5.2 How Fast Is the Fast Path?↩︎

We first assess the best-case scenario, where all messages are processed on the fast path (i.e. in the kernel). We deploy the three realistic applications with their own policies:

We use a testing regime that sends requests at an increasing rate (closed model) beyond the maximum throughput of the respective application. More specifically, the regime takes 200 s and scales the incoming request rate up to 5K req/s for the Social Network and the Media Service, and up to 20K req/s for the Hotel Reservation. We repeat this experiment 30 times to reduce noise. 7 shows the result. On the left, it shows the CDF of the request latency across the entire experiment. On the right, it shows the request completion rate. We found these trends to be consistent with the applications, even if we swapped their respective policies.

5.2.0.1 Specializing the data plane improves performance

optimizes L7 policies by synthesizing a specialized data plane that is specific to the policy. As we will see in 5.4, this data plane is, compared to Envoy, more efficient at parsing messages, and enforces the policy without IPC costs. This yields significant performance improvements. Compared to the L4 fast path, reduces the median request latency of the Social Network by \(10\times\), the Media Service by \(2.5\times\), and the Hotel Reservation by \(6\times\). As a result, all applications serve more traffic, too. Compared to the L4 fast path, serves up to 49%, 38%, and \(3\times\) more traffic. The performance benefits become particularly significant for applications with complex data planes. stacks employ expensive state management and coordinate with the peer using control messages. requires little state, and can forward most of the control messages without processing them.

5.2.0.2 Optimizing IPC alone is not enough

State-of-the-art service meshes improve the performance of L7 policies by optimizing IPC. As 7 shows, this yields only minor performance benefits. For the Social Network and Media Service, it serves up to 15% more traffic, for the Hotel Reservation, which is much more overloaded than the other two applications, the throughput improvement is negligible.

5.3 What is the Overhead of the Slow Path?↩︎

Next, we assess the worst-case scenario, where all messages are processed on the slow path. In this case, redirects all messages to Envoy for processing. To this end, we use the Echo Service. It exhibits minimal IPC overhead, which makes the overhead of the slow path the most apparent. We configure with the following policy:

In this experiment, we scale the length of the HTTP headers from 1 kB to 16 kB. We increment the header length by 1 kB at a time and repeat the experiment with two different HTTP header compositions. In the first composition, Single Header, we fix the number of parsed and matched headers to one. In the second composition, Multiple Headers, we fix the size of each HTTP header to 1 kB. The load generator establishes 3000 connections and sends as many messages as possible in one minute. 8 summarizes the results.

5.3.0.1 accelerates the slow path too

The number of HTTP headers has a negligible impact on the performance of . In both configurations, remains more performant than Envoy for HTTP headers that are smaller than 6.8 kB. However, as the size of the HTTP headers increases, the throughput decreases, e.g., by 11% for 10 kB headers.

In this experiment, ’s runtime is dominated by its parsing overhead, which is a function of the number of parsed bytes. For a 1 kB HTTP header, 73% of the runtime is spent on parsing. This overhead is fundamental, as must iterate over the full HTTP header to match the message against the configured policy. The remainder of the runtime is implementation-specific and accounts for data copies, string comparisons and eBPF map queries (see 5.4).

We want to emphasize that this experiment pushes the ratio between IPC cost and parsing overhead to the extreme. In this experiment, IPC cost is minor with only two pods that process the request. On the other hand, the request sizes are large in comparison to cloud-scale deployments. [@seemakhupt_cloud] reports that half of the observed RPC calls in production have a median request size below 1.5 kB, with responses blow 315 B. For requests of this size, achieves roughly 8% more throughput than Envoy. Finally, it goes without saying that should not be deployed in a service mesh where it cannot process any message directly in the kernel. Despite this, can efficiently record traffic telemetry. Future versions can use this data to dynamically (de)active the fast path on a per-connection basis. Thus, we expect to be able to accelerate a vast majority of workloads, even if in some cases it has to resort to the slow path.

5.4 How does the Fast Path Scale?↩︎

The previous experiments show that ’s fast path accelerates realistic workloads significantly, and the slow path only induces a slowdown in extreme cases. In this experiment, we validate whether the benefits of the fast path also hold as policies become increasingly complex. As in 5.3, we deploy the Echo Service to minimize IPC overhead, and consequently limit to rely more on its parsing and processing pipelines for performance gains. We configure the service proxy with four different policies, each of which can be tuned to be more complex via a tuneable parameter c:

These policies are designed to scale the complexity along two dimensions: the number of actions and the complexity of the individual parsers and actions. To build increasingly complex policies, we can nest actions recursively. As we can see from the policy description, the complexity parameter c affects multiple policy components simultaneously. For example, RBAC\([c=1]\) parses one 16 kB-sized HTTP header and enforces 100 RBAC rules. The final policy, Mutate, represents the most complex policy that the current implementation of can offload (see 4 for a discussion on how to alleviate this limit). Given that the parsing complexity is a function of the total number of bytes parsed and remains largely independent of the number of HTTP headers (5.3), this experiment uses only one header.

5.4.0.1 ’s performance is a function of policy complexity

As shown in 9, we observe the same trend for all four policies. In general, consistently outperforms both configurations of Envoy. However, this performance gain diminishes as policies become more complex. For example, consider the policy JWT\([c=0.5]\), which parses and matches an 8 kB long HTTP header, enforces 50 RBAC policies, and authenticates a 1.5 kB long JWT. With this configuration, achieves a 59% higher throughput than the L4 fast path. For JWT\([c=1]\), this improvement shrinks to 44%.

We note that our per-node deployment is an extreme case. Typically, the service proxy’s overhead is multiplied in a per-pod deployment because more than two instances process each request. Additionally, as mentioned earlier, [@seemakhupt_cloud] reports that half of all RPC calls in their data center have a median request size below 1.5 kB in their production systems, which corresponds to policies with \(c\leq0.1\). To summarize, for all policies that the current implementation of is able to offload, it is more efficient than Envoy, even when accelerated with an L4 fast path.

5.4.0.2 Understanding ’s overheads

The previous experiment shows that the performance of degrades more quickly than Envoy’s. This is to be expected, as eBPF programs are executed in a virtual machine that lacks some runtime optimizations like SIMD [@miano_fast; @shahinfar_demystifying]. Moreover, calls to a bpf_helper function can be expensive, hurting performance further. Thus, the more complex a policy is, the smaller the performance gain of becomes.

We illustrate this by measuring the main sources of overhead: parsing the message, enforcing the policy, and IPC. Note that we only measure the overhead induced by the service proxy, and ignore the runtime of the application itself. It follows that does not exhibit any IPC overhead, as it runs in kernel space. For each data point, the load generator sends 5K req/s for three minutes⁴ to collect enough samples. The result is shown in 10: While Envoy’s overhead grows by 206% (251% with the L4 fast path), grows by 877%. It also becomes apparent that for Envoy, only the parsing overhead grows, whereas for , the policy enforcement overhead grows at the same rate as parsing. Despite this, even for the maximum complexity \(c=1\), ’s overhead is only 0.30 ms, whereas Envoy’s overhead reaches 0.78 ms for every request (0.68 ms with the L4 fast path).

To summarize, eBPF’s runtime is indeed less efficient than that of the user space. Therefore, offloading significantly more complex policies will inevitably lead to a point where the policy enforcement in eBPF is disadvantageous. However, we consider such a policy to be an extreme case that is unlikely to be used in practice.

6 Discussion and Limitations↩︎

We start this section by discussing the future work that this project has made possible. Then we discuss the main limitations of , and outline what it takes to alleviate them.

6.0.0.1 Accelerating other applications

introduces a practical way to parse and process L7 protocols in the kernel. As long as the message processing is not too complex (c.f. 5.4), can significantly improve throughput and request latency. This also opens the door to accelerate other applications, not just service proxies. For example, application-level firewalls could profit heavily from an eBPF offload and facilitate their deployment. ’s core ideas also apply to web servers. It could serve simple HTTP requests directly from kernel space, or cache more complex ones.

6.0.0.2 Parsing more complex protocols

The current version of supports and . Future versions can extend this support, e.g. for gRPC [@online_grpc], by constructing DFAs that operate on the respective encoding. In this regard, presents a special case. It uses QUIC, which the latest Linux kernel does not support. This renders it incompatible with the eBPF runtime. However, in-kernel support for QUIC is on the horizon [@online_quic_kernel].

6.0.0.3 Implementing more policies

currently supports a handful of policies that are, according to our studies, commonly used in today’s service meshes. Adding support for new policies is as simple as implementing a template. Our study has shown that 11% of L7 policies are not fully compatible with the eBPF runtime and require kernel changes. Implementing one of such policies, e.g., a customization policy in the form of a Lua script, requires the user to additionally implement a kernel module that provides the Lua runtime to ’s data plane. Standard eBPF techniques can be used to call the kernel module from within the template.

6.0.0.4 Enforcing more complex policies

’s data plane complexity is bound by the eBPF verifier. It limits the number of instructions an eBPF program may have, and fails to verify the safety of programs with too complex control flows. To address this, the eBPF community is actively working on increasing these limits [@online_taking_bpf], and academia has produced numerous works [@vishwanathan_verifying; @sun_validating; @bhat_formal] that aim to improve the verifier’s accuracy. can also employ standard techniques like kfuncs and tail calls to circumvent this limitation. We emphasize however, that for all experiments in this paper, synthesizes a data plane that fits into a single eBPF program.

7 Related Work↩︎

7.0.0.1 Optimizing L4 policies

Cilium [@online_cilium] and Calico [@online_calico] are the de-facto industry standard for service meshes. They enforce L4 policies in kernel space, but remain dependent on user space service proxies to enforce L7 policies. We replicate their data path with the “L4 Fast Path” in 5.2 and find that achieves up to \(3\times\) higher throughput. Other approaches reduce IPC overheads by bypassing the network stack [@online_dpdk; @rizzo_transparent; @høiland-jørgensen_the; @marinos_network; @belay_ix], or by batching the packets to be processed [@online_io_uring]. Such approaches require application modifications – remains transparent. OnCache [@lin_oncache] accelerates container overlay networks by caching tunnel headers in eBPF. Slim [@zhuo_slim] is a container overlay network that implements lightweight network virtualization by manipulating connection-level metadata when connections are established. Spright [@qi_spright] is a serverless framework that redirects traffic at the socket level. It uses shared memory to implement zero-copy message delivery. Deploying one sidecar per pod incurs a large overhead in the data plane due to the increased CPU utilization, but also makes the control plane more complex. Ambient [@online_istioambient] addresses this by deploying a per-node L4 service proxy. Finally, multiple works [@moon_acceltcp; @brunella_hxdp; @shashidhara_flextoe] offload L4 policies like packet forwarding and filtering to programmable NICs.

7.0.0.2 Optimizing L7 policies

[@sidoretti_application] advocate for L7 policy enforcement in kernel space. [@antichi_fullstack] proposes to incorporate application-layer protocols into SDN architectures. [@shahinfar_automatic] proposes an automatic application offload to kernel space. The Linux kernel uses mechanisms like the kernel connection multiplexor (KCM) [@online_kcm] to perform message delineation in kernel space. ’s parser augments KCM to provide an HTTP-based message interface. [@kumar_feasibility] studies the potential of eBPF to read L7 payloads. They find that eBPF-based telemetry is more efficient than state-of-the-art systems, but is also limited to matching only 48 B of data. We show that recent advances in the Linux kernel have made it possible to match up to 16 kB of data. [@ngai_regex] is another orthogonal approach to filter packets using regexes in eBPF. employs a similar approach (Aho-Corasick DFAs) to parse messages efficiently. Copper [@saxena_copper] proposes a policy DSL that facilitates the management of diverse data planes and helps minimize the data plane resources. CanalMesh [@song_canal] implements a multi-tenant remote mesh gateway, eliminating the need for a sidecar proxy. This approach improves throughput and latency while simplifying orchestration and pod intrusion. Nevertheless, the deployment of a secure and error-free multi-tenant proxy is challenging. Finally, mRPC [@chen_remote] and ServiceRouter [@saokar_servicerouter] eliminate the service proxy completely by reintegrating policy enforcement back into the application. This yields significant performance gains, but couples the life cycle of the pod to that of the policy enforcement.

7.0.0.3 Specializing the kernel.

There is a long line of work that aims at specializing the kernel [@chick_shadow; @pu_optimistic; @Perianayagam_profile; @pu_microlanguages], or specifically the network stack [@chen_a]. [@bhatia_automatic] propose an automatic specialization of protocol stacks, tailored towards the current usage context. [@marinos_network] employ Netmap [@rizzo_transparent] to implement a specialized zero-copy network stack operating from user space. LinuxFP [@abranches_linuxfp] continuously profiles the kernel to automatically synthesize and deploy a message processing fast path.

8 Conclusion↩︎

The service mesh simplifies the development of web applications by abstracting away the network layer. But its data plane, the service proxy, is responsible for major performance degradations, resulting in higher request latencies.

We presented , a transparent kernel space fast path for service proxies. We demonstrated the need for application-layer processing in kernel space and showed that this has become a possibility with the latest advances in the Linux kernel. improves the performance of service meshes dramatically, lowering the request latency while sustaining more throughput. The service proxy remains agnostic of , allowing for a flexible deployment.

We consider to be the first step that shows the performance benefits of application-layer processing in the kernel. This opens the door for performance optimization of many different applications that profit from application-level processing in the kernel.

This work does not raise any ethical issues.

---

1. We choose this rate because, in our setup, it exercises the full application without exceeding the maximum throughput of all configurations (see 5)↩︎

2. To reduce attribution error, we instrument the proxy with in-process atomic counters rather than relying on eBPF; the additional measurement overhead is negligible relative to the reported magnitudes.↩︎

3. Note that the Linux kernel already implements an extensive crypto API [@online_cryptoapi] and starting from version 6.10, exposes limited functionality to the eBPF runtime. In 4, we discuss how to extend the existing interface with minimal kernel changes.↩︎

4. We choose this rate because, in our setup, it exercises the full application without exceeding the maximum throughput that all configurations can process (see 9)↩︎