Today, most organizations and individuals use Linux and the Linux kernel with a “one-size-fits-all” approach. This differs from how Linux was used in the past–for example, 20 years ago, many users would compile their kernel and modify it to fit their specific needs, architectures and use cases. This is no longer the case, as one-size-fits-all has become good enough. But, like anything in life, “good enough” is not the best you can get.
Enter: Extended Berkeley Packet Filter (eBPF). eBPF allows users to modify one-size-fits-all to fit their specific needs. While this was not impossible before, it was cumbersome and often unsecure.
eBPF is a feature available in Linux kernels that allows users to safely load programs into the kernel, to customize its operation. With eBPF, the kernel and its behavior become highly customizable, instead of being fixed.
Utilizing eBPF, users can load a program into the kernel and instruct the kernel to execute their program if, for example, a certain packet is seen or another event occurs. eBPF lets programs run without needing to add additional modules or modify the kernel source code. Users can think of it as a lightweight, sandboxed virtual machine (VM) within the Linux kernel that makes use of certain kernel resources. Using eBPF removes the necessity to modify the kernel source code and improves the capacity of software to make use of existing layers.
This technology revolutionizes how observability, security, and networking are delivered.
Making One-Size-Fits-All Fit Specific Needs
Companies like Google and Meta are using eBPF, including in environments without containers or cloud. Instead of having to maintain their own in-house version of Linux, they can maintain their in-house version of eBPF code. This approach is more productive, less expensive, and easier to understand. In general, it is much easier to maintain something that modifies one-size-fits-all instead of maintaining something very specific in-house. It’s more secure due to the verification, less error-prone because it’s a smaller code base, and users can tune it for their ecosystem.
With eBPF, organizations can take code to customers and run it on their side, in their kernel: the privileged part of the operating system responsible for enforcing security, segmentation and isolation. With eBPF, organizations can inject their code into a customer’s environment and run it safely, and the customers can be confident that the code itself won’t introduce security holes into their system.
This creates a fundamental change in how services such as observability, security, and networking are delivered.
Enabling Security & Performance
When it comes to cloud-native environments, the juxtaposition between security and performance creates challenges for developers as well as IT and security teams. Teams can make things secure, but they’ll typically be slower, and customers do not want to pay for something slower. People pay attention to the price of the security, and if the cost of security is performance, a plethora of problems will ensue. At the same time, lack of security also creates a host of issues.
eBPF is changing the ecosystem by enabling security and performance to co-exist. Extending the basic capabilities of seeing and interpreting all system calls and providing packet and socket-level views of all networking operations at low performance overhead enables the development of revolutionary approaches to system security. Typically, entirely independent tools have handled different aspects of system call filtering, process context tracing, and network-level filtering. With eBPF, a single tool can control visibility into all parts of the system. This allows users to develop security systems that operate with more context and an improved level of control.
Additionally, the combination of efficiency and programmability makes eBPF a good candidate for most networking solutions’ packet processing requirements. The programmability of eBPF provides a means of adding additional protocol parsers, and smoothly programs any forwarding logic to address changing requirements without ever exiting the Linux kernel’s packet processing context. The effectiveness offered by the Just-In-Time (JIT) compiler offers execution performance near that of natively compiled in-kernel code.
Visibility in a Cloudy World
In the cloud-native world, users often lack visibility into what is happening in the cluster, and that lack of visibility can lead to both security issues and performance issues. In fact, this lack of visibility helped spark the broad use of eBPF in cloud-native environments. Brendan Gregg, a former Netflix performance engineer, was a pioneer in this area. He wanted insight into performance issues for Netflix, and to get customized data out of the kernel, he started a project heavily leaning on eBPF to get this data out. Now, this is common practice.
Rather than relying on pre-defined gauges and counters exposed by the operating system, eBPF allows for the generation of events and the collection and in-kernel aggregation of custom metrics based on a broad range of potential sources. This increases the depth of visibility that might be attained and decreases the overall system overhead dramatically. This is achieved by collecting only the required visibility data and by producing histograms and similar data structures at the source of the event, rather than depending on the export of samples.
The ability to attach eBPF programs to arbitrary kernel functions in addition to kernel and user application probe points enables visibility into the runtime behavior of applications as well as the system. By providing introspection capabilities to both the system and the application side, both views can be combined. This gives unique and powerful insights to troubleshoot system performance issues. A wide range of different data structures lets users extract useful visibility data in an effective way, without needing the export of vast amounts of sampling data that is typical for similar systems.
A Great Technology for the Right Use Case
Project Calico uses eBPF heavily to customize network processing. Traditionally, iptables (now being replaced by nftables) were the workhorse for packet filtering and forwarding. Is eBPF going to replace this or any other subsystem? Perhaps at some point, but not in the near future. There is a symbiosis between these technologies as they complement each other. eBPF is used to inject custom code where encoding it in the kernel would be impractical. The host network stack with its iptables is often bypassed on the fast path, but still relied on for handling cumbersome corner cases. Network security offerings can greatly benefit from XDP, a specific use of eBPF, to do super low cost packet processing super early, and connect-time load balancing can be used to remove the extra cost of per packet address translation. However, Project Calico does not shy away from letting netfilter do other tasks that would be difficult to achieve purely in eBPF right now.
For now, the two co-exist and complement each other, and it’s up to you as the user to weigh the price-performance tradeoff and decide which feature to use when, given your specific needs. Working with a vendor that provides options to combine benefits of both is a strategic approach.
Image credit: Adonis1969 / Dreamstime.com
Tomas Hruby is Staff Software Engineer, Tigera.
