Live patching is a way of updating a running system without stopping it. It is best known as a technique for keeping Linux servers updated to the latest security levels without affecting downtime. This article provides some background to the technique and explains the advantages of using it.
What is Live Patching?
Live patching lets you keep Linux server kernels up-to-date with the latest security updates without the need to reboot. Although the practice is a decade old – once seen as a convenience tool easing the lives of system administrators – it is now coming to the attention of security managers and CISOs in the wake of the recent flurry of Linux-related kernel vulnerabilities.
Until the advent of live security updates, server managers had to choose between running their systems with known vulnerabilities, or taking their servers down to install security updates. System administrators now see Linux kernel live security updates are becoming an essential component of an enterprise’s cybersecurity toolkit, not merely a convenience for system maintainers.
Where Did Live Patching Come From?
In 2008, Jeff Arnold announced Ksplice, software to apply security updates to running Linux kernels without restarting them. The company was based on selling a live patching software solution Arnold and his colleagues had developed during their MIT student days. The software itself came from a real need Arnold had experienced as a member of a volunteer group administering servers for the student community. One of the servers under his remit had a kernel security update pending. The installation was deferred so as not to inconvenience other students on the server, but before the update could be applied, the system suffered a hack exploiting the unpatched vulnerability.
After Oracle bought Ksplice and closed the source in 2011, the Linux community was spurred into developing its own solutions. These matured in 2014 as Red Hat’s Kpatch and SUSE’s Kgraft. In May of the same year, CloudLinux, producers of a Linux variant well known in web hosting spheres, released their own commercial Linux kernel live patching service under the name of KernelCare.
Why Use Linux Kernel Live Patching?
The prime motive for users of automated security updates of Linux kernels is to avoid server reboots. However, the reason to avoid reboots varies from company to company. The primary reason live patching customers give can be boiled down to three main drivers:
Compliance: Being able to keep large numbers of servers automatically up-to-date is a boon for companies seeking or maintaining compliance certifications (such as SOC 2). Taking on the burden of gaining a compliance certificate can be eased by automating certain aspects of the specification. Automation of security updates fits in wherever there is a conflict between the need to have security fixes installed within a certain time frame (usually 30 days) and the need to keep services active.
Availability: Companies that work with SLA contracts may be penalized if their system accessibility and uptime statistics fall below defined levels. Live patching allows a system to be patched without incurring downtime. Even without the constraints of an SLA, companies can find their revenue streams are impacted when their services are interrupted, especially those companies making money from services such as continuous, online multiplayer games, cryptocurrency mining or audio/video streaming.
Convenience (cost): Updating a system takes time, and requires a high level of system administration skill. Live patching relieves staff of the chore of routine maintenance, allowing highly-skilled staff to concentrate on more esoteric system challenges.
What Live Patching Can’t Do
Despite the obvious benefits of automated security updates, there are caveats to its adoption that need to be made clear:
- Live patching is only for critical security problems. Linux kernel patches can fix vulnerabilities if the problem can be isolated to small and specific portions of kernel code. However, if the problem is complex and affects many functions, or affects data structures, live patching can’t be done
- Not all kernels support live security updates. The various live solutions use different techniques for managing the patching process and for creating patches, and some are specific to the Linux family for which they were designed
- Linux kernel security fixes must be written by experts. Even simple patches require an advanced knowledge of Linux and C. If the patch is intended for production servers, it must be thoroughly tested across a wide range of platforms and kernel versions. This takes enterprise-level equipment and skills to do properly
Who Offers Linux Kernel Live Patching?
Here is a list of the main Linux kernel live patching vendors, in alphabetical order:
- Canonical Livepatch: For Ubuntu 14.04, 16.04, 18.04 LTS versions only and available as part of the Ubuntu Advantage program
- KernelCare: Developed independently and available for multiple platforms on a per-server, per-month subscription basis
- Kgraft: Developed by SUSE and now offered by them under the name SUSE Linux Enterprise Live Patching
- Kpatch: Developed by Red Hat. Source code is freely available on github but patches must be purchased via a commercial license for Red Hat Enterprise Linux Server
- Ksplice: The first to offer commercial live patching for Linux. Available only on Oracle Linux as part of their Oracle Linux Premier Support
There are many options for enterprises wishing to enjoy the benefits of live patching. The choice of vendor depends on which flavor of Linux they are already using. Despite being a mature technology for over a decade, Linux kernel live patching still has some way to go before it is adopted as a security best practice, and not merely seen as a system administrator’s convenience.
