Supporting Hybrid Virtualization Orchestration for Edge Computing

Microservice architectures allow developers to decompose their applications into independently deployable functional blocks, each with its own requirements. In order to support a wide range of constraints, service virtualization can be customized across microservices but is typically homogeneous within a cluster. As there is no clear one size fit all approach, we can improve resource utilization and performance by using virtualization as a new dimension in orchestration, especially in edge computing environments. For instance, Unikernels represent a lightweight virtualization technology that offers a performant alternative to traditional containers. While we find different studies analyzing and comparing these virtualization technologies, (a) the performance results might vary when including the overhead of the orchestration platform, and (b) it’s not trivial to select the perfect virtualization technology for an entire cluster. In this paper, we explore the benefits of hybrid container-unikernel deployments by extending an orchestration framework for edge computing to allow for seamless mixing and matching of both technologies. Our evaluation shows how hybrid deployments can lead up to 44% CPU reduction cluster-wide while there are scenarios where containers are still preferable.

In the landscape of modern computing, microservice architectures have increasingly become the standard approach for designing highly scalable and available applications. Microservices are typically deployed in containers [13, 28, 54]. With the OCI standardization [7], containers are nowadays decoupled from the underlying runtime, allowing for seamless portability across different environments and reusability of common functions – like nginx web servers, Redis, etc. This calls for a new dimension in orchestration, where the virtualization technology can be chosen based on the service requirements. Unfortunately, state-of-the-art still considers virtualization coupled with infrastructure provisioning. Edge computing significantly alters this assumption, given its inherently heterogeneous infrastructure offering variations in (CPU/memory) hardware, OS support, etc. [39, 45]. Previous studies highlight the operational overheads caused by cloud-native assumptions at the edge [15, 41] and benefits of lightweight virtualization in conjunction with containers [21, 29]. Unikernels are a good candidate for the edge because of their small footprint, faster instantiation, improved performance, and flexibility [35, 53]. However, despite advancements in unikernel toolchains, such as Unikraft [31], which allows porting existing Linux applications, the ecosystem does not support a wide range of applications and driver functionality [25]. Moreover, as shown in the remainder of this paper, unikernels are not the best choice for all services at all times. We envision a future where, given a standardized packaging format like OCI, the runtime can be chosen dynamically based on application requirements, with the orchestration platform effectively becoming a middleware for multi-virtualization setups. Take, for example, a stream-processing video analytics pipeline, which can include several GPU-intensive services that operate more suitably as containers with full-fledged OS providing complex driver support [55]. However, services within the pipeline, such as load balancers, may be more performant as unikernels using a hypervisor as resource multiplexer [29, 33, 36]. Hybrid virtualization also enables a gradual transition of complex containerized applications to unikernels – as the build toolchain evolves to support more system calls and libraries [1, 6, 23]. While several papers have empirically evaluated and compared the performance of different isolation technologies [29, 33, 49], they do not consider (i) the overheads of compatibility layers which allow these virtualizations to operate on common hardware and (ii) orchestration overhead for managing deployments with different virtualizations at runtime.

This paper explores the feasibility of container-unikernel hybrid orchestration. Our contributions are as follows.

(1) We extend Oakestra [15], a lightweight orchestration framework for edge computing. We implement a compatibility layer that allows Unikraft [31] unikernels to behave as containers from an orchestration perspective. We extend the control plane to aggregate clusters’ virtualization information and the scheduling workflows to consider virtualization requirements. We introduce service hot-swap to change the service’s virtualization technology at runtime.

(2) We evaluate the suitability of hybrid virtualization orchestration via real-world application pipelines. Specifically, we dissect the overhead of the compatibility layer performing cross-deployment of containers via runc, unikernels, and gVisor secure containers. Our results showcase the potential for hybrid virtualization, achieving up to ≈ 44% CPU usage reduction in our cluster.

Background and Related Work

Due to their small memory footprint (approx. a few MB) and reduced system call dependencies [32], unikernels offer faster boot times (≈ 10×) compared to containers [26, 31] and are easy to scale and migrate [37, 40, 51]. The shared application and kernel address space allows all code to run in the same CPU privilege domain, which improves performance by avoiding application and kernel context switches [52]. Early unikernel frameworks, such as MirageOS [4], required developers to write applications from scratch. However, in recent years, unikernels have evolved as a capable alternative to containers. Unikraft [31] provides a streamlined approach to building and porting existing Linux applications. The toolchain provides a high degree of POSIX compatibility (≈ 160+ out of 224 syscalls required for popular Linux install [34]). EVE-OS [1], a Linux Foundation project, is a universal, vendor-agnostic OS for edge computing hardware (including embedded devices) and adds native support for both containerized and unikernels workloads. Container runtimes (e.g., Firecracker [2], gVisor [3]) enhance container security and isolation by executing them as para-virtualized microVMs over qemu/kvm similar to unikernels. The OCI standards help supporting both container and unikernel runtimes simultaneously [7, 11]. Experimental runtimes like urunc [42] and runu [46] represent the first steps towards kernel-level compatibility unifying containers and unikernels [47]. Unfortunately, it is not clear what the overhead of such compatibility layers is in real-world deployments, and how they affect the orchestration of services, especially on constrained hardware at the edge. Edge infrastructure is generally less powerful and more heterogeneous than cloud datacenters, often comprising of smaller devices with varying CPU architectures and capabilities, e.g., Intel NUCs, Jetson Xavier, Raspberry Pis, etc. As edge computing is often seen as an extension of the cloud, the majority of orchestration solutions adapt the popular cloud-native Kubernetes (K8s) framework [19]. Solutions like KubeEdge [20], KubeFed [24], and MicroK8s [17] modify Kubernetes by simplifying control-plane operations and removing non-essential components to make it applicable for edge. On the other hand, Oakestra [12, 15] rearchitects the orchestration control plane from the ground up to address the hardware heterogeneity and geographical diversity in edge infrastructures with minimal overhead. In Oakestra, computational devices (leaf nodes) are grouped into (logical) clusters managed by local cluster orchestrators (see fig. 1). The worker node includes NodeEngine component for managing service deployment and operation and NetManager for network communication. Each cluster orchestrator is responsible for keeping track of fine-grained resource and service management within its cluster. The root orchestrator acts as an “orchestrator of clusters” and the point-of-contact of developers to deploy their applications.

Unfortunately, all state-of-the-art orchestration frameworks treat virtualization as a cluster constraint. We finally have an opportunity to exploit virtualization as a dimension to improve resource utilization and application performance. In [38], the authors examine approaches for orchestrating sandboxed containers as microVMs over qemu/kvm via extensions to K8s. FADES [21] leverages MirageOS [4] unikernels to deploy application microservices in Xenbootable images suitable for edge devices. However, arguably (i) not all applications perform better as unikernels, and (ii) the virtualization technology must be dictated by application requirements and not by the infrastructure availability alone.

Orchestration Support

In our exploration, we extend the Oakestra [12, 15] orchestration framework to support hybrid container-unikernel deployments and measure the overheads and benefits of such hybrid virtualization setups. We choose Oakestra due to its lightweight implementation and extensible design, which allows us to integrate unikernels orchestration metrics alongside containers with minimal changes and reduced overhead. We use Unikraft [31] as the unikernel runtime for our experiments, as it provides a wide range of unikernel configurations and supports a variety of applications. The proposed architecture provides an extensible interface that is used to evaluate unikernel virtualization as an additional orchestration dimension, but that easily allows for further runtimes support and optimal virtualization selection.

3.1 Hybrid Service Deployment

To enable hybrid virtualization support, we must ensure that the worker nodes’ hardware can support container and unikernel execution. Oakestra supports integration of new runtimes via (a) using the runtime dispatcher interface or (b) integration to containerd thanks to OCI runtime-spec compatibility. Initial experiments with runu [46], an OCI-compatible runtime for containerd, showed inconsistent behavior. This runtime is currently under development [47], so misalignments with the latest Unikraft versions are expected. Moreover, managing runu as OCI runtime involves the additional overhead of the containerd middleware managing the hypervisor, which can be avoided by directly interacting with qemu. To overcome this, we design and implement a Unikernel Runtime Abstractor component in Oakestra’s NodeEngine, which instead of controlling unikernels via containerd, adds abstractions for directly managing and monitoring Unikraft services within the orchestration framework (see fig. 2). This approach, while Oakestraspecific, is not replacing the OCI runtimes such as runu/urunc. These runtimes can be easily integrated as containerd runtimes when they mature, but at the cost of additional overhead. Unikernels (and the abstractor) are only enabled on machines supporting qemu and kvm targets (e.g., node 1 and 2 in fig. 1) while containers are enabled only in nodes supporting containerd. The Unikernel Runtime Abstractor (i) manages the lifecycle of unikernels and interacts directly with qemu for the virtualization, (ii) performs the downloading and unpacking of kernel images, and (iii) binds a routine to read the qemu qmp socket interface and update the internal service status (running/paused/failed) to the cluster orchestrator.

3.2 Resource Management and Scheduling

As shown in fig. 1, typical edge deployment with Oakestra may include worker nodes with heterogeneous hardware architecture and runtime target support. While some nodes execute both container and unikernel deployments (e.g., node 1), others only support either one of the two (e.g., node 2 or 3). At worker startup, the NodeEngine checks the nodes’ CPU architecture and virtualization support. Then, it advertises its runtimes to the associated cluster orchestrator as virt tuple (). In Oakestra, the cluster orchestrator hides the internal infrastructure details of its workers by only sending the aggregated cluster statistics to the root orchestrator [14]. We extend the control plane to support multi-virtualization by propagating the set of all available virtualization and hardware architecture combinations in each cluster (see Cluster1 → Root in fig. 1). Further, we extend the Oakestra schedulers to consider virtualization during scheduling. Specifically, the service schedulers, before they apply one of the available scheduling policies [15], they first prune the most suitable cluster(s)/worker(s) list based on the available virtualization options matched with SLAs requirements.

3.3 Virtualization Hot-Swapping

We extend the control plane with a runtime switch functionality for stateless applications supporting multiple virtualization technologies. Suppose service1.instance1 is deployed as a container but a unikernel implementation is available. By triggering the hot-swap, the control plane performs the deployment of service1.instance2 unikernel alongside the first instance. The network component gradually balances and shifts the traffic from the container to the unikernel instance. Once the traffic migration is complete the first containerized instance is removed.

3.4 Inter-Service Networking

To achieve agile hybrid virtualization, it is important that the orchestrated services can interact with both unikernel and containerbased services without additional overhead. Oakestra utilizes a semantic overlay network to enable multi-cluster container networking and load balancing. Each service is allocated IP addresses mapped to different load-balancing strategies across available instances. The NetManager interprets packets to/from a semantic address and re-assigns them to the correct instance IP address – forming a tunnel between communicating services. To achieve similar seamless networking between containers and unikernels (and across unikernels), we extend the NetManager to provision (i) a network namespace for unikernels and (ii) a local namespace IP address that can be used to translate network packets irrespective of the virtualization target. Unlike containers, unikernels do not share the host kernel but require a dedicated network stack. We overcome this by connecting a macvtap interface in bridge mode directly to the veth of the service’s network namespace (see ). The runtime abstractor is providing such interface to qemu for the unikernel startup. With this extension, unikernel and container namespace IP addresses are provisioned using the same Oakestra mechanism, giving out-of-the-box support for semantic load balancing and service discovery for both supported virtualization techniques.

Application Performance

tldr

Discussion and Future Work

While advancements in toolchains like Kraftkit [48] are enhancing the portability of applications to the unikernel domain, not all libraries, drivers, and consequently applications are currently supported. Moreover, as we show in §4, determining the most suitable virtualization for a given application is not straightforward. Generally, we observed that within an orchestrated infrastructure, containers are a more efficient choice for network-dominant and latency-critical applications, while unikernels are better suited for CPU-intensive applications and scalability, achieving up to 44% CPU usage reduction in our cluster. It is crucial to recognize that the real-world performance of unikernels might not always align with conceptual expectations [22] and aspects such as flexibility, compatibility and security may be more relevant factors for considering optimal virtualization choice [44]. Our findings motivate for joint transparent orchestration of unikernels and containers as well as the need for a platform that abstracts its complexity. Summing up, there is no one-size-fits-all approach for service virtualization.

In future extensions, we envision a closer integration between Oakestra, Unikraft, and qemu to reduce the virtual network bottlenecks experienced with the unikernels. We also plan to investigate intelligent scheduling solutions for performance forecasting and a telemetry-based feedback loop for performance monitoring across virtualizations from the application to the runtime layer. Moreover, we plan to integrate cross-virtualization checkpointing to enable the data migration of stateful applications with minimal loss and downtime across different virtualization technologies.