Do we really want Virtual Machines?

• They require a guest OS
  • Overhead of deployment and maintenance
• Very slow: a new VM starts in dozens of seconds, plus provisioning
  • Allocate disk, deploy image, create VM, boot guest OS
  • Not quick enough for bursty workloads
• Coarse grained:
  • For resource management: allocate for a full OS
  • For application architecture: monolithic layout
    • Horizontal scaling must replicate whole VM instead of components

Introducing: OS-level virtualization

• Cloud users do not want to run OSes
  • They want to run their applications
• How to share cloud resources closer to the applications?
  • Virtualization layer just between the OS and the application
• Virtualize the OS for multiple applications at the same time!
• An OS executes a container runtime that uses a container engine to run containers
  • Docker, LXC, OpenVZ…

Actors of OS-level virtualization

1. Container runtime
2. Container engine
3. Container

Containers runtimes

• High-level management of containers, artifacts and runtime configuration
  • Business-oriented container lifecycle
  • Build, download container images
  • Configure networking, volumes, security, etc.
• A container image packages an application and its runtime
  • Business core, dependencies, pre-configuration
• Ecosystem of reusable images stored in registries (DockerHub, GitLab registries, local registry…)
  • Images are built immutable for portability, reusability and composability
• Examples: Docker (containerd), Podman, Apptainer…

Container engines

• Low-level management of containers
  • Create, start, stop, destroy…
  • Prepare images for usage
  • Last-mile setup of networking, mounts, security…
• Different engines for different usages or orientations: generic, security-oriented, scientific…
• Examples: runc, Kata Container, gVisor, wasmtime…

Containers

• Container: isolated and limited virtual copy of the host OS
  • Deploys the image to “fill in” the virtual copy
• Isolation: users, devices, processes…
  • Virtual filesystem: built from container image
• Limits: CPU, memory, I/O…
  • Also monitoring

Comparison with hardware virtualization: stack

Comparison with hardware virtualization: features

• Containers are better overall for cloud-native applications
  • Applications are architectured to be deployed on the cloud
• Security concerns
  • Kernel shared between containers
• VMs still have use cases: persistent, interactive environments, robustness, first-level resource provisioning…

Demo: Docker

• Creation and usage of a Docker container
  • Run an interactive image
  • Pull and run a daemon service
  • List images, monitor containers
• Docker is rather “low-level” for applications: compose multiple components (containers) in a single application with Docker Compose

Building containers: two ways

1. Interactively
   1. From a base distribution image
   • Linux distributions: Ubuntu, Alpine…
   • Runtime distributions (based on Linux distributions): Python…
   2. Use the package manager to add software
   3. docker commit tags the current state of the container as an image
   • Efficient for testing and experimenting
2. Writing a Dockerfile
   • DSL to describe how to install and configure the bundled applications
   • Proper method: clean, reusable, reproducible, auditable…

Building containers with a Dockerfile

# Starting from a base image.
FROM alpine

# Execute commands to build and configure the image.
RUN apk add --no-cache perl

# Add local files.
COPY cowsay /usr/local/bin/cowsay
COPY docker.cow /usr/local/share/cows/default.cow

# Set the default executable.
ENTRYPOINT ["/usr/local/bin/cowsay"]

• Build the image with docker build --tag namespace/name:tag
• Can start from an empty image: FROM scratch
  • Rarely used, only by base images of distributions, where the image is built from an archive
• May also include users, volumes, network ports…

Internal of a container engine

1. Isolation
2. Limit
3. Operation control
4. Virtual filesystem

Isolation: namespaces

• Provide an isolated view of the OS
• 8 dimensions:

Limit: control groups (cgroups)

• Constrain resource usage
  • Also prioritization, accounting, control
• 8 “dimensions” (controllers):
  • cpu: CPU time
  • cpuset: task placement on memory and CPU nodes
  • memory: memory usage
  • io: block I/O
  • pid: number of PIDs (i.e., of processes)
  • device: access to device files
    • special: only through BPF
  • perf_event: performance monitoring
  • net: network packets priority and classes for QoS
• Other specialized controllers: rdma, hugetlb, misc

Operation control: capabilities and MAC

• Capabilities: selectively drop root privileges
• Mandatory Access Control (MAC): system-level operational policies with Linux Security Modules
  • SELinux, AppArmor, seccomp…
• More than 40 capabilities (CAP_XXX):

Low-level view of a container engine

• Most features that make a container, come from the Linux kernel!

Demo: namespaces and cgroups

• Spawn a new process in namespaces
• Put a process in control groups
  • Set limit and monitor resource usage
• Using the virtual filesystem interface
  • There are also syscalls

Containers for the cloud

1. Application architecture in the cloud
2. Micro-services
3. Orchestration

Cloud application architecture

• Historic pattern: monolithic application
  • All components are ad-hoc, tightly coupled
• Unfit for the cloud
  • Must manage all components at once for scalability, deployment, service quality
  • Hard to reconfigure
• New paradigm enabled by containers: micro-services

Micro-services

Orchestration

• Composition: build applications as micro-services
  • Roughly: manage multiple containers as one application
  • Example: Docker Compose
• Orchestration: manage micro-services
  • Deployment
  • Distribution
  • Replication
  • Load-balancing
  • Availability
  • Rolling updates
  • …
• Orchestration exposes higher-level interfaces to the features of composition
  • In the end, the orchestrator is the user front-end
• Examples: Kubernetes, Docker Swarm
• Abstraction of management unit: the pod

Orchestration: scheduling

• Manual criteria: filters
  • Handle host heterogeneity
    • Settings of container runtime, host OS…
  • Container affinity: force placement for resource access
    • Image availability, volume placement, other container…
• Strategies for deployment on physical hosts
  • Spread: balance load over hosts
  • Binpack: colocate as much as possible
• Handle colocation of tightly-coupled containers: pods
  • Containers in a pod share the same network namespace and same volumes
  • Pod = service container + helper (sidecar) containers (logging, interfacing…)

Orchestration of pods

Demo: Kubernetes

• Create and use a pod
• Create and use a deployment
  • Scalability
  • Roll-out

Kubernetes application: example of deployment.yaml

kind: Deployment
# [...]
spec:
  # Scalability: set number of replicas.
  replicas: 3
  selector:
    matchLabels:
      app: simpleserver
  template:
    metadata:
      labels:
        app: simpleserver
    spec:
      # Pod: composition of containers.
      containers:
      - name: pythonserver
        image: python:simpleserver
        resources:
          requests:
            cpu: 0.5
        ports:
        - containerPort: 8080

Operating system-level virtualization

• Virtualize the OS instead of the hardware
  • Containers: simpler, lighter, faster
    • Not safer!
• Based on the Linux kernel (LinuX Containers, LXC): namespaces, cgroups, etc.
  • Container engines wrap those features and deliver unified specifications
  • Container runtimes bring usability, networking, development processes…
• Enabling new cloud-native application architecture: micro-services
  • Compositions of containers managed by orchestrators