The Stack

Runs the cluster’s automatic “ID-card office.” It issues the TLS certificates that make the padlock in https:// work, and renews them before they expire — so connections stay encrypted and trusted without anyone manually managing certificates.

A live status board for the cluster itself. Instead of resource usage, it reports the state of everything running — how many apps are deployed, how many are healthy, what’s pending or failing — so the health of the whole cluster can be tracked and alerted on.

A scheduler and queue for big batch compute jobs — the classic high-performance-computing (HPC) workflow used for things like model training. It works like a ticketing system: heavy jobs line up, and Slurm assigns each one to machines as capacity frees up, keeping expensive GPU hardware busy.

A smart traffic router for AI requests sitting in front of multiple vLLM model servers. Like a receptionist who remembers which desk already has your file open, it routes each request to the server that already has the relevant context cached — so answers come back faster and the GPUs are used more efficiently at scale.

Think of it as a strong isolation boundary clusterdOS offers — instead of sharing the host kernel, a Kata pod boots inside a real QEMU/KVM VM, so a guest kernel compromise never reaches the host or neighboring pods. It's the opt-in escape hatch for untrusted or multi-tenant workloads, and for anything gVisor's syscall sandbox can't run.

The actual “engine” that runs large language models. It loads an AI model onto GPUs and answers prompts efficiently, squeezing maximum throughput out of the hardware — the workhorse that turns GPUs into a working AI model server.

The cluster’s networking — the roads, wiring, and traffic cops that let all the apps find and talk to each other. Built on a fast in-kernel technology (eBPF), it also acts as a high-performance firewall and gives visibility into network traffic. It’s the very first thing clusterdOS starts, because nothing else can communicate until the network exists.

The cluster’s log plumbing. It collects the logs every app produces across all machines and pipes them into one central place, so problems can be searched and diagnosed from a single stream instead of hunting machine by machine.

The cluster’s speedometer and fuel gauge. It continuously reports how much CPU and memory each app is using — which powers kubectl top and lets clusterdOS automatically scale workloads up or down based on real-time demand.

Connects the cluster to extremely fast shared storage. It’s like giving every machine access to one giant, high-speed shared drive — important for AI/GPU work where many machines need to read the same massive datasets at the same time without waiting.

Acts as an in-house database administrator for PostgreSQL. It sets up, runs, and looks after Postgres databases inside the cluster so applications have a reliable place to store data, without a human hand-configuring each database.

Runs untrusted or risky programs inside a sandbox — a sealed “padded room” — so they can’t reach or harm the real machine underneath. clusterdOS uses it to safely run things like on-demand workspace/shell pods. It installs itself automatically onto nodes and re-installs after a machine is replaced, with no manual login required.

Takes inventory of each machine’s hardware and capabilities and puts labels (“stickers”) on them — what CPU features, what kind of GPU, etc. Those labels let clusterdOS send each workload to a machine that actually has the hardware it needs, instead of guessing.

Lets passwords and keys be stored safely, even in a public git repo. It works like a locked mailbox: anyone can drop in an encrypted secret, but only the cluster holds the key to open it. This is what makes clusterdOS’s “everything in git” (GitOps) approach safe for sensitive values.

The same automatic driver-and-plumbing setup as the NVIDIA operator, but for AMD GPUs. clusterdOS supports both vendors, so a cluster can run on whichever brand of accelerator the hardware happens to have.

Automatically installs and maintains everything a machine needs to use NVIDIA GPUs — drivers, the container runtime hooks, the device plugin, and monitoring. Like plugging in a graphics card and having the computer instantly install and keep the right drivers up to date, except clusterdOS does this across every GPU machine in the cluster so apps can actually “see” and share the GPUs.