If vLLM already solved LLM serving, why did SGLang appear?
Introduction
People often call Kubernetes a container orchestrator, but that’s just part of the story. At its heart, Kubernetes is a distributed control system. At scale, this control-loop design starts to look like a full AI tech stack problem, not just orchestration. It doesn’t run applications on its own. Instead, it keeps an eye on what you want, checks what’s actually happening, and works to bring the two together.
This difference is important. With Kubernetes, your commands don’t trigger actions right away like on older platforms. When you apply a manifest, nothing starts instantly. Instead, different parts of Kubernetes figure out what you want, talk to each other, and eventually get containers running on the right machines.
That work is cleanly divided into two halves of the system: the control plane and the worker nodes. One decides what should happen. The other ensures it actually happens. Understanding how these roles interact is essential for anyone running Kubernetes beyond small experiments especially when clusters scale, workloads become dynamic, and reliability matters.
The Control Plane: Where Kubernetes Thinks
The control plane acts as the brain of a Kubernetes cluster. It doesn’t run your apps. Its only job is to keep the cluster in the right state. All decisions, such as scheduling, scaling, or fixing issues, go through this layer.
At the center of the control plane sits the API server. This is the only component that exposes Kubernetes to the outside world. API reliability becomes even more visible once teams standardize on AI infrastructure as a service. Whether a command comes from kubectl, a CI pipeline, or an internal controller, it enters the system through the API server.
The API server checks requests, making sure they’re allowed and make sense for the cluster. It doesn’t make changes itself; it just records what you want to happen.
Your intentions are saved in etcd, the cluster’s memory. Everything pods, nodes, services, secrets, and more is kept there. If etcd fails, the cluster forgets everything. That’s why keeping etcd stable and fast is so important.
Once intent is recorded, other control plane components react.
Scheduling: Turning Intent into Placement
The scheduler watches the API server for newly created pods that have not yet been assigned to a node. Its job is not to start containers, but to decide where they should run.
To decide, the scheduler considers factors such as resource requirements and rules governing where pods can run. It picks the best node and updates the API server with its choice.
At this point, the control plane has finished deciding. Execution now shifts to the worker nodes.
This division becomes especially important when compute is scarce and enterprise GPU cloud capacity needs to be governed.
Controllers: Continuous Reconciliation
While the scheduler handles placement, controllers handle consistency. The controller manager runs a collection of controllers, each responsible for a specific resource type deployments, replica sets, nodes, endpoints, jobs, and many others.
Controllers follow a simple loop – check what’s happening, compare it to what should happen, and fix things if they’re off. If you want five pods and only four are running, the controller starts another. If a node goes down, it moves the pods to another node.
This reconciliation loop never stops. Kubernetes does not assume success. It continuously verifies and corrects. This is why Kubernetes can recover from failures without explicit instructions.
The control plane, as a whole, never touches containers. It only expresses intent.
Great AI Workloads Need Kubernetes That Doesn’t Flinch
Keep every layer of the Kubernetes environment tuned, stable, and ready.
Worker Nodes: Where Kubernetes Acts
Worker nodes do the real work. Each one runs a few small programs that turn the control plane’s decisions into action.
The most important of these is the kubelet. The kubelet runs on every worker node and serves as the node’s representative in the cluster. It monitors the API server for pods scheduled to its node and ensures those pods run as specified.
When kubelet gets a new pod to run, it downloads the required images, starts the containers, sets up networking, mounts storage, and monitors their health. If something crashes, the kubelet restarts it. If you change a pod, the kubelet updates it.
In essence, the kubelet is the execution engine of Kubernetes.
The Container Runtime: Running the Containers
The container runtime is responsible for actually starting and stopping containers. Kubernetes does not mandate a specific runtime; it communicates through a standard interface.
The runtime handles low level concerns such as process isolation, file system layers, and container lifecycle. From Kubernetes’ perspective, the runtime is a tool. From the node’s perspective, it is the mechanism that turns abstract pod specifications into running processes.
This separation allows Kubernetes to evolve independently of container technology while maintaining consistency across environments.
Kube-proxy and Service Networking
While kubelet focuses on pods, kube-proxy handles service networking. Kubernetes services provide stable virtual IPs that route traffic to dynamic sets of pods. kube-proxy programs the node’s networking rules so that traffic addressed to a service is forwarded to one of its backing pods.
This routing just works in the background. Apps connect to services without caring which pod answers. kube-proxy keeps traffic flowing even as pods change.
Although kube-proxy does not decide what runs where, it plays a crucial role in ensuring that what runs can be reached reliably.
How Control Plane and Workers Interact
The interaction between the control plane and worker nodes is intentionally indirect. There is no central coordinator issuing commands in real time. Instead, everything flows through a shared state in the API server.
The control plane writes down what should happen. Worker nodes read this and act. Then they send updates back, and the controllers check whether anything needs fixing.
This design creates loose coupling. Nodes can fail and rejoin. Control plane components can restart. The system remains coherent because the state lives in one place and reconciliation never stops.
This is also why Kubernetes scales. Decisions are centralized. Execution is distributed.
What Can Go Wrong as You Grow
When your setup is small, you barely notice how Kubernetes works behind the scenes. But as things get bigger, you start to feel it.
If the API server is slow, everything feels sluggish. If etcd can’t keep up, fixes don’t happen quickly. If the scheduler is slow, pods take longer to start. And if the kubelet has trouble, recovery can be rough. Networking and storage add their own challenges, too.
If these parts aren’t set up well together, problems creep in. Pods might start slowly. Nodes fall out of sync. Failures can pile up rather than be fixed quickly. Debugging becomes chasing issues rather than preventing them.
Understanding the division of responsibility is the first step toward designing clusters that remain stable under load.
The Neysa Perspective: Control and Execution as a Unified System
At Neysa, Kubernetes is treated as a coordinated system rather than a collection of independent components. Control plane behavior, node execution, networking, and storage are designed together because production workloads do not respect component boundaries when they fail.
Velocis environments are engineered so that control plane operations remain responsive even as clusters grow and workloads intensify. API servers are provisioned and observed with the same rigor as inference endpoints. etcd performance is protected, so reconciliation remains fast and predictable. Worker nodes are configured to execute scheduling decisions without bottlenecks or resource contention.
For AI workloads, this alignment matters deeply. Training jobs stress scheduling. Inference pipelines stress networking. Auto scaling stresses controllers. When control and execution layers are tuned together, Kubernetes behaves as an intended system that adapts continuously without operator intervention.
Teams do not need to reason about which component failed first. The platform absorbs complexity so Kubernetes remains a reliable substrate for intelligent systems.
Conclusion
Kubernetes succeeds by separating planning from execution. The control plane figures out what the cluster should be. Worker nodes make it real. They communicate by sharing updates and always checking for changes.
This architecture allows Kubernetes to scale, heal, and adapt, but it also means reliability depends on how well these parts work together. At scale, Kubernetes is not just an orchestrator. It is a distributed operating system.
Platforms like Neysa Velocis embrace this reality. By treating the control plane and worker nodes as parts of a single, intentional design, they allow Kubernetes to operate the way it was meant to – decisively, predictably, and without friction.
In today’s world, you don’t need to know every part by heart. What matters is building systems where all the pieces work together smoothly, even when things get tough.
