I Replaced Our Entire Kubernetes Cluster with a Single Erlang Node
Andika's AI AssistantPenulis
I Replaced Our Entire Kubernetes Cluster with a Single Erlang Node
The modern DevOps landscape is currently obsessed with complexity. For years, we followed the industry trend of containerizing every minor function, resulting in a sprawling infrastructure that required a dedicated team just to keep the lights on. However, after months of battling YAML indentation, persistent volume claims, and networking latency, I made a radical decision: I replaced our entire Kubernetes cluster with a single Erlang node.
This move wasn't just about reducing our cloud bill; it was a fundamental shift in how we perceive distributed systems and fault tolerance. While Kubernetes is an incredible tool for massive-scale orchestration, many mid-sized engineering teams are finding that the "Kubernetes tax" outweighs the benefits. By leveraging the power of the BEAM VM (the Erlang Virtual Machine), we achieved better performance, higher reliability, and a significantly simplified deployment pipeline.
The Kubernetes Complexity Tax
Before the migration, our infrastructure consisted of a standard managed Kubernetes (EKS) cluster. We were running roughly 25 microservices, each with its own Dockerfile, CI/CD pipeline, and resource limits. To manage communication between these services, we had to implement a service mesh, configure Ingress controllers, and manage complex secrets.
The operational overhead was staggering. We spent more time debugging Kubelet issues and OOMKills than writing features. The "Kubernetes tax" manifested in three ways:
Resource Overhead: Each pod required its own overhead for the container runtime and sidecar proxies.
Cognitive Load: Developers needed to understand Helm charts and Kubernetes manifests just to deploy a "Hello World" service.
Networking Latency: Every internal request traveled through multiple layers of virtual networking, adding milliseconds to every transaction.
By moving to a single Erlang node, we eliminated these layers entirely. In the Erlang ecosystem, "services" are simply processes within the VM, communicating via high-speed internal messaging rather than HTTP calls over a virtual network.
Why Erlang? The Power of the BEAM VM
To understand why a single Erlang node can replace a cluster of containers, you have to understand the Actor Model. Erlang was designed by Ericsson in the 1980s to handle telecommunications switching—systems that require "nine fives" (99.9999999%) of uptime.
Fault Tolerance and the "Let it Crash" Philosophy
In a traditional Kubernetes environment, if a service fails, the orchestrator detects the failure and restarts the container. This is a slow, heavy-handed approach to recovery. In contrast, Erlang utilizes a Supervision Tree architecture.
If a specific process within the Erlang node fails, its supervisor instantly restarts it in a clean state. This happens in microseconds, not seconds. Because Erlang processes are incredibly lightweight (costing only a few kilobytes of RAM), a single node can easily manage millions of them simultaneously.
Native Distribution Without the Service Mesh
One of the primary reasons people choose Kubernetes is for service discovery and distribution. However, Erlang has native distribution baked into the language. You can send a message to a process on a different node as easily as sending one to a local process. While we consolidated to a single large bare-metal node for simplicity, the system remains "distributed" internally, allowing us to scale back out to a cluster of Erlang nodes seamlessly without adding an external orchestrator.
The Migration Strategy: From 50 Pods to One Node
The transition required us to move away from the "polyglot microservices" mindset and embrace a modular monolith approach using Elixir (a modern language built on the Erlang VM). We consolidated our various functional areas—authentication, billing, and data processing—into separate applications within a single Umbrella project.
Instead of managing 25 separate deployments, we now manage one. Here is a simplified example of how we handle internal "service" calls now:
# Instead of an HTTP call to a Billing Service# GenServer.call(:billing_service, {:process_payment, user_id, amount})defmoduleMyApp.BillingdouseGenServerdefhandle_call({:process_payment, user_id, amount}, _from, state)do# Logic for processing payment{:reply,:ok, state}endend
By using internal message passing, we removed the need for JSON serialization, API gateways, and internal load balancers. The performance gains were immediate.
Performance Benchmarks and Resource Efficiency
The most shocking result of the migration was the reduction in resource consumption. Our Kubernetes cluster was costing us approximately $2,500 per month in cloud fees. The single Erlang node, running on a high-end dedicated server, costs $300 per month.
We found that the BEAM VM's scheduler is far more efficient at utilizing CPU cores than the Linux kernel is at balancing dozens of competing Docker containers. Because Erlang uses "preemptive multitasking" at the VM level, one runaway process cannot starve the rest of the system of resources.
Operational Simplicity: Deployment and Maintenance
One of the greatest features of Erlang is Hot Code Swapping. In a Kubernetes environment, a deployment involves building a new image, pushing it to a registry, and performing a rolling update. This often results in dropped connections or state loss.
With Erlang, we can technically update the code of a running system without stopping it. While we typically use "blue-green" deployments for safety, the ability to upgrade stateful processes in-place is a superpower for maintaining high availability.
Furthermore, monitoring became significantly easier. Instead of aggregating logs from 25 different containers into a costly ELK stack, we use Observer, a built-in Erlang tool that provides a real-time graphical interface into the VM's processes, memory usage, and load.
Is This Right for You? (The Nuance)
I am not suggesting that Kubernetes is obsolete. If you are operating at the scale of Netflix or Uber, or if your team is forced to use five different programming languages, container orchestration is a necessity.
However, for startups and mid-sized companies, the Erlang/Elixir ecosystem offers a compelling alternative. If you can standardize your backend on the BEAM, you gain the benefits of a distributed system without the "distributed systems overhead." You trade YAML configuration for actual application logic.
Conclusion: Embracing Radical Simplification
By deciding to replace our entire Kubernetes cluster with a single Erlang node, we didn't just save money; we reclaimed our engineering velocity. We no longer spend our mornings debugging ingress controllers or waiting for pods to initialize. Instead, we focus on building features within a stable, high-performance environment designed specifically for concurrency.
If you find yourself overwhelmed by the complexity of modern cloud-native tooling, it might be time to look back at the battle-tested technologies of the past. Sometimes, the best way to scale up is to simplify down.
Ready to simplify your stack? Start exploring the Elixir language and the Phoenix Framework to see how the BEAM VM can transform your development workflow.
Created by Andika's AI Assistant
Full-stack developer passionate about building great user experiences. Writing about web development, React, and everything in between.
