Build the Network Automation Layer First

Assumption 1: The vendor stands it up, then we’re done

A hardware vendor or professional implementation team can do an excellent job bringing up an AI cluster, but that doesn’t mean it has built the system you’ll use to operate the network for years afterward.

Notice in the image below that the initial deployment is a brief, though critical, first step. After that, the ongoing operations of day 2 and beyond consume the vast majority of resources and effort in operating an AI cloud.

The implementation team has a specific objective, which is to deploy the infrastructure, configure it, validate it, bring it online, and move to the next project. It’s important to understand that this isn’t a failure of the hardware vendor. It’s actually an incentive mismatch. The hardware transaction is designed to get the cluster deployed and operational. The cloud operator, however, owns Day 2 as well as the years of changes that follow, and those changes begin quickly.

From a networking perspective, this means making frequent changes across tens of thousands of links (or more) as customers are added, resized, or removed. Those changes must be applied consistently across the entire environment while maintaining the security and isolation required by customers and regulators.

Assumption 2: The compute orchestrator handles networking

Compute orchestration and network orchestration solve different problems. A compute orchestrator can determine where a workload can run, which resources it needs, and where a tenant should live to provide a “soft” isolation, or in other words, the isolation of tenant workloads in software at the compute layer. However, it doesn’t translate those requirements across the physical network. Instead, “hard” isolation (enforced on networking hardware) provides true multi-tenancy through workload isolation enforced on networking hardware across bare metal, VM, and container workloads, on every fabric Netris manages.

This is important because in AI infrastructure, tenant isolation can’t exist only at the compute or software layer. It also has to be enforced in hardware across the entire network. As workloads are provisioned, changed, and removed, the relevant network state has to change across VLANs, VXLANs, routing, switch ports, and network-adjacent services.

In the next graphic, we can see that the compute orchestrator is just one piece of the entire workflow. The network orchestrator takes that intent from the compute orchestrator and translates it into the actual configuration required in the physical world.

This distinction matters because many compute orchestration platforms claim to handle networking, and at the software layer, they do. An operator might define a new tenant network, subnet, or security policy through the compute orchestrator and reasonably assume the networking is taken care of.

But consider what happens when a new tenant workload is provisioned. The compute orchestrator may create the logical network and determine which servers will run the workload. It does not necessarily configure the physical switches carrying that traffic. The appropriate VLANs still need to be provisioned across the right leaf switches, routing and policy need to be applied, and the tenant needs to remain isolated from every other tenant as traffic moves across the physical network.

That is the gap the network orchestration layer fills. It takes the intent defined at the compute layer and translates it into the configuration and policy required across the underlying network infrastructure.

This becomes especially important in highly regulated industries, sovereign AI clouds, and high-security environments, where software-based workload isolation alone may not satisfy infrastructure requirements. Hard isolation may need to be enforced by the network itself.

The challenge goes beyond isolation. AI data center networks can span thousands of devices and hundreds of thousands of interfaces. The compute orchestrator shouldn’t need to understand that complexity, but something does. With human error responsible for a significant share of network outages, automating changes and abstracting complexity is an essential part of operating AI infrastructure at scale.

Assumption 3: Bare metal or single-tenant means no multi-tenancy

Imagine an AI cloud with exactly one major customer. Everything is running fine. The network is humming along, GPUs are communicating, and the customer is happy. Then, a GPU server fails inside that customer’s environment.

This is where it starts to get interesting. The server may still contain sensitive customer data, so the operator can’t just send a tech to the rack to start swapping fans and hard drives. The customer likely requires its data to be destroyed using its own tools and the server to be programmatically removed from its network environment before anyone can physically service it.

Only then can the failed system be moved into a service or maintenance environment where the operator’s technicians can work on it. That means even a supposedly “single-tenant” cloud already has multiple operational domains.

Multi-tenancy isn’t only about putting Customer A next to Customer B. It’s about controlling which infrastructure belongs to which environment, what that infrastructure can communicate with, and how those relationships change throughout its lifecycle.

As you can see in the image above, once you account for the production environment, service or maintenance domains, management, and multiple network planes, even a single-customer AI cloud requires multi-tenant network operations.

Assumption 4: We can build our own automation

Networking is a mature discipline, so a common assumption is to just hire experienced network engineers, give them the infrastructure, and let them run it.

But when an AI cloud needs to change dynamically, something else starts happening. The engineers begin building the missing operational layer themselves. They extend the scripts used during bring-up, perhaps adding Ansible playbooks, Python tools, Git repositories, and CI/CD pipelines. They create bespoke methods to encode device dependencies, tenant requirements, network state, and operational processes into custom automation.

In an AI infrastructure, scripts, playbooks, and repositories have to be updated every time a new customer is onboarded, resized, or decommissioned. In effect, what initially seemed like a programmatic strategy for managing the network becomes almost as manual as managing devices one at a time.

Eventually, the organization isn’t simply operating a network. It’s developing network automation software, and that distinction matters a lot to the business.

The company may now own a proprietary software layer it never really wanted to build. Critical operational knowledge may live inside scripts, repositories, workflows, and inside the heads of one or two engineers who understand how everything fits together.

That’s a lot of risk, and the business didn’t necessarily decide to take on that risk when it set out to build and operate an AI cloud. It happened because something had to fill the gap.

The answer isn’t to replace those engineers. Experienced network engineers are still extremely important. What we want to do is augment the engineer and make their expertise scalable.

Instead of requiring a handful of people to understand how every change maps across thousands of switches and dependencies, the operational knowledge can be encoded into an abstraction that allows the team to express what the network should do.

Think of it as giving the engineer ten meaningful controls instead of a dashboard with thousands of individual knobs.