Building Private Cloud Infrastructure across Three Colocation Data Centers.

Rick Fowler and Craig Smith are Systems Engineers at Databento. They work on the Infrastructure team, whose mission is to ensure stable operations of the network, compute, and storage systems, as well as to ensure DevOps and InfraOps engineers can elastically spin up network services such (i.e., load balancers, V-Nets, NAT, DHCP, VPN, etc.)

Standing up Databento’s Private Cloud

During Databento’s early seed stage days, Luca Lin, the founding CTO, had originally put together a simple two-track infrastructure to get Databento off the ground. 

As business picked up and started growing fast, Rick Fowler was hired by Luca Lin further to redesign and stand up their version 2 infrastructure with growth, agility, and scalability in mind. They wanted to achieve cloud-like operational efficiency but run their own infrastructure in colocation data centers.

Evaluating the Existing Technology Stack

Rick first examined the existing network toolset to see if it would solve his problem. That initial setup was based on traditional enterprise data center architecture with manual CLI configuration of switches and routers, storing IP address subnet and configuration info in spreadsheets, and it lacked any automation. It was a simple solution for the early days, never meant for scalability. 

Rick is savvy with Ansible, so he initially considered buying Cisco/Juniper routers and firewalls, F5 for load balancing, and some switches and leveraging Ansible for automation. But this solution had three significant downsides for Databento. 1) These traditional networking vendors provide very little abstraction. Even with the most advanced Ansible skills, Rick must design imperative procedural automation, maintain it across multiple vendors, and apply individual logic for each network functionality. It’s very time-consuming and prone to human error. 2) DevOps and InfraOps engineers would not get any self-serviceability; every request would need to go through Rick and Craig, the Network Engineering team introducing delays and risk for configuration drift. 3) Hard to ask for vendor help when multiple products are involved — Lack of centralized support (single throat to choke).

Rick wanted, and Luca required, a general and elegant solution to let them quickly and easily manage the infrastructure and provide self-serviceability to the entire engineering team. They needed to make infrastructure robust and highly scalable – but preserve the startup-spirited agility.

Netris to the Rescue

One of Databento’s investors is Mike Dvorkin, a well-known figure in the network infrastructure space commonly referred to as “the creator of Cisco ACI,” and he suggested Rick check out Netris. 

Netris allows engineers to design, build, and operate fully automated network infrastructures providing the entire engineering team with cloud-like VPC networking abstraction via the intuitive web console, API, and IaC (Infrastructure as Code). Netris leverages SmartNIC accelerated Linux networking running over conventional servers for (Layer 3-7) VPC networking services (Routing, firewalling, load balancing, NAT, DHCP, VPN, etc.) For physical (Layer 1/2) switching, Netris can automate the lifecycle of open-networking switches (Nvidia, Dell, Edge-Core) with Linux OS (Cumulus, SONiC, Ubuntu/SwitchDev), can automate API-driven bare-metal platforms (like Equinix, PhoenixNAP, etc.), or can run anywhere over ethernet switches statically pre-configured with a carved-out range of VLANs.  

Before investing in hardware, how do you evaluate a solution of this scale (3 regions, six border routers, 30+ switches)? 

Rick has used Netris’s on-demand sandbox to learn and evaluate the platform. He reviewed pre-configured examples and walked through step-by-step “learning by doing” guide. Netris Sandbox quickly allowed him to simulate his desired architecture, create the anticipated services, and perform end-to-end testing with realistic end-user traffic flows. Unlike Cisco/Juniper/F5 sales motion, there’s no need to wait for loaner equipment – Netris Sandbox is available immediately at https://netris.io/sandbox.

For switches (Layer 1-2), Rick chose Nvidia Spectrum switches (formerly Mellanox) with 100Gbps ports and Cumulus Linux NOS because he worked with Nvidia/Mellanox switches before at Canonical (the maintainer of Ubuntu Linux) and was impressed with the hardware quality and low switching latency.

Instead of buying Cisco/Juniper routers, firewalls, and F5, Rick chose Netris SoftGate PRO, the 100Gbps version of the VPC gateway combined with the Nvidia ConnectX-6 SmartNIC card running on a Dell server with dual Intel Xeon CPUs.  

Rick says, “I could use Ansible to scratch-build in-house automation and undertake the ongoing maintenance of my automation pipeline, which would take much of my time. Also, my DevOps/InfraOps colleagues may not be able to self-service. Netris came with my desired automation already built-in and with cloud-like VPC Networking abstraction that our entire engineering team found intuitive. When I need automation specific to Databento’s business, I can leverage Netris API through config management like Ansible, Salt, or Terraform, or use Python.”

Netris subscription includes 24/7/365 support which is responsible for the end-to-end operation of the entire system, not just Netris software. Even if there is a hardware issue Netris support team can help to narrow that down and make it easier for you to work it out with your hardware supplier, or if there is an issue with one of the open-source packages, Netris will take care of that too. 

Rick says, “Working with Netris TAC over slack feels like working with colleagues on my team.”