Data Center Network Architecture: Leaf-Spine, ECMP & VXLAN

data center network architecture

Data Center Network Architecture Explained: Leaf-Spine, Core, Access and East-West Traffic

Modern data centers depend on far more than fast switches.

Applications now communicate with databases, caches, storage systems, container clusters, APIs, security services, and other application components inside the same data center. As a result, the network must handle large volumes of both north-south traffic between users and applications and east-west traffic between systems inside the data center.

This is where data center network architecture becomes critical.

A traditional architecture might resemble:

Internet
   |
   v
Core
   |
   v
Aggregation
   |
   v
Access
   |
   v
Servers

A modern data center may instead use a leaf-spine fabric:

             Spine 1       Spine 2
             /  |  \       /  |  \
            /   |   \     /   |   \
           /    |    \   /    |    \
        Leaf 1 Leaf 2 Leaf 3 Leaf 4
          |      |      |      |
        Servers Servers Storage Servers

Every leaf typically connects to every spine, creating multiple equal-cost paths across the fabric. Cisco describes spine-leaf as a common architecture for modern data centers, particularly where east-west traffic has increased because of distributed applications. Cisco

This guide explains how traditional three-tier architecture compares with leaf-spine networking, how east-west and north-south traffic move through a data center, and where technologies such as ECMP, BGP, VXLAN, EVPN, VLANs, Top-of-Rack switching, border leafs, load balancing, security, and network monitoring fit into the design.

For the broader infrastructure layer, start with Data Center Architecture Explained and Servers & Infrastructure Explained.


What Is Data Center Network Architecture?

Data center network architecture defines how servers, storage systems, switches, routers, security devices, load balancers, and external networks are connected and how traffic moves among them.

A production design must answer questions such as:

  • How do servers communicate with each other?
  • How does traffic enter and leave the data center?
  • What happens when a switch fails?
  • Are multiple equal-cost paths available?
  • Where are routing boundaries placed?
  • How large should Layer 2 domains be?
  • How are workloads segmented?
  • How is east-west traffic secured?
  • How are multiple tenants isolated?
  • How does the network scale when hundreds or thousands of servers are added?
  • How are failures detected and routed around?

A simplified architecture can be represented as:

                    External Networks
                           |
                           v
                     Border Layer
                           |
                           v
                     Data Center Fabric
                      /      |      \
                     /       |       \
                    v        v        v
                Compute   Storage   Services

The correct architecture depends on traffic patterns, scale, availability requirements, workload type, and operational complexity.


Traditional Three-Tier Data Center Architecture

For many years, enterprise networks commonly used a hierarchical three-tier structure:

                  Core Layer
                 /          \
                /            \
        Aggregation 1     Aggregation 2
           /    \            /    \
          /      \          /      \
      Access    Access    Access    Access
        |         |         |         |
      Servers   Servers   Servers   Servers

The three logical layers are:

Core

The core provides high-speed connectivity between major sections of the infrastructure and often connects toward WAN, Internet, or other network domains.

Aggregation or Distribution

The aggregation layer combines traffic from access switches and may provide:

  • Layer 3 routing
  • Policy enforcement
  • VLAN boundaries
  • Redundancy
  • Service insertion

Access

Access switches connect directly to:

  • Servers
  • Hypervisors
  • Storage systems
  • Appliances

This architecture remains useful in some environments, particularly where traffic is strongly hierarchical.


Limitations of Traditional Three-Tier Designs

Traditional hierarchical architectures were built during a period when much data-center traffic primarily traveled between internal servers and external clients.

That traffic pattern is often described as north-south.

User
 |
 v
Internet
 |
 v
Core
 |
 v
Aggregation
 |
 v
Access
 |
 v
Server

Modern distributed applications can generate far more server-to-server communication.

For example:

Web Server
     |
     v
Application Server
     |
     v
Cache
     |
     v
Database

That traffic moves horizontally within the data center.

Traditional Layer 2 designs may also depend heavily on Spanning Tree Protocol, where some redundant links can remain blocked to avoid loops rather than actively carrying traffic.

Cisco’s documentation contrasts these traditional designs with spine-leaf fabrics, which use multiple active paths and are optimized for modern east-west traffic patterns. Cisco


North-South vs East-West Traffic

Understanding traffic direction is fundamental to data center network design.

North-South Traffic

North-south traffic enters or leaves the data center.

Examples include:

Internet User
      |
      v
Data Center Application

or:

Branch Office
     |
     v
WAN
     |
     v
Data Center

Typical north-south traffic may pass through:

Internet
   |
Firewall
   |
Load Balancer
   |
Application

This makes components such as border routers, firewalls, NAT gateways, and load balancers important in north-south design.

For a deeper explanation of traffic distribution, see Load Balancing Explained.


East-West Traffic

East-west traffic moves between systems inside the data center.

For example:

Frontend
   |
   v
API
   |
   v
Cache
   |
   v
Database

or:

VM A ------> VM B

Modern cloud applications, virtualization, Kubernetes platforms, distributed databases, analytics systems, and storage clusters can generate large amounts of east-west traffic.

Cisco explicitly identifies increased east-west traffic as one of the reasons modern data center fabrics use spine-leaf topology. Cisco


What Is Leaf-Spine Architecture?

A leaf-spine architecture normally contains two major switching layers:

                 Spine Layer

              Spine1      Spine2
              / |  \      / |  \
             /  |   \    /  |   \
            /   |    \  /   |    \

                 Leaf Layer

           Leaf1   Leaf2   Leaf3   Leaf4
             |       |       |       |
          Servers Servers Storage Servers

Each leaf normally connects to every spine.

Leaf switches do not need to connect directly to every other leaf because traffic between leaves traverses the spine layer.

Likewise, in the canonical Clos-style topology, the spine layer provides transit between leaves rather than building a traditional three-tier hierarchy.

Cisco’s recent VXLAN/EVPN design guidance describes spine switches as the underlay transit layer through which leaf-to-leaf east-west traffic crosses the fabric. Cisco


Leaf Switches

Leaf switches connect workloads to the fabric.

Typical endpoints include:

Leaf Switch
   |
   +-- Server
   +-- Hypervisor
   +-- Storage Node
   +-- Firewall
   +-- Load Balancer
   +-- Kubernetes Node

A leaf may therefore act as the first network hop for servers.

Depending on the design, it can also perform:

  • Layer 2 switching
  • Layer 3 routing
  • VXLAN tunnel endpoint functions
  • Policy enforcement
  • Anycast gateway functions

The exact feature set depends on the chosen architecture.


Spine Switches

Spine switches provide high-speed transit between leaf switches.

Conceptually:

Leaf A
   |
   +-------- Spine 1 -------+
   |                        |
   +-------- Spine 2 -------+---- Leaf B

Rather than forcing traffic through a single core switch, several equivalent paths can exist.

Cisco’s VXLAN BGP EVPN design guide describes the spine as providing underlay connectivity among leaf nodes, with leaf-to-leaf tenant traffic crossing the spine. Cisco


Why Every Leaf Connects to Every Spine

Consider four leaves and two spines:

                  Spine 1        Spine 2
               /   |   |  \   /  |   |   \
              /    |   |   \ /   |   |    \
          Leaf1  Leaf2 Leaf3 Leaf4

Each leaf can reach another leaf through either spine.

For example:

Server A
   |
Leaf 1
   |
   +---- Spine 1 ----+
   |                 |
   +---- Spine 2 ----+
                     |
                   Leaf 4
                     |
                  Server B

This provides multiple paths with similar topology depth.

It is one of the key reasons leaf-spine architectures can provide predictable latency characteristics compared with networks where some server-to-server paths cross very different numbers of network tiers.


What Is ECMP?

ECMP — Equal-Cost Multi-Path — allows routing to use multiple paths with equal routing cost.

Suppose Leaf 1 needs to reach Leaf 4.

              Spine 1
             /       \
Leaf 1 -----           ----- Leaf 4
             \       /
              Spine 2

Both paths may be eligible:

Leaf1 → Spine1 → Leaf4

Leaf1 → Spine2 → Leaf4

Instead of keeping one path idle, the network can distribute flows across multiple available paths.

This improves utilization and resilience.

RFC 7938 describes large-scale data center architectures using BGP and explicitly discusses ECMP as part of horizontally scaled Clos-style networks. RFC Editor


What Happens When a Spine Switch Fails?

Consider:

Spine 1 ✓
Spine 2 ✓

Leaf switches have paths through both.

If Spine 1 fails:

Spine 1 X
Spine 2 ✓

traffic can continue through Spine 2, assuming capacity and routing convergence are sufficient.

The result is:

Before Failure
Path 1 + Path 2

After Failure
Path 2

Availability therefore depends not only on redundant paths existing, but also on ensuring the surviving infrastructure has sufficient capacity to carry traffic during failures.


Top-of-Rack Switching

A common data center design places a leaf switch at or near the top of a server rack.

This is called Top-of-Rack — ToR — switching.

Rack

+-------------------+
| ToR Switch        |
+-------------------+
| Server 1          |
| Server 2          |
| Server 3          |
| Server 4          |
| Storage Node      |
+-------------------+

Servers use short local cables to connect to the ToR switch.

The ToR then connects upward to the spine layer.

Server
  |
  v
ToR / Leaf
  |
  +------ Spine 1
  |
  +------ Spine 2

RFC 8365 specifically recognizes designs where the network virtualization edge can reside on a Top-of-Rack device. RFC Editor


Top-of-Rack vs End-of-Row

Another model is End-of-Row — EoR — switching.

Instead of putting a switch in every rack:

Rack 1 ----\
Rack 2 -----\
Rack 3 ------ EoR Switch
Rack 4 -----/

the switch serves several racks.

ToR advantages can include:

  • Shorter server cables
  • More modular racks
  • Easier rack-level expansion
  • Failure isolation

EoR advantages can include:

  • Fewer switches
  • Potentially centralized management
  • Different port-utilization economics

There is no universally correct choice.

The physical topology should match rack density, cabling constraints, port requirements, failure domains, and operational model.


Leaf-Spine and Oversubscription

A leaf-spine network is not automatically non-blocking.

Suppose a leaf connects:

48 × 25 Gbps server ports

Maximum theoretical server-facing bandwidth is:

48 × 25
=
1,200 Gbps

Now suppose it has:

4 × 100 Gbps uplinks
=
400 Gbps

Then the access side contains significantly more theoretical capacity than the uplinks.

That architecture is oversubscribed.

A simplified ratio is:

Downlink Capacity
-----------------
Uplink Capacity

In this example:

1200 / 400
=
3:1

Whether this is acceptable depends on actual traffic patterns.

A highly east-west intensive storage or AI workload may require significantly more fabric bandwidth than ordinary enterprise application servers.


Scaling a Leaf-Spine Fabric

Suppose the current architecture has:

2 Spine Switches
4 Leaf Switches

More server capacity may require adding another leaf:

                 Spine1     Spine2
               / | | | \   / | | | \
              /  | | |  \ /  | | |  \
           Leaf1 Leaf2 Leaf3 Leaf4 Leaf5

The new leaf connects to all spines.

This allows horizontal expansion of access capacity without completely redesigning the network.

Eventually, however, spine port capacity becomes a scaling limit.

At much larger scale, architectures can introduce additional hierarchy through multiple fabrics, super-spines, or Clos stages.


Layer 2 vs Layer 3 in the Data Center

Traditional data centers often extended large Layer 2 domains across many switches.

Modern fabrics frequently push Layer 3 routing closer to the servers.

Traditional:

Server
  |
Layer 2
  |
Layer 2
  |
Gateway

Routed fabric:

Server
  |
Leaf
  |
Layer 3 Fabric
  |
Leaf
  |
Destination

One reason for using routed underlays is that IP routing can take advantage of multiple active paths without relying on spanning-tree blocking.

For networking fundamentals, see Routers, Switches and Network Devices Explained.


VLANs in Data Center Networks

VLANs remain useful for logical Layer 2 segmentation.

For example:

VLAN 10 → Web
VLAN 20 → Application
VLAN 30 → Database
VLAN 40 → Management

However, large virtualized data centers can require significantly more segmentation and mobility than traditional VLAN-only architectures were designed to provide.

For a practical explanation of VLAN tagging, access ports, and trunks, see VLAN Explained: Access and Trunk Ports.


What Is VXLAN?

VXLAN — Virtual eXtensible LAN — provides an overlay mechanism that can carry Layer 2 network segments across a Layer 3 IP network.

Conceptually:

Tenant Network
      |
      v
   VTEP A
      |
      |  Layer 3 Underlay
      |
      v
   VTEP B
      |
      v
Tenant Network

RFC 7348 describes VXLAN as a Layer 2 overlay over a Layer 3 network and defines the VXLAN Network Identifier — VNI — used to identify overlay segments. RFC Editor

VXLAN is particularly useful where many virtualized workloads or tenants need isolated logical networks while the physical network remains routed.


Underlay vs Overlay Network

VXLAN-based data centers introduce an important distinction.

Underlay

The underlay is the physical IP network.

Leaf
 |
Spine
 |
Leaf

Its job is primarily to provide IP reachability between fabric endpoints.

Overlay

The overlay creates logical tenant networks on top of that infrastructure.

Tenant A
VNI 10010

Tenant B
VNI 10020

Tenant C
VNI 10030

Conceptually:

          Logical Overlay
       Tenant A / Tenant B
                |
                v
       -------------------
         IP Underlay
       -------------------
                |
        Leaf-Spine Fabric

RFC 8365 assumes IP connectivity in the underlay and defines how EVPN can support network virtualization overlays, including VXLAN encapsulation. RFC Editor


What Is BGP EVPN?

VXLAN provides a data-plane encapsulation mechanism.

The network still needs a scalable method of distributing endpoint and reachability information.

EVPN — Ethernet VPN — uses BGP as a control-plane mechanism.

A simplified architecture is:

           BGP EVPN Control Plane
             <------------>
        Leaf A            Leaf B
          |                 |
        VTEP A            VTEP B
          \                 /
           \               /
               VXLAN

RFC 8365 specifies the use of EVPN as a network virtualization overlay solution and discusses VXLAN as one of its encapsulation options. RFC Editor

This combination is commonly referred to as:

VXLAN BGP EVPN


Why BGP Is Used in Modern Data Centers

BGP was originally associated primarily with Internet routing, but it is also used inside large data centers.

RFC 7938 documents operational experience using eBGP as the routing protocol in large-scale data centers, emphasizing operational simplicity, stability, and horizontally scaled Clos networks. RFC Editor

A simplified routed fabric might use:

Leaf ASN 65001
    |
   BGP
    |
Spine ASN 65100
    |
   BGP
    |
Leaf ASN 65002

Actual BGP designs vary.

The important architectural point is that dynamic routing allows the fabric to learn paths and react when links or devices fail.


Underlay Routing Protocol Choices

Not every leaf-spine network uses BGP.

Possible underlay routing choices can include:

  • eBGP
  • OSPF
  • IS-IS

The correct choice depends on factors such as:

  • Network scale
  • Operational expertise
  • Vendor architecture
  • Failure convergence requirements
  • Automation
  • Troubleshooting model

Large hyperscale architectures have demonstrated BGP successfully, but that does not mean every enterprise data center should automatically replace its existing routing protocol with BGP.


Border Leaf Architecture

Internal leaves connect servers and workloads.

External connectivity can be provided through border leaf switches.

                    WAN / Internet
                          |
                          v
                     Border Leaf
                          |
                +---------+---------+
                |                   |
             Spine 1             Spine 2
                |                   |
              Leaf                 Leaf

A border leaf may connect the fabric to:

  • Internet edge
  • WAN
  • Firewalls
  • Data center interconnect
  • External routers

Cisco’s VXLAN EVPN architecture guidance describes border leafs as points for external connectivity while spine switches continue to provide fabric transit. Cisco


North-South Traffic Through a Leaf-Spine Fabric

An external request may follow:

Internet
   |
   v
Firewall
   |
   v
Border Leaf
   |
   v
Spine
   |
   v
Leaf
   |
   v
Application Server

The exact path depends on where external services such as firewalls and load balancers are attached.


East-West Traffic Through the Fabric

Server-to-server communication might look like:

Server A
   |
   v
Leaf A
   |
   v
Spine
   |
   v
Leaf B
   |
   v
Server B

A properly designed fabric should provide multiple available paths between the leaves.

Modern Cisco leaf-spine guidance specifically targets these high-volume east-west traffic patterns. Cisco


Load Balancing and Data Center Networking

Application load balancing is separate from network ECMP.

ECMP distributes network flows among equal-cost network paths.

Application load balancing distributes requests or connections across application targets.

Users
  |
  v
Load Balancer
  |
  +--------+--------+
  |        |        |
  v        v        v
Server1 Server2 Server3

Both can exist simultaneously.

For example:

Application Load Balancer
          |
          v
      Leaf-Spine
          |
     ECMP Paths
          |
          v
 Application Servers

See Load Balancing Explained for the application and transport-layer side of this architecture.


Data Center Network Redundancy

A production network should avoid unnecessary single points of failure.

Weak architecture:

All Servers
    |
One Switch
    |
One Router

A stronger design might provide:

        Spine A       Spine B
          |             |
          +------+------+
                 |
           Leaf Pair
           /      \
        Server   Server

But redundant hardware alone is not sufficient.

Failure-domain analysis should ask:

Do both paths depend on the same upstream device?

Are both switches using the same power source?

Does one configuration error affect both?

Can the surviving path handle the full required traffic?

Redundancy should be validated end to end.


Dual-Homed Servers

Critical systems can connect through multiple network interfaces.

                 Server
                /      \
               /        \
            NIC A      NIC B
              |          |
            Leaf A      Leaf B

This can protect against:

  • NIC failure
  • Cable failure
  • Leaf failure

The host and network design must still support the required bonding, teaming, multihoming, or routing behavior.

VXLAN EVPN standards also include mechanisms intended to support multihomed network virtualization edges. RFC Editor


Data Center Network Security

High east-west traffic means security cannot exist only at the Internet perimeter.

Traditional thinking:

Internet
   |
Firewall
   |
Trusted Data Center

is often insufficient.

A compromised internal workload could attempt to communicate laterally with:

  • Application servers
  • Databases
  • Management networks
  • Storage systems
  • Other tenants

A stronger model considers both north-south and east-west controls.

North-South Security
        +
East-West Segmentation

For broader principles, see Network Security Basics and Firewalls Explained.


Network Segmentation

Segmentation limits unnecessary communication between workloads.

For example:

Web Tier
   |
Allowed
   |
   v
Application Tier
   |
Allowed
   |
   v
Database Tier

but perhaps:

Internet
   |
   X
Database Tier

A segmentation strategy can use combinations of:

  • VLANs
  • VRFs
  • ACLs
  • Firewalls
  • Security groups
  • EVPN/VXLAN segmentation
  • Kubernetes network policies

The specific technology matters less than ensuring that required communication is permitted while unnecessary access is restricted.


Data Center Network Monitoring

A fabric can appear operational while performance degrades.

Useful metrics include:

Interface utilization
Packet drops
Errors
Latency
Link state
Routing adjacency state
ECMP path health
Buffer utilization
BGP session state
MAC table utilization
ARP / ND state
VXLAN tunnel status

Monitoring should also track changes.

For example:

Normal:
4 ECMP Paths

Failure:
3 ECMP Paths

Traffic may still work, but redundancy has degraded.

For protocol-level monitoring concepts, CorgiNetwork also has SNMP Explained.


Network Telemetry and Observability

Traditional SNMP remains valuable, but modern data-center environments may also use:

  • Streaming telemetry
  • Syslog
  • Flow records
  • API-based monitoring
  • Routing telemetry
  • Distributed tracing at the application layer

These data sources answer different questions.

Interface telemetry may tell you:

Link utilization = 95%

while application tracing may reveal:

Database request latency = 400 ms

Network monitoring and application observability should therefore complement one another.


Troubleshooting East-West Traffic

Suppose Server A cannot reach Server B.

Use a layered workflow.

Server A
   |
NIC
   |
Leaf A
   |
Spine
   |
Leaf B
   |
NIC
   |
Server B

Check:

  1. Does Server A have correct addressing?
  2. Is its interface operational?
  3. Is the leaf port up?
  4. Is the destination reachable through routing?
  5. Are ECMP paths available?
  6. Are ACL or firewall policies blocking traffic?
  7. Is VXLAN/EVPN state correct where used?
  8. Is the destination server actually listening?

Avoid assuming immediately that the spine is responsible.

The complete path should be examined.


Troubleshooting North-South Traffic

If internal server-to-server communication works but external users cannot connect, focus on the external path:

Internet
   |
Border Router
   |
Firewall
   |
Border Leaf
   |
Load Balancer
   |
Application

Possible causes include:

  • External routing
  • Firewall policy
  • NAT
  • DNS
  • Load balancer health
  • Border-leaf routing
  • Return-path problems

A routing issue can also create asymmetric behavior where traffic reaches the application but the response follows the wrong path.


Network Failure Scenario: One Leaf Fails

Suppose a server is single-homed:

Server
   |
Leaf A

If Leaf A fails:

Leaf A X

the server becomes isolated.

Compare this with a properly designed dual-homed system:

        Server
       /      \
   Leaf A    Leaf B

If one leaf fails, the remaining path may continue carrying traffic.

This demonstrates why switch redundancy and endpoint attachment strategy must be considered together.


Network Failure Scenario: One Spine Fails

Normal:

Leaf A
 |   |
 |   |
Sp1 Sp2
 |   |
 |   |
Leaf B

Failure:

Leaf A
     |
     |
    Sp2
     |
     |
Leaf B

Connectivity may continue, but available aggregate bandwidth is lower.

Therefore resilient designs must consider:

Connectivity after failure
            +
Capacity after failure

not merely whether packets still pass.


Network Failure Scenario: Link Failure

If each leaf has connections to multiple spines:

Leaf
 |
 +---- Link 1 ---- Spine 1
 |
 +---- Link 2 ---- Spine 2

losing one link does not necessarily isolate the leaf.

Routing should remove the failed path from forwarding and continue using remaining paths.

This is one of the operational advantages of routed fabrics with dynamic routing and ECMP.


Network Capacity Planning

Capacity planning should begin with workload traffic patterns.

Suppose a rack contains 40 servers.

Each has:

25 Gbps

Theoretical server-facing capacity is:

40 × 25
=
1,000 Gbps

But the servers may never all transmit at line rate simultaneously.

Therefore the design must consider:

  • Average utilization
  • Peak utilization
  • East-west traffic
  • Storage traffic
  • Backup traffic
  • Replication
  • Failure conditions
  • Future growth

The objective is not to maximize every link automatically.

It is to provide sufficient capacity for actual application requirements and failure scenarios.


AI and High-Performance Data Center Networks

AI and distributed compute workloads can create particularly demanding east-west traffic.

A distributed workload may require:

Compute Node A
      ↔
Compute Node B
      ↔
Compute Node C
      ↔
Compute Node D

These applications can require high bandwidth and low latency among compute nodes.

Cisco’s data-center networking designs for AI/ML workloads use leaf-spine connectivity with high-speed links between leaf and spine switches. Cisco

This is another example of why traffic characteristics should drive architecture.


Storage Traffic in the Data Center

Storage can also generate substantial east-west bandwidth.

For example:

Application Server
       |
       v
Storage Network
       |
       v
Storage Cluster

Traffic may include:

  • Database I/O
  • VM storage
  • Replication
  • Backups
  • Distributed storage synchronization

The storage architecture should therefore be planned together with the network.

See Storage Architecture Explained for block, file, object, SAN, NAS, replication, and storage performance.


Traditional Three-Tier vs Leaf-Spine

Feature Traditional Three-Tier Leaf-Spine
Main layers Core/Aggregation/Access Spine/Leaf
Typical historical focus North-south traffic High east-west demand
Path consistency Can vary More uniform
Multipath utilization Architecture-dependent Commonly ECMP
Horizontal expansion More hierarchical Add leaf capacity
Routing Often higher in hierarchy Often pushed toward leaf
Modern overlay use Possible Very common
Large distributed workloads Can work Often better suited

Neither architecture is automatically correct for every environment.

The decision should follow workload and operational requirements.


Data Center Network Architecture Example

A mature fabric could look like:

                         Internet / WAN
                              |
                              v
                        Border Routers
                              |
                              v
                           Firewalls
                              |
                              v
                         Border Leafs
                          /        \
                         /          \
                    Spine 1        Spine 2
                  /   |   \        /   |   \
                 /    |    \      /    |    \
              Leaf1 Leaf2 Leaf3 Leaf4 Leaf5 Leaf6
                |     |     |     |     |     |
                v     v     v     v     v     v
             Compute Storage App DB  Containers Services

The physical fabric may operate as a Layer 3 underlay.

VXLAN can provide the overlay.

EVPN can distribute overlay reachability.

ECMP can distribute traffic across equal-cost underlay paths.

Border leaves connect external networks.

Security policies control both north-south and east-west communication.

Monitoring observes the complete fabric.


A Practical Data Center Network Design Process

Begin with workloads, not switch models.

First determine:

How many servers?

Then:

How much bandwidth per server?

Then:

What traffic is east-west?
What traffic is north-south?

Then evaluate:

Rack count
Port density
Uplink capacity
Oversubscription
Failure tolerance
Routing requirements
Segmentation
External connectivity
Growth
Operations

Only after these requirements are understood should specific hardware and protocols be selected.


Data Center Network Architecture Best Practices

A strong design should keep the physical topology predictable, avoid unnecessary Layer 2 extension, use multiple active paths where appropriate, understand oversubscription instead of assuming the fabric is non-blocking, separate failure domains, size surviving paths for expected failures, monitor routing and link health, segment sensitive workloads, automate repetitive configuration, document external dependencies, and test actual failure scenarios.

It is also important to resist adding complexity without a clear requirement. VXLAN, EVPN, BGP, microsegmentation, automation platforms, and sophisticated overlays can solve real scaling problems, but a smaller data center should not deploy them simply because large hyperscale networks do.

Architecture should solve the environment’s actual problem.


Frequently Asked Questions

What is data center network architecture?

Data center network architecture defines how servers, storage, switches, routers, security appliances, and external networks are connected and how traffic moves between them.

What is leaf-spine architecture?

Leaf-spine is a two-tier Clos-style architecture in which leaf switches connect workloads and each leaf normally connects to every spine switch.

What does a spine switch do?

A spine switch provides high-speed transit between leaf switches.

What does a leaf switch do?

A leaf switch connects servers and other endpoints to the data center fabric and may also provide routing, gateway, or VXLAN functions.

What is east-west traffic?

East-west traffic moves between systems inside the data center, such as application servers communicating with databases or storage.

What is north-south traffic?

North-south traffic enters or leaves the data center, such as users accessing an application from the Internet.

What is ECMP?

ECMP means Equal-Cost Multi-Path. It allows multiple routes with the same routing cost to be used simultaneously.

Why is leaf-spine good for east-west traffic?

Leaf-spine provides multiple short, predictable paths between leaf switches and commonly uses ECMP to use those paths concurrently.

Is leaf-spine always non-blocking?

No.

The fabric can still be oversubscribed if server-facing bandwidth exceeds available uplink or spine capacity.

What is a ToR switch?

A Top-of-Rack switch is located in or near a server rack and connects the rack’s devices to the wider data center network.

What is VXLAN?

VXLAN is an overlay technology that carries virtual Layer 2 networks across a Layer 3 IP infrastructure. RFC 7348 defines the deployed VXLAN framework. RFC Editor

What is EVPN?

EVPN is a BGP-based control-plane technology for distributing Ethernet and network-virtualization reachability information. RFC 8365 defines the use of EVPN with network virtualization overlays such as VXLAN. RFC Editor

What is VXLAN BGP EVPN?

It combines VXLAN data-plane encapsulation with an EVPN/BGP control plane to build scalable overlay networks across a routed IP fabric.

Does a leaf-spine network require BGP?

No.

Different implementations may use BGP, OSPF, IS-IS, or platform-specific mechanisms. RFC 7938 documents BGP as a proven option for large-scale data centers. RFC Editor

What is a border leaf?

A border leaf provides connectivity between the internal data center fabric and external networks such as WANs, Internet edges, firewalls, or data center interconnects.

What happens if a spine fails?

If multiple spines and valid routing paths exist, traffic can continue through surviving spines, although total network capacity may decrease.

What is data center network oversubscription?

Oversubscription occurs when the potential aggregate bandwidth of downstream devices exceeds available upstream fabric capacity.


Conclusion

Data center network architecture has evolved because application traffic has changed.

A traditional architecture often followed:

Core
  |
Aggregation
  |
Access
  |
Servers

Modern distributed workloads frequently benefit from:

          Spine
        /   |   \
       /    |    \
    Leaf   Leaf   Leaf
      |      |      |
   Servers Storage Applications

The architecture separates several concerns.

The leaf-spine topology creates a predictable fabric.

ECMP allows multiple network paths to be active.

Layer 3 routing provides scalable underlay connectivity.

VXLAN can create logical overlays over that routed infrastructure. RFC 7348 defines VXLAN as a Layer 2 overlay over Layer 3. RFC Editor

EVPN can provide a scalable control plane for those overlays. RFC Editor

BGP can provide underlay routing at very large scale, as documented in RFC 7938. RFC Editor

And the final design must still address:

Redundancy
+
Bandwidth
+
Security
+
Monitoring
+
Failure Domains
+
Operational Simplicity

The most important lesson is that there is no universally correct data center topology.

A good design starts with traffic patterns and workload requirements, then selects the topology, routing, segmentation, and overlay technologies that satisfy them.

Continue through the CorgiNetwork infrastructure cluster with Data Center Architecture Explained, Routers, Switches and Network Devices Explained, VLAN Explained, Load Balancing Explained, Storage Architecture Explained, and Cloud Networking Explained.

Leave a Reply

Your email address will not be published. Required fields are marked *