800G and 1.6T for Hyperscale AI:
Latency, Density, and Design Constraints
How all-to-all GPU communication patterns, sub-microsecond latency targets, and extreme power density requirements are reshaping optical interconnect design in hyperscale AI infrastructure.
Introduction
The shift from conventional cloud computing to large-scale AI training has altered almost every dimension of data centre network design. Where traditional workloads generated predictable, north-south traffic flows between users and servers, AI training clusters create a fundamentally different pattern: dense, synchronised, east-west communication between thousands of GPU accelerators that must collectively function as a single logical compute engine. This traffic pattern — often called all-to-all communication — demands bandwidth, latency, and reliability characteristics that push optical interconnect technology to its limits.
The response from the industry has been a rapid transition through optical port speeds. The 400G generation, which was considered advanced three years ago, is now the baseline access tier for AI data centres. The 800G generation — based on eight lanes of 100 Gb/s each — has become the standard for AI fabric buildouts in 2025, with deployments doubling year-over-year. The 1.6T generation, leveraging 200 Gb/s per-lane technology, is entering early volume production and is already specified in hyperscale procurement contracts for the next infrastructure cycle.
This article covers the engineering rationale behind these transitions — the traffic patterns that demand them, the optical and electrical technologies that enable them, the latency constraints that shape them, and the power and density challenges that complicate them. It examines where pluggable optics remain the practical choice, where Linear Pluggable Optics (LPO) and its asymmetric variant Linear Receive Optics (LRO) offer compelling improvements, and where Co-Packaged Optics (CPO) become the preferred path. It also covers emerging fibre technologies — hollow-core and multi-core fibres — and optical circuit switching, all of which are shaping the next phase of AI cluster design.
This reference covers intra-data-centre optical interconnects for AI training and inference clusters, including the switch fabric, top-of-rack, and server attachment layers. Data centre interconnect coherent optics, metro transport, and submarine systems are outside scope. Forward-looking sections on emerging fibre and switching technologies are clearly labelled as such.
2. AI Traffic Patterns and Why They Demand New Interconnect Designs
2.1 The All-to-All Communication Pattern
Training a large-scale AI model requires distributing computation across hundreds or thousands of GPU accelerators simultaneously. The work is split in several ways — across training data (data parallelism), across model layers (pipeline parallelism), and across the weights within a layer (tensor parallelism). Each parallelism strategy produces a specific communication pattern, and in combination they create demand for high-bandwidth, low-latency collective communication operations such as AllReduce, AllGather, and ReduceScatter.
An AllReduce operation requires every accelerator in a training group to share its locally computed gradient updates with all other accelerators in the group, aggregate the results, and distribute the aggregated result back to every accelerator — all before the next training step begins. In a cluster of thousands of GPUs, this generates a simultaneous, bidirectional, many-to-many traffic burst that is wholly unlike web traffic, database queries, or file I/O. It is a synchronised collective operation, meaning the fastest accelerator can only advance as fast as the slowest interconnect allows.
2.2 The Scale-Up and Scale-Out Network Layers
Modern AI cluster networks are organised into two physically distinct layers. The scale-up network connects GPU accelerators within a single server or a tightly coupled node group at the highest possible bandwidth and lowest latency. This layer uses specialised chip-to-chip interconnects, PCIe, or CXL switches, and is internal to the compute node — transparent to the Ethernet fabric.
The scale-out network (also called the backend AI fabric) connects compute nodes across the cluster. This is where 800G and 1.6T optical transceivers are deployed. It must carry collective communication traffic with near-zero congestion, provide a non-blocking topology, and deliver end-to-end latency that keeps collective operations within the compute step window.
2.3 Rail Architecture for AllReduce Efficiency
A rail architecture groups GPU servers so that each GPU within a server connects to a different spine switch through dedicated top-of-rack (ToR) switches on separate rails. In a single-rail design, all GPU ports from a server route through one ToR. In a multi-rail design, GPU ports are distributed across multiple ToR switches, each connected to different spine switches.
Multi-rail architectures reduce AllReduce congestion because traffic from different GPUs within the same server exits through different uplinks, reducing the probability that synchronisation traffic from multiple GPUs contends for the same spine links simultaneously. The optical design implication is that each rack requires multiple ToR switches with 800G uplinks, increasing total fibre count and switch port count per rack — a direct planning input for fabric sizing and cabling infrastructure.
2.4 Why East-West Traffic Dominates
In AI training clusters, the dominant pattern is server-to-server (east-west) gradient synchronisation traffic. In a cluster with thousands of GPU accelerators, a single training step may require each node to transmit its full gradient update to every other node simultaneously — meaning aggregate east-west bandwidth scales with the square of the number of nodes, while north-south traffic scales only linearly. This structural difference is why AI data centres invest in switch silicon with dramatically higher port counts, and why the ratio of optical transceivers to GPU accelerators has increased substantially compared to traditional cloud deployments.
3. Foundational Concepts: 800G and 1.6T Optical Technology
3.1 Lane Architecture and Modulation
The progression from 400G to 800G to 1.6T in intra-data-centre optics follows a consistent pattern: the industry increases speed either by increasing the number of lanes at a fixed lane rate, or by increasing the per-lane data rate while keeping the lane count fixed, or both. For short-reach intra-data-centre applications, direct-detection with four-level Pulse Amplitude Modulation (PAM4) is the dominant modulation scheme. PAM4 encodes 2 bits per symbol, doubling the data rate compared to non-return-to-zero (NRZ) signalling at the same baud rate — at the cost of reduced noise margin that makes Forward Error Correction (FEC) mandatory.
Read the Full Analysis with Premium
The remaining 80% of this article — the design numbers, trade-offs and field guidance — is part of MapYourTech Premium, along with the full premium library, courses and professional tools.
Optical Communications & Network Automation Expert | Author of 3 Books for Optical Engineers | Founder, MapYourTech
Optical networking engineer with nearly two decades of experience across DWDM, OTN, coherent optics, submarine systems, and cloud infrastructure. Founder of MapYourTech. Read full bio →
Follow on LinkedInRelated Articles on MapYourTech
Share:
You May Also Like
-
Free
-
July 10, 2026
-
Free
-
July 10, 2026
-
Free
-
July 10, 2026