Sub-pJ/bit Optical Connectivity for AI Clusters

High-performance AI clusters are increasingly bottlenecked by the energy and communications costs of interconnecting numerous compute and memory resources. Current systems face a gap of nearly two orders of magnitude between on-chip, intra-socket, communication capacities, and the capacities of links transporting data over longer distances. The per bit energy cost of data movement dominates that of data processing, as does density, throughput, and latency. Integrated silicon photonics offer the opportunity of optical connectivity that delivers high off-chip communication bandwidth densities with low power consumption. To realize these benefits the co-integration of photonics with the compute and memory is critical. This talk will cover approaches for leveraging photonic IO that can scale to realize Petabit/s chip escape bandwidths with sub-picojoule/bit energy consumption, as well as new architectural approaches that enable flexible connectivity tailored to accelerate distributed AI/ML applications.