Title: Balanced Sparse Tree: A Scalable Network Topology for Large Language Models
Authors: Shaoteng Liu, Huitian Wang, Xiangyu Chen, Yi Zhang, Xiaotian Zhou, Peng Dong, Rui Meng, Fuguang Huang, Xia Zhu, and Bingyang Liu (Huawei); Dejun Kong (Shanghai Jiao Tong University and University of New South Wales); Hongji Dong, Xiaofeng Gao, and Guihai Chen (Shanghai Jiao Tong University)
Introduction
Large-scale LLM training and inference increasingly require networks that connect thousands of accelerators with low hop count, high collective-communication throughput, and acceptable switch cost. Clos networks provide predictable bandwidth but become expensive and add layers as clusters scale, while direct or sparse designs such as Dragonfly+ and Zettafly either introduce asymmetric paths that interact poorly with All-to-All and All-Reduce or remain far below the degree-diameter upper bound imposed by finite switch radix. Because contemporary radix is physically limited, the central problem is not simply adding ports but using them in a topology that balances scalability, symmetry, latency, and cost for LLM communication patterns.
Key idea and contribution:
Balanced Sparse Tree (BST) is a three-layer, switch-centric topology consisting of endpoint clusters, grouped lower-layer switches, and sparsely connected upper-layer switches. Its construction uses hypergraph theory and Steiner systems so that pairwise connectivity remains balanced and symmetric. A tunable sparsity factor allocates switch radix to horizontal expansion while retaining a fixed maximum of four switch hops between endpoint clusters. The resulting bipartite biregular construction approaches the relevant Moore-bound upper limit and, for the configurations studied, supports 1.7x the scale of Zettafly at the same radix.
The topology is complemented by differentiated routing (DiffR), which normally uses minimal paths but permits selected detours when they expose otherwise idle bandwidth; the design also includes a deadlock-avoidance mechanism for the lossless fabric. Topology-affined rank and expert placement maps collective algorithms and MoE workloads onto BST’s balanced domains. The authors provide mathematical scale and throughput models, packet-level simulation, fault analysis, and a 24-NPU Ascend testbed that can switch logically between BST and Clos using port isolation.
Evaluation
The paper combines analysis, simulations at 320 and 2,560 nodes, fault injection, and experiments on four Ascend 910B servers using 24 NPUs and 200/400-Gbps links. At equivalent scale, BST is reported to reduce network cost by 50% relative to Clos while maintaining comparable AI-workload performance; simulations show 3.9-11.8% gains for collective communication and a 13.4% improvement over other scalable topologies on a profiled DeepSeek-V3 MoE workload. On the testbed, BST closely matches Clos for Pairwise All-to-All and NHR All-Reduce, improves Ring All-Reduce completion time by about 10%, and runs Qwen3-30B-A3B inference within roughly 0.2-0.6% of Clos across service and per-request metrics while using lower-radix spine switches. This result is significant because it suggests that an LLM fabric can substantially reduce switching cost and increase supported scale without paying the large collective-communication penalty usually associated with sparse topologies.
Q1: When comparing different topologies, such as Clos and BST, you discussed performance benefits in terms of bandwidth and ease of cabling. Could you provide some intuition about fault tolerance? The evaluation seems to indicate that BST is also resilient to link failures. Is this resilience a consequence of the structure of the topology?
A1: Some of my colleagues have asked a similar question. BST mainly relies on DiffR, our differentiated routing mechanism, to utilize alternative paths when the minimal paths are affected by failures or congestion.
In our evaluation, we considered relatively low link-failure rates, up to approximately 4% or 5%. A single link failure can usually be handled by rerouting traffic through alternative paths. Simultaneous failures of many links are much less likely, although they can cause greater performance degradation. For the failure scenarios evaluated in the paper, we found that DiffR was sufficient to maintain reasonable performance.
Q2: How would you handle cabling in a data center? More specifically, given that BST may appear to be an irregular topology, have you considered how to simplify its physical cabling?
A2: Although BST is sparse, it has a symmetric and structured topology. Its switches and links are organized into regular groups, which makes the connection pattern predictable. The network can also be expanded systematically by scaling the spine layer or adding additional ToR groups. Therefore, the topology’s symmetry and grouping structure can help simplify cabling and incremental expansion.
Q3: Are there multiple BST configurations that can be evaluated, or did you test only one specific configuration?
A3: There are many possible BST configurations. The paper provides a large table listing different feasible BST designs for switch radices ranging from 128 to 256. These configurations are constructed using different Steiner systems from combinatorial design theory. Therefore, BST is a family of topologies rather than a single fixed topology.
Q4: Did you study the trade-off between the deployment cost and performance of these different BST configurations?
A4: In the current comparison, we mainly evaluated a relatively small-scale BST configuration and compared it with Clos and other baseline topologies. We also analyzed topology cost and scalability for a broader range of configurations. However, a more comprehensive comparison between large-scale BST configurations and equivalent Clos deployments remains future work.
Personal thoughts
The paper’s most appealing feature is its end-to-end treatment of topology design. It connects a combinatorial construction to routing, deadlock freedom, workload placement, cost modeling, large simulations, and a real accelerator testbed. The explicit adversarial and failure experiments are especially useful because they reveal where sparsity hurts rather than presenting only favorable collective workloads. The near-Clos Qwen inference result at lower spine radix makes the economic motivation concrete.