AuthorsF. Zahid, E. G. Gran, B. Bogdanski, B. D. Johnsen and T. Skeie
TitleA weighted fat-tree routing algorithm for efficient load-balancing in InfiniBand enterprise clusters
AfilliationCommunication Systems, Communication Systems
Publication TypeProceedings, refereed
Year of Publication2015
Conference NameProceedings of the 23rd Euromicro International Conference on Parallel, Distributed Network-based Processing (PDP 2015)
Date Published03/2015
Place PublishedTurku, Finland
ISSN Number1066-6192
Accession Number15090056
Keywordsfat-tree networks, InfiniBand, Load-balancing, Routing algorithms

InfiniBand (IB) has become a popular network interconnect for high-performance computing (HPC) systems. Many of the large IB-based HPC systems use some variant of the fat-tree topology to take advantage of the useful properties fat-trees offer. The fat-tree routing algorithm is one of the most efficient deterministic routing algorithms for fat-tree topologies. The algorithm ensures that the number of routes assigned to each link are balanced across the fabric. However, one problem with its load-balancing technique is that it assumes uniform traffic distribution in the network. When routes towards nodes that mainly consume large amount of data are assigned to share links in the fabric while alternative links are underutilized, sub-optimal network throughput is obtained. Also, as the fat-tree algorithm routes nodes according to the indexing order, the performance may differ for two systems cabled in the exact same way.

In this paper, we propose wFatTree, a novel fat-tree routing algorithm, which considers node traffic characteristics to balance load across the network links more evenly, and with predictable network performance. Our experiments and simulations show an improvement of up to 60% in total network throughput on large fat-tree installations when using wFatTree routing. Furthermore, wFatTree can also be used to prioritize traffic flowing towards the critical nodes in the network.

Citation Key19102