IRON: Reducing Workload Interference on Massively Parallel Platforms

Interconnect networks with Dragonfly and Fat tree configurations are dominant in high-performance computing facilities and data centers. A key challenge of managing these shared networks is workload interference. In a multi-user computing environment, interference among applications for shared network resources can cause a vicious cycle of events (workload interference, low productivity, selfish user behavior, and poor scheduling) aggravating each other. This project aims to address this fundamental problem on massively parallel systems by developing the IRON (Interference ReductiON) framework. The project consists of three research thrusts: (1) flit-level network modeling and simulation to gain insights into workload interference and further to explore various what-if questions in terms of workload interference, (2) interference-aware scheduling to mitigate network congestion/contention among applications, and (3) experiments to quantitatively characterize workload interference and to assess the interference-aware scheduling design.

The project has three key outcomes. First is the experimental study to quantitatively measure workload interference of representative applications on production systems. The second is the development of high-fidelity modeling tool to simulate and study complex network interference on high-performance computing systems. The third is the design of interference aware methods, including routing and scheduling, to reduce network contention among applications.  The project delivers several open-source software artifacts for interference analysis and reduction on large-scale interconnect networks. The integrated education and outreach plan has enhanced the Computer Science curriculum and broadened the participation by underrepresented groups.