HPC Performance Metrics: Should We Drop FLOPS ?


Panel Theme:

As computer architectures have advanced, making floating-point operations relatively inexpensive, it is still very ingrained in HPC to compare systems by their FLOPS rates, and to optimize algorithms to minimize their floating-point operation count even at the cost of increased memory bandwidth demands or programming effort. Should we drop FLOPS as a metric? If the answer is yes, then is there a way to gracefully (or perhaps suddenly) move to a metric other than FLOPS that has the predictive value and ease-of-analysis that FLOPS had in the 1960s and 1970s?

The panelists will give their position on this question and their suggestions on the general issue of using simplistic measures like FLOPS as a performance metric.

 

Panel Moderator :

John Gustafson

Chief Technology Officer, HPC

Bio:John joined ClearSpeed in 2005 after leading high-performance computing efforts at Sun Microsystems. He has 33 years experience using and designing compute-intensive systems, including the first matrix algebra accelerator and the first commercial massively-parallel cluster while at Floating Point Systems. His pioneering work on a 1024-processor nCUBE at Sandia National Laboratories created a watershed in parallel computing, for which he received the inaugural Gordon Bell Award. He also has received three R&D 100 Awards for innovative performance models, including the model commonly known as Gustafson's Law or Scaled Speedup.

He received his B.S. degree from Caltech and his M.S. and Ph.D. degrees from Iowa State University, all in Applied Mathematics.


Panelists ( Alphabetical Order ) :

David.H. Bailey

Chief Technologist
Computational Research Dept.
Lawrence Berkeley National Laboratory

Bio:David H. Bailey received his B.S. in mathematics from Brigham Young University, and his Ph.D. from Stanford University. For 15 years he worked at the NAS supercomputer center at NASA's Ames Research Center, and for the last 10 years at the Lawrence Berkeley Laboratory, where he is currently the Chief Technologist of the Computational Research Dept. Dr. Bailey has received the Sidney Fernbach Award from the IEEE Computer Society, and the Chauvenet and Hasse awards from the Mathematical Association of America. He has authored three books and over 100 papers in the area of computer science and mathematics.

David.Koester

The MITRE Corporation

Bio David Koester received his Masters in Applied Statistics (MAS) from The Ohio State University in 1978 and his doctorate from Syracuse University in 1996 under the guidance of Dr. Geoffrey Fox and Dr. Sanjay Ranka. He joined the MITRE Corporation at the MITRE-Rome site in 1978 and continues to work from that office which is co-located with the Air Force Research Laboratory (AFRL) Site Rome, NY. Dr. Koester's present areas of interest in High End Computing (HEC) technologies include understanding the high-level mappings of applications to computing architectures and metrics to evaluate system performance and productivity. Dr. Koester is currently working with DARPA/IPTO to examine the future ultrascale HEC applications that will drive technologies in the 2015-2020 time frame. Dr. Koester was a member of the DARPA High Productivity Computing Systems (HPCS) Productivity Team and led the Benchmarks Working Group. On HPCS, Dr. Koester developed the RandomAccess benchmark for the HPC Challenge benchmark suite.

Karen L. Karavanic

Computer Science Dept.
Portland State University

Bio: Karen L. Karavanic is an Associate Professor of Computer Science at Portland State University. Her research interests include operating systems, performance measurement, and performance tools for high end computing. Professor Karavanic received her M.S. and Ph.D. in Computer Science from the University of Wisconsin - Madison, where she was a member of the Paradyn Parallel Performance Tools research group.

Jeffrey Vetter

Computer Scientist
Oak Ridge National Laboratory

Bio: Jeffrey Vetter is a computer scientist in the Computer Science and Mathematics Division (CSM) of Oak Ridge National Laboratory (ORNL), where he leads the Future Technologies Group and directs the Experimental Computing Laboratory. Dr. Vetter is also a Joint Professor in the College of Computing at the Georgia Institute of Technology, where he earlier earned his PhD. He joined ORNL in 2003, after four years at Lawrence Livermore National Laboratory. Vetter's interests span several areas of high-end computing (HEC) -- encompassing architectures, system software, and tools for performance and correctness analysis of applications.