SC15 Invited Talk Spotlight: Reproducibility in High Performance Computing

The number of lines of code published in ACM Transactions on Mathematical Software, 1960–2012, on a log scale. The proportion of articles that published code remained roughly constant at about a third, with standard error of about 0.12, and the journal consistently published around thirty-five articles each year. Source: click here and click here.

Ensuring reliability and reproducibility in computational research raises unique challenges in the

supercomputing context. Specialized architectures, extensive and customized software, and complex workflows all raise barriers to transparency, while established concepts such as Validation, Verification, and Uncertainty Quantification point ways forward.

Invited speaker Victoria Stodden co-edited this book.

The topic has attracted national attention: President Obama's July 29, 2015 Executive Order "Creating a National Strategic Computing Initiative" includes accessibility and workflow capture as objectives; an XSEDE14 workshop released a report "Standing Together for Reproducibility in Large-Scale Computing"; on May 5, 2015 ACM Transactions in Mathematical Software released a "Replicated Computational Results Initiative"; and this conference is host to a new workshop "Numerical Reproducibility at Exascale", to name but a few examples. In this context I will outline a research agenda to establish reproducibility and reliability as a cornerstone of scientific computing.



Speaker Background:

Dr. Victoria Stodden

Dr. Victoria Stodden is an associate professor in the School of Information Sciences at the University of Illinois at Urbana-Champaign, with a faculty appointment at the National Center for Supercomputing Applications. She completed both her PhD in statistics and her law degree at Stanford University.

Her research centers on the multifaceted problem of enabling reproducibility in computational science. This includes studying adequacy and robustness in replicated results, designing and implementing validation systems, developing standards of openness for data and code sharing, and resolving legal and policy barriers to disseminating reproducible research.

She is the co-chair the Advisory Committee for the National Science Foundation's Division of Advanced CyberInfrastructure, and is a member of the NSF CISE directorate's Advisory Committee.