Wake Up and Smell the Coffee: Evaluation Methodology for the 21st Century
Stephen M. Blackburn,
Kathryn S. McKinley,
Robin Garner,
Chris Hoffmann,
Asjad M. Khan,
Rotem Bentzur,
Amer Diwan,
Daniel Feinberg,
Daniel Frampton,
Samuel Z. Guyer,
Martin Hirzel,
Antony Hosking,
Maria Jump,
Han Lee,
J. Eliot B. Moss,
Aashish Phansalkar,
Darko Stefanovic,
Thomas VanDrunen,
Daniel Von Dincklage,
Benjamin Wiedermann
August, 2008
Abstract
Evaluation methodology underpins all innovation in experimental computer science. It requires relevant workloads, appropriate experimental design, and rigorous analysis. Unfortunately, methodology is not keeping pace with the changes in our field.
Publication
Communications of the ACM (Research Highlights), 51 (8), 83-89