Advancing reproducibility and replicability in simulation: Challenges and opportunities.

Abstract

Computer simulation has become increasingly complex and widely applied across different domains. However, the reproducibility and replicability (R&R) of simulation models remain limited. Despite recent improvements, independent reproduction or replication of simulation experiments is rare in the literature. This paper provides an overview of the state of research on R&R in simulation, highlights recent developments, and discusses key concepts and future perspectives. It first examines how R&R has been viewed, approached, and evaluated and then outlines typical challenges and defining characteristics of R&R. Emerging opportunities are also discussed in light of community-driven practices, artificial intelligence, and quantum computing. Given the significant role of simulation in modern science, this paper argues that R&R studies of simulation are valuable research outputs and should be regarded as an integral and equally important part of scientific progress. R&R should be explicitly addressed and embedded into modelling and simulation practices, and supported by stronger community efforts. Researchers engaging in these efforts face substantial challenges, including those related to recognition and rewards, methodology, and scalability, many of which are under-researched.