AboutDownloadsDocumentsForumsWikiSource CodeNews
Primary Publication
Erdemir, A, Mulugeta, L, Ku, JP, Drach, A, Horner, M, Morrison, TM, Peng, GCY, Vadigepalli, R, Lytton, WW, Myers, JG, "Credible Practice of Modeling and Simulation in Healthcare: Ten Rules from a Multidisciplinary Perspective," Journal of Translational Medicine, 18:369, 2020. (2020)  View
Abstract

The complexities of modern biomedicine are rapidly increasing. Thus, modeling and simulation have become increasingly important as a strategy to understand and predict the trajectory of pathophysiology, disease genesis, and disease spread in support of clinical and policy decisions. In such cases, inappropriate or ill-placed trust in the model and simulation outcomes may result in negative outcomes, and hence illustrate the need to formalize the execution and communication of modeling and simulation practices. Although verification and validation have been generally accepted as significant components of a model’s credibility, they cannot be assumed to equate to a holistic credible practice, which includes activities that can impact comprehension and in-depth examination inherent in the development and reuse of the models. For the past several years, the Committee on Credible Practice of Modeling and Simulation in Healthcare, an interdisciplinary group seeded from a U.S. interagency initiative, has worked to codify best practices. Here, we provide Ten Rules for credible practice of modeling and simulation in healthcare developed from a comparative analysis by the Committee’s multidisciplinary membership, followed by a large stakeholder community survey. These rules establish a unified conceptual framework for modeling and simulation design, implementation, evaluation, dissemination and usage across the modeling and simulation life-cycle. While biomedical science and clinical care domains have somewhat different requirements and expectations for credible practice, our study converged on rules that would be useful across a broad swath of model types. In brief, the rules are: (1) Define context clearly. (2) Use contextually appropriate data. (3) Evaluate within context. (4) List limitations explicitly. (5) Use version control. (6) Document appropriately. (7) Disseminate broadly. (8) Get independent reviews. (9) Test competing implementations. (10) Conform to standards. Although some of these are common sense guidelines, we have found that many are often missed or misconstrued, even by seasoned practitioners. Computational models are already widely used in basic science to generate new biomedical knowledge. As they penetrate clinical care and healthcare policy, contributing to personalized and precision medicine, clinical safety will require established guidelines for the credible practice of modeling and simulation in healthcare.

Related Publications
Mulugeta, L. and Erdemir, A., "Credible Practice of Modeling & Simulation in Healthcare", Proceedings of the ASME/FDA 2013 1st Annual Frontiers in Medical Devices: Applications of Computer Modeling and Simulation, 11-13 September 2013, Washington, DC, FMD2013-16080. [Poster] (2013)  View
Abstract

Computational modeling and simulation (M&S) methods have substantial potential to support clinical research and decision support. Consequently, substantial investment is being made by government agencies and industry to advance research and development activities in simulation-based medicine and notable discoveries are being made. However, the mechanisms or processes necessary to appropriately translate these research activities and discoveries in computational methods to clinical research and practice are lacking. Moreover, there is substantial research diversity in the field such that subject matter experts within and across mathematical and biological disciplines tend to have their own interpretation of credible practice in M&S. This can result in misuse and distrust of the tools among medical practitioners, ultimately leading to their under-utilization across all aspects of medicine. To help fill this gap, the “Committee on Credible Practice of Modeling & Simulation in Healthcare” was established under the Interagency Modeling and Analysis Group (IMAG) and the Multiscale Modeling (MSM) Consortium. The IMAG and MSM are organized by the National Institutes of Health (NIH) in collaboration with other government agencies and academic researchers to promote the advancement of computational medicine. The objectives of Committee on Credible Practice of Modeling & Simulation in Healthcare are to establish guidelines and identify new areas of research for the development and implementation of credible computational models and simulations for healthcare research and intervention.

Mulugeta, L. and Erdemir, A., "Credible Practice of Modeling & Simulation in Healthcare", Proceedings of the ASME/FDA 2013 1st Annual Frontiers in Medical Devices: Applications of Computer Modeling and Simulation, 11-13 September 2013, Washington, DC, FMD2013-16080. (2013)  View
J. Barhak, A. Erdemir, A. Hunt, J. Ku and L. Mulugeta, “Establishing Credible Practice Guidelines for Simulation-Based Medicine”, 14thInternational Meeting on Simulation in Healthcare, 25-29 Jan. 2014, San Francisco, CA. (2014)
L. Mulugeta and A. Erdemir, “Common Practice Guidelines: A Significant Gap in Computational Modeling and Simulation in Healthcare”, 2014 Interagency Modeling and Analysis Group Multiscale Modeling Consortium Meeting – Satellite Meeting, 05 September 2014, Bethesda, MD. (2014)
A. Erdemir, L. Mulugeta and W.W. Lytton, “Ten ‘Not So’ Simple Rules for Credible Practice of Modeling and Simulation in Healthcare: A Multidisciplinary Committee Perspective”, Frontiers in Medical Devices Conference: Innovations in Modeling and Simulation, 18-20 May 2015, Washington DC. (2015)  View
J.E. Bischoff, L. Mulugeta, A. Erdemir and A. Hunt, “Towards the Establishment of Guidelines for the Credible Practice of Modeling and Simulation in Healthcare”, Latsis Symposium ETH Zurich on Personalized Medicine 2016, 27-29 June 2016, Zurich, Switzerland. (2016)  View
M. Horner, L. Mulugeta, A. Erdemir, G. An, D.M. Eckmann, J.E. Bischoff, C.A. Hunt, J. Ku, D. Lochner, W.W. Lytton, V. Marmarelis, J.G. Myers, G. Peng, M. Steele and M. Walton, “A Comparison of Community-Based Guidelines and Standards for the Credible Use of Computational Methods in Healthcare”, 2016 BMES/FDA Frontiers in Medical Devices, Washington, DC. (2016)  View
M. Horner, L. Mulugeta, A. Erdemir, G. An, D.M. Eckmann, J.E. Bischoff, C.A. Hunt, J. Ku, D. Lochner, W.W. Lytton, V. Marmarelis, J.G. Myers, G. Peng, M. Steele and M. Walton, “Credibility of Computational Methods in Healthcare: A Comparison of Community-based Standards and Guidelines”, 2016 ASME V&V Symposium, Las Vegas Neva. (2016)  View
L. Mulugeta, L. Tian, M.J. Steele, M. Horner, A. Erdemir, J.E. Bischoff, D.M. Eckmann, C.A. Hunt, J.P. Ku, D.R. Lochner, W.W. Lytton, V. Marmarelis, T.M. Morrison, J.G. Myers, G.C.Y. Peng, and M. Walton, “Developing Credible Practice Guidelines for Modeling and Simulation in Healthcare: A Multifaceted Approach”, 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 16-20 August 2016, Orlando, FL. (2016)  View
Credibility, Replicability, Reproducibility in Simulation for research and clinical application (2017)  View
L. Mulugeta, A. Erdemir, J. Ku and T. Morrison, “Pathway to Credible Practice Guidelines for Computational Modeling in Healthcare “, ASME 2014 Verification and Validation Symposium, 7-9 May 2014, Las Vegas, NV. (2014)
Feedback