Committee Presentation

This page will contain the up to date committee presentation. Discussions regarding the page can be found at the bottom of the page.

Committee on Credible Practice of Modeling & Simulation in Healthcare

prepared by

Committee Co-Chairs - Including comments from committee members

Interagency Modeling and Analysis Group

Multiscale Modeling Consortium

May 30, 2013

https://simtk.org/home/cpms

DEFINITIONS

Credible Practice of Modeling & Simulation in Healthcare

dependable with a desired certainty level to guide research or support decision making within a prescribed application domain and intended use; establishing reproducibility & accountability

Credible Practice of Modeling & Simulation in Healthcare

any activity involving development, solution, interpretation and application of computational representation of biological, environmental and man-made systems and their interaction thereof

Credible Practice of Modeling & Simulation in Healthcare

specifically computational modeling; virtual representation of system(s) of interest in a usable form in order to provide descriptive and predictive metrics for timely and systematic exploration of the system(s)

Credible Practice of Modeling & Simulation in Healthcare

computational solution of models to quantify descriptive and predictive metrics of system(s) of interest; including related post-processing efforts to calculate these metrics from raw analysis results

Credible Practice of Modeling & Simulation in Healthcare

any activity involving development, maintenance, advancement, or administration of medical care; including research, diagnosis, risk assessment, prevention, therapy, rehabilitation, surgery, intervention design, and regulation

NEED

• Clinical urgency
  • Modeling and simulation offers the capabilities to potentially expedite, improve the quality, and increase the efficiency of healthcare delivery by supporting clinical research and decision making.
  • There is a gap in mechanisms or processes for translating computational research models to the clinical practice.
  • Current computing technology can now replace many human tasks and decisions. It is important that the ability of computers is neither exaggerated nor diminished. It is important to gage this transition of tasks from human to machine in a manner that will be most efficient while diminishing negative phenomena. Establishing the credibility level of models will help smooth this transition.
• Research diversity
  • Within and across Engineering, mathematical and biological disciplines, subject matter experts tend to have their own interpretation of credibility assessment.
  • Tools and good practice recommendations do not readily transfer across disciplines or are not adopted by different fields.
• Challenges of multiscale analysis
  • Multiscale simulations require re-use of subsidiary models, which may have varying (or unknown) credibility.
  • Propagation of uncertainties of a given scale for multiscale analysis can be especially complicated.
• Scattered activities

  • In modeling & simulation (M&S) world, a plethora of commendable yet unconnected scientific and community activities exist for

  • verification & validation

  • uncertainty estimation and propagation
  • error propagation
  • evidence based assessment
  • reporting recommendations
  • model exchange and dissemination
  • standards and mark-up development
  • documentation
  • Sensitivity Analysis / Results Robustness

None of us are experts in everything. We need to learn from each other. Credible practice of modeling and simulation in healthcare requires ongoing inclusive complete communications to establish adaptive workflows that can be utilized broadly.

CHARGE

• Adopt a consistent terminology and provide definitions
• Propose guidelines and procedures for credible practice
• Demonstrate workflows for credible practice
• Promote good practice
• Identify and promote innovative game changing technologies establishing model credibility

Adopt a consistent terminology

• Clearly describe committee's understanding of modeling and simulation vocabulary, e.g.,
  • model
  • simulation
  • multiscale
  • verification
  • validation
  • uncertainty
  • sensitivity
  • credibility
  • accreditation
  • Abstraction
  • Assumption
  • Intended Use
  • Referent
  • qualification
• Consistently use appropriate vocabulary for communication
• Unify discipline-specific appreciation of modeling and simulation vocabulary

Propose guidelines and procedures for credible practice

• Identify translational research directions to leverage readily available techniques to increase credibility of modeling and simulation in healthcare, e.g.,
  • verification and validation
  • uncertainty quantification
• Establish guidelines to implement readily available model evaluation techniques which are aware of the constraints of clinical practice
• Define novel translational workflows to enhance credibility of models and simulation processes
• Move towards models that directly tie claims to results

Demonstrate workflows for credible practice

• Summarize state-of-the-art by identifying examples in literature and in field practice for the development of model credibility assessment procedures
• Conduct studies, ideally with proposed guidelines and procedures, for the implementation of credibility assessment procedures in modeling and simulation in healthcare

• Disseminate examples of credibility assessment by providing a repository of case studies on the implementation of proposed guidelines and procedures for different types of M&S

Promote good practice

• Bridge synergistic activities of establishing confidence in simulation-based medicine conducted by
  • modeling and simulation communities
  • scientific societies
  • clinical practitioners
  • regulatory agencies
• Outreach to advocate credible practice of modeling and simulation through increased awareness and training; by providing
  • webinars
  • lectures
  • guidelines
• Reward Self Criticism
  • Suggest methods and promote cultures and environments that allow admitting failure to speed up the development cycle

Identify and promote innovative game changing technologies establishing model credibility

• Engage with modelers and accumulate technologies in a list
• Identify technologies that are successful in one modeling field and check if those are applicable in other modeling fields.
• Assess possible benefits of each technology from certain to highly speculative.
• Disseminate the list of technologies and findings with the modelers and modeling community.

ORGANIZATION

• COMMITTEE Co-Chairs & Members: EXECUTE & CHARGE

• ADVISORY COUNCIL: REVIEW & ADVISE

Interagency Modeling and Analysis Group (IMAG)

Multiscale Modeling Consortium (MSM)

ACTIVITY CYCLE

attachment:ActivityCycle.jpg

ROLES & EXPECTATIONS

Committee Co-Chairs (2)

• Provide leadership and guidance to the Committee
• Assign tasks to Committee Members to realize charge of the Committee and incorporate their work to reach Committee end-products
  • Serve as Committee Members, i.e., take on and fulfill assigned tasks to realize charge of the Committee
  • Report to Advisory Council and implement their feedback
  • Report to IMAG/MSM and implement their feedback
  • Evaluate performance of Committee Members
  • Evaluate performance of Advisory Council
  • Share authority and responsibilities equally

ROLES & EXPECTATIONS

• Committee Members (10 including Co-Chairs)
  • Complete assigned tasks (on schedule) to realize charge of the Committee and prepare Committee end-products
  • Report to the Committee and implement their feedback
  • Represent a particular discipline and an area of practice to diversify Committee interests and reach out
  • Evaluate performance of Committee Co-Chairs
  • Evaluate performance of Advisory Council
  • Expect to invest 8 hours of effort per month

ROLES & EXPECTATIONS

• Advisory Council
  • Review Committee activities (on schedule) and provide feedback on the pathway for realization of the charge of the Committee and for preparation of Committee end-products
  • Represent a particular discipline and an area of practice to broaden Committee affairs and reach out
  • Evaluate performance of Committee Co-Chairs
  • Evaluate performance of Committee Members
  • Expect to invest 1 hour of effort per month

PARTICIPATION

Committee Co-Chairs (2)

• Nominated and elected by the Committee Members and the Advisory Council
• 2 year term with possibility of reappointment
• Elections 6 months prior to end of term
• Continuing post subject to commitment to leadership
• Ideally, one from IMAG and the other from MSM
• First-term (04/2013- 04/2015) appointed via IMAG consensus
  • Lealem Mulugeta (NASA, IMAG representative)
  • Ahmet Erdemir (Cleveland Clinic, MSM-WG representative)

PARTICIPATION

Committee Members (10 including Co-Chairs)

• Participation through nomination and invitation
• 2 year term with possibility of reappointment
• Continuing membership subject to commitment to timely completion of tasks
• Multidisciplinary background - clinicians - mathematicians - biologists - engineers - statisticians - computer scientists - product developers

PARTICIPATION

Committee Members (10 including Co-Chairs)

• Wide variety of groups
  • IMAG representatives
  • MSM working groups
  • External representatives
  • Professional Societies
  • Regulatory Agencies
  • Funding Agencies
  • Scientists
  • Physicists
  • Physicians
  • Pharmacists

PARTICIPATION

Advisory Council

• Participation through nomination and invitation
  • 2 year term with possibility of reappointment
  • Continuing membership subject to commitment to providing feedback in a timely manner
  • Substantial experience and expertise in respective areas of interest

FIRST-TERM (2013-2015) GOALS

• Form the Committee and the Advisory Council
• Establish infrastructure for communications between Committee Co-Chairs and Members, and Advisory Council
• Establish a web-based framework for development, community interactions, and dissemination
• Develop a guidance document on establishing credible practice of modeling and simulation in healthcare
• Draft a proposed model certification process

FIRST-TERM (2013-2015) TIMELINE

by April 30, 2013

• Establish communication infrastructure
• Establish web-based framework
• Appoint Committee Members
• Introductory meeting with Committee Members

• Discuss objectives, roles & assign tasks

by May 31, 2013

• Meeting with Committee Member to review first set of tasks
• Appoint Advisory Council
• Introductory meeting with Advisory Council

• Discuss objectives, roles & summarize progress

by June 30, 2013

• Working outline of guidance document

by MSM 2013 meeting

• Community outreach to summarize progress and plans

by April 30, 2014

• First draft of guidance document for external review

by October 31, 2014

• Release of guidance document

by November 30, 2014

• Journal article/white paper based on guidance document
• Outline of model certification process
• Co-Chairs; nominations and election

through December 2014 – April 2015

• Co-Chairs; transition
• First draft of proposed model certification process

CONTACT

CHECK https://simtk.org/home/cpms

Lealem Mulugeta

Committee Co-Chair

mulugeta@dsls.usra.edu

lealem.mulugeta@nasa.gov

Ahmet Erdemir

Committee Co-Chair

erdemira@ccf.org

COMMENTS & DISCUSSION

Committee members, please explain major changes here in text to keep a discussion going. The discussion will be periodically removed, yet still be available in previous versions of the wiki page.

This version contains combined changes from multiple committee members as collected and posted to the committee forum by Jacob Barhak on 16 May 2013 and May 30 2013 . See: https://simtk.org/forums/viewtopic.php?f=848&t=4094&start=30