Review of summary document providing Committee overview

[Martin Steele]

Definition for "multiscale model."

Lealem, this is interesting. In my academic upbringing, one rule is to keep an M&S at one level of detail; that is, do not have one part of an M&S a one level of detail and another part of an M&S at a different level of detail. In practice, of course, that is more ... fuzzy.

Is there some papers or reference information on this topic? ... perhaps that justify having various levels of abstraction in M&S?

[Lealem Mulugeta]

Definition for "multiscale model."

Is there some papers or reference information on this topic? ... perhaps that justify having various levels of abstraction in M&S?

Multiscale modeling has become pretty common thing these days. In fact Digital Astronaut and a few other groups within NASA have been looking at this issue for a while now. One example is using a time resolved lumped parameter model of the cardiovascular system to generate boundary conditions for a CFD model of the heart to assess the influence of cardiovascular system changes on the shape and performance of the heart in microgravity conditions. The visual impairment issue that we are currently working on modeling is very much a multiscale problem. I have a poster we presented at the HRP Investigators' Workshop if you want me to send it to you. I can't attach it to this post because it is too large.

An open source mutlisclae model integration tool has also been published by Glenn Research Center because of the need for multiscale in a number of engineering M&S problems (the name slips my mind). In the wider research community, there are multiple research groups that have published on best approaches on how to deal with multiple scales. I believe the Physiome Project has been trying to establish standardized methods and markup languages for this: http://link.springer.com/chapter/10.100 ... 0-1935-5_8

Here is another presentation that summarizes other works and the challenges of multiscale modeling for computational biology/medicine and why it is necessary: http://www.imagwiki.nibib.nih.gov/media ... 3_2012.ppt

But there are numerous other works that try to deal with this problem and why it is necessary. I will dig up a couple of other sources I have that get at the heart of why multiscale is needed and how one might go about to tackling the problem.

I hope that answers your question. If not, let me know if you were looking for something more specific.

[Lealem Mulugeta]

Jacob,

Thanks a lot for taking the point on this. See below for my concurrences, reservations, and recommended alternative changes.

Slide 7: add Engineering before mathematical
Slide 8: add especially before complicated

Agreed these two changes

Slide 8: extend uncertainty estimation by adding and propagation

If this suggested change must be made, then I recommend that the bullet that states "Error propagation" should also be extended to be, "Error estimation and propagation". I agree that error and uncertainty propagation are important. However, for cases wen one cannot directly quantify error, error estimation is important. Depending on the problem at hand, the difference between the true value and the calculated valued may be important (e.g. there may be an accuracy threshold requirement by the decision maker using the model).

Moreover, it is also important to note that the list is not intended to be comprehensive, but only a good list of examples. So what I would also recommend to edit the major bullet that precedes the bullet list to read as follows:

"In the modeling & simulation (M&S) world, a plethora of commendable yet unconnected scientific and community activities exist for, but not limited to:"

This will alleviate any future concerns that may be raised due to the incompleteness of the list.

Slide 9: add complete after inclusive

I don't agree with this change. I don't know how we can be complete in our inclusive communication efforts because to me that suggests it means we need to ensure that everyone understands everything we're communicating at all times. This does not seem realistic. The best we can do is be simply inclusive in our communication by being as transparent as possible, and do our best to be responsive to the greater community. So I think we should leave this slide as it is.

Slide 11: add the phrase and provide definitions

I don't quite understand this edit suggestion.

Slide 11: add qualification to the list

What is meant by "qualification"? This is the first time I am hearing of qualification with respect to M&S. We can add it, but it is not clear to me if it is a term that is used widely enough in M&S practice to be added to this list. Perhaps, it is better to just add it to the list of terms we need to define as well and leave it off this example list.

With that said, it is important to note that the list is not intended to be complete just like on slide 8. The terms listed are merely examples. This is why the abbreviation "e.g." is added at the end of the major bullet preceding the list of terms. But it can be added if people feel strongly enough about it.

Martin asked to add Improve quality to slide 7

I like this suggestion, as well as the suggestion made to change the first bullet completely to read,

"Modeling and simulation offers the capabilities to potentially expedite, improve the quality, and increase the efficiency of healthcare delivery by supporting clinical research and decision making"

I will also provide comments to the post on May 16.

Thanks gain!

Lealem

[Lealem Mulugeta]

Slide 7: Need, under Clinical Urgency:
- Change the first bullet
From:
* There is a pressing need to utilize computational modeling & simulation to support clinical research and decision making in healthcare.
To :
* Modeling and simulation offers the capabilities to potentially expedite and increase the efficiency of healthcare delivery by supporting clinical research and decision making

I agree with this change in conjunction with the augmentation the Martin suggested as indicated in my previous response.

From:
* There is a gap in mechanisms or processes for translating computational models to the clinical practice.
To:
* There is a gap in mechanisms or processes for translating computational research models to the clinical practice.

I agree.

- Add a new third bullet:
* Current computing technology can now replace many human tasks and decisions. It is important that the ability of computers is neither exaggerated nor diminished. It is important to gage this transition of tasks from human to machine in a manner that will be most efficient while diminishing negative phenomena. Establishing the credibility level of models will help smooth this transition.

This is very lengthy. It needs to be truncated to something more concise so that we stay consistent with the rest of the presentation format... I don't have an immediate suggestion on how we may reduce it, but it needs to be condensed so that it is consistent with the presentation and gets the point across quickly.

Slide 8: Need, Under the long list of Scattered activities:
Add the following bullet at the end of the slide:
* Sensitivity Analysis / Results Robustness

These are important, but we have run out of room to add anymore items. As I indicated in a previous response, the list is not intended to be complete. I have no issues with adding it, but we may need to drop something else. But my suggestion would be to take all of the items that have been suggested, and add them to the glossary of terms/phrases so that they are not lost.

Slide 10: Charge
Add a new bullet:
* Identify and promote innovative game changing technologies establishing model credibility

I think this is an integral part of the second charge, "Propose guidelines and procedures for
credible practice". As part of this second charge we are gong to be reviewing the various methods currently available, new and out of the box methods, as well as those we as a committee we are going to recommend as a unique contribution to the field. So I don't see the need to add this bullet.

Slide 11 Charge - Adopt a consistent terminology:
Add 4 bullets to the long list of vocabulary:
* Abstraction
* Assumption
* Intended Use
* Referent

Once again, the list is not intended to be complete. So I recommend adding these terms to the glossary so that they are captured in our end product. If there are strong feelings about including them, I am fine with adding them as long as they fit and the presentation does not get too busy. The intention of this presentation is to communicate our mission in a concise fashion, and I am concerned that adding to these lists that are intended to be examples may make the slides too busy to take away from the main message. But if you can creatively add them so that things look clean, I am fine with it.

Slide 12: Charge - Propose guidelines and procedures for credible practice:
Add the bullet
* Move towards models that directly tie claims to results
The consensus with Ahmet was: Proposing guidelines and procedures for credible practice is a step towards endorsing models that directly tie claims to results – please feel free to rephrase/omit

Agreed

Slide 14: Charge - promote good practice:
Add a new bullet:
* Reward Self Criticism: Suggest methods and promote cultures and environments that allow admitting failure to speed up the development cycle

by the statement, "admitting failure", is this getting at making clear statements of a model's limitations and not exaggerating the capabilities of the model? Perhaps an example will help me understand what is meant by this statement because right now I am not sure what it means exactly. I can interpret it in different ways.

Add a new Slide after Slide 14 with the Title Charge:
Sub title in bold is:
Identify and promote innovative game changing technologies establishing model credibility

Under this title add the following bullets:
* Engage with modelers and accumulate technologies in a list
* Identify technologies that are successful in one modeling field and check if those are applicable in other modeling fields.
* Assess possible benefits of each technology from certain to highly speculative.
* Disseminate the list of technologies and findings with the modelers and modeling community.

*** Please feel free to modify these since these points have not been reviewed by others and if you feel it is too cumbersome, you can omit and simplify.

Once again, this read very much like what the second charge is intended for. If you read the last bullet on the slide that covers the second charge, it reads as follows:
"Define novel translational workflows to enhance credibility of models and simulation processes"

Perhaps the best compromise is to take some of the above suggested content and augment the "Propose guidelines and procedures for credible practice" slide.

Slide 21: Participation:
Please add the following bullets to the multidisciplinary background list at the end of the slide:
* Scientists
* Physicists
* Physicians
* Pharmacists

I am good with this change, but I would drop "Scientists" since no one is a general scientist. A scientist will fall under some category, such as Physicists, Biologists, Mathematicians, Chemists, etc.

Also this list is not intended to be comprehensive, so I would edit the major bullet above the list to, "Multidisciplinary background, including but not limited to:"

Hope that is all helpful. Thanks for all the work you and Jerry did on this, Jacob!

Lealem

[Jacob Barhak]

Please vote on changes to the committee document

Hello to all,

Can you please vote on the changes to the committee document. So far only Martin and Lealem have provided responses after my request. Recall that we should have a meeting, most likely this week, and if you do not vote it will be assumed that you abstain.

Meanwhile, here are some issues Lealem has raised
First, here is the change Tina suggested in slide 11 that I did not explain well in my shorthand:
The first bullet is: "Adopt a consistent terminology and provide definitions"
I made a poor representation of her suggestion. And this change also should carry to slide 10 since it is the charge title. So if you accept this change, it also applies to slide 10.

And Lealem, this is an example of admitting mistake. The sooner one does this, the better since a new version can be created sooner. A shorted correction cycle improves the work significantly and will support better models. Tony calls this process model falsification. I come from a computing and Quality Assurance view. The ideas behind Test Driven Development (TDD) should be applied to modeling and simulation. Also it is important from a psychological approach. Humans make mistakes and if this is realized and tests are devised then better models will evolve. I used the term admitting mistake to emphasize this. I believe we are on the same page here - so if you feel a need to change the text – I am fine as long as the idea is represented.

Lealem also wanted to merge technological innovations with the 2nd charge item. Here I feel strongly that the words technology and innovation be kept in the charge slide. I believe that we are now in a unique time frame where innovative technology plays an important role and it is not adopted fast enough due to an overly traditional approach. In my mind we should recognize this opportunity of innovative technology and we should give it priority and reflect this in the committee's charge. I am ok to merge things, as long as technological innovation remains a visible priority. And Lealem, thanks for bringing this into the spot light.

Finally, I also suggest that to simplify our work we move to use wiki technology for our documents. We can check with MSM wiki if it is possible to get space there yet there are other alternatives. The advantages of wiki technology are: it uses a simple web interface, it keeps transitions, and it keeps all versions while showing the last one and supports viewing changes. It will allow all of us to communicate better and keep a live document. It would be extremely useful for glossary. What do you think?

I look forward to see more responses before the meeting.

[Ahmet Erdemir]

Finally, I also suggest that to simplify our work we move to use wiki technology for our documents. We can check with MSM wiki if it is possible to get space there yet there are other alternatives. The advantages of wiki technology are: it uses a simple web interface, it keeps transitions, and it keeps all versions while showing the last one and supports viewing changes. It will allow all of us to communicate better and keep a live document. It would be extremely useful for glossary. What do you think?

The SimTk framework provides a wiki as well. If there is interest from members, we can activate it. As long as people use it, it can be a beneficial feature, providing version control. For building a glossary document, I am initially tempted to try the 1) discuss definitions in the forum and 2) update a structured document in subversion workflow. We can certainly switch to wiki, if that workflow seems easier for a larger group of members. Yet, for a given task, we should keep focus on one tool or the other as implementing both will likely create confusion.

Please also remember that the site provides various useful tools, e.g. surveys, mailing lists etc., that can be utilized for communication. I am not inclined to use the tools that are provided other websites. This will require users potentially needing different authentication, and us trying to maintain and keep an eye on multiple places.

Ahmet

[Ahmet Erdemir]

Please find below my responses to proposed changes.

Slide 7: Need, under Clinical Urgency:
- Change the first bullet
From:
* There is a pressing need to utilize computational modeling & simulation to support clinical research and decision making in healthcare.
To :
* Modeling and simulation offers the capabilities to potentially expedite and increase the efficiency of healthcare delivery by supporting clinical research and decision making

Yes.

- Change the second bullet
From:
* There is a gap in mechanisms or processes for translating computational models to the clinical practice.
To:
* ]There is a gap in mechanisms or processes for translating computational research models to the clinical practice.

Yes.

- Add a new third bullet:
* Current computing technology can now replace many human tasks and decisions. It is important that the ability of computers is neither exaggerated nor diminished. It is important to gage this transition of tasks from human to machine in a manner that will be most efficient while diminishing negative phenomena. Establishing the credibility level of models will help smooth this transition.

No (for now). I am not opposed to this idea but I am against using a long descriptions. We need to be concise. Also how does this relate to Clinical Urgency?

Slide 8: Need, Under the long list of Scattered activities:
Add the following bullet at the end of the slide:
* Sensitivity Analysis / Results Robustness

No, as this list is not intended to be complete. I am also in favor of keeping the slides less crowded.

Slide 10: Charge
Add a new bullet:
* Identify and promote innovative game changing technologies establishing model credibility

No, as this is part of charges 2 and 3. In slide 12, we noted that "Define novel translational workflows to enhance credibility of models and simulation processes"

Slide 11 Charge - Adopt a consistent terminology:
Add 4 bullets to the long list of vocabulary:
* Abstraction
* Assumption
* Intended Use
* Referent

No, as this list is not intended to be complete. I am also in favor of keeping the slides less crowded.

Slide 12: Charge - Propose guidelines and procedures for credible practice:
Add the bullet
* Move towards models that directly tie claims to results
The consensus with Ahmet was: Proposing guidelines and procedures for credible practice is a step towards endorsing models that directly tie claims to results – please feel free to rephrase/omit

No (for now). What is the action in here? All the list items in this slide are specific activities to realize the charge of proposing guidelines. The phrase "move towards models that directly tie claims to results" simply is a motivation for this charge.

Slide 14: Charge - promote good practice:
Add a new bullet:
* Reward Self Criticism: Suggest methods and promote cultures and environments that allow admitting failure to speed up the development cycle

Yes.

Add a new Slide after Slide 14 with the Title Charge:
Sub title in bold is:
Identify and promote innovative game changing technologies establishing model credibility

Under this title add the following bullets:
* Engage with modelers and accumulate technologies in a list
* Identify technologies that are successful in one modeling field and check if those are applicable in other modeling fields.
* Assess possible benefits of each technology from certain to highly speculative.
* Disseminate the list of technologies and findings with the modelers and modeling community.

No. This is part of charges 2 & 3 (slides 11 and 12). "Identify and promote innovative game changing technologies establishing model credibility" is not different than "Define novel translational workflows to enhance credibility of models and simulation processes"
Many of these listed items are implied by charges 2 & 3. Maybe we'll find a better way to clearly incorporate them in slides 11 and 12 without bloating the presentation.

Slide 21: Participation:
Please add the following bullets to the multidisciplinary background list at the end of the slide:
* Scientists
* Physicists
* Physicians
* Pharmacists

[/quote]

No, as this list is not intended to be complete. I am also in favor of keeping the slides less crowded.

Slide 7: add Engineering before mathematical

Yes.

Slide 8: add especially before complicated

Yes.

Slide 8: extend uncertainty estimation by adding and propagation

No, as this list is not intended to be complete, it is simply illustrative of scattered activities.

Slide 9: add complete after inclusive

No. You cannot guarantee completeness.

Slide 11: add the phrase and provide definitions

Maybe. This is already implied.

Slide 11: add qualification to the list

No, as this list is not intended to be complete.

Martin asked to add Improve quality to slide 7

Recommended changes for Slide 7 seem to take care of this.

[Joy Ku]

This document was a good overview of the committee for someone, like myself, who was deciding whether or not to join the committee. Is that the target audience for this document? If not, and the document is actually intended for a broader audience (e.g., the general researcher who is interested in what the committee does but may not necessarily be thinking of joining), then I'd like to suggest that the slides on the organization of the committee and its activity cycle be condensed into a single slide.

I agree with Tony's earlier comment on spelling out the need for modeling & simulation. This is not something everyone buys into yet (see a recent story we did in our BCR magazine: http://www.bcr.org/content/meet-skeptic ... hem-over-0). If the audience for this document is the general researcher, then we need to emphasize the need, motivation, and goals more in these slides, and perhaps decrease the details on the methods.

Joy

[Jacob Barhak]

Hi Joy, Hi Ahmet,

First Joy - if you want additional reasoning for joining the committee, please look at the discussion thread "need for the committee". And thank you for the fascinating article. It does explain the resistance to new technology very well.

This is why I am trying to emphasize the need in technological innovation and trying to promote it into the charge slide. Ahmet, I believe I used the term you yourself suggested on April 28nd in this forum, yet you chose to reject it from the list of changes. Perhaps you changed your mind seeing the overall picture, yet you did reject a point that is very important to me. Please see my previous post in response to Lealem. I ask to raise this in our next meeting.

As for wiki technology. Yes! Please do add it to out set of SIMTK tools we can access. It is supirior to SVN for drafting. For example, this entire discussion would have been much easier on wiki - if your wiki version supports storing the discussions along side the document. And for sure it is best to define our glossary that will evolve through time.

I hope the move to wiki will save us time and effort.

[C. Anthony Hunt]

Re: Voting on changes

Ahmet, Jacob:

It's hard to keep track of these various changes as posted in the Forum. A document with edits would be helpful. Some suggested changes seem helpful, others do not.

Maybe when all pending revisions are made, a new Forum can focus on the revised document. I expect that new revision ideas will be suggested as the Committee's work progresses.