Should “technology” be included in the Committee Charge?

[Martin Steele]

First: Definition
I suggest not uniquely defining the word “technology,” but rely on the existing, well-established, ones. http://www.answers.com/topic/technology

If we need a definition of “M&S Technology”:
The application of scientific knowledge, methods, and materials to the development, production, and use of models and simulations towards a practical objective.

Second: My short answer is that I choose Option 1.
If I’m reading the latest version of Committee Charges, technology is inherently part of them.

Technology has an inherent role in M&S Credibility. Going back to this Forum’s premise, we are specifically addressing the “Credible Practice of Modeling & Simulation,” and technology plays a role in that as well.

The platform on which M&S is accomplished most definitely affects the credibility of the model or simulation. Many times, the platform is discounted because it is in such wide use, and thus inherently accepted. This has at least some risk associated with it. In the case of a computational model, it is built on some computer hardware with some operating system – this is the aforementioned platform. Most often, some application software or computer language is used to implement the model. The model is then used with some set of inputs to yield some result. Model building, verification, validation, and use is the typically what is considered in M&S credibility. However, the technology upon which models are built and run, i.e., the hardware, operating system, and application software, also affect the quality & credibility of the effort. If any of them are changed (read: revised, as in new computer, updated operating system or application), then the whole basis for the M&S effort should come into question (credibility should be re-assessed).

{I have a diagram that more readily depicts the above, if I can find a way to upload it.}

Having said all that, a model or simulation can have complete credibility without a lot of technology. Technology is a means to better, faster, cheaper M&S. So, technology can have a distinct impact on the Credible Practice of Modeling & Simulation, but it is inherently part of doing the business of modeling & simulation.

[Tina Morrison]

i vote for option 1. most of the reasons have already been shared.

[Pras Pathmanathan]

Having read the discussion, I vote for option 1. I think technology can be taken into account as appropriate without formally being introduced into the Commitee Charge.

[Jacob Barhak]

Since most people contributed an opinion and since there was a call for further discussion, I decided to re-engage in the discussion with some relevant examples from one presentation in IMSH.

Joy Ku and Nathan Wilson both presented simulations in a panel in IMSH.

Nathan presented a cardiovascular model and was asked a question that involved timing of the model. His answer was that when he started his Ph.D. He ran long simulations on an expensive super computer, and today he can run simulations on a computer that costs less than 10k$ in less than a day.

My interpretation to this was that the creeping technology improvement made his model more credible than in the past.

Another example related to technology in the same presentation was from Joy Ku from our committee. She presented OpenSim where motion data is captured by markers and then imported to the computer for simulation. One of the medical people in the crowd asked about data and validation. Joy answered that data is captured from a group of about hundred people - Joy can give the exact number. Then the discussion continued about the ability of the model to generalize and have more data. I asked Joy if in the future capture technology can improve the availability if data. Her answer was somewhat positive - she described that there is research on capturing motion without markers and in the future it may be possible to collect more data and therefore improve the model.

My interpretations from these examples is that model credibility is sometimes highly tied to technological advancement. I imagined capture of this data using a cell phone and training the model using many movies of people walking - that are easy to get. I hope I had the right image in mind.

I like the definitions Ahmet added of disruption technology brings, yet the creeping technology also facilitates constant improvement and tied to our ability to determine credibility. Therefore I think we must include technology somehow in the committee. I notice that there is acceptance of the idea that technology has place in model improvement - then why not add this single word to the charge of the committee? It seems an important enough driver.

If someone is interested in continuing this discussion I will be happy to converse further.

And Joy, do feel free to correct my description and bring your perspective of the event I described.

[Marlei Walton]

After reading everyone's posts, my short answer is that I support option 1; I believe consideration of technology is an intrinsic part of model credibility and doesn't warrant separate consideration.

[Jerry Myers]

I am going to go with Option 1.

Dr. Steele makes the most sound reasoning in the assessment of most of the arguments.

For my part, I view M&S technology as an enabler of credibility, but not a quatifiable measure of the credibility itself. In general, I accept that new technology can increase efficiency, application of models and lead to new methods for developing models and performing simulations. i also think it reasonable to assume that any quantification of such advances can be captured within the basic measures that we have been discussing and this is the area where such a discussion should be focused, not on making a change to the charter.

I love "whiz-bang" technology innovations, such as newer and faster computers, as much as the next persion, but in the context of a particular problem, in most cases, it increases efficency of the processes that are already measured within the scope of assessing credibility. One of the discussions above mentioned time to complete a simulation as a measure of improved innovation. I would argue that if the question being asked included "within a certain abount of time" then the verification factor of the model and simulation would capture this part of the innovation since verifying that the simulation ran in the required time (with the required accuracy, a cross over with validation) is a neccesary part of the credbility assessment. If time to completion of the model is not in the context of its informing the decision making, then the model is more efficient, but I would hesitate to say it is more credible for this particular case.

I would also caution that defining a particular technology innovation criteria not be allowed todrive the technology in some way. Rather letting the problems at hand drive the technology is more appropriate in this case. If memeory serves, it was the need of distributed modeling developers in the open source community that drove the need for subversion (also mentioned in the dicussion) over other pracitices that achieved similar outcomes with less efficincey and certainity, not because the technology was defined by a standards committee. Thus, I am more in favor of focusing on giving the community a small set of measures that are fundamentally quantifiable and applicable across a wide range of situations.

[James Thomas]

I am new enough to the group that I probably do not understand all the implications of this decision. It seems technological development (whether hardware or algorithmic) is critical to taking a model from something that is feasible to something that is usable. This would make technology implicit in the charge of a committee dedicated to validating M&S. Whether this requires an explicit charge (in the form of option 2) or just the implicit fact (option 1) I don't really care. Is there a parallel committee to ours that is more dedicated to technology per se? If so, then I'd go with option 1; otherwise, option 2.

[Ronald Germain]

Having read the back and forth on this topic, I have a mixed view I am afraid. I agree with many that it 'expands' the mandate rather far and there is a risk of dilution of effort. On the other hand, a major issue in the modeling field is the inability of others to re-use / test a published model if it is done 'by hand' rather than using a widely available software tool. Thus, standards or at least suggestions for the importance of validated modeling environments that allow ready model sharing and simulation are issues that directly bear on whether a particular modeling effort is actually useful to the field and hence, appear to be within the primary mandate. What I don't know is if we can rely on other efforts to qualify modeling environments to handle this 'technology issue' - if it was the case that adequate evaluation and recommendations in this sphere exist elsewhere, we could then just refer to them at the appropriate points in any recommendations we make dealing with our core issues, rather than seeking to evaluate these tools ourselves.

[Lealem Mulugeta]

Dear all,

Ahmet and I have reviewed the voting results for the three options, and they are summarized below. Please note that a few people provided their votes via email instead of responding to the forum thread.

Vote Results
Option 1: Do not add "technology" in Committee charges - 15 Votes
Option 2: Add "technology" in Committee charges blended with an existing charge balancing the emphasis - 2 Votes
Option 3: Add "technology" as a separate Committee charge - 0 Votes
Unclear / Undecided - 5 Votes

As stated in the original conditions of this pole, we will move forward with the majority vote. So the decision is to implement Option 1: to NOT add "technology" as part of the Committee’s charges.

This is the final decision the Co-chairs will enforce as the Committee moves toward the development of the credible practice guidelines. In keeping with this, we will include explicit statements on the CPMS project site, wiki and other relevant documents to indicate the Committee’s charges do not extend to technology. We will further substantiate this statement with the numerous arguments made in favor of Option 1.

This does not, however, prevent any of the Committee members from independently (i.e. outside of the Committee’s activities) promoting technology for their own personal interests. However, such independent activities should not interfere with the goals of the Committee.

With that said, we see the value in the feedback we received regarding Option 2. Therefore, the CPMS site will maintain a wiki page titled, “Technologies in Modeling & Simulation” (Renamed from Technologies for Credibility). The sole purpose of this wiki page is to help outline a framework which another committee/working group/task force can utilize to align the different aspects and categorization of "technologies" that are relevant to the NIBIB or other relevant institutions. However, the contents on the wiki page will not be used in the development of the "Guidelines for Credible Practice of Modeling and Simulation in Healthcare".

Thank you all for participating in the discussion. Ahmet and I were very excited to see every member of the Committee get involved in this important discussion. We hope to see this level of interaction as we move forward with the bigger decisions we will need to make in the coming months.

Regards,

Lealem

[Jacob Barhak]

This will not change results since there is an overwhelming majority not to add technology. For the record do count my vote as a yes for option 3 - this is how it all started. Yet I also supported option 2 which was a good compromise. I am proud to belong to this minority.

Never the less, looking at the text of the discussion there seems to be a recognition that technology is a major driver that impacts credibility. It would be interesting to revisit this topic in a few years to see if technological advancements will change views.

I ask a majority representative to update the wiki with the committee presentation accordingly since it currently does not reflect this vote results.