ISPOR- Possible resource?

[Joy Ku]

Is anyone familiar with the International Society for Pharmacoeconomics and Outcomes Research (ISPOR)? I just came across their journal called Value in Health (http://www.valueinhealthjournal.com/home), whose motto is "Translating outcomes research to healthcare decision." The Society also has extensive documentation related to good research practices: http://www.ispor.org/workpaper/practices_index.asp. They may have some good pointers and practices for our group to adopt.

Joy

[Jacob Barhak]

Yes Joy,

If you look at the meeting minutes you will find reference to this organization. I am a member of the organization and presented there this year in New Orleans.

The members get to see the task force draft and get to send responses.

I have engaged in multiple communications with the organization some of which were made public.

Please look at the links I am sending here to see previous interactions that were made public:

http://www.linkedin.com/groups/Technolo ... na_4158822

http://www.linkedin.com/groups/Public-R ... mp_4158822

Note that I have more feedback that was not approved for publication by ISPOR. If anyone is interested I will be happy to post this feedback to the committee forum. I was trying to separate this from the committee work, yet if others from the committee are interested I will gladly divert the discussion on ISPOR issues to this forum that is open and not heavily moderated.

I hope I have a second.

Jacob

[Jacob Barhak]

Joy, you have raised an important point.

In the last month I received 3 requests to review ISPOR task force drafts on different subjects. All 3 have the word credibility in the subject.

Those review requests are basically requests for information/feedback.

I offer the following observation:

Credibility is an important theme these days that multiple groups are trying to define in different scopes.

Does anyone from the committee other than myself has connection/affiliation/influence with ISPOR?

Do we want to formally connect to such efforts?

It would be nice if the work and products from our committee will influence other groups such as ISPOR.

[Jacob Barhak]

This posting is with regards to the ISPOR Webinar - Parameter Estimation and Uncertainty Analysis held on November 21, 2013 and presented by Andrew Briggs.

The webinar pretty much followed the guidelines in the manuscript: Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-6. Here is a link to the paper: http://www.ispor.org/workpaper/Modeling ... inty-6.pdf

The main message the speaker tried to convey was that both deterministic and probabilistic methods could be used for parameter estimation.

At the end of the webinar I asked the speaker on comments regarding game changing technologies such as High Performance Computing and Symbolic Math. The speaker answered that he was not familiar with these technologies. I am referring the speaker to a response letter written to the task force last year that was published in the ISPOR LinkedIn discussion list:
http://www.linkedin.com/groups/Public-R ... mp_4158822

Here is the long PDF version of the attached letter:
http://sites.google.com/site/jacobbarha ... pr2013.pdf

I am in hope that the speaker and the ISPOR task force will recognize the importance of these innovative technologies.

[Jacob Barhak]

Andrew Briggs did reply to my previous post. The continued public discussion moved to the ISPOR discussion list that can be seen in this link:

http://www.linkedin.com/groups/Technolo ... na_4158822

It is also appropriate to note that this discussion represents the opinions of the participants rather than the groups they are affiliated with or the forums they post to, and specifically my personal views may not represent the views of other CPMS committee members.

If CPMS members wish to have their opinion in this matter known they are welcome to continue this discussion in this forum or on the ISPOR LinkedIn discussion list.

[Jacob Barhak]

This is a summary of an email conversation that continued this discussion. The entries have been edited to remove repetition and side conversation. This summary has been reviewed and approved before release by both authors.



Andrew Briggs
4-Dec-2013

I am preparing a second edition of my text book ‘Decision Analytic Modelling for Health Economic Evaluation’. The focus of the text is very practical and I make lots of examples and illustrations available on the web demonstrating how models can be implemented in a simple spreadsheet. (Follow the link to downloads from our homepage: http://www.gla.ac.uk/hehta )

... it would be great to see some of these examples implemented in symbolic math in a form that could be made available alongside the Excel examples.

I like using Excel as a tool because most people have it on their machines and are familiar with simple spreadsheet operations. That gives the model one level of transparency. However, how Excel handles mathematical formulae is archaic and makes it extremely easy to introduce errors making debugging of Excel models a nightmare. The idea of being able to write the mathematical formula in a natural form is very attractive and leads to a different form of transparency.




Jacob Barhak
19-Dec-2013

In this email I take your challenge of implementing one of the modeling examples on your web page using symbolic math. It took me a while to find the examples you were discussing and I took the first one:

http://www.gla.ac.uk/media/media_94488_en.xls

SymbolicExample.zip

A zip archive containing Python code that uses Symbolic Math to replicate example and the output of this code

(5.37 KiB) Downloaded 138 times

The example itself on your web page may be implemented using Symbolic Math, yet to be truthful, if you would have asked me to model this, I would have chosen a spreadsheet or a numeric library such as Matlab or numpy. The question you ask there is just too simple and limited to the technology you use. With better technology the questions you ask can be more complicated.

For this reason I added another section to the program that shows where Symbolic Math can elevate your modeling to the next level. For example Symbolic Math comes useful when you deal with estimation problems that require optimization of some sort. Therefore I added another question to your exercise where I ask to optimize risk reduction to reach a certain Life year target after 5 years. This estimation problem is not trivial to solve with a spreadsheet - although possible. Symbolic Math has an advantage here.... I believe you should address Symbolic Math as a useful tool that can:
1. Double check math calculations and help eliminate human error
2. Improve understanding of model structure by allowing to explore the function behind the model
3. Help in optimization

These capabilities will help elevate the modeler to the next level where more difficult questions can be asked. This is not only an implementation issue - it is a new capability offered by technological advancement.




Andrew Briggs
26-Dec-2013

Thanks for this — though I notice that you chose to replicate the simplest example of a Markov process without time dependency in the transitions.. I’m afraid, for me at least, the illustration if anything strengthens my view that the symbolic math methods you are advocating are about implementation. As such, don’t see any particular need to champion this particular platform over, say, R, or C++ as programming languages.



Jacob Barhak
26-Dec-2013

If you look at the third part of the solution it may open your eyes. This is not about implementation alone - it is about the power of the tools used and the complexity of the questions you can ask and answer.



Andrew Briggs
12-Jan-2014

Sorry Jacob,

But you didn’t really address the challenge. You replicated the most simple example from our book that used fixed transition probabilities. As you noted yourself, the example is trivial.

Perhaps you should consider writing a scientific article that demonstrates the potential of the tools you advocate in the field of HTA and submit it for peer-review in the scientific literature.




Jacob Barhak
12-Jan-2014

You are calling for more evidence. I can help there by being specific.

Here are direct links to the papers:
http://onlinelibrary.wiley.com/doi/10.1 ... 9/abstract
http://www.sciencedirect.com/science/ar ... 351000076X



Andrew Briggs
13-Jan-2014

A quick review of your papers suggests that these are an important contribution. I can see the arguments around the flexibility of the estimation method. Nevertheless, the methods you describe are not unique and others have written similarly of the need to estimate ‘latent’ parameters and have variously used maximum likelihood estimation / R / WinBugs etc. to implement their solutions.



Jacob Barhak
13-Jan-2014

Your recognition of the work is appreciated. And yes, you are correct, with additional effort it is even possible to implement the same without using Symbolic Math, just like it is possible to implement a model using Markov, DES, or as an equation. These are all tools. And Symbolic Math and HPC are additional tools in the arsenal of the modeler. Specifically those are highly useful for estimation as demonstrated - this was the point I was trying to make in your seminar and this discussion. These are game changers that should be addressed just like Markov and DES are addressed specifically in your guidelines.m