Quantifying Structural Uncertainty in Health Economic Decision Models
Within a health economic decision model there are multiple sources of uncertainty. These uncertainties must be properly accounted for if a decision maker is to have confidence in the model output. Structural uncertainty is present when we are uncertain about the model output because we are uncertain about the functional form of the model. We are not certain that our model adequately reflects reality, and so we are not certain that our result would be correct, even if the true values of all input parameters were known.
Standard probabilistic sensitivity analysis will not account for structural uncertainty, which may result in spuriously precise estimate of the model outputs. Moreover, structural uncertainty may well have a greater impact on the model outputs than parameter and methodological uncertainty, yet methods for dealing with structural uncertainty are relatively underdeveloped.
To resolve the problem of structural uncertainty two broad approaches have been proposed; model averaging and discrepancy modelling. In the model averaging approach, we calculate the sum of the outputs of a set of plausible models, weighted by some measure of their adequacy. If we believe our weighted average result then we are implicitly assuming that our set of models contains the "true" model, although we do not know which the true model is. In the discrepancy modelling approach, we do not have to assume that any of our plausible models are necessarily "true", instead we try to make judgements about the discrepancy between the model output and the "true" target value. In the context of health economic decision modelling there are difficulties with both these approaches (Strong et al, 2009).
Research at CHEBS
Our current approach to quantifying structural uncertainty in health economic decision models is to build on the discrepancy modelling research that was initiated by the Managing Uncertainty in Complex Models (MUCM) project. Mark Strong (working with Jeremy Oakley and Jim Chilcott) explored this as part of his MRC Fellowship in Health Services/Public Health Research.
In March of 2009, CHEBS hosted a fruitful meeting of researchers from around the UK with an interest in Structural Uncertainty in the context of health technology assessment. The presentations from this event are available here.
Papers and Reports
- Strong M, Oakley JE. When is a model good enough? Deriving the expected value of model improvement via specifying internal model discrepancies. SIAM/ASA Journal on Uncertainty Quantification. 2014;2(1):106-125. Open Access. DOI:10.1137/120889563
- Strong M, Oakley JE, Chilcott J. Managing structural uncertainty in health economic decision models: a discrepancy approach. Journal of the Royal Statistical Society, Series C. 2012;61(1):25-45. doi: 10.1111/j.1467-9876.2011.01014.x. Preprint here.