Regression methods
Decision analytic models (DAM) used to evaluate the cost-effectiveness of interventions are pivotal sources of evidence used in the NICE Technology Appraisal (TA) process.
It is becoming increasingly common for parameter estimates used in the DAMs to be informed by some kind of regression analysis on individual patient level data but there is currently little guidance relating to reporting standards for such inputs.
Objectives
- To identify the frequency of use of regression models in NICE TA submissions; the parameters they inform, and the amount of information reported to describe and support the analyses.
- To produce suggestions for guidance on good practice in this area.
Method
A random sample of 79 Appraisal submissions was selected from all appraisals (n=111) issued since the publication of the updated NICE Methods Guide in June 2008. An extensive data extraction form was developed and used to extract information on model formulation, diagnostics, performance, and how the results (and their variability) are fed-into and propagated through the DAM. The focus was on the reporting and transparency of the analyses; we did not seek to make judgements about the appropriateness or otherwise of the analyses.
On completion of the review, our expert working group convened to discuss the results in detail. Recommendations for good practice were drafted together with a checklist for critiquing reporting standards in this area. Consensus and final versions were achieved iteratively through email correspondence.
Results
Of the 79 technology appraisals examined, 47 included at least one regression analysis and a total of 91 separate regression analyses were reported. 56 were de novo analyses provided by the manufacturer/sponsor of the technology (34 from Single Technology Appraisals and 22 from Multiple Technology Appraisals), while the remaining 35 were sourced from existing published literature. Over 50% involved health state utility values with the balance involving health care costs (11%) or probabilities of clinical events (35%).
For the de novo analyses, reporting was poorest around the sample size used, the justification of the type of model estimated, the selection of covariates used, the strategy for identifying the preferred final model and any validation used.
Across all the analyses, there was potential for improvement in the reporting of: the description of the dataset, the model type, the rationale for inclusion of model covariates, the validity of the final model and the uncertainty in the model.
Conclusion
Statistical regression models are in widespread use in NICE TAs yet reporting standards relating to basic information are poor. Whilst some of this may be due to the word limit imposed on TAs, there is still scope for improvement. This is important as increasing levels of reporting transparency enable policy decision makers to have increasing levels of confidence in the resulting estimates of cost-effectiveness.
We suggest a series of recommendations that could be used for the minimum reporting requirements for any statistical regression analyses used in a DAM.
DSU report
Related publication
B Kearns, R Ara, A Wailoo, A Manca, M Hernandez Alava, K Abrams, M Campbell (2013) Good practice guidelines for the use of statistical regression models in economic evaluations. PharmacoEconomics; 31 (8): 643-652