Observational data TSD
A technical support document about observational data produced by the Nice DSU.
Download
About
Non-randomised studies may be used either to complement the evidence base represented by randomised controlled trials (RCTs) or as the source of evidence for a specific effectiveness parameter if randomised data are not available.
The current 2013 NICE Guide to the Methods of Technology Appraisal (Methods Guide) recognises the potential biases that may arise from the use of non-RCT data, namely from confounding, lack of blinding, incomplete follow-up and lack of a pre-specified end-point. It recommends that potential biases should be identified, and ideally quantified and adjusted for. However, it does not provide guidance on how to estimate treatment effect or the appropriate methods to deal with potential biases.
Lack of clear guidance on the use of evidence from non-RCTs may lead to inappropriate and inconsistent use of methods which in turn lead to biased estimates which may have potential adverse consequences for decisions on the effectiveness and cost-effectiveness of health technologies.
The objectives of this Technical Support Document (TSD) are to:
- Summarise commonly available methods to analyse comparative individual patient data (IPD) from non-RCTs to obtain estimates of treatment effect to inform NICE Technology Appraisals (TAs).
- Propose a set of recommendations to improve the quality and transparency of future assessments.
It includes a summary of:
- The most commonly used methods for non-randomised IPD (Section 2).
- TAs which used non-randomised data to inform estimates of treatment effect for the cost-effectiveness analysis (Section 3).
It also includes:
- An algorithm to aid selection of the appropriate method(s) for the analysis (Section 4.1).
- A review of existing checklists for quality assessment of the analysis of non-randomised studies (Section 4.2.1).
- A novel checklist (the QuEENS checklist) to assess the quality of the analysis of non-randomised studies (Section 4.2.2).
- A summary of findings and final recommendations (Sections 5.1-5.2).
This TSD provides practical guidance on the methods that are relatively straightforward to apply and are most commonly used in statistical and econometric analysis of non-randomised data. Specifically, it focuses on approaches that can be applied using standard statistical software without additional bespoke programming and advanced econometric/statistical skills. It is therefore aimed at those typically engaging in the NICE TA process whether submitting or reviewing evidence. The reviews and tools presented in this TSD are intended to help improve the quality of the analysis, reporting, critical appraisal and interpretation of estimates of treatment effect from non-RCT studies.
Related work on real world data (RWD)
The use of real world data for the estimation of treatment effects in NICE decision making