Evidence synthesis TSD series
A series of TSDs have been produced in the area of evidence synthesis.
Downloads
TSD 1
Introduction to evidence synthesis for decision making (PDF, 106KB)
TSD 2
WinBUGS system (.odc) files (last updated Sept 2016) (.zip, 6.1MB)
TSD 3
Heterogeneity: subgroups, meta-regression, bias and bias-adjustment (PDF, 492KB)
WinBUGS system (.odc) files (.zip, 197KB)
TSD 4
Inconsistency in networks of evidence based on randomised controlled trials (last updated April 2014) (PDF, 429KB)
WinBUGS system (.odc) files (last updated March 2013) (.zip, 13KB)
TSD 5
Evidence synthesis in the baseline natural history model (PDF, 178KB)
WinBUGS system (.odc) files (.zip, 24KB)
TSD 6
Embedding evidence synthesis in probabilistic cost effectiveness analysis: software choices (PDF, 122KB)
TSD 7
Evidence synthesis of treatment efficacy in decision making: a reviewer’s checklist (PDF, 147KB)
This report refers to a checklist table. (Word document, 81KB)
TSD 18
Methods for population-adjusted indirect comparisons in submissions to NICE (PDF, 1.5MB)
Appendix D: Worked example of MAIC and STC (PDF, 295KB)
Code for R (.zip, 3KB)
The documents above are also available in a zip file (.zip, 1.3MB).
Related publications
Phillipo D, Ades A, Dias S, Palmer S, Abrams K and Welton N (2018) Methods for population-adjusted indirect comparisons in health technology appraisal. Medical Decision Making, 38(2), pp. 200-211.
TSD 20
Multivariate meta-analysis of summary data for combining treatment effects on correlated outcomes and evaluating surrogate endpoints (PDF, 1.2MB) – October 2019 – Updated December 2022: Trivariate meta-analysis model in product normal formulation with unstructured covariance has been checked and replaced in version 2.
TSD 20 one-page summary (PDF, 549KB)
About
- TSDs 1-7
-
TSDs 1 to 7 have been published as a series in Medical Decision Making (July 2013).
The intention behind the TSDs is not to be prescriptive, but rather to explain the requirements for evidence syntheses set out in the 2008 Methods Guide and to provide guidance on methods that meet these requirements.
TSDs 2, 3, 4 and 5 include an extensive set of worked examples of analyses using Bayesian Markov chain Monte Carlo methods, using WinBUGS. The WinBUGS code should not be copied and pasted from the text; users are instead advised to download the WinBUGS system (.odc) files.
When all seven documents have been completed, those already on the website will be updated in order to harmonise and complete the cross-referencing. Instructions for citing these documents can be found on page 3 of each TSD.
- TSD 18
-
A standard methodology for indirect comparisons is now used routinely in submissions to NICE. This allows estimates on the relative effect of C vs B to be estimated from an AB trial and an AC trial.
Recently, publications have appeared on “Matching-adjusted Indirect Comparisons” (MAICs) and on “Simulated Treatment Comparisons” (STCs). The aim of these methods is to hold out the promise of improving on standard indirect comparisons by using individual patient data (IPD) in a subset of trials to control for imbalances in “baseline characteristics” in the trial evidence.
These methods are now being used increasingly in STA submissions to NICE, particularly when the manufacturer has access to individual patient data on its own AB trial, but only aggregate data on the competitor’s AC trial.
The purpose of this TSD is to comment on the reliability of these methods and to provide recommendations on their use in the context of NICE appraisals.
- TSD 20
-
The TSD describes the use of multivariate meta-analytic methods for synthesis of data from multiple correlated outcomes as well as for evaluation of surrogate endpoints, in the context of HTA.
Health technology assessment (HTA) agencies such as the National Institute for Health and Care Excellence (NICE) require evidence synthesis of existing studies to inform their decisions. For example; gathering information about the best available treatments with respect to multiple efficacy and safety outcomes. However relevant studies may not provide direct evidence about all the treatments or outcomes of interest. Studies that do not provide direct evidence about a particular outcome or treatment of interest are often excluded from a meta-analysis evaluating that outcome or treatment.
This exclusion is unwelcome, especially if participants are otherwise representative of the population, clinical settings and condition of interest. Research studies require considerable costs and time, and involve precious patient participation, and simply discarding them could be viewed as research waste. Statistical models for multivariate and network meta-analysis address this by simultaneously analysing multiple outcomes and multiple treatments, respectively. This allows more studies to contribute toward each outcome and treatment comparison.
This Technical Support Document (TSD) describes the key methods of multivariate meta-analysis and their extensions to network meta-analysis of multiple outcomes. We focus on the use of multivariate meta-analytic methods for combining data from multiple correlated outcomes with the aim of including all relevant evidence and borrowing of information across outcomes, in particular when not all of the relevant studies report an outcome of interest. We also devote considerable attention to the use of multivariate meta-analysis for the purpose of surrogate endpoint evaluation.
We describe and apply the methods in the Bayesian framework of estimation and provide WinBUGS code for a Bayesian analysis using Markov chain Monte Carlo (MCMC) simulation.