Evidence synthesis TSD series

Downloads

TSD 1

Introduction to evidence synthesis for decision making (PDF, 106KB)

TSD 2

A general linear modelling framework for pair-wise and network meta-analysis of randomised controlled trials (last updated Sept 2016) (PDF, 1.6MB)

WinBUGS system (.odc) files (last updated Sept 2016) (.zip, 6.1MB)

TSD 3

Heterogeneity: subgroups, meta-regression, bias and bias-adjustment (PDF, 492KB)

WinBUGS system (.odc) files (.zip, 197KB)

TSD 4

Inconsistency in networks of evidence based on randomised controlled trials (last updated April 2014) (PDF, 429KB)

WinBUGS system (.odc) files (last updated March 2013) (.zip, 13KB)

TSD 5

Evidence synthesis in the baseline natural history model (PDF, 178KB)

WinBUGS system (.odc) files (.zip, 24KB)

TSD 6

Embedding evidence synthesis in probabilistic cost effectiveness analysis: software choices (PDF, 122KB)

TSD 7

Evidence synthesis of treatment efficacy in decision making: a reviewer’s checklist (PDF, 147KB)

This report refers to a checklist table. (Word document, 81KB)

TSD 18

Methods for population-adjusted indirect comparisons in submissions to NICE (PDF, 1.5MB)

Appendix D: Worked example of MAIC and STC (PDF, 295KB)

Code for R (.zip, 3KB)

The documents above are also available in a zip file (.zip, 1.3MB).

Related publications

Phillipo D, Ades A, Dias S, Palmer S, Abrams K and Welton N (2018) Methods for population-adjusted indirect comparisons in health technology appraisal. Medical Decision Making, 38(2), pp. 200-211.

TSD 20

Multivariate meta-analysis of summary data for combining treatment effects on correlated outcomes and evaluating surrogate endpoints (PDF, 1.2MB) – October 2019 – Updated December 2022: Trivariate meta-analysis model in product normal formulation with unstructured covariance has been checked and replaced in version 2.

TSD 20 one-page summary (PDF, 549KB)

TSD 25

Evidence Synthesis of Diagnostic Test Accuracy for Decision Making (PDF, 1,740KB)

TSD 25: One page summary (PDF, 256KB)

---

About

TSDs 1-7

TSDs 1 to 7 have been published as a series in Medical Decision Making (July 2013).

The intention behind the TSDs is not to be prescriptive, but rather to explain the requirements for evidence syntheses set out in the 2008 Methods Guide and to provide guidance on methods that meet these requirements.

TSDs 2, 3, 4 and 5 include an extensive set of worked examples of analyses using Bayesian Markov chain Monte Carlo methods, using WinBUGS. The WinBUGS code should not be copied and pasted from the text; users are instead advised to download the WinBUGS system (.odc) files.

When all seven documents have been completed, those already on the website will be updated in order to harmonise and complete the cross-referencing. Instructions for citing these documents can be found on page 3 of each TSD.

TSD 18

A standard methodology for indirect comparisons is now used routinely in submissions to NICE. This allows estimates on the relative effect of C vs B to be estimated from an AB trial and an AC trial.

Recently, publications have appeared on “Matching-adjusted Indirect Comparisons” (MAICs) and on “Simulated Treatment Comparisons” (STCs). The aim of these methods is to hold out the promise of improving on standard indirect comparisons by using individual patient data (IPD) in a subset of trials to control for imbalances in “baseline characteristics” in the trial evidence.

These methods are now being used increasingly in STA submissions to NICE, particularly when the manufacturer has access to individual patient data on its own AB trial, but only aggregate data on the competitor’s AC trial.

The purpose of this TSD is to comment on the reliability of these methods and to provide recommendations on their use in the context of NICE appraisals.

TSD 20

The TSD describes the use of multivariate meta-analytic methods for synthesis of data from multiple correlated outcomes as well as for evaluation of surrogate endpoints, in the context of HTA.

Health technology assessment (HTA) agencies such as the National Institute for Health and Care Excellence (NICE) require evidence synthesis of existing studies to inform their decisions. For example; gathering information about the best available treatments with respect to multiple efficacy and safety outcomes. However relevant studies may not provide direct evidence about all the treatments or outcomes of interest. Studies that do not provide direct evidence about a particular outcome or treatment of interest are often excluded from a meta-analysis evaluating that outcome or treatment.

This exclusion is unwelcome, especially if participants are otherwise representative of the population, clinical settings and condition of interest. Research studies require considerable costs and time, and involve precious patient participation, and simply discarding them could be viewed as research waste. Statistical models for multivariate and network meta-analysis address this by simultaneously analysing multiple outcomes and multiple treatments, respectively. This allows more studies to contribute toward each outcome and treatment comparison.

This Technical Support Document (TSD) describes the key methods of multivariate meta-analysis and their extensions to network meta-analysis of multiple outcomes. We focus on the use of multivariate meta-analytic methods for combining data from multiple correlated outcomes with the aim of including all relevant evidence and borrowing of information across outcomes, in particular when not all of the relevant studies report an outcome of interest. We also devote considerable attention to the use of multivariate meta-analysis for the purpose of surrogate endpoint evaluation.

We describe and apply the methods in the Bayesian framework of estimation and provide WinBUGS code for a Bayesian analysis using Markov chain Monte Carlo (MCMC) simulation.

TSD 25

This TSD describes methods for meta-analysis of diagnostic test accuracy, distinguishing between situations where (1) only a single estimate of sensitivity and specificity is available from each study, or (2) some studies report accuracy at multiple explicit diagnostic threshold values. It additionally discusses use of meta-analysis results in decision models, including choice of output to incorporate, identification of the optimal threshold according to some criterion, and the critical role of prevalence in the latter.