DTS statisticians’ authors of “highly accessed” papers on pilot trials
DTS statisticians have been working collaboratively and productively in teams to produce three statistical methodological papers on the topic of pilot studies.
One of the most frequently asked questions of a statistician by researchers is “I am designing a study, how many patients do I need?” Research funding bodies are now frequently asking for pilot or feasibility studies to be conducted prior to the main study; to determine whether or not it is actually feasible to carry out the larger main study. Statisticians, in DTS, are now regularly being asked to estimate sample sizes for pilot or feasibility studies. External pilot or feasibility studies can be used to estimate key unknown parameters to inform the design of the definitive randomised controlled trial (RCT). However, there is little consensus on how large pilot studies need to be, and some suggest inflating estimates to adjust for the lack of precision when planning the definitive RCT.
The statisticians in DTS (Dawn Teare, Munyaradzi Dimairo, Neil Shephard, Alex Hayman, Amy Whitehead and Stephen Walters) got together and wrote a paper to try and answer this question. They used a simulation approach to illustrate the sampling distribution of the standard deviation for continuous outcomes and the event rate for binary outcomes.
The authors concluded that an external pilot study should have at least 70 measured subjects (35 per group) when estimating the variability for a continuous outcome. If the event rate in an intervention group needs to be estimated by the pilot then a total of 60 to 100 subjects is required. Hence if the primary outcome is binary a total of at least 120 subjects (60 in each group) may be required in the pilot trial. It is very much more efficient to use a larger pilot study, than to guard against the lack of precision by using inflated estimates. The work was published online in July 2014 in the Trials Journal and has been “highly accessed” since then.
On the related theme of pilot trials another team of statisticians (Ellen C Lee†, Amy L Whitehead†, Richard M Jacques† and Steven A Julious) collaborated on the issue of how to analyse the outcome date from pilot studies; and in particular the statistical interpretation of pilot trials and should significance thresholds be reconsidered?
The paper describes significance thresholds, confidence intervals and surrogate markers in the context of pilot studies and how Bayesian methods can be used in pilot trials.
The authors concluded that in pilot trials the focus should be on descriptive statistics and estimation, using confidence intervals, rather than formal hypothesis testing and that confidence intervals other than 95% confidence intervals, such as 85% or 75%, be used for the estimation.
The work was published online in March 2014 in the Trials Journal and has been “highly accessed” since then.
Pilot and feasibility studies: Is there a difference from each other and from a randomised controlled trial?
On the related theme of pilot trials another team of statisticians (Amy L. Whitehead, Benjamin G.O. Sully, Michael J. Campbell) collaborated on the issue of is there a difference between a pilot and a feasibility study? The terms are frequently interchanged and used synonymously. The team aimed to provide an introduction to the topic area of pilot and feasibility trials and draw together the work of others in the area on defining what is a pilot or feasibility study?
The authors concluded that the term ‘feasibility’ should be used as an overarching term for preliminary studies and the term ‘pilot’ refers to a specific type of study which resembles the intended trial in aspects such as, having a control group and randomisation. However, studies labelled ‘pilot’ should have different aims and objectives to main trials and also should include an intention for future work. Researchers should not use the title ‘pilot’ for a trial which evaluates a treatment effect.