HAR6062: Evaluation Methods for Complex Settings

Introduction

The course introduces key principles for evaluating complex health and social care interventions, and where and why these methods are useful. It gives broad perspectives while focusing on health programmes and technologies as examples. Participants are introduced to complex situations, complex interventions and relevant innovative evaluation methodologies. Combined methods, for instance including Patient-reported outcome measures (PROMs), are explored.

The course will benefit anyone interested in evaluation, when there is a need to address issues of complexity, which are often overlooked in traditional evaluation methods. Students will apply their learning to the development of an evaluation design in their chosen area.

Objectives

This course aims to introduce the key principles of evaluation for complex interventions and where and why these methods are useful for research in health and social care settings. Teaching activities will give broad perspectives while focusing on programme and technology evaluations as examples. Students will demonstrate learning by developing an evaluation design, in an area of their choosing.

Learning Outcomes

By the end of this course, students should be able to:

  • Describe what constitutes a complex intervention or situation
  • Plan the application of methodologies and principles for evaluating complex interventions
  • Develop theories to guide evaluation
  • Appreciate the key issues for data analysis and presentation of findings
  • Discuss the strengths and limitations of methodologies for evaluating complex interventions
  • Demonstrate awareness of the limitation of traditional evaluation methods including PROMs in complex situations

Teaching Methods

A “blended learning” approach will incorporate directed independent learning and structured group sessions. Seminar sessions (4x 2-hour) will be supported by MOLE on line resources and materials, including 3x half-hour audio recorded lectures with slide shows. Further, students are expected to do preparatory work for (3x 2-hour) tutorials including reading recommended materials and completing preliminary exercises. Students will also take part in 2X 2 hour presentation sessions with group discussion and feedback.

Assessment

Component Weighting
Essay assignment; 2500 - 3000 words 90%
Presentation; Evaluation Study Design 10%

Assessment is carried out through formative and summative assessments.
Formative assessment gives an indication of students’ progress and feedback is provided to students on the accuracy and quality of their work throughout the module. Students’ participation in group work and on-line exercises will be peer assessed to allow them to identify and reflect on their learning needs. A tutor is always present in small group assessment and feedback sessions to ensure that the feedback is focused and to provide technical and subject specific expertise.

Summative evaluation:
Students are expected to give a short presentation approximately half-way through the course (‘An outline proposal for a complex evaluation’). This will contribute towards 10% of their final mark. They will be expected to describe complex elements of the evaluation setting, and plan the application of suitable methodologies for programme theory development, data collection, analysis and dissemination. This will provide a point of focus leading to development of their final proposal.

Students will also submit a 2500-3000 word assignment. The assignment consists of an evaluation proposal in which students should consider methods of data collection, data analysis and underlying theories for evaluating a specific complex health intervention. Headings (similar to those below) will be provided, which align with the learning outcomes and teaching activities:

• What is the setting and scope of the evaluation?
• What are the complex elements of the evaluation?
• Describe the methodology and methods of investigation
• What potential programme theories to guide the investigation?
• How will data analysis and dissemination of findings be carried out?
• Strengths and limitations of the proposal and consideration of alternative methodologies

The minimum pass mark to achieve a satisfactory pass for ScHARR programmes is 50%.