This guide to 'holistic evaluation' results from a pilot project within the Student Services Department during 2008-09. The original inspiration for the project was the publication in 2008 of a new US-based research resource on evaluation in a student services context, Assessment Reconsidered: Institutional Effectiveness for Student Success (Richard Keeling, Andrew Wall, Ric Underhile & Gwen Dungy, NASPA 2008): http://assessmentreconsidered.org/.
The core argument in Assessment Reconsidered is that student services professionals should move beyond an approach to service evaluation which is constrained by an operational and rather instrumental focus on process and user satisfaction - also referred to as an 'accountability' model. This is an approach which is very well developed in the Student Services Department. We have around 40 separate mechanisms in place for evaluating our provision: many, though not all, are based on such satisfaction (or accountability) indicators. Our Service Level Agreements reflect similar thinking.
Assessment Reconsidered suggests that we should aim for a broader and more holistic approach to evaluation which considers the impact of our provision. It is a richer, more sophisticated form of evaluation which encourages us to reflect on our purpose. There is an emphasis on reflective practice and evidence and the key evaluative questions become what difference do we make to our students' lives? How do our services help students to develop, to grow, to achieve, to learn, to succeed, etc? The output from such evaluation is likely to be less operational and more strategic. Arguably it enables us to be more proactive in influencing policy, rather than merely to demonstrate levels of usage or customer satisfaction in a somewhat reactive way. This organic model of evaluation encourages us to focus on learning and it is a balanced approach which is closely in tune with the Balanced Scorecard/Strategy Map concepts which underpin our departmental strategy. Holistic evaluation should help us move beyond one-dimensional satisfaction indicators. In short it could be seen to put the 'values' back into evaluation.
The guidance in this 'toolkit' is drawn from the success of four pilots of this broader approach to evaluation which took place in the English Language Teaching Centre, the Disabled Student Support Service, the Counselling Service, and Student Support and Guidance. Guidance is provided on the concept of holistic evaluation (this introductory page); preparing for holistic evaluation; conducting holistic evaluation; and follow up action. Some practical examples - drawn from the pilot projects - are also included. Support for all evaluation activity in the department is available from the Service Improvement Team.
Catherine McKeown, email@example.com
Andrew West, firstname.lastname@example.org
For more information, download the Holistic Evaluation Toolkit.