Observations

Types of observation

Observations can be made of many aspects of change in learning and teaching (L&T), e.g. of learning and teaching activities (lectures, seminars, lab classes), documents and other teaching material presented to students, learning resources, learning environments, interactions between participants.
  • Highly structured observations will consist of a checklist of e.g. the incidence, presence, or frequency of predetermined evidence to be observed in the situation which will either support or refute a preconceived theory.
  • Semi-structured and unstructured observations allow for issues to emerge from the observation, although they may be semi-structured around issues considered to be relevant to the evaluation.


Why use observational methods?

  • To provide contextual information needed to frame the evaluation and make sense of data collected using other methods.
  • To develop insight into the L&T context, the environment, events, activities, interactions, language used etc. This might point to issues requiring further exploration using other methods.
  • To collect information about how a change in L&T has been implemented, independently of participant perceptions.
  • To learn about sensitive issues that participants may be unwilling to talk about.


Using observational methods

  • Your selection of activities, resources, or documents to be observed will depend on why you are using observational methods, your evaluation questions, how the data is to be used with other methods, and the extent to which you have access to the context of the evaluation.
  • In the case of a structured or semi-structured observation, it can help to draft beforehand a checklist to guide observations you will need to record.
  • When you have considered how and when you would like to make observations you may have to negotiate what is feasible with others, depending on the extent to which you have control over access to the context of the evaluation.
  • Making sense of observed and perceived outcomes, and how these have come about, involves comparing what is observed with the original plans for change. Noting what does not happen in relation to what was planned can be as important as what does happen.
  • Analysis of documents and materials may provide access to relevant contextual information. If large amounts of similar documents or other textual information are to be observed (e.g. discussion postings in a virtual learning environment), a content analysis approach might be appropriate.

Strengths

  • Observations can provide good insights into how the different participants are behaving and interacting.
  • Observations may enable you to see things that are taken for granted by participants in the learning and teaching context. Their perceived lack of importance by participants may mean that they would not be picked up by other methods that explore participant perceptions.

Limitations

  • Observations can be time consuming. Getting a representative picture of the implementation over the duration of a pilot or embedding phase of a change in learning and teaching will involve attending more than one learning and teaching activity or event.
  • Observation of activity may affect the behaviour of those involved in it and hence what you observe. Participants may be concerned about what you are actually evaluating. Academic staff may be concerned the quality of their teaching is being evaluated and students may be concerned their academic performance is being assessed.
  • The thinking that underlies participants’ observed actions cannot be observed. Observations are therefore used with other methods that seek insight into this thinking.
  • Being able to make sense of the context of evaluation in a limited amount of time with limited resources may require some knowledge of the academic discipline and its culture.


Practicalities

  • Minimising the impact of evaluation activity involves making a judgement about the extent to which your observations are likely to influence the behaviour and experience of those you are observing and develop strategies to address this. Explaining the purpose and nature of the evaluation activity and seeking permission from participants to be present in any classes you would not normally participate in is one measure you could take.
  • Video recording, if not too intrusive, may provide a good record of the physical environment and the activities and interactions taking place in it.
  • When you come to make sense of your notes of observations it will help if you have differentiated between description and interpretation.
  • Recording your thoughts and feelings about your experiences and observations will help you be reflexive about how your presence may be influencing the behaviour of those you are observing.


Further reading

Cohen, L., Manion, L. and Morrison, K. (2000) Research Methods in Education 5th Edition. London: RoutledgeFalmer. See Chapter 17 pp 306 - 316. [Available from University Main Library.]