Electronic questionnaires

Using group response technology

Why use electronic questionnaires?

To distribute questionnaires more quickly than using a postal service.

To eliminate the costs associated with printing and distributing paper-based questionnaires (does not apply to OMR forms/cards).

To collect data in electronic format, reducing time and costs required for data processing.

To increase student confidence in the anonymity of their responses (in relation to distributing papr-based questionnaires in class-time, handwriting recognition etc.)However, not all electronic methods guarantee anonymity.

In the case of group response technology, to receive instant feedback.

Email

Questions are either sent in the body of an email or as file attachments to the email in Word or other electronic file format. If the questions are set within the body of the email respondents receiving it need only select the ‘reply’ function and add their answers to questions before sending. Attachments require that respondents open the attached questionnaire and save it with their completed responses to their local file storage. They then create a reply to the email and attach the completed questionnaire before sending. The response will not be anonymous, although some element of anonymity can be introduced if a different person is responsible for processing the incoming data prior to analysis, so that the person analysing the data does not have access to the original email source. It is also advisable to save text-based questionnaires as Rich Text Format (.rtf) files for transfer. This file format provides for easier transfer of formatted documents between users that may be using different computer operating systems and word processing applications.

This approach may be appropriate if you have a good knowledge that your intended recipients will have an appropriate application to open, edit and save your questionnaire, and sufficient ability and confidence in the use of technology to do this. If you are unsure of the technical capability of all your students, then include instructions on how to do this in the body of the email.



Web forms

The questionnaire is created using HTML (Hyper Text Mark-up Language) and made available to potential respondents via a web server. When the respondent has completed the questionnaire they ‘submit’ the web form back to the web server. The web server processes the data by executing programs created as part of the web form for either sending data to a designated email address, or inserting it into a database created specially for the purpose of storing and retrieving the data.

It is also possible to write web forms for University web pages using cPanel, and in a more limited way in the University's content management system (CMS). Explanation of how to do this can be obtained from the University's cPanel and CMS support pages.

Some websites, such as SurveyMonkey.com, enable you to easily create your own questionnaires, although the format may be fairly limited.

Optical mark readable forms or cards

Questions are printed either separately to the OMR form, or on the OMR form itself. The OMR form has predefined spaces for answers to multiple choice questions, and the respondent’s selection is made with appropriate shading in dark pencil or pen. The forms are fed through a reader which notes the positions of the answer marks in relation to pre-printed marks on the forms. The OMR software application adds the data captured to a database and processes it for analysis, e.g. by producing frequency counts and charts for the distribution of responses for each question.

Questions requiring free text responses will need to be asked using a supplementary questionnaire, probably paper-based so that it can be completed in the same class time.

A service for OMR forms is provided by Learning and Teaching Services.
Automated student course evaluation

Group response technology

Group response handset
Software enables the generation of questions that can projected onto a screen. Each of the respondents in the group has a hand-held electronic device which allows them to select an immediate response from a range of choices within a limited time period (normally seconds). The distribution of the responses is automatically processed by software and the results can be immediately projected onto the screen for the whole group to see. Questions and responses are stored in a database for later retrieval and analysis.


The technology may raise expectations amongst respondents that they will be able to see the distribution of the responses immediately. This raises a number of issues that may need to be carefully managed during the process. For example, your questions may yield unexpected response distributions that highlight sensitive issues you may be unequipped to respond to at the time. The trend in responses may influence how some respondents answer subsequent questions, as they may not want to feel ‘out of step’ with the general body of opinion. It might be advisable to be prepared with strategies to cope with these eventualities. Examples might include saving feedback to the group until all questions have been answered or providing a short open format questionnaire that students can use to give fuller written responses to explain their answers. This might be particularly useful when unexpected response distributions highlight sensitive issues. It will also help if you are in a position to explain how feedback will be handled at the course or departmental level, and how and when students will be informed of any action that will be taken as a consequence of their feedback.

A portable system is available within the University. The hardware and technical support is booked through Audio Visual Service. A quick tutorial guide to the use of this system has been produced by LeTS (see link at the top of this page).

Synthesis XPiq– Electronic Group Response System: Quick Start Tutorial

Strengths and limitations

Technology Strengths Limitations
All Data can be processed much more quickly, reducing the time taken to inform all interested parties of the outcome, agree action to make improvements, and to take this action.

Less paper is generated, reducing storage needs and waste.

Many of the techniques require little manual intervention in transferring data collected to a suitable application for data processing.
All potential respondents must have access to the appropriate technology and appropriate technical skills to use it, although the requirements are not excessively demanding.

Many of the approaches are quite limiting with respect to the formatting of questions and the structure of the questionnaire.

It can be difficult to motivate students to return electronic questionnaires. Unlike paper-based questionnaires, completion cannot be easily incorporated into a slot at the end of class time and supervised by a member of staff. Instead specific strategies for encouraging completion need to be employed.
Email
- Email respondents are not guaranteed anonymity.

Heavily formatted questionnaires attached to emails can create large file sizes which might be fairly slow to download for modem users. Questionnaires created by one application (e.g. Word) may not be able to be opened and edited by users without the same application. Those that are able to open the file, in addition to adding their own responses, are also able to edit the content and format of the original questionnaires.
Web forms
Web-based questionnaires can be made to look more visually appealing, and the data generated is in a predictable format.
Web-based questionnaires may require more technical expertise to create them, the cost of which may not be justifiable for the relatively small number of students to be included in a survey in the context of a single course or new curriculum development.
OMR forms
- OMR forms can be quite costly to design and print and initial investment is also needed in equipment and software. These costs may be justified by the savings made in data processing if large numbers are involved, but this approach is not really cost-effective if a questionnaire is to be used only once in the context of evaluating a change in learning and teaching.
Group response technology
- Presentation of the distribution of responses as the questionnaire proceeds may influence future responses, as individuals do not want to feel ‘out of step’ with the group.
MOLE - Data exported from MOLE is not always exported in a format that can be immediately processed by receiving applications. Data may need to be reformatted or converted in some way.