There are a number of advantages in using an on-line system like COSSET to collect students' feedback on teaching.
1. Increase in efficiency in the data collection process.
A paper-based system usually creates a lot of administrative workload. Before administering a questionnaire, a lot of preparatory work has to be done. For example, it involves the printing, packaging and distributing of a large quantity of questionnaire and answer sheets. All these materials have to be returned to a central location for processing, which involves sorting and scanning of the answer sheets. If the questionnaire also contains free-response questions, as does the one used in HKUST, it often proves difficult to process them properly because of the lack of manpower.2. Increase in flexibilityWith a Web-based system, data collection involves only the inputting of student and instructor data into the system. Such data, enormous in size though they may be, are already in electronic format and inputting them into the report system involves merely the conversion of a data file from one format into another. Once the system is set up, students can access the questionnaire using any computer which is connected to the university network at any time they choose within a specified period. Even the answers students make to the free-response questions can be easily captured electronically.
An on-line system with the processing power of its host computer allows much greater flexibility in a number of aspects. It supports complex questionnaire designs and has other benefits.An on-line system also makes sampling of students in a questionnaire survey a viable option. As students are asked more and more frequently to provide feedback on teaching, courses and activities, or participation in research activities, there is a genuine risk of "questionnaire burnout". With an on-line system, it is possible to sample only a fraction of the students in a group for their feedback, instead of doing a full survey. Statistically speaking, such probability sampling has the advantage that the margins of error of the survey results are estimable. At the moment, with the paper-based system, the students doing an evaluation are often a self-selected group. With such a sample, it would bot be possible to estimate the margin of error in the survey results. Another advantage is that students have to do less evaluations. If on average only half of the students in each course are asked to evaluate, then each student on average will only have to evaluate half of the courses he/she attends. With less evaluations to do, hopefully students will spend more time on each evaluation.
- No need to schedule time for teaching evaluation:
Since students can do the evaluation anywhere, anytime, where and when they have access to a computer with connection to the university network, there is no need to schedule a time slot for the course evaluation. At the moment, most evaluations are done in class time.
- Flexibility in questionnaire design:
Like all computer-based testing systems, an on-line system to capture students' responses allows great flexibility in questionnaire design. A practical example is the evaluation of courses with more than one instructor. With a paper-based system, because the questionnaire is of a fixed formats. With an on-line system, questionnaire can always and easily be modified to contain additional sections to evaluate any number of instructors in the course.
An on-line system can also support complex branching. Unlike in paper based systems, respondents will not notice when they are directed to a different set of questions based on their specific answers to some previous questions because the system simply chooses and presents only the next question needing to be answered. In a paper questionnaire, such branching will often confuse the respondents.
In an on-line system questionnaires can also be customized in accordance with the needs of the individual course or instructor. Each course or instructor can have a different set of questions, in addition to the core set.