Description and results of the Remote Assessment Systems Pilot - SAR
Registration is now closed.
Motivated by the need to find a solution for student assessment at a time of emergency remote teaching, the FCT's FCCN Unit is challenging the higher education community to take part in a study to help find the preferred solution.
For this purpose the following platforms will be available: ProctorExam, ExamNet, TestWe or Respondus. The teacher has the possibility to perform a set of tests in order to evaluate these Remote Assessment Systems (RAS), and thus identify which best meets the necessary requirements.
Through the participation of various higher education institutions, with the coordination of a team from the FCT's National Scientific Computing Unit (Unidade FCCN), it was possible to take a learning path as to what should be done for the future.
The first and main conclusion drawn is that the remote assessment solutions existing in the market present several implementation difficulties, taking into account the reality and the objectives of Portuguese Higher Education. Besides their high cost (the most advanced platforms in fraud prevention have values that are around 10€ per student per exam), most of the teachers who participated in the study indicate that these solutions are not very effective in fraud prevention.
On the other hand, with regard to Data Protection - one of the main issues associated with this topic - the study concluded that the existing remote assessment solutions do not convey the level of trust, in terms of data management, that is required of Portuguese teachers and higher education institutions. One of the main lessons learned from this project relates to Colibri's complementary role. As some existing platforms do not allow real-time video monitoring, this service from FCCN, which is based on Zoom technology, was used in conjunction by some of the member institutions.
According to the project's conclusions, these combined solutions seem to be sufficient for most of the uses. The integration of Colibri with other tools is well accepted by the teachers, even though it is necessary to find an integrated solution adjusted to the needs of the national HEIs that are not solved by the commercial tools. In this sense, the integration of Colibri in this study allowed us to draw some conclusions about which functionalities are critical for a future assessment platform to be successful.
The possibility of two-way communication (teacher/student and teacher/class), the cost compatible with the financial reality of the HEIs, the recording and analysis of data (audio and video) in a controlled place and with clarified legislation were pointed out as some of the strategic and necessary steps, as well as the eventual integration of the main functions into a single application that can run on various devices.