Description and results of the Remote Assessment Systems Pilot - SAR
Registration is now closed.
Motivated by the need to find a solution for student assessment at a time of emergency remote teaching, the FCT FCCN Unit challenges the Higher Education community to participate in a study to help find the preferred solution.
For this purpose the following platforms will be available: ProctorExam, ExamNet, TestWe or Respondus. The teacher has the possibility to perform a set of tests in order to evaluate these Remote Assessment Systems (RAS), and thus identify which best meets the necessary requirements.
Pilot Results
Through the participation of various higher education institutions, with the coordination of a team from the National Scientific Computation Unit (FCCN Unit) of the FCT, it was possible to make a learning path regarding what should be done for the future.
The first and main conclusion drawn is that the remote assessment solutions existing in the market present several implementation difficulties, taking into account the reality and the objectives of Portuguese Higher Education. Besides their high cost (the most advanced platforms in fraud prevention have values that are around 10€ per student per exam), most of the teachers who participated in the study indicate that these solutions are not very effective in fraud prevention.
On the other hand, regarding Data Protection - one of the main issues associated to this theme - the study concluded that the existing remote assessment solutions do not convey the level of confidence, regarding data management, that is required from teachers and Portuguese higher education institutions. One of the main lessons learned from this project is related to the complementary role of Colibri. As some existing platforms do not allow real-time monitoring by video, this FCCN service, which is based on Zoom technology, was used in conjunction by some of the member institutions.
According to the project's conclusions, these combined solutions seem to be sufficient for most of the uses. The integration of Colibri with other tools is well accepted by the teachers, even though it is necessary to find an integrated solution adjusted to the needs of the national HEIs that are not solved by the commercial tools. In this sense, the integration of Colibri in this study allowed us to draw some conclusions about which functionalities are critical for a future assessment platform to be successful.
The possibility of two-way communication (teacher/student and teacher/class), the cost compatible with the financial reality of the HEIs, the recording and analysis of data (audio and video) in a controlled place and with clarified legislation were pointed out as some of the strategic and necessary steps, as well as the eventual integration of the main functions into a single application that can run on various devices.
