Piloto de Sistemas de Avaliação Remota • FCCN

Description and results of the Remote Assessment Systems Pilot – SAR

Registration is now closed.

Motivated by the need to find a solution for student assessment during a time of emergency remote learning, the FCT FCCN Unit is challenging the Higher Education community to participate in a study to help find the preferred solution.

The following platforms will be available for this purpose: ProctorExam, ExamNet, TestWe, or Respondus. Teachers can take a series of tests to evaluate these Remote Assessment Systems (RAS) and identify the one that best meets the necessary requirements.

Pilot Results

Through the participation of several higher education institutions, coordinated by a team from the National Scientific Computing Unit (FCCN Unit) of the FCT, it was possible to create a learning path regarding what should be done in the future.

The first and main conclusion drawn is that the remote assessment solutions currently on the market present several implementation challenges, given the reality and objectives of Portuguese higher education. In addition to their high cost (the most advanced fraud prevention platforms cost around €10 per student per exam), most faculty members who participated in the study consider these solutions to be ineffective in preventing fraud.

Furthermore, regarding Data Protection—one of the main issues associated with this topic—the study concluded that existing remote assessment solutions do not convey the level of trust, when it comes to data management, that is required of Portuguese faculty and higher education institutions. One of the main lessons learned from this project relates to the complementary role of Colibri. Since some existing platforms do not allow for real-time video monitoring, this FCCN service, based on Zoom technology, was used in conjunction with some of the participating institutions.

According to the project's conclusions, these combined solutions appear to be sufficient for most applications. The integration of Colibri with other tools is well-received by faculty, although it is necessary to find an integrated solution tailored to the needs of national HEIs, which are not met by commercial tools. Therefore, the integration of Colibri in this study allowed us to draw some conclusions about the critical functionalities for a future assessment platform to be successful.

The possibility of two-way communication (teacher/student and teacher/class), the cost compatible with the financial reality of HEIs, the recording and analysis of data (audio and video) in a controlled location and with clarified legislation were highlighted as some of the strategic and necessary steps, as well as the eventual integration of the main functions in a single application that can be executed on several devices.

Best Practices for Remote Assessment Download

Remote Assessment Systems Pilot Download

Pioneering platform in assessment tools for e-learning systems.

Integrated into LMS e-learning platforms (Moodle, Blackboard, etc.);

Includes several steps to verify the student's identity and environment;

Use of your own browser to lock screens, applications and keyboard shortcuts;

AI-based recording and detection of suspicious behavior.

Platform managed by University of Aveiro

Fully customizable platform to adapt to all remote assessment requirements.

Identification through Citizen Card;
360º view using a smartphone;
Screen sharing.

Platform managed by University of Lisbon

FAQS

Who can use this Pilot?
All higher education faculty members belonging to RCTS. Check if your institution belongs to RCTS at “RCTS Entity List“

I want to take 5 exams for my 10 classes, can I?
The goal of this Pilot is to collect data from different solutions and different teachers. On platforms with a limited number of licenses, we suggest conducting assessments with a reasonable number of students, but allowing others to participate as well.

Some of the platforms are paid, do I pay anything?
No. The FCT and its partners provide faculty with free access to the platforms. In exchange, they expect them to complete a set of forms so they can objectively carry out their data collection mission to support the subsequent decision on whether or not to scale one or more of these services to the entire academic community.

Do I need to involve my entity in the pilot?
The pilot aims to be straightforward for faculty to facilitate the process and avoid delays, but this does not justify implementing internal rules at your institution. Whenever necessary, validate internally with distance learning services before requesting access to the pilot.

How is access to data controlled?
All platforms have their own terms and conditions, and all are committed to respecting the GDPR.

What platforms are part of the pilot?
ProctorExam; ExamNet; TestWe; Respondus.

Why are partner entities part of the pilot?
The FCT's FCCN Unit wants the community to actively participate in the search for solutions, so it took advantage of initiatives that were already underway and that demonstrated willingness. These entities do not favor others.

I am a teacher at a partner institution. Do I have to use the platform managed by my institution?
No. You can request access to any of the other pilot platforms.

I'm a student, I don't intend to participate, am I obliged?
Teachers should always offer alternatives to remote assessment. Contact your teacher.

The exam was taking place and there was a network problem, what do I do now?
If you fail the exam, you should contact your instructor as soon as possible using the means indicated by the instructor. The solution varies depending on the failure.

Can I still apply for SAR Pilot?

Registration for the SAR pilot is now closed. Faculty who tested the four products (ProctorExam, ExamNet, TestWe, and Respondus) have moved on to a second phase of the pilot to analyze the data obtained.