**3. Impact of the pandemic on student assessment at Universiti Putra Malaysia**

The Malaysian government imposed a Movement Control Order (MCO) in March 2020 as a result of the unprecedented situation. Consequently, the planned academic schedule and all teaching and learning activities in higher education institutions in Malaysia were severely impacted. All face-to-face activities were suspended and were conducted remotely. Medical schools were given the flexibility to amend their teaching and learning activities and assessments based on the guidelines published by the Malaysian Medical Council (MMC) and the Malaysian Qualification Agency (MQA). This was to ensure that all assessments were valid, reliable and fair without compromising the programme's educational objectives.

Universiti Putra Malaysia (UPM) experienced the first online examination with a small number of preclinical students for a remedial examination [11]. This became the inception for the faculty to make further improvements for subsequent examinations with a larger cohort. Due to some limitations of the in-house learning management system (LMS), we used commercially available platforms to deliver the theory and practical examination questions. A mock examination was conducted prior to the actual examination to provide hands-on experience for both faculty members and students. Some amendments and improvements were made based on the findings from the mock examination and feedback from faculty and students.

On the day of the examination, a video conferencing platform was used to proctor the students and conduct OSCE. Earlier, the blueprint for OSCE was revised and only history taking, data interpretation and communication skills were possible to be assessed. In order to facilitate coordination and ensure strict adherence to COVID-19 standard operating procedures (SOPs), all examiners and simulated patients (SPs) were convened in the faculty. They were seated according to their stations, maintaining a suitable physical distance between them. Through the video conferencing platform, the

*Perspective Chapter: Paradigm Shift on Student Assessment Due to COVID-19 Pandemic... DOI: http://dx.doi.org/10.5772/intechopen.109555*

#### **Figure 2.**

*An examiner is observing an examinee interacting with a simulated patient remotely in an online OSCE.*

#### **Figure 3.**

*An example of an OSCE circuit consisting of four stations conducted in breakout rooms in a video-conferencing platform.*

examiners could observe the students communicating with the SPs remotely (**Figure 2**). Online OSCE was possible by transferring the students into multiple breakout rooms which mimic the physical stations in an OSCE circuit. The examiners and SPs remained in the same breakout room and the instructions for students were shared on the

computer screen. The example of an online circuit with four OSCE stations is presented in **Figure 3**. We employed several strategies to prevent cheating attempts which include shuffling the questions and answer options, using a 'lock-down' browser and requesting the students to sign an integrity form before the examination.

Later, we successfully conducted an online examination with several cohorts of students. We also modified the typical synchronous online OSCE into written examination and video OSCE [12]. However, these modifications still have their limitations in assessing the intended clinical skills. For instance, we had to postpone our final year examination because it was not feasible to conduct the clinical examinations for a high-stake examination remotely.

When the government allowed medical students in clinical years to return to campus, the final examination was conducted with tight compliance to COVID-19 SOPs. All staff and students were required to complete their vaccination status to be on campus. There were some amendments made to the examination which include a prolonged duration of the exam and a reduced of examiners for each long case and short case examination. The students also had to sit for only two short cases examination as opposed to the usual three short cases. Several discussions with the Dean, Deputy Dean, coordinators and medical education unit were made to ensure that the validity was not compromised. The amendments were also implemented in accordance with the guidelines made by the MMC and MQA related to medical education procedures during and post-COVID-19.

### **4. Experience at other medical schools in Malaysia**

The outbreak of COVID-19 has changed the perspectives of conducting assessments online and remotely. Medical schools in Malaysia had no choice but to innovate and engage in online methods to conduct the examinations despite being in lockdown. This is to avoid any delay in the student's graduation or progression to the subsequent years.

The assessment method in Universiti Sains Malaysia underwent a major revamp, with the final examination being converted into a continuous or online assessment [13]. Theory examinations and OSCE were conducted online through their e-learning platform [14]. The unexpected circumstances have caused medical lecturers to embrace their e-learning portal more than prior to the pandemic.

Universiti Tunku Abdul Rahman (UTAR), a private medical school conducted their Year 3 MBBS Professional Examination using Google forms for theory examination. They used Microsoft Teams to proctor 49 students during the examination and to run OSCE. The students had to use two devices during the examination; one for assessing the exam questions and the other one placed at the corner back to act as a video camera. To enhance the integrity, the students were required to show that no unauthorised materials were within their vicinity and the examination was recorded and monitored by their Department of Exam. They claimed that the online exam conduct was more efficient and required fewer resources compared to the traditional physical setting [15].

Meanwhile, Universiti Malaysia Sabah used their LMS system to conduct their theory examination online through Google Meet. They convert all methods of theory examinations into MCQs as it was more feasible and practical. Similar to UTAR, the students used two devices during the examination. Meanwhile, the clinical examinations were conducted asynchronously. The students had to record videos while

*Perspective Chapter: Paradigm Shift on Student Assessment Due to COVID-19 Pandemic... DOI: http://dx.doi.org/10.5772/intechopen.109555*

performing procedures and examinations on family members, friends or manikins. They were given feedback and were allowed to resubmit another video until their performance was deemed satisfactory. The online examinations were only done on the cohorts involved in low stake examinations and they would have opportunities to improve themselves in the remaining years before graduation [16].

The traditional long case and short case examinations were converted into physical OSCE in the Final Professional Examination at Universiti Sains Islam Malaysia. The examination was conducted in strict adherence to COVID-19 SOPs. It was their first time conducting OSCE in the high-stake examination, however, they managed to train the faculty, students and all personnel involved as well as recruit adequate simulated patients within the constraint. Despite mixed reactions from the faculty and students, the faculty has decided to resume with OSCE in all clinical examinations from there on after careful deliberation of the benefits of OSCE [17].

### **5. Challenges of online assessment**

The adoption of online assessment is not without obstacles. Internet connection has been a major concern among the faculty and students [11, 15, 16]. A synchronous online examination could be a challenge, especially for students living in rural areas. Some faculties did a mock examination to test logistic capabilities. Students who discovered that their internet connection or hardware was not adequate for the exams had ample time to make required adjustments before the actual examination. These include upgrading their hardware or taking the examination in other places with more stable internet connections. Extra time can be given to the students who have unstable internet connections to complete the examination.

The unusual feeling of taking an important examination remotely may cause the students to develop anxiety. In a study in China, medical students felt anxiety during an online exam due to their poor capacity to adapt to the online platform and worries regarding the fairness of the online assessments [18]. This also occurred at UPM [11]. The pressure of having to familiarise themselves with online platforms might be burdening and cause emotional distress, not just to the students but also to the educators [13]. Administrative support is very crucial to ensure a smooth implementation of the online assessment.

Another challenge was to ensure the constructive alignment of the online assessment. The assessment of affective and psychomotor domains might be difficult to conduct [12, 13]. Assessments related to the 'shows how' and 'does' in Miller's pyramid such as long case examination and workplace-based assessment with patients are not possible in an online environment. It can also be tough to assess clinical skills and professional attributes such as empathy and teamwork remotely. To assess physical examination skills, faculties might need to be creative either by using simulated patients [15] or manikin. History-taking and communication skills were assessable either synchronously or asynchronously [11, 12], however, other performance tests are difficult to conduct without introducing threats to the validity of the assessment [1, 19].

### **6. Strategies to conduct online assessment**

Given the technological advancement we have made in adapting to the pandemic, medical schools may consider remaining to adapt the innovative methods as long as

they are valid and reliable. Several strategies can be taken to conduct online synchronous and asynchronous examinations, especially if it is conducted for the first time. These can be divided into pre-, during and post-assessment (**Figure 4**).

#### **6.1 Pre-examination**

#### *6.1.1 Ensure adequate facilities and human resources*

The most important factor for a successful online assessment is to identify whether the institution has sufficient infrastructure to run the online assessment. Facilities such as computers with stable internet connection are essential. The computers must be compatible with all online software and platforms needed for the assessment. Additional accessories such as a microphone, camera, speaker and headphones are required to enable two-way communication. The internet connection must have adequate bandwidth to support the transmission of data between faculty and all students for the whole assessment process.

The LMS system subscribed by the institutions usually may be used to deliver the assessment. Some examples include Moodle, Canvas and Blackboard can be used to develop student assessments, obtain performance data and provide feedback to students [20]. Additionally, numerous commercial online examination platforms can be subscribed to by the institution to be used for synchronous and asynchronous examinations. Some of these platforms have additional features such as a question bank besides being able to perform item analysis and alignment of assessment to the course learning outcomes that can facilitate further administrative work.

Some practical considerations should be given for the online examination platform. For example, the platform should be secure and dependable and can cater to the faculty and students. To ensure security and prevent any dishonesty attempt, some of these platforms are equipped with a remote proctoring system to confirm the examinee's identity and monitor behaviour while taking the examination. For tests consisting of MCQs, most of the platforms allow automated scoring while manual marking can also be done for open-ended questions. However, some platforms can only accommodate written-based examination, but not performance-based examination. OSCE for example may be conducted through a video conference platform such as Microsoft Teams and Zoom application by creating multiple break-out rooms for individual OSCE stations. However, other assessments that involve actual patients and require examinees to perform clinical skills such as physical examination and clinical procedures can be quite tricky. This may not be feasible and still need to be done physically in a proper clinical setting.

Paperless examination using an online platform is more environmental-friendly and can be cost-effective in the long run. The institution is able to save many resources in terms of effort, money and time [21]. Automated analysis can be done instantly and this enhances instant and timely feedback for the benefit of faculty and students. A high volume of assessment data and documentation can be managed efficiently with a proper and stable online examination platform.

However, the institution needs to ensure there are adequate technical experts to manage and support the whole assessment system. This is essential to prevent interference, especially while the examination is running. Detailed guidelines and SOPs can be prepared to be used as a reference. The faculty must also be familiar with the navigation of the online platform used for the examination. Academics need to be well informed on how to set up an assessment using the platform, how to *Perspective Chapter: Paradigm Shift on Student Assessment Due to COVID-19 Pandemic... DOI: http://dx.doi.org/10.5772/intechopen.109555*

#### **Figure 4.**

*A flowchart of several strategies to conduct an online assessment.*

insert questions and the answer scheme, how to retrieve answer scripts and grade the students and how to provide feedback if needed. Maintaining the security of the exam questions and students' information needs to be paid great attention to. If a video conferencing platform is used, faculty members need to know how to admit students, invigilate students, set up breakout rooms and use many other features in the platform. Adequate training is essential to familiarise the faculty with the workflow and navigation of the online platform. It is crucial for the faculty to be well-prepared and have enough confidence to conduct an online examination.

#### *6.1.2 Identify assessment blueprint*

An institution must ensure that the assessment methods are aligned with the intended learning outcomes. Converting to an online examination for a written assessment is highly possible with an appropriate online platform. As a matter of fact, exams with MCQs can be more efficient and cost-effective in an online setting [15].

However, remote or online assessment of some clinical skills may not be feasible due to limited standardisation and inadequate resources. The majority of clinical skills that require the presence of patients are not possible to be assessed remotely. Therefore, a clear and detailed blueprint should be prepared to guarantee that all learning outcomes are covered. An example of a blueprint for Orthopaedic course is shown in **Table 1**.

As shown in the table, written assessment is conducted online while the assessment of performance is maintained as an in-person examination. It is also indicated whether the online assessment is done synchronously or asynchronously. This is flexible for any institution depending on their feasibility and practicality. The conduct


#### **Table 1.**

*An example of an assessment blueprint for Orthopaedic course.*

of asynchronous examinations is also flexible. For instance, students are given some clinical problems and are required to submit an assignment in the form of essays about the diagnosis, investigations and treatments within an allotted duration.

#### *6.1.3 Explore student situations and needs*

One advantage of conducting an online assessment is that students do not have to be present on campus and can sit for the examination remotely. However, we need to consider various factors that may discriminate the student's capacity for an online assessment. An institution cannot ignore the fact that students living in rural areas may have limited access to a seamless internet connection. Some students may not have the necessary resources such as a personal computer or laptop. It is essential to put these considerations in the first place, especially before implementing an online examination.

The faculty should first gather feedback from students if they decide to convert to online examination. Explore their needs and capacity. If an internet connection is an issue, faculty may conduct an asynchronous examination with a reasonable time to complete it. This allows some flexibility for students to complete the examination at their convenience. An open-book examination can be carried out in the asynchronous method. In this scenario, the questions or assignments should be assessing higher-order thinking skills that answers can not just be directly obtained from any references. If a synchronous online examination is deemed necessary, facilities should be made available for students without a proper device or access to a stable internet connection. An example would be allowing the students to sit for the examination in the computer lab in the faculty. However, invigilators must be present similar to physical examination to prevent discussion among the students.

Students should be briefed about the conduct of online examinations and the expectations. A mock examination or a trial run is obligatory before running the real online examination. It is encouraged to execute the mock similar to the actual exam in terms of the method of assessment, duration and time. This is to provide a hands-on experience to students and faculty and identify any difficulties during the examination. For example, we noted that the long duration of the examination was physically and mentally challenging, so the actual exam schedule was amended by breaking down the exam into multiple parts with breaks in between [11]. There should be an adequate period between the mock and actual examinations so that proper interventions can be taken by both faculty and students.

Some students may also have limited capabilities concerning digital use such as keyboard typing skills and familiarity with certain computer applications. An additional time for the examination can be considered, especially if it involves many multimedia such as images and videos. Practice questions can also be given to students for them to familiarise themselves with the online platform prior to the actual examination.

#### **6.2 During examination**

#### *6.2.1 Ensure technical support is on standby*

It is important to expect the unexpected worst-case scenario on the day of the examination, particularly for a synchronous examination. Therefore, we need to ensure there is adequate support and assistance available if needed. For example, if there is network interference occurs on the examinees or faculty's side, there is an SOP on how to handle the situation. In a synchronous examination, the faculty needs to prepare an alternative medium for examinees to immediately inform if they face any glitches to get online, such as by a telephone call. In certain cases, examinees may suddenly face internet or device problems while taking the exams. *What kind of intervention should be done? Do we allow extra time given for them to complete the examination? How long should it be?* It is also recommended to prepare an extra set of exam questions beforehand in case there are students who do not manage to complete the examination due to some circumstances.

There is also a possibility that the faculty encounters technical problems. For instance, there is an electricity outage in the institution or the online platform cannot be accessed for whatever reasons. *Do we have a backup platform? Do we need to postpone the examination?* This needs to be considered. A backup platform or devices can be arranged earlier in case it happens. An extra set of exam questions is useful if these problems occur when the examinees have already been exposed to the questions.

#### *6.2.2 Maintain high standards of examination security*

There is much evidence showing that cheating and academic dishonesty are common in online assessments [22, 23]. This can be a threat to the validity of the assessment. Given the availability of technologies like Bluetooth, wireless networking, mobile phones and wearable technology, preventing cheating during online exams might be challenging. Tech-savvy students will always come up with new ways to cheat. If the examination involves a large number of students at a time, there is always a possibility that they may communicate with each other.

Some strategies can be employed to deter cheating, particularly for a synchronous examination. Using a remote proctoring system, that is able to confirm student identity, and track eye movements, keystrokes and background noises are convenient to recognise potential cheating attempts. Students need to download specific software onto their computers which can allow a third-party service provider to monitor or record their webcam, microphone and desktop feeds completing the examination. However, this software can be quite expensive to subscribe to. Some institutions may have reservations about online proctoring systems due to worries about privacy and ethical issues [24]. An alternative is to have human invigilators to monitor students' behaviour in real-time through a video conferencing platform. Yet, to invigilate a large number of students, for example 100 students, in one setting can be difficult. Therefore, many invigilators are needed by breaking down the students into smaller groups for easier observation.

For assessments involving MCQs, the questions and answer options can be shuffled so that each student's paper would have a distinct order of questions. With this tactic, it can be challenging for them to exchange information and share answers within the allotted exam time. In a closed-book exam, examinees may also have the opportunity to access other applications or search online references while answering the questions. There are many software available to lock down the browser to discourage this attempt. Some online examination platforms or LMS are already been embedded with these enhanced security features to provide a secure test environment. Another solution is by requesting the students to use two devices while taking the examination. One device is their computer, to access the exam questions and the second device should be with a camera, such as another computer, tablet or smartphone to be placed behind them. Hence, we can have an almost 360-degree view

*Perspective Chapter: Paradigm Shift on Student Assessment Due to COVID-19 Pandemic... DOI: http://dx.doi.org/10.5772/intechopen.109555*

and ensure that no other references are available within their surroundings or on the computer screen.

An integrity agreement that students need to read and sign can be distributed prior to the examination. This agreement includes statements that students shall conduct themselves honestly and ethically in the exam. They also need to be briefed and frequently reminded of the consequences of violating the policy.

In an asynchronous and open-book exam, there is also a possibility that students may copy the work of others. There is commercially available cost-effective software that can be used to detect any duplication and plagiarism, such as Turnitin and Dupli Checker. The software can give an instant and comprehensive report to confirm whether the student's work is authentic by comparing it with billions of resources across the internet.

#### **6.3 Post-examination**

#### *6.3.1 Ensure accurate scoring and reporting*

The use of an online assessment platform can significantly boost the efficiency of administrative work like marking, gathering and organising data. Automation can reduce the workload and burden of faculty members when testing large student cohorts. However, human error cannot be totally eliminated. Generally, faculty needs to check and verify that the scoring is correct prior to reporting. Having cloud storage or backup folders to save all information is recommended just in case the platform crashes or fails to work.

#### *6.3.2 Evaluate and improve based on feedback*

The transition from a traditional pen-and-paper examination to an online assessment may emotionally affect the faculty and students. Hesitancy and anxiety in dealing with the online environment may be observed in a few cases. Therefore, allowing two-way feedback is crucial for building trust and confidence. Frequent training should be conducted to enhance proficiency with the online system, especially for a new cohort of students.

For any kind of assessment, regular monitoring and evaluation should be done to identify any areas for improvement. An institution should invest their effort and time to analyse successes and failures in the assessment conduct. Any problems detected during the examination should be recorded and troubleshot. It is also important to ensure that the online platform is stable and well-maintained to avoid interference for subsequent assessments. Further enhancements can be made to smoothen the assessment process and to be more convenient and cost-effective for both students and faculty.

### **7. Conclusion**

The COVID-19 pandemic has forced a rapid transition from traditional pen-andpaper examination to alternative student assessment methods which encompasses the use of a variety of digital platforms. We can observe that many examples of successful innovative online assessment conduct have been done in Malaysian medical schools, and globally. They have achieved favourable results in terms of validity, reliability and acceptability from both faculty and students. Such efforts serve as a stepping stone for medical education reforms that previously might be proposed but never completely materialised.

Online assessment is no longer impossible, albeit the limitation on assessing clinical performance. Without a doubt, we must not overlook the fact that not all faculty and students have equal access to technology and digital competency. Nevertheless, it is time for us to use technology to create a better educational experience and find creative ways to increase equity. Despite all the challenges, we need to remain vigilant and may take appropriate measures to minimise complications. Above all, we need to ensure that the assessment conducted is valid, reliable, feasible, secure and fair.
