ONG Pei Shi1, *, and SOO Yuen Jien2,3
1Department of Pharmacy and Pharmaceutical Sciences, Faculty of Science (FoS), National University of Singapore (NUS)
2Department of Computer Science, School of Computing (SOC), NUS
3Centre For Teaching, Learning and Technology (CTLT), NUS
Sub-Theme
Building Learning Relationships
Keywords
Technology-enhanced learning, autograder, peer assessment, feedback, student learning
Category
Paper Presentation
Background
Peer assessment is an approach where students are tasked to evaluate and provide feedback on the work of other students (Topping, 1998; Duchy et al., 1999). When performed effectively, peer assessment improves students’ engagement and understanding of the subject matter while allowing them to hone their critical thinking skills (Topping, 1998; Morris et al., 2001; Li et al., 2010; Topping, 2017). This strategy has been utilised in the teaching of healthcare professional students with positive outcomes (Nofziger et al., 2010; Theising et al., 2014; Ramm et al., 2015; Lerchenfeldt et al., 2019). Peer assessment has likewise been utilised for the learning of Pharmaceutical Compounding (PC) in National University of Singapore (NUS). Here, students peer assesses their classmates’ handwritten compounding exercises, where they would provide written feedback on errors made on their classmates’ worksheets. This process is termed conventional peer assessment. This would be followed by instructor-led small group feedback using errors identified by students to close the learning loop. Despite the purported advantages of peer assessment, a preliminary survey of students performing conventional peer assessment for PC showed that this approach was not optimal, as students were unable to capture all errors made by their peers to allow thorough feedback on all errors by instructors. Therefore, we hypothesized that the use of technology can improve peer assessment and instructor-led feedback for learning of PC.
In this study, we built an in-house digital platform cum autograder, named the Pharmacy Peer Assessment and Feedback System (PECS), which allowed students to log in their answers for each exercise digitally and peers to evaluate their classmates’ submissions and provide feedback on the errors identified for each submission into PECS. Concurrently, the autograder within PECS will also identify errors for each submission, which can be compared against errors identified during the peer assessment. We further evaluated students’ and instructors’ perceptions of the usefulness of PECS to improve peer assessment and feedback provision for learning of PC. We also quantified and compared the errors captured by the peer assessor or those captured by the autograder versus those captured by an expert grader to further substantiate the usefulness of PECS as a tool to enhance peer assessment, feedback provision, and student learning.
Methodology
A total of 316 students and seven instructors were recruited. A cross-over study design was adopted for two compounding exercises. For each exercise, half of the students were assigned to conventional peer assessment, while the other half were assigned to perform peer assessment using PECS. Students who utilised conventional peer assessment for the first exercise were assigned to perform peer assessment using PECS for the second exercise via versa. For each exercise, following the completion of either conventional peer assessment or peer assessment using PECS, instructors performed small group feedback on students’ work using errors identified by the peers without or with those identified by the autograder, respectively. Students’ and instructors’ perceptions of PECS as a learning tool to improve the effectiveness of peer assessment and instructors’ feedback were surveyed and described using descriptive statistics. Errors captured by the autograder or those captured by a peer assessor versus those captured by an expert grader were quantified and analysed using non-parametric statistics.
Results
Majority of students (>65%) and instructors (>71.0%) either strongly agree or agree that PECS possessed good ability to identify errors in different sections of the exercises (Figure 1).
The autograder function in PECS captured errors identified by the expert grader with 66.7% accuracy compared to the peer assessor with only 12.5% accuracy (p<0.001) (Figure 2), confirming that PECS was superior in capturing errors made by students compared to the peer assessor.
(*p<0.001 using Mann Whitney U test for comparison between autograder versus peer assessor)
Most students also either strongly agree or agree that PECS improved the quality of their learning (78.5%), their engagement in learning (77.2%), the effectiveness of peer assessment (82.9%), and allowed them to better learn from their peers’ errors (90.5%) (Table 1).
Students’ response (n=316) on the usefulness of PECS as a tool for learning
| Question | Strongly Agree or Agree (%) | Strongly Disagree or Disagree (%) |
| PECS has improved the quality of my learning | 78.5 | 21.5 |
| The use of PECS made me more engaged in my learning during peer assessment | 77.2 | 22.8 |
| The use of PECS improved the effectiveness of peer assessment | 82.9 | 17.1 |
| The use of PECS allowed me to better learn from my peers’ errors | 90.5 | 9.5 |
Similarly, 100% of instructors either strongly agree or agree that PECS improved student learning and their effectiveness at performing peer assessment (Table 2).
Instructors’ response (n=7) on the usefulness of PECS as a tool for learning
| Question | Strongly Agree or Agree (%) | Strongly Disagree or Disagree (%) |
| PECS has improved the quality of students’ learning | 100.0 | 0 |
| The use of PECS improved the effectiveness of students’ ability to peer assess each other’s work | 100.0 | 0 |
Seventy-six percent of students either strongly agree or agree that the use of PECS improved the quality of feedback received, while 85.7% of instructors either strongly agree or agree that the use of PECS improved the quality of feedback given (Figure 3).
More than 70% of students and instructors further agreed that PECS improved the interaction between students and instructors during the instructor-led feedback session (Figure 4). All instructors agreed that PECS is a useful teaching aid.
Conclusion and Significance
This study successfully showed that the use of technology, such as PECS, enabled better capture of students’ errors versus conventional peer assessment. This, in turn, facilitated the peer assessment process by allowing students to learn from errors not captured by their peers. It further created a more conducive learning environment for feedback provision, where instructors can provide more thorough feedback on both errors captured by peers and PECS. This consecutively led to greater interaction between students and instructors. Overall, the use of PECS enhanced peer assessment, improved feedback provision, and student learning of PC.
References
Duchy, F., Segers, M., & Sluijsmans, D. (1999). The use of self-, peer and co-assessment in higher education: A review. Studies in Higher Education, 24(3), 331–350. https://doi.org/10.1080/03075079912331379935
Li, L., Liu, X., & Steckelberg, A. L. (2010). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525–536. https://doi.org/10.1111/j.1467-8535.2009.00968.x
Lerchenfeldt, S., Mi, M., & Eng, M. (2019). The utilization of peer feedback during collaborative learning in undergraduate medical education: A systematic review. BMC Medical Education, 19, 321-310. https://doi.org/10.1186/s12909-019-1755-z
Morris, J. (2001). Peer assessment: A missing link between teaching and learning? A review of the literature. Nurse Education Today, 21(7), 507-515. https://doi.org/10.1054/nedt.2001.0661
Nofziger, A. C., Naumburg, E. H., Davis, B. J., Mooney, C. J., & Epstein, R. M. (2010). Impact of peer assessment on the professional development of medical students: A qualitative study. Academic Medicine, 85(1): 140-147. https://doi.org/10.1097/ACM.0b013e3181c47a5b
Ramm, D., Thomson, A., & Jackson, A. (2015). Learning clinical skills in the simulation suite: The lived experiences of student nurses involved in peer teaching and peer assessment. Nurse Education Today, 35(6), 823-827. https://doi.org/10.1016/j.nedt.2015.01.023
Theising, K., Wu, K., & Sheehan, A. H. (2014). Impact of peer assessment on student pharmacists’ behaviors and self-confidence. Currents in Pharmacy Teaching and Learning, 6(1), 10-14. https://doi.org/10.1016/j.cptl.2013.09.020
Topping, K. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68(3), 249–276. https://doi.org/10.3102/00346543068003249
Topping, K. (2017). Peer assessment: Learning by judging and discussing the work of other learners. Interdisciplinary Education and Psychology, 1(1), 1–17. https://doi.org/10.31532/InterdiscipEducPsychol.1.1.007