Sub-Theme
Building Technological and Community Relationships
Keywords
AI Technology in education, ChatGPT–assisted learning, Java programming,
Category
Lightning Talks
Background
In recent years, Artificial Intelligence (AI) has been increasingly used in education, particularly in programming studies (Jin & Kim, 2023; Haindl & Weinberger, 2024; Vukojičić & Krstić, 2023). AI-driven tools such as ChatGPT (OpenAI, 2023) have provided an enhancing learning experience for learners as they can retrieve detailed explanations and examples for various programming concepts through chat (Ouh et al., 2023). AI-based techniques are able to provide diverse and dynamic learning materials for learners based on learners’ characteristics, such as their level of programing proficiency (Ghimire & Edwards, 2024; Mikac et al., 2024). However, the effectiveness of using AI tools such as ChatGPT to help students, especially those without any coding experience is unknown. This study aims to explore the potential benefits of ChatGPT for coding beginners in a Java programming course.
Purpose
The main purpose of this paper is: (1) Develop a web application that integrates the ChatGPT API to generate dynamic and personalised contents for a Java fundamental programing course. (2) Provide complementary learning materials for coding beginners, including stage-based learning assessment quizzes and personalised e-Learning recommendations.
Research Questions
(1) What are the potential benefits of ChatGPT in assisting Java programming study, particularly for students with no prior coding experience?
Method
Design and develop a ChatGPT-assisted eLearning web application that leverages the power of ChatGPT to generate dynamic learning materials, including topic slides, exercises and quizzes. Implement an experiment to evaluate the effectiveness of ChatGPT-assisted learning in Java foundation course. A total of 27 students from the Graduate Diploma in System Analysis are involved in this study. We propose a methodology that compares student performance with and without the ChatGPT-assisted eLearning application. This study follows a controlled experimental design, dividing students into two groups: Group A (students without coding background, who will go through the course with the ChatGPT-assisted eLearning application) and Group B (students with coding experience, who will study the course without the ChatGPT-assisted application). In the end, both groups will take the final quiz to evaluate students’ performance.
Key Findings
The pre-course survey shows that 68% of students are classified as Group A while 32% fall into Group B. The p-value of the Hypothesis test is 1.83 × 10⁻⁵, which is extremely smaller than the significance level of 0.05. This indicates that Group A and Group B are significantly different. The results of the final quiz are shown in Table 1 with the p-value of 0.573. It is greater than the significance level of 0.05. This indicates that there is no significant difference between Group A and Group B. The experiment results clearly indicate that students, especially those who lack a basic understanding of programming concepts can benefit from the ChatGPT-assisted learning strategy.
Future Work
In the future, we will extend this research work to the following aspects: (1) for the next batch of students, creating groups with mixed population and conduct the experiment to further evaluate out method. (2) Generating dynamic and personalised course materials for other object-oriented programming languages, such as C++, C# and Python (3) Enhancing the learning framework by incorporating additional instructional methods, such as hands-on workshops, auto-grading, learning feedback and report (4) Extending ChatGPT-assisted learning across other academic disciplines.
Statistical Comparison of The Final Quiz of Group A and Group B
References
Ghimire, A., & Edwards, J. (2024). Coding with AI: How are tools like ChatGPT being used by students in foundational programming courses? In International Conference on Artificial Intelligence in Education (pp. 259–267). Springer Nature.
Haindl, P., & Weinberger, G. (2024). Students’ experiences of using ChatGPT in an undergraduate programming course. IEEE Access, 12, 43519–43529. https://dx.doi.org/10.1109/ACCESS.2024.3380909
Jin, J., & Kim, M. (2023). GPT-empowered personalized eLearning system for programming languages. Applied Sciences, 13(23), 12773. https://doi.org/10.3390/app132312773
Mikac, M., Horvatić, M., Logožar, R., & Dumić, E. (2024). ChatGPT in education—Use cases in an introductory web programming course. In INTED2024 Proceedings (pp. 3173–3182). IATED.
OpenAI (2023). ChatGPT: Optimizing Language Models for Dialogue. https://openai.com/index/chatgpt/.
Ouh, E. L., Gan, B. K. S., Shim, K. J., & Wlodkowski, S. (2023). ChatGPT, can you generate solutions for my coding exercises? An evaluation on its effectiveness in an undergraduate Java programming course. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education (Vol. 1, pp. 54–60).
Vukojičić, M., & Krstić, J. (2023). ChatGPT in programming education: ChatGPT as a programming assistant. InspirED Teachers’ Voice, 2023(1), 7–13. https://www.teachers-voice.org/index.php/inspired/article/view/10