Exploring the Impact of Language Models on Undergraduate Thesis Writing Skills among Higher Education Students

Authors

  • Agnia Ilma Universitas Brawijaya, Malang, Indonesia
  • Riza Ady Sampurna Pojok Peradaban Institute, Malang, Indonesia

DOI:

https://doi.org/10.59001/pjier.v2i1.152

Keywords:

language models, undergraduate thesis, writing skills, higher education

Abstract

This article explores the impact of language models on undergraduate thesis writing skills among higher education students. With advancements in technology, language models have become integral to the digital writing landscape. This study examined the influence of language models on students' confidence and motivation in academic writing. The research employs a qualitative approach, conducting semi-structured interviews with five students from an English Language Education program who utilized AI tools in their thesis writing process. Despite acknowledging the advantages, participants underscored challenges encompassing overreliance, inaccuracies, and disruptions in the natural flow of their writing. The impact of these tools on participants' confidence and motivation in academic writing manifested diversely, introducing varying degrees of perceived artificiality in writing proficiency. There were already numerous researches regarding the pretinence of language models usage on students’ writing aptitude.  However, there remains a gap in understanding their specific influence on undergraduate thesis writing. Thematic analysis of the interview data reveals insights into the influence of language models, on the writing proficiency of higher education students engaged in the process of crafting undergraduate theses. Employing a methodology encompassing interviews, participants reported noteworthy enhancements in the coherence, clarity, grammar, syntax, and vocabulary of their theses facilitated by the utilization of these language tools. Overall, the incorporation of language models in the writing process emerged as a catalyst for heightened efficiency, skills development, and enhanced clarity in undergraduate theses, affirming the constructive role of AI technology in educational settings. Individualized experiences with language models exhibited variance contingent upon personal preferences and attitudes towards the writing process.

References

Alkhawaldeh, M.A., Khasawneh, M.A.S. (2023). Language Learning Tools And Improvements In Speaking And Writing Skills. Journal of Southwest Jiaotong University, 58(5). https://doi.org/10.35741/issn.0258-2724.58.5.33

Alqahtani, T., Badreldin, H. A., Alrashed, M., Alshaya, A. I., Alghamdi, S. S., Alowais, S. A., ... & Albekairy, A. M. (2023). The emergent role of artificial intelligence, natural learning processing, and large language models in higher education and research. Research in Social & Administrative Pharmacy: RSAP, S1551-7411. https://doi.org/10.1016/j.sapharm.2023.05.016

Cacciuttolo, C., Vásquez, Y., Cano, D., & Valenzuela, F. (2023). Research Thesis for Undergraduate Engineering Programs in the Digitalization Era: Learning Strategies and Responsible Research Conduct Road to a University Education 4.0 Paradigm. Sustainability, 15(14), 11206. https://doi.org/10.3390/su151411206

Ciaccio, E. J. (2023). Use of artificial intelligence in scientific paper writing. Informatics in Medicine Unlocked, 101253. https://doi.org/10.1016/j.imu.2023.101253

Chauncey, S. A., & McKenna, H. P. (2023). A framework and exemplars for ethical and responsible use of AI Chatbot technology to support teaching and learning. Computers and Education: Artificial Intelligence, 5, 100182. https://doi.org/10.1016/j.caeai.2023.100182

Deng, X., & Yu, Z. (2023). A meta-analysis and systematic review of the effect of chatbot technology use in sustainable education. Sustainability, 15(4), 2940. https://doi.org/10.3390/su15042940

El Shazly, R. (2021). Effects of artificial intelligence on English speaking anxiety and speaking performance: A case study. Expert Systems, 38(3), 12667. https://doi.org/10.1111/exsy.12667

Essel, H. B., Vlachopoulos, D., Essuman, A. B., & Amankwa, J. O. (2024). ChatGPT effects on cognitive skills of undergraduate students: Receiving instant responses from AI-based conversational large language models (LLMs). Computers and Education: Artificial Intelligence, 6, 100198. https://doi.org/10.1016/j.caeai.2023.100198

Gayed, J. M., Carlon, M. K. J., Oriola, A. M., & Cross, J. S. (2022). Exploring an AI-based writing Assistant's impact on English language learners. Computers and Education: Artificial Intelligence, 3, 100055. https://doi.org/10.1016/j.caeai.2022.100055

Hadi, M. U., Qureshi, R., Shah, A., Irfan, M., Zafar, A., Shaikh, M. B., ... & Mirjalili, S. (2023). A survey on large language models: Applications, challenges, limitations, and practical usage. Authorea Preprints. https://doi.org/10.36227/techrxiv.23589741.v1

Hadi, M. U., Qureshi, R., Shah, A., Irfan, M., Zafar, A., Shaikh, M. B., ... & Mirjalili, S. (2023). Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects. Authorea Preprints. https://doi.org/10.36227/techrxiv.23589741.v4

Hapsari, I.P. and Wu, T.T. (2022) AI Chatbots Learning Model in English Speaking Skill: Alleviating Speaking Anxiety, Boosting Enjoyment, and Fostering Critical Thinking. Innovative Technologies and Learning: 5th International Conference, Online, August 2022. Cham: Springer International Publishing, 444-453. https://doi.org/10.1007/978-3-031-15273-3_49

Hsiao, Y. P., Klijn, N., & Chiu, M. S. (2023). Developing a framework to re-design writing assignment assessment for the era of Large Language Models. Learning: Research and Practice, 9(2), 148-158. https://doi.org/10.1080/23735082.2023.2257234

Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., ... & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and individual differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274

Khan, W., Daud, A., Khan, K., Muhammad, S., & Haq, R. (2023). Exploring the frontiers of deep learning and natural language processing: A comprehensive overview of key challenges and emerging trends. Natural Language Processing Journal, 100026. https://doi.org/10.1016/j.nlp.2023.100026

Koka, N. A., Khan, A. S., & Jan, N. (2023). Exploring the attitudes and perceptions of foreign language lecturers on artificial intelligence-driven writing evaluation and feedback tools for improving undergraduate writing skills. Journal of Southwest Jiaotong University, 58(5). https://doi.org/10.35741/issn.0258-2724.58.5.6

Livberber, T., & Ayvaz, S. (2023). The impact of Artificial Intelligence in academia: Views of Turkish academics on ChatGPT. Heliyon, 9(9), 19688. https://doi.org/10.1016/j.heliyon.2023.e19688

Malik, A. R., Pratiwi, Y., Andajani, K., Numertayasa, I. W., Suharti, S., & Darwis, A. (2023). Exploring Artificial Intelligence in Academic Essay: Higher Education Student's Perspective. International Journal of Educational Research Open, 5, 100296. https://doi.org/10.1016/j.ijedro.2023.100296

Marghany, M. M. (2023). Using artificial intelligence-based instruction to develop EFL higher education students’ essay writing skills. CDELT Occasional Papers in the Development of English Education, 82(1), 219-240. https://doi.org/10.21608/OPDE.2023.313623

Meyer, J. G., Urbanowicz, R. J., Martin, P. C., O’Connor, K., Li, R., Peng, P. C., ... & Moore, J. H. (2023). ChatGPT and large language models in academia: opportunities and challenges. BioData Mining, 16(1), 20. https://doi.org/10.1186/s13040-023-00339-9

Myers, D., Mohawesh, R., Chellaboina, V. I., Sathvik, A. L., Venkatesh, P., Ho, Y. H., ... & Jararweh, Y. (2023). Foundation and large language models: fundamentals, challenges, opportunities, and social impacts. Cluster Computing, 1-26. https://doi.org/10.1007/s10586-023-04203-7

Olsson, A., & Engelbrektsson, O. (2022). A thesis that writes itself: On the threat of AI-generated essays within academia. https://urn.kb.se/resolve?urn=urn:nbn:se:hh:diva-47095

Sallam, M. (2023). The utility of ChatGPT as an example of large language models in healthcare education, research and practice: Systematic review on the future perspectives and potential limitations. medRxiv, 2023-02. https://doi.org/10.1101/2023.02.19.23286155

Song, C., & Song, Y. (2023). Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students. Frontiers in Psychology, 14, 1260843. https://doi.org/10.3389/fpsyg.2023.1260843

Schwenke, N., Söbke, H., & Kraft, E. (2023). Potentials and Challenges of Chatbot-Supported Thesis Writing: An Autoethnography. Trends in Higher Education, 2(4), 611-635. https://doi.org/10.3390/higheredu2040037

Starfield, S., & Paltridge, B. (2019). Thesis and dissertation writing in a second language: Context, identity, genre. Journal of Second Language Writing, 43, 1-3. https://doi.org/10.1016/j.jslw.2018.10.002

Tayan, O., Hassan, A., Khankan, K., & Askool, S. (2023). Considerations for adapting higher education technology courses for AI large language models: A critical review of the impact of ChatGPT. Machine Learning with Applications, 100513. https://doi.org/10.1016/j.mlwa.2023.100513

Yan, L., Sha, L., Zhao, L., Li, Y., Martinez‐Maldonado, R., Chen, G., ... & Gašević, D. (2024). Practical and ethical challenges of large language models in education: A systematic scoping review. British Journal of Educational Technology, 55(1), 90-112. https://doi.org/10.1111/bjet.13370

Zekaj, R. (2023). AI Language Models as educational allies: enhancing instructional support in higher education. International Journal of Learning, Teaching and Educational Research, 22(8), 120-134. https://doi.org/10.26803/ijlter.22.8.7

Downloads

Published

2024-03-01

Issue

Section

Articles