Review Article

A bibliometric analysis of publications on ChatGPT in education: Research patterns and topics

Marina R. Zheltukhina 1 , Olga V. Sergeeva 2 , Alfiya R. Masalimova 3 * , Roza L. Budkevich 4 , Nikolay N. Kosarenko 5 , Georgy V. Nesterov 6
More Detail
1 Pyatigorsk State University, Pyatigorsk, RUSSIA2 Kuban State University, Krasnodar, RUSSIA3 Kazan Federal University, Kazan, RUSSIA4 Almetyevsk State Oil Institute, Almetyevsk, RUSSIA5 Plekhanov Russian University of Economics, Moscow, RUSSIA6 I.M. Sechenov First Moscow State Medical University (Sechenov University), Moscow, RUSSIA* Corresponding Author
Online Journal of Communication and Media Technologies, 14(1), January 2024, e202405, https://doi.org/10.30935/ojcmt/14103
Published: 05 January 2024
OPEN ACCESS   2099 Views   2099 Downloads
Download Full Text (PDF)

ABSTRACT

This paper aims to conduct a bibliometric analysis and a comprehensive overview of publications on ChatGPT in educational research. This research also aimed to present the bibliometric results to interpret the research patterns and themes of the application of ChatGPT in educational research. The researchers used the VOSviewer program to conduct a bibliometric analysis and identify research patterns and topics in publications indexed in the Scopus database. For this purpose, the researchers used the Scopus database to find related publications. After applying inclusion and exclusion criteria, they found 82 publications and analyzed them using the bibliometric method. This study showed that researchers from 42 countries examined various topics, including academic writing, artificial intelligence’s (AI) potential, and benefits, using ChatGPT in research, exploring best practices, and reviewing AI. The keyword analysis results showed that five clusters emerged from the current studies on ChatGPT in education research. These results showed that researchers focused on understanding the use of ChatGPT in medical and nursing education, generative AI’s ethical dimensions, the effects of ChatGPT on educational outcomes, large language models and medical education, and ChatGPT and AI. In general, the use of ChatGPT in educational contexts and research is frequently discussed in the publications analyzed in this study. In addition, medical and nursing education was the most studied of the many research studies. Based on the obtained results, recommendations for further studies are drawn.

CITATION (APA)

Zheltukhina, M. R., Sergeeva, O. V., Masalimova, A. R., Budkevich, R. L., Kosarenko, N. N., & Nesterov, G. V. (2024). A bibliometric analysis of publications on ChatGPT in education: Research patterns and topics. Online Journal of Communication and Media Technologies, 14(1), e202405. https://doi.org/10.30935/ojcmt/14103

REFERENCES

  1. Alneyadi, S., & Wardat, Y. (2023a). ChatGPT: Revolutionizing student achievement in the electronic magnetism unit for eleventh-grade students in Emirates schools. Contemporary Educational Technology, 15(4), ep448. https://doi.org/10.30935/cedtech/13417
  2. Alneyadi, S., Wardat, Y., Alshannag, Q., & Abu-Al-Aish, A. (2023b). The effect of using smart e-learning app on the academic achievement of eighth-grade students. EURASIA Journal of Mathematics, Science and Technology Education, 19(4), em2248. https://doi.org/10.29333/ejmste/13067
  3. Atlas, S. (2023). ChatGPT for higher education and professional development: A guide to conversational AI. University of Rhode Island. https://digitalcommons.uri.edu/cba_facpubs/548
  4. Baidoo-Anu, D., & Owusu Ansah, L. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. SSRN. https://doi.org/10.2139/ssrn.4337484
  5. Bauer, E., Greisel, M., Kuznetsov, I., Berndt, M., Kollar, I., Dresel, M., Fischer, M. R., & Fischer, F. (2023). Using natural language processing to support peer-feedback in the age of artificial intelligence: A cross-disciplinary framework and a research agenda. British Journal of Educational Technology, 54(5), 1222-1245. https://doi.org/10.1111/bjet.13336
  6. Bearman, M., & Ajjawi, R. (2023). Learning to work with the black box: Pedagogy for a world with artificial intelligence. British Journal of Educational Technology, 54(5), 1160-1173. https://doi.org/10.1111/bjet.13337
  7. Chaudhry, I. S., Sarwary, S. A. M., El Refae, G. A., & Chabchoub, H. (2023). Time to revisit existing student’s performance evaluation approach in higher education sector in a new era of ChatGPT–A case study. Cogent Education, 10(1), 2210461. https://doi.org/10.1080/2331186x.2023.2210461
  8. Cooper, G. (2023). Examining science education in ChatGPT: An exploratory study of generative artificial intelligence. Journal of Science Education and Technology, 32, 444-452. https://doi.org/10.1007/s10956-023-10039-y
  9. Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International. https://doi.org/10.1080/14703297.2023.2190148
  10. Crawford, J., Cowling, M., & Allen, K. (2023). Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). Journal of University Teaching & Learning Practice, 20(3). https://doi.org/10.53761/1.20.3.02
  11. Dawson, P., & Sutherland-Smith, W. (2018). Can markers detect contract cheating? Results from a pilot study. Assessment & Evaluation in Higher Education, 43(2), 286-293. https://doi.org/10.1080/02602938.2017.1336746
  12. Dawson, P., Sutherland-Smith, W., & Ricksen, M. (2020). Can software improve marker accuracy at detecting contract cheating? A pilot study of the Turnitin authorship investigate alpha. Assessment & Evaluation in Higher Education, 45(4), 473-482. https://doi.org/10.1080/02602938.2019.1662884
  13. Day, T. (2023). A preliminary investigation of fake peer-reviewed citations and references generated by ChatGPT. The Professional Geographer. https://doi.org/10.1080/00330124.2023.2190373
  14. Deng, J., & Lin, Y. (2023). The benefits and challenges of ChatGPT: An overview. Frontiers in Computing and Intelligent Systems, 2(2), 81-83. https://doi.org/10.54097/fcis.v2i2.4465
  15. Dowling, M., & Lucey, B. (2023). ChatGPT for (finance) research: The Bananarama conjecture. Finance Research Letters, 53, 103662. https://doi.org/10.1016/j.frl.2023.103662
  16. Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., . . ., & Wright, R. (2023). Opinion paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642
  17. Emenike, M. E., & Emenike, B. U. (2023). Was this title generated by ChatGPT? Considerations for artificial intelligence text-generation software programs for chemists and chemistry educators. Journal of Chemical Education, 100(4), 1413-1418. https://doi.org/10.1021/acs.jchemed.3c00063
  18. Eysenbach, G. (2023). The role of ChatGPT, generative language models, and artificial intelligence in medical education: A conversation with ChatGPT and a call for papers. JMIR Medical Education, 9, e46885. https://doi.org/10.2196/46885
  19. Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International. https://doi.org/10.1080/14703297.2023.2195846
  20. Fernandez, P. (2023). “Through the looking glass: Envisioning new library technologies” AI-text generators as explained by ChatGPT. Library Hi Tech News, 40(3), 11-14. https://doi.org/10.1108/lhtn-02-2023-0017
  21. Ferrucci, D., Levas, A., Bagchi, S., Gondek, D., & Mueller, E. T. (2013). Watson: Beyond jeopardy! Artificial Intelligence, 199-200, 93-105. https://doi.org/10.1016/j.artint.2012.06.009
  22. Floridi, L., & Chiriatti, M. (2020). GPT-3: Its nature, scope, limits, and consequences. Minds and Machines, 30(4), 681-694. https://doi.org/10.1007/s11023-020-09548-1
  23. Frederick, D. E. (2023). ChatGPT: A viral data-driven disruption in the information environment. Library Hi Tech News, 40(3), 4-10. https://doi.org/10.1108/lhtn-04-2023-0063
  24. Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023). Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. npj Digital Medicine, 6, 75. https://doi.org/10.1038/s41746-023-00819-6
  25. Gardner, D. E., & Giordano, A. N. (2023). The challenges and value of undergraduate oral exams in the physical chemistry classroom: A useful tool in the assessment toolbox. Journal of Chemical Education, 100(5), 1705-1709. https://doi.org/10.1021/acs.jchemed.3c00011
  26. Giannos, P., & Delardas, O. (2023). Performance of ChatGPT on UK standardized admission tests: Insights from the BMAT, TMUA, LNAT, and TSA examinations. JMIR Medical Education, 9, e47737. https://doi.org/10.2196/47737
  27. Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023). How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Medical Education, 9, e45312. https://doi.org/10.2196/45312
  28. González-Pérez, L. I., & Ramírez-Montoya, M. S. (2022). Components of education 4.0 in 21st century skills frameworks: Systematic review. Sustainability, 14(3), 1493. https://doi.org/10.3390/su14031493
  29. Halaweh, M. (2023). ChatGPT in education: Strategies for responsible implementation. Contemporary Educational Technology, 15(2), ep421. https://doi.org/10.30935/cedtech/13036
  30. Hamzaçebi, C., Akay, D., & Kutay, F. (2009). Comparison of direct and iterative artificial neural network forecast approaches in multi-periodic time series forecasting. Expert Systems With Applications, 36(2), 3839-3844. https://doi.org/10.1016/j.eswa.2008.02.042
  31. Hassabis, D. (2017). Artificial intelligence: Chess match of the century. Nature, 544(7651), 413-414. https://doi.org/10.1038/544413a
  32. Humphry, T., & Fuller, A. L. (2023). Potential ChatGPT use in undergraduate chemistry laboratories. Journal of Chemical Education, 100(4), 1434-1436. https://doi.org/10.1021/acs.jchemed.3c00006
  33. Ivanov, S., & Soliman, M. (2023). Game of algorithms: ChatGPT implications for the future of tourism education and research. Journal of Tourism Futures, 9(2), 214-221. https://doi.org/10.1108/jtf-02-2023-0038
  34. Jarrah, A. M., Wardat, Y., & Fidalgo, P. (2023). Using ChatGPT in academic writing is (not) a form of plagiarism: What does the literature say? Online Journal of Communication and Media Technologies, 13(4), e202346. https://doi.org/10.30935/ojcmt/13572
  35. Jeon, J., & Lee, S. (2023). Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT. Education and Information Technologies, 28, 15873-15892. https://doi.org/10.1007/s10639-023-11834-1
  36. Karaali, G. (2023). Artificial intelligence, basic skills, and quantitative literacy. Numeracy, 16(1), 9. https://doi.org/10.5038/1936-4660.16.1.1438
  37. Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., . . ., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
  38. Kooli, C. (2023). Chatbots in education and research: A critical examination of ethical implications and solutions. Sustainability, 15(7), 5614. https://doi.org/10.3390/su15075614
  39. Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I., & Pechenkina, E. (2023). Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Education, 21(2), 100790. https://doi.org/10.1016/j.ijme.2023.100790
  40. Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410. https://doi.org/10.3390/educsci13040410
  41. Masters, K. (2023a). Ethical use of artificial intelligence in health professions education: AMEE guide no. 158. Medical Teacher, 45(6), 574-584. https://doi.org/10.1080/0142159x.2023.2186203
  42. Masters, K. (2023b). Medical teacher’s first ChatGPT’s referencing hallucinations: Lessons for editors, reviewers, and teachers. Medical Teacher, 45(7), 673-675. https://doi.org/10.1080/0142159x.2023.2208731
  43. McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (2006). A proposal for the Dartmouth summer research project on artificial intelligence, august 31, 1955. AI Magazine, 27(4), 12.
  44. Mhlanga, D. (2023). Open AI in education, the responsible and ethical use of ChatGPT towards lifelong learning. SSRN. https://doi.org/10.2139/ssrn.4354422
  45. OpenAI. (2022). ChatGPT. https://openai.com/blog/ChatGPT/
  46. Perkins, M. (2023). Academic integrity considerations of AI large language models in the post-pandemic era: ChatGPT and beyond. Journal of University Teaching and Learning Practice, 20(2). https://doi.org/10.53761/1.20.02.07
  47. Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching, 6(1), 342-363. https://doi.org/10.37074/jalt.2023.6.1.9
  48. Skavronskaya, L., Hadinejad, A. H., & Cotterell, D. (2023). Reversing the threat of artificial intelligence to opportunity: A discussion of ChatGPT in tourism education. Journal of Teaching in Travel & Tourism, 23(2), 253-258. https://doi.org/10.1080/15313220.2023.2196658
  49. Sok, S., & Heng, K. (2023). ChatGPT for education and research: A review of benefits and risks. SSRN. https://doi.org/10.2139/ssrn.4378735
  50. Strzelecki, A. (2023). To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interactive Learning Environments. https://doi.org/10.1080/10494820.2023.2209881
  51. Subramani, M., Jaleel, I., & Krishna Mohan, S. (2023). Evaluating the performance of ChatGPT in medical physiology university examination of phase I MBBS. Advances in Physiology Education, 47(2), 270-271. https://doi.org/10.1152/advan.00036.2023
  52. Sun, G. H., & Hoelscher, S. H. (2023). The ChatGPT storm and what faculty can do. Nurse Educator, 48(3), 119-124. https://doi.org/10.1097/nne.0000000000001390
  53. Teixeira da Silva, J. A. (2023). Is ChatGPT a valid author? Nurse Education in Practice, 68, 103600. https://doi.org/10.1016/j.nepr.2023.103600
  54. Thorp, H. H. (2023). ChatGPT is fun, but not an author. Science, 379(6630), 313. https://doi.org/10.1126/science.adg7879
  55. Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10, 15. https://doi.org/10.1186/s40561-023-00237-x
  56. Uddin, S. M. J., Albert, A., Ovid, A., & Alsharef, A. (2023). Leveraging ChatGPT to aid construction hazard recognition and support safety education and training. Sustainability, 15(9), 7121. https://doi.org/10.3390/su15097121
  57. van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023). ChatGPT: Five priorities for research. Nature, 614(7947), 224-226. https://doi.org/10.1038/d41586-023-00288-7
  58. van Eck, N. J., & Waltman, L. (2020). VOSviewer manual. https://www.vosviewer.com/documentation/Manual_VOSviewer_1.6.8.pdf
  59. Wardat, Y., Tashtoush, M. A., AlAli, R., & Jarrah, A. M. (2023). ChatGPT: A revolutionary tool for teaching and learning mathematics. EURASIA Journal of Mathematics, Science and Technology Education, 19(7), em2286. https://doi.org/10.29333/ejmste/13272
  60. Wu, R., & Yu, Z. (2023). Do AI chatbots improve students learning outcomes? Evidence from a meta-analysis. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13334
  61. Yan, D. (2023). Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation. Education and Information Technologies, 28, 13943-13967. https://doi.org/10.1007/s10639-023-11742-4
  62. Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education–Where are the educators? International Journal of Educational Technology in Higher Education, 16, 39. https://doi.org/10.1186/s41239-019-0171-0