This paper is in the following e-collection/theme issue:

Natural Language Processing (873) Generative Language Models Including ChatGPT (636) Chatbots and Conversational Agents (704) Artificial Intelligence (2006) Artificial Intelligence (AI) in Medical Education (151) Human Factors and Usability Case Studies (963)

Published on in Vol 12 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/54345, first published .
Reference Hallucination Score for Medical Artificial Intelligence Chatbots: Development and Usability Study

Reference Hallucination Score for Medical Artificial Intelligence Chatbots: Development and Usability Study

Reference Hallucination Score for Medical Artificial Intelligence Chatbots: Development and Usability Study

Authors of this article:

Fadi Aljamaan1 Author Orcid Image ;   Mohamad-Hani Temsah1 Author Orcid Image ;   Ibraheem Altamimi1 Author Orcid Image ;   Ayman Al-Eyadhy1 Author Orcid Image ;   Amr Jamal1 Author Orcid Image ;   Khalid Alhasan1 Author Orcid Image ;   Tamer A Mesallam2 Author Orcid Image ;   Mohamed Farahat2 Author Orcid Image ;   Khalid H Malki2 Author Orcid Image
Article Authors Cited by (43) Tweetations (26) Metrics

Journals

  1. Temsah M, Jamal A, Alhasan K, Temsah A, Malki K. OpenAI o1-Preview vs. ChatGPT in Healthcare: A New Frontier in Medical AI Reasoning. Cureus 2024 View
  2. Giray L. Benefits and Challenges of Using AI for Peer Review: A Study on Researchers’ Perceptions. The Serials Librarian 2024;85(5-6):144 View
  3. Zhang P, Shi J, Kamel Boulos M. Generative AI in Medicine and Healthcare: Moving Beyond the ‘Peak of Inflated Expectations’. Future Internet 2024;16(12):462 View
  4. Barrot J. Balancing Innovation and Integrity: An Emerging Technology Report on SciSpace in Academic Writing. Technology, Knowledge and Learning 2025;30(1):587 View
  5. Thandla S, Armstrong G, Menon A, Shah A, Gueye D, Harb C, Hernandez E, Iyer Y, Hotchner A, Modi R, Mudigonda A, Prokos M, Rao T, Thomas O, Beltran C, Guerrieri T, LeBlanc S, Moorthy S, Yacoub S, Gardner J, Greenberg B, Hubal A, Lapina Y, Moran J, O’Brien J, Winnicki A, Yoka C, Zhang J, Zimmerman P. Comparing new tools of artificial intelligence to the authentic intelligence of our global health students. BioData Mining 2024;17(1) View
  6. Ullah M, Bin Naeem S, Kamel Boulos M. Assessing the Guidelines on the Use of Generative Artificial Intelligence Tools in Universities: A Survey of the World’s Top 50 Universities. Big Data and Cognitive Computing 2024;8(12):194 View
  7. Pergantis P, Bamicha V, Skianis C, Drigas A. AI Chatbots and Cognitive Control: Enhancing Executive Functions Through Chatbot Interactions: A Systematic Review. Brain Sciences 2025;15(1):47 View
  8. Temsah M, Nazer R, Altamimi I, Aldekhyyel R, Jamal A, Almansour M, Aljamaan F, Alhasan K, Temsah A, Al-Eyadhy A, Aljafen B, Malki K. OpenAI's Sora and Google's Veo 2 in Action: A Narrative Review of Artificial Intelligence-driven Video Generation Models Transforming Healthcare. Cureus 2025 View
  9. Temsah A, Alhasan K, Altamimi I, Jamal A, Al-Eyadhy A, Malki K, Temsah M. DeepSeek in Healthcare: Revealing Opportunities and Steering Challenges of a New Open-Source Artificial Intelligence Frontier. Cureus 2025 View
  10. Abouammoh N, Alhasan K, Aljamaan F, Raina R, Malki K, Altamimi I, Muaygil R, Wahabi H, Jamal A, Alhaboob A, Assiri R, Al-Tawfiq J, Al-Eyadhy A, Soliman M, Temsah M. Perceptions and Earliest Experiences of Medical Students and Faculty With ChatGPT in Medical Education: Qualitative Study. JMIR Medical Education 2025;11:e63400 View
  11. Choo S, Yoo S, Endo K, Truong B, Son M. Advancing Clinical Chatbot Validation Using AI-Powered Evaluation With a New 3-Bot Evaluation System: Instrument Validation Study. JMIR Nursing 2025;8:e63058 View
  12. Salyers A, Bull S, Silvasstar J, Howell K, Wright T, Banaei-Kashani F. Building and Beta-Testing Be Well Buddy Chatbot, a Secure, Credible and Trustworthy AI Chatbot That Will Not Misinform, Hallucinate or Stigmatize Substance Use Disorder: Development and Usability Study. JMIR Human Factors 2025;12:e69144 View
  13. Sweed T, Mabrouk A, Dawson M. Transforming orthopaedics with AI: Insights from a custom ChatGPT on ESSKA osteotomy consensus. Knee Surgery, Sports Traumatology, Arthroscopy 2025;33(5):1557 View
  14. Khawaja Z, Adhoni M, Byrnes K. Generative artificial intelligence powered chatbots in urology. Current Opinion in Urology 2025;35(3):243 View
  15. Zhou J, Zhang J, Wan R, Cui X, Liu Q, Guo H, Shi X, Fu B, Meng J, Yue B, Zhang Y, Zhang Z. Integrating AI into clinical education: evaluating general practice trainees’ proficiency in distinguishing AI-generated hallucinations and impacting factors. BMC Medical Education 2025;25(1) View
  16. Whitfield S, Yang S. Evaluating AI Language Models for Reference Services: A Comparative Study of ChatGPT, Gemini, and Copilot. Internet Reference Services Quarterly 2025;29(2):153 View
  17. Branda F, Stella M, Ceccarelli C, Cabitza F, Ceccarelli G, Maruotti A, Ciccozzi M, Scarpa F. The Role of AI-Based Chatbots in Public Health Emergencies: A Narrative Review. Future Internet 2025;17(4):145 View
  18. Triola M, Rodman A. Integrating Generative Artificial Intelligence Into Medical Education: Curriculum, Policy, and Governance Strategies. Academic Medicine 2025;100(4):413 View
  19. Temsah M, Al-Eyadhy A, Jamal A, Alhasan K, Malki K. Authors’ Reply: Citation Accuracy Challenges Posed by Large Language Models. JMIR Medical Education 2025;11:e73698 View
  20. Alhuzaimi A, Aljamaan F, AL-Ajwad F, Alabdulkareem W, Alshahrani F, Altamimi I, Al-Eyadhy L, Bukhari A, BinOmair A, Al-Subaie S, Shayah Y, Alhaboob A, Alanteet A, Alanteet A, Alharbi M, Jamal A, Barry M, Assiri R, Alhasan K, Al-Tawfiq J, Temsah M. Public knowledge of food poisoning, risk perception and food safety practices in Saudi Arabia: A cross-sectional survey following foodborne botulism outbreak. Medicine 2025;104(15):e41593 View
  21. Barra F, Rodella G, Costa A, Scalogna A, Carenzo L, Monzani A, Corte F. From prompt to platform: an agentic AI workflow for healthcare simulation scenario design. Advances in Simulation 2025;10(1) View
  22. Chen J, Ge X, Yuan C, Chen Y, Li X, Zhang X, Chen S, Zheng W, Miao C. Comparing orthodontic pre-treatment information provided by large language models. BMC Oral Health 2025;25(1) View
  23. Kring T, Akula S, Prasad S, Sokhn E, Thaller S. Evaluating AI Chatbots for Preoperative and Postoperative Counseling for Mandibular Distraction Osteogenesis. Journal of Craniofacial Surgery 2025 View
  24. Hunter J, Dentino P, Jayakumar P. The Emergence of Applied Artificial Intelligence in the Realm of Value Based Musculoskeletal Care. Current Reviews in Musculoskeletal Medicine 2025;18(10):379 View
  25. Udagedara T, Tran A, Bokhari S, Shiraga S, Abel S, Houghton C, Galvin K, Samakar K, Putnam L. A Comparative Analysis of the Accuracy and Readability of Popular Artificial Intelligence-Chat Bots for Inguinal Hernia Management. The American Surgeon™ 2025;91(10):1729 View
  26. Siyad A, Akhila A, Ramanarayanan S, Mustafa S, James J, Babu P. Assessing the Validity and Accuracy of Artificial Intelligence Technologies for Identifying Relevant Literature in Dentistry. Journal of Nature and Science of Medicine 2025;8(2):135 View
  27. Chow J, Li K. Large Language Models in Medical Chatbots: Opportunities, Challenges, and the Need to Address AI Risks. Information 2025;16(7):549 View
  28. Yang H, Li M, Zhou H, Xiao Y, Fang Q, Zhou S, Zhang R. Large Language Model Synergy for Ensemble Learning in Medical Question Answering: Design and Evaluation Study. Journal of Medical Internet Research 2025;27:e70080 View
  29. Menezes P, Santos A, Andrade M, Nascimento Júnior J, Duarte M, Saravanan S, Serafini M. Innovative data extraction in patent reviews with SciSpace. World Patent Information 2025;82:102377 View
  30. Camlet A, Kusiak A, Ossowska A, Świetlik D. Advances in Periodontal Diagnostics: Application of MultiModal Language Models in Visual Interpretation of Panoramic Radiographs. Diagnostics 2025;15(15):1851 View
  31. Kring T, Prasad S, Dadi S, Sokhn E, Franzmann E. A comparison of quality and readability of Artificial Intelligence chatbots in triage for head and neck cancer. American Journal of Otolaryngology 2025;46(5):104710 View
  32. Ng J. Prompt engineering for generative artificial intelligence chatbots in health research: A practical guide for traditional, complementary, and integrative medicine researchers. Integrative Medicine Research 2025;14(4):101222 View
  33. Wei Q, Cui M, Liu Z, Liu Z, Feng G, Li Y, Christiani D, Li L, Wang J, Hao Y, Wei Y. Integrating statistical design and inference: A roadmap for robust and trustworthy medical AI. The Innovation Medicine 2025;3(3):100145 View
  34. Asiri S. Assessing the Reliability of ChatGPT and Gemini in Identifying Relevant Orthodontic Literature. European Journal of General Dentistry 2025 View
  35. . Artificial Intelligence in Scholarly Publishing: New Tools, Same Standards. Clinical Journal of Oncology Nursing 2025 View
  36. Ito Y, Ikehara H, Okamoto Y, Kako J. Is Large Language Model-Assisted Citation Screening Feasible in a Scoping Review on Nonpharmacological Interventions for Delirium in Patients With Cancer?. Cureus 2025 View
  37. Jain A, Nimonkar P, Jadhav P. Citation integrity in the age of AI: evaluating the risks of reference hallucination in maxillofacial literature. Journal of Cranio-Maxillofacial Surgery 2025;53(10):1871 View
  38. Hasnain M, Aurangzeb K, Alhussein M, Ghani I, Mahmood M. AI in conjunctivitis research: assessing ChatGPT and DeepSeek for etiology, intervention, and citation integrity via hallucination rate analysis. Frontiers in Artificial Intelligence 2025;8 View
  39. Cheema A. The Future of Artificial Intelligence and Artificial Intelligence in Primary Care. Primary Care: Clinics in Office Practice 2025 View
  40. Altunel Kılınç E, Çabuk Çelik N. Evaluation of artificial ıntelligence use in ankylosing spondylitis with ChatGPT-4: patient and physician perspectives. Clinical Rheumatology 2025 View
  41. Safran E, Çalı A. Fabricated or accurate? Ethical concerns and citation hallucination in aI-generated scientific writing on musculoskeletal topics. Anatolian Current Medical Journal 2025;7(5):695 View

Books/Policy Documents

  1. Kancilia C, Nguyen A. Impact of Emergent Technologies on Writing Centers and Pedagogy. View

Conference Proceedings

  1. Tran T, Nguyen M, Tran M, Do T. Proceedings of the 2nd Workshop on Security-Centric Strategies for Combating Information Disorder. Enhancing Medical Chatbot Reliability: A Multi-Step Verification Approach to Prevent Hallucinations View