TY  - JOUR
AU  - Steele, Brian
AU  - Fairie, Paul
AU  - Kemp, Kyle
AU  - D'Souza, Adam G
AU  - Wilms, Matthias
AU  - Santana, Maria Jose
PY  - 2025
DA  - 2025/2/24
TI  - Identifying Patient-Reported Care Experiences in Free-Text Survey Comments: Topic Modeling Study
JO  - JMIR Med Inform
SP  - e63466
VL  - 13
KW  - natural language processing
KW  - patient-reported experience
KW  - topic models
KW  - inpatient
KW  - artificial intelligence
KW  - AI
KW  - patient reported
KW  - feedback
KW  - survey
KW  - patient experiences
KW  - bidirectional encoder representations from transformers
KW  - BERT
KW  - sentiment analysis
KW  - pediatric caregivers
KW  - patient safety
KW  - safety
AB  - Background: Patient-reported experience surveys allow administrators, clinicians, and researchers to quantify and improve health care by receiving feedback directly from patients. Existing research has focused primarily on quantitative analysis of survey items, but these measures may collect optional free-text comments. These comments can provide insights for health systems but may not be analyzed due to limited resources and the complexity of traditional textual analysis. However, advances in machine learning–based natural language processing provide opportunities to learn from this traditionally underused data source. Objective: This study aimed to apply natural language processing to model topics found in free-text comments of patient-reported experience surveys. Methods: Consumer Assessment of Healthcare Providers and Systems–derived patient experience surveys were collected and linked to administrative inpatient records by the provincial health services organization responsible for inpatient care. Unsupervised topic modeling with automated labeling was performed with BERTopic. Sentiment analysis was performed to further assist in topic description. Results: Between April 2016 and February 2020, 43.4% (43,522/100,272) adult patients and 46.9% (3501/7464) pediatric caregivers included free-text responses on completed patient experience surveys. Topic models identified 86 topics among adult survey responses and 35 topics among pediatric responses that included elements of care not currently surveyed by existing questionnaires. Frequent topics were generally positive. Conclusions: We found that with limited tuning, BERTopic identified care experience topics with interpretable automated labeling. Results are discussed in the context of person-centered care, patient safety, and health care quality improvement. Furthermore, we note the opportunity for the identification of temporal and site-specific trends as a method to identify patient care and safety concerns. As the use of patient experience measurement increases in health care, we discuss how machine learning can be leveraged to provide additional insight on patient experiences. 
SN  - 2291-9694
UR  - https://medinform.jmir.org/2025/1/e63466
UR  - https://doi.org/10.2196/63466
DO  - 10.2196/63466
ID  - info:doi/10.2196/63466
ER  -