We conducted spatiotemporal analysis of the volume, sentiment, and topic of patient experience–related posts on the Weibo platform. Figure 1 shows an overview of this study, which included three distinct stages: (1) data collection and cleaning, (2) data selection and coding, and (3) data analysis and interpretation.

First, this study shows that the integration of logistic regression (LR), multinomial naïve Bayes (MNB), and random forest (RF) yields the highest prediction power, and the performance is better than that of any single one machine learning model.

Second, the results of sentiment analysis show that people tend to write negative posts on social media platforms, which could attract more responses and trigger more discussions.

Third, the results of topic modeling highlight the interpersonal and functional aspects of care, especially the interpersonal aspects, which are often the “moments of truth” during a service encounter in which patients make a critical evaluation of hospital services.

Weibo is the most widely used social media platform in China, with more than 550 million monthly active users [15], on which people publicly express their feelings and attitudes. We therefore selected the Weibo platform as the source to systematically search for posts about patient experience, defined as patients,’ friends,’ or family members’ discussions of hospital experience. Weibo supports the retrieval of textual content from a defined period that contains specific keywords. Weibo posts are open and easily accessible to everyone, and we did not extract users’ identities, such as poster ID and name, to maintain the users’ anonymity and avoid any invasion of individuals’ privacy. Moreover, only the post content, release date, and location were extracted to prevent identification of the users. Thus, there were no ethical issues to address.

To improve the search efficiency, we used a public free Python (Python Software Foundation) crawler tool to search for posts related to patient experience that contained predefined keywords. Hospital service–related terms and experience-related terms were combined to construct the final search strategy. Hospital service–related terms included hospital, health care service, doctor, nurse, and medical staff. With respect to experience-related terms, both broad and specific search terms were used to minimize the loss of relevant posts; broad search terms included experience, satisfaction, good, terrible, poor, and quality; specific search terms were determined on the basis of the 2012 National Health Service (NHS) patient experience framework, and terms including respect, environment, technology, information, education, communication, continuity, waiting time, trust, decision-making, access, emotional support, attitude, pain, and comfort were used. Patients’ opinions have time validity. To obtain representative patient voices, we searched for posts published in the last 10 years. Therefore, the release date of the posts was set from 2013 to 2023. We obtained 149,811 Weibo posts and stored the extracted posts locally in an informal text database. After some duplicate and reposted posts were filtered, 97,526 posts remained.

The content of posts on the Weibo platform is unstructured. For further processing and analysis, we preprocessed the text by removing garbled codes, hashtags, outliers, and non-Chinese characters and additional spaces.

A manual method was used to identify relevant data. We set criteria including three stages for the inclusion and exclusion of retrieved posts for detailed analysis. First, posts were included if they covered any information about health care. We then estimated whether the post was released from the perspective of patients or their caregivers, and posts related to health-related knowledge, propaganda promotion from hospitals, and the working experience of medical staff or students were excluded. Finally, we determined whether patient experience was the main topic of the post; we excluded retrieved posts if they expressed only personal feelings of some specific illnesses.

XC and ZS selected relevant posts; if there was any disagreement, two other authors, YZ and Wei Qin, MD, were consulted, and a consensus was reached on whether the post should be included. Finally, 4008 valid posts were further analyzed.

The statistical analysis was conducted via Python and IBM SPSS Statistics (version 26; IBM Corp). To efficiently extract and count each topic, all qualitative data were binarized to address multilabel classification via one hot encoding. The machine-coded data were subsequently imported into SPSS software to describe the characteristics of the discussion topics and sentiments of the posts and to calculate interrater agreement via Cohen κ values. Word clouds were created to visualize patient experience from frequently occurring topics in the posts [21].

Patients or the public were not involved in the design, conduct, reporting, or dissemination plans of this research.

This study was approved by the Research Ethics Committee of Zhongshan Hospital of Fudan University (approval B2022-613R). We did not extract users’ identities, such as poster ID and name. Moreover, only the post content, release date, and location were extracted to prevent identification of the users. Therefore, there is not any invasion of individuals’ privacy.

A total of 4008 posts pertaining to patient experience–related text were identified. The average length of the posts was 257 (SD 241) words. Among these posts, 47.68% (n=1911) commented on outpatient departments, 24.63% (n=987) commented on inpatient departments, 6.81% (n=273) commented on emergency departments, 3.42% (n=137) commented on physical examination departments, and 19.31% (n=774) commented on hospital services and did not focus on a specific care setting. In terms of comment objects, 56.69% (2272/4008) described doctors, 49.15% (1970/4008) described nurses, 0.62% (25/4008) described health care assistants, 2.35% (94/4008) described nonhealth care workers, and 12.67% (508/4008) used the term “medical staff” without the particular objects.

The number of patient experience–related posts between 2003 and 2022 is illustrated in Figure 2. Despite occasional fluctuations, patient experience–related posts revealed a general trend of rapid increase. The peaks in 2020 and 2022 were contemporaneous with the outbreaks of COVID-19.

Tables 1 and 2 list the performance metrics of the machine learning models. The accuracy, precision, recall, and F-measure of the integration of LR, MNB, and RF for patient experience themes were 0.93, 0.95, 0.80, 0.77, and 0.84, respectively. For the patient experience sentiment, the accuracy, precision, recall, and F-measure were 0.80, 0.89, 0.80, and 0.83, respectively.

The sentiment analysis of the included Weibo posts revealed that negative sentiment posts constituted the highest proportion (3319/4008, 82.81%), followed by positive sentiment posts (635/4008, 15.84%), and then mixed sentiment posts (n=54, 1.35%). The word count of negative sentiment posts was significantly greater than that of positive posts (264 vs 229, P<.001). The results of zero-shot emotion recognition revealed that angry emotion constituted the highest proportion (1491/4008, 37.20%), followed by sad emotion (1029/4008, 25.67%), surprised emotion (647/4008, 16.14%), happy emotion (528/4008, 13.17%), and afraid emotion (313/4008, 7.81%). The outbreak of the COVID-19 pandemic was used as the cutoff value to divide this study’s period into the two time frames (2003‐2019 and 2020‐2023) to analyze the temporal aspects of Weibo post sentiment. From 2003‐2019, the percentage of negative sentiment posts was 89.74% (1006/1121), which was significantly greater than that from 2020‐2023, which was 80.12% (2313/2887). We further compared the sentiment distributions in different setting categories.

Among the public discourse included in this content analysis, 3.89% (156/4008) of patient experience posts were nonspecifically about some specific care aspects and were labeled as “general.” Twenty patient experience themes were discussed on the social media platform (Table 3), and 52.74% (2114/4008) discussed more than one theme. The majority of the posts described the interpersonal aspects of care (73.53%, 2947/4008). The thematic distributions of the whole sample and of the user groups and sentiments are reported in Table 3. The theme distribution in the two time frames (2003‐2019 and 2020‐2023) is reported in Table 4.

The five most frequent themes were “health care professionals’ attitude,” “access to care,” “communication, information, and education,” “technical competence,” and “efficacy of treatment.” Multimedia Appendix 2 includes some specific examples of each theme.

The majority of patient experience–related posts (2712/4008, 67.66%) discussed health care professionals’ attitudes. People reported positive feelings and high satisfaction levels when they found health care providers to be kind, courteous, and friendly, whereas rude, impatient, and indifferent providers became the most common source of people’s complaints. The word “rude” was mentioned in 15.82% (429/2712) of the posts complaining about health care professionals’ attitude.

A total of 24.4% (978/4008) of the posts mentioned the availability of an appointment for some professionals or the time spent waiting for an appointment or visit in the outpatient department. Among these posts, most complained about the long waiting times.

A total of 22.18% (889/4008) of the posts mentioned communication, information, and education they received about their clinical status, progress, prognosis, and processes of care. As health needs improve, people are no longer satisfied with receiving health care passively; instead, they expect to be involved in their treatment and care. When health care providers ignore their information needs, people report negative feelings and low satisfaction levels.

There were 393 posts (9.81%) describing providers’ technical competence, which referred mainly to venipuncture procedures. More than half of these posts (206/393, 52.4%) reported that nurses had excellent or poor invasive venipuncture skills or muscle injection skills.

A better health outcome is the primary intention of visiting hospitals; therefore, in addition to the service behaviors mentioned above, efficacy is also an important discussion topic. A total of 332 posts (8.28%) described satisfactory or unsatisfactory treatment efficacy.

Understanding patient experience is fundamental to providing patient-centred care, and social media platforms provide an additional source of information to complement traditional survey methods. This study revealed that people tend to release negative posts on social media platforms, which could attract more responses and trigger more discussions. The results of topic modeling highlight the interpersonal and functional aspects of care, especially the interpersonal aspects, which are often the “moment of truth” during a service encounter in which patients make a critical evaluation of hospital services.

This time-based content analysis using Weibo data revealed that social media platforms are gaining popularity for communicating and sharing hospital experience among the Chinese public. In this study, the frequency of patient experience–related posts significantly increased over time. From 2013‐2018, the average number of valid posts was less than 100 per year. However, after 2019, the average number of valid posts was nearly 1000 per year. Owing to the accessibility and authenticity of free-text online data, summarizing and analyzing the public’s disclosure on social media platforms can generate deeper and new insight into patient experience and serve as an important opportunity to identify areas for quality improvement action. Health care facilities need to recognize and seize this opportunity. At the policy level, social media has been viewed as an important avenue for monitoring the performance of health care providers. For example, the England NHS actively monitors social media to better understand and respond to patients’ voices [22]. However, few studies have monitored and mined unsolicited patient experience–related data on the internet. To our knowledge, this is the first study in China to understand patient experience using social media.

Many data-mining techniques have been used to understand free-text feedback regarding health care, such as LR, RF classifier, naïve Bayes, MNB classifier, support vector machine, and decision tree classifier [23,24]. The advantages of different methods vary. This study compared the performance metrics of each machine learning model and found that the LR, MNB, and RF had the highest precision and accuracy in predicting discussion topics related to patient experience; this finding is consistent with the findings of other studies [25]. This study therefore integrated LR, MNB, and RF to predict patient experience themes and sentiment, and the performance was higher than that of any single machine learning model [11,26]. The accuracy, precision, recall, and F-measure values in this study are all greater than 0.8. Accuracy reflects overall correctness, precision evaluates positive prediction quality, recall assesses sensitivity to positive instances, and the F-measure balances precision and recall. Our results indicate that the machine learning classification approach could effectively label free text and automatically classify patients’ views into one or more topic categories from given examples. Hospital administrators and clinicians can gain deep insight into patient experience via a machine learning approach.

Among the public discourse included in this content analysis, a wide range of patient experience themes were discussed. These themes can be divided into interpersonal and functional aspects. Interpersonal aspects include health care professionals’ attitude, emotional support, information provision, privacy protection, involvement of family members, and responding requests, which constitute the highest proportion of patient experience topics. Larson et al [27] also reported that patient experience mainly reflects the interpersonal aspects of health care services. This is especially true for provider attitude, which has been universally discussed and demonstrated as the critical attribute of patient experience and satisfaction in previous research [26,28]. This study found that themes associated with positive and negative emotional feedback both focus on the health care professionals’ attitude (eg, “rude” and “friendly”); the rude behavior experienced by patients may trigger dissatisfaction, while friendly behavior can receive praise. Maramba et al [29] conducted a textual analysis of free-text comments from patients and reported that “rude” was significantly associated with a worse experience. The functional aspects include access to care, technical competence, efficacy of treatment, physical comfort, error in treatment, fairness of care, and coordination of care. Similar findings were found in previous studies [30]. Many topics frequently appeared in the traditional hospital-initiated surveys (eg, access to care and physical comfort). There were also some topics that were not typically addressed, such as technical competence, efficacy of treatment, error in treatment, fairness of care and coordination of care, indicating that health care organizations can use social media platforms to identify unexpected patient experience aspects that may not be viewed by hospitals [31].

As shown in Table 3, the theme distribution of the whole sample and sentiments indicate that the interpersonal aspects of health care services play an important role in shaping a good patient experience. However, in most cases, when health care organizations develop quality improvement activities, they always focus on functional aspects, such as access to care, technical competence, efficacy of treatment, and physical comfort. The authentic voices of patients on social media offer a precious opportunity for health care organizations to reshape their services. Therefore, the use of the public’s comments on social media platforms will eventually lead to a more patient-centred health care system that will improve on interpersonal aspects of care.

This study also revealed that negative sentiment posts represented the highest share of all posts. Previous research has also demonstrated that many health care-related tweets are negative [32-34], which may be partly explained by the fact that social media platforms often serve as outlets for individuals to voice complaints and negative experiences more readily than positive experiences. Compared with positive posts, negative posts tend to attract more responses and trigger more discussions [35], therefore having a greater impact on the health care system. The health care system should monitor the rise of negative voices regarding services.

The Weibo platform provides an online public sphere in which people can express and discuss their authentic opinions. However, despite its widespread use, Weibo users do not represent the Chinese population; younger and higher socioeconomic populations are the main user groups on Weibo. Therefore, future studies could explore patient experience using other social media platforms, such as Douyin (Beijing Weibo Vision Technology Co, Ltd), which may attract more older adult individuals. Additionally, dissatisfied patients are more likely to voice complaints and their negative experiences than satisfied patients. Therefore, a selection bias exists, and other sources of complementary data, such as patients’ free-text comments in surveys and semistructured interviews, are necessary to further understand patient experience. However, owing to the benefits of accessibility, flexibility, and anonymity, social media has become a ubiquitous tool that allows individuals to openly gather, communicate, and share real-time information and feedback about their interactions with the health care services. We thus suggest that this research provides a starting point for Chinese hospital administrators and clinicians in terms of how social media analysis can improve health care.

With the application of a mixed methods approach involving literature review, human annotation, and machine learning–based models, monitoring public disclosure about their interactions with hospitals on social media platforms can help to better understand gaps and potential opportunities for improving health care quality, serving as a supplemental source of data about patient experience. People tend to release negative posts on social media platforms, which could attract more responses and trigger more discussions. A wide range of patient experience themes were discussed, and the five most frequently discussed topics were “health care professionals’ attitude,” “access to care,” “communication, information, and education,” “technical competence,” and “efficacy of treatment,” highlighting the interpersonal and functional aspects of care.

Social media has become common place and allows individuals to openly gather, communicate, and share real-time information and feedback about their interactions with health care services. Hospital administrators and clinicians should consider the value of social media and pay attention to what patients and their family members are communicating on social media. To increase the utility of this information, a machine learning algorithm can be used for topic modeling of patients’ views about health care providers and their care.