This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.
Previous research shows that being a “digital native” or growing up in a digital environment does not necessarily lead to increased digital competencies, such as digital health literacy or evaluation of webpage quality.
This study showed how medical students searched for health information online, specifically the recommended testing for histamine intolerance, by comparing the use of various search engines (Google, Medisuch, and a website of the student’s choice) to find out more about search strategies in future health professionals. As Medisuch presents a qualitatively better search engine, we assumed that medical students using this search engine might find valid information faster on more reliable webpages, and might recommend the correct diagnostic steps for histamine intolerance to their patients more often than students using a generic search engine like Google.
Medical students in their third year of study were asked to find the relevant diagnostic steps of histamine intolerance online. They were randomly assigned to use one search engine: Google, their personal choice, or Medisuch. Their process of seeking information online was video recorded.
In total, 140 medical students participated in this study. The total number of webpages found did not differ among the groups (
The results indicate that medical students were able to find trustworthy health-related information online independent of the search engine used. Medical students that are digital natives seem to have proper internet skills and a knowledge of how to use them. They entered specific medical terms (evidence-based diagnostic steps) or names of reliable webpages (DocCheck) in the search engines to gain correct information. However, it remains to be seen if this behavior can be called true “digital literacy”.
The internet is omnipresent and has become the primary source of information for many [
Furthermore, it has been shown that falling in the category of “digital natives” [
One such approach involves introducing specific search engines that prefilter for the user and only show reliable results, such as the German webpage Medisuch [
To properly advise patients, medical students need to be generally aware of diagnostic steps; if they lack the knowledge, they should know how to find reliable information. As medical students are still learning, specific training can impact their attitude and behavior. This study compared various search engines to investigate how medical students search for health information online. The aim of this study was to track medical students’ searches for health information online and identify potentially weak strategies, which could be addressed specifically within the medical curriculum. We wanted to compare three different approaches and focus on the medical students’ findings on histamine intolerance. As Medisuch presents a qualitatively better search engine [
This study was performed in an explanatory cross-sectional manner. Third-year students from the Faculty of Medicine of the University Hospital Tuebingen, Germany, were recruited from their curricular courses in the Department of Internal Medicine VI. They had all received theoretical input on functional disorders and differential diagnoses with a focus on intolerances before participating in the study. Group sizes consisted of 8 to 16 students per teaching session. Students were taught by experienced physicians. Participation in the course was mandatory, but participation in the study was on a voluntary basis.
The study received ethical approval from the Ethics Committee of the Tuebingen Medical Faculty (443/2018BO2). All participants gave written informed consent. They did not receive a reimbursement for their participation.
Students of the university must attend a 2-week course on psychosomatic medicine in their third year. Every 2 weeks, a new group of approximately 12 to 15 students start the course. The topic is “somatoform disorders” on the third day of the course. Students received a case report involving a patient who consulted her physician due to abdominal pain. The patient in the fictitious case had already researched her health problem beforehand and asked specifically about potential causes such as problems with digestion of histamines. The patient wanted to know which steps had to be taken to rule out or confirm histamine intolerance. Each student had access to one stationary computer with connection to the internet via Internet Explorer or Firefox (both were installed and students were free to choose). Students were instructed to complete a worksheet about histamine intolerance by searching for related information online. They were randomly assigned to one of three groups: Google [
Process of study including teaching and assessment.
Students were asked about demographic variables such as age and gender, as well as experience with electronics such as possession of electronic devices and time spent online per day. They also rated their competence on the topic of histamine intolerance on a 6-point Likert scale (0 = not competent at all and 5 = highly competent).
Before starting the teaching session, students were asked to start Morae (TechSmith, Michigan) [
All Morae videos were coded according to this categorical system by experienced members of the research team. All webpages students had visited were listed alphabetically, and their quality was classified according to a rating sheet by two independent raters. In cases of a disagreement, a third independent rater was consulted.
Students answered specific questions about the search engine(s) used, keywords entered, and the number of webpages they consulted before making their diagnostic recommendation. Additionally, they rated the number of pages considered as trustworthy as well as the number of pages they would recommend to a patient. Trustworthy was defined as an evidence-based and certified webpage; thus, untrustworthy was defined as a nonevidence-based one such as an advertisement or self-made webpage by an individual. To get an idea on the effectiveness of medical students’ searches, the number of pages opened and the keywords needed to get to the desired answer were considered as relevant markers.
Students had to give specific recommendations on the next diagnostic steps. They were asked to choose one or more of the following options: antibody testing, assessment of diaminoxydase, nutritional diary, elimination diet, H2 breathing test, histamine testing, provocation test, and test of urine and feces. To pass, students needed to mark one or more of the following options: provocation test, nutritional diary, or elimination diet, as these were the diagnostic factors recommended by a clinical expert.
Statistical analysis was performed using SPSS 25 (IBM Corp, Armonk, New York). Mean values, associated SDs, frequencies, and percentages of relevant factors such as age and gender were calculated. To test possible relationships among the variables, Chi-squared tests were used. Analysis of variance (ANOVAs) were conducted to compare differences between mean values. In addition, the Pearson correlations were calculated. Beforehand, distribution of data was tested by the Kolmogorov-Smirnov test.
There were 140 students who took part in the survey, and the response rate was 83.3% (140 /168). The average age was 24.36 (SD 2.84) years. Females constituted 61.7% (85) of the participating students. There were 50 students randomly assigned to Google, 46 to Medisuch, and 44 to the free choice group (
There were no significant differences between the groups with regards to age (
Videos were successfully recorded from 125 students (44 from the Google group, 45 from the Medisuch group, and 36 from the free choice group). The remaining 15 recordings were lost due to technical reasons such as the recording or storing not working. The following data are based on these video recordings except those on diagnostic recommendation, which were based on the worksheet.
There was no significant difference between groups regarding the total number of webpages students considered before making their diagnostic choice. The same result was shown for the number of pages regarded as useful. However, students of the free choice group (mean 0.88, SD 0.79) reported significantly fewer pages as recommendable to patients than the other two groups (
Information seeking-behavior of students regarding the total number of webpages, the number of pages considered useful, and the number of pages considered recommendable for each group, as well as the means and SDs of the webpages are shown in
There were no significant differences between groups in regards to the number of keywords entered in the search field or the number of webpages accessed (
There was no significant difference regarding the amount of keywords and webpage names found by students for each group. Means and the corresponding SDs are shown in
There was a highly significant difference between groups in whether or not the students entered specific medical terminology in the search engine (
Information-seeking behavior of students regarding the total number of webpages.
Number of keywords and webpage names found by students for each group.
In total, 53 different webpages were accessed by the whole sample. Quality ratings showed that 34 pages fulfilled the criteria of a qualified webpage. Interrater-correlation after Cohen
There were significantly high Pearson correlations between the number of webpages and the number of reliable webpages for all three groups (Google:
There were no significant differences in the frequencies of trustworthy webpages found among the three groups with
Quotient of trustworthy or untrustworthy webpages found by students separated by group.
Independent of the search group, approximately 60% of students identified nutrition diary as a correct diagnostic step (
Diagnostic decision after online search.
Diagnostic step (correct or wrong) | Google (N=50) | Free choice (N=44) | Medisuch (N=46) | Chi-square ( |
|
|
|||||
|
yes | 2 (4%) | 1 (2%) | 7 (15%) | 6.84 (2) |
|
no | 48 (96%) | 43 (98%) | 39 (85%) | |
|
|||||
|
yes | 19 (38%) | 15 (34%) | 13 (28%) | 1.03 (2), |
no | 31 (62%) | 29 (66%) | 33 (72%) | ||
|
|||||
|
yes | 29 (58%) | 23 (52%) | 18 (39%) | 3.55 (2), |
no | 21 (42%) | 21 (48%) | 28 (61%) | ||
|
|||||
|
yes | 0 (0%) | 0 (0%) | 0 (0%) | N/Aa |
no | 50 (100%) | 44 (100%) | 46 (100%) | ||
|
|||||
|
yes | 7 (14%) | 4 (9%) | 7 (15 %) | 0.84 (2), |
no | 43 (86%) | 40 (91%) | 39 (85%) | ||
|
|||||
|
yes | 30 (60%) | 27 (61%) | 28 (61%) | 0.02 (2), |
no | 20 (40%) | 17 (39%) | 18 (39%) | ||
|
|||||
|
yes | 26 (52%) | 19 (43%) | 33 (72%) | 7.87 (2), |
no | 24 (48%) | 25 (57%) | 13 (28%) | ||
|
|||||
|
yes | 25 (50%) | 21 (48%) | 22 (48%) | 0.06 (2), |
no | 25 (50%) | 23 (52%) | 24 (52%) |
aNot applicable.
In this explanatory study, we examined medical students’ information-seeking behavior when assigned to use a generic search engine or a search engine of high quality. The hypothesis that students of the qualitatively better search engine would find the correct diagnostic steps for histamine intolerance more often was not fully supported. The Medisuch group significantly identified the elimination diet as a correct diagnostic step more frequently; however, this was not the case for the nutrition diary or the provocation test. The students of the Google and free choice groups reported the antibody detection as wrong more often than the Medisuch group. Furthermore, students of all three groups identified reliable webpages, which indicates that they do have internet skills that allow them to identify reliable information online. They were able to search for information online without navigation issues by entering specific webpages (like Flexicon) or medical terminology (diagnostic steps) in order to find reliable results [
A general question arises when viewing these results: Is entering specific terms or webpages into a generic search engine part of “digital literacy”? Many students searched at DocCheck [
Finding health information online still includes the risk of being misinformed by unreliable information or information providers, which ultimately affects patients [
One limitation of this study might be that search results and choices of relevant information is only one step out of many. It might be relevant to both track students’ search behavior and assess their cognitive processes preceding the search, such as defining the problem, choosing a certain source of information, or formulating the search strategies [
This study showed that medical students are able to search and find relevant medical information online regardless of the search engine used, and thus, this study confirms previous findings of medical students having internet skills in a professional sense. Future studies could focus on how to best integrate these internet skills into the medical curriculum. Furthermore, it needs to be determined if the online behavior of the students involved can be considered proper digital literacy. The next step could also focus on patients and their online search strategies.
analysis of variance
Health On the Net Foundation Code of Conduct
information technology.
We thank all the lecturers of the Department of Psychosomatic Medicine and Psychotherapy. We would also like to thank our study assistant Lisa Maria Wiesner for her help. We also acknowledge the support with financing publication fees by Deutsche Forschungs-gemeinschaft and Open Access Publishing Fund of the University of Tuebingen
None declared.