Published on 21.03.19 in Vol 7, No 1 (2019): Jan-Mar
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/13008, first published Dec 17, 2018.
Completeness of Electronic Dental Records in a Student Clinic: Retrospective Analysis
Background: A well-designed, adequately documented, and properly maintained patient record is an important tool for quality assurance and care continuity. Good clinical documentation skills are supposed to be a fundamental part of dental student training.
Objective: The goal of this study was to assess the completeness of electronic patient records in a student clinic.
Methods: Completeness of patient records was assessed using comparative review of validated cases of alveolar osteitis treated between August 2011 and May 2017 in a student clinic at Columbia University College of Dental Medicine, New York, USA. Based on a literature review, population-based prevalence of nine most frequently mentioned symptoms, signs, and treatment procedures of alveolar osteitis was identified. Completeness of alveolar osteitis records was assessed by comparison of population-based prevalence and frequency of corresponding items in the student documentation. To obtain all alveolar osteitis cases, we ran a query on the electronic dental record, which included all cases with diagnostic code Z1820 or any variation of the phrases “dry socket” and “alveolar osteitis” in the notes. The resulting records were manually reviewed to definitively confirm alveolar osteitis and to extract all index items.
Results: Overall, 296 definitive cases of alveolar osteitis were identified. Only 22% (64/296) of cases contained a diagnostic code. Comparison of the frequency of the nine index categories in the validated alveolar osteitis cases between the student clinic and the population showed the following results: severe pain: 94% (279/296) vs 100% (430/430); bare bone/missing blood clot: 27% (80/296) vs 74% (35/47) to 100% (329/329); malodor: 7% (22/296) vs 33%-50% (18/54); radiating pain to the ear: 8% (24/296) vs 56% (30/54); lymphadenopathy: 1% (3/296) vs 9% (5/54); inflammation: 14% (42/296) vs 50% (27/54); debris: 12% (36/296) vs 87% (47/54); alveolar osteitis site noted: 96% (283/296) vs 100% (430/430; accepted documentation requirement); and anesthesia during debridement: 77% (20/24) vs 100% (430/430; standard of anesthetization prior to debridement).
Conclusions: There was a significant discrepancy between the index category frequency in alveolar osteitis cases documented by dental students and in the population (reported in peer-reviewed literature). More attention to clinical documentation skills is warranted in dental student training.
JMIR Med Inform 2019;7(1):e13008
A clinical record is a fundamental part of patient care delivery . Its completeness is important for many reasons. A record’s main purpose is to serve as a means of communication among providers themselves and between providers and their colleagues [ ]. Clinical decision support is dependent on accurate and complete dental records [ ], which also aid in the evaluation of a patient’s care [ ]. In the event of a lawsuit, the record serves as evidence [ ] and its contents are necessary to determine whether the diagnosis and treatment met appropriate standards [ ]. With the expansion of dental informatics applications, it is even more essential to have a complete record in order to ensure proper analysis and results in outcomes research [ ]. However, there are known issues with the completeness of electronic medical records (EMRs).
In the past decade, electronic patient records became a ubiquitous part of dental care delivery . Oral health data accumulated in the process of clinical care represent a rich and readily available recourse for scientific investigation and data analytics [ ]. Recent analysis of electronic dental records (EDRs) helped identify predictors of implant survival [ ] as well as the prevalence and risk factors of peri-implantitis [ ]. Application of machine learning techniques [ ] and temporal analytics [ ] resulted in new opportunities for knowledge discovery and predictive analytics [ ]. With the increasing use of EMRs as an important resource for scientific discovery, potential barriers for secondary analysis of EMR have been recognized [ ]. Completeness of EMR is one of the most frequently discussed issues that may limit the use of EMR data for clinical and population health research [ ]. This issue was reported to be particularly relevant in the evaluation of medical student documentation [ ]; however, no systematic assessment of the completeness of EMR in dental student clinics has been performed.
Since the widespread introduction of EMRs, numerous studies have indicated gaps in documentation. A study at the University of Michigan examined whether there were differences in the reported eye symptoms between EMRs and eye symptom questionnaires that patients fill out. Exact agreement was found in only 23.5% (38/162) of cases. In cases where patients reported three or more symptoms, data from the eye symptom questionnaire always varied from the EMR data . A similar study at the Mayo Clinic compared symptoms of chest pain, dyspnea, and cough between information forms patients received prior to the appointment and EMRs, for the purpose of identifying stable angina pectoris. They found that the two documents had varying levels of positive agreement (ratio): 74 for chest pain, 70 for dyspnea, and 63 for cough [ ]. Researchers at Duke University analyzed records for completeness for quality purposes, but concluded that improper documentation for colorectal cancer impeded their ability to accurately calculate patient performance. Of the 499 patients eligible for the analysis, only 66 had sufficient documentation. In addition, only 86% (427/499) of EMRs confirmed a diagnosis, 29% (143/499) were missing the age, and only 38% (188/499) stated the TMN stage [ ].
Other studies indicated that the quality of documents in EDRs may be suboptimal [, ]. However, there is a lack of systematic studies on the completeness of EDRs. The goal of this project was to review the documentation quality of EDRs in a dental school clinic.
EDRs of patients examined at Columbia University’s College of Dental Medicine were analyzed.
The study was a retrospective analysis of EDRs from patients with dry sockets diagnosed at Columbia University’s College of Dental Medicine between August 2011 and May 2017. We ran a query on the college’s database to find EDRs containing diagnostic code Z1820, the phrases “dry socket” and “alveolar osteitis,” or a variation of those two phrases. All queries were performed using structured query language in the Oracle database containing data from an EDR system called axiUm (Exan Group, Las Vegas, NV).
For the purpose of this study, a dry socket was defined by a diagnostic Z code at the initial encounter. Alternatively, it was defined by the presence of a key word in a note combined with clinical evidence and explicit documentation of the patient with alveolar osteitis or dry socket. Z codes are represented by a list of diagnostic codes and terms developed for use with EDR, as previously described .
After reviewing the literature on dry socket [- ], we compiled a list of 17 criteria related to dry socket. Five were related to treatment: curettage, irrigation, anesthesia, intra-alveolar medication, and medication. Three were symptoms: pain, radiating pain, and tenderness on palpation. Nine were signs: lack of blood clot, malodor, low-grade fever, bare bone, lymphadenopathy, pus, erythema, inflamed gingiva/socket, and debris. The last category was the socket site of dry socket. Severe pain was a necessary symptom for dry socket diagnosis [ , ].
A dental student reviewed each of the query results to confirm the presence of a dry socket. The cases were also analyzed for the presence of any of the 17 criteria mentioned above. Pertinent positive and negative results for dry socket criteria were both recorded for further analysis. The student then reviewed the available literature on dry socket statistics to determine the prevalence of symptoms in the literature. Both the positive and negative criteria were compared to the baseline figures from this literature.
A total of 150 records with diagnostic Z codes were identified, and another 787 records were identified by searching for a mention of dry socket or alveolar osteitis in the notes (). Both queries resulted in a number of duplicate cases, which were removed. The dry socket/alveolar osteitis dataset had 11 duplicate cases, and the Z code data set contained 13 duplicate cases. In addition, 101 of the remaining cases overlapped (ie, had both diagnostic Z codes and dry socket or alveolar osteitis in the notes). These overlapping cases were not counted twice in the study. After removing duplicates and overlapping cases, the final number of combined cases was 812.
Cases were then categorized based on the dental record content (). An explicit diagnosis of dry socket was found in 278 EMRs. Another 18 EMRs had a diagnostic Z code and the patient received treatment for dry sockets, but the dentist never explicitly stated that the patient presented with a dry socket; these were also included as definitive dry sockets for the purpose of this study. Totally, 296 cases (of 812, 36.5%) were categorized as definitive dry sockets, and 108 cases (of 812, 13.3%) of possible dry sockets had notes of treatment of dry sockets, but the dentist did not include a diagnostic code or any definitive diagnoses of dry socket in the note. This misdocumentation is important, but could not be included in the data, as there is no way to confirm the patient had a dry socket from the records. The remaining cases were classified as follows:
- Follow-up treatment of both definitive dry sockets and possible dry sockets (226/812, 27.8%).
- Cases that were definitively classified as nondry sockets: These patients’ records were included in the query because they contained the key word to rule out the diagnosis, history of previous dry socket, mention of dry socket paste, or similar reference (117/812, 14.4%).
- Referrals to the oral surgery clinic: Such referrals for further evaluation were also a common finding. The referrals were not followed up to determine if these patients were later seen in Columbia (43/812, 5.3%). An additional segment of charts belonged to patients whose only encounter populated in our search was for a follow-up in our clinic for a dry socket (19/812, 2.3%). The final and smallest group of charts contained a Z code, but no note was found in the system for the encounter. This may have been a kink in the system or oversight by the provider (3/812, 0.4%).
Definitive dry socket cases were analyzed for correctness of EMR documentation. The demographics of patients of the 296 records are outlined in. Women (196/296, 66.3%) were affected by dry socket more frequently than men. The age group with the highest frequency of dry socket included adults between the ages of 20 and 39 years (146/296, 49.3%). demonstrates the extraction characteristics of the definitive dry sockets studied. Molars were the most frequent sites for dry socket, with the third molar represented in 37.5% of the cases (111/296). Tooth extraction was performed under the supervision of faculty from the Department of Oral and Maxillofacial Surgery in 82.8% (245/296) cases. About half of the tooth extraction procedures were carried out by predoctoral students (145/296, 46.3%) and the remaining extractions were performed by postdoctoral students, the majority of whom were represented by residents of the Department of Oral and Maxillofacial Surgery. A similar pattern was observed for location and provider type for the tooth extraction follow-up visits during which dry socket was diagnosed and treated.
|Definitive dry socket||Diagnostic code with signs, symptoms, and treatment, or explicit clinical diagnosis||296|
|Dry socket follow-up||Follow-up to case classified as definitive dry socket, possible dry socket, or other treatment||230|
|Possible dry socket||Presented with symptoms and treated for dry socket, but no explicit diagnosis or Z code||108|
|Referral||Referred to the Department of Oral and Maxillofacial Surgery||43|
|Not dry socket||Diagnosis excluded by provider, history of previous dry socket in notes, or similar finding||117|
|Dry socket treatment follow-up or missing other documentation||Only the follow-up on previous treatment was contained in the query||15|
|Incomplete notes||Notes were not populated but contained the Z code||3|
|African American||23 (7.8)|
|Not disclosed||1 (0.3)|
Two general types of misdocumentation were encountered and calculated while analyzing the 296 cases of definitive dry socket (). The first was any missing gross documentation necessary for diagnosis of dry socket to support the clinical diagnosis of this condition and to provide documentation necessary for patient follow-up and care continuity. These included documented pain, visible bone, or lack of blood clot. We also included the missing socket position when diagnosing a dry socket. Of the 296 cases, 220 (74.3%) were missing at least one of the abovementioned factors. The second type of misdocumentation was the number of cases without a diagnostic Z code. Of the 296 cases, 232 (78.4%) did not contain the diagnostic code. All cases analyzed were documented after the Columbia University College of Dental Medicine implemented diagnostic codes.
|Third molar||111 (37.5)|
|First and second molars||125 (42.2)|
|Department of extraction treatment|
|Department of Oral and Maxillofacial Surgery||245 (82.8)|
|Provider group for extraction|
|Department of Oral and Maxillofacial Surgery||65 (24.1)|
|General practice residency||23 (8.5)|
|Advanced education in general dentistry||18 (6.7)|
|Location of dry socket treatment|
|Department of Oral and Maxillofacial Surgery||242 (81.8)|
|Advanced education in general dentistry||17 (5.7)|
|Provider groups for dry socket diagnosis|
|Department of Oral and Maxillofacial Surgery||49 (16.55)|
|General practice residency||42 (14.19)|
|Advanced education in general dentistry||26 (8.78)|
Possible misdocumentation was also found in the remaining 516 cases that were not labeled as definitive dry socket. There were 108 cases of patients who presented with symptoms of dry sockets and received treatment for dry sockets. However, their charts did not include a Z code or any definitive diagnoses by the provider. Although many of these patients fulfilled the requirements to be classified with a dry socket, they could not be included in the analysis because they lacked a basic documented confirmation. As previously stated, 206 cases were classified as follow-up appointments to initial treatment of definitive dry socket. From these cases, another category of misdocumentation emerged: 16 of the cases contained a diagnostic Z code, although the actual diagnosis and initial treatment of the dry socket did not contain this code. The latter type of misdocumentation does not have any direct consequences, but reveals a reluctance to use dry socket diagnosis or lack of education regarding diagnostic codes.
The final group of misdocumentation was found in the 19 cases categorized as follow-up to the initial appointment for dry socket, in which the patient’s first documented encounter with the word dry socket or alveolar osteitis was in the follow-up. The clinical note in 14 of these 19 cases indicated that the patient was previously seen in the clinic or emergency room to treat the dry socket. The implication is that these patients were seen and treated for a dry socket previously in the clinic, but the earlier documentation lacked any of the three search terms (dry socket, alveolar osteitis, and Z code) used to identify cases for this analysis.
|Type of misdocumentation||n (%)|
|Missing documentation necessary for diagnosis (pain, bone visible, or socket position)||220 (74.3)|
|No Z codes||232 (78.4)|
|Treated for dry socket and had symptoms, but no definitive diagnosis||108 (36.5)a|
|Dry socket follow-up is Z code but no initial encounter||16 (5.4)a|
|Dry socket follow-up was searchable but not the original case||14 (4.7)a,b|
aThese cases were categorized for purposes of misdocumentation but were not included in the analysis of misdocumentation of definitive dry sockets.
bIncludes 2 from the emergency room.
Definitive dry socket signs, symptoms, and treatments completeness compared to the best values available in the literature are presented in. Presence of severe pain was considered a necessary symptom for dry socket diagnosis. The clinical notes only contained pain as a symptom in 94% (279/296) of the cases. Bare bone [ ], open socket [ ], or missing blood clot [ ] was mentioned in 74% (35/47) to 100% (329/329) of the dry socket cases. In our clinic, any of these related terms were mentioned in 27% (80/296) of cases. In the 296 definitive dry socket cases, the word “bone” was only found in 50 cases and “clot” was found only 25 times; some of these cases overlapped. These terms are germane and necessary for a diagnosis of dry socket, and the lack of documentation is troublesome. Malodor [ ] was documented in 33% (18/54) to >50% of the dry socket cases in the literature [ ]; however, its documentation was only present in 7% (22/296) of our cases. Radiating pain toward the ear was present in 56% (30/54) of the cases in the literature [ ], but was mentioned only in 7% (24/296) of our cases (documentation of radiating pain alone was included in this number, as we did not require ear or side of face to be documented). Lymphadenopathy [ ] was present in 9% (5/54) of the cases in the literature [ ], but in only 1% (3/296) of our cases. Inflammation was present in 50% (27/54) of the cases in the literature [ ]; however, in the dental record, only 14% (42/296) of cases mentioned inflammation and 15% (44/296) (with some overlap with inflammation cases) mentioned erythema.
A comparison between the notes of predoctoral dental students and postdoctoral faculty and residents is outlined in. The nine selected signs, symptoms, and other documentation related to dry socket diagnosis were analyzed. Predoctoral students were more likely to properly document the location of the dry socket, while postdoctoral students were more likely to document malodor, presence of debris, and anesthesia administration during socket debridement. Misdocumentation of the remaining categories was equal (both groups were within 1% of one another for the other five categories) as compared to the expected values ( ).
|Characteristic||Predoctorate (n=143), n (%)||Postdoctorate (n=151), n (%)|
|Severe pain||135 (95)||144 (94)|
|Bare bone/missing blood clot||31 (22)||49 (32)|
|Malodor||6 (4)||16 (10)|
|Radiating pain to ear||12 (8)||12 (8)|
|Lymphadenopathy||1 (1)||2 (1)|
|Inflammation||21 (15)||21 (14)|
|Debris||12 (8)||24 (16)|
|Dry socket site noted||140 (98)||143 (93)|
|Anesthesia mentioned during debridement||2a (50)||18b (82)|
aTotal for this group is 4 students.
bTotal for this group is 22 faculty and residents.
Our ultimate finding was that misdocumentation occurs in EDRs to varying degrees within a dental school clinic. The dental records reviewed lack many signs and symptoms that are necessary and expected to be recorded for a diagnosis of dry socket. This misdocumentation was prevalent in the notes of both predoctoral and postdoctoral students. The major limitation of this study, as with other electronic record retrospective studies, was our ability to confirm the diagnosis of dry socket. We also could not confirm whether patients actually had the symptoms omitted. We were forced to rely on the information provided by the documenter.
Evaluation of completeness of symptom documentation in this study followed recent guidance on EMR data quality–assessment methodology  that promulgates the use of validated population-based prevalence as a gold standard [ - ]. Following these guidelines, the comparison between the validated symptom prevalence and symptom frequency found in our EDR review was used to identify completeness of symptom documentation in EDR in this study. This approach has been successfully used in a number of previous studies to identify completeness of smoking status records [ ], obesity reporting [ ], hypertension records [ ], and depression prevalence [ ]. To minimize bias toward inflating misdocumentation rates, whenever several verified estimates of a population-based frequency for a particular symptom were available, a comparison was made between the lowest population-based frequency and the frequency found in the EDR, as previously described [ - ].
Our findings of the demographics and tooth extraction characteristics among patients with dry socket are congruent with previous reports. As previously determined, women are more likely to have dry socket then men [, ]. The tooth position is most likely to be a third molar compared to any other individual area [ ]. People in their 20s and 30s are at a higher risk for alveolar osteitis [ ]. Two major types of misdocumentation that were found and analyzed in this study have been previously mentioned in the literature, including lack of supporting documentation for clinical diagnosis [ , ] and absence of appropriate diagnostic codes [ ].
Analysis of characteristics of providers who performed tooth extraction and dry socket diagnosis confirmed the external validity  of our study sample, as the resulting characteristics accurately reflected routine dental care delivery patterns for these types of procedures occurring in the student clinic. The majority of the cases were carried out under supervision of a preceptor from the Department of Oral and Maxillofacial Surgery. About half of the extractions and dry socket diagnoses were carried out by predoctoral students, and the rest were performed by postdoctoral residents of whom approximately half were represented by residents of the Department of Oral and Maxillofacial Surgery. The fact that the predoctoral and postdoctoral dental surgeons were equally presented in our analysis supported the unbiased comparison between documentation quality of predoctoral and postdoctoral students.
The results of documentation completeness in EDRs in this study are corroborated by the following reports. In Minnesota, a discrepancy was found between the American Dental Association’s recommendation for dental record accuracy and the actual accuracy in dental practices . In Finland, researchers observed a discrepancy between the quality of treatment a dentist believed he/she provided and the treatment the patient actually received, as contained in the EDR [ ]. EMRs have also been shown to have issues with accuracy [ ]. In an adult cardiology clinic, researchers discovered “very poor” completeness values for signs such as chest pain and shortness of breath [ ]. In a systemic review of EMR completeness in primary care, Thiru [ ] found that records of diagnoses with clear clinical criteria had a higher rate of completeness than those without clear criteria. This is relevant to dry socket, a diagnosis with unclear criteria. A trauma center study also found incompleteness of certain categories in the EMR [ ]. Similarly, a study conducted with inpatient records at Menelik II Referral Hospital, a government hospital in Addis Ababa, Ethiopia, found “low” EMR completeness compared to the expected standard of 100% [ ]. Legal Medical Record Standards stated that “Each Medical Record shall contain sufficient, accurate information to identify the patient, support the diagnosis, justify the treatment, document the course and results, and promote continuity of care among health care providers” [ ]. Following this simple instruction can vastly improve the delivery of care.
To resolve the issues with incomplete or inaccurate records, dental education should emphasize more on proper documentation and ensure its incorporation into the clinic routine. Documenters need to remember that their records are not simply for their own convenience, but may serve legal, research-related, or forensic purposes [, ]. To this end, each record must be complete, without implicit assumptions, and follow a method that makes it easily accessible to any reader. Thierer [ ] found an improvement in EDR accuracy by incorporating an in-service intervention for faculty members and a Moodle site course on documentation for students.
Our findings have important implications for future research that uses EMR data. Better understanding of the potential limitations of electronic health record data use promotes fidelity and reproducibility of secondary data analysis . A variety of approaches are being implemented to address the potential limitations of EMR data [ ] such as deep learning techniques [ ] for imputing missing data, symbolic operations for time interval analytics [ ], and calibration to reduce measurement error in prevalence estimates based on EMR data [ ]. A growing number of studies employ common data models combined with cross-linked semantic ontologies to harmonize EMR data [ ] and confirm with the Findability, Accessibility, Interoperability, and Reusability principles [ ].
We believe there are three sequential steps necessary for improving EDRs. The first step is additional training predoctoral and postdoctoral students on the importance of note comprehensiveness. As per a systemic review by the Accreditation Council for Continuing Medical Education, a live intervention with interactive techniques is the most effective way to change a physician’s behavior to influence patient outcomes . Interventions should therefore be constructed accordingly. The next step is adding disease- and condition-specific worksheets to the EDR. The worksheets should contain categories pertinent to the specific diagnosis, with drop-down boxes for the practitioner to complete. This prevents the inadvertent omission of crucial categories. The last step is adding a clinical decision support tool to EDRs. The tool embodies evidence-based dentistry, an approach being adopted by an increasing number of dental schools and practitioners. However, this tool operates properly only if practitioners enter complete and accurate data into EDRs in a way that computers can easily analyze. Thus, strict compliance with the first two steps is critical. This will result in refined EDRs, which can potentially lead to superior and safer delivery of care at a lower cost [ ]. Although some information in EDRs may seem largely irrelevant, the EDR is a critical depository of data, with limitless research possibilities. If properly executed, it may improve diagnoses, treatment, and dentistry as a whole.
Conflicts of Interest
- Devadiga A. What's the deal with dental records for practicing dentists? Importance in general and forensic dentistry. J Forensic Dent Sci 2014 Jan;6(1):9-15 [FREE Full text] [CrossRef] [Medline]
- Ho Y, Gadd CS, Kohorst KL, Rosenbloom ST. A qualitative analysis evaluating the purposes and practices of clinical documentation. Appl Clin Inform 2014 Dec;5(1):153-168 [FREE Full text] [CrossRef] [Medline]
- Wright A, Phansalkar S, Bloomrosen M, Jenders RA, Bobb AM, Halamka JD, et al. Best practices in clinical decision support: the case of preventive care reminders. Appl Clin Inform 2010 Dec;1(3):331-345 [FREE Full text] [CrossRef] [Medline]
- American Dental Association. 2010. Dental Records URL: http://www.ada.org/~/media/ADA/Public%20Programs/Files/MPRG_Dental_Records.pdf?la=en [accessed 2019-03-18] [WebCite Cache]
- Mangalmurti S, Murtagh L, Mello M. Medical malpractice liability in the age of electronic health records. N Engl J Med 2010 Nov 18;363(21):2060-2067. [CrossRef] [Medline]
- Acharya A, Shimpi N, Mahnke A, Mathias R, Ye Z. Medical care providers' perspectives on dental information needs in electronic health records. J Am Dent Assoc 2017 Dec;148(5):328-337. [CrossRef] [Medline]
- Acharya A, Schroeder D, Schwei K, Chyou P. Update on electronic dental record and clinical computing adoption among dental practices in the United States. Clin Med Res 2017 Dec;15(3-4):59-74 [FREE Full text] [CrossRef] [Medline]
- Gordon S, Camargo G, Mejia G, Sutherland J. Use of the dental electronic health record for research: Assessing demographic and oral health characteristics data for clinic patients. J Dent Educ 2018 Dec;82(12):1249-1257. [CrossRef] [Medline]
- Hickin M, Shariff J, Jennette P, Finkelstein J, Papapanou P. Incidence and Determinants of Dental Implant Failure: A Review of Electronic Health Records in a U.S. Dental School. J Dent Educ 2017 Oct;81(10):1233-1242 [FREE Full text] [CrossRef] [Medline]
- Kordbacheh Changi K, Finkelstein J, Papapanou P. Peri-implantitis prevalence, incidence rate, and risk factors: A study of electronic health records at a U.S. dental school. Clin Oral Implants Res 2019 Feb 15. [CrossRef] [Medline]
- Yoon S, Choi T, Odlum M, Mitchell D, Kronish I, Davidson K, et al. Machine Learning to Identify Behavioral Determinants of Oral Health in Inner City Older Hispanic Adults. Stud Health Technol Inform 2018;251:253-256 [FREE Full text] [Medline]
- Shknevsky A, Shahar Y, Moskovitch R. Consistent discovery of frequent interval-based temporal patterns in chronic patients' data. J Biomed Inform 2017 Nov;75:83-95 [FREE Full text] [CrossRef] [Medline]
- Wu J, Roy J, Stewart W. Prediction modeling using EHR data: challenges, strategies, and a comparison of machine learning approaches. Med Care 2010 Jun;48(6 Suppl):S106-S113. [CrossRef] [Medline]
- Verheij R, Curcin V, Delaney B, McGilchrist M. Possible Sources of Bias in Primary Care Electronic Health Record Data Use and Reuse. J Med Internet Res 2018 May 29;20(5):e185 [FREE Full text] [CrossRef] [Medline]
- Weiskopf N, Hripcsak G, Swaminathan S, Weng C. Defining and measuring completeness of electronic health records for secondary use. J Biomed Inform 2013 Oct;46(5):830-836 [FREE Full text] [CrossRef] [Medline]
- Seo J, Kong H, Im S, Roh H, Kim D, Bae H, et al. A pilot study on the evaluation of medical student documentation: assessment of SOAP notes. Korean J Med Educ 2016 Jun;28(2):237-241 [FREE Full text] [CrossRef] [Medline]
- Valikodath NG, Newman-Casey PA, Lee PP, Musch DC, Niziol LM, Woodward MA. Agreement of Ocular Symptom Reporting Between Patient-Reported Outcomes and Medical Records. JAMA Ophthalmol 2017 Mar 01;135(3):225-231 [FREE Full text] [CrossRef] [Medline]
- Pakhomov S, Jacobsen S, Chute C, Roger V. Agreement between patient-reported symptoms and their documentation in the medical record. Am J Manag Care 2008 Aug;14(8):530-539 [FREE Full text] [Medline]
- Abernethy AP, Herndon JE, Wheeler JL, Rowe K, Marcello J, Patwardhan M. Poor documentation prevents adequate assessment of quality metrics in colorectal cancer. J Oncol Pract 2009 Jul;5(4):167-174 [FREE Full text] [CrossRef] [Medline]
- Kopsala JT. WiredSpace. 2014. Quality Assurance and Dental Patient Records URL: http://wiredspace.wits.ac.za/handle/10539/14625 [accessed 2019-03-18] [WebCite Cache]
- White JM, Kalenderian E, Stark PC, Ramoni RL, Vaderhobli R, Walji MF. Evaluating a dental diagnostic terminology in an electronic health record. J Dent Educ 2011 May;75(5):605-615 [FREE Full text] [Medline]
- Singh V, Pal US, Singh R, Soni N. Honey a sweet approach to alveolar osteitis: A study. Natl J Maxillofac Surg 2014 Jan;5(1):31-34 [FREE Full text] [CrossRef] [Medline]
- Soni N, Singh V, Mohammad S, Singh RK, Pal US, Singh R, et al. Effects of honey in the management of alveolar osteitis: A study. Natl J Maxillofac Surg 2016;7(2):136-147 [FREE Full text] [CrossRef] [Medline]
- Baqain Z, Karaky AA, Sawair F, Khraisat A, Khaisat A, Duaibis R, et al. Frequency estimates and risk factors for postoperative morbidity after third molar removal: a prospective cohort study. J Oral Maxillofac Surg 2008 Nov;66(11):2276-2283. [CrossRef] [Medline]
- MacGregor A. Aetiology of dry socket: A clinical investigation. British Journal of Oral Surgery 1968 Jan;6(1):49-58. [CrossRef]
- Hedström L. Dry socket--an evaluation of 163 cases of the surgical removal of mandibular wisdom teeth. Tandlakartidningen 1987 Mar;79(6):298-303. [Medline]
- Blum I. Contemporary views on dry socket (alveolar osteitis): a clinical appraisal of standardization, aetiopathogenesis and management: a critical review. Int J Oral Maxillofac Surg 2002 Jun;31(3):309-317. [CrossRef] [Medline]
- Bortoluzzi M, Manfro R, De Déa BE, Dutra T. Incidence of dry socket, alveolar infection, and postoperative pain following the extraction of erupted teeth. J Contemp Dent Pract 2010 Jan 01;11(1):E033-E040. [Medline]
- Zozus M, Hammond W, Richesson R. NIH Collaboratory. 2014. Phenotypes, Data Standards, and Data Quality Core URL: https://www.nihcollaboratory.org/Documents/Hammond_Zozus_2014-0816-PDSDQCore_update.pdf [accessed 2019-03-18] [WebCite Cache]
- Chen H, Hailey D, Wang N, Yu P. A review of data quality assessment methods for public health information systems. Int J Environ Res Public Health 2014 May 14;11(5):5170-5207 [FREE Full text] [CrossRef] [Medline]
- Faulconer E, de Lusignan S. An eight-step method for assessing diagnostic data quality in practice: chronic obstructive pulmonary disease as an exemplar. Inform Prim Care 2004;12(4):243-254 [FREE Full text] [Medline]
- Booth H, Prevost A, Gulliford M. Validity of smoking prevalence estimates from primary care electronic health records compared with national population survey data for England, 2007 to 2011. Pharmacoepidemiol Drug Saf 2013 Dec;22(12):1357-1361 [FREE Full text] [CrossRef] [Medline]
- McVeigh K, Newton-Dame R, Chan P, Thorpe L, Schreibstein L, Tatem K, et al. Can Electronic Health Records Be Used for Population Health Surveillance? Validating Population Health Metrics Against Established Survey Data. EGEMS (Wash DC) 2016;4(1):1267 [FREE Full text] [CrossRef] [Medline]
- Perlman S, McVeigh K, Thorpe L, Jacobson L, Greene C, Gwynn R. Innovations in Population Health Surveillance: Using Electronic Health Records for Chronic Disease Surveillance. Am J Public Health 2017 Dec;107(6):853-857. [CrossRef] [Medline]
- Thorpe L, McVeigh K, Perlman S, Chan P, Bartley K, Schreibstein L, et al. Monitoring Prevalence, Treatment, and Control of Metabolic Conditions in New York City Adults Using 2013 Primary Care Electronic Health Records: A Surveillance Validation Study. EGEMS (Wash DC) 2016;4(1):1266 [FREE Full text] [CrossRef] [Medline]
- Akinbami B, Godspower T. Dry socket: incidence, clinical features, and predisposing factors. Int J Dent 2014;2014:796102 [FREE Full text] [CrossRef] [Medline]
- Kolokythas A, Olech E, Miloro M. Alveolar osteitis: a comprehensive review of concepts and controversies. Int J Dent 2010;2010:249073 [FREE Full text] [CrossRef] [Medline]
- Abu Younis MH, Abu Hantash RO. Dry socket: frequency, clinical picture, and risk factors in a palestinian dental teaching center. Open Dent J 2011 Feb 07;5:7-12 [FREE Full text] [CrossRef] [Medline]
- Sweet J, Butler D. Predisposing and operative factors: effect on the incidence of localized osteitis in mandibular third-molar surgery. Oral Surg Oral Med Oral Pathol 1978 Aug;46(2):206-215. [Medline]
- Steckler A, McLeroy K. The importance of external validity. Am J Public Health 2008 Jan;98(1):9-10. [CrossRef] [Medline]
- Osborn J, Stoltenberg J, Newell K, Osborn S. Adequacy of dental records in clinical practice: a survey of dentists. J Dent Hyg 2000;74(4):297-306. [Medline]
- Helminen S, Vehkalahti M, Murtomaa H. Dentists' perception of their treatment practices versus documented evidence. Int Dent J 2002 Apr;52(2):71-74. [Medline]
- Majeed A, Car J, Sheikh A. Accuracy and completeness of electronic patient records in primary care. Fam Pract 2008 Aug;25(4):213-214. [CrossRef] [Medline]
- Oksuzyan A, Demirchyan A, Thompson M. Evaluation of Medical Records Completeness in the Adult Cardiology Clinic at Nork Marash Medical Center. Yerevan, Armenia: American University of Armenia; 2003 Aug. URL: http://aua.am/chsr/UserFiles/File/Anahit/Evaluation of SEF Completeness_Eng.PDF [accessed 2019-03-16] [WebCite Cache]
- Thiru K, Hassey A, Sullivan F. Systematic review of scope and quality of electronic patient record data in primary care. BMJ 2003 May 17;326(7398):1070 [FREE Full text] [CrossRef] [Medline]
- de Mul M, Berg M. Completeness of medical records in emergency trauma care and an IT-based strategy for improvement. Med Inform Internet Med 2007 Jun;32(2):157-167. [CrossRef] [Medline]
- Tola K, Abebe H, Gebremariam Y, Jikamo B. Improving Completeness of Inpatient Medical Records in Menelik II Referral Hospital, Addis Ababa, Ethiopia. Advances in Public Health 2017;2017:1-5. [CrossRef]
- University of California. 2008. Legal Medical Record Standards URL: http://policy.ucop.edu/doc/1100168/LegalMedicalRecord [accessed 2019-03-16] [WebCite Cache]
- Waleed P, Baba F, Alsulami S, Tarakji B. Importance of dental records in forensic dental identification. Acta Inform Med 2015 Feb;23(1):49-52 [FREE Full text] [CrossRef] [Medline]
- Thierer TE, Delander KA. Improving Documentation, Compliance, and Approvals in an Electronic Dental Record at a U.S. Dental School. J Dent Educ 2017 Apr;81(4):442-449 [FREE Full text] [CrossRef] [Medline]
- Farmer R, Mathur R, Bhaskaran K, Eastwood SV, Chaturvedi N, Smeeth L. Promises and pitfalls of electronic health record analysis. Diabetologia 2018 Dec;61(6):1241-1248. [CrossRef] [Medline]
- Casey J, Schwartz B, Stewart W, Adler N. Using Electronic Health Records for Population Health Research: A Review of Methods and Applications. Annu Rev Public Health 2016;37:61-81. [CrossRef] [Medline]
- Yoon S, Odlum M, Lee Y, Choi T, Kronish I, Davidson K, et al. Applying Deep Learning to Understand Predictors of Tooth Mobility Among Urban Latinos. Stud Health Technol Inform 2018;251:241-244 [FREE Full text] [Medline]
- Moskovitch R, Polubriaginof F, Weiss A, Ryan P, Tatonetti N. Procedure prediction from symbolic Electronic Health Records via time intervals analytics. J Biomed Inform 2017 Nov;75:70-82 [FREE Full text] [CrossRef] [Medline]
- Chan P, Zhao Y, Lim S, Perlman S, McVeigh K. Using Calibration to Reduce Measurement Error in Prevalence Estimates Based on Electronic Health Records. Prev Chronic Dis 2018 Dec 13;15:E155 [FREE Full text] [CrossRef] [Medline]
- Hripcsak G, Duke J, Shah N, Reich C, Huser V, Schuemie M, et al. Observational Health Data Sciences and Informatics (OHDSI): Opportunities for Observational Researchers. Stud Health Technol Inform 2015;216:574-578 [FREE Full text] [Medline]
- Boeckhout M, Zielhuis G, Bredenoord A. The FAIR guiding principles for data stewardship: fair enough? Eur J Hum Genet 2018 Dec;26(7):931-936 [FREE Full text] [CrossRef] [Medline]
- Cervero R, Gaines J. Accreditation Council for Continuing Medical Education. 2014. Effectiveness of Continuing Medical Education: Updated Synthesis of Systematic Reviews URL: http://www.accme.org/sites/default/files/2014_Effectiveness_of_Continuing_Medical_Education_Cervero_and_Gaines_0.pdf [accessed 2019-03-16] [WebCite Cache]
- Bates D, Kuperman G, Wang S, Gandhi T, Kittler S, Volk L, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003;10(6):523-530 [FREE Full text] [CrossRef] [Medline]
|AEGD: advanced education in general dentistry|
|EDR: electronic dental record|
|EMR: electronic medical record|
|GPR: general practice residency|
|OMFS: Department of Oral and Maxillofacial Surgery|
Edited by C Lovis; submitted 17.12.18; peer-reviewed by F Shaya, R Moskovitch, M El Tantawi; comments to author 28.12.18; revised version received 11.02.19; accepted 13.03.19; published 21.03.19.
©Seth Aaron Levitin, John T Grbic, Joseph Finkelstein. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 21.03.2019.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.