Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 21.03.19 in Vol 7, No 1 (2019): Jan-Mar

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/13008, first published Dec 17, 2018.

This paper is in the following e-collection/theme issue:

    Original Paper

    Completeness of Electronic Dental Records in a Student Clinic: Retrospective Analysis

    1Division of Foundational Sciences, Columbia University College of Dental Medicine, New York, NY, United States

    2Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, NY, United States

    *these authors contributed equally

    Corresponding Author:

    Joseph Finkelstein, MD, PhD

    Department of Population Health Science and Policy

    Icahn School of Medicine at Mount Sinai

    1425 Madison Ave, Rm L2-36

    New York, NY, 10029

    United States

    Phone: 1 212 659 9596

    Email: Joseph.Finkelstein@mssm.edu


    ABSTRACT

    Background: A well-designed, adequately documented, and properly maintained patient record is an important tool for quality assurance and care continuity. Good clinical documentation skills are supposed to be a fundamental part of dental student training.

    Objective: The goal of this study was to assess the completeness of electronic patient records in a student clinic.

    Methods: Completeness of patient records was assessed using comparative review of validated cases of alveolar osteitis treated between August 2011 and May 2017 in a student clinic at Columbia University College of Dental Medicine, New York, USA. Based on a literature review, population-based prevalence of nine most frequently mentioned symptoms, signs, and treatment procedures of alveolar osteitis was identified. Completeness of alveolar osteitis records was assessed by comparison of population-based prevalence and frequency of corresponding items in the student documentation. To obtain all alveolar osteitis cases, we ran a query on the electronic dental record, which included all cases with diagnostic code Z1820 or any variation of the phrases “dry socket” and “alveolar osteitis” in the notes. The resulting records were manually reviewed to definitively confirm alveolar osteitis and to extract all index items.

    Results: Overall, 296 definitive cases of alveolar osteitis were identified. Only 22% (64/296) of cases contained a diagnostic code. Comparison of the frequency of the nine index categories in the validated alveolar osteitis cases between the student clinic and the population showed the following results: severe pain: 94% (279/296) vs 100% (430/430); bare bone/missing blood clot: 27% (80/296) vs 74% (35/47) to 100% (329/329); malodor: 7% (22/296) vs 33%-50% (18/54); radiating pain to the ear: 8% (24/296) vs 56% (30/54); lymphadenopathy: 1% (3/296) vs 9% (5/54); inflammation: 14% (42/296) vs 50% (27/54); debris: 12% (36/296) vs 87% (47/54); alveolar osteitis site noted: 96% (283/296) vs 100% (430/430; accepted documentation requirement); and anesthesia during debridement: 77% (20/24) vs 100% (430/430; standard of anesthetization prior to debridement).

    Conclusions: There was a significant discrepancy between the index category frequency in alveolar osteitis cases documented by dental students and in the population (reported in peer-reviewed literature). More attention to clinical documentation skills is warranted in dental student training.

    JMIR Med Inform 2019;7(1):e13008

    doi:10.2196/13008

    KEYWORDS



    Introduction

    A clinical record is a fundamental part of patient care delivery [1]. Its completeness is important for many reasons. A record’s main purpose is to serve as a means of communication among providers themselves and between providers and their colleagues [2]. Clinical decision support is dependent on accurate and complete dental records [3], which also aid in the evaluation of a patient’s care [4]. In the event of a lawsuit, the record serves as evidence [5] and its contents are necessary to determine whether the diagnosis and treatment met appropriate standards [6]. With the expansion of dental informatics applications, it is even more essential to have a complete record in order to ensure proper analysis and results in outcomes research [5]. However, there are known issues with the completeness of electronic medical records (EMRs).

    In the past decade, electronic patient records became a ubiquitous part of dental care delivery [7]. Oral health data accumulated in the process of clinical care represent a rich and readily available recourse for scientific investigation and data analytics [8]. Recent analysis of electronic dental records (EDRs) helped identify predictors of implant survival [9] as well as the prevalence and risk factors of peri-implantitis [10]. Application of machine learning techniques [11] and temporal analytics [12] resulted in new opportunities for knowledge discovery and predictive analytics [13]. With the increasing use of EMRs as an important resource for scientific discovery, potential barriers for secondary analysis of EMR have been recognized [14]. Completeness of EMR is one of the most frequently discussed issues that may limit the use of EMR data for clinical and population health research [15]. This issue was reported to be particularly relevant in the evaluation of medical student documentation [16]; however, no systematic assessment of the completeness of EMR in dental student clinics has been performed.

    Since the widespread introduction of EMRs, numerous studies have indicated gaps in documentation. A study at the University of Michigan examined whether there were differences in the reported eye symptoms between EMRs and eye symptom questionnaires that patients fill out. Exact agreement was found in only 23.5% (38/162) of cases. In cases where patients reported three or more symptoms, data from the eye symptom questionnaire always varied from the EMR data [17]. A similar study at the Mayo Clinic compared symptoms of chest pain, dyspnea, and cough between information forms patients received prior to the appointment and EMRs, for the purpose of identifying stable angina pectoris. They found that the two documents had varying levels of positive agreement (ratio): 74 for chest pain, 70 for dyspnea, and 63 for cough [18]. Researchers at Duke University analyzed records for completeness for quality purposes, but concluded that improper documentation for colorectal cancer impeded their ability to accurately calculate patient performance. Of the 499 patients eligible for the analysis, only 66 had sufficient documentation. In addition, only 86% (427/499) of EMRs confirmed a diagnosis, 29% (143/499) were missing the age, and only 38% (188/499) stated the TMN stage [19].

    Other studies indicated that the quality of documents in EDRs may be suboptimal [5,20]. However, there is a lack of systematic studies on the completeness of EDRs. The goal of this project was to review the documentation quality of EDRs in a dental school clinic.


    Methods

    Data Source

    EDRs of patients examined at Columbia University’s College of Dental Medicine were analyzed.

    Data Collection

    The study was a retrospective analysis of EDRs from patients with dry sockets diagnosed at Columbia University’s College of Dental Medicine between August 2011 and May 2017. We ran a query on the college’s database to find EDRs containing diagnostic code Z1820, the phrases “dry socket” and “alveolar osteitis,” or a variation of those two phrases. All queries were performed using structured query language in the Oracle database containing data from an EDR system called axiUm (Exan Group, Las Vegas, NV).

    For the purpose of this study, a dry socket was defined by a diagnostic Z code at the initial encounter. Alternatively, it was defined by the presence of a key word in a note combined with clinical evidence and explicit documentation of the patient with alveolar osteitis or dry socket. Z codes are represented by a list of diagnostic codes and terms developed for use with EDR, as previously described [21].

    After reviewing the literature on dry socket [22-28], we compiled a list of 17 criteria related to dry socket. Five were related to treatment: curettage, irrigation, anesthesia, intra-alveolar medication, and medication. Three were symptoms: pain, radiating pain, and tenderness on palpation. Nine were signs: lack of blood clot, malodor, low-grade fever, bare bone, lymphadenopathy, pus, erythema, inflamed gingiva/socket, and debris. The last category was the socket site of dry socket. Severe pain was a necessary symptom for dry socket diagnosis [25,28].

    Data Analysis

    A dental student reviewed each of the query results to confirm the presence of a dry socket. The cases were also analyzed for the presence of any of the 17 criteria mentioned above. Pertinent positive and negative results for dry socket criteria were both recorded for further analysis. The student then reviewed the available literature on dry socket statistics to determine the prevalence of symptoms in the literature. Both the positive and negative criteria were compared to the baseline figures from this literature.


    Results

    A total of 150 records with diagnostic Z codes were identified, and another 787 records were identified by searching for a mention of dry socket or alveolar osteitis in the notes (Figure 1). Both queries resulted in a number of duplicate cases, which were removed. The dry socket/alveolar osteitis dataset had 11 duplicate cases, and the Z code data set contained 13 duplicate cases. In addition, 101 of the remaining cases overlapped (ie, had both diagnostic Z codes and dry socket or alveolar osteitis in the notes). These overlapping cases were not counted twice in the study. After removing duplicates and overlapping cases, the final number of combined cases was 812.

    Figure 1. Breakdown of the reviewed cases. DS: dry socket; AO: alveolar osteitis; FU: follow-up; OMFS: Department of Oral and Maxillofacial Surgery.
    View this figure

    Cases were then categorized based on the dental record content (Table 1). An explicit diagnosis of dry socket was found in 278 EMRs. Another 18 EMRs had a diagnostic Z code and the patient received treatment for dry sockets, but the dentist never explicitly stated that the patient presented with a dry socket; these were also included as definitive dry sockets for the purpose of this study. Totally, 296 cases (of 812, 36.5%) were categorized as definitive dry sockets, and 108 cases (of 812, 13.3%) of possible dry sockets had notes of treatment of dry sockets, but the dentist did not include a diagnostic code or any definitive diagnoses of dry socket in the note. This misdocumentation is important, but could not be included in the data, as there is no way to confirm the patient had a dry socket from the records. The remaining cases were classified as follows:

    • Follow-up treatment of both definitive dry sockets and possible dry sockets (226/812, 27.8%).
    • Cases that were definitively classified as nondry sockets: These patients’ records were included in the query because they contained the key word to rule out the diagnosis, history of previous dry socket, mention of dry socket paste, or similar reference (117/812, 14.4%).
    • Referrals to the oral surgery clinic: Such referrals for further evaluation were also a common finding. The referrals were not followed up to determine if these patients were later seen in Columbia (43/812, 5.3%). An additional segment of charts belonged to patients whose only encounter populated in our search was for a follow-up in our clinic for a dry socket (19/812, 2.3%). The final and smallest group of charts contained a Z code, but no note was found in the system for the encounter. This may have been a kink in the system or oversight by the provider (3/812, 0.4%).

    Definitive dry socket cases were analyzed for correctness of EMR documentation. The demographics of patients of the 296 records are outlined in Table 2. Women (196/296, 66.3%) were affected by dry socket more frequently than men. The age group with the highest frequency of dry socket included adults between the ages of 20 and 39 years (146/296, 49.3%). Table 3 demonstrates the extraction characteristics of the definitive dry sockets studied. Molars were the most frequent sites for dry socket, with the third molar represented in 37.5% of the cases (111/296). Tooth extraction was performed under the supervision of faculty from the Department of Oral and Maxillofacial Surgery in 82.8% (245/296) cases. About half of the tooth extraction procedures were carried out by predoctoral students (145/296, 46.3%) and the remaining extractions were performed by postdoctoral students, the majority of whom were represented by residents of the Department of Oral and Maxillofacial Surgery. A similar pattern was observed for location and provider type for the tooth extraction follow-up visits during which dry socket was diagnosed and treated.

    Table 1. Categories of classifications.
    View this table
    Table 2. Patient demographics.
    View this table

    Two general types of misdocumentation were encountered and calculated while analyzing the 296 cases of definitive dry socket (Table 4). The first was any missing gross documentation necessary for diagnosis of dry socket to support the clinical diagnosis of this condition and to provide documentation necessary for patient follow-up and care continuity. These included documented pain, visible bone, or lack of blood clot. We also included the missing socket position when diagnosing a dry socket. Of the 296 cases, 220 (74.3%) were missing at least one of the abovementioned factors. The second type of misdocumentation was the number of cases without a diagnostic Z code. Of the 296 cases, 232 (78.4%) did not contain the diagnostic code. All cases analyzed were documented after the Columbia University College of Dental Medicine implemented diagnostic codes.

    Table 3. Extraction characteristics.
    View this table

    Possible misdocumentation was also found in the remaining 516 cases that were not labeled as definitive dry socket. There were 108 cases of patients who presented with symptoms of dry sockets and received treatment for dry sockets. However, their charts did not include a Z code or any definitive diagnoses by the provider. Although many of these patients fulfilled the requirements to be classified with a dry socket, they could not be included in the analysis because they lacked a basic documented confirmation. As previously stated, 206 cases were classified as follow-up appointments to initial treatment of definitive dry socket. From these cases, another category of misdocumentation emerged: 16 of the cases contained a diagnostic Z code, although the actual diagnosis and initial treatment of the dry socket did not contain this code. The latter type of misdocumentation does not have any direct consequences, but reveals a reluctance to use dry socket diagnosis or lack of education regarding diagnostic codes.

    The final group of misdocumentation was found in the 19 cases categorized as follow-up to the initial appointment for dry socket, in which the patient’s first documented encounter with the word dry socket or alveolar osteitis was in the follow-up. The clinical note in 14 of these 19 cases indicated that the patient was previously seen in the clinic or emergency room to treat the dry socket. The implication is that these patients were seen and treated for a dry socket previously in the clinic, but the earlier documentation lacked any of the three search terms (dry socket, alveolar osteitis, and Z code) used to identify cases for this analysis.

    Table 4. Types of misdocumentation encountered (N=296).
    View this table

    Definitive dry socket signs, symptoms, and treatments completeness compared to the best values available in the literature are presented in Figure 2. Presence of severe pain was considered a necessary symptom for dry socket diagnosis. The clinical notes only contained pain as a symptom in 94% (279/296) of the cases. Bare bone [23], open socket [25], or missing blood clot [26] was mentioned in 74% (35/47) to 100% (329/329) of the dry socket cases. In our clinic, any of these related terms were mentioned in 27% (80/296) of cases. In the 296 definitive dry socket cases, the word “bone” was only found in 50 cases and “clot” was found only 25 times; some of these cases overlapped. These terms are germane and necessary for a diagnosis of dry socket, and the lack of documentation is troublesome. Malodor [22] was documented in 33% (18/54) to >50% of the dry socket cases in the literature [27]; however, its documentation was only present in 7% (22/296) of our cases. Radiating pain toward the ear was present in 56% (30/54) of the cases in the literature [22], but was mentioned only in 7% (24/296) of our cases (documentation of radiating pain alone was included in this number, as we did not require ear or side of face to be documented). Lymphadenopathy [22] was present in 9% (5/54) of the cases in the literature [23], but in only 1% (3/296) of our cases. Inflammation was present in 50% (27/54) of the cases in the literature [22]; however, in the dental record, only 14% (42/296) of cases mentioned inflammation and 15% (44/296) (with some overlap with inflammation cases) mentioned erythema.

    A comparison between the notes of predoctoral dental students and postdoctoral faculty and residents is outlined in Table 5. The nine selected signs, symptoms, and other documentation related to dry socket diagnosis were analyzed. Predoctoral students were more likely to properly document the location of the dry socket, while postdoctoral students were more likely to document malodor, presence of debris, and anesthesia administration during socket debridement. Misdocumentation of the remaining categories was equal (both groups were within 1% of one another for the other five categories) as compared to the expected values (Figure 2).

    Figure 2. Signs and symptoms in misdocumentation. DS: dry socket; CDM: clinical dental record.
    View this figure
    Table 5. Comparison between the notes of predoctoral dental students and postdoctoral faculty and residents.
    View this table

    Discussion

    Our ultimate finding was that misdocumentation occurs in EDRs to varying degrees within a dental school clinic. The dental records reviewed lack many signs and symptoms that are necessary and expected to be recorded for a diagnosis of dry socket. This misdocumentation was prevalent in the notes of both predoctoral and postdoctoral students. The major limitation of this study, as with other electronic record retrospective studies, was our ability to confirm the diagnosis of dry socket. We also could not confirm whether patients actually had the symptoms omitted. We were forced to rely on the information provided by the documenter.

    Evaluation of completeness of symptom documentation in this study followed recent guidance on EMR data quality–assessment methodology [29] that promulgates the use of validated population-based prevalence as a gold standard [30-31]. Following these guidelines, the comparison between the validated symptom prevalence and symptom frequency found in our EDR review was used to identify completeness of symptom documentation in EDR in this study. This approach has been successfully used in a number of previous studies to identify completeness of smoking status records [32], obesity reporting [33], hypertension records [34], and depression prevalence [35]. To minimize bias toward inflating misdocumentation rates, whenever several verified estimates of a population-based frequency for a particular symptom were available, a comparison was made between the lowest population-based frequency and the frequency found in the EDR, as previously described [29-32].

    Our findings of the demographics and tooth extraction characteristics among patients with dry socket are congruent with previous reports. As previously determined, women are more likely to have dry socket then men [25,36]. The tooth position is most likely to be a third molar compared to any other individual area [37]. People in their 20s and 30s are at a higher risk for alveolar osteitis [38]. Two major types of misdocumentation that were found and analyzed in this study have been previously mentioned in the literature, including lack of supporting documentation for clinical diagnosis [25,28] and absence of appropriate diagnostic codes [39].

    Analysis of characteristics of providers who performed tooth extraction and dry socket diagnosis confirmed the external validity [40] of our study sample, as the resulting characteristics accurately reflected routine dental care delivery patterns for these types of procedures occurring in the student clinic. The majority of the cases were carried out under supervision of a preceptor from the Department of Oral and Maxillofacial Surgery. About half of the extractions and dry socket diagnoses were carried out by predoctoral students, and the rest were performed by postdoctoral residents of whom approximately half were represented by residents of the Department of Oral and Maxillofacial Surgery. The fact that the predoctoral and postdoctoral dental surgeons were equally presented in our analysis supported the unbiased comparison between documentation quality of predoctoral and postdoctoral students.

    The results of documentation completeness in EDRs in this study are corroborated by the following reports. In Minnesota, a discrepancy was found between the American Dental Association’s recommendation for dental record accuracy and the actual accuracy in dental practices [41]. In Finland, researchers observed a discrepancy between the quality of treatment a dentist believed he/she provided and the treatment the patient actually received, as contained in the EDR [42]. EMRs have also been shown to have issues with accuracy [43]. In an adult cardiology clinic, researchers discovered “very poor” completeness values for signs such as chest pain and shortness of breath [44]. In a systemic review of EMR completeness in primary care, Thiru [45] found that records of diagnoses with clear clinical criteria had a higher rate of completeness than those without clear criteria. This is relevant to dry socket, a diagnosis with unclear criteria. A trauma center study also found incompleteness of certain categories in the EMR [46]. Similarly, a study conducted with inpatient records at Menelik II Referral Hospital, a government hospital in Addis Ababa, Ethiopia, found “low” EMR completeness compared to the expected standard of 100% [47]. Legal Medical Record Standards stated that “Each Medical Record shall contain sufficient, accurate information to identify the patient, support the diagnosis, justify the treatment, document the course and results, and promote continuity of care among health care providers” [48]. Following this simple instruction can vastly improve the delivery of care.

    To resolve the issues with incomplete or inaccurate records, dental education should emphasize more on proper documentation and ensure its incorporation into the clinic routine. Documenters need to remember that their records are not simply for their own convenience, but may serve legal, research-related, or forensic purposes [1,49]. To this end, each record must be complete, without implicit assumptions, and follow a method that makes it easily accessible to any reader. Thierer [50] found an improvement in EDR accuracy by incorporating an in-service intervention for faculty members and a Moodle site course on documentation for students.

    Our findings have important implications for future research that uses EMR data. Better understanding of the potential limitations of electronic health record data use promotes fidelity and reproducibility of secondary data analysis [51]. A variety of approaches are being implemented to address the potential limitations of EMR data [52] such as deep learning techniques [53] for imputing missing data, symbolic operations for time interval analytics [54], and calibration to reduce measurement error in prevalence estimates based on EMR data [55]. A growing number of studies employ common data models combined with cross-linked semantic ontologies to harmonize EMR data [56] and confirm with the Findability, Accessibility, Interoperability, and Reusability principles [57].

    We believe there are three sequential steps necessary for improving EDRs. The first step is additional training predoctoral and postdoctoral students on the importance of note comprehensiveness. As per a systemic review by the Accreditation Council for Continuing Medical Education, a live intervention with interactive techniques is the most effective way to change a physician’s behavior to influence patient outcomes [58]. Interventions should therefore be constructed accordingly. The next step is adding disease- and condition-specific worksheets to the EDR. The worksheets should contain categories pertinent to the specific diagnosis, with drop-down boxes for the practitioner to complete. This prevents the inadvertent omission of crucial categories. The last step is adding a clinical decision support tool to EDRs. The tool embodies evidence-based dentistry, an approach being adopted by an increasing number of dental schools and practitioners. However, this tool operates properly only if practitioners enter complete and accurate data into EDRs in a way that computers can easily analyze. Thus, strict compliance with the first two steps is critical. This will result in refined EDRs, which can potentially lead to superior and safer delivery of care at a lower cost [59]. Although some information in EDRs may seem largely irrelevant, the EDR is a critical depository of data, with limitless research possibilities. If properly executed, it may improve diagnoses, treatment, and dentistry as a whole.

    Conflicts of Interest

    None declared.

    References

    1. Devadiga A. What's the deal with dental records for practicing dentists? Importance in general and forensic dentistry. J Forensic Dent Sci 2014 Jan;6(1):9-15 [FREE Full text] [CrossRef] [Medline]
    2. Ho Y, Gadd CS, Kohorst KL, Rosenbloom ST. A qualitative analysis evaluating the purposes and practices of clinical documentation. Appl Clin Inform 2014 Dec;5(1):153-168 [FREE Full text] [CrossRef] [Medline]
    3. Wright A, Phansalkar S, Bloomrosen M, Jenders RA, Bobb AM, Halamka JD, et al. Best practices in clinical decision support: the case of preventive care reminders. Appl Clin Inform 2010 Dec;1(3):331-345 [FREE Full text] [CrossRef] [Medline]
    4. American Dental Association. 2010. Dental Records   URL: http://www.ada.org/~/media/ADA/Public%20Programs/Files/MPRG_Dental_Records.pdf?la=en [accessed 2019-03-18] [WebCite Cache]
    5. Mangalmurti S, Murtagh L, Mello M. Medical malpractice liability in the age of electronic health records. N Engl J Med 2010 Nov 18;363(21):2060-2067. [CrossRef] [Medline]
    6. Acharya A, Shimpi N, Mahnke A, Mathias R, Ye Z. Medical care providers' perspectives on dental information needs in electronic health records. J Am Dent Assoc 2017 Dec;148(5):328-337. [CrossRef] [Medline]
    7. Acharya A, Schroeder D, Schwei K, Chyou P. Update on electronic dental record and clinical computing adoption among dental practices in the United States. Clin Med Res 2017 Dec;15(3-4):59-74 [FREE Full text] [CrossRef] [Medline]
    8. Gordon S, Camargo G, Mejia G, Sutherland J. Use of the dental electronic health record for research: Assessing demographic and oral health characteristics data for clinic patients. J Dent Educ 2018 Dec;82(12):1249-1257. [CrossRef] [Medline]
    9. Hickin M, Shariff J, Jennette P, Finkelstein J, Papapanou P. Incidence and Determinants of Dental Implant Failure: A Review of Electronic Health Records in a U.S. Dental School. J Dent Educ 2017 Oct;81(10):1233-1242 [FREE Full text] [CrossRef] [Medline]
    10. Kordbacheh Changi K, Finkelstein J, Papapanou P. Peri-implantitis prevalence, incidence rate, and risk factors: A study of electronic health records at a U.S. dental school. Clin Oral Implants Res 2019 Feb 15. [CrossRef] [Medline]
    11. Yoon S, Choi T, Odlum M, Mitchell D, Kronish I, Davidson K, et al. Machine Learning to Identify Behavioral Determinants of Oral Health in Inner City Older Hispanic Adults. Stud Health Technol Inform 2018;251:253-256 [FREE Full text] [Medline]
    12. Shknevsky A, Shahar Y, Moskovitch R. Consistent discovery of frequent interval-based temporal patterns in chronic patients' data. J Biomed Inform 2017 Nov;75:83-95 [FREE Full text] [CrossRef] [Medline]
    13. Wu J, Roy J, Stewart W. Prediction modeling using EHR data: challenges, strategies, and a comparison of machine learning approaches. Med Care 2010 Jun;48(6 Suppl):S106-S113. [CrossRef] [Medline]
    14. Verheij R, Curcin V, Delaney B, McGilchrist M. Possible Sources of Bias in Primary Care Electronic Health Record Data Use and Reuse. J Med Internet Res 2018 May 29;20(5):e185 [FREE Full text] [CrossRef] [Medline]
    15. Weiskopf N, Hripcsak G, Swaminathan S, Weng C. Defining and measuring completeness of electronic health records for secondary use. J Biomed Inform 2013 Oct;46(5):830-836 [FREE Full text] [CrossRef] [Medline]
    16. Seo J, Kong H, Im S, Roh H, Kim D, Bae H, et al. A pilot study on the evaluation of medical student documentation: assessment of SOAP notes. Korean J Med Educ 2016 Jun;28(2):237-241 [FREE Full text] [CrossRef] [Medline]
    17. Valikodath NG, Newman-Casey PA, Lee PP, Musch DC, Niziol LM, Woodward MA. Agreement of Ocular Symptom Reporting Between Patient-Reported Outcomes and Medical Records. JAMA Ophthalmol 2017 Mar 01;135(3):225-231 [FREE Full text] [CrossRef] [Medline]
    18. Pakhomov S, Jacobsen S, Chute C, Roger V. Agreement between patient-reported symptoms and their documentation in the medical record. Am J Manag Care 2008 Aug;14(8):530-539 [FREE Full text] [Medline]
    19. Abernethy AP, Herndon JE, Wheeler JL, Rowe K, Marcello J, Patwardhan M. Poor documentation prevents adequate assessment of quality metrics in colorectal cancer. J Oncol Pract 2009 Jul;5(4):167-174 [FREE Full text] [CrossRef] [Medline]
    20. Kopsala JT. WiredSpace. 2014. Quality Assurance and Dental Patient Records   URL: http://wiredspace.wits.ac.za/handle/10539/14625 [accessed 2019-03-18] [WebCite Cache]
    21. White JM, Kalenderian E, Stark PC, Ramoni RL, Vaderhobli R, Walji MF. Evaluating a dental diagnostic terminology in an electronic health record. J Dent Educ 2011 May;75(5):605-615 [FREE Full text] [Medline]
    22. Singh V, Pal US, Singh R, Soni N. Honey a sweet approach to alveolar osteitis: A study. Natl J Maxillofac Surg 2014 Jan;5(1):31-34 [FREE Full text] [CrossRef] [Medline]
    23. Soni N, Singh V, Mohammad S, Singh RK, Pal US, Singh R, et al. Effects of honey in the management of alveolar osteitis: A study. Natl J Maxillofac Surg 2016;7(2):136-147 [FREE Full text] [CrossRef] [Medline]
    24. Baqain Z, Karaky AA, Sawair F, Khraisat A, Khaisat A, Duaibis R, et al. Frequency estimates and risk factors for postoperative morbidity after third molar removal: a prospective cohort study. J Oral Maxillofac Surg 2008 Nov;66(11):2276-2283. [CrossRef] [Medline]
    25. MacGregor A. Aetiology of dry socket: A clinical investigation. British Journal of Oral Surgery 1968 Jan;6(1):49-58. [CrossRef]
    26. Hedström L. Dry socket--an evaluation of 163 cases of the surgical removal of mandibular wisdom teeth. Tandlakartidningen 1987 Mar;79(6):298-303. [Medline]
    27. Blum I. Contemporary views on dry socket (alveolar osteitis): a clinical appraisal of standardization, aetiopathogenesis and management: a critical review. Int J Oral Maxillofac Surg 2002 Jun;31(3):309-317. [CrossRef] [Medline]
    28. Bortoluzzi M, Manfro R, De Déa BE, Dutra T. Incidence of dry socket, alveolar infection, and postoperative pain following the extraction of erupted teeth. J Contemp Dent Pract 2010 Jan 01;11(1):E033-E040. [Medline]
    29. Zozus M, Hammond W, Richesson R. NIH Collaboratory. 2014. Phenotypes, Data Standards, and Data Quality Core   URL: https://www.nihcollaboratory.org/Documents/Hammond_Zozus_2014-0816-PDSDQCore_update.pdf [accessed 2019-03-18] [WebCite Cache]
    30. Chen H, Hailey D, Wang N, Yu P. A review of data quality assessment methods for public health information systems. Int J Environ Res Public Health 2014 May 14;11(5):5170-5207 [FREE Full text] [CrossRef] [Medline]
    31. Faulconer E, de Lusignan S. An eight-step method for assessing diagnostic data quality in practice: chronic obstructive pulmonary disease as an exemplar. Inform Prim Care 2004;12(4):243-254 [FREE Full text] [Medline]
    32. Booth H, Prevost A, Gulliford M. Validity of smoking prevalence estimates from primary care electronic health records compared with national population survey data for England, 2007 to 2011. Pharmacoepidemiol Drug Saf 2013 Dec;22(12):1357-1361 [FREE Full text] [CrossRef] [Medline]
    33. McVeigh K, Newton-Dame R, Chan P, Thorpe L, Schreibstein L, Tatem K, et al. Can Electronic Health Records Be Used for Population Health Surveillance? Validating Population Health Metrics Against Established Survey Data. EGEMS (Wash DC) 2016;4(1):1267 [FREE Full text] [CrossRef] [Medline]
    34. Perlman S, McVeigh K, Thorpe L, Jacobson L, Greene C, Gwynn R. Innovations in Population Health Surveillance: Using Electronic Health Records for Chronic Disease Surveillance. Am J Public Health 2017 Dec;107(6):853-857. [CrossRef] [Medline]
    35. Thorpe L, McVeigh K, Perlman S, Chan P, Bartley K, Schreibstein L, et al. Monitoring Prevalence, Treatment, and Control of Metabolic Conditions in New York City Adults Using 2013 Primary Care Electronic Health Records: A Surveillance Validation Study. EGEMS (Wash DC) 2016;4(1):1266 [FREE Full text] [CrossRef] [Medline]
    36. Akinbami B, Godspower T. Dry socket: incidence, clinical features, and predisposing factors. Int J Dent 2014;2014:796102 [FREE Full text] [CrossRef] [Medline]
    37. Kolokythas A, Olech E, Miloro M. Alveolar osteitis: a comprehensive review of concepts and controversies. Int J Dent 2010;2010:249073 [FREE Full text] [CrossRef] [Medline]
    38. Abu Younis MH, Abu Hantash RO. Dry socket: frequency, clinical picture, and risk factors in a palestinian dental teaching center. Open Dent J 2011 Feb 07;5:7-12 [FREE Full text] [CrossRef] [Medline]
    39. Sweet J, Butler D. Predisposing and operative factors: effect on the incidence of localized osteitis in mandibular third-molar surgery. Oral Surg Oral Med Oral Pathol 1978 Aug;46(2):206-215. [Medline]
    40. Steckler A, McLeroy K. The importance of external validity. Am J Public Health 2008 Jan;98(1):9-10. [CrossRef] [Medline]
    41. Osborn J, Stoltenberg J, Newell K, Osborn S. Adequacy of dental records in clinical practice: a survey of dentists. J Dent Hyg 2000;74(4):297-306. [Medline]
    42. Helminen S, Vehkalahti M, Murtomaa H. Dentists' perception of their treatment practices versus documented evidence. Int Dent J 2002 Apr;52(2):71-74. [Medline]
    43. Majeed A, Car J, Sheikh A. Accuracy and completeness of electronic patient records in primary care. Fam Pract 2008 Aug;25(4):213-214. [CrossRef] [Medline]
    44. Oksuzyan A, Demirchyan A, Thompson M. Evaluation of Medical Records Completeness in the Adult Cardiology Clinic at Nork Marash Medical Center. Yerevan, Armenia: American University of Armenia; 2003 Aug.   URL: http://aua.am/chsr/UserFiles/File/Anahit/Evaluation of SEF Completeness_Eng.PDF [accessed 2019-03-16] [WebCite Cache]
    45. Thiru K, Hassey A, Sullivan F. Systematic review of scope and quality of electronic patient record data in primary care. BMJ 2003 May 17;326(7398):1070 [FREE Full text] [CrossRef] [Medline]
    46. de Mul M, Berg M. Completeness of medical records in emergency trauma care and an IT-based strategy for improvement. Med Inform Internet Med 2007 Jun;32(2):157-167. [CrossRef] [Medline]
    47. Tola K, Abebe H, Gebremariam Y, Jikamo B. Improving Completeness of Inpatient Medical Records in Menelik II Referral Hospital, Addis Ababa, Ethiopia. Advances in Public Health 2017;2017:1-5. [CrossRef]
    48. University of California. 2008. Legal Medical Record Standards   URL: http://policy.ucop.edu/doc/1100168/LegalMedicalRecord [accessed 2019-03-16] [WebCite Cache]
    49. Waleed P, Baba F, Alsulami S, Tarakji B. Importance of dental records in forensic dental identification. Acta Inform Med 2015 Feb;23(1):49-52 [FREE Full text] [CrossRef] [Medline]
    50. Thierer TE, Delander KA. Improving Documentation, Compliance, and Approvals in an Electronic Dental Record at a U.S. Dental School. J Dent Educ 2017 Apr;81(4):442-449 [FREE Full text] [CrossRef] [Medline]
    51. Farmer R, Mathur R, Bhaskaran K, Eastwood SV, Chaturvedi N, Smeeth L. Promises and pitfalls of electronic health record analysis. Diabetologia 2018 Dec;61(6):1241-1248. [CrossRef] [Medline]
    52. Casey J, Schwartz B, Stewart W, Adler N. Using Electronic Health Records for Population Health Research: A Review of Methods and Applications. Annu Rev Public Health 2016;37:61-81. [CrossRef] [Medline]
    53. Yoon S, Odlum M, Lee Y, Choi T, Kronish I, Davidson K, et al. Applying Deep Learning to Understand Predictors of Tooth Mobility Among Urban Latinos. Stud Health Technol Inform 2018;251:241-244 [FREE Full text] [Medline]
    54. Moskovitch R, Polubriaginof F, Weiss A, Ryan P, Tatonetti N. Procedure prediction from symbolic Electronic Health Records via time intervals analytics. J Biomed Inform 2017 Nov;75:70-82 [FREE Full text] [CrossRef] [Medline]
    55. Chan P, Zhao Y, Lim S, Perlman S, McVeigh K. Using Calibration to Reduce Measurement Error in Prevalence Estimates Based on Electronic Health Records. Prev Chronic Dis 2018 Dec 13;15:E155 [FREE Full text] [CrossRef] [Medline]
    56. Hripcsak G, Duke J, Shah N, Reich C, Huser V, Schuemie M, et al. Observational Health Data Sciences and Informatics (OHDSI): Opportunities for Observational Researchers. Stud Health Technol Inform 2015;216:574-578 [FREE Full text] [Medline]
    57. Boeckhout M, Zielhuis G, Bredenoord A. The FAIR guiding principles for data stewardship: fair enough? Eur J Hum Genet 2018 Dec;26(7):931-936 [FREE Full text] [CrossRef] [Medline]
    58. Cervero R, Gaines J. Accreditation Council for Continuing Medical Education. 2014. Effectiveness of Continuing Medical Education: Updated Synthesis of Systematic Reviews   URL: http:/​/www.​accme.org/​sites/​default/​files/​2014_Effectiveness_of_Continuing_Medical_Education_Cervero_and_Gaines_0.​pdf [accessed 2019-03-16] [WebCite Cache]
    59. Bates D, Kuperman G, Wang S, Gandhi T, Kittler S, Volk L, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003;10(6):523-530 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    AEGD: advanced education in general dentistry
    EDR: electronic dental record
    EMR: electronic medical record
    GPR: general practice residency
    OMFS: Department of Oral and Maxillofacial Surgery


    Edited by C Lovis; submitted 17.12.18; peer-reviewed by F Shaya, R Moskovitch, M El Tantawi; comments to author 28.12.18; revised version received 11.02.19; accepted 13.03.19; published 21.03.19

    ©Seth Aaron Levitin, John T Grbic, Joseph Finkelstein. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 21.03.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.