Large national surveys and registry data provide epidemiological and population-level health information. Although such studies will remain as gold standards in evaluating the health state at a population level, the more recent development of large real-world data (RWD) from electronic medical records (EMRs) and claims data for therapeutic management and population-level safety evaluations provide additional and unique opportunities to expand our understanding in a broad class of clinical, epidemiological, and public health–related questions [1-6].

EMR data are collected during routine medical care, offering the opportunity to investigate clinical questions from a real-world perspective. Although randomized clinical trials (RCTs) allow the evaluation of the safety and efficacy of interventions in a design-led population, the EMR-based studies allow for comparative effectiveness and safety studies, apart from revolutionizing the approach to efficient pharmacovigilance. RWD-based studies also provide opportunities to explore clinical questions in populations that are often excluded from RCTs, such as pregnant, older, or comorbid patients. Furthermore, real-world studies allow us to investigate questions that may be unethical for testing in RCTs. EMRs are also used to track how clinical guidelines are implemented in real-world practices and to research the quality of clinical care.

The epidemiological value of EMR-based research directly depends on the size of the EMR network. Several EMR systems were implemented at the national level, and most familiar representatives include databases from the United States, the United Kingdom, Sweden, Norway, and Denmark [7-10]. The representativeness of some of these databases in terms of demographics and chronic and rare diseases has been shown in some studies [8,9,11-14].

Apart from health research based on data from individual practices, pharmacies, insurers, claims, or prescriptions, the MarketScan Commercial Claims and Encounters Database, owned by Truven Health Analytics, is one of the most commonly used data source for health research in the United States [15,16]. The Veteran Affairs–integrated health care system is another widely used data source in the United States [17,18]. One of the oldest primary and ambulatory EMR systems in the United States is the Centricity Electronic Medical Record (CEMR), owned by General Electric, which provides an opportunity for research using deidentified data on more than 45 million patients from all states of the United States [19,20].

The CEMR database has been extensively used for health outcome academic research worldwide in the fields of diabetes [21-26], cardiovascular research [27-31], obesity [32-34], inflammatory diseases [35-38], mental health [39-41], and other diseases [42,43]. To draw meaningful inferences from such studies and to generalize the results, it is essential to understand the underlying population. For instance, using the CEMR, Montvida et al [44] described trends in the antidiabetic drug prescription patterns during the years 2005-2016. However, the study should be interpreted in the light of overall representativity of the CEMR with respect to diabetes as well as gender and ethnic differences, as all these factors affect drug choices.

To the best of our knowledge, only two studies have investigated the representativeness of the CEMR database [14,45]. Brixner et al [14] evaluated the BMI and laboratory data from the CEMR in 2003 to 2004 in comparison with the US national health surveys. Crawford et al [45] compared the National Ambulatory Medical Care Survey (NAMCS) with CEMR’s office visits during 2005 and concluded that CEMR data provide a more accurate estimate of the distribution of diagnoses in ambulatory visits in the United States.

Given the significant increase in CEMR coverage since the last report was published and exponentially increasing volume of RWD-based research, we aimed to repeat and expand the exploration of the generalizability and representativity of the CEMR database with two of the most widely used and relevant survey results from the United States. Specifically, the goals of this study were to compare (1) patient demographics in the CEMR with the NAMCS and (2) the prevalence of obesity, hypertension, high total cholesterol, and diabetes in the CEMR with the respective reports based on the National Health and Nutrition Examination Surveys (NHANES).

Data on percent distribution of office visits and number of office visits per 100 patients by various subgroups from the NAMCS were compared with similar data from the CEMR. All office visits in 2016 for patients with nonmissing age and sex from the CEMR were aggregated to match the NAMCS report as close as possible.

In the NHANES, obesity was defined as a BMI of 30 kg/m² or greater for adults aged 20 years and older [50].

In the CEMR, the proportion of obese people was estimated among people aged older than 20 years and with at least one BMI measure (direct or estimated using weight and height) during the years 2015-2016. Women who had pregnancy-related records before or within the estimated time frame were excluded.

In the NHANES, systolic blood pressure (SBP) of 140 mm Hg or greater, diastolic blood pressure (DBP) of 90 mm Hg or greater, or currently taking medication to lower high blood pressure were defining hypertension for people aged 18 years and older [51].

In the CEMR, the proportion of patients with hypertension during the years 2015-2016 was estimated among people aged older than 18 years. On average, patients had 4 blood pressure measures during a 2-year time frame. Those who had an average of available measures for SBP of 140 mm Hg or greater, those who had an average of available measures for DBP of 90 mm Hg or greater, or those who were taking medication to lower high blood pressure during the respective time frame were considered to have hypertension. Blood pressure–lowering medications included diuretics, peripheral vasodilators, beta blockers, calcium channel blockers, angiotensin-converting enzyme inhibitors, angiotensin II receptor blockers, and other agents acting on the renin-angiotensin system. Only medications that are indicated to lower blood pressure were preserved within these drug classes.

In the NHANES, proportions of participants aged 20 years and older with high total cholesterol (≥240 mg/dL) were reported [52]. In the CEMR, among people aged older than 20 years and with at least one available cholesterol measure, the proportions of those with total cholesterol of 240 mg/dL or greater were estimated during the years 2015-2016.

In the NHANES, participants were classified as having diagnosed diabetes if they answered “yes” to the question, “Other than during pregnancy, have you ever been told by a doctor or health professional that you have diabetes or sugar diabetes?” [53]. Participants were classified as having undiagnosed diabetes if they did not report a diagnosis of diabetes by a health care provider and their fasting (8-24 hours) plasma glucose level was 126 mg/dL or greater or their hemoglobin A_1c (HbA_1c) level was 6.5% or greater. Participants were randomly assigned to a morning, afternoon, or evening examination. Fasting plasma glucose data from the morning examination (after an 8- to 24-hour fast) were used to define total and undiagnosed diabetes.

In the CEMR, an algorithm to identify patients with diabetes was developed on the basis of (1) diabetes diagnostic codes (International Classification of Diseases and SNOMED), (2) antidiabetic medication prescription patterns, (3) availability of 2 measurements of HbA_1c level of 6.5% or greater or fasting blood glucose level of 126 mg/dL or greater or random blood glucose level of 200 mg/dL or greater within 1 year, and (4) keyword searching procedures for diabetic-related terms from the clinical notes of every patient. The algorithm was developed on the basis of clinical guidelines and machine learning suggestions described by Adjah et al [54] for a database from the United Kingdom. Patients who were prescribed metformin for polycystic ovary syndrome were detected and excluded. In the case of nondefinite diabetes subtype, a patient’s age and insulin and noninsulin prescription patterns were used to distinguish subtypes. The off-label use of antidiabetic drugs was not explored. For analyses in this study, patients with prediabetes and gestational diabetes were excluded. The proportion of patients with coded and noncoded diabetes was estimated among adults aged 20 years and older and who were active in the CEMR during the years 2013-2016.

Proportional distributions between the CEMR and the NAMCS and NHANES were compared using the chi-square test, where appropriate. Office visit estimates in the NAMCS report are based on sample data weighted to produce annual national estimates and include SEs. All estimates in the NHANES reports were age adjusted using the 2000 US Census population. For the NAMCS and the NHANES estimates, 95% CIs were calculated using the available SE estimates from the reports.

Crude estimates from the CEMR data were calculated and presented in this study, 95% CI for percentages were calculated based on binomial distribution assumption, and 95% CIs for number of visits per 100 persons per year were calculated assuming a Poisson distribution.

Statistical equivalence for the pairwise comparisons of proportional distributions, where appropriate, was evaluated using the two one-sided test (TOST) of equivalence [55,56] with a ±2.5, ±5, and ±7.5 percentage point equivalence margins. Using population summary statistics on mean, SD, and total number of office visits, the TOST procedure available in SAS 9.4 (SAS Institute Inc) was employed.

The NAMCS estimated 883,725,000 office visits in the United States in 2016. In the CEMR, 29,207,860 office visits in 2016 occurred for patients with nonmissing age and sex. In the NAMCS and the CEMR, sex distribution was similar (equivalent at 2.5 percentage point margin), where 58.0% (95% CI 56.2-59.8) and 59.8% (95% CI 59.8-59.8) of all visits were by females (Table 1). The number of visits per 100 females per year was similar in the NAMCS and the CEMR: 315.0 (95% CI 291.5-338.5) versus 294.6 (95% CI 294.5-294.8), whereas the number of visits per 100 males per year was lower in the NAMCS compared with the CEMR: 239.0 (220.6-257.4) versus 270.8 (270.6-270.9).

Looking into office visits’ percent distribution by age, it was similar between data sources (P=.22 for overall and P=.23 by age and sex). The CEMR contains fewer visits by younger patients: the age group <15 years did not reach equivalence at the 5 percentage point equivalence margin for overall comparison and was not equivalent at the 2.5 percentage point margin in comparisons by sex. Age groups of 15-24/25-44 years were equally likely to have a visit with proportions of 7.4% (95% CI 6.6-8.2)/19.3% (95% CI 17.5-21.1) and 7.5% (95% CI 7.4-7.5)/20.0% (95% CI 20.0-20.0) in the NAMCS and the CEMR, respectively. Overall, there were 277.9 (95% CI 259.3-296.5) and 284.6 (95% CI 284.4-284.7) office visits per 100 persons in 2016 in the NAMCS and the CEMR, respectively. Younger patients had similar numbers of visits per year per 100 persons: 257.4 (95% CI 205.5-309.3) and 276.5 (95% CI 276.2-276.8) in the NAMCS and the CEMR, respectively; middle age groups (15-44 years) had significantly fewer visits in the NAMCS; and patients older than 65 years had significantly more visits in the NAMCS compared with the CEMR (P<.05).

The overall ethnicity distribution was similar between the NAMCS and CEMR groups (P=.20). The proportion of visits by white among all visits were similar in the CEMR (85.7% [95% CI 85.7-85.7]) and NAMCS (83.8% [95% CI 82.0-85.6]; equivalent at 2.5 percentage point margin). The number of visits per 100 persons per year was also similar: CEMR, 289.7 (95% CI 289.6-289.8); and NAMCS, 302.3 (95% CI 281.1-323.5). Although the share of office visits by black or African Americans was similar in both data sources (11%, equivalent at 2.5 percentage point margin), there were significantly fewer office visits per 100 persons per year in NAMCS compared with CEMR in this ethnic group: 224.3 (95% CI 192.2-256.4) versus 293.7 (95% CI 293.3-294.0); P<.05.

The geographical distribution of office locations in the CEMR and the NAMCS was similar (Table 1; P=.23), with underrepresented Midwest and West (not equivalent at 2.5 percentage point margin) and overrepresented South in the CEMR compared with the NAMCS (not equivalent at 5 percentage point margin).

In this study, we compared the CEMR ambulatory and primary care database with federal reports based on the NAMCS and the NHANES. Although the CEMR and the CDC reports may not be directly compared because of the differences in the data collection nature and methodologies applied, in this study, we have observed that the CEMR is a good source of population health research with regard to cardiometabolic conditions. Specifically, we observed that (1) on average, there were 3 office visits per patient per year in both the NAMCS and the CEMR; (2) the distribution of age at office visits in the CEMR is biased toward older population; (3) although the proportional share of all visits by males and females were similar, females/males had more/fewer visits in the NAMCS, compared with the CEMR; (4) the distribution of office visits were similar for ethnic groups; (5) West regions are underrepresented and South region is overrepresented in the CEMR compared with the NAMCS; (6) compared with the CDC reports based on the NHANES data, the prevalence of obesity and high total cholesterol is similar in the CEMR, whereas hypertension prevalence is 5% higher; and (7) the prevalence of diabetes in the CEMR reflects the diagnosed US population well.

A decade ago, Crawford et al [45] compared CEMR’s office visits during the year 2005 with the NAMCS report. Crawford et al [45] reported that the CEMR had higher proportions of visits by younger patients and by females, compared with the NAMCS. Maintaining similar methods, we observed a reversed trend in age and no difference in the distribution of visits by gender. The overall prevalence in adults with obesity, high cholesterol, and diabetes was similar between the CEMR and the NHANES reports, and the proportion of those with hypertension was higher in the CEMR than in the NHANES. A possible explanation for this result is the ability to track longitudinal information in EMRs, which is especially prominent in chronic conditions. This feature of EMRs provides exceptional opportunities in terms of extending and modifying the classical epidemiologic theory [57].

Closed systems in the West (the Kaiser Permanente [58]) may explain the finding of lower rates for office visits in the West region in the CEMR. Although the CDC reports were adjusted and weighted with the US Census data, CEMR’s crude prevalence estimates of chronic conditions reported in this study are thus biased by geographic regions. This issue should be carefully taken care of in future population health research based on the CEMR data.

As with any survey, the results in the NAMCS and the NHANES are subject to sampling and nonsampling errors. Nonsampling errors include reporting and processing errors as well as biases because of nonresponse and incomplete response. In the NAMCS, ethnicity data were missing for 25% of visits [47,48], whereas in the CEMR, unknown race accounted for only 9% of visits.

It is important to highlight that the population that seeks medical care is biased from the general population, and healthy individuals will be underrepresented in any EMR database. For this reason, rather than comparing the CEMR with US Census data, we compared the demographics at the time of office visits in the CEMR with the NAMCS, which is carefully weighted and adjusted for the US population. Although a significant subset of CEMR users is participating in the Medical Quality Improvement Consortium, the CEMR research database is biased toward these practices. The participation and response rates in the NAMCS of less than 40% also introduce a selection bias, although the NAMCS estimates were corrected [47,48].

The limitations of this study include the nonavailability of provider specialty and insurance data in the CEMR for deeper comparisons with the NAMCS. Certain specialties might adopt EMRs at a slower rate, and insured individuals might be overrepresented in commercial databases, compared with the general population. Owing to large cohort sizes in the CEMR, reported CIs are very narrow, and we believe they do not reflect meaningful differences. Adopting TOST with 2.5, 5.0, and 7.5 percentage point equivalence margins for data comparison provides another overview of the equivalence of data sources. As mentioned earlier, CEMR and survey methods are not directly comparable, and we have done our best to match the data as closely as possible. The cardiometabolic prevalence estimates should be interpreted carefully, in the light of methodological and regional differences. However, we believe that the results of this study demonstrate the ability of the CEMR to reflect population health quite well.

To conclude, epidemiological and population health findings based on the CEMR database might reflect trends in the general US population; however, the possible region, age, and gender biases presented in this study should be treated and interpreted carefully.