New York University Langone Health (NYULH) is a large academic health care system in New York City, with over 8000 health care providers across 4 hospitals and over 500 ambulatory faculty group practices. The system is connected via a single EHR system, Epic, with over 7.5 million active patient accounts. Prior to the COVID-19 pandemic, NYULH offered limited telemedicine services only through pilot programs such as “virtual urgent care” (in emergency medicine), postoperative wound checks (in orthopedics), and some mental health services. Telemedicine for primary care and other routine health services was not available. During the pandemic, NYULH rapidly scaled its telemedicine offerings to include primary care, ambulatory specialty practice, and urgent care. NYULH “virtual health” was comprised of a single, enterprise-wide instance of synchronous, video-based telecommunications encounters between physicians and patients in remote locations accessed through a standardized EHR-based patient portal system and a third-party videoconferencing vendor. This platform provided a unified patient and provider experience between clinical practice sites and across specialties. At the height of the pandemic, this system saw an 8595% increase in monthly telemedicine visits between February (n=1699) and April (n=147,736), with over 2000 unique physicians engaging in video visits [15].

This is an EHR-based retrospective cohort study including all ambulatory care physicians continuously practicing (defined as at least 5 appointments scheduled per week in the reporting period) at any New York-based NYULH faculty group practice site between January 1, 2020, and August 31, 2020. Nonphysician practitioners (eg, advanced-practice providers) and residents were not included in the study cohort, as with few exceptions, they did not provide telemedicine-based care during this period.

This study was deemed part of a quality improvement and met the criteria for exemption from institutional review board’s review according to NYULH institutional policy. All data were collected as part of routine clinical care and administrative management.

Definitions of key variables associated with study measures and analysis are provided in Table 1 and Table 2.

We first computed telemedicine intensity, clinical load, WOW, and WOW per appointment for all physicians in the EHR that met our inclusion criteria. To evaluate whether WOW significantly varied across time periods, we ran one-way ANOVAs on both WOW and WOW per appointment. To evaluate the effect of telemedicine intensity and time period on after-hours work burden, as well as whether the relationship between telemedicine intensity and after-hours work varied across time periods, we conducted a hierarchical linear regression analysis in which the dependent variable was WOW per appointment. We first entered the main effects of telemedicine intensity and pandemic time period, followed by the interaction of telemedicine intensity and pandemic time period. To understand the nature of the interaction of telemedicine intensity and pandemic time period, we partitioned the data by time period and regressed WOW per appointment on telemedicine intensity in each time period. All analyses were conducted using SPSS (version 28; IBM Corp).

Our study found that telemedicine was less efficient than in-person–based care and increased physicians’ WOW burden. The overall EHR-based WOW declined for physicians in the context of the COVID-19 pandemic and the rapid transition to telemedicine; however, when controlling for changes in patient volume and clinical hours of care, physicians who devoted a higher proportion of their clinical time to telemedicine had higher levels of EHR-based WOW than those who used telemedicine less intensively. This relationship was present during all phases of the study (before the pandemic, during acute pandemic, and after acute pandemic) and was amplified over time, including in the after acute pandemic phase. These findings suggest that the observed decrease in the average WOW during the pandemic was the result of the overall decrease in clinical load for physicians rather than any benefits or efficiencies of telemedicine itself. Further, the amplification of the relationship between WOW per appointment and telemedicine intensity in the time period beyond the acute pandemic suggests that the WOW increasing effect of telemedicine was exacerbated over time, and therefore, the unique circumstances of the early COVID-19 pandemic alone are insufficient to explain the behavior patterns of physicians.

There are several limitations to this study that future research could address. First, limitations in our Epic-based data set preclude the ability to review and analyze physician EHR activity with sufficient granularity beyond certain time periods; for example, time periods more specific than a calendar month or physician activity log data at smaller than 15-minute increments. Specifically, Epic does not count WOW in its time outside of scheduled hours if that work occurs within the 30 minutes before or after patient scheduled hours (a “shoulder period”), which our analysis is unable to reliably differentiate as WOW time and therefore excludes, resulting in a systematic underestimation of the true WOW. Moreover, because shoulder time is added for each clinical day regardless of length, this underestimation bias is greater for physicians who spread their patient time over more scheduled days relative to those who see the same number of patients on fewer days [11]. Similarly, we are unable to target more specific times of pandemic disruption (eg, March 15, which is the exact date when most of our institution’s ambulatory clinics closed for in-person care). Second, we are limited in our ability to analyze activity at the level of physician or patient demographics; therefore, we are unable to comment on whether factors such as gender, age, or years in practice may have affected clinical load, telemedicine intensity, or WOW, and whether patient features such as patient complexity or acuity contributed to these outcomes. It is possible, for example, that telemedicine-based visits are overall less clinically intense compared to in-person visits due to differences in patient case mix, in which case our analysis would underestimate the time costs associated with telemedicine-based visits. Third, EHR-based data and work represent only part of the overall nonclinical burden of physicians. Additional time spent reviewing non-EHR based records, discussing care plans, working with interdisciplinary teams (eg, nurses and care managers), or advocating with insurers is not captured in this study; this work may have been increased during the pandemic due to disruptions in traditional office practices and workflows. Additionally, as our data are behavioral, we are unable to directly associate our measures with important factors such as physicians’ attitudes (eg, stress and burnout). Finally, our findings represent only the experience of physicians at a single health care system during the unusual period of the COVID-19 pandemic and the rapid transition to telemedicine, which may limit generalizability across provider type, practice environment, or geographic location.

To our knowledge, this is the first study to systematically evaluate the impact of the transition to telemedicine during the COVID-19 pandemic on physicians’ after-hours workload and one of a few studies that used EHR-based data to objectively evaluate after-hours work burden [15-17]. Although research documenting the experience of health systems undergoing the transition to telemedicine in response to the pandemic has increased [18-20], there is limited research exploring the effects of telemedicine on health care delivery areas such as clinical workflows, administrative load, or practice efficiencies, either during the pandemic or prior to it; the most robust of these works are almost a decade old and reflect a dated telemedicine environment that may no longer be relevant to the current context of health care delivery [21,22]. Similarly, literature exploring the impact of telemedicine on important aspects of physician work experience such as burnout and quality of life are limited, with the majority of work prior to the pandemic coming out of the field of telepsychiatry as an “early adopter” of the technology [23,24]. This study contributes to the literature on telemedicine in health care by exploring both the novel context of its expansion during the COVID-19 pandemic and its relationship to EHR-based work burden for clinicians.

A number of factors may be responsible for our findings that telemedicine increased the after-hours work burden of physicians. First, it is possible that organizational and technological inefficiencies in the early design, deployment, and scaling of telemedicine may have resulted in increased after-hours EHR work burden for physicians using telemedicine more intensively. These include early and ongoing technological issues relating to the computer hardware, software functionality and integrations, and user experience of the “virtual health” platform deployed by our system. These issues have been highlighted elsewhere in EHR and digital health technology implementation research, particularly regarding usability and user experience barriers [25-27] exacerbated by the scale and abruptness of the transition to telemedicine due to the pandemic [28]. However, technological inefficiencies should be at least partially ameliorated over time as physicians learn to navigate and optimize their setup and systems (the “learning curve”), an assumption that is not supported by our after acute pandemic period findings of a continued “amplified” relationship between telemedicine intensity and after-hours EHR work. Similarly, telemedicine training for physicians during this period of rapid expansion was often ad hoc and likely suboptimal for the development of effective telemedicine competencies (eg, efficient platform navigation, technical troubleshooting, “virtual health” EHR documentation), and thereby, potentially worsening WOW; however, this would be expected to improve with time as physicians adapted their workflows and learned new skills, rather than, as our results found, establishing a pattern of increasing work burden in the later periods of telemedicine of use, even as access to quality telemedicine trainings and best practice knowledge sharing improved among institutions. This suggests that “virtual health” training as it existed during the early phases of the pandemic was not sufficient to improve after-hours work burden for physicians. Further exploration of the relationship between telemedicine training and “virtual health” practice patterns (including EHR-based activities) is warranted as training becomes more regularly integrated into medical education and professional development.

The second factor that might have impacted our findings is that it is likely that significant disruptions to the work norms of clinical practices during the pandemic affected after-hours work patterns. In clinics, individual- and practice-level adjustments to the demands of care provision during the pandemic likely resulted in a number of unique work structures and arrangements that could have likely affected physicians’ work schedules, including time spent doing after-hours work. In particular, the shift to a telemedicine-based platform—particularly one with limited multiparty functionality—may have inhibited effective team-based care between physicians and clinical support staff (eg, medical assistants) and shifted both clinical and administrative tasks that had prior been completed by other staff members onto physicians. This “doctor does it all” phenomenon has been recently described as an unintended effect of the rapid transition to telemedicine during the pandemic [29]; within our own system, much of the current WOW involves responding to patient messages, phone calls, refill requests, and completing various EHR documentation requirements often left for the end of the day after direct patient care responsibilities are ended. NYULH is actively engaged in reducing this burden on providers by redistributing relevant work to support staff, as well as using novel technologies including machine learning to facilitate message triage and management, for example by suppressing messages that are not actionable by providers. More work is needed to fully understand the impact of the new virtual-first models of care delivery on interdisciplinary teams and team-based practice.

Overall, our results suggest that telemedicine is not panacea for the work challenges facing clinicians. In fact, our evidence during the acute pandemic and after the acute pandemic suggests that rather than reducing administrative burden, telemedicine intensity may increase it, shifting the work temporally and spatially to after-hours work and home. This suggests that a more thorough understanding of the implications of telemedicine in clinical practice is necessary prior to its indiscriminate expansion to ensure policies and practices that increase efficiency and work-life quality and counteract inefficiencies, waste, and work-related stress and burnout are implemented. Given the limited data available on the impact of telemedicine on important aspects of physicians’ experience of work, it may be instructive to look to fields outside of medicine, where the study of “telework” (defined as a work arrangement that allows employees to perform work at approved alternative or remote worksites) [29] is more robust. Research in the industries of engineering, consulting, and software development has demonstrated varying effects of remote work on key elements of employees’ work experience. Positive effects of telework in these fields include increased job satisfaction, performance, and work-life balance, as well as reduced employee turnover, real estate costs, commute time, and environmental impact [30-32]. Conversely, negative effects include reduced career development and feelings of reduced energy, confidence, and engagement due to the loss of high-quality interaction with colleagues and clients [33,34]. Significantly, telework has been associated with workers’ inability to disconnect from their work and increased stress-inducing work intensification [35,36]. This relationship may apply to telemedicine and help explain the findings in this study. Although more investigation is needed to understand the full scope and implications of medical telework beyond the direct care provided by telemedicine (including tasks such as remote teaching, non-EHR–based clinical work, administrative work, and research), general learning from these fields may help identify and guide key areas of future telemedicine and telework research.

In this study, we evaluated the impact of the transition to telemedicine during the COVID-19 pandemic on physicians’ EHR-based after-hours workload; we found that when controlling for the clinical load of patient visits, physicians who devoted a higher proportion of their clinical time to telemedicine engaged in higher levels of EHR-based after-hours work compared to those who used telemedicine less intensively; this relationship persisted and was amplified over time, even after the acute pandemic period. This suggests that telemedicine, as currently delivered, may be less efficient than in-person–based care and may contribute to after-hours work burden of physicians. Further study is needed on the detailed impacts of telemedicine on physician work practices, particularly in contexts beyond the COVID-19 pandemic and relating to administrative burden, after-hours clinical responsibilities (particularly the EHR-related in-basket and patient portal messaging responsibilities), and experience of work. Learning from other industries where telework is more established can help identify areas of need and opportunity in future telemedicine care delivery.