The practice was a semirural interprofessional clinic in southeastern Ontario, Canada, where 12 family physicians and other allied health providers provide comprehensive primary care under a single-payer model. Under this publicly funded model, physician compensation is primarily through capitation payments for rostered patients. The primary care patient portal initially offered access to laboratory results, the ability to enter vital signs such as blood pressure measurements, and the ability to view when certain screening maneuvers were due. Additional features were introduced over time, including the ability to receive secure messages (in 2012), send secure messages (in 2015), book appointments (in 2016), and renew prescriptions (in 2018.) All practice patients were invited to join the portal via email, posters, and telephone reminders and at in-person encounters.

We collected data for all practice patients except those seen exclusively for focused care (eg, obstetrical care). We retained data from all patients only for the period they were aged 18 years or older. Among patients who adopted the portal, we excluded those for whom we did not have at least one year of data prior to and following their portal registration (ie, index) date. For non–portal patients, we excluded those who did not have at least two consecutive years of data between 2009 and 2019.

We calculated propensity scores to estimate the probability of individuals registering for the portal using logistic regression [27,28]. Propensity scores were derived based on sex, age, whether the patient was rostered to the family health team, the presence of specific diagnoses on the index date, and the number of in-person and telephone encounters, as well as the number of unique medications prescribed in the 12 months prior to the index date. During the study time period, all appointments with medical doctors (MDs), nurse practitioners (NPs), and physician assistants (PAs) were in person. Since education and income level were recorded for approximately a third of patients, these measures were not included in the propensity score matching. Control patients were entered into the equation for each year they were eligible (ie, for each year they had at least one year of data prior to and after the index date), with their corresponding profile for that year. July 1 of that year was considered the index date.

The study period was January 2002 through December 2019. We extracted electronic medical record information on patient age, sex, education, income, enrollment with the practice, and presence or absence of specific diagnoses on the index date. We also extracted the dates of in-person encounters with MDs, NPs, and PAs; dates of triage calls (TCs) to the practice triage nurses; and prescription dates and prescribed medications. Prescribed medications included only those that were identified as distinct medications using Anatomical Therapeutic Chemical codes. Diagnoses were defined based on diagnostic codes, using the earliest date when the diagnostic code was applied.

In order to study the clinician workload associated with the portal, two providers (KF and MF) time stamped their portal messages between February 20, 2020, and February 25, 2021. This allowed us to estimate the average provider time spent per message. We also collected the total number of portal messages sent by all providers to all portal patients between January 1, 2019, and December 31, 2019, in order to determine the average amount of time spent per patient on portal-related work.

We described the profile of eligible patients prior to matching on their index date for portal patients on July 1 of the median year for which they were eligible to be matched for non–portal patients and again for the matched patients on their index date in both arms. The main study outcome was the frequency of in-person encounters with MDs, NPs, or PAs, as well as frequency of TCs. We used an ITS design to evaluate the impact of portal registration on use of these traditional health care services over time and compared use by portal users to their matched controls. We present the results in the usual ITS format, defining time relative to the index date with year 0 representing the 12 months preceding and including the index date and each time unit representing a 12-month interval. The ITS model includes time as a linear variable to model for an underlying linear time trend and the portal enrollment (ie, the intervention) as a dummy variable. Intervention and time interaction is also included in the model to identify the effect of the intervention on both arms (ie, portal and non–portal) over time.

We plotted the annual number of in-person encounters with MDs, in-person encounters with NPs or PAs, and TCs across time and overlaid the estimates derived from the ITS equations. During the study time period, all appointments with MDs, NPs, and PAs were in person. Because the year-0 results showed a spike in service use in both study groups, likely related to the attribution of the index date, we excluded that year from the ITS model. Also, although the spike in service use at year 1 in the portal arm may represent a transient change in behavior associated with the initial adoption, we also excluded this from the ITS to obtain a more reliable estimate of the impact of portal adoption over time, recognizing that this approach omits significant use; this should be considered in result interpretation.

We also depicted the number of visits per calendar year for patients who adopted the portal grouped by year of portal registration to demonstrate the pattern of changes in these visits over time for the intervention arm.

Ethics approval was received from the Bruyère Research Ethics Board (M16-20-012).

Of the 14,894 patients who met the study criteria, 7247 (48.7%) were portal participants. Of these, 3696 were matched one to one with a control patient (Figure 1). The profile of all eligible patients before and after propensity matching is shown in Table 1. Before matching, portal users differed from non–portal users, but after matching, the mean propensity scores of the 2 groups and their index years, the prevalence of chronic conditions, sex, rostering status, and total visits and medications in the previous years showed good agreement. Income and education levels, which could not be included in the propensity score derivation because of poor data completeness, remained higher in the portal group.

We used a caliper of 0.2 for matching and limited the potential matching pool for each portal patient to non–portal patients with an index date that was within 1 year of the portal patient’s index date. We identified all potential controls for each portal patient and assigned matches prioritizing first portal patients who had a unique match, then non–portal patients who had a unique portal match. We repeated this after each match to minimize loss of controls. When more than one match was possible, we attributed the control patient whose propensity score was closest to the portal patient’s score. The balance of baseline covariates between the matched portal users and non–portal users was assessed using standardized differences, with values <0.1 representing negligible differences.

We plotted the number of visits in relation to the index year for portal and control patients, the estimated slopes for the years before and after the transition, and the shift in visits at the index date derived from the ITS equation (Figures 2-4). The information for the years prior to the index date for portal patients demonstrates their annual visits for the years prior to their registration on the portal. The index date for the control patients was assigned to be within 1 year of the portal’s patient index date in order to control for temporal factors such as health care use trends. The outputs of the ITS analyses are provided in Table 2. The intercepts and slopes prior to the index year were similar in the control and portal arms for MDs, NPs/PAs, and TCs (P>.05). After the index year, there was a significant jump in MD visits in the portal arm (0.42 more visits/year vs control, P<.001) but not for NP or PA visits. The TCs also showed a greater increase in visits in the portal arm compared to the control arm after the index year (0.102 more visits/year vs control, P=.006).

We also plotted the visit rates for each year for patients having enrolled in the portal, grouped by year of portal registration (Figures 5-7).

The 2 physicians who time stamped 2061 portal messages spent an average of 3.83 minutes on each message. We also extracted the total number of portal messages sent by all providers between January 1 and December 31, 2019, and found that an average of 1.49 messages were sent to each portal patient in the practice. Thus, the average amount of provider time devoted to portal messages was estimated to be 5.7 minutes per portal patient per year.

Our findings suggest that portal registration is associated with an increase in service use, but that some reductions may be expected over subsequent years. Compared to matched controls, portal registration was associated with a significant initial increase in the number of in-person MD encounters and telephone calls, but a subsequent drop in the rate of MD visits and increase in NP visits over time. MDs spent an estimated 5.7 minutes per patient annually to respond to portal messages.

We believe that ours is the first study to examine the trend in encounters after portal registration over an extended time span and the first study to examine the impact of an exclusively primary care portal on traditional health care usage. It is possible that the observed increase in encounters was due to differences between the two groups that were not captured in the propensity matching. For instance, patients might have registered on the portal when they developed a new health concern, anticipating an increased requirement for health services. The reason for the gradual decrease in MD visits but increase in NP visits that took place after the initial jump in MD visits associated with portal registration is difficult to determine without further study. It is possible that patients initially presented to their own physician after sending them a portal message or viewing results, but the physician shared follow-up care with the nurse practitioner or physician assistant.

There may have been differences in areas such as electronic literacy or internet access that were not identified. It is also possible that the higher frequency of in-person encounters after portal registration was due to increased engagement by patients in their health. For instance, access to laboratory results may have generated questions from patients [29]. Increased awareness of being due for cancer screening or diabetes or blood pressure monitoring may have resulted in a higher number of encounters but improved quality of care or patient satisfaction. We did not examine these areas as they were beyond the scope of this study, but they would benefit from future research. While some past studies demonstrated improvements in certain health outcomes associated with electronic patient portals [30-33], only a few were based in primary care [31,34]. Several systematic reviews that evaluated a variety of portals in different practice settings suggested that portals or similar digital health services may result in improved patient satisfaction, but they did not demonstrate a meaningful impact on health outcomes, cost, or use [35-40].

We found that providers spent less than 6 minutes per year on clerical work for each patient registered on the portal. This is a small amount of time per patient but is significant when considering the context of an entire primary care practice. We note that the time-stamping of messages was performed during the COVID-19 pandemic, while the number of messages sent by all providers was collected prior to 2020. We consider that even if the COVID-19 pandemic resulted in an increased number of messages, the provider time per message would not have changed significantly. Therefore, our estimate of portal-related clerical work reflects prepandemic time requirements, and these may have increased since 2020 due to increased patient interest in asynchronous virtual care. This would also be an area for further study. Portals that do not allow incoming messages or any secure messaging would reduce or eliminate this time requirement but might also limit patient engagement and other potential benefits of the portal. Since the clerical burden associated with electronic environments in health care has been associated with professional burnout, [22,26] it is important to consider the provider time requirement associated with patient portals. The time and cost associated with incorporating a patient portal are currently not specifically addressed in either fee-for-service or capitated Canadian primary care funding models.

There are other limitations to this study. We examined the long-term impact of an electronic patient portal in a single primary care practice, which may not be reflective of the impact in other primary care practices. However, portal adoption has not been widespread for long enough to allow study of the long-term impact of portals across multiple sites. Additionally, the impact of patient portals in other settings, such as hospitals, laboratories, or specialist practices, may be quite different. Further research is needed into electronic patient portals in different settings to determine their impact on various health outcomes.

Electronic patient portals are increasingly being adopted by providers and sought after by patients. We found that portal registration was associated with a subsequent increase in the number of some traditional encounters and a small increase in clerical workload for providers. Portals have enormous potential to truly engage patients as partners in their own health care, but their impact on use of traditional health care services and clerical burden must also be considered when they are incorporated into primary care.