BACKGROUND: Access to primary care outside of regular working hours is limited in many countries. This study investigates the relation between the after-hours premium, an incentive for primary care physicians to provide services after hours, and less-urgent visits to the emergency department in Ontario, Canada.
METHODS: We analyzed a retrospective cohort of a random sample of Ontario residents from April 2002 to March 2006, and a subcohort of patients followed from April 2005 to March 2016. We linked patient and primary care physician data with emergency department visit data. We used fixed-effects regression models to analyze the association between the introduction of the after-hours premium, as well as subsequent increases in the value of the premium, and the number of monthly emergency department visits.
RESULTS: The sample consisted of 586 534 patients between 2002 and 2006, and 201 594 patients from 2005 to 2016. After controlling for patient and physician characteristics, seasonality and time-invariant patient confounding factors, introduction of the after-hours premium was associated with a reduction of 1.26 less-urgent visits to the emergency department per 1000 patients per month (95% confidence interval −1.48 to −1.04). Most of this reduction was observed in after-hours visits. Sensitivity analysis showed that the monthly reduction in less-urgent visits to the emergency department was in the range of −1.24 to −1.16 per 1000 patients. Subsequent increases in the after-hours premium were associated with a small reduction in less-urgent visits to the emergency department.
INTERPRETATION: Ontario’s experience suggests that incentivizing physicians to improve access to after-hours primary care reduces some less-urgent visits to the emergency department. Other jurisdictions may consider incentives to limit less-urgent visits to the emergency department.
One prominent health policy issue confronting many countries is overcrowding of the emergency department.1 Not only does overcrowding result in longer wait times in the emergency department, but it may be associated with patient dissatisfaction and higher risk of death,2–4 as well as contribute to higher health system costs.5–7 Use of the emergency department by patients with conditions treatable in primary care may be a factor that contributes to emergency department overcrowding,4,8,9 and improved after-hours access to primary care is a potential solution.10
Following Canada’s initiatives on primary care reform in the early 2000s, the Ontario government introduced several patient enrolment models (PEMs) for primary care delivery.11 These models were characterized by mandatory patient enrolment, group-based practice and blended remuneration, including retrospective and prospective payments and pay-for-performance incentives. In July 2003, the Ontario government introduced the after-hours premium, an incentive for physicians practising in PEMs to claim an additional 10% on specific services provided to enrolled patients after regular business hours (5 pm to 8 am on weekdays, and any time on weekends and holidays). The after-hours premium increased to 15% in April 2005, 20% in April 2006 and to 30% in September 2011. One study examined the effect of enrolment in a PEM on overall emergency department visits, thereby masking any differential effects on urgent and less-urgent visits.12 Using physician-level data, a recent study examined the impact of the increase in the after-hours premium from 10% to 20%.13 We build on this literature and examine whether the introduction of Ontario’s after-hours premium, and subsequent increases in the premium, were associated with changes in emergency department visits, stratified by visit urgency. In particular, we examine whether the premium was associated with reductions in less-urgent visits to the emergency department.
We conducted a retrospective cohort study using linked health administrative data to investigate the effect of the after-hours premium on emergency department visits over 2 periods. The results are reported according to the Strength in Reporting of Observational Studies in Epidemiology guidelines.14 We examined a 10% random sample of Ontario residents in 2002, following them in 2 discrete periods up to 2016. We excluded patients who did not have complete data over the entire study period, as well as patients who lived in rural areas because of rural–urban differences in primary care practice.15 In the first period, Apr. 1, 2002, to Mar. 31, 2006, we examine the effect of the premium’s introduction and of the change in its value from 10% to 15%. Follow-up was limited to 2006 as several other incentives and new PEMs were introduced after 2006.11 In the second period, we followed a subcohort of patients, all of whom were enrolled in a PEM from Apr. 1, 2005, to Mar. 31, 2016, investigating the effect of subsequent increases in the premium.
Data sources and variables
Seven health administrative databases were linked using encoded identifiers, as follows: the Corporate Provider Database, for physician demographic characteristics and physician model type; the ICES Physician Database, for additional physician practice characteristics; the Client Agency Program Enrolment database, for linking enrolled patients with their physicians; the Ontario Health Insurance Plan Claims Database, for physician billings; the Registered Persons Database, for patient demographic characteristics; Canadian Census data, for dissemination area-level income; and the National Ambulatory Care Reporting System database, for emergency department visits.
The Client Agency Program Enrolment database captures all patients enrolled to family physicians in PEMs, enabling us to link patients with physicians in the PEM group.16 All other patients were assigned as fee-for-service (FFS) patients to the physician who claimed the highest billings in the previous 2 years.17,18 Patients were categorized by their physician’s eligibility to bill for the after-hours premium each month. Physicians who practised in PEMs were eligible to bill for the after-hours premium for their enrolled patients; physicians who practised in the traditional FFS model were not. PEMs are comprised of blended FFS, blended capitation and other specialized models, the details of which are described elsewhere.11
The outcome was the number of emergency department visits per patient per month, stratified by urgency and timing. Urgency was defined using the Canadian Triage and Acuity Scale scores.19 Emergency department visits with a score of 1 or 2 were defined as very urgent, a score of 3 was defined as urgent and scores of 4 or 5 were defined as less urgent (hereafter referred to as very urgent, urgent and less-urgent visits, respectively). Timing was split into regular hours (weekdays 8 am to 5 pm) and after hours (5 pm to 8 am on weekdays, and weekends and statutory holidays).
We included patient age, sex, income and comorbidity in our models. We defined low income as being in the 2 lowest area-level income quintiles. We used the Johns Hopkins Adjusted Clinical Groups software (Version 11.0) to define comorbidity as the number of adjusted diagnosis groups (maximum 32 groups). Physician characteristics included age, sex, international medical graduate status, experience (defined as years since graduation) and physician practice group size.
We used fixed-effects linear regression models to assess the effect of the introduction of the after-hours premium and increases in the premium on monthly emergency department visits at the patient level. Under certain reasonable assumptions, the results of a fixed-effects model may be interpreted to show causality.20 Fixed-effects models control for the observable and unobservable patient confounding factors that do not vary over time. Clustering of patients within physicians was allowed, to obtain valid standard errors.21 The after-hours premium was constructed as a categorical variable and set to the value of the premium (i.e., 10%, 15%, 20% or 30%) for patients whose physician was eligible to bill for the after-hours premium. The value of the premium for all patients before its introduction and for patients whose physician practised in the FFS model was set to 0. We adjusted for patient and physician characteristics, as well as for seasonal effects using monthly dummies.
One issue with the fixed-effects approach is that patients whose physician switched to a PEM between 2002 and 2006 and those who remained in the traditional FFS may be different. To address this potential selection bias, we used inverse probability weighting based on estimated propensity scores22–25 and entropy balancing weights26,27 to ensure that all observable physician characteristics, patient characteristics and emergency department visits were similar across both groups of patients before the introduction of the premium. Thus, any change in visits to the emergency department could be attributed to the after-hours premium. This methodological approach has been used in several recent papers to study the effect of reform on other outcomes.28–31 We also assessed the effect of the after-hours premium for a subsample of patients whose physician billed the premium at least once in a month.
Analysis was conducted at the patient level using separate models for each level of urgency and timing. The estimated coefficients were multiplied by 1000 for ease of interpretation. The β coefficient is the impact of the after-hours premium on the change in the number of emergency department visits per 1000 patients per month. Analysis was performed using Stata 15.1 and the user-written Stata program, reghdfe.
The use of data in this study was authorized under Section 45 of Ontario’s Personal Health Information Protection Act; thus, review by a research ethics board was not needed.
The construction of our study cohorts is presented in Figure 1. The sample for the first study period (2002–2006) consisted of 586 534 patients. The rate of emergency department visits increased from 28.19 per 1000 patients per month in 2002/03 to 28.99 per 1000 patients per month in 2005/06 (Table 1). This trend was driven by an increase in very urgent and urgent visits, while less-urgent visits decreased from 15.27 to 12.09 per 1000 patients per month. A slightly larger reduction in less-urgent visits was found during after-hours periods compared with regular hours (Figure 2).
We found that the introduction of the after-hours premium was associated with a reduction in less-urgent visits by 1.26 per 1000 patients per month (95% confidence interval [CI] −1.48 to −1.04) (Table 2). However, the reduction was larger after hours than during regular hours, with a reduction of 0.87 (95% CI −1.03 to −0.72) and 0.39 (95% CI −0.51 to −0.26) less-urgent visits per 1000 patients per month, respectively (Table 3). Although a reduction was found in less-urgent visits, there were small increases in urgent and very urgent visits. The increase in the after-hours premium from 10% to 15% was not associated with a change in less-urgent visits; however, the premium was associated with a small decrease in less-urgent visits after hours by 0.17 per 1000 patients per month (95% CI −0.32 to −0.02). The increase in the value of the premium was associated with increases in urgent and very urgent visits.
Results using inverse propensity score weights and entropy balancing weights (Appendices 1–3, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200277/tab-related-content) produced slight attenuation of the effects on less-urgent visits, in the range of −1.24 (95% CI −1.46 to −1.02) and −1.16 (95% CI −1.39 to −0.94) per 1000 patients per month (Table 4). However, with entropy balancing weights, increases in the value of the premium were no longer associated with a reduction in less-urgent visits after hours. The effect of introduction of the premium was slightly attenuated when examining only patients whose physician billed the after-hours premium (Appendix 4, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200277/tab-related-content). In this subsample, the effect of increases in the premium value on less-urgent visits after hours was no longer significant.
Of the initial sample, 201 594 patients remained enrolled in a PEM for the entire period between Apr. 1, 2005, and Mar. 31, 2016 (Appendix 5, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200277/tab-related-content). The rate of emergency department visits for this subcohort increased over time, from 23.9 per 1000 patients per month in 2005/06 to 33.2 per 1000 patients per month in 2015/16. Similar to the first period, this increase was driven by very urgent and urgent visits, while less-urgent visits decreased from 11.2 to 9.7 per 1000 patients per month, with a larger reduction in visits after hours compared with regular hours.
Although the increase in the premium from 15% to 20% was not associated with reductions in less-urgent visits, the increase from 20% to 30% was associated with a small reduction in less-urgent visits, especially after hours (Table 5). We found a reduction of 0.17 (95% CI −0.31 to −0.03) and 0.13 (95% CI −0.25 to −0.02) emergency department visits per 1000 patients per month after hours when the premium increased from 15% to 20% and from 20% to 30%, respectively. Although less-urgent visits decreased, we found an increase in very urgent and urgent visits to the emergency department.
The introduction of an after-hours premium for primary care practitioners in Ontario reduced less-urgent visits to the emergency department, with much of the reduction found in visits after hours rather than during regular hours. The effect of the introduction of the after-hours premium was similar when accounting for potential selection bias. Using the lower bound from the entropy-balanced weighted estimate of 1.16 fewer less-urgent visits per 1000 patients per month (95% CI −1.39 to −0.94), and given that about 10 million residents were enrolled in a PEM by March 2011,32 the introduction of the after-hours premium was associated with a reduction of 139 600 less-urgent visits (95% CI 112 600 to 166 700) per annum. Subsequent increases in the premium were associated with a very small reduction in less-urgent visits after hours. We found that the introduction of the after-hours premium reduced some less-urgent visits, suggesting that some emergency department visits can be avoided through improved access to primary care.
Previous studies found that interventions to improve access to after-hours primary care can be associated with a reduction in emergency department use.33–38 Our results are consistent with studies that found that incentivizing physicians to be available for longer hours was associated with a small reduction in nonurgent emergency department visits.39 However, our study conflicts with another that found that enrolment in a PEM was associated with an increase in emergency department visits in Ontario.12 That study’s findings are likely owing to the use of overall emergency department visits without a comparison group, and the lack of control for patient-specific time-invariant confounding factors. In contrast, a more recent study at the physician level found that increases in the after-hours premium from 10% to 20% were associated with a small reduction in less-urgent visits to the emergency department.13 Many studies have explored the effects of interventions not directly related to the emergency department on its use; however, the impact of these programs varies vastly in the context of managed care setting with financial incentives.40 Our results regarding less-urgent visits to the emergency department fall closer to the lower end of the effectiveness of financial incentives found in the literature.
Further research is required to understand the cost-effectiveness of the after-hours premium and determine whether the after-hours premium is a cost-saving measure for the health care system. Further research is also required to examine whether it has led to improved access to primary care for patients with unmet needs and improved patient satisfaction.
First, we were unable to account for the role of individual-level socioeconomic factors that may influence emergency department visits. Second, while the first period analyses reflect the effect of the after-hours premium, as it was not contaminated with other policies, the analyses in our second period capture the combined effects of the after-hours premium and other pay-for-performance incentives introduced in the post-2006 era. We were not able to disentangle the effects of the increased after-hours premium (20% to 30%) from other incentives as some services are covered by multiple incentives in the post-2006 period. Third, whether the premium is actually associated with urgent visits or whether other factors contribute to this association requires further investigation. About 25% of the sample were excluded with incomplete follow-up data, including patients who had died, who had moved out of province, or who could not be traced in ICES databases. Although exclusion of patients who died is unlikely to affect the number of less-urgent visits to the emergency department, it may bias our results of very urgent and urgent visits. Fourth, there may be some residual confounding if there is unobservable physician selection into PEMs. Fifth, we are unable to account for the role of walk-in clinics as we did not have accurate data on these clinics. Finally, our data set was unable to capture whether patients attempted to seek care from a primary care physician before seeking care in the emergency department or whether they were referred to the emergency department by their physicians.
The after-hours premium reduced some less-urgent visits to the emergency department in Ontario. Interventions to reduce potentially avoidable emergency department visits are of considerable interest to policy-makers; the use of incentives to promote access to after-hours primary care and divert patients away from the emergency department in a cost-effective manner is one policy option to consider.
The authors thank former ICES analyst Lihua Li and current ICES analyst Alex Ouedraogo for cutting the data for this study and other administrative support.
Competing interests: None declared.
This article has been peer reviewed.
Contributors: All authors contributed to the conceptualization and design of the study. Michael Hong and Sisira Sarma analyzed and interpreted the data. Michael Hong drafted the manuscript. All of the authors critically revised the manuscript for important intellectual content, approved the final version to be published and agreed to be accountable for all aspects of the work.
Funding: This study was supported by the Canadian Institutes of Health Research (CIHR) Operating Grant (MOP-130354) and the Ontario Ministry of Research and Innovation Early Researcher Award. Michael Hong was supported by the CIHR Canadian Graduate Scholarship and the Ontario Graduate Scholarship.
Data sharing: The data used in this study are held securely in coded form at ICES Unix server. Although data sharing agreements prohibit ICES from making the data publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at https://www.ices.on.ca/DAS. The Stata syntax codes to run various models are available from the authors on request.
Disclaimer: This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). Parts of this material and data are based on information from the Canadian Institute for Health Information (CIHI). The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES, CIHI or the Ontario MOHLTC is intended or should be inferred.
- Accepted August 19, 2020.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/