We previously conducted a stepped wedge cluster-randomized clinical trial to evaluate the impact of a video game intervention, designed to motivate hospitalists toward having more Advance Care Planning (ACP) conversations with older patients, on ACP billing frequency. The trial involved 163 hospitalists from 35 hospitals across the U.S., with intervention rollout occurring from October 2020 to February 2021. A summary of baseline characteristics of included patients stratified by the occurrence of ACP billing status is shown in Table 1. Participating physicians were provided iPads preloaded with the game and asked to engage with it for at least two hours. The primary outcome, ACP billing, served as a proxy for ACP conversation initiation, with data abstractors blinded to the intervention status of physicians. Results indicated no significant increase in ACP billing after the intervention, with a pooled odds ratio (OR) of 0.96 (95% CI 0.88–1.06, p = 0.42). Secondary analyses revealed a heterogeneous intervention effect across steps, with increases in ACP billing in the initial three steps but declines in the latter two, possibly due to confounding from the COVID-19 pandemic, which varied in impact across trial sites (Mohan et al. 2021, 2023).

Table 1 Baseline characteristics of patients by occurrence of advance care planning (ACP) discussions5.1 Evaluating spillover in ACP

Our prior work demonstrated that within hospitals, hospitalists and primary care physicians form nearly complete social networks based on shared patient care, remaining densely connected even under stringent thresholds for shared encounters (Bobak et al. 2023). This pervasive interconnectivity suggests extensive opportunities for collaboration and peer influence among hospital-based physicians. Beyond individual institutions, hospitalists also participate in a broader, though sparser, inter-hospital network linking multiple sites through shared clinical activity. Together, these patterns indicate that physician behavior is shaped by overlapping local and cross-institutional ties, implying that interventions targeting individual hospitals may be susceptible to spillover effects as professional interactions extend beyond cluster boundaries.

Following these observations, we sought to directly measure spillover by evaluating all physicians involved in patient encounters, aiming to identify the extent to which patients received care from a physician who had already been exposed to the intervention in a prior step of the trial (because the intervention is at the hospital-level, these physicians must be at a different hospital to the physician(s) a patient encounters at a hospital yet to receive the intervention), rather than the expected step assigned to the hospital where the encounter occurred. Our analysis revealed that 2305 (5.21%) eligible patient stays involved encounters with a physician who had been intervened upon prior to the assigned trial step.

5.2 Visualizing spillover networks

The initial spillover measure does not capture how intervened physicians may further diffuse the intervention’s influence to their peers-either directly through collaborative ACP activities or indirectly via shifts in shared practice norms. To examine these broader, cascading effects, we constructed physician networks based on shared patient encounters and visualized their evolution across trial steps (Fig. 1). Panel A shows the baseline network colored by each hospital’s assigned intervention step, while Panels B–F depict the sequential rollout, highlighting direct intervention recipients (dark green), their immediate collaborators (light green), and second-degree neighbors (yellow). This visualization illustrates the progressive spread of intervention influence through both direct and indirect professional connections over time.

Fig. 1

Progression of Intervention influence across physician networks. This figure illustrates the evolving physician network across trial steps, visualizing the potential spread of the intervention’s impact from directly intervened physicians to their broader collaborative network. A Baseline network with nodes colored by each hospital’s assigned intervention step. B–F represent the sequential rollout of the intervention across steps 1 to 5, where dark green nodes denote direct intervention recipients, light green nodes represent first-degree neighbors (direct collaborators of intervention recipients), and yellow nodes depict second-degree neighbors (collaborators of the collaborators)

5.3 Adjusting for physician spillover5.3.1 Intent-to-treat formulation (cross-hospital spillover only)

We begin by presenting results from the intent-to-treat (ITT) formulation, which defines exposure based on the randomized cluster assignment of hospitals to the intervention. In this specification, the intervention indicator reflects whether a patient’s hospital had initiated the intervention at a given step, while spillover is quantified using a time-varying network restricted to cross-hospital connections. This approach preserves the randomized structure of the stepped-wedge trial and estimates the direct effect of assignment to intervention while adjusting for potential contamination arising from professional interactions across institutions. By isolating within-hospital diffusion to the cluster-level intervention term, the ITT models provide a conservative estimate of the intervention’s effectiveness under real-world conditions of partial interference.

Table 2 Model estimates for the intent-to-treat formulation (cross-hospital spillover only)

Table 2 summarizes model estimates for the intent-to-treat formulation, in which spillover was calculated using time-varying networks that excluded within-hospital connections. In the intervention-only model, the estimated effect of the intervention on ACP billing was not statistically significant (OR 0.96, 95% CI 0.88–1.06, \(p = 0.42\)). After including the cross-hospital spillover term, the estimated intervention effect decreased slightly in magnitude (OR 0.91, 95% CI 0.83–1.00, \(p = 0.042\)), reaching borderline statistical significance. When the interaction between intervention and spillover was added, the direct intervention effect again became non-significant (OR 0.97, 95% CI 0.88–1.07, \(p = 0.53\)).

Spillover exposure was strongly associated with higher odds of ACP billing in both models that included terms involving the physician the network. In the additive model, a one-unit increase in spillover exposure corresponded to a 5.3% increase in the odds of ACP billing (OR 1.053, 95% CI 1.045–1.061, \(p < 0.001\)). When the interaction term was included, the spillover effect increased further (OR 1.19, 95% CI 1.13–1.26, \(p < 0.001\)), suggesting that cross-hospital diffusion played a substantial role in propagating the intervention’s influence.

The significant negative interaction between intervention and spillover (\(\hat_3\) = 0.88, 95% CI 0.84–0.93, \(p < 0.001\)) indicates that the direct intervention effect diminishes in the presence of greater spillover exposure. This pattern is consistent with the interpretation that diffusion of intervention-related behaviors across connected hospitals attenuates the marginal benefit of direct assignment to the intervention condition.

5.3.2 Adjusting for step-wise differences in intervention impact

Secondary analyses from our prior work indicated that the intervention may have increased ACP billing during the first three steps of the trial, suggesting a heterogeneous impact across trial stages (Mohan et al. 2023). Building on this observation, we aimed to assess the variation in intervention effects across the steps more rigorously, incorporating physician spillover to better understand the factors contributing to this heterogeneity.

Table 3 Extended intent-to-treat model including step and step-by-intervention interactions

In this extended intent-to-treat specification (Table 3), the estimated effect of the intervention (now representing the intervention effect specific to the first step of the trial) became positive for the first time (OR 1.08, 95% CI 0.94–1.24, \(p = 0.26\)), suggesting a potential increase in ACP billing among directly intervened physicians after accounting for step-level heterogeneity and pandemic-related disruption. Although the association did not reach statistical significance, the change in direction from previous negative estimates indicates that, once temporal variation and contextual effects are considered, the intervention’s impact appears more favorable. However, the above effect only pertains to the first step of the trial; the modification of this effect over the subsequent steps is described below.

The network-derived spillover effect remained strong and statistically significant (OR 1.20, 95% CI 1.13–1.27, \(p < 0.001\)), consistent with the prior model. Likewise, the negative interaction between intervention and spillover (\(\hat_3 = 0.88\), 95% CI 0.83–0.93, \(p < 0.001\)) again indicates attenuation of the direct intervention effect in settings with greater spillover exposure.

Introducing step-by-intervention interactions revealed substantial heterogeneity in intervention effects across the stepped-wedge rollout. The interaction for Step 4 (\(OR 0.63\), 95% CI 0.54–0.74, \(p < 0.001\)) was notably and significantly negative, indicating that the intervention’s effect was sharply reduced during this period. Step 4 corresponded to January 5–26, 2021, which coincided with the peak of the winter COVID-19 surge and the deadliest month of the pandemic in the United States (CDC 2025). The timing and direction of this interaction suggest that pandemic-related disruptions-such as staff turnover, changes in patient volume, and heightened clinical burden may have suppressed ACP activity even among physicians already exposed to the intervention. In contrast, interaction terms for Steps 2, 3, and 5 were small and non-significant, implying relatively stable intervention effects (similar to that for Step 1) outside of the pandemic spike. The hospital-level covariate representing the proportion of hospitalizations involving COVID-19 remained significantly associated with reduced ACP billing (OR 0.43, 95% CI 0.27–0.69, \(p < 0.001\)), reinforcing the broader influence of the pandemic on ACP documentation across the study period.

While the multiple interaction terms in a model might give the appearance of a high-level of complexity and potential instability in the estimated coefficients, the fact that the estimates of the spillover effect and the intervention-spillover interaction effect are much the same as for the model without the interactions of trial-step with the intervention suggests that such concerns are unfounded. In fact, it appears as though the intervention effect by trial-step interaction is largely orthogonal to the intervention by spillover interaction.

5.3.3 As-treated formulation (inclusive network spillover)

Reformulating the intervention as a continuous dose–response variable, based on the number of intervened physicians who billed for a given patient’s stay, yielded a notable shift in the estimated effect of the intervention. In contrast to the original cluster-level specification used in the stepped-wedge trial (OR 0.96, 95% CI 0.88–1.06), this more granular measure produced a positive and statistically significant association with ACP billing when examined independently (OR 1.06, 95% CI 1.01–1.11, \(p = 0.015\); Table 4). This result suggests that increasing the number of directly intervened physicians involved in a patient’s care was associated with higher odds of ACP documentation, supporting the interpretation that the intervention’s effects scale with the degree of exposure among treating physicians.

Table 4 Model estimates for the as-treated formulation (inclusive network spillover, controlling for the number of physicians seen)

In this formulation, the spillover variable captures diffusion of intervention practices both within and across hospitals. The additive spillover effect (\(\hat_2\)) was statistically significant (OR 1.01, 95% CI 1.00–1.02, \(p = 0.015\)), indicating that each 10 percentage-point increase in network exposure to intervened physicians was associated with approximately a 1% increase in the odds of ACP billing. Given that many physicians were connected to multiple peers, this effect translates into a meaningful cumulative influence of indirect exposure on clinical practice patterns.

When both exposure and spillover were included together, the direct intervention effect attenuated and lost statistical significance (OR 1.04, 95% CI 0.99–1.09, \(p = 0.089\)), consistent with shared explanatory variance between direct and indirect pathways of influence. The interaction between the number of intervened physicians and the spillover measure was particularly informative (OR 1.013, 95% CI 1.005–1.021, \(p = 0.001\)). This positive and statistically significant interaction suggests a synergistic effect: patients exposed to more intervened physicians benefited even more when those physicians were embedded in highly connected networks of other intervened peers. In other words, the intervention’s impact was amplified when local intensity of exposure coincided with broader network diffusion, reflecting a reinforcing mechanism between direct implementation and peer-driven propagation of intervention behaviors.

5.3.4 Extended as-treated model with temporal effects

Extending the as-treated model to include step-by-intervention interactions revealed a strengthened and statistically significant direct intervention effect in Step 1 (OR 1.11, 95% CI 1.03–1.20, \(p = 0.006\); Table 5). This finding reinforces the positive association between the number of intervened physicians involved in a patient’s care and the likelihood of ACP billing, consistent with a dose–response interpretation of intervention exposure. The interaction between intervention and spillover remained positive and statistically significant (OR 1.009, 95% CI 1.001–1.017, \(p = 0.029\)), suggesting that the beneficial effects of direct exposure were amplified in settings where physicians were also connected to a larger network of other intervened peers. Together, these results indicate that both direct and indirect diffusion mechanisms contribute meaningfully to the overall impact of the intervention.

Table 5 Extended as-treated model including step-by-intervention interactions and COVID-19 adjustment

Similar to the intent-to-treat formulation, we observe a temporary attenuation in intervention impact during Step 4 (OR 0.744, 95% CI 0.677–0.816, \(p < 0.001\)). However, this reduction appears transient, with the effect largely sustained through Step 5 (OR 0.847, 95% CI 0.763–0.941, \(p = 0.002\)). Alternatively, this pattern may partially reflect network saturation: as shown in Fig. 1, by Step 3 nearly all physicians were connected to an intervened peer when considering both first- and second-degree ties, suggesting that opportunities for further diffusion diminished in later rollout periods.