Our visual environment is structured hierarchically, akin to the classic “forest before trees” analogy (Navon, 1977). Processing this hierarchy requires flexible attentional control to shift the focus between global configurations and local details. Research on hierarchical letter stimuli has consistently demonstrated a “global precedence” effect—faster and more accurate responses to global features than to local ones (Navon, 1977, Navon, 1981). Content-to-level binding (CLB) theory (Hübner, 2014, Hübner and Volberg, 2005) explains global precedence via dual-stage processing. During the perceptual encoding stage, greater global saliency yields a global advantage effect (Baumann and Kuhl, 2005, Gerlach and Poirel, 2020, Kimchi, 1992), which may reflect an inherent attentional bias toward global forms. During the response decision stage, inconsistent global information disrupts feature-level binding more strongly, causing a global interference effect (Lamb and Robertson, 1988, Wu et al., 2020, Zmigrod et al., 2015; Grice et al., 1983); this interference underscores the challenge of suppressing irrelevant information when attention is focused on the local level.

Although the visual system is often studied in isolation, perception in natural environments is inherently multisensory. Auditory input influences the perceptual encoding and decision-making stages of visual processing. At the perceptual encoding level, sound can enhance visual contrast detection (Lippert et al., 2007), facilitate letter recognition (Hirst et al., 2019, Xu et al., 2024), and aid in object disambiguation (Mercier and Cappe, 2020, Williams et al., 2022). During the decision-making stage, behaviorally relevant auditory stimuli systematically modulate visual task performance. Converging evidence indicates that meaningful sounds can accelerate evidence accumulation during visual decision-making (Noppeney et al., 2010), sharpen categorical judgments (Adam & Noppeney, 2010; Kayser et al., 2017), and alter response selection thresholds (Bolam et al., 2024, Van Der Burg et al., 2008). Conversely, task-irrelevant auditory signals may compete for cognitive resources and attentional capacity, impairing visual decision efficiency (Donohue et al., 2013, Talsma et al., 2010). Cross-modal interaction is thought to arise from the dynamic modulation of response-level processes, including evidence integration and conflict resolution (Regenbogen et al., 2018, Stein and Stanford, 2008; Rohe and Noppeney, 2015, Rohe and Noppeney, 2016). However, much less is known about how auditory information affects hierarchical forms of visual processing.

Few studies have examined whether auditory input influences global–local visual processing. Colzato et al. (2017) reported that pre-task exposure to high-frequency binaural beats reduced the global advantage effect. In their study, the sound was not presented synchronously with the visual stimuli but preceded the visual task without any semantic information. Real-world sensory input is not only presented sequentially but also simultaneously and semantically congruent (Adam and Noppeney, 2010, Bolam et al., 2024, Noppeney et al., 2010, Regenbogen et al., 2018). Studies using the cross-modal Stroop paradigm have shown that semantically congruent auditory information improves visual task performance (Chan & Zhou, 2016). Crucially, even in the presence of visual dominance, semantically incongruent auditory stimuli impair visual task performance (Donohue, Appelbaum, et al., 2013). Thus, it is unclear whether synchronously presented auditory stimuli affect global–local visual processing and, if so, in what way.

Moreover, the nature of auditory influences on global–local processing warrants further exploration. Research in visual attentional filtering, such as the Flanker task, demonstrates that distractor interference can stem from distinct sources: generalized attentional capture or specific response competition (Bräutigam et al., 2024, Donohue et al., 2013, Fong et al., 2018, Fu et al., 2021, Kang et al., 2018). For example, the impact of a spoken letter (“H” or “S”) on visual letter recognition may depend not only on its auditory nature but crucially on whether it matches or mismatches the target response (Donohue et al., 2013, Hirst et al., 2019, Chan and Zhou, 2016). This parallel suggests that global–local processing, like Flanker tasks, functions as an attentional filter potentially susceptible to both modality-general resource depletion and semantic-specific conflict. Understanding how concurrent auditory stimuli interact with this hierarchical attentional process, especially when task-relevant and task-irrelevant information coexist, is therefore a critical question.

To dissociate perceptual effects from decision-level effects of auditory input during global–local visual processing, we employed hierarchical drift diffusion modeling (HDDM; Ratcliff and McKoon, 2008, Wiecki et al., 2013). Traditional behavioral metrics, such as reaction times and accuracy, do not distinguish between different cognitive stages and may conflate perceptual encoding with response selection. As shown in Fig. 1a, the HDDM addresses this limitation by decomposing response behavior into interpretable latent parameters: non-decision time (t), which indexes the duration of perceptual and motor components; drift rate (v), which reflects the quality and speed of evidence accumulation during decision-making; boundary separation (a), which indicates response caution; and the starting point (z), which reflects response bias. Prior work has shown that semantically congruent auditory stimuli can increase the drift rate in visual categorization tasks, suggesting enhanced decision efficiency (Bolam et al., 2024). By applying HDDM within the framework of CLB theory (Hübner & Volberg, 2005), we aimed to test whether auditory effects on global–local processing arise at the level of perceptual encoding, decision formation, or both.

Drawing on established findings in visual global–local processing research (Navon, 1977), we hypothesize that under visual-only conditions, classic within-domain effects will emerge: a global advantage effect and a global interference effect. Building on CLB theory and multisensory integration frameworks, we further posit that cross-domain auditory congruency dynamically modulates these hierarchical processes through distinct mechanisms: First, for the global advantage effect, response-congruent sounds should enhance perceptual encoding efficiency, whereas response-incongruent sounds will attenuate it by disrupting early feature integration. Second, for the global interference effect, response-congruent sounds may facilitate feature-hierarchy binding during decision-making, reducing conflict costs; conversely, response-incongruent sounds should exacerbate interference by impairing evidence accumulation for conflict resolution. Finally, pure tones—devoid of semantic content—are predicted to exert minimal influence on either phenomenon, providing an essential baseline to isolate semantic-specific effects.

To test these hypotheses, we conducted two experiments using a hierarchical letter identification task. In Experiment 1, participants completed trials under four auditory conditions: no sound, pure tone, response-congruent speech, and response-incongruent speech. This design systematically examined how different types of auditory input—ranging from non-semantic tones to semantically meaningful speech—influence global–local processing. Furthermore, the global precedence effect is based on confounding the advantage effect with the interference effect (Gerlach & Poirel, 2020). Therefore, in Experiment 2, we increased the visual angle of the hierarchical stimuli to reduce the saliency of global forms in peripheral vision, a manipulation that eliminates the global precedence effect (Gerlach and Poirel, 2020, Kinchla and Wolfe, 1979). This approach allowed for a more precise examination of global interference in local target processing. By combining behavioral and computational approaches, our study aims to determine not only whether auditory semantics influence global–local visual processing but also how and at what processing stage such effects emerge—advancing theoretical models of multisensory integration, attentional control, and hierarchical perception.