Research / Inside the model
Attention-Side Transformer Computations Organize Shared Multilingual Brain Alignment during Naturalistic Story Listening
Maps where shared multilingual brain alignment sits inside XLM-R-base and NLLB-200 encoders, showing it is organized primarily around attention-side and post-attention computations rather than late FFN states.
- Published
- 23 March 2026
- Author
- Ali Uyar
Abstract
Most brain-language-model alignment studies rely on returned hidden states, leaving unclear which internal transformer computations carry cross-lingually shared signal during naturalistic comprehension. We analyzed English, French, and Chinese listeners from the Le Petit Prince multilingual naturalistic fMRI corpus (94 participants; 9 runs per language cohort) using ROI-average encoding models built from intermediate states of XLM-R-base and the encoder of NLLB-200-distilled-600M. For each sentence span and target language, we decomposed state representations into a leave-target-out shared component and an orthogonalized specific residual, then contrasted attention-side and FFN-side contributions within semantic and auditory ROI families. The planned semantic FFN-preference hypothesis failed uniformly: all six semantic model-language contrasts were negative (mean effect −0.0083; 0/6 Holm-significant in the planned direction), and supplementary reverse-direction tests favored attention-side states in all six cells. By contrast, the auditory attention-preference hypothesis was positive in 5/6 cells and Holm-significant in 4/6 (mean effect 0.0033). Representative winners concentrated in attn and post-attn rather than ffn states, semantic representative settings showed a larger mean content bias than auditory settings (0.663 vs. 0.610), and deleting top-attributed words hurt held-out prediction more than matched-random deletion in all 20 summary rows (mean top-k minus random difference 0.0381). These results do not support a late FFN-centered account of shared multilingual brain alignment. Instead, they indicate that the shared multilingual advantage is organized primarily around attention-side and post-attention computations, with the clearest confirmatory support in auditory cortex and a consistent semantic shift away from FFN-side states.
Why it matters
Brain–model alignment studies usually read returned hidden states. Reading intermediate attention-side, post-attention and FFN states instead shows which internal computations carry the signal shared across languages.
Cite
@misc{uyar2026inside,
title = {Attention-Side Transformer Computations Organize Shared Multilingual Brain Alignment during Naturalistic Story Listening},
author = {Uyar, Ali},
year = {2026},
doi = {10.5281/zenodo.19185601},
publisher = {Zenodo},
url = {https://doi.org/10.5281/zenodo.19185601}
}