Token–Layer Activation Event Cascades in LLMs: Rate-Matched Connectivity under Gain Scaling

Uyar, Ali

doi:10.5281/zenodo.18435924

Research / Inside the model

Token–Layer Activation Event Cascades in LLMs: Rate-Matched Connectivity under Gain Scaling

Models gated-MLP activations as a token-by-layer event lattice and shows, under rate-matched and marginals-preserving shuffle controls, that gain scaling changes local event connectivity in two Qwen2.5-7B checkpoints.

Published: 30 January 2026
Author: Ali Uyar
DOI: 10.5281/zenodo.18435924
Code: github.com/aliuyar1234/AvalancheLLM

Read the paper (PDF)

Abstract

Activation patterns in large language models are often studied via static sparsity or outlier magnitudes, but less is known about how activation events form connected cascades across token positions and depth. We model gated-MLP activations as a token-by-layer event lattice by standardizing a fixed internal tensor and thresholding it into sparse binary events. This enables connected-component analysis that yields avalanche-like cascades on a two-dimensional lattice. We introduce directional branching metrics that decompose local propagation into token-direction and depth-direction components, and we evaluate a low-compute gain intervention that scales each layer’s MLP residual contribution at inference time.

To isolate connectivity effects from trivial rate effects, we design two strong controls: (i) per-layer rate-matched thresholds that equalize marginal event rates across gains, and (ii) a marginals-preserving raster shuffle that permutes token order within each layer while preserving per-layer event counts exactly. We avoid equating heavy-tailed component statistics with criticality by treating tail fits as descriptive diagnostics and using multiple falsifiable signatures. Finally, we calibrate a gain g⋆ on Dataset A using a mechanistic criterion based on branching, then evaluate the same g⋆ unchanged on Dataset B and ARC multiple-choice, comparing against the baseline g=1.

In the provided 7B runs, total branching b_tot varies with gain under rate matching (Table 8 and Figure 3) and the null-controlled residual Δb_tot remains non-zero (Figure 4). This framework supports both positive and negative transfer outcomes under strong controls by reporting mechanistic signatures alongside task metrics.

Why it matters

Activation patterns are often studied through static sparsity or outlier magnitudes. Holding event rates fixed across gains isolates how events propagate across tokens and depth, and the framework reports mechanistic signatures such as directional branching alongside task metrics.

Cite

@misc{uyar2026avalanchellm,
  title     = {Token–Layer Activation Event Cascades in LLMs: Rate-Matched Connectivity under Gain Scaling},
  author    = {Uyar, Ali},
  year      = {2026},
  doi       = {10.5281/zenodo.18435924},
  publisher = {Zenodo},
  url       = {https://doi.org/10.5281/zenodo.18435924}
}

Lit lamp: an active site (layer ℓ, token t), where at least one unit of the standardized gated-MLP tensor crosses τ_ℓ; the full raster is 28 layers × 256 tokens, and aval. counts avalanches, 4-connected groups of active sites, over 64 sequencesGain knob: every block's MLP residual scaled by g, 11 detents from 0.70 to 1.30; the marker sits at g*, the gain whose b_tot is nearest 1Trimmers: per-layer thresholds τ_ℓ(g) re-set at every gain so each layer's event rate stays on its target for r*Coincidence meter: the needle reads b_tot from 0.25 to 1.75 with 1.00 straight up; the faint pointer reads b_perm, the shuffled baseline, so the gap between them is ΔbShuffle: token order permuted within each layer, per-layer event counts kept exactly; with it on, the needle drops onto b_permAvalancheLLM thresholds the gated-MLP activations of Qwen2.5-7B-Instruct into a token-by-layer event lattice and measures how events propagate: b_time is the fraction of active sites whose next token is also active, b_depth the fraction whose next layer is, and b_tot their sum. A gain g scales every block's MLP residual, per-layer thresholds keep each layer's event rate on its target for r* at every gain, and a within-layer token shuffle that keeps event counts exactly gives the baseline for Δb. One-sided at r* = 4×10⁻⁵, b_tot goes from 1.109 at g = 0.70 to 1.089 at g = 1.30 and Δb from 0.233 to 0.210, with every layer's rate within 5.23×10⁻⁷ of target. The lamps show 8 of the 28 layers and 16 of the 256 tokens, each window drawn to the measured b_time and b_depth of its setting.