New Machine Learning (Statistics) (stat.ML) papers, Thursday 22 October 2026
39 papers announced on arXiv. Get them ranked for you, free →
Interdomain Attention: Beyond Token-Level Key-Value Memory
AI summaryInterdomain Attention mixes softmax attention and state‑space recurrent memory in a single layer, compressing keys/values into an SSM. Across 125M–1.3B models, IA beats S4D baselines and softmax Transformers on language, reasoning, and genomic tasks.
Borrowing Strength Across Stores: Transfer Learning for Choice-Based Revenue Management
AI summaryThe paper introduces a shrinkage estimator for the multinomial logit demand model that balances store‑specific estimation with cross‑store pooling. It proves finite‑sample bounds on estimation error and downstream assortment/regret, showing empirically that moderate shrinkage boosts revenue for hundreds of apparel stores.
A Reverse-Gold Theorem: Positive-Data Identification by Substrate-Bounded Contrastive Learners
Gold's theorem shows that no learner can identify an arbitrary language in a superfinite class from positive presentations alone. We prove a positive result of a different shape. Fix an innate \emph{recursive contrast operator} $R$ mapping any string to a finite set of…
ReAPO: Retry-Aware Policy Optimization for Efficient Exploration in LLM Reasoning
AI summaryReAPO improves LLM reasoning by retry‑aware policy optimization that assigns credit to each attempt via a stochastic shortest‑path view, allocating more generations to failed trajectories. It outperforms GRPO and baselines on Qwen3/OLMo benchmarks, achieving higher first‑attempt accuracy with fewer tokens.
Depth Separation for Neural Networks under Gaussian Measure
AI summaryDepth‑3 ReLU networks of polynomial size can approximate a particular radial function under Gaussian measure to fixed small L² error, while any depth‑2 network achieving the same error must have exponential size or subexponential weights; the result follows from a Hermite‑frequency lower bound verified in Lean 4.
Actually, All Learnable Distribution Classes Are Privately Learnable
AI summaryAny class of distributions learnable with finitely many samples up to arbitrary total‑variation error is also privately learnable with finitely many samples, proving Ashtiani’s conjecture under the standard learnability definition.
Backtracking Direct, Indirect and Total Effects
Natural direct and indirect effects are defined through cross-world interventional counterfactuals. We propose an alternative construction based on backtracking counterfactuals. Rather than intervening on the treatment or mediator, we use a backtracking conditional distribution…
Auditing Conformal Prediction for Language Models Under Inference-Time Model Shift
AI summaryThis paper formalizes inference-time model shift in language model conformal prediction, demonstrating that changing generation settings between calibration and deployment breaks theoretical guarantees. Consequently, standard evaluations substantially overstate true deployment coverage.
Is In-Context Reinforcement Learning A Posterior Sampler? An Empirical Study
AI summaryICRL, specifically DPT and variants, is tested against two Bayesian posterior sampler conditions: sufficient‑statistic invariance and martingale property. In Gaussian, Bernoulli, and linear bandit experiments, both conditions fail, indicating ICRL does not act as a posterior sampler in practice.
From Common Information to Canonical Correlation Analysis and Beyond: Bridging Information Theory, Signal Processing, and Machine Learning
Many problems in signal processing, machine learning, and neuroimaging provide several jointly observed measurements of one process, each mixing a component common to all of them with a component private to itself, and require the common component to be recovered without…
A Jump-Modified Wasserstein Gradient Flow for Maximum Mean Discrepancy with Energy Kernels
We study a jump-modified Wasserstein gradient flow of the squared maximum mean discrepancy generated by the energy kernels $K(z)=-|z|^q$, $0<q<2$, in every dimension $d\ge2$ as well as the one-dimensional range $1\le q<2$. Under subcritical density and finite-moment assumptions,…
Internal Guidance as a Sampling Method for Tilted Diffusion Models
AI summaryThe paper introduces internal guidance for diffusion models, shifting the denoiser’s input instead of adding guidance terms. By training with random diagonal rescalings, a single denoiser samples binary reward tilts, class labels, and infilling without fine‑tuning or external guidance.
Curvature-Scaled Sharpness-Aware Minimization: Convergence Beyond Smoothness
Sharpness-Aware Minimization (SAM) has established itself as a widely used method for training deep neural networks. Despite its popularity, existing convergence guarantees for SAM-type methods require smoothness, a restrictive assumption. Generalized smoothness conditions have…
Schrödinger-Bass Bridge Barycenter Models for Measure-Valued Data
AI summarySchrödinger‑Bass Bridge barycenter models use directional asymmetry and dual drift‑diffusion control to encode, decode, symmetrize, and Markovianize measure‑valued data. They guarantee barycenter existence, compute Gaussian barycenters via finite‑dimensional optimization, and outperform existing methods on image and scRNA interpolation tasks.
Approximate Solomonoff Induction: On the Robustness of Learned Universal Predictors
AI summaryThe paper bounds how close an approximate universal predictor must be to the Solomonoff mixture to keep prediction error low. It shows worst‑case excess error is Θ(η/w_μ), but averaged over the prior it is only O(η), and extends to distribution shift and finite‑sample settings.
ExpEvolve: Structuring and Reusing Experience for Open-Ended Discovery
AI summaryExpEvolve injects idea-level extraction into evolutionary coding agents, identifying high‑level factors that explain program performance and using them for guided exploitation and exploration. Across 11 math tasks it boosts final performance and sample efficiency by up to 5×, with transfer to other LLMs and MLS‑Bench.
GaP-MoE : Explicit Expert Specialization in Mixture-of-Experts via Orthogonal Gradient Accumulation and Projection
Sparse Mixture-of-Experts (SMoE) models improve scalability by routing input tokens to only a subset of expert networks, but they often suffer from representation collapse, in which multiple experts converge toward similar functions and lose their intended diversity. Prior work…
Cascading Bandits with Noisy Observations: Algorithms and Regret Bounds
We propose a novel stochastic cascading bandit setup where probing an ordered shortlist of $K$ out of $L$ arms incurs position-dependent costs, and the learner observes only noisy feedback regarding each arm's true output. We identify a sub-class of noise models for which the…
The Predictive--Optimization Gap in GuanDan Self-Play
AI summaryThe paper shows that in GuanDan, a predictive signal (state–gain interactions) improves outcome prediction but does not consistently improve policy optimization: a single checkpoint outperforms a baseline, yet across multiple seeds and iterations the effect averages to zero.
When Thoughts Become Memory: Unifying In-Context Learning and Fine-Tuning
AI summaryProposes a certified rule that quantifies whether new data should be added to a model’s context or its weights by measuring the code length saved on held‑out queries. The rule correctly selects between in‑context learning and fine‑tuning across transformers, TabPFN, occupancy and stereo networks, achieving 61,948/64,000 correct decisions and ≤3.3% error on null streams.
One Act, Many Masks: Effect-Marginal Policy Optimization for Long-Horizon Agents
AI summaryEffect-Marginal Policy Optimization (EMPO) aligns RL updates to execution effects rather than specific text, correcting credit‑misalignment in long‑horizon LLM agents. EMPO boosts task success and learning speed on ALFWorld, WebShop, and Sokoban without extra critics or rollouts.
Density-Level Backward Error for Low-NFE Generative Flows
AI summaryDensity-level backward error identifies which part of a flow’s numerical defect alters the density, enabling a solver‑order‑aware compute‑allocation rule. The method allocates evaluations more efficiently than curvature‑based schemes and improves low‑NFE Euler sampling on pretrained ODE flows.
Generative Modeling via Nonlinear Schrödinger Bridges and Branching Diffusions
AI summaryGenerative modeling is built on forward‑backward PDEs from mean‑field games, yielding a nonlinear Schrödinger bridge system (SBNL). Using branching diffusions, the method solves this efficiently, scaling to high dimensions and recovering classical, nonlinear, and score‑based generative tasks.
A unified approach for estimation of unnormalized models via Stein operators
AI summaryThe paper unifies estimation of unnormalized models by linking proper scoring rules (e.g., score matching, pseudolikelihood, NCE) to Stein operators. It shows these rules induce Stein operators, providing a geometric framework that characterizes the asymptotic efficiency of the estimators.
BRIDGE: An Acceptance Gate to Block False Improvements in Model Updates, with a Retrospective Clinical Evaluation
Models are updated repeatedly, and an update that looks better on validation data but not on new data is a false improvement that can misguide clinical decisions. The naive rule accepts every positive validation gain and is optimistic when the same data choose and judge the…
Residual-Based Causal Discovery via Pretrained In-Context Quantile Regression
AI summaryLoscar extends RESIT to location‑scale noise models by estimating conditional mean and scale with pretrained in‑context quantile regression, avoiding parametric assumptions. It outperforms an FGLS variant on bivariate and multivariate causal‑discovery benchmarks.
Private Online Prediction from Experts with Small Losses
We consider online prediction from experts, a fundamental problem in machine learning, under differential privacy constraints. Existing private algorithms achieve near-optimal regret $\tilde{O}(\sqrt{T})$, where $T$ is the time horizon. However, this bound becomes suboptimal…
Fragment-Level Graph Encoding for Interpretable Molecular Property Prediction
Graph neural networks (GNNs) are among the standard tools for molecular property prediction, but the atom-level explanations of their predictions are hard for medicinal chemists to act on. We propose a change of input representation rather than a new model: molecules are encoded…
Near Minimax Rate for Quantile Distributional Reinforcement Learning
In this paper, we study quantile distributional policy evaluation, where the return distribution under a given policy is approximated by the quantile Bellman fixed point $\bm\eta_m$. We investigate the statistical accuracy of estimating $\bm\eta_m$ across quantile resolutions…
Lifting PCA in the Wasserstein Space
Principal Component Analysis (PCA) is a fundamental tool for statistical dimension reduction, yet no canonical extension to distribution-valued data has emerged within the framework of optimal transport. While Wasserstein barycenters and Wasserstein least-squares regression…
Post-hoc Epistemic Uncertainty for Vision-Language-Action Policies
Vision-language-action (VLA) policies need to recognize gaps in their training knowledge to request useful demonstrations or assistance in new situations, especially when failures are costly. Disagreement within an ensemble of independently trained policies can guide these…
Asymmetric Likelihood-Free Hypothesis Testing
AI summaryThis paper characterizes sample budgets in asymmetric likelihood‑free hypothesis testing: with different numbers of samples from reference distributions, one can test a third distribution using only $\tilde O(n^{1/4})$ samples instead of the $\Omega(\sqrt n)$ needed when budgets are equal. The results also give matching bounds for non‑adaptive 2‑clustering and show an adaptivity gap of $\tilde\Theta(k^{1/4})$.
How Do DiTs Denoise? An Aggregator-Predictor View of Attention and MLPs
How do diffusion transformers (DiTs) represent image content and learn rules for reusing it to generalize? We develop an aggregator--predictor view of DiTs as input-adaptive local denoisers, in which context aggregation and patch prediction provide functional roles for…
Silent Alignment in Two-Layer Linear Networks: A High-Dimensional Perspective
AI summaryThe paper analyzes gradient flow in high-dimensional two-layer linear networks, deriving exact asymptotic spectral dynamics. It shows a silent alignment phase where singular directions align quickly while values stay zero, followed by a slower spectral learning phase where singular values grow and loss decreases.
Mastering Model Fit: Applications of the InterModel Vigorish for Interpretable Machine Learning
Accuracy alone gives little information about which features or model components improve probabilistic fit, which outcomes benefit, or how fit changes during training. We introduce the InterModel Vigorish (IMV) into the Machine Intelligence space in order to express differences…
The Price of Current-Effect Inference in Nonstationary Multi-Armed Bandits
Adaptive experiments use bandit algorithms to favor interventions with higher expected rewards. However, decision-makers also require reliable inference on current treatment effects: differences between intervention and reference-arm mean rewards at the present time, rather than…
Mitigating Model Collapse through Dreaming Learning
Training generative models on synthetic data causes loss of diversity and a decrease in performance, a phenomenon known as model collapse. We study this degeneration spectrally in a controlled Markov setting. The eigenvalue spectrum of the learned transition operator offers a…
The role of locality and weight sharing in convolutional neural networks
AI summaryThe paper analyzes a teacher–student convolutional model to separate the effects of locality and weight sharing. It shows that for uncorrelated data convolution is P‑times faster; for correlated data weight sharing hurts and locally connected nets eventually outperform CNNs, with a predictable transition.
Geometric Algebra for Neural Networks: A Survey
Geometric algebra (GA) unifies geometric primitives and transformations in a single algebraic framework. Neural networks designed around this framework often represent features as multivectors and constrain layer operations to respect the algebraic structure, enabling principled…