New Machine Learning (Statistics) (stat.ML) papers, Friday 23 October 2026
26 papers announced on arXiv. Get them ranked for you, free →
Why Constrained Updates Still Forget: A Functional-Path Analysis of Continual SFT
AI summaryThe paper decomposes how constrained updates in supervised fine‑tuning affect old‑task performance, showing that update placement and granularity drive forgetting. It demonstrates that matched replay schedules and subspace regularization can better preserve old behavior while improving new‑task gains.
Differentially Private Synthetic Data via Generalized Moment Matching
AI summaryPrivate synthetic data are generated by estimating moments with differential privacy, fitting a distribution, and sampling. The method achieves minimax‑optimal error (up to logs) for IPMs such as MMD with Matern/Laplace kernels, sliced Wasserstein‑1, and ReLU‑network metrics.
AI Mindmeld: A Latent Interconnect Protocol
AI summaryThe paper introduces the Latent Interconnect Protocol, a framework that learns lightweight connectors to map specialist models’ high‑dimensional representations directly into an LLM’s internal state. Experiments show that frozen LLMs can use these connectors for zero‑shot transfer and improved compositional reasoning across domains.
When Reconstruction Is Not Enough: Doubly Robust Counterfactual and Interventional Mediation Analysis with Learned Multimodal Representations
AI summaryThis paper shows that standard representation learning objectives like reconstruction can fail to preserve causal identification in multimodal health data, leading to severe bias in doubly robust mediation estimators. The authors propose a validated workflow including diagnostics and cross-fitted estimators to ensure reliable counterfactual and interventional analyses.
High-Value Training Data from Noisy, Diverse Sources
AI summaryThe paper introduces scalable data‑weighting via metagradients, learning parametric rules from a small high‑quality target set and transferring them to larger models and streaming data. It shows that learned weights improve reward‑model performance and combine human and synthetic signals better than either alone.
Flow-based Spectral Kernel Learning for Nonstationary Attention
AI summaryFlow-based Spectral Kernel Attention learns nonstationary attention kernels via normalizing flows over spectral densities, enabling flexible, regularized kernel learning while retaining linear‑attention efficiency. Experiments show competitive performance and 40‑47 % fewer projection parameters than baselines.
Lifetime-valid decisions by continually evolving AI analysts
AI agents increasingly act as analysts whose decision can often be cast as hypothesis tests. Classical tests control the Type I error rate, the probability of rejecting a true null hypothesis, for fixed procedures. However, AI analysts introduce internal randomness and can…
Residual Calibration via Local Feature-Space Refinement
AI summaryThe paper introduces LAPE, a post‑hoc residual calibration technique that refines global calibrator outputs by non‑parametric local evidence in feature space, and extends it to multi‑view inputs via CFAF; experiments show improved calibration for several baselines with low overhead.
Width and Depth Limits Commute During Training
AI summaryShows that for linear residual nets scaled by 1/√depth, the infinite-width and infinite-depth limits commute during training with gradient clipping, and still hold in probability without clipping. The network outputs converge in L² for any fixed number of steps.
xVGAE: A Hierarchical Variational Graph Autoencoder for Exchangeable Graphs
AI summaryxVGAE learns a graphon from training graphs using a graph-level latent variable, node-level exchangeable latents, and an edge probability function, producing new graphs that match training statistics and generate up to two orders of magnitude faster than current models.
Continuous-depth Deep Gaussian Processes
AI summaryThis paper introduces continuous-depth deep Gaussian processes, replacing discrete layer stacks with stochastic differential equations governed by Gaussian process priors. The proposed models improve optimization stability and regression performance across various benchmarks compared to traditional discrete deep Gaussian processes.
Random Projections for Spectral Algorithms: Optimal Rates in Sobolev Norms
Random projections (RP) offer an effective method for reducing computational and storage overhead while preserving the geometric structure of data. However, existing studies have primarily focused on the optimal generalization performance of specific kernel-regularized…
Transformers with CoT Discover Heavy-Ball Acceleration Through Training
AI summaryTransformers trained with chain‑of‑thought supervision first learn to imitate gradient descent for linear regression weight prediction. Additional trajectory supervision using true weights causes the model to discover the Heavy‑Ball accelerated method, improving convergence without a Heavy‑Ball teacher.
What Comes Next and Why: Interpretable Next-Event Prediction with Neuro-Symbolic Rules
In real-world applications, being able to explain why a prediction is made is often as important - if not more so - than achieving high accuracy. However, existing methods for sequential event prediction typically prioritize accuracy at the expense of interpretability.…
Prediction Uncertainty Controlled Computer Model Calibration
Computer-model calibration is usually undertaken in service of prediction, yet neither Bayesian Kennedy-O'Hagan (KOH) calibration nor amortized simulation-based inference is built to control the predictive error of the calibrated simulator. When parameters are weakly identified…
Local Discrimination of Binary Classifiers via Localized AUC Functional
Risk prediction models are frequently evaluated with AUC, which means that during evaluation we are blind to performance heterogeneity, which could lead to bad decision-making in clinics. In this work, we aim to change that by identifying patient cohorts for whom the risk…
Deep Learning for Random Objects in Metric Spaces: Beyond Manifold Assumptions
Deep learning (DL) methods have substantially improved predictive performance across a wide range of downstream prediction tasks, enabling new discoveries in multiple domains. However, principled DL methods for non-Euclidean random objects remain relatively scarce, despite the…
Metric Conformal Prediction via the Conditional Expected Local Radius
AI summaryThis paper defines the conditional expected local radius (CELR) and uses it to form a conformal prediction set for non‑Euclidean responses. It proves uniform consistency, asymptotic conditional validity, and shows improved computation and coverage over existing methods.
Dimension-free regimes for unadjusted Langevin in ridge-separable models: A subspace robust Wasserstein analysis
AI summaryULA’s bias is analyzed in subspace robust Wasserstein distance, revealing a bell-shaped mixing time governed by data rank N instead of dimension d. In overparameterized and large‑sample regimes, convergence is dimension‑free; near the proportional regime dimension‑free guarantees fail.
SABLE: Scalable Affine Bayesian Latent-space Exploration
Complex objects such as engineering designs and proteins are often difficult to optimise because their design spaces exhibit high-dimensional structure and objective evaluations are expensive. Generative models provide continuous latent representations, but their latent spaces…
Nonparametric autoregressive estimation of probability distributions with parallel gradient boosting
Nonparametric estimation of joint probability distributions is a core problem in statistics, which is notoriously challenging with high-dimensional data. Such challenges also arise with conditional distribution estimation and have led to flexible algorithms that aim to tackle…
Diffusion Language Models for Perplexity-Aware Black-Box Attacks
\emph{Continuous-space} attacks find prompts that trigger malicious behaviors, such as overthinking, in black-box LLMs by sampling or optimizing over a low-dimensional Gaussian latent space and decoding it to text through a \emph{random mapping} and nearest-token lookup. Because…
When to Follow the Teacher: Advantage-Based On-Policy Distillation
Online Policy Distillation (OPD) often outperforms offline methods such as Supervised Fine-Tuning (SFT) by training on trajectories sampled directly from the student. However, standard OPD trains the student to follow the teacher at every state of the trajectory, assuming that…
SympFormer: Inertial Attention Blocks from Damped Hamiltonian Dynamics on Spaces of Probability Measures
The attention mechanism underpins modern large language models, yet their dynamics remain only partially understood. Recent perspectives interpret attention blocks as interacting particle systems, whose mean-field limits correspond to gradient flows of energy functionals on…
Reliable AI-Assisted Decisions via Selective Auditing
AI summaryThe paper introduces an online optimization framework for selectively auditing AI decisions to minimize human review while guaranteeing that unaudited error rates remain below a specified threshold. Experiments across LLM safety screening and data labeling demonstrate that the proposed algorithms maintain strict error control while significantly reducing auditing frequency compared to baselines.
Phase Transition from Random Features to Fully Trainable Networks
Constructing accurate and computationally tractable approximation spaces is a fundamental task in scientific computing. Partially trainable neural networks bridge fixed random features and fully adaptive representations. This paper quantifies a phase transition pattern in…