New Machine Learning (Statistics) (stat.ML) papers, Monday 19 October 2026
31 papers announced on arXiv. Get them ranked for you, free →
Cointegrated Regressors: Where Folded Concave Penalties Go Beyond LASSO
LASSO is one of the most popular regularization methods for high-dimensional regression models. While cointegration is common in practical applications of predictive regressions with many regressors, the convex penalty of LASSO overly suppresses the coefficients of cointegrated…
Causal Integrated Gradients: Scalable Feature Attribution Along Causal Paths
Existing feature attribution methods that account for feature dependencies do not scale to high dimensions. We propose causal integrated gradients (CIG) and causal expected gradients (CEG), which extend integrated gradients to incorporate prior causal structure among input…
Upper bounds on the number of causal pieces in spiking neural networks
AI summaryThe paper upper‑bounds the number of causal pieces a feedforward single‑spike neural network can generate. It shows a general poly‑Bernoulli combinatorial bound and tighter hyperplane‑based bounds for various integrate‑and‑fire and ReLU‑type neuron models.
Source-Aware Dynamic Pricing with Strategic Buyers
AI summaryThis paper introduces a source-aware dynamic pricing framework that handles both verified and strategically manipulated self-reported buyer features. The proposed SALE policy achieves a sublinear expected regret bound by combining localized exploration with strategic correction.
Learning smooth dynamics via Kinematic Interpolants
AI summaryThe paper introduces Kinematic Interpolants, a framework leveraging position and velocity measurements to infer nonlinear dynamics. It achieves superior reconstruction of dynamical systems while requiring a fraction of the training time compared to existing baselines.
Computing Fréchet on Unknown Submanifolds of Riemannian Manifolds
AI summaryThis paper introduces a graph-based algorithm to compute intrinsic Frechet means on unknown Riemannian submanifolds using only local samples and an ambient metric. The method achieves an O(1/T) convergence rate and extends ISOMAP guarantees to Riemannian ambient spaces.
Flat Loss, Evolving Features: Grokking in Modular Addition
AI summaryReLU networks on modular addition can maintain 100 % training accuracy while training cross‑entropy stays flat above zero, and test accuracy rises above 90 %; this is provable for one‑hidden‑layer networks and observed empirically in deeper nets and transformers.
Is a Linear Probe Evidence of a Linear Representation?
AI summaryLinear probes can be misleading: they succeed on any rich feature map, even random kernels, because they only capture a linear subspace. The paper introduces reverse predictivity, spectral concentration, and OOD tests to show pretrained transformers encode more complex, curved structures, not a purely linear representation.
Optimal approximation of smooth functions by ReLU neural networks
AI summaryThis paper establishes optimal ReLU network constructions for uniformly approximating Hölder functions across all dimensions and smoothness levels. The resulting architectures achieve the information-theoretic optimal approximation error and minimax regression rates without extraneous logarithmic factors.
Unit-level aggregation breaks coverage in causal foundation models
AI summaryCausal foundation models struggle with group-level uncertainty quantification because they assume unit-level errors are independent, ignoring correlations from posterior uncertainty over the data-generating process. Jointly predictive neural process models fix this failure and restore correct coverage.
Conformal Policy Selection: When to Deploy AI Decisions for Targeted Intervention
AI summaryConformal Policy Selection tests each individual’s predicted outcome against a threshold and selects those exceeding it while controlling false discovery rate. It guarantees finite‑sample FDR control and is doubly robust, offering a safety gate for deploying AI policies in high‑stakes settings.
Towards Hyperparameter Transfer for Differentially Private Optimization
AI summaryDP‑µP extends maximal update parametrization to differentially private training, deriving a signal‑to‑noise ratio condition for layer‑wise learning‑rate scaling. It yields practical scaling rules for DP‑SGD, DP‑Adam, and DP‑Muon that transfer learning rates across widths, matching or surpassing tuned baselines.
Distinguishability and Recovery for a Doubly Sparse Spiked Wigner Model
AI summaryThe paper shows that a leave‑one‑out quadratic thresholding method recovers the support of a sparse rank‑one signal in a spiked Wigner model with independent sparse noise, and sign synchronization or spectral refinement yields exact or strong recovery, giving sparsity‑SNR tradeoffs.
Information-Geometric Blind Spots of Conformal Test Martingales
Conformal test martingales (CTMs) are e-processes for sequentially testing exchangeability via p-values produced by a conformity function $A$. However, any distribution shift that preserves the push-forward $P\circ A^{-1}$ leaves the p-value distribution invariant, rendering it…
A Class of Rényi-based Concentration Metrics for Kolmogorov-Arnold Geometry in Multilayer Perceptrons
AI summaryThe paper introduces α‑RC, a family of Rényi‑based concentration metrics for measuring how much a few Jacobian minors dominate in multilayer perceptrons, bounded 0–1 regardless of network size. For α = 2 the metric equals a transformed Participation Ratio, improving interpretability.
The Loss or the Architecture? What Matters in Survival Analysis with Competing Risks
AI summaryThe study independently tests loss functions and backbones for deep competing‑risk survival models across tabular and image data, finding that a cumulative‑incidence loss dominates over architecture choice; the best performance comes from combining a proper loss with a modality‑matched backbone, yielding state‑of‑the‑art Brier scores.
Minimax-optimal Scaling Laws in Ridge Regression via Transfer Learning
AI summaryThe paper analyzes ridge regression transfer learning: a pretrained source model regularized by ridge and a target model regularized by its distance to the source. It derives deterministic risk equivalents, shows minimax‑optimal scaling under power‑law covariance spectra, and confirms the result with experiments.
Towards optimal selection schedules for online active learning
We consider the problem of online linear classification, where a stream of unlabeled samples is available, and annotation is costly. We investigate the optimal query policy, that prescribes whether and when a sample should be labeled and used for a training step, or instead…
Adaptive conformal prediction on the Grassmannian
We introduce a conformal prediction framework for regression with Grassmann-valued responses. The proposed method constructs prediction regions in the form of metric balls directly on the Grassmann manifold, using basis-invariant geometric distances as non-conformity measures…
Causal Discovery beyond Conditional Independence: Identifiability, Algorithms, and Statistical Inference for Directed Acyclic and Cyclic Graphs
AI summaryThe paper reviews causal discovery methods that use distributional features beyond conditional independence, separating identification, estimation, and computation, and discusses latent confounding, feedback, and cyclic graphs, offering consistency, uncertainty, and algorithmic insights for diverse data settings.
Beyond Expected Values: Risk-Sensitive Planning with Distributional Monte-Carlo Tree Search
Monte-Carlo Tree Search (MCTS) typically stores only scalar mean estimates, which makes it poorly suited for tail-risk objectives such as conditional value-at-risk (CVaR). We introduce Categorical Thompson Sampling with Optimism (CATSO) and Particle Thompson Sampling with…
A general recipe for data-adaptive anytime-valid envelopes for iterative algorithms
Most convergence guarantees for iterative stochastic algorithms are stated at a fixed iteration count and do not justify stopping at a data-dependent time. Recent work has constructed anytime-valid envelopes that do, but deterministic envelopes are typically too conservative for…
Boundary Detection for High-Dimensional Point Clouds
We introduce two new methods for boundary detection in high-dimensional point cloud data, intended to work in situations with non-uniform density, high ambient dimension, and substantial noise. Our first method fits a separating hyperplane on local neighborhoods after doing…
No-Regret Language Generation
We introduce no-regret generation, which requires a generator to make a sublinear number of mistakes uniformly over all demonstration sequences and compatible target languages. We also formulate an agnostic variant that allows arbitrary demonstrations that may not be compatible…
Equivalence testing and run-count design for stochastic machine-learning experiments
Claims that two training methods perform equally well often rest on a non-significant difference between two or three runs per method. We develop a protocol, with its power theory, for certifying practical equivalence in per-run metrics such as the AUROC (area under the receiver…
Exact Tail Laws for Nonlinear SGD with Infinite-Variance Gradients
AI summaryThis paper characterizes the exact tail probabilities and accuracy of nonlinear SGD under infinite-variance gradient noise. The main result proves that the limiting tail behavior is governed by a single large gradient shock transported through the local Hessian flow.
Mesoscopic learning dynamics of extensive-width neural networks
Understanding how stochastic gradient descent learns internal representations in wide neural networks trained on rich target functions is a central problem in machine learning. We study this question in two-layer neural networks learning target functions represented by networks…
Parametric Resonance in Primal-Dual Learning for Slowly Non-Stationary Constrained MDPs
Primal-dual methods for constrained Markov decision processes are analyzed under a two-timescale assumption, with non-stationarity entering through two channels. In the additive channel, where the constraint level oscillates, the dual oscillation saturates at a floor set by the…
Task-aware Kernel Stein certificates for Rényi differential privacy
Rènyi differential privacy, based on Rènyi divergence, has become a notion of choice for privacy guarantees. However, outside the Gaussian setting, Rènyi divergence can be difficult to compute. In particular, Rènyi differential privacy guarantees that exploit task-specific…
Learning without relearning physical symmetry: an SO(3)-equivariant convolutional network (CNN) on the sphere
Many of the most important datasets in the physical sciences live natively on the sphere, and these data often carry no preferred position or orientation. Using methods that do not reflect this symmetry leads to far higher simulation requirements, which can be catastrophic for…
ENFR: Evidence Networks with Flow-Matched References for Amortized Simulation-Based Model Inference
The evidence, $\mathcal{Z} = p(x | \mathcal{M})$, of a data-generating model $\mathcal{M}$ is a powerful tool in Bayesian frameworks: it enables model inference, model misspecification tests, and goodness-of-fit assessment as an alternative to standard discrepancy statistics,…