Publications

2024

• Compression with Exact Error Distribution for Federated Learning
  
  • Hegazy Mahmoud
  • Leluc Rémi
  • Li Cheuk Ting
  • Dieuleveut Aymeric
  , 2023, 238, pp.613-621. Compression schemes have been extensively used in Federated Learning (FL) to reduce the communication cost of distributed learning. While most approaches rely on a bounded variance assumption of the noise produced by the compressor, this paper investigates the use of compression and aggregation schemes that produce a specific error distribution, e.g., Gaussian or Laplace, on the aggregated data. We present and analyze different aggregation schemes based on layered quantizers achieving exact error distribution. We provide different methods to leverage the proposed compression schemes to obtain compression-for-free in differential privacy applications. Our general compression methods can recover and improve standard FL schemes with Gaussian perturbations such as Langevin dynamics and randomized smoothing.
• Stackelberg Mean Field Games: convergence and existence results to the problem of Principal with multiple Agents in competition
  
  • Djete Mao Fabrice
  , 2023. In a situation of moral hazard, this paper investigates the problem of Principal with $n$ Agents when the number of Agents $n$ goes to infinity. There is competition between the Agents expressed by the fact that they optimize their utility functions through a Nash equilibrium criterion. Each Agent is offered by the Principal a contract which is divided into a Markovian part involving the state/production of the Agent and a non--Markovian part involving the states/productions of all the other Agents. The Agents are in interactions. These interactions are characterized by common noise, the empirical distribution of states/productions and controls, and the contract which is not assumed to be a map of the empirical distribution. By the help of the mean field games theory, we are able to formulate an appropriate $limit$ problem involving a Principal with a $representative$ Agent. We start by solving the problem of both the Principal and the $representative$ Agent in this $limit$ problem. Then, when $n$ goes to infinity, we show that the problem of Principal with $n$ Agents converges to the $limit$ problem of Principal with a $representative$ Agent. A notable result is that, despite allowing a general type of contracts, it is approximately optimal for the Principal to offer contracts to the $n$ Agents that are maps of the empirical distribution of states/productions and controls of the Agents.
• Generative Flow Networks as Entropy-Regularized RL
  
  • Tiapkin Daniil
  • Morozov Nikita
  • Naumov Alexey
  • Vetrov Dmitry
  , 2023, vol. 238 of PMLR, pp.4213-4221. The recently proposed generative flow networks (GFlowNets) are a method of training a policy to sample compositional discrete objects with probabilities proportional to a given reward via a sequence of actions. GFlowNets exploit the sequential nature of the problem, drawing parallels with reinforcement learning (RL). Our work extends the connection between RL and GFlowNets to a general case. We demonstrate how the task of learning a generative flow network can be efficiently redefined as an entropy-regularized RL problem with a specific reward and regularizer structure. Furthermore, we illustrate the practical efficiency of this reformulation by applying standard soft RL algorithms to GFlowNet training across several probabilistic modeling tasks. Contrary to previously reported results, we show that entropic RL approaches can be competitive against established GFlowNet training methods. This perspective opens a direct path for integrating RL principles into the realm of generative flow networks. (10.48550/arXiv.2310.12934)
  DOI : 10.48550/arXiv.2310.12934
• Mean Field Game of Mutual Holding with common noise
  
  • Bassou Leila
  • Djete Mao Fabrice
  • Touzi Nizar
  , 2024. We consider the mean field game of cross--holding introduced in \citeauthor*{DjeteTouzi} \cite{DjeteTouzi} in the context where the equity value dynamics are affected by a common noise. In contrast with \cite{DjeteTouzi}, the problem exhibits the standard paradigm of mean--variance trade off. Our crucial observation is to search for equilibrium solutions of our mean field game among those models which satisfy an appropriate notion of no--arbitrage. Under this condition, it follows that the representative agent optimization step is reduced to a standard portfolio optimization problem with random endowment.
• Stochastic Approximation with Biased MCMC for Expectation Maximization
  
  • Gruffaz Samuel
  • Kim Kyurae
  • Durmus Alain Oliviero
  • Gardner Jacob R
  , 2024. <div><p>The expectation maximization (EM) algorithm is a widespread method for empirical Bayesian inference, but its expectation step (E-step) is often intractable. Employing a stochastic approximation scheme with Markov chain Monte Carlo (MCMC) can circumvent this issue, resulting in an algorithm known as MCMC-SAEM. While theoretical guarantees for MCMC-SAEM have previously been established, these results are restricted to the case where asymptotically unbiased MCMC algorithms are used. In practice, MCMC-SAEM is often run with asymptotically biased MCMC, for which the consequences are theoretically less understood. In this work, we fill this gap by analyzing the asymptotics and non-asymptotics of SAEM with biased MCMC steps, particularly the effect of bias. We also provide numerical experiments comparing the Metropolisadjusted Langevin algorithm (MALA), which is asymptotically unbiased, and the unadjusted Langevin algorithm (ULA), which is asymptotically biased, on synthetic and real datasets. Experimental results show that ULA is more stable with respect to the choice of Langevin stepsize and can sometimes result in faster convergence.</p><p>Z S(y, z)p(z|θ, y) dz, set s k = s(θ k ). ❷ Maximization: Set θ k+1 = θ(s k ), which implies l(θ k+1 ) ≥ l(θ k ).</p><p>This algorithm quickly converges towards a local maximum of l under mild conditions. (See the review by McLachlan and Krishnan (2007).)</p></div>
• Proving Linear Mode Connectivity of Neural Networks via Optimal Transport
  
  • Ferbach Damien
  • Goujaud Baptiste
  • Gidel Gauthier
  • Dieuleveut Aymeric
  , 2024, 238, pp.3853-3861. The energy landscape of high-dimensional non-convex optimization problems is crucial to understanding the effectiveness of modern deep neural network architectures. Recent works have experimentally shown that two different solutions found after two runs of a stochastic training are often connected by very simple continuous paths (eg, linear) modulo a permutation of the weights. In this paper, we provide a framework theoretically explaining this empirical observation. Based on convergence rates in Wasserstein distance of empirical measures, we show that, with high probability, two wide enough two-layer neural networks trained with stochastic gradient descent are linearly connected. Additionally, we express upper and lower bounds on the width of each layer of two deep neural networks with independent neuron weights to be linearly connected. Finally, we empirically demonstrate the validity of our approach by showing how the dimension of the support of the weight distribution of neurons, which dictates Wasserstein convergence rates is correlated with linear mode connectivity.
• Mean field game of mutual holding with defaultable agents, and systemic risk
  
  • Djete Mao Fabrice
  • Guo Gaoyue
  • Touzi Nizar
  , 2023. We introduce the possibility of default in the mean field game of mutual holding of Djete and Touzi [11]. This is modeled by introducing absorption at the origin of the equity process. We provide an explicit solution of this mean field game. Moreover, we provide a particle system approximation, and we derive an autonomous equation for the time evolution of the default probability, or equivalently the law of the hitting time of the origin by the equity process. The systemic risk is thus described by the evolution of the default probability.
• Gaussian process regression with Sliced Wasserstein Weisfeiler-Lehman graph kernels
  
  • Carpintero Perez Raphaël
  • da Veiga Sébastien
  • Garnier Josselin
  • Staber Brian
  , 2024. Supervised learning has recently garnered significant attention in the field of computational physics due to its ability to effectively extract complex patterns for tasks like solving partial differential equations, or predicting material properties. Traditionally, such datasets consist of inputs given as meshes with a large number of nodes representing the problem geometry (seen as graphs), and corresponding outputs obtained with a numerical solver. This means the supervised learning model must be able to handle large and sparse graphs with continuous node attributes. In this work, we focus on Gaussian process regression, for which we introduce the Sliced Wasserstein Weisfeiler-Lehman (SWWL) graph kernel. In contrast to existing graph kernels, the proposed SWWL kernel enjoys positive definiteness and a drastic complexity reduction, which makes it possible to process datasets that were previously impossible to handle. The new kernel is first validated on graph classification for molecular datasets, where the input graphs have a few tens of nodes. The efficiency of the SWWL kernel is then illustrated on graph regression in computational fluid dynamics and solid mechanics, where the input graphs are made up of tens of thousands of nodes.
• Thorough mathematical modeling and analysis of Uniswap v3
  
  • Echenim Mnacho
  • Gobet Emmanuel
  • Maurice Anne-Claire
  , 2024.
• Second order quantitative bounds for unadjusted generalized Hamiltonian Monte Carlo
  
  • Camrud Evan
  • Durmus Alain Oliviero
  • Monmarché Pierre
  • Stoltz Gabriel
  , 2023. This paper provides a convergence analysis for generalized Hamiltonian Monte Carlo samplers, a family of Markov Chain Monte Carlo methods based on leapfrog integration of Hamiltonian dynamics and kinetic Langevin diffusion, that encompasses the unadjusted Hamiltonian Monte Carlo method. Assuming that the target distribution $\pi$ satisfies a log-Sobolev inequality and mild conditions on the corresponding potential function, we establish quantitative bounds on the relative entropy of the iterates defined by the algorithm, with respect to $\pi$. Our approach is based on a perturbative and discrete version of the modified entropy method developed to establish hypocoercivity for the continuous-time kinetic Langevin process. As a corollary of our main result, we are able to derive complexity bounds for the class of algorithms at hand. In particular, we show that the total number of iterations to achieve a target accuracy $\varepsilon >0$ is of order $d/\varepsilon^{1/4}$, where $d$ is the dimension of the problem. This result can be further improved in the case of weakly interacting mean field potentials, for which we find a total number of iterations of order $(d/\varepsilon)^{1/4}$.
• Friction damping for turbomachinery: A comprehensive review of modelling, design strategies, and testing capabilities
  
  • Yuan Jie
  • Gastaldi Chiara
  • Denimal Enora
  • Chouvion Benjamin
  Progress in Aerospace Sciences, Elsevier, 2024, 147, pp.101018. This paper presents a comprehensive review of recent advancements in modelling approaches, design strategies, and testing techniques applied to friction damping in turbomachinery. It critically evaluates experimental testing, design processes, and optimisation studies, along with the latest developments in numerical modelling techniques. The review begins with an overview of vibration mitigation methods and the historical development of friction dampers for bladed disk systems. Subsequent sections explore research efforts aimed at enhancing numerical and simulation modelling capabilities, encompassing contact friction models, reduced-order modelling methods, and numerical solvers suitable for real-world applications and industrial high-fidelity models. The paper also delves into available testing rigs for experimental validation and characterisation of various friction damper types, as well as the literature on uncertainty quantification in friction damping. It concludes by highlighting recent trends in novel concepts, modelling techniques, and testing technologies shaping the design of next-generation friction dampers. (10.1016/j.paerosci.2024.101018)
  DOI : 10.1016/j.paerosci.2024.101018
• Initialisation from lattice Boltzmann to multi-step Finite Difference methods: Modified equations and discrete observability
  
  • Bellotti Thomas
  Journal of Computational Physics, Elsevier, 2024, 504, pp.112871. Latitude on the choice of initialisation is a shared feature between one-step extended state-space and multi-step methods. The paper focuses on lattice Boltzmann schemes, which can be interpreted as examples of both previous categories of numerical schemes. We propose a modified equation analysis of the initialisation schemes for lattice Boltzmann methods, determined by the choice of initial data. These modified equations provide guidelines to devise and analyze the initialisation in terms of order of consistency with respect to the target Cauchy problem and time smoothness of the numerical solution. In detail, the larger the number of matched terms between modified equations for initialisation and bulk methods, the smoother the obtained numerical solution. This is particularly manifest for numerical dissipation. Starting from the constraints to achieve time smoothness, which can quickly become prohibitive for they have to take the parasitic modes into consideration, we explain how the distinct lack of observability for certain lattice Boltzmann schemes---seen as dynamical systems on a commutative ring---can yield rather simple conditions and be easily studied as far as their initialisation is concerned. This comes from the reduced number of initialisation schemes at the fully discrete level. These theoretical results are successfully assessed on several lattice Boltzmann methods. (10.1016/j.jcp.2024.112871)
  DOI : 10.1016/j.jcp.2024.112871
• Spine for interacting populations and sampling
  
  • Bansaye Vincent
  Bernoulli, Bernoulli Society for Mathematical Statistics and Probability, 2024, 30 (2). We consider Markov jump processes describing structured populations with interactions via density dependance. We propose a Markov construction with a distinguished individual which allows to describe the random tree and random sample at a given time via a change of probability. This spine construction involves the extension of type space of individuals to include the state of the population. The jump rates outside the spine are also modified. We apply this approach to some issues concerning evolution of populations and competition. For single type populations, we derive the diagram phase of a growth fragmentation model with competition and the growth of the size of birth and death processes with multiple births. We also describe the ancestral lineages of a uniform sample in multitype populations. (10.3150/23-BEJ1645)
  DOI : 10.3150/23-BEJ1645
• Generative models for ECG data : theory and application.
  
  • Victorino Cardoso Gabriel
  , 2024. This thesis contributes to the vast domain of Generative models, with a particular interest in applying such models to electrocardiogram (ECG) data for inference and uncertainty quantification.In a first part, we develop two novel methods for reducing bias in Importance Sampling and Sequential Monte Carlo (SMC) methods, which are two important tools of Bayesian inference.The issuing algorithms can both be viewed as a wrapper around current existing algorithms providing effortless bias reduction. We also provide new non-assymptotic convergence bounds for using such algorithms for parameter learning in Hidden Markov Models (HMM).In a second part, we focus on using SMC for solving Bayesian linear inverse problems with generative models serving as informative priors.Finally, we apply this method on several ECG based inverse problems, namely missing lead completion and out-of-distribution detection.
• Multispecies cross-diffusions: from a nonlocal mean-field to a porous medium system without self-diffusion
  
  • Doumic Marie
  • Hecht Sophie
  • Perthame Benoît
  • Peurichard Diane
  Journal of Differential Equations, Elsevier, 2024, 389, pp.228-256. Systems describing the long-range interaction between individuals have attracted a lot of attention in the last years, in particular in relation with living systems. These systems are quadratic, written under the form of transport equations with a nonlocal self-generated drift. We establish the localisation limit, that is the convergence of nonlocal to local systems, when the range of interaction tends to 0. These theoretical results are sustained by numerical simulations. The major new feature in our analysis is that we do not need diffusion to gain compactness, at odd with the existing literature. The central compactness result is provided by a full rank assumption on the interaction kernels. In turn, we prove existence of weak solutions for the resulting system, a cross-diffusion system of quadratic type. (10.1016/j.jde.2024.01.017)
  DOI : 10.1016/j.jde.2024.01.017
• Yet another analysis of the SP500 at-the-money skew: crossover of different power-law behaviours
  
  • Delemotte Jules
  • Ségonne Florent
  • de Marco Stefano
  , 2023.
• Control of isolated response curves through optimization of codimension-1 singularities
  
  • Mélot Adrien
  • Denimal Enora
  • Renson Ludovic
  Computers & Structures, Elsevier, 2024, pp.1-19. We introduce a computational framework for controlling the location of isolated response curves, i.e. responses that are not connected to the main solution branch and form a closed curve in parameter space. The methodology relies on bifurcation tracking to follow the evolution of fold bifurcations in a codimension-2 parameter space. Singularity theory is used to distinguish points of isola formation and merger from codimension-2 bifurcations and an optimization problem is formulated to delay or advance the onset or merger of isolated response curves or control their position in the state/parameter space. We illustrate the methodology on three examples: a finite element model of a cantilever beam with cubic nonlinearity at its tip, a two-degree-of-freedom oscillator with asymmetry and a two-degree-of-freedom base-excited oscillator exhibiting multiple isolas. Our results show that the location of points of isola formation and mergers can effectively be controlled through structural optimization. (10.1016/j.compstruc.2024.107394)
  DOI : 10.1016/j.compstruc.2024.107394
• Active learning of Boltzmann samplers and potential energies with quantum mechanical accuracy
  
  • Molina-Taborda Ana
  • Cossio Pilar
  • Lopez-Acevedo Olga
  • Gabrié Marylou
  , 2024. Extracting consistent statistics between relevant free-energy minima of a molecular system is essential for physics, chemistry and biology. Molecular dynamics (MD) simulations can aid in this task but are computationally expensive, especially for systems that require quantum accuracy. To overcome this challenge, we develop an approach combining enhanced sampling with deep generative models and active learning of a machine learning potential (MLP). We introduce an adaptive Markov chain Monte Carlo framework that enables the training of one Normalizing Flow (NF) and one MLP per state. We simulate several Markov chains in parallel until they reach convergence, sampling the Boltzmann distribution with an efficient use of energy evaluations. At each iteration, we compute the energy of a subset of the NF-generated configurations using Density Functional Theory (DFT), we predict the remaining configuration's energy with the MLP and actively train the MLP using the DFT-computed energies. Leveraging the trained NF and MLP models, we can compute thermodynamic observables such as free-energy differences or optical spectra. We apply this method to study the isomerization of an ultrasmall silver nanocluster, belonging to a set of systems with diverse applications in the fields of medicine and catalysis. (10.48550/arXiv.2401.16487)
  DOI : 10.48550/arXiv.2401.16487
• A practical global existence and uniqueness result for stochastic differential equations on Riemannian manifolds of bounded geometry
  
  • Rakotomalala Matthias
  , 2024. In this paper, we establish a result for existence and uniqueness of stochastic differential equa- tions on Riemannian manifolds, for regular inhomogeneous tensor coefficients with stochastic drift, under geometrical-only hypothesis on the manifold, so-called manifolds of bounded geometry, this hypothesis is consistent with the maximal regularity result for parabolic equations obtained by Herbert Amann. Furthermore, we provide a stochastic flow estimate for the solutions.
• Branching diffusion processes and spectral properties of Feynman-Kac semigroup
  
  • Collet Pierre
  • Méléard Sylvie
  • San MARTIN Jaime
  , 2024. In this article we study the long time behavior of linear functionals of branching diffusion processes as well as the time reversal of the spinal process by means of spectral properties of the Feynman-Kac semigroup. We generalize for this non Markovian semigroup the theory of quasi-stationary distribution (q.s.d.) and Q-process. The most amazing result is the identification of the law of the reversal time spinal process issued from q.s.d. with the Q-process of the Feynman-Kac semigroup.
• Banning new gas boilers as a no-regret mitigation option
  
  • Escribe Célia
  • Vivier Lucas
  , 2024, pp.054018. The low uptake of low-carbon heating systems across Europe has prompted authorities to consider more ambitious measures, including a complete ban on the installation of new fossil fuel boilers. In this analysis, we simulate the impacts of introducing this ban in France under 11,664 scenarios covering major uncertainties. We find that the ban induces major changes in the energy system, leading to efficiency gains. Additionally, we find that the ban increases the likelihood of reaching carbon neutrality while reducing total system cost in over 75% of scenarios. Finally, we show that the implementation of the ban, when coupled with the existing subsidy framework, mitigates inequalities among owner-occupied households but generates adverse effects for those in privately rented homes.
• Glass optimization for optical design: CMA-ES optimizer with integer handling
  
  • Marty Tristan
  • Héron Sébastien
  • Semet Yann
  • Auger Anne
  • Hansen Nikolaus
  , 2024, pp.7. Traditional optimization of lens material uses a two dimensional continuous space to describe a material along with a local optimizer like Levenberg-Marquardt. However other algorithms may be more suited for harder problems, in particular if the problem has several local minima, large number of variables and integer variables. The presented optimization method make use of evolution strategy with covariance matrix adaptation (CMA-ES). A modified version of this algorithm is used to handle the optimization of lens material as integer variables. Results will be focused on the performance obtained with complex camera lens composed of 31 diopters and optimized for 3 configurations corresponding to 3 different f number. (10.1117/12.3016937)
  DOI : 10.1117/12.3016937
• Constructive approaches to worst-case complexity analyses of gradient methods for convex optimization : contributions, new insights, and novel results
  
  • Goujaud Baptiste
  , 2024. In the current era marked by an unprecedented surge in available data and computational prowess, the field of machine learning, and more specifically deep learning, has witnessed an extraordinary evolution.Machine learning algorithms heavily rely on optimization techniques to tune their parameters and enhance predictive accuracy.Amidst the myriad of optimization approaches, the first-order optimization methods have emerged as stalwarts, demonstrating a remarkable balance between efficacy and computational efficiency.Crucially, the development of strong optimization theory is pivotal in unraveling the full potential of first-order optimization.Theoretical underpinnings not only deepen our understanding of optimization landscapes but also pave the way for the design of novel algorithms.The momentum-based algorithms have proven their mettle by significantly accelerating convergence in optimization problems.The conceptual foundation provided by optimization theory has enabled the formulation of momentum, turning theoretical insights into a powerful and widely adopted practical optimization tool.The role of this thesis is to pursue and accelerate the effort to develop a strong theoretical foundation of first-order optimization.We proved various results, exploiting the general structures of the certificate proofs.(i) We used the link between quadratic optimization and polynomial theory to explain empirically observed phenomena.(ii) We implemented a Python package to support the emph{Performance estimation} framework.(iii) We wrote a tutorial to explain how to derive natural proofs in optimization based on this framework.(iv) We applied this methodology, with the help of our python package, to derive a complete first-order optimization theory on a very large class of functions.(v) We complemented the theoretical emph{Performance estimation} framework to disprove the convergence of a specific family of methods and applied it to the famous Heavy-ball method to provably disprove an acceleration over the class of smooth and strongly convex functions.
• Entropic Lower Bound of Cardinality for Sparse Optimization
  
  • Jacquet Quentin
  • Bialecki Agnes
  • Ghaoui Laurent El
  • Gaubert Stéphane
  • Zorgati Riadh
  , 2024. We introduce a family of cardinality's lower bounds, defined as ratios of norms. We prove that the tightest bound of the family is obtained as a limit case, and involves a Shannon entropy. We then use this entropic lower bound in sparse optimization problems to approximate cardinality requirements. This provides a nonlinear nonconvex relaxed problem, which can be efficiently solved by off-the-shelf nonlinear solvers. In the numerical study, we focus on the case where the optimization is performed on the simplex, and where the classical l1 penalization does not yield sparse solution. The Finance Index Tracking problem is taken as an example and illustrates the efficiency of the proposed approach.
• Learning of extreme Expected Shortfall with neural networks. Application to cryptocurrency data
  
  • Allouche Michaël
  • Girard Stéphane
  • Gobet Emmanuel
  , 2024. We propose new parametrizations for neural networks in order to estimate extreme Value-at-Risk and Expected-Shortfall in heavy-tailed settings. All proposed neural network estimators feature a bias correction based on an extension of the usual second-order condition to an arbitrary order.The convergence rate of the uniform error between extreme log quantities and their neural network approximation is established. The finite sample performances of the neural network estimator are compared to other bias-reduced extreme-value competitors on both real and simulated data. It is shown that our method outperforms them in difficult heavy-tailed situations where other estimators almost all fail.