Mathematical Sciences Department Math Seminar
Yu-Jui Huang, University of Colorado
Friday, April 18th
11:00am - 12:00 pm
Title: Langevin diffusions with density-dependent temperature
Abstract: In non-convex optimization, Langevin diffusions are commonly used in search of an (approximate) global minimizer. The fundamental rationale is to perturb gradient descent by a Brownian motion, whose influence is controlled by a "temperature" process, thereby allowing the diffusion to escape from local minimizers. In the literature, the temperature process is usually exogenously given as a constant or a time-dependent function, which is in itself independent of the diffusion. In this talk, we introduce a new temperature process that endogenously depends on the probability density function of the diffusion. As the Langevin dynamics is now self-regulated by its own probability density at each time, it forms a distribution-dependent stochastic differential equation (SDE) of the Nemytskii type, distinct from the standard McKean-Vlasov equations. For the existence of a solution to the SDE, we first show that the corresponding Fokker-Planck equation has a solution, relying on appropriate regularization and approximation arguments; next, by Trevisan's superposition principle, a weak solution to the SDE is constructed from the solution to the Fokker-Planck equation. Furthermore, based on suitable SDE estimates, we prove that as time goes to infinity, the probability density functions induced by the SDE has a well-defined limit, which admits an explicit formula in terms of the Lambert W function. We demonstrate in several numerical examples that the density-regulated Langevin dynamics approaches the global minimum faster than the classical Langevin dynamics.