consensus-based optimization (CBO)

Consensus-based optimization is the name for a first-order stochastic interacting particle algorithms that seeks to find the global minimum of a non-convex objective function. The idea is to combine ideas from swarm optimization and opinion formation to obtain an algorithm that allows for analytical convergence results.
More details can be found in the following articles:

  • A consensus-based model for global optimization and its mean-field limit
    R. Pinnau, C. Totzeck, O. Tse, S. Martin Mathematical Models and Methods in Applied Sciences 27 (1), 2017.
  • An analytical framework for a consensus-based global optimization method
    J. A. Carrillo, Y.-P. Choi, C. Totzeck, O. Tse Mathematical Models and Methods in Applied Sciences 28 (06), pp. 1037-1066, 2018.
  • J.A. Carrillo, S. Jin, L. Li and Y. Zhu (2019) A consensus-based global optimization method for high dimen-sional machine learning problems, preprint arXiv:1909.09249
  • S.-Y. Ha, S. Jin and D. Kim (2019) Convergence and error estimates for time-discrete consensus-based opti-mization algorithms
    preprint arXiv:2003.05086
  • M. Fornasier, H. Huang, L. Pareschi and P. Sünnen (2020) Consensus-Based Optimization on the Sphere I:Well-Posedness and Mean-Field Limit
    preprint arXiv:2001.11994
  • M. Fornasier, H. Huang, L. Pareschi and P. Sünnen (2020) Consensus-based Optimization on the Sphere II:Convergence to Global Minimizers and Machine Learning preprint arXiv:2001.11988
  • C. Totzeck and M.-T. Wolfram (2020) Consensus-Based Global Optimization with Personal Best
    preprint arXiv:2005.07084
  • C. Totzeck, Trends in consensus-based optimization, in: Active Particles, Volume 3, Springer International Publishing, 2022.
  • S. Göttlich, C. Totzeck, Parameter calibration with Consensus-based Optimization for interaction dynamics driven by neural networks, in: Ehrhardt, M., Günther, M. (eds) Progress in Industrial Mathematics at ECMI 2021. ECMI 2021. Mathematics in Industry, vol 39. Springer, Cham, 2022.
  • K. Klamroth, M. Stiglmayr, C. Totzeck (2022) Consensus-Based Optimization for Multi-Objective Problems: A Multi-Swarm Approach
  • J. A. Carrillo, C. Totzeck and U. Vaes, Consensus-based Optimization and Ensemble Kalman Inversion for Global Optimization Problems with Constraints, in: Modeling and Simulation for Collective Dynamics, Lecture Notes Series, Institute for Mathematical Sciences, NUS: Volume 40, 2023.

optimization with ode/pde

Via mean-field limits the optimization of interacting particle systems is related to the optimization of kinetic equations such as Vlasov- or Fokker-Planck type PDEs. Usually the spaces are equipped with the Wasserstein metric which bring us to the task of optimization with PDEs in spaces without Hilbert structure. This is an interesting area of research that I look forward to explore further.
Some articles related to this:

  • Instantaneous control of interacting particle systems in the mean-field limit
    M. Burger, R. Pinnau, C. Totzeck, O. Tse, A. Roth Journal of Computational Physics 405, pp. 109181, 2020.
  • M. Burger, R. Pinnau, C. Totzeck, O. Tse (2019) Mean-field optimal control and optimality conditions in the space of probability measures
  • Parameter identification in uncertain scalar conservation laws discretized with the discontinuous stochastic Galerkin Scheme
    L. Schlachter, C. Totzeck, 2020. Accepted for publication in Communications in Computational Physics
  • C. Totzeck (2019) An anisotropic interaction model with collision avoidance

space mapping for stochastic interactig particle systems

Optimization problems are not directly solvable but allow for substitute problems which can be optimization are an interesting playgrounds for space mapping. The main idea is to approximate the optimizer by iterative solving the substitute problem. The method gives reasonable results for a stochastic optimization problem with interacting particles, where the substitute model is the corresponding determinstic problem. This motivates me to analyse the method in more detail. See here for the promising approach in stochastic interacting particle systems:

  • Space mapping-based optimization with the macroscopic limit of interacting particle systems
    J. Weißen, S. Göttlich, C. Totzeck
    Optimization and Engineering, online first, 2021.
  • Space mapping-based receding horizon control for stochastic interacting particle systems: dogs herding sheep
    R. Pinnau, C. Totzeck Journal of Mathematics in Industry 10 (11), 2020.

mean-field limits and optimization of port-Hamiltonian systems

Port-Hamiltonian modeling focusses on energy-preserving coupling of (sub)systems. Originating from physical applications the framework is transferred to more abstract mathematical systems such as interacting particle systems in the recent years. In my opinion the structure is interesting as it combines Hamiltonian dynamics with dissipative effects. Clearly, it is desirable to preserve the port-Hamiltonian structure in data-based calibration or optimization tasks. This turns out to be challenging as for example the the matrix spaced underlying port-Hamiltonian ODE are in general not flat. The following works are concerned with these topics:

  • B. Jacob, C. Totzeck (2023) Port-Hamiltonian structure of interacting particle systems and its mean-field limit
  • M. Günther, B. Jacob, C. Totzeck (2023) Data-driven adjoint-based calibration of port-Hamiltonian systems in time domain
  • A. Tordeux, C. Totzeck (2022) Multi-scale description of pedestrian collective dynamics with port-Hamiltonian systems
  • M. Günther, B. Jacob, C. Totzeck (2022) Structure-preserving identification of port-Hamiltonian systems - a sensitivity-based approach