Consensus-based optimization is the name for a first-order stochastic interacting particle algorithms that seeks to find the global minimum of a non-convex objective function. The idea is to combine ideas from swarm optimization and opinion formation to obtain an algorithm that allows for analytical convergence results.
More details can be found in the following articles:
- A consensus-based model for global optimization and its mean-field limit
R. Pinnau, C. Totzeck, O. Tse, S. Martin Mathematical Models and Methods in Applied Sciences 27 (1), 2017.
- An analytical framework for a consensus-based global optimization method
J. A. Carrillo, Y.-P. Choi, C. Totzeck, O. Tse Mathematical Models and Methods in Applied Sciences 28 (06), pp. 1037-1066, 2018.
- J.A. Carrillo, S. Jin, L. Li and Y. Zhu (2019) A consensus-based global optimization method for high dimen-sional machine learning problems, preprint arXiv:1909.09249
- S.-Y. Ha, S. Jin and D. Kim (2019) Convergence and error estimates for time-discrete consensus-based opti-mization algorithms
- M. Fornasier, H. Huang, L. Pareschi and P. Sünnen (2020) Consensus-Based Optimization on the Sphere I:Well-Posedness and Mean-Field Limit
- M. Fornasier, H. Huang, L. Pareschi and P. Sünnen (2020) Consensus-based Optimization on the Sphere II:Convergence to Global Minimizers and Machine Learning preprint arXiv:2001.11988
- C. Totzeck and M.-T. Wolfram (2020) Consensus-Based Global Optimization with Personal Best
Via mean-field limits the optimization of interacting particle systems is related to the optimization of kinetic equations such as Vlasov- or Fokker-Planck type PDEs. Usually the spaces are equipped with the Wasserstein metric which bring us to the task of optimization with PDEs in spaces without Hilbert structure. This is an interesting area of research that I look forward to explore further.
Some articles related to this:
- Instantaneous control of interacting particle systems in the mean-field limit
M. Burger, R. Pinnau, C. Totzeck, O. Tse, A. Roth Journal of Computational Physics 405, pp. 109181, 2020.
- M. Burger, R. Pinnau, C. Totzeck, O. Tse (2019) Mean-field optimal control and optimality conditions in the space of probability measures
- Parameter identification in uncertain scalar conservation laws discretized with the discontinuous stochastic Galerkin Scheme
L. Schlachter, C. Totzeck, 2020. Accepted for publication in Communications in Computational Physics
- C. Totzeck (2019) An anisotropic interaction model with collision avoidance
Optimization problems are not directly solvable but allow for substitute problems which can be optimization are an interesting playgrounds for space mapping. The main idea is to approximate the optimizer by iterative solving the substitute problem. The method gives reasonable results for a stochastic optimization problem with interacting particles, where the substitute model is the corresponding determinstic problem. This motivates me to analyse the method in more detail. See here for the promising approach in stochastic interacting particle systems:
- Space mapping-based receding horizon control for stochastic interacting particle systems: dogs herding sheep
R. Pinnau, C. Totzeck Journal of Mathematics in Industry 10 (11), 2020.