SciML/Optimization.jl

Optimization.jl

Optimization.jl is a package with a scope that is beyond your normal global optimization package. Optimization.jl seeks to bring together all of the optimization packages it can find, local and global, into one unified Julia interface. This means, you learn one package and you learn them all! Optimization.jl adds a few high-level features, such as integrating with automatic differentiation, to make its usage fairly simple for most cases, while allowing all of the options in a single unified interface.

Installation

Assuming that you already have Julia correctly installed, it suffices to import OptimizationBase in the standard way:

using Pkg
Pkg.add("OptimizationBase")

The packages relevant to the core functionality of Optimization.jl will be imported accordingly and, in most cases, you do not have to worry about the manual installation of dependencies. Below is the list of packages that need to be installed explicitly if you intend to use the specific optimization algorithms offered by them:

• OptimizationAuglag for augmented Lagrangian methods
• OptimizationBBO for BlackBoxOptim.jl
• OptimizationCMAEvolutionStrategy for CMAEvolutionStrategy.jl
• OptimizationEvolutionary for Evolutionary.jl (see also this documentation)
• OptimizationGCMAES for GCMAES.jl
• OptimizationIpopt for Ipopt.jl
• OptimizationLBFGSB for LBFGSB.jl
• OptimizationMadNLP for MadNLP.jl
• OptimizationManopt for Manopt.jl (optimization on manifolds)
• OptimizationMetaheuristics for Metaheuristics.jl (see also this documentation)
• OptimizationMOI for MathOptInterface.jl (usage of algorithm via MathOptInterface API; see also the API documentation)
• OptimizationMultistartOptimization for MultistartOptimization.jl (see also this documentation)
• OptimizationNLopt for NLopt.jl (usage via the NLopt API; see also the available algorithms)
• OptimizationNLPModels for NLPModels.jl
• OptimizationNOMAD for NOMAD.jl (see also this documentation)
• OptimizationODE for optimization of steady-state and time-dependent ODE problems
• OptimizationOptimJL for Optim.jl
• OptimizationOptimisers for Optimisers.jl (machine learning optimizers)
• OptimizationPolyalgorithms for polyalgorithm optimization strategies
• OptimizationPRIMA for PRIMA.jl
• OptimizationPyCMA for Python's CMA-ES implementation via PythonCall.jl
• OptimizationQuadDIRECT for QuadDIRECT.jl
• OptimizationSciPy for SciPy optimization algorithms via PythonCall.jl
• OptimizationSophia for Sophia optimizer (second-order stochastic optimizer)
• OptimizationSpeedMapping for SpeedMapping.jl (see also this documentation)

Tutorials and Documentation

For information on using the package, see the stable documentation. Use the in-development documentation for the version of the documentation, which contains the unreleased features.

Examples

using OptimizationBase
rosenbrock(x, p) = (p[1] - x[1])^2 + p[2] * (x[2] - x[1]^2)^2
x0 = zeros(2)
p = [1.0, 100.0]

prob = OptimizationProblem(rosenbrock, x0, p)

using OptimizationOptimJL
sol = solve(prob, NelderMead())

using OptimizationBBO
prob = OptimizationProblem(rosenbrock, x0, p, lb = [-1.0, -1.0], ub = [1.0, 1.0])
sol = solve(prob, BBO_adaptive_de_rand_1_bin_radiuslimited())

The output of the first optimization task (with the NelderMead() algorithm) is given below:

* Status: success

* Candidate solution
   Final objective value:     3.525527e-09

* Found with
   Algorithm:     Nelder-Mead

* Convergence measures
   √(Σ(yᵢ-ȳ)²)/n ≤ 1.0e-08

* Work counters
   Seconds run:   0  (vs limit Inf)
   Iterations:    60
   f(x) calls:    118

We can also explore other methods in a similar way:

using ForwardDiff, ADTypes
f = OptimizationFunction(rosenbrock, ADTypes.AutoForwardDiff())
prob = OptimizationProblem(f, x0, p)
sol = solve(prob, BFGS())

For instance, the above optimization task produces the following output:

* Status: success

* Candidate solution
   Final objective value:     7.645684e-21

* Found with
   Algorithm:     BFGS

* Convergence measures
   |x - x'|               = 3.48e-07 ≰ 0.0e+00
   |x - x'|/|x'|          = 3.48e-07 ≰ 0.0e+00
   |f(x) - f(x')|         = 6.91e-14 ≰ 0.0e+00
   |f(x) - f(x')|/|f(x')| = 9.03e+06 ≰ 0.0e+00
   |g(x)|                 = 2.32e-09 ≤ 1.0e-08

* Work counters
   Seconds run:   0  (vs limit Inf)
   Iterations:    16
   f(x) calls:    53
   ∇f(x) calls:   53

prob = OptimizationProblem(f, x0, p, lb = [-1.0, -1.0], ub = [1.0, 1.0])
sol = solve(prob, Fminbox(GradientDescent()))

The examples clearly demonstrate that Optimization.jl provides an intuitive way of specifying optimization tasks and offers a relatively easy access to a wide range of optimization algorithms.