Rosenbrock¶
Minimize 2-variable nonsmooth Rosenbrock function, subject to a simple bound constraint. Taken from: GRANSO demo examples 1, 2, & 3
Problem Description¶
\[\min_{x_1,x_2} w|x_1^2-x_2|+(1-x_1)^2,\]
\[\text{s.t. }c_1(x_1,x_2) = \sqrt{2}x_1-1 \leq 0, c_(x_1,x_2)=2x_2-1\leq0,\]
where \(w\) is a constant (e.g., \(w=8\))
Modules Importing¶
Import all necessary modules and add PyGRANSO src folder to system path.
[1]:
import time
import torch
from pygranso.pygranso import pygranso
from pygranso.pygransoStruct import pygransoStruct
Function Set-Up¶
Encode the optimization variables, and objective and constraint functions.
Note: please strictly follow the format of comb_fn, which will be used in the PyGRANSO main algortihm.
[2]:
device = torch.device('cpu')
# variables and corresponding dimensions.
var_in = {"x1": [1], "x2": [1]}
def comb_fn(X_struct):
x1 = X_struct.x1
x2 = X_struct.x2
# objective function
f = (8 * abs(x1**2 - x2) + (1 - x1)**2)
# inequality constraint, matrix form
ci = pygransoStruct()
ci.c1 = (2**0.5)*x1-1
ci.c2 = 2*x2-1
# equality constraint
ce = None
return [f,ci,ce]
User Options¶
Specify user-defined options for PyGRANSO
[3]:
opts = pygransoStruct()
# option for switching QP solver. We only have osqp as the only qp solver in current version. Default is osqp
# opts.QPsolver = 'osqp'
# set an intial point
# All the user-provided data (vector/matrix/tensor) must be in torch tensor format.
# As PyTorch tensor is single precision by default, one must explicitly set `dtype=torch.double`.
# Also, please make sure the device of provided torch tensor is the same as opts.torch_device.
opts.x0 = torch.ones((2,1), device=device, dtype=torch.double)
opts.torch_device = device
Main Algorithm¶
[4]:
start = time.time()
soln = pygranso(var_spec = var_in,combined_fn = comb_fn, user_opts = opts)
end = time.time()
print("Total Wall Time: {}s".format(end - start))
print(soln.final.x)
╔═════ QP SOLVER NOTICE ════════════════════════════════════════════════════════════════════════╗
║ PyGRANSO requires a quadratic program (QP) solver that has a quadprog-compatible interface, ║
║ the default is osqp. Users may provide their own wrapper for the QP solver. ║
║ To disable this notice, set opts.quadprog_info_msg = False ║
╚═══════════════════════════════════════════════════════════════════════════════════════════════╝
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation ║
Version 1.2.0 ║
Licensed under the AGPLv3, Copyright (C) 2021-2022 Tim Mitchell and Buyun Liang ║
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
Problem specifications: ║
# of variables : 2 ║
# of inequality constraints : 2 ║
# of equality constraints : 0 ║
═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣
║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║
Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║
═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣
0 ║ 1.000000 │ 1.41421356237 ║ 0.00000000000 ║ 1.000000 │ - ║ - │ 1 │ 0.000000 ║ 1 │ 0.579471 ║
1 ║ 1.000000 │ 0.70773811042 ║ 0.70773811042 ║ 0.000000 │ - ║ S │ 3 │ 1.500000 ║ 1 │ 10.07366 ║
2 ║ 1.000000 │ 0.25401310554 ║ 0.25401310554 ║ 0.000000 │ - ║ S │ 3 │ 0.250000 ║ 1 │ 0.198885 ║
3 ║ 1.000000 │ 0.21478744238 ║ 0.21478744238 ║ 0.000000 │ - ║ S │ 3 │ 0.250000 ║ 1 │ 0.135710 ║
4 ║ 1.000000 │ 0.21422378595 ║ 0.21422378595 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.332997 ║
5 ║ 1.000000 │ 0.15330884270 ║ 0.15330884270 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.122691 ║
6 ║ 1.000000 │ 0.14804462353 ║ 0.14804462353 ║ 0.000000 │ - ║ S │ 2 │ 0.500000 ║ 1 │ 0.012623 ║
7 ║ 1.000000 │ 0.10856024489 ║ 0.10856024489 ║ 0.000000 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 0.042111 ║
8 ║ 1.000000 │ 0.10482595154 ║ 0.10482595154 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.003211 ║
9 ║ 0.810000 │ 0.07758251262 ║ 0.09438278485 ║ 0.001132 │ - ║ S │ 3 │ 1.500000 ║ 1 │ 0.038778 ║
10 ║ 0.810000 │ 0.07197699268 ║ 0.08886048479 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.012307 ║
11 ║ 0.810000 │ 0.07055904204 ║ 0.08710992844 ║ 0.000000 │ - ║ S │ 3 │ 0.250000 ║ 1 │ 0.003100 ║
12 ║ 0.810000 │ 0.07048871361 ║ 0.08702310322 ║ 0.000000 │ - ║ S │ 7 │ 0.046875 ║ 1 │ 0.003061 ║
13 ║ 0.810000 │ 0.07020995506 ║ 0.08667895687 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.001026 ║
14 ║ 0.810000 │ 0.06962027906 ║ 0.08595096180 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 5.51e-06 ║
15 ║ 0.810000 │ 0.06952233581 ║ 0.08581975963 ║ 8.33e-06 │ - ║ S │ 3 │ 4.000000 ║ 2 │ 5.10e-06 ║
16 ║ 0.729000 │ 0.06255153247 ║ 0.08579440422 ║ 4.17e-06 │ - ║ S │ 2 │ 0.500000 ║ 3 │ 3.98e-06 ║
17 ║ 0.729000 │ 0.06254666970 ║ 0.08579790082 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 4 │ 7.59e-19 ║
═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣
Optimization results: ║
F = final iterate, B = Best (to tolerance), MF = Most Feasible ║
═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣
F ║ │ ║ 0.08579790082 ║ 0.000000 │ - ║ │ │ ║ │ ║
B ║ │ ║ 0.08578643763 ║ 0.000000 │ - ║ │ │ ║ │ ║
MF ║ │ ║ 0.08578643763 ║ 0.000000 │ - ║ │ │ ║ │ ║
═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣
Iterations: 17 ║
Function evaluations: 40 ║
PyGRANSO termination code: 0 --- converged to stationarity and feasibility tolerances. ║
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝
Total Wall Time: 0.18392133712768555s
tensor([[0.7071],
[0.5000]], dtype=torch.float64)
PyGRANSO Restarting¶
(Optional) The following example shows how to set various PyGRANSO options (such as simpler ASCII printing) and how to restart PyGRANSO
[5]:
opts = pygransoStruct()
opts.torch_device = device
# set an infeasible initial point
opts.x0 = 5.5*torch.ones((2,1), device=device, dtype=torch.double)
# By default PyGRANSO will print using extended ASCII characters to 'draw' table borders and some color prints.
# If user wants to create a log txt file of the console output, please set opts.print_ascii = True
opts.print_ascii = True
# By default, PyGRANSO prints an info message about QP solvers, since
# PyGRANSO can be used with any QP solver that has a quadprog-compatible
# interface. Let's disable this message since we've already seen it
# hundreds of times and can now recite it from memory. ;-)
opts.quadprog_info_msg = False
# Try a very short run.
opts.maxit = 10 # default is 1000
# PyGRANSO's penalty parameter is on the *objective* function, thus
# higher penalty parameter values favor objective minimization more
# highly than attaining feasibility. Let's set PyGRANSO to start off
# with a higher initial value of the penalty parameter. PyGRANSO will
# automatically tune the penalty parameter to promote progress towards
# feasibility. PyGRANSO only adjusts the penalty parameter in a
# monotonically decreasing fashion.
opts.mu0 = 100 # default is 1
# start main algorithm
soln = pygranso(var_spec = var_in,combined_fn = comb_fn, user_opts = opts)
==================================================================================================================
PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation |
Version 1.2.0 |
Licensed under the AGPLv3, Copyright (C) 2021-2022 Tim Mitchell and Buyun Liang |
==================================================================================================================
Problem specifications: |
# of variables : 2 |
# of inequality constraints : 2 |
# of equality constraints : 0 |
==================================================================================================================
| <--- Penalty Function --> | | Total Violation | <--- Line Search ---> | <- Stationarity -> |
Iter | Mu | Value | Objective | Ineq | Eq | SD | Evals | t | Grads | Value |
=====|===========================|================|=================|=======================|====================|
0 | 100.0000 | 21841.7781746 | 218.250000000 | 10.00000 | - | - | 1 | 0.000000 | 1 | 9732.770 |
1 | 34.86784 | 1509.66611872 | 42.9789783006 | 11.08181 | - | S | 10 | 0.001953 | 1 | 546.6040 |
2 | 34.86784 | 1378.57842221 | 39.2815768212 | 8.914529 | - | S | 3 | 1.500000 | 1 | 4.455384 |
3 | 12.15767 | 285.452828054 | 22.6935200208 | 9.552604 | - | S | 2 | 2.000000 | 1 | 0.297144 |
4 | 12.15767 | 264.999595731 | 21.0732808630 | 8.797697 | - | S | 2 | 2.000000 | 1 | 0.603629 |
5 | 4.239116 | 60.5144787250 | 12.1478493778 | 9.018338 | - | S | 2 | 2.000000 | 1 | 0.111610 |
6 | 4.239116 | 53.5399399407 | 10.5181947367 | 8.952094 | - | S | 2 | 0.500000 | 1 | 0.164082 |
7 | 3.815204 | 48.9917031616 | 10.4947962860 | 8.951912 | - | S | 4 | 0.125000 | 1 | 0.033640 |
8 | 3.815204 | 48.7011303503 | 10.4372013183 | 8.881076 | - | S | 2 | 2.000000 | 1 | 0.018555 |
9 | 3.815204 | 48.2564717826 | 10.3422772655 | 8.798572 | - | S | 2 | 2.000000 | 1 | 0.057946 |
10 | 3.815204 | 39.4225027901 | 9.27057783616 | 4.053355 | - | S | 5 | 16.00000 | 1 | 0.001796 |
==================================================================================================================
Optimization results: |
F = final iterate, B = Best (to tolerance), MF = Most Feasible |
==================================================================================================================
F | | | 9.27057783616 | 4.053355 | - | | | | | |
MF | | | 9.27057783616 | 4.053355 | - | | | | | |
==================================================================================================================
Iterations: 10 |
Function evaluations: 35 |
PyGRANSO termination code: 4 --- max iterations reached. |
==================================================================================================================
Let’s restart PyGRANSO from the last iterate of the previous run
[6]:
opts = pygransoStruct()
opts.torch_device = device
# set the initial point and penalty parameter to their final values from the previous run
opts.x0 = soln.final.x
opts.mu0 = soln.final.mu
opts.opt_tol = 1e-6
# PREPARE TO RESTART PyGRANSO IN FULL-MEMORY MODE
# Set the last BFGS inverse Hessian approximation as the initial
# Hessian for the next run. Generally this is a good thing to do, and
# often it is necessary to retain this information when restarting (as
# on difficult nonsmooth problems, PyGRANSO may not be able to restart
# without it). However, your mileage may vary. In the test, with
# the above settings, omitting H0 causes PyGRANSO to take an additional
# 16 iterations to converge on this problem.
opts.H0 = soln.H_final # try running with this commented out
# When restarting, soln.H_final may fail PyGRANSO's initial check to
# assess whether or not the user-provided H0 is positive definite. If
# it fails this test, the test may be disabled by setting opts.checkH0
# to false.
# opts.checkH0 = False % Not needed for this example
# If one desires to restart PyGRANSO as if it had never stopped (e.g.
# to continue optimization after it hit its maxit limit), then one must
# also disable scaling the initial BFGS inverse Hessian approximation
# on the very first iterate.
opts.scaleH0 = False
# Restart PyGRANSO
opts.maxit = 100 # increase maximum allowed iterations
# Main algorithm
soln = pygranso(var_spec = var_in,combined_fn = comb_fn, user_opts = opts)
╔═════ QP SOLVER NOTICE ════════════════════════════════════════════════════════════════════════╗
║ PyGRANSO requires a quadratic program (QP) solver that has a quadprog-compatible interface, ║
║ the default is osqp. Users may provide their own wrapper for the QP solver. ║
║ To disable this notice, set opts.quadprog_info_msg = False ║
╚═══════════════════════════════════════════════════════════════════════════════════════════════╝
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation ║
Version 1.2.0 ║
Licensed under the AGPLv3, Copyright (C) 2021-2022 Tim Mitchell and Buyun Liang ║
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
Problem specifications: ║
# of variables : 2 ║
# of inequality constraints : 2 ║
# of equality constraints : 0 ║
═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣
║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║
Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║
═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣
0 ║ 3.815204 │ 39.4225027901 ║ 9.27057783616 ║ 4.053355 │ - ║ - │ 1 │ 0.000000 ║ 1 │ 0.161642 ║
1 ║ 2.503156 │ 27.1659262914 ║ 9.40498130422 ║ 3.623796 │ - ║ S │ 2 │ 2.000000 ║ 1 │ 0.052069 ║
2 ║ 2.252840 │ 24.6931839860 ║ 9.60278347893 ║ 3.059650 │ - ║ S │ 2 │ 2.000000 ║ 1 │ 0.121925 ║
3 ║ 2.027556 │ 22.3469956308 ║ 9.74369172380 ║ 2.591115 │ - ║ S │ 2 │ 2.000000 ║ 1 │ 0.107446 ║
4 ║ 2.027556 │ 21.7313246108 ║ 9.83432474079 ║ 1.791681 │ - ║ S │ 2 │ 2.000000 ║ 1 │ 0.249443 ║
5 ║ 2.027556 │ 18.8070548305 ║ 9.27572664351 ║ 0.000000 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 1.149786 ║
6 ║ 2.027556 │ 15.5789878011 ║ 7.13239951765 ║ 1.117649 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 1.812487 ║
7 ║ 2.027556 │ 5.32456682805 ║ 2.62610104757 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.324572 ║
8 ║ 2.027556 │ 5.06514188742 ║ 2.49815146400 ║ 0.000000 │ - ║ S │ 3 │ 0.250000 ║ 1 │ 0.840053 ║
9 ║ 2.027556 │ 4.34855971809 ║ 2.14472981556 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.161357 ║
10 ║ 2.027556 │ 4.17458588015 ║ 2.05892511204 ║ 0.000000 │ - ║ S │ 2 │ 2.000000 ║ 1 │ 0.438844 ║
11 ║ 2.027556 │ 3.88777301389 ║ 1.91746767656 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.026078 ║
12 ║ 2.027556 │ 3.02755206246 ║ 1.49320271480 ║ 0.000000 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 0.341024 ║
13 ║ 2.027556 │ 2.82661618788 ║ 1.39410020980 ║ 0.000000 │ - ║ S │ 2 │ 0.500000 ║ 1 │ 0.138146 ║
14 ║ 2.027556 │ 2.61942625386 ║ 1.29191317368 ║ 0.000000 │ - ║ S │ 4 │ 3.000000 ║ 1 │ 0.271047 ║
15 ║ 2.027556 │ 2.41981099682 ║ 1.19346200337 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.022472 ║
16 ║ 2.027556 │ 1.68687275803 ║ 0.83197346564 ║ 0.000000 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 0.273910 ║
17 ║ 2.027556 │ 1.62715982554 ║ 0.80252277047 ║ 0.000000 │ - ║ S │ 4 │ 0.125000 ║ 1 │ 0.038027 ║
18 ║ 2.027556 │ 1.53557783057 ║ 0.75735410592 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.015193 ║
19 ║ 2.027556 │ 1.04198868285 ║ 0.51391364968 ║ 0.000000 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 0.094309 ║
═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣
║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║
Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║
═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣
20 ║ 2.027556 │ 0.93149970489 ║ 0.45941997346 ║ 0.000000 │ - ║ S │ 3 │ 0.250000 ║ 1 │ 0.110416 ║
21 ║ 2.027556 │ 0.82149497454 ║ 0.40516513040 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.029324 ║
22 ║ 2.027556 │ 0.71807054560 ║ 0.35415572251 ║ 0.000000 │ - ║ S │ 2 │ 2.000000 ║ 1 │ 0.173107 ║
23 ║ 2.027556 │ 0.71364643723 ║ 0.35197373175 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.012969 ║
24 ║ 2.027556 │ 0.54336780853 ║ 0.26799152256 ║ 0.000000 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 0.049872 ║
25 ║ 2.027556 │ 0.52920946359 ║ 0.26100856119 ║ 0.000000 │ - ║ S │ 2 │ 0.500000 ║ 1 │ 0.090110 ║
26 ║ 2.027556 │ 0.38297864798 ║ 0.18888684491 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.078443 ║
27 ║ 2.027556 │ 0.34625417862 ║ 0.17077416634 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.148051 ║
28 ║ 2.027556 │ 0.33665070462 ║ 0.16603768844 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.003684 ║
29 ║ 2.027556 │ 0.26789115070 ║ 0.13212515763 ║ 0.000000 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 0.089877 ║
30 ║ 2.027556 │ 0.25588826429 ║ 0.12620527840 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.054336 ║
31 ║ 2.027556 │ 0.19518453283 ║ 0.09626591659 ║ 0.000000 │ - ║ S │ 2 │ 2.000000 ║ 1 │ 0.028392 ║
32 ║ 2.027556 │ 0.19483151740 ║ 0.09609180774 ║ 0.000000 │ - ║ S │ 5 │ 0.062500 ║ 1 │ 0.026507 ║
33 ║ 2.027556 │ 0.17882219593 ║ 0.08819593616 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 7.43e-05 ║
34 ║ 2.027556 │ 0.17424853428 ║ 0.08592330867 ║ 3.42e-05 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 4.88e-05 ║
35 ║ 1.077526 │ 0.09246135145 ║ 0.08580889928 ║ 0.000000 │ - ║ S │ 3 │ 0.250000 ║ 2 │ 6.04e-18 ║
═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣
Optimization results: ║
F = final iterate, B = Best (to tolerance), MF = Most Feasible ║
═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣
F ║ │ ║ 0.08580889928 ║ 0.000000 │ - ║ │ │ ║ │ ║
B ║ │ ║ 0.08580889928 ║ 0.000000 │ - ║ │ │ ║ │ ║
MF ║ │ ║ 0.08580889928 ║ 0.000000 │ - ║ │ │ ║ │ ║
═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣
Iterations: 35 ║
Function evaluations: 77 ║
PyGRANSO termination code: 0 --- converged to stationarity and feasibility tolerances. ║
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝
[7]:
soln.final.x
[7]:
tensor([[0.7071],
[0.5000]], dtype=torch.float64)
Results Logs¶
(Optional) opts below shows the importance of using an initial point that is neither near nor on a nonsmooth manifold, that is, the functions (objective and constraints) should be smooth at and about the initial point.
[8]:
opts = pygransoStruct()
opts.torch_device = device
# Set a randomly generated starting point. In theory, with probability
# one, a randomly selected point will not be on a nonsmooth manifold.
opts.x0 = torch.randn((2,1), device=device, dtype=torch.double) # randomly generated is okay
opts.maxit = 100 # we'll use this value of maxit later
opts.opt_tol = 1e-6
# However, (0,0) or (1,1) are on the nonsmooth manifold and if PyGRANSO
# is started at either of them, it will break down on the first
# iteration. This example highlights that it is imperative to start
# PyGRANSO at a point where the functions are smooth.
# Uncomment either of the following two lines to try starting PyGRANSO
# from (0,0) or (1,1), where the functions are not differentiable.
# opts.x0 = torch.ones((2,1), device=device, dtype=torch.double) # uncomment this line to try this point
# opts.x0 = torch.zeros((2,1), device=device, dtype=torch.double) # uncomment this line to try this point
# Uncomment the following two lines to try starting PyGRANSO from a
# uniformly perturbed version of (1,1). pert_level needs to be at
# least 1e-3 or so to get consistently reliable optimization quality.
# pert_level = 1e-3
# opts.x0 = (torch.ones((2,1)) + pert_level * (torch.randn((2,1)) - 0.5)).to(device=device, dtype=torch.double)
The opts below show how to use opts.halt_log_fn to create a history of iterates
NOTE: NO NEED TO CHANGE ANYTHING BELOW
[9]:
# SETUP THE LOGGING FEATURES
# Set up PyGRANSO's logging functions; pass opts.maxit to it so that
# storage can be preallocated for efficiency.
class HaltLog:
def __init__(self):
pass
def haltLog(self, iteration, x, penaltyfn_parts, d,get_BFGS_state_fn, H_regularized,
ls_evals, alpha, n_gradients, stat_vec, stat_val, fallback_level):
# DON'T CHANGE THIS
# increment the index/count
self.index += 1
# EXAMPLE:
# store history of x iterates in a preallocated cell array
self.x_iterates.append(x)
self.f.append(penaltyfn_parts.f)
self.tv.append(penaltyfn_parts.tv)
# keep this false unless you want to implement a custom termination
# condition
halt = False
return halt
# Once PyGRANSO has run, you may call this function to get retreive all
# the logging data stored in the shared variables, which is populated
# by haltLog being called on every iteration of PyGRANSO.
def getLog(self):
# EXAMPLE
# return x_iterates, trimmed to correct size
log = pygransoStruct()
log.x = self.x_iterates[0:self.index]
log.f = self.f[0:self.index]
log.tv = self.tv[0:self.index]
return log
def makeHaltLogFunctions(self,maxit):
# don't change these lambda functions
halt_log_fn = lambda iteration, x, penaltyfn_parts, d,get_BFGS_state_fn, H_regularized, ls_evals, alpha, n_gradients, stat_vec, stat_val, fallback_level: self.haltLog(iteration, x, penaltyfn_parts, d,get_BFGS_state_fn, H_regularized, ls_evals, alpha, n_gradients, stat_vec, stat_val, fallback_level)
get_log_fn = lambda : self.getLog()
# Make your shared variables here to store PyGRANSO history data
# EXAMPLE - store history of iterates x_0,x_1,...,x_k
self.index = 0
self.x_iterates = []
self.f = []
self.tv = []
# Only modify the body of logIterate(), not its name or arguments.
# Store whatever data you wish from the current PyGRANSO iteration info,
# given by the input arguments, into shared variables of
# makeHaltLogFunctions, so that this data can be retrieved after PyGRANSO
# has been terminated.
#
# DESCRIPTION OF INPUT ARGUMENTS
# iter current iteration number
# x current iterate x
# penaltyfn_parts struct containing the following
# OBJECTIVE AND CONSTRAINTS VALUES
# .f objective value at x
# .f_grad objective gradient at x
# .ci inequality constraint at x
# .ci_grad inequality gradient at x
# .ce equality constraint at x
# .ce_grad equality gradient at x
# TOTAL VIOLATION VALUES (inf norm, for determining feasibiliy)
# .tvi total violation of inequality constraints at x
# .tve total violation of equality constraints at x
# .tv total violation of all constraints at x
# TOTAL VIOLATION VALUES (one norm, for L1 penalty function)
# .tvi_l1 total violation of inequality constraints at x
# .tvi_l1_grad its gradient
# .tve_l1 total violation of equality constraints at x
# .tve_l1_grad its gradient
# .tv_l1 total violation of all constraints at x
# .tv_l1_grad its gradient
# PENALTY FUNCTION VALUES
# .p penalty function value at x
# .p_grad penalty function gradient at x
# .mu current value of the penalty parameter
# .feasible_to_tol logical indicating whether x is feasible
# d search direction
# get_BFGS_state_fn function handle to get the (L)BFGS state data
# FULL MEMORY:
# - returns BFGS inverse Hessian approximation
# LIMITED MEMORY:
# - returns a struct with current L-BFGS state:
# .S matrix of the BFGS s vectors
# .Y matrix of the BFGS y vectors
# .rho row vector of the 1/sty values
# .gamma H0 scaling factor
# H_regularized regularized version of H
# [] if no regularization was applied to H
# fn_evals number of function evaluations incurred during
# this iteration
# alpha size of accepted size
# n_gradients number of previous gradients used for computing
# the termination QP
# stat_vec stationarity measure vector
# stat_val approximate value of stationarity:
# norm(stat_vec)
# gradients (result of termination QP)
# fallback_level number of strategy needed for a successful step
# to be taken. See bfgssqpOptionsAdvanced.
#
# OUTPUT ARGUMENT
# halt set this to true if you wish optimization to
# be halted at the current iterate. This can be
# used to create a custom termination condition,
return [halt_log_fn, get_log_fn]
mHLF_obj = HaltLog()
[halt_log_fn, get_log_fn] = mHLF_obj.makeHaltLogFunctions(opts.maxit)
# Set PyGRANSO's logging function in opts
opts.halt_log_fn = halt_log_fn
# Main algorithm with logging enabled.
soln = pygranso(var_spec = var_in,combined_fn = comb_fn, user_opts = opts)
# GET THE HISTORY OF ITERATES
# Even if an error is thrown, the log generated until the error can be
# obtained by calling get_log_fn()
log = get_log_fn()
╔═════ QP SOLVER NOTICE ════════════════════════════════════════════════════════════════════════╗
║ PyGRANSO requires a quadratic program (QP) solver that has a quadprog-compatible interface, ║
║ the default is osqp. Users may provide their own wrapper for the QP solver. ║
║ To disable this notice, set opts.quadprog_info_msg = False ║
╚═══════════════════════════════════════════════════════════════════════════════════════════════╝
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation ║
Version 1.2.0 ║
Licensed under the AGPLv3, Copyright (C) 2021-2022 Tim Mitchell and Buyun Liang ║
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
Problem specifications: ║
# of variables : 2 ║
# of inequality constraints : 2 ║
# of equality constraints : 0 ║
═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣
║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║
Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║
═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣
0 ║ 1.000000 │ 16.4067374414 ║ 16.4067374414 ║ 0.000000 │ - ║ - │ 1 │ 0.000000 ║ 1 │ 16.89371 ║
1 ║ 1.000000 │ 12.1088353467 ║ 11.5118588213 ║ 0.596977 │ - ║ S │ 4 │ 0.125000 ║ 1 │ 19.28340 ║
2 ║ 1.000000 │ 1.11708224486 ║ 1.11708224486 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.478277 ║
3 ║ 1.000000 │ 0.55748343428 ║ 0.55748343428 ║ 0.000000 │ - ║ S │ 4 │ 0.125000 ║ 1 │ 0.825482 ║
4 ║ 1.000000 │ 0.54324826834 ║ 0.54324826834 ║ 0.000000 │ - ║ S │ 3 │ 0.250000 ║ 1 │ 0.072797 ║
5 ║ 1.000000 │ 0.43272406722 ║ 0.43272406722 ║ 0.000000 │ - ║ S │ 2 │ 2.000000 ║ 1 │ 0.029511 ║
6 ║ 1.000000 │ 0.39143728403 ║ 0.39143728403 ║ 0.000000 │ - ║ S │ 2 │ 2.000000 ║ 1 │ 0.165019 ║
7 ║ 1.000000 │ 0.33585127390 ║ 0.33585127390 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.111788 ║
8 ║ 1.000000 │ 0.27769332848 ║ 0.27769332848 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.016351 ║
9 ║ 1.000000 │ 0.27206257507 ║ 0.27206257507 ║ 0.000000 │ - ║ S │ 6 │ 0.093750 ║ 1 │ 0.185187 ║
10 ║ 1.000000 │ 0.25369049840 ║ 0.25369049840 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.018059 ║
11 ║ 1.000000 │ 0.24095795441 ║ 0.24095795441 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.002996 ║
12 ║ 1.000000 │ 0.22956867248 ║ 0.22956867248 ║ 0.000000 │ - ║ S │ 2 │ 2.000000 ║ 1 │ 0.014434 ║
13 ║ 1.000000 │ 0.16940732170 ║ 0.16940732170 ║ 0.000000 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 0.080774 ║
14 ║ 1.000000 │ 0.14128118763 ║ 0.14128118763 ║ 0.000000 │ - ║ S │ 2 │ 0.500000 ║ 1 │ 0.100849 ║
15 ║ 1.000000 │ 0.13056475076 ║ 0.13056475076 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.005391 ║
16 ║ 1.000000 │ 0.12030008103 ║ 0.12030008103 ║ 0.000000 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 0.035053 ║
17 ║ 1.000000 │ 0.09388862233 ║ 0.09388862233 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.001793 ║
18 ║ 1.000000 │ 0.08668315753 ║ 0.08668315753 ║ 0.000000 │ - ║ S │ 2 │ 0.500000 ║ 1 │ 0.001835 ║
19 ║ 1.000000 │ 0.08658279210 ║ 0.08658279210 ║ 0.000000 │ - ║ S │ 4 │ 0.125000 ║ 1 │ 0.001426 ║
═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣
║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║
Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║
═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣
20 ║ 1.000000 │ 0.08597722224 ║ 0.08597722224 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 3.76e-05 ║
21 ║ 0.900000 │ 0.07729405862 ║ 0.08588228736 ║ 0.000000 │ - ║ S │ 2 │ 0.500000 ║ 2 │ 1.39e-05 ║
22 ║ 0.900000 │ 0.07725449555 ║ 0.08583832839 ║ 0.000000 │ - ║ S │ 5 │ 0.062500 ║ 1 │ 1.42e-04 ║
23 ║ 0.900000 │ 0.07725013449 ║ 0.08583348277 ║ 0.000000 │ - ║ S │ 2 │ 0.500000 ║ 2 │ 5.96e-05 ║
24 ║ 0.900000 │ 0.07723808110 ║ 0.08580155461 ║ 9.62e-06 │ - ║ S │ 4 │ 1.250000 ║ 2 │ 7.31e-06 ║
25 ║ 0.900000 │ 0.07721211695 ║ 0.08579124105 ║ 6.63e-13 │ - ║ S │ 1 │ 1.000000 ║ 3 │ 4.06e-13 ║
═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣
Optimization results: ║
F = final iterate, B = Best (to tolerance), MF = Most Feasible ║
═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣
F ║ │ ║ 0.08579124105 ║ 6.63e-13 │ - ║ │ │ ║ │ ║
B ║ │ ║ 0.08579124105 ║ 6.63e-13 │ - ║ │ │ ║ │ ║
MF ║ │ ║ 0.08580793766 ║ 0.000000 │ - ║ │ │ ║ │ ║
═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣
Iterations: 25 ║
Function evaluations: 60 ║
PyGRANSO termination code: 0 --- converged to stationarity and feasibility tolerances. ║
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝
[10]:
print(log.f[0:3])
print(log.x[0:3])
[16.406737441369938, 11.51185882133243, 1.117082244864862]
[tensor([[-0.8448],
[-0.9117]], dtype=torch.float64), tensor([[ 1.1292],
[-0.1617]], dtype=torch.float64), tensor([[0.3125],
[0.0171]], dtype=torch.float64)]
LFBGS Restarting¶
(Optional)
(Note that this example problem only has two variables!)
If PyGRANSO runs in limited-memory mode, that is, if opts.limited_mem_size > 0, then PyGRANSO’s restart procedure is slightly different from the BFGS restarting, as soln.H_final will instead contain the most current L-BFGS state, not a full inverse Hessian approximation.
Instead the BFGS standard procedure, users should do the following: 1) If you set a specific H0, you will need to set opts.H0 to whatever you used previously. By default, PyGRANSO uses the identity for H0.
Warm-start PyGRANSO with the most recent L-BFGS data by setting: opts.limited_mem_warm_start = soln.H_final;
NOTE: how to set opts.scaleH0 so that PyGRANSO will be restarted as if it had never terminated depends on the previously used values of opts.scaleH0 and opts.limited_mem_fixed_scaling.
[11]:
opts = pygransoStruct()
opts.torch_device = device
# set an infeasible initial point
opts.x0 = 5.5*torch.ones((2,1), device=device, dtype=torch.double)
opts.print_ascii = True
opts.quadprog_info_msg = False
opts.maxit = 10 # default is 1000
opts.mu0 = 100 # default is 1
opts.print_frequency = 2
# By default, PyGRANSO uses full-memory BFGS updating. For nonsmooth
# problems, full-memory BFGS is generally recommended. However, if
# this is not feasible, one may optionally enable limited-memory BFGS
# updating by setting opts.limited_mem_size to a positive integer
# (significantly) less than the number of variables.
opts.limited_mem_size = 1
# start main algorithm
soln = pygranso(var_spec = var_in,combined_fn = comb_fn, user_opts = opts)
==================================================================================================================
PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation |
Version 1.2.0 |
Licensed under the AGPLv3, Copyright (C) 2021-2022 Tim Mitchell and Buyun Liang |
==================================================================================================================
Problem specifications: |
# of variables : 2 |
# of inequality constraints : 2 |
# of equality constraints : 0 |
==================================================================================================================
Limited-memory mode enabled with size = 1. |
NOTE: limited-memory mode is generally NOT |
recommended for nonsmooth problems. |
==================================================================================================================
| <--- Penalty Function --> | | Total Violation | <--- Line Search ---> | <- Stationarity -> |
Iter | Mu | Value | Objective | Ineq | Eq | SD | Evals | t | Grads | Value |
=====|===========================|================|=================|=======================|====================|
0 | 100.0000 | 21841.7781746 | 218.250000000 | 10.00000 | - | - | 1 | 0.000000 | 1 | 9732.770 |
2 | 34.86784 | 1378.57842221 | 39.2815768212 | 8.914529 | - | S | 3 | 1.500000 | 1 | 4.455384 |
4 | 12.15767 | 262.610250639 | 20.8774186596 | 8.789579 | - | S | 2 | 2.000000 | 1 | 0.604009 |
6 | 4.239116 | 57.9458175917 | 11.5708792009 | 8.895520 | - | S | 3 | 0.750000 | 1 | 0.165224 |
8 | 1.642320 | 26.2056730861 | 10.5532019880 | 8.873935 | - | S | 1 | 1.000000 | 1 | 0.027766 |
10 | 1.642320 | 25.9163909350 | 10.4174724535 | 8.807564 | - | S | 2 | 2.000000 | 1 | 0.021455 |
==================================================================================================================
Optimization results: |
F = final iterate, B = Best (to tolerance), MF = Most Feasible |
==================================================================================================================
F | | | 10.4174724535 | 8.807564 | - | | | | | |
MF | | | 75.6886238113 | 8.192103 | - | | | | | |
==================================================================================================================
Iterations: 10 |
Function evaluations: 29 |
PyGRANSO termination code: 4 --- max iterations reached. |
==================================================================================================================
[12]:
# Restart
opts = pygransoStruct()
opts.torch_device = device
# set the initial point and penalty parameter to their final values from the previous run
opts.x0 = soln.final.x
opts.mu0 = soln.final.mu
opts.limited_mem_size = 1
opts.quadprog_info_msg = False
opts.print_frequency = 2
opts.limited_mem_warm_start = soln.H_final
opts.scaleH0 = False
# In contrast to full-memory BFGS updating, limited-memory BFGS
# permits that H0 can be scaled on every iteration. By default,
# PyGRANSO will reuse the scaling parameter that is calculated on the
# very first iteration for all subsequent iterations as well. Set
# this option to false to force PyGRANSO to calculate a new scaling
# parameter on every iteration. Note that opts.scaleH0 has no effect
# when opts.limited_mem_fixed_scaling is set to true.
opts.limited_mem_fixed_scaling = False
# Restart PyGRANSO
opts.maxit = 100 # increase maximum allowed iterations
# Main algorithm
soln = pygranso(var_spec = var_in,combined_fn = comb_fn, user_opts = opts)
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation ║
Version 1.2.0 ║
Licensed under the AGPLv3, Copyright (C) 2021-2022 Tim Mitchell and Buyun Liang ║
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
Problem specifications: ║
# of variables : 2 ║
# of inequality constraints : 2 ║
# of equality constraints : 0 ║
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
Limited-memory mode enabled with size = 1. ║
NOTE: limited-memory mode is generally NOT ║
recommended for nonsmooth problems. ║
═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣
║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║
Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║
═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣
0 ║ 1.642320 │ 25.9163909350 ║ 10.4174724535 ║ 8.807564 │ - ║ - │ 1 │ 0.000000 ║ 1 │ 0.142170 ║
2 ║ 1.642320 │ 13.8167164344 ║ 6.50516654793 ║ 3.133149 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 3.838176 ║
4 ║ 1.642320 │ 6.43976368442 ║ 3.92113741712 ║ 0.000000 │ - ║ S │ 4 │ 8.000000 ║ 1 │ 3.895256 ║
6 ║ 1.642320 │ 4.81165411491 ║ 2.92979027070 ║ 0.000000 │ - ║ S │ 3 │ 0.250000 ║ 1 │ 0.052493 ║
8 ║ 1.642320 │ 4.63155845135 ║ 2.82013099132 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.017665 ║
10 ║ 1.642320 │ 4.03639569243 ║ 2.45773959349 ║ 0.000000 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 0.013749 ║
12 ║ 1.642320 │ 3.00639292821 ║ 1.83057645887 ║ 0.000000 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 0.028975 ║
14 ║ 1.642320 │ 2.28891992127 ║ 1.39371100989 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.048897 ║
16 ║ 1.642320 │ 1.95780587454 ║ 1.19209745051 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.265478 ║
18 ║ 1.642320 │ 1.49471266813 ║ 0.91012249177 ║ 0.000000 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 0.084726 ║
20 ║ 1.642320 │ 1.34989358124 ║ 0.82194292989 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.216509 ║
22 ║ 1.642320 │ 1.13057014213 ║ 0.68839806928 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.018354 ║
24 ║ 1.642320 │ 1.00727210829 ║ 0.61332256067 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.009416 ║
26 ║ 1.642320 │ 0.91876697706 ║ 0.55943226303 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.012977 ║
28 ║ 1.642320 │ 0.79484835473 ║ 0.48397888143 ║ 0.000000 │ - ║ S │ 3 │ 0.250000 ║ 1 │ 0.096405 ║
30 ║ 1.642320 │ 0.65714700682 ║ 0.40013327247 ║ 0.000000 │ - ║ S │ 2 │ 0.500000 ║ 1 │ 0.142885 ║
32 ║ 1.642320 │ 0.57064376781 ║ 0.34746191622 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.005519 ║
34 ║ 1.642320 │ 0.50072121946 ║ 0.30488645320 ║ 0.000000 │ - ║ S │ 3 │ 4.000000 ║ 1 │ 0.002562 ║
36 ║ 1.642320 │ 0.46398321205 ║ 0.28251687839 ║ 0.000000 │ - ║ S │ 2 │ 0.500000 ║ 1 │ 0.010241 ║
38 ║ 1.642320 │ 0.38322497032 ║ 0.23334362003 ║ 0.000000 │ - ║ S │ 3 │ 0.250000 ║ 1 │ 0.145492 ║
═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣
║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║
Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║
═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣
40 ║ 1.642320 │ 0.31487530037 ║ 0.19172587420 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.006704 ║
42 ║ 1.642320 │ 0.30050834308 ║ 0.18297791129 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.007435 ║
44 ║ 1.642320 │ 0.26424400929 ║ 0.16089675380 ║ 0.000000 │ - ║ S │ 2 │ 0.500000 ║ 1 │ 0.795419 ║
46 ║ 1.642320 │ 0.20338505133 ║ 0.12384006214 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.004990 ║
48 ║ 1.642320 │ 0.17520417328 ║ 0.10668087731 ║ 0.000000 │ - ║ S │ 4 │ 0.125000 ║ 1 │ 0.003565 ║
50 ║ 1.077526 │ 0.10049441692 ║ 0.08991368997 ║ 0.002780 │ - ║ S │ 6 │ 0.031250 ║ 1 │ 0.033750 ║
52 ║ 1.077526 │ 0.09397044638 ║ 0.08720941715 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 2.31e-04 ║
54 ║ 1.077526 │ 0.09254093483 ║ 0.08588275676 ║ 0.000000 │ - ║ S │ 5 │ 0.062500 ║ 2 │ 2.09e-04 ║
56 ║ 1.077526 │ 0.09244295357 ║ 0.08579182510 ║ 0.000000 │ - ║ S │ 2 │ 0.500000 ║ 2 │ 1.78e-06 ║
58 ║ 1.077526 │ 0.09243954005 ║ 0.08578865717 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 4 │ 6.93e-15 ║
═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣
Optimization results: ║
F = final iterate, B = Best (to tolerance), MF = Most Feasible ║
═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣
F ║ │ ║ 0.08578865717 ║ 0.000000 │ - ║ │ │ ║ │ ║
B ║ │ ║ 0.08578865717 ║ 0.000000 │ - ║ │ │ ║ │ ║
MF ║ │ ║ 0.08578865717 ║ 0.000000 │ - ║ │ │ ║ │ ║
═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣
Iterations: 58 ║
Function evaluations: 127 ║
PyGRANSO termination code: 0 --- converged to stationarity and feasibility tolerances. ║
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝