Sphere Manifold¶
Optimization on a sphere manifold, taken from this Manopt example
Problem Description¶
Rayleigh quotient problem:
\[\min_{x}-x^TAx,\]
\[\text{s.t. }x^Tx=1,\]
where \(A\in R^{n\times n}\) is a symmetric matrix.
Modules Importing¶
Import all necessary modules and add PyGRANSO src folder to system path.
[1]:
import time
import torch
from pygranso.pygranso import pygranso
from pygranso.pygransoStruct import pygransoStruct
Data Initialization¶
Specify torch device, and generate data
[2]:
device = torch.device('cpu')
torch.manual_seed(0)
n = 300
# All the user-provided data (vector/matrix/tensor) must be in torch tensor format.
# As PyTorch tensor is single precision by default, one must explicitly set `dtype=torch.double`.
# Also, please make sure the device of provided torch tensor is the same as opts.torch_device.
A = torch.randn((n,n)).to(device=device, dtype=torch.double)
A = .5*(A+A.T)
Function Set-Up¶
Encode the optimization variables, and objective and constraint functions.
Note: please strictly follow the format of comb_fn, which will be used in the PyGRANSO main algortihm.
[3]:
# variables and corresponding dimensions.
var_in = {"x": [n,1]}
def user_fn(X_struct,A):
x = X_struct.x
# objective function
f = -x.T@A@x
# inequality constraint, matrix form
ci = None
# equality constraint
ce = pygransoStruct()
ce.c1 = x.T@x-1
return [f,ci,ce]
comb_fn = lambda X_struct : user_fn(X_struct,A)
User Options¶
Specify user-defined options for PyGRANSO
[4]:
opts = pygransoStruct()
opts.torch_device = device
opts.print_frequency = 10
opts.x0 = torch.randn((n,1)).to(device=device, dtype=torch.double)
opts.mu0 = 0.1 # increase penalty contribution
opts.opt_tol = 1e-6
Main Algorithm¶
[5]:
start = time.time()
soln = pygranso(var_spec = var_in,combined_fn = comb_fn,user_opts = opts)
end = time.time()
print("Total Wall Time: {}s".format(end - start))
╔═════ QP SOLVER NOTICE ════════════════════════════════════════════════════════════════════════╗
║ PyGRANSO requires a quadratic program (QP) solver that has a quadprog-compatible interface, ║
║ the default is osqp. Users may provide their own wrapper for the QP solver. ║
║ To disable this notice, set opts.quadprog_info_msg = False ║
╚═══════════════════════════════════════════════════════════════════════════════════════════════╝
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation ║
Version 1.2.0 ║
Licensed under the AGPLv3, Copyright (C) 2021-2022 Tim Mitchell and Buyun Liang ║
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
Problem specifications: ║
# of variables : 300 ║
# of inequality constraints : 0 ║
# of equality constraints : 1 ║
═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣
║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║
Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║
═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣
0 ║ 0.100000 │ 233.598444885 ║ -777.138392664 ║ - │ 311.3123 ║ - │ 1 │ 0.000000 ║ 1 │ 45.87915 ║
10 ║ 0.038742 │ 4.73697237561 ║ -1135.54118646 ║ - │ 48.73016 ║ S │ 2 │ 2.000000 ║ 1 │ 0.703910 ║
20 ║ 0.038742 │ -0.90394750818 ║ -24.2316527975 ║ - │ 0.034836 ║ S │ 1 │ 1.000000 ║ 1 │ 0.063645 ║
30 ║ 0.038742 │ -0.94016470001 ║ -24.2720808397 ║ - │ 1.85e-04 ║ S │ 1 │ 1.000000 ║ 1 │ 0.005112 ║
40 ║ 0.038742 │ -0.94079615436 ║ -24.2856228608 ║ - │ 7.86e-05 ║ S │ 1 │ 1.000000 ║ 1 │ 0.004916 ║
50 ║ 0.038742 │ -0.94088141602 ║ -24.2858835595 ║ - │ 3.47e-06 ║ S │ 1 │ 1.000000 ║ 1 │ 8.26e-04 ║
60 ║ 0.038742 │ -0.94088588797 ║ -24.2859210930 ║ - │ 4.55e-07 ║ S │ 1 │ 1.000000 ║ 1 │ 3.31e-04 ║
70 ║ 0.038742 │ -0.94088637134 ║ -24.2859220156 ║ - │ 6.97e-09 ║ S │ 1 │ 1.000000 ║ 1 │ 2.97e-05 ║
80 ║ 0.038742 │ -0.94088637916 ║ -24.2859220393 ║ - │ 6.69e-11 ║ S │ 1 │ 1.000000 ║ 1 │ 3.02e-06 ║
═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣
Optimization results: ║
F = final iterate, B = Best (to tolerance), MF = Most Feasible ║
═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣
F ║ │ ║ -24.2859220401 ║ - │ 6.78e-12 ║ │ │ ║ │ ║
B ║ │ ║ -24.2859222959 ║ - │ 2.60e-07 ║ │ │ ║ │ ║
MF ║ │ ║ -24.2859220401 ║ - │ 6.78e-12 ║ │ │ ║ │ ║
═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣
Iterations: 85 ║
Function evaluations: 96 ║
PyGRANSO termination code: 0 --- converged to stationarity and feasibility tolerances. ║
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝
Total Wall Time: 0.6032183170318604s