Sphere Manifold

Optimization on a sphere manifold, taken from this Manopt example

Problem Description

Rayleigh quotient problem:

\[\min_{x}-x^TAx,\]
\[\text{s.t. }x^Tx=1,\]

where \(A\in R^{n\times n}\) is a symmetric matrix.

Modules Importing

Import all necessary modules and add PyGRANSO src folder to system path.

[1]:
import time
import torch
from pygranso.pygranso import pygranso
from pygranso.pygransoStruct import pygransoStruct

Data Initialization

Specify torch device, and generate data

[2]:
device = torch.device('cpu')
torch.manual_seed(0)
n = 300
# All the user-provided data (vector/matrix/tensor) must be in torch tensor format.
# As PyTorch tensor is single precision by default, one must explicitly set `dtype=torch.double`.
# Also, please make sure the device of provided torch tensor is the same as opts.torch_device.
A = torch.randn((n,n)).to(device=device, dtype=torch.double)
A = .5*(A+A.T)

Function Set-Up

Encode the optimization variables, and objective and constraint functions.

Note: please strictly follow the format of comb_fn, which will be used in the PyGRANSO main algortihm.

[3]:
# variables and corresponding dimensions.
var_in = {"x": [n,1]}


def user_fn(X_struct,A):
    x = X_struct.x

    # objective function
    f = -x.T@A@x

    # inequality constraint, matrix form
    ci = None

    # equality constraint
    ce = pygransoStruct()
    ce.c1 = x.T@x-1

    return [f,ci,ce]

comb_fn = lambda X_struct : user_fn(X_struct,A)

User Options

Specify user-defined options for PyGRANSO

[4]:
opts = pygransoStruct()
opts.torch_device = device
opts.print_frequency = 10
opts.x0 = torch.randn((n,1)).to(device=device, dtype=torch.double)
opts.mu0 = 0.1 # increase penalty contribution
opts.opt_tol = 1e-6

Main Algorithm

[5]:
start = time.time()
soln = pygranso(var_spec = var_in,combined_fn = comb_fn,user_opts = opts)
end = time.time()
print("Total Wall Time: {}s".format(end - start))


╔═════ QP SOLVER NOTICE ════════════════════════════════════════════════════════════════════════╗
║  PyGRANSO requires a quadratic program (QP) solver that has a quadprog-compatible interface,  ║
║  the default is osqp. Users may provide their own wrapper for the QP solver.                  ║
║  To disable this notice, set opts.quadprog_info_msg = False                                   ║
╚═══════════════════════════════════════════════════════════════════════════════════════════════╝
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation                                             ║
Version 1.0.0                                                                                                    ║
Licensed under the AGPLv3, Copyright (C) 2021 Tim Mitchell and Buyun Liang                                       ║
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
Problem specifications:                                                                                          ║
 # of variables                     :   300                                                                      ║
 # of inequality constraints        :     0                                                                      ║
 # of equality constraints          :     1                                                                      ║
═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣
     ║ <--- Penalty Function --> ║                ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║
Iter ║    Mu    │      Value     ║    Objective   ║ Ineq │    Eq    ║ SD │ Evals │     t    ║ Grads │    Value   ║
═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣
   0 ║ 0.100000 │  233.598444885 ║ -777.138392664 ║   -  │ 311.3123 ║ -  │     1 │ 0.000000 ║     1 │ 45.87915   ║
  10 ║ 0.038742 │  4.73657610523 ║ -1135.29853488 ║   -  │ 48.72037 ║ S  │     2 │ 2.000000 ║     1 │ 0.704227   ║
  20 ║ 0.038742 │ -0.90688325527 ║ -23.4396021953 ║   -  │ 0.001215 ║ S  │     5 │ 1.125000 ║     1 │ 0.071368   ║
  30 ║ 0.038742 │ -0.94016725008 ║ -24.2720353768 ║   -  │ 1.81e-04 ║ S  │     1 │ 1.000000 ║     1 │ 0.005056   ║
  40 ║ 0.038742 │ -0.94079722542 ║ -24.2856657523 ║   -  │ 7.92e-05 ║ S  │     1 │ 1.000000 ║     1 │ 0.004941   ║
  50 ║ 0.038742 │ -0.94088146445 ║ -24.2858822238 ║   -  │ 3.37e-06 ║ S  │     1 │ 1.000000 ║     1 │ 8.10e-04   ║
  60 ║ 0.038742 │ -0.94088589472 ║ -24.2859211796 ║   -  │ 4.51e-07 ║ S  │     1 │ 1.000000 ║     1 │ 3.30e-04   ║
  70 ║ 0.038742 │ -0.94088637155 ║ -24.2859220141 ║   -  │ 6.70e-09 ║ S  │     1 │ 1.000000 ║     1 │ 2.91e-05   ║
  80 ║ 0.038742 │ -0.94088637916 ║ -24.2859220393 ║   -  │ 6.54e-11 ║ S  │     1 │ 1.000000 ║     1 │ 2.99e-06   ║
═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣
F = final iterate, B = Best (to tolerance), MF = Most Feasible                                                   ║
Optimization results:                                                                                            ║
═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣
   F ║          │                ║ -24.2859220401 ║   -  │ 6.61e-12 ║    │       │          ║       │            ║
   B ║          │                ║ -24.2859223114 ║   -  │ 2.56e-07 ║    │       │          ║       │            ║
  MF ║          │                ║ -24.2859220401 ║   -  │ 6.61e-12 ║    │       │          ║       │            ║
═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣
Iterations:              85                                                                                      ║
Function evaluations:    100                                                                                     ║
PyGRANSO termination code: 0 --- converged to stationarity and feasibility tolerances.                           ║
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝
Total Wall Time: 0.5655074119567871s