{ "cells": [ { "cell_type": "markdown", "id": "5257bb27", "metadata": {}, "source": [ "# Eigenvalue Optimization\n", "\n", "Eigenvalue Optimization taken from: [GRANSO](http://www.timmitchell.com/software/GRANSO/) demo example 4." ] }, { "cell_type": "markdown", "id": "16ae1959", "metadata": {}, "source": [ "## Problem Description" ] }, { "cell_type": "markdown", "id": "72e3a77a", "metadata": {}, "source": [ "We have $M=A+BXC$,\n", "where the matrices $A\\in R^{N,N},B\\in R^{N,M}$ and $C\\in R^{P,N}$ are given, $X\\in R^{M,P}$ is the matrix form optimization variable.\n", "\n", "We have the nonconvex, nonsmooth, and constrained optimization problem\n", "$$\\min_{X}\\max| \\mathrm{Im} (\\Lambda(A+BXC))|,$$\n", "$$\\text{s.t. }\\alpha(A+BXC)+\\xi \\leq 0,$$\n", "where $\\mathrm{Im}(\\cdot)$ is the imaginary part of complex number, $\\xi$ is the stability margin, and $\\Lambda(\\cdot)$ is the spectrum of a square matrix $\\cdot$, and $\\alpha(\\cdot)$ is the spectral abscissa of a square matrix, i.e., the maximum real part of its eigenvalues." ] }, { "cell_type": "markdown", "id": "08dfdd50", "metadata": {}, "source": [ "## Modules Importing\n", "Import all necessary modules and add PyGRANSO src folder to system path." ] }, { "cell_type": "code", "execution_count": 1, "id": "90ed32f9", "metadata": {}, "outputs": [], "source": [ "import time\n", "import torch\n", "from pygranso.pygranso import pygranso\n", "from pygranso.pygransoStruct import pygransoStruct\n", "import scipy.io\n", "from torch import linalg as LA" ] }, { "cell_type": "markdown", "id": "17a1b7fe", "metadata": {}, "source": [ "## Data Initialization \n", "Specify torch device, and read the data from a provided file.\n", "\n", "Use GPU for this problem. If no cuda device available, please set *device = torch.device('cpu')*" ] }, { "cell_type": "code", "execution_count": 2, "id": "8b4842e1", "metadata": {}, "outputs": [], "source": [ "device = torch.device('cuda')\n", "\n", "file = \"/home/buyun/Documents/GitHub/PyGRANSO/examples/data/spec_radius_opt_data.mat\"\n", "mat = scipy.io.loadmat(file)\n", "mat_struct = mat['sys']\n", "mat_struct = mat_struct[0,0]\n", "# All the user-provided data (vector/matrix/tensor) must be in torch tensor format. \n", "# As PyTorch tensor is single precision by default, one must explicitly set `dtype=torch.double`.\n", "# Also, please make sure the device of provided torch tensor is the same as opts.torch_device.\n", "A = torch.from_numpy(mat_struct['A']).to(device=device, dtype=torch.double)\n", "B = torch.from_numpy(mat_struct['B']).to(device=device, dtype=torch.double)\n", "C = torch.from_numpy(mat_struct['C']).to(device=device, dtype=torch.double)\n", "p = B.shape[1]\n", "m = C.shape[0]\n", "stability_margin = 1" ] }, { "cell_type": "markdown", "id": "ec80716b", "metadata": {}, "source": [ "## Function Set-Up\n", "\n", "Encode the optimization variables, and objective and constraint functions.\n", "\n", "Note: please strictly follow the format of comb_fn, which will be used in the PyGRANSO main algortihm." ] }, { "cell_type": "code", "execution_count": 3, "id": "fb360e75", "metadata": {}, "outputs": [], "source": [ "# variables and corresponding dimensions.\n", "var_in = {\"X\": [p,m] }\n", "\n", "def user_fn(X_struct,A,B,C,stability_margin):\n", " # user defined variable, matirx form. torch tensor\n", " X = X_struct.X\n", "\n", " # objective function\n", " M = A + B@X@C\n", " [D,_] = LA.eig(M)\n", " f = torch.max(D.imag)\n", "\n", " # inequality constraint, matrix form\n", " ci = pygransoStruct()\n", " ci.c1 = torch.max(D.real) + stability_margin\n", "\n", " # equality constraint \n", " ce = None\n", " \n", " return [f,ci,ce]\n", "\n", "comb_fn = lambda X_struct : user_fn(X_struct,A,B,C,stability_margin)" ] }, { "cell_type": "markdown", "id": "f0f55ace", "metadata": {}, "source": [ "## User Options\n", "Specify user-defined options for PyGRANSO" ] }, { "cell_type": "code", "execution_count": 4, "id": "f3a65b57", "metadata": {}, "outputs": [], "source": [ "opts = pygransoStruct()\n", "opts.torch_device = device\n", "opts.maxit = 200\n", "opts.x0 = torch.zeros(p*m,1).to(device=device, dtype=torch.double)\n", "# print for every 10 iterations. default: 1\n", "opts.print_frequency = 10" ] }, { "cell_type": "markdown", "id": "8bca18c7", "metadata": {}, "source": [ "## Main Algorithm" ] }, { "cell_type": "code", "execution_count": 5, "id": "632976b3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[33m╔═════ QP SOLVER NOTICE ════════════════════════════════════════════════════════════════════════╗\n", "\u001b[0m\u001b[33m║ PyGRANSO requires a quadratic program (QP) solver that has a quadprog-compatible interface, ║\n", "\u001b[0m\u001b[33m║ the default is osqp. Users may provide their own wrapper for the QP solver. ║\n", "\u001b[0m\u001b[33m║ To disable this notice, set opts.quadprog_info_msg = False ║\n", "\u001b[0m\u001b[33m╚═══════════════════════════════════════════════════════════════════════════════════════════════╝\n", "\u001b[0m═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗\n", "PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation ║ \n", "Version 1.2.0 ║ \n", "Licensed under the AGPLv3, Copyright (C) 2021-2022 Tim Mitchell and Buyun Liang ║ \n", "═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣\n", "Problem specifications: ║ \n", " # of variables : 200 ║ \n", " # of inequality constraints : 1 ║ \n", " # of equality constraints : 0 ║ \n", "═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣\n", " ║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║ \n", "Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║ \n", "═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣\n", " 0 ║ 1.000000 │ 16.2063030241 ║ 13.7635444107 ║ 2.442759 │ - ║ - │ 1 │ 0.000000 ║ 1 │ 28.28938 ║ \n", " 10 ║ 1.000000 │ 14.3596190745 ║ 12.9268516490 ║ 1.432767 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.035504 ║ \n", " 20 ║ 1.000000 │ 13.7030952290 ║ 12.6546385386 ║ 1.048457 │ - ║ S │ 2 │ 0.500000 ║ 1 │ 0.039570 ║ \n", " 30 ║ 1.000000 │ 12.8620021270 ║ 12.2684505963 ║ 0.593552 │ - ║ S │ 4 │ 0.125000 ║ 1 │ 0.048312 ║ \n", " 40 ║ 1.000000 │ 12.6579998061 ║ 12.1060352760 ║ 0.551965 │ - ║ S │ 8 │ 0.007812 ║ 1 │ 0.035025 ║ \n", " 50 ║ 0.900000 │ 11.1959291986 ║ 11.9651110834 ║ 0.427329 │ - ║ S │ 7 │ 0.015625 ║ 1 │ 0.019151 ║ \n", " 60 ║ 0.900000 │ 11.0023393275 ║ 11.8890675575 ║ 0.302179 │ - ║ S │ 4 │ 0.125000 ║ 1 │ 0.024557 ║ \n", " 70 ║ 0.900000 │ 10.7433220769 ║ 11.7789577618 ║ 0.142260 │ - ║ S │ 5 │ 0.062500 ║ 1 │ 0.025465 ║ \n", " 80 ║ 0.590490 │ 7.01603991319 ║ 11.7158332169 ║ 0.097958 │ - ║ S │ 3 │ 0.250000 ║ 1 │ 0.013271 ║ \n", " 90 ║ 0.590490 │ 6.92114034141 ║ 11.6658843316 ║ 0.032552 │ - ║ S │ 8 │ 0.007812 ║ 1 │ 0.086947 ║ \n", " 100 ║ 0.590490 │ 6.87608100731 ║ 11.6215913630 ║ 0.013648 │ - ║ S │ 5 │ 0.062500 ║ 1 │ 0.005812 ║ \n", " 110 ║ 0.590490 │ 6.83585456951 ║ 11.5622624393 ║ 0.008454 │ - ║ S │ 4 │ 0.125000 ║ 1 │ 0.039093 ║ \n", " 120 ║ 0.590490 │ 6.78145717776 ║ 11.4799718129 ║ 0.002649 │ - ║ S │ 12 │ 4.88e-04 ║ 1 │ 8.554103 ║ \n", " 130 ║ 0.590490 │ 6.75685012537 ║ 11.4427850182 ║ 0.000000 │ - ║ S │ 6 │ 0.031250 ║ 1 │ 0.008980 ║ \n", " 140 ║ 0.282430 │ 3.22489665521 ║ 11.4062621765 ║ 0.003431 │ - ║ S │ 9 │ 0.003906 ║ 2 │ 0.003855 ║ \n", " 150 ║ 0.282430 │ 3.20049021475 ║ 11.3319954231 ║ 0.000000 │ - ║ S │ 7 │ 0.140625 ║ 1 │ 0.097628 ║ \n", " 160 ║ 0.282430 │ 3.18442979945 ║ 11.2751302117 ║ 0.000000 │ - ║ S │ 7 │ 0.015625 ║ 2 │ 0.001811 ║ \n", " 170 ║ 0.282430 │ 3.17160540367 ║ 11.2297227945 ║ 0.000000 │ - ║ S │ 5 │ 0.062500 ║ 1 │ 0.012334 ║ \n", " 180 ║ 0.282430 │ 3.15443814807 ║ 11.1689385869 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.003354 ║ \n", " 190 ║ 0.282430 │ 3.14459744559 ║ 11.1340955510 ║ 0.000000 │ - ║ S │ 7 │ 0.015625 ║ 1 │ 0.019961 ║ \n", "═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣\n", " ║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║ \n", "Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║ \n", "═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣\n", " 200 ║ 0.282430 │ 3.13379717527 ║ 11.0958549673 ║ 0.000000 │ - ║ S │ 3 │ 0.250000 ║ 1 │ 0.003635 ║ \n", "═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣\n", "F = final iterate, B = Best (to tolerance), MF = Most Feasible ║ \n", "Optimization results: ║ \n", "═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣\n", " F ║ │ ║ 11.0958549673 ║ 0.000000 │ - ║ │ │ ║ │ ║ \n", " B ║ │ ║ 11.0958549673 ║ 0.000000 │ - ║ │ │ ║ │ ║ \n", " MF ║ │ ║ 11.0958549673 ║ 0.000000 │ - ║ │ │ ║ │ ║ \n", "═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣\n", "Iterations: 200 ║ \n", "Function evaluations: 977 ║ \n", "PyGRANSO termination code: 4 --- max iterations reached. ║ \n", "═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝\n", "Total Wall Time: 73.09481072425842s\n" ] } ], "source": [ "start = time.time()\n", "soln = pygranso(var_spec = var_in,combined_fn = comb_fn,user_opts = opts)\n", "end = time.time()\n", "print(\"Total Wall Time: {}s\".format(end - start))" ] }, { "cell_type": "markdown", "id": "2eec4c44", "metadata": {}, "source": [ "## LBFGS \n", "(Optional) LBFGS and feasibility related options" ] }, { "cell_type": "code", "execution_count": 6, "id": "f97cf86d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[33m╔═════ QP SOLVER NOTICE ════════════════════════════════════════════════════════════════════════╗\n", "\u001b[0m\u001b[33m║ PyGRANSO requires a quadratic program (QP) solver that has a quadprog-compatible interface, ║\n", "\u001b[0m\u001b[33m║ the default is osqp. Users may provide their own wrapper for the QP solver. ║\n", "\u001b[0m\u001b[33m║ To disable this notice, set opts.quadprog_info_msg = False ║\n", "\u001b[0m\u001b[33m╚═══════════════════════════════════════════════════════════════════════════════════════════════╝\n", "\u001b[0m═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗\n", "PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation ║ \n", "Version 1.2.0 ║ \n", "Licensed under the AGPLv3, Copyright (C) 2021-2022 Tim Mitchell and Buyun Liang ║ \n", "═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣\n", "Problem specifications: ║ \n", " # of variables : 200 ║ \n", " # of inequality constraints : 1 ║ \n", " # of equality constraints : 0 ║ \n", "═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣\n", "\u001b[33mLimited-memory mode enabled with size = 40. \u001b[0m ║ \n", "\u001b[33mNOTE: limited-memory mode is generally NOT \u001b[0m ║ \n", "\u001b[33mrecommended for nonsmooth problems. \u001b[0m ║ \n", "═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣\n", " ║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║ \n", "Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║ \n", "═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣\n", " 0 ║ 1.000000 │ 16.2063030241 ║ 13.7635444107 ║ 2.442759 │ - ║ - │ 1 │ 0.000000 ║ 1 │ 28.28938 ║ \n", " 10 ║ 1.000000 │ 14.3596190745 ║ 12.9268516490 ║ 1.432767 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.035504 ║ \n", " 20 ║ 1.000000 │ 13.7030952290 ║ 12.6546385386 ║ 1.048457 │ - ║ S │ 2 │ 0.500000 ║ 1 │ 0.039570 ║ \n", " 30 ║ 1.000000 │ 12.8620021270 ║ 12.2684505963 ║ 0.593552 │ - ║ S │ 4 │ 0.125000 ║ 1 │ 0.048312 ║ \n", " 40 ║ 1.000000 │ 12.6579998061 ║ 12.1060352760 ║ 0.551965 │ - ║ S │ 8 │ 0.007812 ║ 1 │ 0.035025 ║ \n", " 50 ║ 1.000000 │ 12.3635830043 ║ 11.9482819495 ║ 0.415301 │ - ║ S │ 3 │ 0.250000 ║ 1 │ 0.024637 ║ \n", " 60 ║ 1.000000 │ 12.1866328732 ║ 11.8573351538 ║ 0.329298 │ - ║ S │ 6 │ 0.031250 ║ 1 │ 0.049341 ║ \n", " 70 ║ 1.000000 │ 12.0726567906 ║ 11.8152322843 ║ 0.257425 │ - ║ S │ 9 │ 0.003906 ║ 1 │ 0.087016 ║ \n", " 80 ║ 0.900000 │ 10.8099812281 ║ 11.7322840024 ║ 0.250926 │ - ║ S │ 6 │ 0.031250 ║ 1 │ 0.053814 ║ \n", " 90 ║ 0.900000 │ 10.7423772961 ║ 11.6888546812 ║ 0.222408 │ - ║ S │ 10 │ 0.001953 ║ 1 │ 1.097996 ║ \n", " 100 ║ 0.900000 │ 10.6934954335 ║ 11.6498556823 ║ 0.208625 │ - ║ S │ 8 │ 0.023438 ║ 1 │ 0.104976 ║ \n", " 110 ║ 0.900000 │ 10.6737493784 ║ 11.6419405074 ║ 0.196003 │ - ║ S │ 10 │ 0.001953 ║ 1 │ 0.486377 ║ \n", " 120 ║ 0.590490 │ 7.01684023305 ║ 11.6335924856 ║ 0.147320 │ - ║ S │ 15 │ 1.83e-04 ║ 1 │ 13.80324 ║ \n", " 130 ║ 0.590490 │ 6.99652751546 ║ 11.6311862406 ║ 0.128428 │ - ║ S │ 9 │ 0.003906 ║ 1 │ 0.166842 ║ \n", " 140 ║ 0.590490 │ 6.98605019503 ║ 11.6116595852 ║ 0.129481 │ - ║ S │ 14 │ 1.22e-04 ║ 2 │ 0.201047 ║ \n", " 150 ║ 0.590490 │ 6.96537265265 ║ 11.6137692506 ║ 0.107558 │ - ║ S │ 17 │ 1.07e-04 ║ 1 │ 390.6339 ║ \n", " 160 ║ 0.590490 │ 6.94088780302 ║ 11.6615791338 ║ 0.054842 │ - ║ S │ 12 │ 4.88e-04 ║ 1 │ 0.955100 ║ \n", " 170 ║ 0.590490 │ 6.92412727036 ║ 11.6075095895 ║ 0.070009 │ - ║ S │ 14 │ 1.22e-04 ║ 1 │ 0.127275 ║ \n", " 180 ║ 0.282430 │ 3.33270686212 ║ 11.5989449670 ║ 0.056822 │ - ║ S │ 11 │ 9.77e-04 ║ 1 │ 3.557972 ║ \n", " 190 ║ 0.282430 │ 3.32413687788 ║ 11.6020951504 ║ 0.047363 │ - ║ S │ 16 │ 3.05e-05 ║ 1 │ 3.739864 ║ \n", "═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣\n", " ║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║ \n", "Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║ \n", "═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣\n", " 200 ║ 0.098477 │ 1.14935778481 ║ 11.6713215456 ║ 0.000000 │ - ║ S │ 9 │ 0.003906 ║ 1 │ 11.56244 ║ \n", "═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣\n", "F = final iterate, B = Best (to tolerance), MF = Most Feasible ║ \n", "Optimization results: ║ \n", "═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣\n", " F ║ │ ║ 11.6713215456 ║ 0.000000 │ - ║ │ │ ║ │ ║ \n", " B ║ │ ║ 11.6713215456 ║ 0.000000 │ - ║ │ │ ║ │ ║ \n", " MF ║ │ ║ 11.6713215456 ║ 0.000000 │ - ║ │ │ ║ │ ║ \n", "═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣\n", "Iterations: 200 ║ \n", "Function evaluations: 1835 ║ \n", "PyGRANSO termination code: 4 --- max iterations reached. ║ \n", "═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝\n", "Total Wall Time: 138.43107748031616s\n" ] } ], "source": [ "opts = pygransoStruct()\n", "opts.torch_device = device\n", "opts.maxit = 200\n", "opts.x0 = torch.zeros(p*m,1).to(device=device, dtype=torch.double)\n", "# print for every 10 iterations. default: 1\n", "opts.print_frequency = 10\n", "\n", "# Limited-memory mode is generally not recommended for nonsmooth\n", "# problems, such as this one, but it can nonetheless enabled if\n", "# desired/necessary. opts.limited_mem_size == 0 is off, that is, \n", "# limited-memory mode is disabled.\n", "# Note that this example has 200 variables.\n", "opts.limited_mem_size = 40\n", "\n", "start = time.time()\n", "soln = pygranso(var_spec = var_in,combined_fn = comb_fn,user_opts = opts)\n", "end = time.time()\n", "print(\"Total Wall Time: {}s\".format(end - start))" ] }, { "cell_type": "code", "execution_count": 7, "id": "8dff3916", "metadata": {}, "outputs": [], "source": [ "# We can also tune PyGRANSO to more aggressively favor satisfying\n", "# feasibility over minimizing the objective. Set feasibility_bias to\n", "# true to adjust the following three steering parameters away from\n", "# their default values. For more details on these parameters, type\n", "# import pygransoOptionsAdvanced\n", "# help(pygransoOptionsAdvanced)\n", "import numpy as np\n", "opts = pygransoStruct()\n", "opts.torch_device = device\n", "feasibility_bias = True\n", "if feasibility_bias:\n", " opts.steering_ineq_margin = np.inf # default is 1e-6\n", " opts.steering_c_viol = 0.9 # default is 0.1\n", " opts.steering_c_mu = 0.1 # default is 0.9\n" ] }, { "cell_type": "code", "execution_count": 8, "id": "3fb2e962", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[33m╔═════ QP SOLVER NOTICE ════════════════════════════════════════════════════════════════════════╗\n", "\u001b[0m\u001b[33m║ PyGRANSO requires a quadratic program (QP) solver that has a quadprog-compatible interface, ║\n", "\u001b[0m\u001b[33m║ the default is osqp. Users may provide their own wrapper for the QP solver. ║\n", "\u001b[0m\u001b[33m║ To disable this notice, set opts.quadprog_info_msg = False ║\n", "\u001b[0m\u001b[33m╚═══════════════════════════════════════════════════════════════════════════════════════════════╝\n", "\u001b[0m═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗\n", "PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation ║ \n", "Version 1.2.0 ║ \n", "Licensed under the AGPLv3, Copyright (C) 2021-2022 Tim Mitchell and Buyun Liang ║ \n", "═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣\n", "Problem specifications: ║ \n", " # of variables : 200 ║ \n", " # of inequality constraints : 1 ║ \n", " # of equality constraints : 0 ║ \n", "═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣\n", " ║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║ \n", "Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║ \n", "═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣\n", " 0 ║ 1.000000 │ 16.2063030241 ║ 13.7635444107 ║ 2.442759 │ - ║ - │ 1 │ 0.000000 ║ 1 │ 28.28938 ║ \n", " 10 ║ 0.100000 │ 2.59973391273 ║ 13.5351593895 ║ 1.246218 │ - ║ S │ 4 │ 0.125000 ║ 1 │ 0.021935 ║ \n", " 20 ║ 0.100000 │ 2.16477288097 ║ 13.0916138399 ║ 0.855611 │ - ║ S │ 2 │ 0.500000 ║ 1 │ 0.023498 ║ \n", " 30 ║ 0.100000 │ 1.79745295456 ║ 13.0992702725 ║ 0.487526 │ - ║ S │ 7 │ 0.015625 ║ 1 │ 0.004077 ║ \n", " 40 ║ 0.100000 │ 1.58635141808 ║ 13.0513266695 ║ 0.281219 │ - ║ S │ 8 │ 0.007812 ║ 1 │ 0.004015 ║ \n", " 50 ║ 0.100000 │ 1.42581950214 ║ 13.0841956594 ║ 0.117400 │ - ║ S │ 3 │ 0.250000 ║ 1 │ 0.006101 ║ \n", " 60 ║ 0.100000 │ 1.29645227633 ║ 12.8945466090 ║ 0.006998 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.017645 ║ \n", " 70 ║ 0.100000 │ 1.27161142274 ║ 12.7161142274 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.001977 ║ \n", " 80 ║ 0.100000 │ 1.24712647142 ║ 12.4712647142 ║ 0.000000 │ - ║ S │ 7 │ 0.015625 ║ 1 │ 0.023924 ║ \n", " 90 ║ 0.100000 │ 1.23218670220 ║ 12.2655470686 ║ 0.005632 │ - ║ S │ 2 │ 2.000000 ║ 1 │ 0.004044 ║ \n", " 100 ║ 0.100000 │ 1.21443402902 ║ 12.1443402902 ║ 0.000000 │ - ║ S │ 2 │ 0.500000 ║ 1 │ 0.006803 ║ \n", " 110 ║ 0.100000 │ 1.20645480447 ║ 12.0614085408 ║ 3.14e-04 │ - ║ S │ 7 │ 0.015625 ║ 1 │ 0.001226 ║ \n", " 120 ║ 0.100000 │ 1.19765253592 ║ 11.9765253592 ║ 0.000000 │ - ║ S │ 6 │ 0.031250 ║ 1 │ 0.062770 ║ \n", " 130 ║ 0.100000 │ 1.19273267652 ║ 11.9273267652 ║ 0.000000 │ - ║ S │ 6 │ 0.031250 ║ 1 │ 0.014048 ║ \n", " 140 ║ 0.100000 │ 1.18358712018 ║ 11.8358712018 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.002047 ║ \n", " 150 ║ 0.100000 │ 1.17691829216 ║ 11.7691829216 ║ 0.000000 │ - ║ S │ 5 │ 0.062500 ║ 1 │ 0.107164 ║ \n", " 160 ║ 0.100000 │ 1.17305505323 ║ 11.7305505323 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.001760 ║ \n", " 170 ║ 0.100000 │ 1.16987321991 ║ 11.6987321991 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.001199 ║ \n", " 180 ║ 0.010000 │ 0.11670949970 ║ 11.6709499699 ║ 0.000000 │ - ║ S │ 2 │ 0.500000 ║ 1 │ 3.70e-04 ║ \n", " 190 ║ 0.010000 │ 0.11654589936 ║ 11.6545899360 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 2.54e-04 ║ \n", "═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣\n", " ║ <--- Penalty Function --> ║ ║ Total Violation ║ <--- Line Search ---> ║ <- Stationarity -> ║ \n", "Iter ║ Mu │ Value ║ Objective ║ Ineq │ Eq ║ SD │ Evals │ t ║ Grads │ Value ║ \n", "═════╬═══════════════════════════╬════════════════╬═════════════════╬═══════════════════════╬════════════════════╣\n", " 200 ║ 0.010000 │ 0.11630786772 ║ 11.6307867715 ║ 0.000000 │ - ║ S │ 1 │ 1.000000 ║ 1 │ 0.001599 ║ \n", "═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣\n", "F = final iterate, B = Best (to tolerance), MF = Most Feasible ║ \n", "Optimization results: ║ \n", "═════╦═══════════════════════════╦════════════════╦═════════════════╦═══════════════════════╦════════════════════╣\n", " F ║ │ ║ 11.6307867715 ║ 0.000000 │ - ║ │ │ ║ │ ║ \n", " B ║ │ ║ 11.6307867715 ║ 0.000000 │ - ║ │ │ ║ │ ║ \n", " MF ║ │ ║ 11.6307867715 ║ 0.000000 │ - ║ │ │ ║ │ ║ \n", "═════╩═══════════════════════════╩════════════════╩═════════════════╩═══════════════════════╩════════════════════╣\n", "Iterations: 200 ║ \n", "Function evaluations: 670 ║ \n", "PyGRANSO termination code: 4 --- max iterations reached. ║ \n", "═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝\n", "Total Wall Time: 51.2608208656311s\n" ] } ], "source": [ "opts.maxit = 200\n", "opts.x0 = torch.zeros(p*m,1).to(device=device, dtype=torch.double)\n", "# print for every 10 iterations. default: 1\n", "opts.print_frequency = 10\n", "\n", "start = time.time()\n", "soln = pygranso(var_spec = var_in,combined_fn = comb_fn,user_opts = opts)\n", "end = time.time()\n", "print(\"Total Wall Time: {}s\".format(end - start))" ] }, { "cell_type": "markdown", "id": "c1abe9e7", "metadata": {}, "source": [ "In my testing, with default parameters, PyGRANSO will first obtain a\n", "feasible solution at iter ~= 160 and will reduce the objective to\n", "11.60 by the time it attains max iteration count of 200.\n", "\n", "With feasibility_bias = True, in my testing, PyGRANSO will obtain its\n", "first feasible solution earlier, at iter ~= 60, but it will ultimately\n", "have reduced the objective value less, only to 12.21, by the end of\n", "its 200 maximum allowed iterations." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7" } }, "nbformat": 4, "nbformat_minor": 5 }