Contents

Home

_images/NCVX_logo.png

NCVX Package

NCVX (NonConVeX) is a user-friendly and scalable python software package targeting general nonsmooth NCVX problems with nonsmooth constraints. NCVX is being developed by GLOVEX at the Department of Computer Science & Engineering, University of Minnesota, Twin Cities.

The initial release of NCVX contains the solver PyGRANSO, a PyTorch-enabled port of GRANSO incorporating auto-differentiation, GPU acceleration, tensor input, and support for new QP solvers. As a highlight, PyGRANSO can solve general constrained deep learning problems, the first of its kind.

_images/PyGRANSO_logo_banner.png

PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation

Get the Code

The source code of NCVX Package is available in the NCVX Repository.

The source code of PyGRANSO Solver is available in the PyGRANSO Repository.

Update Logs

Version: 1.2.0 — 2022-07-26

  • Description: major fixes and improvements on LBFGS and Steering Strategy.

  • Fixed: Reducing memory usage for LBFGS. Now PyGRANSO can solve problem with ~15k parameters by using 14 GB memory; Update example: ortho RNN with max folding and orthonormal initialization; Allow high precision for QP solver; Allow part of optimization variables not showing up in objective (see SVM example); Fixed Code 12: terminated with steering failure; Fixed stationary failure: try different stationarity calculation, or set stationarity measure to be inf if encounter numerical issue

  • Added: Reorganize and add examples: perceptual/lp norm attack on ImageNet images. trace optimization with orthogonal constraints; unconstrained deep learning with LeNet5; logistic regression.

Version: 1.1.0 — 2022-02-20

  • Description: major fixes and improvements.

  • Fixed: Avoid gradient accumulating in deep learning problem; Prevent memory leak problem when using torch tensor. See ex6 perceptual attack.

  • Changed: Update format of user-defined variables when using pygranso interface.

  • Packaging: Publish pygranso package on Pypi.

  • Added: ex 10 dictionary learning with torch.nn module; ex 11 orthogonal recurrent neural networks.

Version: 1.0.0 — 2021-12-27

  • Description: initial public release of PyGRANSO.

  • Main features: auto-differentiation, GPU acceleration, tensor input, scalable QP solver, and zero dependency on proprietary packages. Multiple new examples added.

Acknowledgements

We would like to thank Frank E. Curtis and Michael L. Overton for their involvement in creating the BFGS-SQP algorithm that is implemented in the software package GRANSO. This work was supported by UMII Seed Grant Program and NSF CMMI 2038403.

Contact

For questions or bug reports, please either:
Main authors:

  • Buyun Liang (byliang an_at_symbol seas a_dot_symbol upenn a_dot_symbol edu)

  • Tim Mitchell (tim an_at_symbol timmitchell a_dot_symbol com)

  • Ju Sun (jusun an_at_symbol umn a_dot_symbol edu)

Also, we are always looking for contributors and collaborators.