NCVX (NonConVeX) is a user-friendly and scalable python software package targeting general nonsmooth NCVX problems with nonsmooth constraints. NCVX is being developed by GLOVEX at the Department of Computer Science & Engineering, University of Minnesota, Twin Cities.
Our paper is available at https://arxiv.org/abs/2111.13984.
The initial release of NCVX contains the solver PyGRANSO, a PyTorch-enabled port of GRANSO incorporating auto-differentiation, GPU acceleration, tensor input, and support for new QP solvers. As a highlight, PyGRANSO can solve general constrained deep learning problems, the first of its kind.
PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation
Get the Code¶
The source code of NCVX Package is available in the NCVX Repository.
The source code of PyGRANSO Solver is available in the PyGRANSO Repository.
Version: 1.2.0 — 2022-03-XX (In development)
Description: major fixes and improvements on LBFGS and Steering Strategy.
Fixed: Reducing memory usage for LBFGS. Now PyGRANSO can solve problem with ~15k parameters by using 8GB Memory. Improving steering stategy to deal with many constraints.
Improved: improve the performance of ex11 orthogonal constraints on RNN; New suggestion about stationarity and feasibility tolerance, as it depends on problem scaling.
Added: ex 12 perceptual attack on ImageNet images; ex 13 trace optimization with orthogonal constraints; ex 14 unconstrained deep learning with LeNet5; ex 15 logistic regression; ex 16 generic dictionary learning
Version: 1.1.0 — 2022-02-20
Description: major fixes and improvements.
Fixed: Avoid gradient accumulating in deep learning problem; Prevent memory leak problem when using torch tensor. See ex6 perceptual attack.
Changed: Update format of user-defined variables when using pygranso interface.
Packaging: Publish pygranso package on Pypi.
Added: ex 10 dictionary learning with torch.nn module; ex 11 orthogonal recurrent neural networks.
Version: 1.0.0 — 2021-12-27
Description: initial public release of PyGRANSO.
Main features: auto-differentiation, GPU acceleration, tensor input, scalable QP solver, and zero dependency on proprietary packages. Multiple new examples added.
We would like to thank Frank E. Curtis and Michael L. Overton for their involvement in creating the BFGS-SQP algorithm that is implemented in the software package GRANSO. This work was supported by UMII Seed Grant Program and NSF CMMI 2038403.
- For questions or bug reports, please either:
raise issues in the PyGRANSO Repository or
send an email to:
Buyun Liang (liang664 an_at_symbol umn a_dot_symbol edu)
Tim Mitchell (tim an_at_symbol timmitchell a_dot_symbol com)
Ju Sun (jusun an_at_symbol umn a_dot_symbol edu)
Also, we are always looking for contributors and collaborators.