What's Changed [0.7.3] - 2023-11-10

Changed

• Set minimal C++ standard to C++17 by @XuehaiPan in #195.

Fixed

• Fix optree compatibility for multi-tree-map with None values by @XuehaiPan in #195.

Full Changelog: v0.7.2...v0.7.3

What's Changed [0.7.2] - 2023-08-18

Added

• Implement Adadelta, RAdam, Adamax optimizer by @JieRen98 and @Benjamin-eecs in #171.

New Contributors

• @eltociear made their first contribution in #176.

Full Changelog: v0.7.1...v0.7.2

What's Changed [0.7.1] - 2023-05-12

Added

• Enable CI workflow to build CXX/CUDA extension for Python 3.11 by @XuehaiPan in #152.
• Implement AdaGrad optimizer and exponential learning rate decay schedule by @Benjamin-eecs and @XuehaiPan in #80.
• Enable tests on Windows by @XuehaiPan in #140.
• Add ruff and flake8 plugins integration by @XuehaiPan in #138 and #139.
• Add more documentation on implicit differentiation by @Benjamin-eecs and @XuehaiPan in #143.

Fixed

• Fix overloaded annotations of extract_state_dict by @StefanoWoerner in #162.
• Fix transpose empty iterable with zip(*nested) in transformations by @XuehaiPan in #145.

Removed

• Drop Python 3.7 support by @XuehaiPan in #136.

New Contributors

• @StefanoWoerner made their first contribution in #162.

Full Changelog: v0.7.0...v0.7.1

[0.7.0] - 2023-02-16

Added

• Update Sphinx documentation by @XuehaiPan and @Benjamin-eecs and @waterhorse1 and @JieRen98 in #127.
• Add object-oriented modules support for zero-order differentiation by @XuehaiPan in #125.

Changed

• Use postponed evaluation of annotations and update doctring style by @XuehaiPan in #135.
• Rewrite setup CUDA Toolkit logic by @XuehaiPan in #133.

Fixed

• Update tests and fix corresponding bugs by @XuehaiPan and @Benjamin-eecs and @JieRen98 in #78.
• Fix memory leak in implicit MAML omniglot few-shot classification example with OOP APIs by @XuehaiPan in #113.

Full Changelog: v0.6.0...v0.7.0

Highlights

• The implicit gradient and zero-order differentiation APIs become generally available in TorchOpt 0.6.0.
• The project code structure is refactored and reorganized.

[0.6.0] - 2022-12-07

Added

• Add unroll pragma for CUDA OPs by @JieRen98 and @XuehaiPan in #112.
• Add Python implementation of accelerated OP and pure-Python wheels by @XuehaiPan in #67.
• Add nan_to_num hook and gradient transformation by @XuehaiPan in #119.
• Add matrix inversion linear solver with neumann series approximation by @Benjamin-eecs and @XuehaiPan in #98.
• Add if condition of number of threads for CPU OPs by @JieRen98 in #105.
• Add implicit MAML omniglot few-shot classification example with OOP APIs by @XuehaiPan in #107.
• Add implicit MAML omniglot few-shot classification example by @Benjamin-eecs in #48.
• Add object-oriented modules support for implicit meta-gradient by @XuehaiPan in #101.
• Bump PyTorch version to 1.13.0 by @XuehaiPan in #104.
• Add zero-order gradient estimation by @JieRen98 in #93.
• Add RPC-based distributed training support and add distributed MAML example by @XuehaiPan in #83.
• Add full type hints by @XuehaiPan in #92.
• Add API documentation and tutorial for implicit gradients by @Benjamin-eecs and @JieRen98 and @XuehaiPan in #73.
• Add wrapper class for functional optimizers and examples of functorch integration by @vmoens and @Benjamin-eecs and @XuehaiPan in #6.
• Implicit differentiation support by @JieRen98 and @waterhorse1 and @XuehaiPan in #41.

Changed

• Refactor code organization by @XuehaiPan in #92 and #100.

Fixed

• Fix implicit MAML omniglot few-shot classification example by @XuehaiPan in #108.
• Align results of distributed examples by @XuehaiPan in #95.
• Fix None in module containers by @XuehaiPan.
• Fix backward errors when using inplace sqrt_ and add_ by @Benjamin-eecs and @JieRen98 and @XuehaiPan.
• Fix LR scheduling by @XuehaiPan in #76.
• Fix the step count tensor (shape=(1,)) can change the shape of the scalar updates (shape=()) by @XuehaiPan in #71.

Full Changelog: v0.5.0...v0.6.0

[0.5.0] - 2022-09-05

Added

• Implement AdamW optimizer with masking by @Benjamin-eecs and @XuehaiPan in #44.
• Add half float support for accelerated OPs by @XuehaiPan in #67.
• Add MAML example with TorchRL integration by @vmoens and @Benjamin-eecs in #12.
• Add optional argument params to update function in gradient transformations by @XuehaiPan in #65.
• Add option weight_decay option to optimizers by @XuehaiPan in #65.
• Add option maximize option to optimizers by @XuehaiPan in #64.
• Refactor tests using pytest.mark.parametrize and enabling parallel testing by @XuehaiPan and @Benjamin-eecs in #55.
• Add maml-omniglot few-shot classification example using functorch.vmap by @Benjamin-eecs in #39.
• Add parallel training on one GPU using functorch.vmap example by @Benjamin-eecs in #32.
• Add question/help/support issue template by @Benjamin-eecs in #43.

Changed

• Align argument names with PyTorch by @XuehaiPan in #65.
• Replace JAX PyTrees with OpTree by @XuehaiPan in #62.
• Update image link in README to support PyPI rendering by @Benjamin-eecs in #56.

Fixed

• Fix RMSProp optimizer by @XuehaiPan in #55.
• Fix momentum tracing by @XuehaiPan in #58.
• Fix CUDA build for accelerated OP by @XuehaiPan in #53.
• Fix gamma error in MAML-RL implementation by @Benjamin-eecs #47.

[0.4.3] - 2022-08-08

What's Changed

Added

• Bump PyTorch version to 1.12.1 by @XuehaiPan in #49.
• CPU-only build without nvcc requirement by @XuehaiPan in #51.
• Use cibuildwheel to build wheels by @XuehaiPan in #45.
• Use dynamic process number in CPU kernels by @JieRen98 in #42.

Changed

• Use correct Python Ctype for pybind11 function prototype @XuehaiPan in #52.

Full Changelog: v0.4.2...v0.4.3
Pre-built wheels: https://pypi.org/project/torchopt/0.4.3/#files

[0.4.2] - 2022-07-26

What's Changed

Added

• Read the Docs integration by @Benjamin-eecs and @XuehaiPan in #34.
• Update documentation and code styles by @Benjamin-eecs and @XuehaiPan in #22.
• Update tutorial notebooks by @XuehaiPan in #27.
• Bump PyTorch version to 1.12 by @XuehaiPan in #25.
• Support custom Python executable path in CMakeLists.txt by @XuehaiPan in #18.
• Add citation information by @waterhorse1 in #14 and @Benjamin-eecs in #15.
• Implement RMSProp optimizer by @future-xy in #8.

Changed

• Use pyproject.toml for packaging and update GitHub Action workflows by @XuehaiPan in #31.
• Rename the package from TorchOpt to torchopt by @XuehaiPan in #20.

Fixed

• Fixed errors while building from the source and add conda environment recipe by @XuehaiPan in #24.

Full Changelog: v0.4.1...v0.4.2

N/A