Skip to content

feat: Adan optimizer integration #181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 16 commits into
base: main
Choose a base branch
from
Draft

feat: Adan optimizer integration #181

Changes from 2 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
33c89da
test: init
Benjamin-eecs Jul 22, 2023
de1ab8a
feat: init adan alias
Benjamin-eecs Jul 22, 2023
69c9578
feat: init adan optim
Benjamin-eecs Jul 22, 2023
a98001d
feat: init adan transformation
Benjamin-eecs Jul 22, 2023
15fdf13
fix: [pre-commit.ci] auto fixes [...]
pre-commit-ci[bot] Jul 22, 2023
094582a
fix: update docstring
Benjamin-eecs Jul 22, 2023
b421884
Merge remote-tracking branch 'upstream/main' into feature/adan
Benjamin-eecs Jul 22, 2023
4132ada
Merge branch 'feature/adan' of https://github.com/benjamin-eecs/torch…
Benjamin-eecs Jul 22, 2023
53d2bd0
fix: update adan transformation
Benjamin-eecs Jul 23, 2023
5723cf6
fix: [pre-commit.ci] auto fixes [...]
pre-commit-ci[bot] Jul 23, 2023
cb2efb0
fix: update requirements
Benjamin-eecs Jul 24, 2023
8e38169
fix: update adan transformation
Benjamin-eecs Jul 24, 2023
c8b6a1e
Merge branch 'feature/adan' of https://github.com/benjamin-eecs/torch…
Benjamin-eecs Jul 24, 2023
4d94145
test: update Makefile
Benjamin-eecs Jul 24, 2023
501d125
feat: init adan alias
Benjamin-eecs Jul 24, 2023
e11cced
fix: [pre-commit.ci] auto fixes [...]
pre-commit-ci[bot] Jul 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions torchopt/transform/scale_by_adan.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,15 +125,15 @@ def _scale_by_adan(
tree_map = pytree.tree_map # type: ignore[assignment]

def init_fn(params: Params) -> OptState:
zero = tree_map( # count init
tree_map( # count init
lambda t: torch.zeros(1, dtype=torch.int64, device=t.device).squeeze_(),
params,
)
mu = tree_map( # first moment
mu = tree_map( # first moment
lambda t: torch.zeros_like(t, requires_grad=moment_requires_grad),
params,
)
nu = tree_map( # second moment
nu = tree_map( # second moment
torch.zeros_like,
params,
)
Expand Down
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy