Skip to content

gh-136912: fix handling of OverflowError in hmac.digest #136917

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 26, 2025

Conversation

picnixz
Copy link
Member

@picnixz picnixz commented Jul 21, 2025

@picnixz picnixz requested a review from gpshead as a code owner July 21, 2025 10:06
@picnixz picnixz force-pushed the fix/hmac/catch-overflow-error-136912 branch 5 times, most recently from 99d62fe to d0079c6 Compare July 21, 2025 10:25
@picnixz
Copy link
Member Author

picnixz commented Jul 21, 2025

!buildbot bigmem

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @picnixz for commit d0079c6 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136917%2Fmerge

The command will test the builders whose names match following regular expression: bigmem

The builders matched are:

  • aarch64 Ubuntu 22.04 BigMem PR
  • AMD64 Windows11 Bigmem PR

@picnixz picnixz marked this pull request as draft July 21, 2025 14:36
@picnixz
Copy link
Member Author

picnixz commented Jul 21, 2025

!buildbot bigmem

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @picnixz for commit f93e1ba 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136917%2Fmerge

The command will test the builders whose names match following regular expression: bigmem

The builders matched are:

  • aarch64 Ubuntu 22.04 BigMem PR
  • AMD64 Windows11 Bigmem PR

@picnixz picnixz marked this pull request as ready for review July 21, 2025 16:03
@picnixz
Copy link
Member Author

picnixz commented Jul 21, 2025

!buildbot bigmem

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @picnixz for commit df36d7d 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136917%2Fmerge

The command will test the builders whose names match following regular expression: bigmem

The builders matched are:

  • aarch64 Ubuntu 22.04 BigMem PR
  • AMD64 Windows11 Bigmem PR

@picnixz
Copy link
Member Author

picnixz commented Jul 21, 2025

!buildbot bigmem

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @picnixz for commit 74239be 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136917%2Fmerge

The command will test the builders whose names match following regular expression: bigmem

The builders matched are:

  • aarch64 Ubuntu 22.04 BigMem PR
  • AMD64 Windows11 Bigmem PR

@picnixz picnixz marked this pull request as draft July 21, 2025 17:26
@picnixz
Copy link
Member Author

picnixz commented Jul 21, 2025

I'll rewrite this PR tomorrow to use suggestion 4.

Lib/hmac.py Outdated
@@ -241,13 +241,23 @@ def digest(key, msg, digest):
if _hashopenssl and isinstance(digest, (str, _functype)):
try:
return _hashopenssl.hmac_digest(key, msg, digest)
except OverflowError:
try:
return _hashopenssl.hmac_new(key, msg, digest).digest()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gpshead This will create a real HMAC object using OpenSSL and handles chunks in C. Alternatively, I can just catch the OverflowError directly and ignore it. The pure Python implementation already handles chunks as we just call .update() which is implemented in C as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EDIT: actually OpenSSL still requires a key of size at most INT_MAX and HACL* requires the size to be at most UINT32_MAX. I'll just directly switch to the "slow" python implementation.

@picnixz picnixz force-pushed the fix/hmac/catch-overflow-error-136912 branch 4 times, most recently from 1188f4a to 84ea348 Compare July 22, 2025 07:22
@picnixz picnixz force-pushed the fix/hmac/catch-overflow-error-136912 branch from 84ea348 to 7e01d44 Compare July 22, 2025 07:24
@picnixz picnixz force-pushed the fix/hmac/catch-overflow-error-136912 branch from 7e01d44 to 1082bd5 Compare July 22, 2025 07:24
@picnixz picnixz marked this pull request as ready for review July 22, 2025 07:25
@picnixz picnixz added needs backport to 3.14 bugs and security fixes and removed awaiting core review labels Jul 22, 2025
@picnixz
Copy link
Member Author

picnixz commented Jul 22, 2025

!buildbot bigmem

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @picnixz for commit 1082bd5 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136917%2Fmerge

The command will test the builders whose names match following regular expression: bigmem

The builders matched are:

  • aarch64 Ubuntu 22.04 BigMem PR
  • AMD64 Windows11 Bigmem PR

@picnixz picnixz force-pushed the fix/hmac/catch-overflow-error-136912 branch 2 times, most recently from 5686d8f to 2bb7fb6 Compare July 22, 2025 14:49
@picnixz picnixz force-pushed the fix/hmac/catch-overflow-error-136912 branch from 2bb7fb6 to 4d412bd Compare July 22, 2025 14:50
@picnixz
Copy link
Member Author

picnixz commented Jul 22, 2025

!buildbot bigmem

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @picnixz for commit 4d412bd 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136917%2Fmerge

The command will test the builders whose names match following regular expression: bigmem

The builders matched are:

  • aarch64 Ubuntu 22.04 BigMem PR
  • AMD64 Windows11 Bigmem PR

@picnixz
Copy link
Member Author

picnixz commented Jul 25, 2025

!buildbot bigmem

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @picnixz for commit ac6b983 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136917%2Fmerge

The command will test the builders whose names match following regular expression: bigmem

The builders matched are:

  • aarch64 Ubuntu 22.04 BigMem PR
  • AMD64 Windows11 Bigmem PR

@picnixz picnixz enabled auto-merge (squash) July 26, 2025 08:04
@picnixz picnixz merged commit d658b90 into python:main Jul 26, 2025
41 checks passed
@miss-islington-app
Copy link

Thanks @picnixz for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

@miss-islington-app
Copy link

Sorry, @picnixz, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker d658b9053beaacaae80e318f59a5ddd672aa757a 3.14

@picnixz picnixz deleted the fix/hmac/catch-overflow-error-136912 branch July 26, 2025 08:28
@picnixz
Copy link
Member Author

picnixz commented Jul 26, 2025

I'll do the 3.14 backporst in 3.14.1

@picnixz picnixz removed the needs backport to 3.14 bugs and security fixes label Jul 26, 2025
@bedevere-app
Copy link

bedevere-app bot commented Jul 26, 2025

GH-137116 is a backport of this pull request to the 3.14 branch.

picnixz added a commit to picnixz/cpython that referenced this pull request Jul 26, 2025
…st` (pythonGH-136917)

The OpenSSL and HACL* implementations of HMAC single-shot
digest computation reject keys whose length exceeds `INT_MAX`
and `UINT32_MAX` respectively. The OpenSSL implementation
also rejects messages whose length exceed `INT_MAX`.

Using such keys in `hmac.digest` previously raised an `OverflowError`
which was propagated to the caller. This commit mitigates this case by
making `hmac.digest` fall back to HMAC's pure Python implementation
which accepts arbitrary large keys or messages.

This change only affects the top-level entrypoint `hmac.digest`, leaving
`_hashopenssl.hmac_digest` and `_hmac.compute_digest` untouched.
(cherry picked from commit d658b90)

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy