gh-132502: Prototyping adding timestamps to tracebacks. #129337
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This came up at work as a suggestion to make debugging what happened in big async servers with lots of exception groups and exceptions easier. Timestamps when emitting exception groups containing tracebacks often with their own nested causes would allow some semblance of order to be understood. This is a demo. If we want such a feature, we should settle on semantics in a Discuss thread and write it up as a PEP. This should be simpler than exception notes (PEP-678) was. One thought was just to store the timestamp as a note; but that'd involve string and list creation on every exception. Performance testing needs to be done. This is the kind of thing that is visually distracting, so not all applications want to _see_ the timestamps. A knob to turn that on for those who do seems more useful rather than making that the default. But the performance impact of merely collecting the timestamps is worth knowing.
Avoids a 15% regression in the pyperformance async_generators suite.
Tested with `PYTHON_TRACEBACK_TIMESTAMPS=ns` set. First pass. Further review could rework some of these changes. Explicit tests for the new feature have yet to be added.
6c85054 to
53b5500
Compare
Better docs, improved tests. Claude Code using Sonnet 3.7 helped with this, but that was a bit of a battle as our CPython code context size for this type of change is huge.
TODO: performance testing - this increases the conditional load on every BaseException instantiation with that interp->config.field && field[0] check. If needed, we could cache the "collect or not" bool in a static global as it is fair to say this is a process wide setting rather than per interpreter, but ugh.
Use regex assertions with multi-line mode to match end of lines instead of literal newline characters. This allows the tests to pass on Windows which uses \r\n line endings instead of Unix \n line endings. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
|
🤖 New build scheduled with the buildbot fleet by @gpshead for commit b268c82 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F129337%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again. |
Resolves conflicts by integrating traceback timestamp functionality with the new colorization system and maintaining timestamp regex patterns in tests. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Update test_wsgiref, test_pdb, and test_remote_pdb to handle timestamp suffixes in exception messages when PYTHON_TRACEBACK_TIMESTAMPS is set. - test_wsgiref: Use traceback.strip_exc_timestamps() before comparing error messages in validation tests - test_pdb: Update doctest expected output to use ellipsis for timestamp matching in await support test - test_remote_pdb: Strip timestamps from stdout before comparison in do_test method All tests now pass both with and without timestamp functionality enabled. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Implements the missing tests identified in PR python#129337 TODO section: - Test pickle/unpickle of all built-in exception types in hierarchy - Test user-derived exception classes from BaseException, Exception, OSError, ImportError, AttributeError - Test timestamp presence on all exception types except StopIteration and StopAsyncIteration Reorganizes tests into a proper package structure with specialized test modules: - test_basic.py: Original basic functionality tests - test_pickle.py: Exception pickle/unpickle preservation tests - test_user_exceptions.py: Custom exception class tests with inheritance validation - test_timestamp_presence.py: Timestamp behavior verification across all exception types All tests validate behavior both with and without timestamp feature enabled, ensuring proper functionality and performance optimizations for StopIteration/StopAsyncIteration. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Create shared_utils.py with common exception creation logic - Consolidate duplicate code across test_pickle.py, test_timestamp_presence.py, and test_user_exceptions.py - Embed shared functions directly in subprocess test scripts to avoid import issues - Reduce overall test code by ~400 lines while preserving functionality - All 25 tests pass both with and without timestamps enabled 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
|
🤖 New build scheduled with the buildbot fleet by @gpshead for commit 4aaa4dc 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F129337%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again. |
When traceback timestamps are disabled, exception pickles no longer include an empty state dictionary, reducing pickle size to match Python 3.13 baseline. When timestamps are enabled, the state dict is included with timestamp data. - BaseException: Only include dict when timestamp > 0 or custom attributes exist - OSError: Apply same optimization while preserving filename attributes - ImportError: Conditionally include state based on meaningful attributes - AttributeError: Always include state dict for Python 3.13 compatibility Results: - ValueError/RuntimeError: 53 bytes (disabled) -> 103 bytes (enabled) - OSError: 56 bytes (disabled) -> 106 bytes (enabled) - AttributeError: 76 bytes (disabled) -> 120 bytes (enabled) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Include BaseException and Exception in pickle tests by removing unnecessary filtering - Add user-derived exception classes for ImportError and AttributeError - Tests now fully cover all built-in exception types and required user-derived classes 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
52554ea to
77ffb5a
Compare
|
🤖 New build scheduled with the buildbot fleet by @gpshead for commit 77ffb5a 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F129337%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again. |
gpshead
commented
Jun 8, 2025
- Add cross-references between command line flags and environment variables - Clarify default behavior when PYTHON_TRACEBACK_TIMESTAMPS is unset - Simplify traceback.rst wording to use "canonical output" - Fix alphabetical ordering in test_config.py - Improve comment formatting in test_sys.py for better readability 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
The functionality is already provided by force_color(False) and force_not_colorized() which were added since this branch started. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The idea for this came up at work with @njsmith tossing the idea out as "dumb" yet still maybe useful... Why? to make debugging what happened in async servers with lots of exception groups and exceptions easier. Event timestamps reported on everything when emitting exception groups containing tracebacks often with their own nested causes would allow some semblance of order to be more readily understood and correlated with other parts of the distributed system.
This draft PR is a demo. If we want such a feature, we should settle on semantics in a Discuss thread and if necessary, write it up as a PEP. This should be simpler than exception notes (PEP-678) was. One thought was just to store the timestamp as a note; but that'd involve string and list creation on every exception - slow.
Demo
A real world demo involving exception groups in an async application would be more interesting but isn't something I can just tap out in a repl...
Updates: [edits]
I've done performance testing. It now appears to be a no-op (no meaningful regression) as desired performance wise on Linux. For that to be true, I had to add the one special case I suspected might matter: Don't collect the timestamp on
StopIterationandAsyncStopIterationas those are not infrequent error exceptions, but a normal part of application control flow. Without that, one of the async pyperformance benchmarks showed a significant performance hit of over 10%.I've changed it to not emit the timestamps by default. If you set the
PYTHON_TRACEBACK_TIMESTAMPS=environment variable to one of eitherusor1,ns, oriso, timestamps will be enabled intracebackmodule display output. See the documentation in the PR.TODO
isomode. Or decideisois too complicated and simplify by ripping it out?