From f41ecddf0e18ed4e66cbd2d8fc1f1d0a453114a6 Mon Sep 17 00:00:00 2001 From: Disconnect3d Date: Sat, 19 Jul 2025 11:18:37 +0000 Subject: [PATCH 1/3] Doc/c-api/memory.rst: extend --without-pymalloc doc with ASan information This commit extends the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds). I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here). I have discussed ASan and this flag with @encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation. This code can be seen here: https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Objects/obmalloc.c#L2096-L2110 While the annotation macro is defined here: https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Include/pyport.h#L582-L598 And the corresponding attribute is documented in: * for gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute * for clang: https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis --- Doc/c-api/memory.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 61fa49f8681cce..ead542fddf58c2 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -672,6 +672,13 @@ This allocator is disabled if Python is configured with the :option:`--without-pymalloc` option. It can also be disabled at runtime using the :envvar:`PYTHONMALLOC` environment variable (ex: ``PYTHONMALLOC=malloc``). +Typically, it makes sense to disable the pymalloc allocator when building +Python with AddressSanitizer (ASan) which helps uncover low level bugs within +the C code. While pymalloc doesn't break under ASan, the ASan works more +effectively with the system allocator (since we do not add any padding in +between the objects managed/allocated by pymalloc and we do not annotate/poison +the "inaccessible" memory between the pymalloc allocated objects). + Customize pymalloc Arena Allocator ---------------------------------- From 7659c24a752fea3884f0362abed760e2d16814ef Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Sat, 19 Jul 2025 13:27:34 +0200 Subject: [PATCH 2/3] Update Doc/c-api/memory.rst --- Doc/c-api/memory.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index ead542fddf58c2..0e1cc772709430 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -675,9 +675,8 @@ the :envvar:`PYTHONMALLOC` environment variable (ex: ``PYTHONMALLOC=malloc``). Typically, it makes sense to disable the pymalloc allocator when building Python with AddressSanitizer (ASan) which helps uncover low level bugs within the C code. While pymalloc doesn't break under ASan, the ASan works more -effectively with the system allocator (since we do not add any padding in -between the objects managed/allocated by pymalloc and we do not annotate/poison -the "inaccessible" memory between the pymalloc allocated objects). +effectively with the system allocator (ASan does not track +individual allocations done by pymalloc). Customize pymalloc Arena Allocator ---------------------------------- From d3483fcabfcc7799056e5d5087f921507b056d18 Mon Sep 17 00:00:00 2001 From: Disconnect3d Date: Sat, 19 Jul 2025 11:45:13 +0000 Subject: [PATCH 3/3] Improve --with-address-sanitizer and pymalloc docs --- Doc/c-api/memory.rst | 6 ++---- Doc/using/configure.rst | 3 +++ 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 0e1cc772709430..df1bb0ce370919 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -673,10 +673,8 @@ This allocator is disabled if Python is configured with the the :envvar:`PYTHONMALLOC` environment variable (ex: ``PYTHONMALLOC=malloc``). Typically, it makes sense to disable the pymalloc allocator when building -Python with AddressSanitizer (ASan) which helps uncover low level bugs within -the C code. While pymalloc doesn't break under ASan, the ASan works more -effectively with the system allocator (ASan does not track -individual allocations done by pymalloc). +Python with AddressSanitizer (:option:`--with-address-sanitizer`) which helps +uncover low level bugs within the C code. Customize pymalloc Arena Allocator ---------------------------------- diff --git a/Doc/using/configure.rst b/Doc/using/configure.rst index e5fe3c72b1b26e..2cda9587975ddc 100644 --- a/Doc/using/configure.rst +++ b/Doc/using/configure.rst @@ -802,6 +802,9 @@ Debug options .. option:: --with-address-sanitizer Enable AddressSanitizer memory error detector, ``asan`` (default is no). + To improve ASan detection capabilities you may also want to combine this + with :option:`--without-pymalloc` to disable the specialized small-object + allocator whose allocations are not tracked by ASan. .. versionadded:: 3.6 pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy