Add ASAN support to the zend allocator #18858

danog · 2025-06-16T12:12:21Z

This pull request adds ASAN support to the zend allocator, by automatically poisoning all unused pages, chunks and heap management structures before exiting the alloc, free, etc (all ZEND_API) functions.

Internally, the implementation uses the following rules:

Always poison memory (re)allocated by private (non-ZEND_API) allocation functions before returning it
Unpoison memory (re)allocated by public (ZEND_API) allocation functions before returning within the ZEND_API function
Always poison freed memory
Always poison unused memory during reallocation (where new_size < old_size)
When accessing private heap structures and fields, always unpoison before accessing and repoison immediately after
- An exception to the above (for simplicity) is the main heap datastructure, which is poisoned only when entering a ZEND_API function and repoisoned before exiting

This is what allowed me to find #18833, before I found the fast shutdown workaround.

danog · 2025-06-17T13:06:28Z

@dstogov / @nielsdos / @iluuu1994 This is now ready for review!

The implementation is finalized, and has actually revealed a shutdown memory leak in php-fpm when terminated via a SIGTERM, due to the fact that alloc_globals_dtor is not getting invoked (the leak is surprisingly revealed exclusively by the usage of the poison functions, in fact, altering the ZEND_MM_POISON/ZEND_MM_UNPOISON macros to be no-ops re-hides the issue again, i.e. it's not caused by the other code changes/reorganization).

This PR also includes the changes from #18834, as the bug is detectable by ASAN now.

Side note, tracked_malloc also poisons/unpoisons the heap to avoid issues with the observe proxies.

nielsdos

When trying to run ./run-tests.php with this I get an immediate ASAN crash that should be fixed:

==104448==ERROR: AddressSanitizer: use-after-poison on address 0x7b96b5c01080 at pc 0x560e145d67d4 bp 0x7ffd7eb60800 sp 0x7ffd7eb607f8
WRITE of size 8 at 0x7b96b5c01080 thread T0
    #0 0x560e145d67d3 in zend_mm_alloc_small_slow /run/media/niels/MoreData/php-src/Zend/zend_alloc.c:1507:14
    #1 0x560e145d4f8e in zend_mm_alloc_small /run/media/niels/MoreData/php-src/Zend/zend_alloc.c:1546:10
    #2 0x560e145c629b in zend_mm_alloc_heap /run/media/niels/MoreData/php-src/Zend/zend_alloc.c:1625:9
    #3 0x560e145cb2a4 in _emalloc /run/media/niels/MoreData/php-src/Zend/zend_alloc.c:3034:14
    #4 0x560e14c45e82 in cwd_globals_ctor /run/media/niels/MoreData/php-src/Zend/zend_virtual_cwd.c:145:2
    #5 0x560e14c45dce in virtual_cwd_startup /run/media/niels/MoreData/php-src/Zend/zend_virtual_cwd.c:211:2
    #6 0x560e14c5ec09 in zend_startup /run/media/niels/MoreData/php-src/Zend/zend.c:942:2
    #7 0x560e14320b8d in php_module_startup /run/media/niels/MoreData/php-src/main/main.c:2200:2
    #8 0x560e14c733b4 in php_cli_startup /run/media/niels/MoreData/php-src/sapi/cli/php_cli.c:398:9
    #9 0x560e14c6e243 in main /run/media/niels/MoreData/php-src/sapi/cli/php_cli.c:1330:6
    #10 0x7f96bb8906b4  (/usr/lib/libc.so.6+0x276b4) (BuildId: 468e3585c794491a48ea75fceb9e4d6b1464fc35)
    #11 0x7f96bb890768 in __libc_start_main (/usr/lib/libc.so.6+0x27768) (BuildId: 468e3585c794491a48ea75fceb9e4d6b1464fc35)
    #12 0x560e13003584 in _start (/run/media/niels/MoreData/php-src/sapi/cli/php+0xa03584) (BuildId: 04a8f40ab178fb1528246ccec118a5d2e939fd1b)

Zend/zend_alloc.c

danog · 2025-07-07T11:12:03Z

@nielsdos what compiler, platform, configure flags and memory manager envvars are you using?

nielsdos · 2025-07-07T18:25:59Z

what compiler, platform, configure flags and memory manager envvars are you using?

It seems I was testing on an older checkout, I tested it again with a reset to your branch and it works, sorry for the confusion...

Zend/zend_alloc.c

nielsdos

Besides the remaining cs nit I don't see anything obviously wrong. Thanks for the clarifications. I think this patch is worth it. I'll leave this open for some time to see if others want to review this too.

danog · 2025-07-16T09:01:06Z

Ping? :)

nielsdos · 2025-07-21T12:58:11Z

cc @iluuu1994 ?

iluuu1994 · 2025-07-21T12:59:40Z

@nielsdos I'll have a look in a few hours.

iluuu1994

Only approx. halfway through (line 1900 old diff). Overall this seems quite complex and easy to make mistakes. This can't detect buffer overflows for two adjacent allocations within the same bin. I'm also not sure whether this can catch more errors in the allocator itself, as pointers are explicitly unpoisoned before usage, so even if a pointer were wrong, it would just be unpoisoned.

But if others think this should be merged, we should have at least one ASAN build in nightly that doesn't set the USE_ZEND_ALLOC=0 env var. It would also be good to run a test COMMUNITY build to reveal any remaining errors.

Zend/zend_alloc.c

iluuu1994 · 2025-07-22T22:45:56Z

Zend/zend_alloc.c

 	} while (0);
 #endif

 	p = (zend_mm_free_slot*)ptr;
 	zend_mm_set_next_free_slot(heap, bin_num, p, heap->free_slot[bin_num]);
 	heap->free_slot[bin_num] = p;
+
+	ZEND_MM_POISON(p, bin_data_size[bin_num]);


p should already be poisoned.

Only the header and shadow in zend_mm_set_next_free_slot, not the body

iluuu1994 · 2025-07-22T22:51:50Z

Zend/zend_alloc.c

@@ -1586,6 +1812,8 @@ static zend_never_inline void *zend_mm_realloc_slow(zend_mm_heap *heap, void *pt
 		size_t orig_peak = heap->peak;
 #endif
 		ret = zend_mm_alloc_heap(heap, size ZEND_FILE_LINE_RELAY_CC ZEND_FILE_LINE_ORIG_RELAY_CC);
+		ZEND_MM_UNPOISON(ret, size);
+		ZEND_MM_UNPOISON(ptr, copy_size);


ptr should be unpoisoned already.

copy_size is set to the size of the bin if size > old_size, size > size of the bin, old_size < size of the bin, so the unpoison is needed.

iluuu1994 · 2025-07-22T22:54:30Z

Zend/zend_alloc.c

@@ -1627,6 +1855,7 @@ static zend_never_inline void *zend_mm_realloc_huge(zend_mm_heap *heap, void *pt
 #else
 			zend_mm_change_huge_block_size(heap, ptr, new_size ZEND_FILE_LINE_RELAY_CC ZEND_FILE_LINE_ORIG_RELAY_CC);
 #endif
+			ZEND_MM_POISON(ptr, new_size);


Why do we poison here? Edit: I see, it's unpoisoned right away in zend_mm_realloc_heap. Not sure if that's useful.

It just follows the implementation rules

Zend/zend_alloc.c

iluuu1994 · 2025-07-22T23:01:21Z

Zend/zend_alloc.c

 						memcpy(ret, ptr, copy_size);
 						zend_mm_free_small(heap, ptr, old_bin_num);
 					} else {
 						/* reallocation in-place */
 						ret = ptr;
+						ZEND_MM_UNPOISON(ret, size);


Should already be unpoisoned. If anything, the upper bytes should be poisoned.

iluuu1994 · 2025-07-23T09:52:39Z

I'm also not sure whether this can catch more errors in the allocator itself, as pointers are explicitly unpoisoned before usage, so even if a pointer were wrong, it would just be unpoisoned.

Regarding this part: My fear is that, given most accesses unpoison/poison memory very locally, this reverts a lot of the benefits of poisoning. If an access is incorrect, it's very likely for the unpoisoning itself to also be incorrect, whether by depending on an incorrect parameter or applying incorrect logic to both unpoisoning and access. I will discuss this with Arnaud once he's back from vacation, so at some point next week.

Thanks!

danog · 2025-07-23T10:05:29Z

Regarding this part: My fear is that, given most accesses unpoison/poison memory very locally, this reverts a lot of the benefits of poisoning. If an access is incorrect, it's very likely for the unpoisoning itself to also be incorrect, whether by depending on an incorrect parameter or applying incorrect logic to both unpoisoning and access. I will discuss this with Arnaud once he's back from vacation, so at some point next week.

The main class of errors this PR detects is use after frees; buffer overflows are also detected to some extent, but for better detection vanilla ASAN should be used.

However, I have found in my experience that use after frees are the most common type of memory corruption in PHP, mainly due to refcounting bugs, and this PR helps detect those without using the more expensive and slow vanilla ASAN.

github-actions bot added the Category: Engine label Jun 16, 2025

ASAN support for the zend allocator

2f84241

danog force-pushed the asan_zend_alloc branch from 197f4a3 to 2f84241 Compare June 16, 2025 12:14

danog added 18 commits June 16, 2025 17:49

Fix

1880729

Fix

193962c

Fixes

1a259a0

Fixes

4ea8a03

Fixes

ef9a0d2

Improvements

0376092

Improvements

340d985

Improvements

e800e53

Improvements

12eefe4

Fix

6563269

Cleanup

61661d3

Cleanup

4301651

Fix

f11d436

Fix

bba7468

Cleanup

b1b7c1f

Cleanup

835810c

Fixes

c576965

Hotfix newly detected memory leak

e31c4ac

github-actions bot added the SAPI: fpm label Jun 17, 2025

danog added 3 commits June 17, 2025 14:17

Fixes

d9de798

Fixes

e5ca924

Final

e88ec6d

danog force-pushed the asan_zend_alloc branch from 488d33d to e88ec6d Compare June 17, 2025 12:25

danog marked this pull request as ready for review June 17, 2025 13:06

danog requested review from dstogov and bukka as code owners June 17, 2025 13:06

nielsdos requested changes Jul 6, 2025

View reviewed changes

danog added 3 commits July 7, 2025 11:09

cs fixes

8925acf

cs fixes

38e7896

cs fixes

1875711

nielsdos reviewed Jul 7, 2025

View reviewed changes

Zend/zend_alloc.c Show resolved Hide resolved

Zend/zend_alloc.c Outdated Show resolved Hide resolved

nielsdos approved these changes Jul 7, 2025

View reviewed changes

Cs fixes

40936ff

iluuu1994 reviewed Jul 22, 2025

View reviewed changes

danog added 12 commits July 23, 2025 10:28

cs-fix

832dae0

Poison custom chunk_alloc as well

1275855

Unpoison custom chunk_alloc as well

491940e

Add comment

4dd7237

cs

bebc904

cs

15e2df0

Add some docs

3182c4f

Simplify

4a30e71

Simplify

d944845

Fix

2f79a43

Rm redundant unpoison

541098a

Fixup

3085dc1

Fix

607cf2b

danog added 2 commits July 23, 2025 12:22

Fix

587922b

Fix

246b275

Add ASAN support to the zend allocator #18858

Are you sure you want to change the base?

Add ASAN support to the zend allocator #18858

Conversation

danog commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danog commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nielsdos left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

danog commented Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nielsdos commented Jul 7, 2025

Uh oh!

Uh oh!

Uh oh!

nielsdos left a comment

Choose a reason for hiding this comment

Uh oh!

danog commented Jul 16, 2025

Uh oh!

nielsdos commented Jul 21, 2025

Uh oh!

iluuu1994 commented Jul 21, 2025

Uh oh!

iluuu1994 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

iluuu1994 Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

danog Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iluuu1994 Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

danog Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

iluuu1994 Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

danog Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

iluuu1994 Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

danog Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

iluuu1994 commented Jul 23, 2025

Uh oh!

danog commented Jul 23, 2025

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

danog commented Jun 16, 2025 •

edited

Loading

danog commented Jun 17, 2025 •

edited

Loading

danog commented Jul 7, 2025 •

edited

Loading

danog Jul 23, 2025 •

edited

Loading