Add ASAN support to the zend allocator #18858

danog · 2025-06-16T12:12:21Z

This pull request adds ASAN support to the zend allocator, by automatically poisoning all unused pages, chunks and heap management structures before exiting the alloc, free, etc (all ZEND_API) functions.

Internally, the implementation uses the following rules:

Always poison memory (re)allocated by private (non-ZEND_API) allocation functions before returning it
Unpoison memory (re)allocated by public (ZEND_API) allocation functions before returning within the ZEND_API function
Always poison freed memory
Always poison unused memory during reallocation (where new_size < old_size)
When accessing private heap structures and fields, always unpoison before accessing and repoison immediately after
- An exception to the above (for simplicity) is the main heap datastructure, which is poisoned only when entering a ZEND_API function and repoisoned before exiting

This is what allowed me to find #18833, before I found the fast shutdown workaround.

danog · 2025-06-17T13:06:28Z

@dstogov / @nielsdos / @iluuu1994 This is now ready for review!

The implementation is finalized, and has actually revealed a shutdown memory leak in php-fpm when terminated via a SIGTERM, due to the fact that alloc_globals_dtor is not getting invoked (the leak is surprisingly revealed exclusively by the usage of the poison functions, in fact, altering the ZEND_MM_POISON/ZEND_MM_UNPOISON macros to be no-ops re-hides the issue again, i.e. it's not caused by the other code changes/reorganization).

This PR also includes the changes from #18834, as the bug is detectable by ASAN now.

Side note, tracked_malloc also poisons/unpoisons the heap to avoid issues with the observe proxies.

danog · 2025-07-07T11:12:03Z

@nielsdos what compiler, platform, configure flags and memory manager envvars are you using?

nielsdos · 2025-07-07T18:25:59Z

what compiler, platform, configure flags and memory manager envvars are you using?

It seems I was testing on an older checkout, I tested it again with a reset to your branch and it works, sorry for the confusion...

Zend/zend_alloc.c

nielsdos

Besides the remaining cs nit I don't see anything obviously wrong. Thanks for the clarifications. I think this patch is worth it. I'll leave this open for some time to see if others want to review this too.

danog · 2025-07-16T09:01:06Z

Ping? :)

nielsdos · 2025-07-21T12:58:11Z

cc @iluuu1994 ?

iluuu1994 · 2025-07-21T12:59:40Z

@nielsdos I'll have a look in a few hours.

iluuu1994

Only approx. halfway through (line 1900 old diff). Overall this seems quite complex and easy to make mistakes. This can't detect buffer overflows for two adjacent allocations within the same bin. I'm also not sure whether this can catch more errors in the allocator itself, as pointers are explicitly unpoisoned before usage, so even if a pointer were wrong, it would just be unpoisoned.

But if others think this should be merged, we should have at least one ASAN build in nightly that doesn't set the USE_ZEND_ALLOC=0 env var. It would also be good to run a test COMMUNITY build to reveal any remaining errors.

Zend/zend_alloc.c

iluuu1994 · 2025-07-22T22:45:56Z

Zend/zend_alloc.c

 	} while (0);
 #endif

 	p = (zend_mm_free_slot*)ptr;
 	zend_mm_set_next_free_slot(heap, bin_num, p, heap->free_slot[bin_num]);
 	heap->free_slot[bin_num] = p;
+
+	ZEND_MM_POISON(p, bin_data_size[bin_num]);


p should already be poisoned.

Only the header and shadow in zend_mm_set_next_free_slot, not the body

iluuu1994 · 2025-07-22T22:51:50Z

Zend/zend_alloc.c

@@ -1586,6 +1812,8 @@ static zend_never_inline void *zend_mm_realloc_slow(zend_mm_heap *heap, void *pt
 		size_t orig_peak = heap->peak;
 #endif
 		ret = zend_mm_alloc_heap(heap, size ZEND_FILE_LINE_RELAY_CC ZEND_FILE_LINE_ORIG_RELAY_CC);
+		ZEND_MM_UNPOISON(ret, size);
+		ZEND_MM_UNPOISON(ptr, copy_size);


ptr should be unpoisoned already.

copy_size is set to the size of the bin if size > old_size, size > size of the bin, old_size < size of the bin, so the unpoison is needed.

iluuu1994 · 2025-07-22T22:54:30Z

Zend/zend_alloc.c

@@ -1627,6 +1855,7 @@ static zend_never_inline void *zend_mm_realloc_huge(zend_mm_heap *heap, void *pt
 #else
 			zend_mm_change_huge_block_size(heap, ptr, new_size ZEND_FILE_LINE_RELAY_CC ZEND_FILE_LINE_ORIG_RELAY_CC);
 #endif
+			ZEND_MM_POISON(ptr, new_size);


Why do we poison here? Edit: I see, it's unpoisoned right away in zend_mm_realloc_heap. Not sure if that's useful.

It just follows the implementation rules

Zend/zend_alloc.c

iluuu1994 · 2025-07-22T23:01:21Z

Zend/zend_alloc.c

 						memcpy(ret, ptr, copy_size);
 						zend_mm_free_small(heap, ptr, old_bin_num);
 					} else {
 						/* reallocation in-place */
 						ret = ptr;
+						ZEND_MM_UNPOISON(ret, size);


Should already be unpoisoned. If anything, the upper bytes should be poisoned.

iluuu1994 · 2025-07-23T09:52:39Z

I'm also not sure whether this can catch more errors in the allocator itself, as pointers are explicitly unpoisoned before usage, so even if a pointer were wrong, it would just be unpoisoned.

Regarding this part: My fear is that, given most accesses unpoison/poison memory very locally, this reverts a lot of the benefits of poisoning. If an access is incorrect, it's very likely for the unpoisoning itself to also be incorrect, whether by depending on an incorrect parameter or applying incorrect logic to both unpoisoning and access. I will discuss this with Arnaud once he's back from vacation, so at some point next week.

Thanks!

danog · 2025-07-23T10:05:29Z

Regarding this part: My fear is that, given most accesses unpoison/poison memory very locally, this reverts a lot of the benefits of poisoning. If an access is incorrect, it's very likely for the unpoisoning itself to also be incorrect, whether by depending on an incorrect parameter or applying incorrect logic to both unpoisoning and access. I will discuss this with Arnaud once he's back from vacation, so at some point next week.

The main class of errors this PR detects is use after frees; buffer overflows are also detected to some extent, but for better detection vanilla ASAN should be used.

However, I have found in my experience that use after frees are the most common type of memory corruption in PHP, mainly due to refcounting bugs, and this PR helps detect those without using the more expensive and slow vanilla ASAN.

arnaud-lb · 2025-07-25T15:19:02Z

@iluuu1994

This can't detect buffer overflows for two adjacent allocations within the same bin

It does in debug builds due to the debug info surrounding the allocation being poisoned (in addition to any alignment padding or regions past the requested size). I think this is similar in effect to ASAN's red zones, except the zones are fixed size in our case and are not guaranteed to exist before the first block in a bin, or after the last block.

I think the main benefit of this change is not to detect bugs in the allocator itself, but to be able to spot memory management issues without disabling Zend MM. This enables us to run ASAN builds in conditions that are closer to those of non-ASAN builds.

iluuu1994 · 2025-07-25T16:11:26Z

I think the main benefit of this change is not to detect bugs in the allocator itself, but to be able to spot memory management issues without disabling Zend MM.

But is there a big incentive to compile with ASAN but not to use USE_ZEND_ALLOC=0? With USE_TRACKED_ALLOC=1, there aren't too many differences between the two modes in behavior outside of the allocator itself (correct me if I'm wrong). Granted, it's nice to have one less configuration to think about, but this could also be solved by settings these environment variables by default when compiling with ASAN.

I won't object to this, but I am currently failing to see the benefit. In any case, this needs better CI coverage. And I will need to review the second half of this PR. 🙂 And as mentioned, it would be good to see a COMMUNITY build pass without Zend MM being enabled.

github-actions bot added the Category: Engine label Jun 16, 2025

ASAN support for the zend allocator

2f84241

danog force-pushed the asan_zend_alloc branch from 197f4a3 to 2f84241 Compare June 16, 2025 12:14

danog added 18 commits June 16, 2025 17:49

Fix

1880729

Fix

193962c

Fixes

1a259a0

Fixes

4ea8a03

Fixes

ef9a0d2

Improvements

0376092

Improvements

340d985

Improvements

e800e53

Improvements

12eefe4

Fix

6563269

Cleanup

61661d3

Cleanup

4301651

Fix

f11d436

Fix

bba7468

Cleanup

b1b7c1f

Cleanup

835810c

Fixes

c576965

Hotfix newly detected memory leak

e31c4ac

github-actions bot added the SAPI: fpm label Jun 17, 2025

danog added 3 commits June 17, 2025 14:17

Fixes

d9de798

Fixes

e5ca924

Final

e88ec6d

danog force-pushed the asan_zend_alloc branch from 488d33d to e88ec6d Compare June 17, 2025 12:25

danog marked this pull request as ready for review June 17, 2025 13:06

danog requested review from dstogov and bukka as code owners June 17, 2025 13:06

danog added 2 commits July 7, 2025 11:11

cs fixes

38e7896

cs fixes

1875711

nielsdos reviewed Jul 7, 2025

View reviewed changes

Zend/zend_alloc.c Show resolved Hide resolved

Zend/zend_alloc.c Outdated Show resolved Hide resolved

nielsdos approved these changes Jul 7, 2025

View reviewed changes

Cs fixes

40936ff

iluuu1994 reviewed Jul 22, 2025

View reviewed changes

danog added 12 commits July 23, 2025 10:28

cs-fix

832dae0

Poison custom chunk_alloc as well

1275855

Unpoison custom chunk_alloc as well

491940e

Add comment

4dd7237

cs

bebc904

cs

15e2df0

Add some docs

3182c4f

Simplify

4a30e71

Simplify

d944845

Fix

2f79a43

Rm redundant unpoison

541098a

Fixup

3085dc1

Fix

607cf2b

danog added 2 commits July 23, 2025 12:22

Fix

587922b

Fix

246b275

Add ASAN support to the zend allocator #18858

Are you sure you want to change the base?

Add ASAN support to the zend allocator #18858

Conversation

danog commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danog commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danog commented Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nielsdos commented Jul 7, 2025

Uh oh!

Uh oh!

Uh oh!

nielsdos left a comment

Choose a reason for hiding this comment

Uh oh!

danog commented Jul 16, 2025

Uh oh!

nielsdos commented Jul 21, 2025

Uh oh!

iluuu1994 commented Jul 21, 2025

Uh oh!

iluuu1994 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

iluuu1994 Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

danog Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iluuu1994 Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

danog Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

iluuu1994 Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

danog Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

iluuu1994 Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

danog Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

iluuu1994 commented Jul 23, 2025

Uh oh!

danog commented Jul 23, 2025

Uh oh!

arnaud-lb commented Jul 25, 2025

Uh oh!

iluuu1994 commented Jul 25, 2025

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

danog commented Jun 16, 2025 •

edited

Loading

danog commented Jun 17, 2025 •

edited

Loading

danog commented Jul 7, 2025 •

edited

Loading

danog Jul 23, 2025 •

edited

Loading