gh-131338: Disable new stack size code for non-glibc on linux. #134336
Conversation
MUSL claims to support pthread_getattr_np, but the response returned is off by at least an order of magnitude compared to reality. This is apparently a long standing problem. MUSL also does not provide any way to detect directly that it is the libc. It would be possible to add things to configure to detect that MUSL is the compiler, but since this code has caused problems for a lot of non-glibc platforms, it seems easier to just restrict it to glibc.
Python/ceval.c
Outdated
| @@ -442,7 +442,8 @@ _Py_InitializeRecursionLimits(PyThreadState *tstate) | |||
| _tstate->c_stack_soft_limit = _tstate->c_stack_hard_limit + PYOS_STACK_MARGIN_BYTES; | |||
| #else | |||
| uintptr_t here_addr = _Py_get_machine_stack_pointer(); | |||
| # if defined(HAVE_PTHREAD_GETATTR_NP) && !defined(_AIX) && !defined(__NetBSD__) | |||
| # if defined(HAVE_PTHREAD_GETATTR_NP) && !defined(_AIX) && \ | |||
| !defined(__NetBSD__) && defined(__GLIBC__) | |||
There was a problem hiding this comment.
I don't think that this call is correct. It disables the call on FreeBSD.
Maybe check for "glibc or not Linux"?
There was a problem hiding this comment.
That would fix it for alpine, but not for emscripten or wasi (but I don't know what they actually need). I'd rather see it as if glibc or freebsd, and get rid of the rest, but glibc or not linux would work for my immediate concern.
There was a problem hiding this comment.
Well, let's start with fixing Alpine Linux.
There was a problem hiding this comment.
OK, done. Also added an XXX comment, since I think this needs more investigation. I'll put it on my todo list to post to the musl mailing list about it.
This exempts alpine but doesn't affect, eg, FreeBSD.
|
This looks good but it needs a news entry as it is a bugfix and needs to be backported to 3.14. |
Co-authored-by: Victor Stinner <vstinner@python.org>
|
I have added a news entry to move this PR forward and get this merged soon, will run through buildbots and merge. |
|
🤖 New build scheduled with the buildbot fleet by @kumaraditya303 for commit 6498d29 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F134336%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again. |
|
Kumar asked me to test libcst with this PR applied. I manually backported the substantive part of this PR to the 3.14 branch and built Python using pyenv in the musllinux docker container. I still see the segfault: https://gist.github.com/ngoldbaum/16fb37188d049edcd18296f809241798. Also looking close, I don't think the crash we saw in LibCST is caused by this, since we see it in 3.13, and I don't think the new stack size code ever went into 3.13? |
Yes, it is present in 3.14 and above. Looking at the backtrace of libcst it seems unrelated to this, a better test candidate for this would be scipy as that currently errors out with RecursionError like in scipy/scipy#23172. |
MUSL claims to support pthread_getattr_np, but the response returned is off by at least an order of magnitude compared to reality. This is apparently a long standing problem. MUSL also does not provide any way to detect directly that it is the libc. It would be possible to add things to configure to detect that MUSL is the compiler, but since this code has apparently caused problems for a lot of non-glibc platforms, it seems easier to just restrict it to glibc.