Skip to content

xattr support missing in os module under cygwin #90026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
enok2 mannequin opened this issue Nov 22, 2021 · 12 comments
Closed

xattr support missing in os module under cygwin #90026

enok2 mannequin opened this issue Nov 22, 2021 · 12 comments
Labels
3.13 bugs and security fixes build The build process and cross-build OS-windows type-bug An unexpected behavior, bug, or error

Comments

@enok2
Copy link
Mannequin

enok2 mannequin commented Nov 22, 2021 •
edited by bedevere-bot
Loading

BPO 45868
Nosy @iritkatriel
Superseder
  • bpo-45537: Cygwin is unsupported - close all open issues and list them here.
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2022-01-23.20:10:01.267>
created_at = <Date 2021-11-22.12:39:12.588>
labels = ['type-bug', '3.8', 'build']
title = 'xattr support missing in os module under cygwin'
updated_at = <Date 2022-01-23.20:10:01.266>
user = 'https://bugs.python.org/enok2'

    bugs.python.org fields:

    activity = <Date 2022-01-23.20:10:01.266>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2022-01-23.20:10:01.267>
closer = 'iritkatriel'
components = ['Build']
creation = <Date 2021-11-22.12:39:12.588>
creator = 'enok2'
dependencies = []
files = []
hgrepos = []
issue_num = 45868
keywords = []
message_count = 4.0
messages = ['406768', '411414', '411415', '411417']
nosy_count = 2.0
nosy_names = ['iritkatriel', 'enok2']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = '45537'
type = 'behavior'
url = 'https://bugs.python.org/issue45868'
versions = ['Python 3.8']

    Linked PRs
    @enok2
    Copy link
    Mannequin Author

    enok2 mannequin commented Nov 22, 2021

    With the latest Python 3.8.10 under cygwin the following fails:

    >>> import os
>>> os.listxattr('.')
Traceback (most recent call last):
  File "<stdin>", line 1, in < module>
AttributeError: module 'os' has no attribute 'listxattr'

    /usr/include/attr/xattr.h declares the corresponding library functions, so something must be wrong in the configuration/compilation of python itself.

    @enok2 enok2 mannequin added type-crash A hard crash of the interpreter, possibly with a core dump 3.8 (EOL) end of life build The build process and cross-build labels Nov 22, 2021
    @iritkatriel
    Copy link
    Member

    This is listed in the docs as a linux-only feature:

    https://docs.python.org/3/library/os.html#linux-extended-attributes

    @iritkatriel iritkatriel added invalid type-bug An unexpected behavior, bug, or error and removed type-crash A hard crash of the interpreter, possibly with a core dump labels Jan 23, 2022
    @enok2
    Copy link
    Mannequin Author

    enok2 mannequin commented Jan 23, 2022

    Perhaps it could be supported under cygwin though without too much effort? Most likely just a configuration fix?

    After all, cygwin is a sort of linux ...

    @iritkatriel
    Copy link
    Member

    There isn't currently a maintainer for cpython on cygwin. If someone volunteers for this, they can go through the issues listed at the bottom of bpo-45537.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @philcerf
    Copy link
    Contributor

    philcerf commented May 17, 2023 •
    edited
    Loading

    @iritkatriel Any chance to revisit this?

    It seems to me as if Cygwin would just provide the Linux xattr syscalls: https://github.com/search?q=org%3Acygwin%20getxattr&type=code

    If so, wouldn't it be enough to simply have USE_XATTRS defined for Cygwin e.g. at:
    https://github.com/python/cpython/blob/main/Modules/posixmodule.c#LL289C1-L292C2

    Sure that wouldn't be full maintenance of Cygwin in Python,... but some more would perhaps work?

    @iritkatriel
    Copy link
    Member

    If you want to submit a PR, hopefully someone can review it. I'm a Mac user myself, so I can't really help with Cygwin.

    @philcerf
    Copy link
    Contributor

    philcerf commented May 30, 2023 •
    edited
    Loading

    Hey there.

    I played a bit on this with the following results so far:

    First, right now, cygwin contains only Python up to 3.9.9, where we have:

    cpython/Modules/posixmodule.c

    Line 244 in ccb0e6a

    #if defined(HAVE_SYS_XATTR_H) && defined(__GLIBC__) && !defined(__FreeBSD_kernel__) && !defined(__GNU__)

    Here, the check for __GLIBC__ seems the first offender, cause when I remove that, it actually tries to compile the code but then fails with:

    gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong --param=ssp-buffer-size=4 -DOPENSSL_NO_SSL3=1 -I/usr/include/uuid/ -fdebug-prefix-map=/home/calestyo/u/x/python39-3.9.9-1.x86_64/build=/usr/src/debug/python39-3.9.9-1 -fdebug-prefix-map=/home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9=/usr/src/debug/python39-3.9.9-1 -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong --param=ssp-buffer-size=4 -DOPENSSL_NO_SSL3=1 -I/usr/include/uuid/ -fdebug-prefix-map=/home/calestyo/u/x/python39-3.9.9-1.x86_64/build=/usr/src/debug/python39-3.9.9-1 -fdebug-prefix-map=/home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9=/usr/src/debug/python39-3.9.9-1  -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I/home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Include/internal -IObjects -IInclude -IPython -I. -I/home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Include    -DPy_BUILD_CORE_BUILTIN  -DPy_BUILD_CORE_BUILTIN -I/home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Include/internal -c /home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Modules/posixmodule.c -o Modules/posixmodule.o
sed -e "s,@EXENAME@,/usr/bin/python3.9.exe," < /home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Misc/python-config.in >python-config.py
LC_ALL=C sed -e 's,\$(\([A-Za-z0-9_]*\)),\$\{\1\},g' < Misc/python-config.sh >python-config
/home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Modules/posixmodule.c: In function ‘os_getxattr_impl’:
/home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Modules/posixmodule.c:12634:56: error: ‘XATTR_SIZE_MAX’ undeclared (first use in this function); did you mean ‘PY_SIZE_MAX’?
12634 |         static const Py_ssize_t buffer_sizes[] = {128, XATTR_SIZE_MAX, 0};
      |                                                        ^~~~~~~~~~~~~~
      |                                                        PY_SIZE_MAX
/home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Modules/posixmodule.c:12634:56: note: each undeclared identifier is reported only once for each function it appears in
/home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Modules/posixmodule.c: In function ‘os_listxattr_impl’:
/home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Modules/posixmodule.c:12814:57: error: ‘XATTR_LIST_MAX’ undeclared (first use in this function)
12814 |         static const Py_ssize_t buffer_sizes[] = { 256, XATTR_LIST_MAX, 0 };
      |                                                         ^~~~~~~~~~~~~~
In file included from /home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Include/Python.h:137,
                 from /home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Modules/posixmodule.c:12:
/home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Modules/posixmodule.c: In function ‘all_ins’:
/home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Modules/posixmodule.c:15113:33: error: ‘XATTR_SIZE_MAX’ undeclared (first use in this function); did you mean ‘PY_SIZE_MAX’?
15113 |     if (PyModule_AddIntMacro(m, XATTR_SIZE_MAX)) return -1;
      |                                 ^~~~~~~~~~~~~~
/home/calestyo/u/x/python39-3.9.9-1.x86_64/src/Python-3.9.9/Include/modsupport.h:146:67: note: in definition of macro ‘PyModule_AddIntMacro’
  146 | #define PyModule_AddIntMacro(m, c) PyModule_AddIntConstant(m, #c, c)
      |                                                                   ^
make: *** [Makefile:1967: Modules/posixmodule.o] Error 1
make: *** Waiting for unfinished jobs....
*** ERROR: make failed

    and would further fail with a missing XATTR_LIST_MAX, which is because Cygwin doesn't seem to define those two, while e.g. Linux does.

    Meanwhile 8be8101 changed the check to:

    cpython/Modules/posixmodule.c

    Line 277 in 8be8101

    #if defined(HAVE_SYS_XATTR_H) && defined(__linux__) && !defined(__FreeBSD_kernel__) && !defined(__GNU__)

    use __linux__ instead.

    It seems that this is in fact defined by Cygwin (oddly enough), cause when I change the 3.9.9 sources to the new pre processor check and build the Cygwin package, it fails again with the missing XATTR_SIZE_MAX and XATTR_LIST_MAX.

    And if I use e.g.:

    #if (defined(HAVE_SYS_XATTR_H) && defined(__linux__) && !defined(__FreeBSD_kernel__) && !defined(__GNU__)) || defined(__CYGWIN__)
#  define USE_XATTRS
#  define XATTR_SIZE_MAX 65536
#  define XATTR_LIST_MAX 65536
#endif

    it compiles and at least os.getxattr() gives correct data on some test file.

    Of course the *_MAX values may be wrong for Cygwin... I simply took those from Linux.

    I'd guess that it's Cygwin’s duty to properly export XATTR_SIZE_MAX and XATTR_LIST_MAX and will ask them on thir mailing list.

    On the Python side I think we should still add an alternative for __CYGWIN__... I mean it does seem to work there (except for the missing defines, but Cygwin shouldn't be covered by __linux__ should it?

    I'll also ask at the Cygwin ml, whether it's expected to export that symbol... if so, we can just close this (respectively remove it from the meta-bug) as fixed from the Python side... if not I'll do a PR.

    @philcerf
    Copy link
    Contributor

    Just checked again and seems I had some glitch somewhere... Cygwin does not seem to export __linux__.

    Also, if I set the missing defines manually in Cygwin’s /usr/include/cygwin/limits.h they're correctly picked up as expected.

    @philcerf
    Copy link
    Contributor

    btw: A PR will follow, may just take a while since I'll be on some official trip (with no Windows/Cygwin machine at hand) and on vacation right afterwards.

    philcerf added a commit to philcerf/cpython that referenced this issue May 30, 2023
    @philcerf
    pythongh-90026: support XATTRs on Cygwin (python#90026)
    99fa6b0
    @philcerf philcerf mentioned this issue May 30, 2023
    Merged
    @philcerf
    Copy link
    Contributor

    @iritkatriel Since I'm working on getting this done, can you please re-open this issue?

    @iritkatriel iritkatriel reopened this May 30, 2023
    @arhadthedev arhadthedev added 3.13 bugs and security fixes and removed 3.8 (EOL) end of life labels May 30, 2023
    philcerf added a commit to philcerf/cpython that referenced this issue May 30, 2023
    @philcerf
    pythongh-90026: support XATTRs on Cygwin (python#90026)
    a6b7599
    @philcerf
    Copy link
    Contributor

    Hey @iritkatriel.

    Since you're core developer... any chance that you could perhaps review my proposed PR for this? 😇
    Would be awesome.

    Thanks,
    Philippe

    philcerf added a commit to philcerf/cpython that referenced this issue Aug 14, 2023
    @philcerf
    pythongh-90026: support XATTRs on Cygwin (python#90026)
    95e0632
    philcerf added a commit to philcerf/cpython that referenced this issue Oct 21, 2023
    @philcerf
    pythongh-90026: support XATTRs on Cygwin (python#90026)
    3b1bf34
    philcerf added a commit to philcerf/cpython that referenced this issue Nov 2, 2023
    @philcerf
    pythongh-90026: support XATTRs on Cygwin (python#90026)
    1b74ef1
    encukou pushed a commit that referenced this issue Nov 3, 2023
    @philcerf
    gh-90026: support XATTRs on Cygwin (GH-105075)
    7810b69
    @encukou
    Copy link
    Member

    encukou commented Nov 3, 2023

    Thanks for the fix!

    For reference: Cygwin is unsupported platform, meaning we don't test it. The feature may break in the future.

    @encukou encukou closed this as completed Nov 3, 2023
    aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
    @philcerf @aisk
    pythongh-90026: support XATTRs on Cygwin (pythonGH-105075)
    23c76e6
    Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024
    @philcerf @Glyphack
    pythongh-90026: support XATTRs on Cygwin (pythonGH-105075)
    78a0e2c
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.13 bugs and security fixes build The build process and cross-build OS-windows type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @encukou @iritkatriel @arhadthedev @philcerf
    pFad - Phonifier reborn

    Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

    Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


    Alternative Proxies:

    Alternative Proxy

    pFad Proxy

    pFad v3 Proxy

    pFad v4 Proxy