Feature or enhancement

This is a tracking issue for implementing PEP 784. See the PEP text for more details.

Since the diff is significant (~10k lines) I wanted to split up the PRs a bit.

Implementation Plan:

• Add compression module just re-exporting existing compression modules. Move the _compression module.
• Add _zstd native module with Unix build config
• Add Windows build config for _zstd (blocked on adding libzstd to cpython-source-deps) and SBOM config.
• Add zstd Python module with tests
• add NEWS/What's New section
• Add documentation for zstd
• Decide on whether or not to check parameter types for options
• Verify style conformance of C and Python code
  • Python
  • C
• Refactor _train_dict and _finalize_dict to share common code
• Fix test_zstd failed on ubuntu with free-threading #133885
• Improve error message when options value can't be converted to int

Linked PRs

• gh-132983: Introduce compression package and move _compression module #133018
• gh-132983: Introduce _zstd bindings module #133027
• gh-132983: Install compression package contents #133063
• gh-132983: make compression._common a full-fledged package #133076
• gh-132983: Convert compression._common to a package #133086
• gh-132983: [DRAFT] Test _zstd code with Python test suite #133185
• gh-132983: [Draft] Run refleak tests against _zstd with tests #133282
• gh-132983: Add compression.zstd and Python tests #133365
• GH-132983: Build _zstd on Windows #133366
• GH-132983: Use a standard autoconf version check for zstd #133479
• gh-132983: Add What's New entry for PEP 784 implementation #133495
• GH-132983: Remove zstd version check in the header file #133502
• gh-132983: Remove pyzstd in identifiers #133535
• gh-132983: Style improvements for compression.zstd #133547
• GH-132983: Support finding libzstd without pkg-config #133550
• GH-132983: Restore libzstd fallback detection #133565
• [3.14] gh-132983: Remove pyzstd in identifiers (GH-133535) #133629
• gh-132983: Clean-ups for _zstd #133670
• gh-103092: Support subinterpreters in _zstd #133674
• GH-132983: Remove subclassing support from zstd types #133694
• [3.14] gh-103092: Support subinterpreters in _zstd (GH-133674) #133695
• gh-132983: Fix small issues with zstd support in zipfile #133723
• gh-132983: Don't allow trailer data in ZstdFile #133736
• [3.14] gh-132983: Clean-ups for _zstd (GH-133670) #133756
• [3.14] GH-132983: Restore libzstd fallback detection (GH-133565) #133757
• [3.14] GH-132983: Remove subclassing support from zstd types (GH-133694) #133762
• gh-132983: Simplify _zstd_exec() #133775
• gh-132983: Make zstd types immutable #133784
• GH-132983: remove empty_bytes from _zstd module state #133785
• [3.14] gh-132983: Simplify _zstd_exec() (GH-133775) #133786
• [3.14] GH-132983: remove empty_bytes from _zstd module state (GH-133785) #133788
• GH-132983: PEP 7 and Argument Clinic changes for zstd #133791
• [3.14] GH-132983: PEP 7 and Argument Clinic changes for zstd (GH-133791) #133792
• gh-132983: Reduce the size of _zstdmodule.h #133793
• [3.14] gh-132983: Don't allow trailer data in ZstdFile (GH-133736) #133799
• [3.14] gh-132983: Reduce the size of _zstdmodule.h (GH-133793) #133854
• gh-132983: Remove EndlessZstdDecompressor remains #133856
• [3.14] gh-132983: Make zstd types immutable (GH-133784) #133857
• [3.14] gh-132983: Remove leftovers from EndlessZstdDecompressor (GH-133856) #133859
• gh-132983: Convert zstd __new__ methods to Argument Clinic #133860
• gh-132983: Add documentation for compression.zstd #133911
• [3.14] gh-132983: Convert zstd __new__ methods to Argument Clinic (GH-133860) #133915
• gh-132983: Split _zstd_set_c_parameters #133921
• gh-132983: Convert dict_content to take Py_buffer #133924
• gh-132983: Fix zstd compiler warning about unused function mt_continue_should_break() #133947
• [3.14] gh-132983: Fix compiler warning about unused function mt_continue_should_break() (GH-133947) #133950
• gh-132983: Call Py_XDECREF rather than PyObject_GC_Del in failed __new__ #133962
• [3.14] gh-132983: Fix small issues with zstd support in zipfile (GH-133723) #133974
• [3.14] gh-132983: Style improvements for compression.zstd (GH-133547) #134001
• gh-132983: Add zstd version info to test.pythoninfo #134230
• [3.14] gh-132983: Call Py_XDECREF rather than PyObject_GC_Del in failed __new__ (GH-133962) #134305
• [3.14] gh-132983: Add documentation for compression.zstd (GH-133911) #134425
• gh-132983: Refactor shared code in train_dict and finalize_dict #134432
• [3.14] gh-132983: Refactor shared code in train_dict and finalize_dict (GH-134432) #134442
• gh-132983: Fix refleak in zstd dictionary functions #134459
• [3.14] gh-132983: Fix refleak in zstd dictionary functions (gh-134459) #134463
• gh-132983: Slightly tweak error messages for _zstd compressor/decompressor options dict #134601
• [3.14] gh-132983: Slightly tweak error messages for _zstd compressor/decompressor options dict (GH-134601) #134602
• gh-132983: Make _zstd C code PEP 7 compliant #134605
• [3.14] gh-132983: Make _zstd C code PEP 7 compliant (GH-134605) #134609
• [3.14] gh-132983: Convert dict_content to take Py_buffer in ZstdDict() (GH-133924) #134723
• [3.14] gh-132983: Split _zstd_set_c_parameters (GH-133921) #134838
• gh-132983: Minor fixes and clean up for the _zstd module #134930
• [3.14] gh-132983: Minor fixes and clean up for the _zstd module (GH-134930) #134998
• gh-132983: Add missing references to Zstandard in shutil docstrings #136617
• [3.14] gh-132983: Add missing references to Zstandard in shutil docstrings (GH-136617) #137052