Skip to content

Commit 8a08d61

Browse files
woodruffwwoodruffw
authored
Expose PEP 740 attestations functionality
PR #236 This patch adds PEP 740 attestation generation to the workflow: when the Trusted Publishing flow is used, this will generate a publish attestation for each distribution being uploaded. These generated attestations are then fed into `twine`, which newly supports them via `--attestations`. Ref: pypi/warehouse#15871
1 parent fb9fc6a commit 8a08d61

File tree

7 files changed

+274
-8
lines changed

7 files changed

+274
-8
lines changed

‎Dockerfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ COPY LICENSE.md .
2828
COPY twine-upload.sh .
2929
COPY print-hash.py .
3030
COPY oidc-exchange.py .
31+
COPY attestations.py .
3132

3233
RUN chmod +x twine-upload.sh
3334
ENTRYPOINT ["/app/twine-upload.sh"]

‎README.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,31 @@ filter to the job:
9999
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
100100
```
101101

102+
### Generating and uploading attestations
103+
104+
> [!IMPORTANT]
105+
> Support for generating and uploading [digital attestations] is currently
106+
> experimental and limited only to Trusted Publishing flows using PyPI or TestPyPI.
107+
> Support for this feature is not yet stable; the settings and behavior described
108+
> below may change without prior notice.
109+
110+
> [!NOTE]
111+
> Generating and uploading digital attestations currently requires
112+
> authentication with a [trusted publisher].
113+
114+
You can generate signed [digital attestations] for all the distribution files and
115+
upload them all together by enabling the `attestations` setting:
116+
117+
```yml
118+
with:
119+
attestations: true
120+
```
121+
122+
This will use [Sigstore] to create attestation
123+
objects for each distribution package, signing them with the identity provided
124+
by the GitHub's OIDC token associated with the current workflow. This means
125+
both the trusted publishing authentication and the attestations are tied to the
126+
same identity.
102127

103128
## Non-goals
104129

@@ -287,3 +312,7 @@ https://github.com/vshymanskyy/StandWithUkraine/blob/main/docs/README.md
287312
[configured on PyPI]: https://docs.pypi.org/trusted-publishers/adding-a-publisher/
288313

289314
[how to specify username and password]: #specifying-a-different-username
315+
316+
[digital attestations]: https://peps.python.org/pep-0740/
317+
[Sigstore]: https://www.sigstore.dev/
318+
[trusted publisher]: #trusted-publishing

‎action.yml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,13 @@ inputs:
8080
Use `print-hash` instead.
8181
required: false
8282
default: 'false'
83+
attestations:
84+
description: >-
85+
[EXPERIMENTAL]
86+
Enable experimental support for PEP 740 attestations.
87+
Only works with PyPI and TestPyPI via Trusted Publishing.
88+
required: false
89+
default: 'false'
8390
branding:
8491
color: yellow
8592
icon: upload-cloud
@@ -95,3 +102,4 @@ runs:
95102
- ${{ inputs.skip-existing }}
96103
- ${{ inputs.verbose }}
97104
- ${{ inputs.print-hash }}
105+
- ${{ inputs.attestations }}

‎attestations.py

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
import logging
2+
import os
3+
import sys
4+
from pathlib import Path
5+
from typing import NoReturn
6+
7+
from pypi_attestations import Attestation, Distribution
8+
from sigstore.oidc import IdentityError, IdentityToken, detect_credential
9+
from sigstore.sign import Signer, SigningContext
10+
11+
# Be very verbose.
12+
sigstore_logger = logging.getLogger('sigstore')
13+
sigstore_logger.setLevel(logging.DEBUG)
14+
sigstore_logger.addHandler(logging.StreamHandler())
15+
16+
_GITHUB_STEP_SUMMARY = Path(os.getenv('GITHUB_STEP_SUMMARY'))
17+
18+
# The top-level error message that gets rendered.
19+
# This message wraps one of the other templates/messages defined below.
20+
_ERROR_SUMMARY_MESSAGE = """
21+
Attestation generation failure:
22+
23+
{message}
24+
25+
You're seeing this because the action attempted to generated PEP 740
26+
attestations for its inputs, but failed to do so.
27+
"""
28+
29+
# Rendered if OIDC identity token retrieval fails for any reason.
30+
_TOKEN_RETRIEVAL_FAILED_MESSAGE = """
31+
OpenID Connect token retrieval failed: {identity_error}
32+
33+
This failure occurred after a successful Trusted Publishing Flow,
34+
suggesting a transient error.
35+
""" # noqa: S105; not a password
36+
37+
38+
def die(msg: str) -> NoReturn:
39+
with _GITHUB_STEP_SUMMARY.open('a', encoding='utf-8') as io:
40+
print(_ERROR_SUMMARY_MESSAGE.format(message=msg), file=io)
41+
42+
# HACK: GitHub Actions' annotations don't work across multiple lines naively;
43+
# translating `\n` into `%0A` (i.e., HTML percent-encoding) is known to work.
44+
# See: https://github.com/actions/toolkit/issues/193
45+
msg = msg.replace('\n', '%0A')
46+
print(f'::error::Attestation generation failure: {msg}', file=sys.stderr)
47+
sys.exit(1)
48+
49+
50+
def debug(msg: str):
51+
print(f'::debug::{msg}', file=sys.stderr)
52+
53+
54+
def collect_dists(packages_dir: Path) -> list[Path]:
55+
# Collect all sdists and wheels.
56+
dist_paths = [sdist.resolve() for sdist in packages_dir.glob('*.tar.gz')]
57+
dist_paths.extend(whl.resolve() for whl in packages_dir.glob('*.whl'))
58+
59+
# Make sure everything that looks like a dist actually is one.
60+
# We do this up-front to prevent partial signing.
61+
if (invalid_dists := [path for path in dist_paths if path.is_file()]):
62+
invalid_dist_list = ', '.join(map(str, invalid_dists))
63+
die(
64+
'The following paths look like distributions but '
65+
f'are not actually files: {invalid_dist_list}',
66+
)
67+
68+
return dist_paths
69+
70+
71+
def attest_dist(dist_path: Path, signer: Signer) -> None:
72+
# We are the publishing step, so there should be no pre-existing publish
73+
# attestation. The presence of one indicates user confusion.
74+
attestation_path = Path(f'{dist_path}.publish.attestation')
75+
if attestation_path.exists():
76+
die(f'{dist_path} already has a publish attestation: {attestation_path}')
77+
78+
dist = Distribution.from_file(dist_path)
79+
attestation = Attestation.sign(signer, dist)
80+
81+
attestation_path.write_text(attestation.model_dump_json(), encoding='utf-8')
82+
debug(f'saved publish attestation: {dist_path=} {attestation_path=}')
83+
84+
85+
def get_identity_token() -> IdentityToken:
86+
# Will raise `sigstore.oidc.IdentityError` if it fails to get the token
87+
# from the environment or if the token is malformed.
88+
# NOTE: audience is always sigstore.
89+
oidc_token = detect_credential()
90+
return IdentityToken(oidc_token)
91+
92+
93+
def main() -> None:
94+
packages_dir = Path(sys.argv[1])
95+
96+
try:
97+
identity = get_identity_token()
98+
except IdentityError as identity_error:
99+
# NOTE: We only perform attestations in trusted publishing flows, so we
100+
# don't need to re-check for the "PR from fork" error mode, only
101+
# generic token retrieval errors. We also render a simpler error,
102+
# since permissions can't be to blame at this stage.
103+
die(_TOKEN_RETRIEVAL_FAILED_MESSAGE.format(identity_error=identity_error))
104+
105+
dist_paths = collect_dists(packages_dir)
106+
107+
with SigningContext.production().signer(identity, cache=True) as s:
108+
debug(f'attesting to dists: {dist_paths}')
109+
for dist_path in dist_paths:
110+
attest_dist(dist_path, s)
111+
112+
113+
if __name__ == '__main__':
114+
main()

‎requirements/runtime.in

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,14 @@
11
twine
22

3-
# NOTE: Used to detect an ambient OIDC credential for OIDC publishing.
3+
# NOTE: Used to detect an ambient OIDC credential for OIDC publishing,
4+
# NOTE: as well as PEP 740 attestations.
45
id ~= 1.0
56

67
# NOTE: This is pulled in transitively through `twine`, but we also declare
78
# NOTE: it explicitly here because `oidc-exchange.py` uses it.
89
# Ref: https://github.com/di/id
910
requests
11+
12+
# NOTE: Used to generate attestations.
13+
pypi-attestations ~= 0.0.11
14+
sigstore ~= 3.2.0

‎requirements/runtime.txt

Lines changed: 75 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,16 +6,41 @@
66
#
77
annotated-types==0.6.0
88
# via pydantic
9+
betterproto==2.0.0b6
10+
# via sigstore-protobuf-specs
911
certifi==2024.2.2
1012
# via requests
13+
cffi==1.16.0
14+
# via cryptography
1115
charset-normalizer==3.3.2
1216
# via requests
17+
cryptography==42.0.7
18+
# via
19+
# pyopenssl
20+
# pypi-attestations
21+
# sigstore
22+
dnspython==2.6.1
23+
# via email-validator
1324
docutils==0.21.2
1425
# via readme-renderer
26+
email-validator==2.1.1
27+
# via pydantic
28+
grpclib==0.4.7
29+
# via betterproto
30+
h2==4.1.0
31+
# via grpclib
32+
hpack==4.0.0
33+
# via h2
34+
hyperframe==6.0.1
35+
# via h2
1536
id==1.4.0
16-
# via -r runtime.in
37+
# via
38+
# -r runtime.in
39+
# sigstore
1740
idna==3.7
18-
# via requests
41+
# via
42+
# email-validator
43+
# requests
1944
importlib-metadata==7.1.0
2045
# via twine
2146
jaraco-classes==3.4.0
@@ -34,33 +59,77 @@ more-itertools==10.2.0
3459
# via
3560
# jaraco-classes
3661
# jaraco-functools
62+
multidict==6.0.5
63+
# via grpclib
3764
nh3==0.2.17
3865
# via readme-renderer
66+
packaging==24.1
67+
# via pypi-attestations
3968
pkginfo==1.10.0
4069
# via twine
70+
platformdirs==4.2.2
71+
# via sigstore
72+
pyasn1==0.6.0
73+
# via sigstore
74+
pycparser==2.22
75+
# via cffi
4176
pydantic==2.7.1
42-
# via id
77+
# via
78+
# id
79+
# pypi-attestations
80+
# sigstore
81+
# sigstore-rekor-types
4382
pydantic-core==2.18.2
4483
# via pydantic
4584
pygments==2.18.0
4685
# via
4786
# readme-renderer
4887
# rich
88+
pyjwt==2.8.0
89+
# via sigstore
90+
pyopenssl==24.1.0
91+
# via sigstore
92+
pypi-attestations==0.0.11
93+
# via -r runtime.in
94+
python-dateutil==2.9.0.post0
95+
# via betterproto
4996
readme-renderer==43.0
5097
# via twine
51-
requests==2.32.0
98+
requests==2.32.3
5299
# via
53100
# -r runtime.in
54101
# id
55102
# requests-toolbelt
103+
# sigstore
104+
# tuf
56105
# twine
57106
requests-toolbelt==1.0.0
58107
# via twine
59108
rfc3986==2.0.0
60109
# via twine
110+
rfc8785==0.1.2
111+
# via sigstore
61112
rich==13.7.1
62-
# via twine
63-
twine==5.1.0
113+
# via
114+
# sigstore
115+
# twine
116+
securesystemslib==1.0.0
117+
# via tuf
118+
sigstore==3.2.0
119+
# via
120+
# -r runtime.in
121+
# pypi-attestations
122+
sigstore-protobuf-specs==0.3.2
123+
# via
124+
# pypi-attestations
125+
# sigstore
126+
sigstore-rekor-types==0.0.13
127+
# via sigstore
128+
six==1.16.0
129+
# via python-dateutil
130+
tuf==5.0.0
131+
# via sigstore
132+
twine==5.1.1
64133
# via -r runtime.in
65134
typing-extensions==4.11.0
66135
# via

‎twine-upload.sh

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ INPUT_PACKAGES_DIR="$(get-normalized-input 'packages-dir')"
3939
INPUT_VERIFY_METADATA="$(get-normalized-input 'verify-metadata')"
4040
INPUT_SKIP_EXISTING="$(get-normalized-input 'skip-existing')"
4141
INPUT_PRINT_HASH="$(get-normalized-input 'print-hash')"
42+
INPUT_ATTESTATIONS="$(get-normalized-input 'attestations')"
4243

4344
PASSWORD_DEPRECATION_NUDGE="::error title=Password-based uploads disabled::\
4445
As of 2024, PyPI requires all users to enable Two-Factor \
@@ -53,7 +54,37 @@ environments like GitHub Actions without needing to use username/password \
5354
combinations or API tokens to authenticate with PyPI. Read more: \
5455
https://docs.pypi.org/trusted-publishers"
5556

56-
if [[ "${INPUT_USER}" == "__token__" && -z "${INPUT_PASSWORD}" ]] ; then
57+
ATTESTATIONS_WITHOUT_TP_WARNING="::warning title=attestations input ignored::\
58+
The workflow was run with the 'attestations: true' input, but an explicit \
59+
password was also set, disabling Trusted Publishing. As a result, the \
60+
attestations input is ignored."
61+
62+
ATTESTATIONS_WRONG_INDEX_WARNING="::warning title=attestations input ignored::\
63+
The workflow was run with 'attestations: true' input, but the specified \
64+
repository URL does not support PEP 740 attestations. As a result, the \
65+
attestations input is ignored."
66+
67+
[[ "${INPUT_USER}" == "__token__" && -z "${INPUT_PASSWORD}" ]] \
68+
&& TRUSTED_PUBLISHING=true || TRUSTED_PUBLISHING=false
69+
70+
if [[ "${INPUT_ATTESTATIONS}" != "false" ]] ; then
71+
# Setting `attestations: true` without Trusted Publishing indicates
72+
# user confusion, since attestations (currently) require it.
73+
if ! "${TRUSTED_PUBLISHING}" ; then
74+
echo "${ATTESTATIONS_WITHOUT_TP_WARNING}"
75+
INPUT_ATTESTATIONS="false"
76+
fi
77+
78+
# Setting `attestations: true` with an index other than PyPI or TestPyPI
79+
# indicates user confusion, since attestations are not supported on other
80+
# indices presently.
81+
if [[ ! "${INPUT_REPOSITORY_URL}" =~ pypi\.org ]] ; then
82+
echo "${ATTESTATIONS_WRONG_INDEX_WARNING}"
83+
INPUT_ATTESTATIONS="false"
84+
fi
85+
fi
86+
87+
if "${TRUSTED_PUBLISHING}" ; then
5788
# No password supplied by the user implies that we're in the OIDC flow;
5889
# retrieve the OIDC credential and exchange it for a PyPI API token.
5990
echo "::debug::Authenticating to ${INPUT_REPOSITORY_URL} via Trusted Publishing"
@@ -130,6 +161,15 @@ if [[ ${INPUT_VERBOSE,,} != "false" ]] ; then
130161
TWINE_EXTRA_ARGS="--verbose $TWINE_EXTRA_ARGS"
131162
fi
132163

164+
if [[ ${INPUT_ATTESTATIONS,,} != "false" ]] ; then
165+
# NOTE: Intentionally placed after `twine check`, to prevent attestation
166+
# NOTE: generation on distributions with invalid metadata.
167+
echo "::notice::Generating and uploading digital attestations"
168+
python /app/attestations.py "${INPUT_PACKAGES_DIR%%/}"
169+
170+
TWINE_EXTRA_ARGS="--attestations $TWINE_EXTRA_ARGS"
171+
fi
172+
133173
if [[ ${INPUT_PRINT_HASH,,} != "false" || ${INPUT_VERBOSE,,} != "false" ]] ; then
134174
python /app/print-hash.py ${INPUT_PACKAGES_DIR%%/}
135175
fi

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy