fix(pypi): allow pip_compile to work with read-only sources (bazel-contrib#2712)

rbeasley-avgo · aignas · web-flow · commit af9e959538f3 · 2025-05-30T09:25:03.000Z
The validating `py_test` generated by `compile_pip_requirements` chokes when the source `requirements.txt` is stored read-only, such as when managed by the Perforce Helix Core SCM. Though `dependency_resolver` makes a temporary copy of this file, it does so w/ `shutil.copy` which preserves the original read-only file mode. To address this, this commit replaces `shutil.copy` with a `shutil.copyfileobj` such that the temporary file is created w/ permissions according to the user's umask. Resolves (bazel-contrib#2608). --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -93,6 +93,8 @@ END_UNRELEASED_TEMPLATE
   also retrieved from the URL as opposed to only the `--hash` parameter. Fixes
   [#2363](https://github.com/bazel-contrib/rules_python/issues/2363).
 * (pypi) `whl_library` now infers file names from its `urls` attribute correctly.
+* (pypi) When running under `bazel test`, be sure that temporary `requirements` file
+  remains writable.
 * (py_test, py_binary) Allow external files to be used for main
 
 {#v0-0-0-added}
diff --git a/python/private/pypi/dependency_resolver/dependency_resolver.py b/python/private/pypi/dependency_resolver/dependency_resolver.py
@@ -151,9 +151,16 @@ def main(
         requirements_out = os.path.join(
             os.environ["TEST_TMPDIR"], os.path.basename(requirements_file) + ".out"
         )
+        # Why this uses shutil.copyfileobj:
+        #
         # Those two files won't necessarily be on the same filesystem, so we can't use os.replace
         # or shutil.copyfile, as they will fail with OSError: [Errno 18] Invalid cross-device link.
-        shutil.copy(resolved_requirements_file, requirements_out)
+        #
+        # Further, shutil.copy preserves the source file's mode, and so if
+        # our source file is read-only (the default under Perforce Helix),
+        # this scratch file will also be read-only, defeating its purpose.
+        with open(resolved_requirements_file, "rb") as fsrc, open(requirements_out, "wb") as fdst:
+            shutil.copyfileobj(fsrc, fdst)
 
     update_command = (
         os.getenv("CUSTOM_COMPILE_COMMAND") or f"bazel run {target_label_prefix}.update"
