Skip to content

feat: Support main_module with bootstrap_impl=system_python #3072

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

feat: Support main_module with bootstrap_impl=system_python #3072

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ef12d0a
test: Add test for main_module with bootstrap_impl=system_python
brandonchinn178 Jul 11, 2025
e87d7e8
feat: Support main_module with bootstrap_impl=system_python
brandonchinn178 Jul 11, 2025
418a70c
Address feedback
brandonchinn178 Jul 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ END_UNRELEASED_TEMPLATE
* 3.12.11
* 3.13.5
* 3.14.0b3
* (rules) {obj}`main_module` now works without needing to set {obj}`--bootstrap_impl=script`

{#v0-0-0-removed}
### Removed
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ the [Breaking Changes](#breaking-changes) section for how to introduce breaking
changes.

User visible changes, such as features, fixes, or notable refactors, should
be documneted in CHANGELOG.md and their respective API doc. See [Documenting
be documented in CHANGELOG.md and their respective API doc. See [Documenting
changes] for how to do so.

Common `type`s:
Expand Down
9 changes: 9 additions & 0 deletions python/private/main_module_entrypoint_template.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
"""
A shim to get `main_module` working with `bootstrap_impl=system_python`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of creating a shim like this, modify the bootstrap template. All it should really need is to populate a %main_module% variable and pass -m when invoking python. That should simplify things quite a bit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm I could try it again, but it was difficult because then I'd have to update all the locations where %main% is used, since there isn't a main file anymore. It seemed easier to say "we're using the main file logic everywhere still, it just points to a shim"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, it seems like the notion of a main file is hardcoded in a lot of places in python_bootstrap_template.txt. It's also not immediately clear how the translation to -m <main_module> would work for the coverage path. The shim seems like the easiest solution, especially given this is a deprecated codepath and it should have a negligible performance impact

"""

import os
import runpy

if __name__ == "__main__":
runpy.run_module("%main_module%", run_name="__main__", alter_sys=True)
40 changes: 33 additions & 7 deletions python/private/py_executable.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,10 @@ accepting arbitrary Python versions.
# empty target for other platforms.
default = "//tools/launcher:launcher",
),
"_main_module_entrypoint_template": lambda: attrb.Label(
allow_single_file = True,
default = "//python/private:main_module_entrypoint_template.py",
),
"_py_interpreter": lambda: attrb.Label(
# The configuration_field args are validated when called;
# we use the precense of py_internal to indicate this Bazel
Expand Down Expand Up @@ -360,9 +364,8 @@ def _create_executable(
)
else:
stage2_bootstrap = None
extra_runfiles = ctx.runfiles()
zip_main = ctx.actions.declare_file(base_executable_name + ".temp", sibling = executable)
_create_stage1_bootstrap(
extra_runfiles = _create_stage1_bootstrap(
ctx,
output = zip_main,
main_py = main_py,
Expand Down Expand Up @@ -426,16 +429,17 @@ def _create_executable(
if bootstrap_output != None:
fail("Should not occur: bootstrap_output should not be used " +
"when creating an executable zip")
_create_executable_zip_file(
more_runfiles = _create_executable_zip_file(
ctx,
output = executable,
zip_file = zip_file,
stage2_bootstrap = stage2_bootstrap,
runtime_details = runtime_details,
venv = venv,
)
extra_runfiles = extra_runfiles.merge(more_runfiles)
elif bootstrap_output:
_create_stage1_bootstrap(
more_runfiles = _create_stage1_bootstrap(
ctx,
output = bootstrap_output,
stage2_bootstrap = stage2_bootstrap,
Expand All @@ -445,6 +449,7 @@ def _create_executable(
main_py = main_py,
venv = venv,
)
extra_runfiles = extra_runfiles.merge(more_runfiles)
else:
# Otherwise, this should be the Windows case of launcher + zip.
# Double check this just to make sure.
Expand Down Expand Up @@ -796,6 +801,7 @@ def _create_stage1_bootstrap(
is_for_zip,
runtime_details,
venv = None):
extra_runfiles = ctx.runfiles()
runtime = runtime_details.effective_runtime

if venv:
Expand Down Expand Up @@ -833,9 +839,10 @@ def _create_stage1_bootstrap(
)
template = runtime.bootstrap_template
subs["%shebang%"] = runtime.stub_shebang
elif not ctx.files.srcs:
fail("mandatory 'srcs' files have not been provided")
else:
if not ctx.files.srcs and not ctx.attr.main_module:
fail("mandatory 'srcs' files have not been provided")

if (ctx.configuration.coverage_enabled and
runtime and
runtime.coverage_tool):
Expand All @@ -855,6 +862,20 @@ def _create_stage1_bootstrap(
subs["%coverage_tool%"] = coverage_tool_runfiles_path
subs["%import_all%"] = ("True" if ctx.fragments.bazel_py.python_import_all_repositories else "False")
subs["%imports%"] = ":".join(imports.to_list())

if ctx.attr.main_module:
main_module_entrypoint = ctx.actions.declare_file("main_module_entrypoint.py")
ctx.actions.expand_template(
template = ctx.file._main_module_entrypoint_template,
output = main_module_entrypoint,
substitutions = {"%main_module%": ctx.attr.main_module},
)
main_py = main_module_entrypoint
extra_runfiles = extra_runfiles.merge(ctx.runfiles([main_module_entrypoint]))
elif not main_py:
# shouldn't happen
fail("Neither main nor main_module was provided")

subs["%main%"] = "{}/{}".format(ctx.workspace_name, main_py.short_path)

ctx.actions.expand_template(
Expand All @@ -863,6 +884,8 @@ def _create_stage1_bootstrap(
substitutions = subs,
)

return extra_runfiles

def _create_windows_exe_launcher(
ctx,
*,
Expand Down Expand Up @@ -985,7 +1008,7 @@ def _create_executable_zip_file(
sibling = output,
)
if stage2_bootstrap:
_create_stage1_bootstrap(
extra_runfiles = _create_stage1_bootstrap(
ctx,
output = prelude,
stage2_bootstrap = stage2_bootstrap,
Expand All @@ -994,6 +1017,7 @@ def _create_executable_zip_file(
venv = venv,
)
else:
extra_runfiles = ctx.runfiles()
ctx.actions.write(prelude, "#!/usr/bin/env python3\n")

ctx.actions.run_shell(
Expand All @@ -1009,6 +1033,8 @@ def _create_executable_zip_file(
progress_message = "Build Python zip executable: %{label}",
)

return extra_runfiles

def _get_cc_details_for_binary(ctx, extra_deps):
cc_info = collect_cc_info(ctx, extra_deps = extra_deps)
return create_cc_details_struct(
Expand Down
4 changes: 2 additions & 2 deletions tests/base_rules/py_executable_base_tests.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -363,8 +363,8 @@ def _test_main_module_bootstrap_system_python(name, config):
)

def _test_main_module_bootstrap_system_python_impl(env, target):
env.expect.that_target(target).failures().contains_predicate(
matching.str_matches("mandatory*srcs"),
env.expect.that_target(target).default_outputs().contains(
"{package}/{test_name}_subject",
)

_tests.append(_test_main_module_bootstrap_system_python)
Expand Down
19 changes: 16 additions & 3 deletions tests/bootstrap_impls/BUILD.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
# limitations under the License.
load("@rules_pkg//pkg:tar.bzl", "pkg_tar")
load("@rules_shell//shell:sh_test.bzl", "sh_test")
load("//python:defs.bzl", "py_library")
load("//tests/support:py_reconfig.bzl", "py_reconfig_binary", "py_reconfig_test")
load("//tests/support:sh_py_run_test.bzl", "sh_py_run_test")
load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT")
Expand Down Expand Up @@ -123,13 +124,25 @@ py_reconfig_test(
main = "sys_path_order_test.py",
)

py_reconfig_test(
name = "main_module_test",
py_library(
name = "main_module_lib",
srcs = ["main_module.py"],
bootstrap_impl = "script",
imports = ["."],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be kept because it ensures the correct entry is added to sys.path. Without it, it can inadvertently rely on the implicit behavior from the flag that adds repos to sys.path and the PYTHONSAFEPATH setting

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can re-add it, but I don't understand the reasoning. IIUC, imports says "add this directory to PYTHONPATH", but we're specifying main_module with the full absolute path, so imports = ["."] isn't doing anything. Would imports = [] have the behavior you want?

)

py_reconfig_test(
name = "main_module_script_test",
bootstrap_impl = "script",
main_module = "tests.bootstrap_impls.main_module",
target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT,
deps = [":main_module_lib"],
)

py_reconfig_test(
name = "main_module_system_python_test",
bootstrap_impl = "system_python",
main_module = "tests.bootstrap_impls.main_module",
deps = [":main_module_lib"],
)

sh_py_run_test(
Expand Down
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy