Skip to content

deps(gazelle): Bump rules_go from 0.41.0 to 0.55.0; gazelle from 0.31.0 (or 0.33.0 for bzlmod) to 0.40.0; go from 1.19.4 to 1.22.0 #2962

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 31 commits into
base: main
Choose a base branch
from

Conversation

dougthor42
Copy link
Collaborator

@dougthor42 dougthor42 commented Jun 6, 2025

Bump versions:

  • rules_go from 0.41.0 to 0.55.0
  • gazelle from 0.31.0 (or 0.33.0 for bzlmod) to 0.40.0
  • go from 1.19.4 to 1.22.0

Fixes #2956.

  • Run go get github.com/bazelbuild/rules_go@v0.55.0 for version bump.
  • Run go mod tidy to clean up stale deps
  • Run bazel run //:gazelle_update_repos
  • Figure out http_archive hashes and apply them for WORKSPACE users.
  • Also bump bazel-gazelle to address a warning:
    • go get github.com/bazelbuild/bazel-gazelle@v0.40.0
    • Mismatch between versions requested for module
      github.com/bazelbuild/bazel-gazelle
      Bazel dependency version requested in MODULE.bazel: 0.40.0
      Go module version requested in go.mod: 0.31.1
      Please resolve this mismatch to prevent discrepancies between
      native Go and Bazel builds

  • Update CHANGELOG.md accordingly

Bump `rules_go` from 0.41.0 to 0.55.0

Fixes bazel-contrib#2956.

+ Run `go get github.com/bazelbuild/rules_go@v0.55.0` for version bump.
+ Run `go mod tidy` to clean up stale deps
+ Run `bazel run //:gazelle_update_repos`
+ Figure out http_archive hashes and apply them for WORKSPACE users.
+ Also bump bazel-gazelle to address a warning:
    + `go get github.com/bazelbuild/bazel-gazelle@v0.40.0`
    + > Mismatch between versions requested for module
      > github.com/bazelbuild/bazel-gazelle
      > Bazel dependency version requested in MODULE.bazel: 0.40.0
      > Go module version requested in go.mod: 0.31.1
      > Please resolve this mismatch to prevent discrepancies between
      > native Go and Bazel builds
+ Update CHANGELOG.md accordingly
@dougthor42
Copy link
Collaborator Author

I'm not exactly sure how to fix the failing WORKSPACE tests.

(23:42:51) ERROR: no such package '@@org_golang_x_tools//go/vcs': BUILD file not found
in directory 'go/vcs' of external repository @@org_golang_x_tools. Add a BUILD file
to a directory to mark it as a package.
(23:42:51) ERROR: /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/d2393494a560f9893c8389cd6840bb63/external/bazel_gazelle/repo/BUILD.bazel:3:11: no
such package '@@org_golang_x_tools//go/vcs': BUILD file not found in directory 'go/vcs'
of external repository @@org_golang_x_tools. Add a BUILD file to a directory to mark it
as a package. and referenced by '@@bazel_gazelle//repo:repo'

My assumption is that there's an issue with this go_repository being a nested module, and that go_repository is running gazelle on the root org_golang_x_tools instead.

go_repository(
    name = "org_golang_x_tools_go_vcs",
    importpath = "golang.org/x/tools/go/vcs",
    sum = "h1:cOIJqWBl99H1dH5LWizPa+0ImeeJq3t3cJjaeOWUAL4=",
    version = "v0.1.0-deprecated",
)

Any thoughts? I thought that maybe some build_directives could help here, but I didn't see any obvious ones. Maybe gazelle:prefix?

Thus far I'm unable to reproduce the issue locally, so fixing it might result in some CI spam...

@dougthor42
Copy link
Collaborator Author

Thus far I'm unable to reproduce the issue locally

Reproduced via the very simple bazel test --noenable_bzlmod --enable_workspace //... from the gazelle directory 🤦.

Investigating a fix.

@dougthor42 dougthor42 changed the title deps(gazelle): Bump rules_go from 0.41.0 to 0.55.0 deps(gazelle): Bump rules_go from 0.41.0 to 0.55.0; gazelle from 0.31.0 (or 0.33.0 for bzlmod) to 0.40.0; go from 1.19.4 to 1.22.0 Jun 9, 2025
@dougthor42
Copy link
Collaborator Author

@aignas I've solved the WORKSPACE stuff by bumping gazelle to 0.40.0, and in doing so had to bump go from 1.19.4 to 1.22.0. This go bump looks like it may have broken some BuildKite stuff? @@bazel_gazelle_is_bazel_module//:defs.bzl.

Example:

(03:46:18) ERROR: error loading package '@@rules_python_gazelle_plugin//': at /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/ec321eb2cc2d0f8f91b676b6d4c66c29/external/bazel_gazelle/def.bzl:16:5: Unable to find package for @@bazel_gazelle_is_bazel_module//:defs.bzl: The repository '@@bazel_gazelle_is_bazel_module' could not be resolved: Repository '@@bazel_gazelle_is_bazel_module' is not defined.
(03:46:18) ERROR: /workdir/BUILD.bazel:36:10: error loading package '@@rules_python_gazelle_plugin//': at /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/ec321eb2cc2d0f8f91b676b6d4c66c29/external/bazel_gazelle/def.bzl:16:5: Unable to find package for @@bazel_gazelle_is_bazel_module//:defs.bzl: The repository '@@bazel_gazelle_is_bazel_module' could not be resolved: Repository '@@bazel_gazelle_is_bazel_module' is not defined. and referenced by '//:distribution'

Can you take a look?

…ORKSPACE

+ 0.31, 0.32 errored out with the golang_x_tools//go/vcs issue from before
+ 0.33, 0.34, 0.35 have the wrong load sorting.
According to bazel-contrib/bazel-gazelle#1654
we need to call the gazelle_dependencies macro in WORKSPACE. We already
do that in gazelle/WORKSPACE but it's not in the root WORKSPACE. Maybe
we have to do it there, too...
@dougthor42
Copy link
Collaborator Author

Huzzah! I got the rest of the Gazelle stuff passing for WORKSPACE.

Now I need to investigate the Windows builds and my tree-sitter fork...

@dougthor42 dougthor42 force-pushed the u/dthor/bump-rules_go-gh2956 branch from c91f3db to 6849424 Compare July 6, 2025 06:26
@dougthor42
Copy link
Collaborator Author

TL;DR: Might need more updates to rules_go before we can move forward. I'll make an Issue there. But not tonight.


After much application of my head against a brick wall, I think I've come to the conclusion that bazel-contrib/rules_go#4298 did not fully resolve the "not importing C stuff" thing from bazel-contrib/bazel-gazelle#2059.

As you can see from the commit history, I've tried jumping back to smacker/go-tree-sitter, which can't be done because it shares an issue with dougthor42/go-tree-sitter in that things fail on Windows (@shaldengeki saw this in #3060 too).

I've also tried pulling in the official tree-sitter as @jbedard did in #2952. This attempt is why I think that rules_go is still broken, as we get:

_main/external/gazelle++go_deps+com_github_tree_sitter_tree_sitter_python/bindings/go/binding.go:4:11: fatal error: ../../src/parser.c: No such file or directory

The source file is https://github.com/tree-sitter/tree-sitter-python/blob/master/bindings/go/binding.go and looks like:

package tree_sitter_python

// #cgo CFLAGS: -std=c11 -fPIC
// #include "../../src/parser.c"
// #include "../../src/scanner.c"
import "C"
...

Looking at the BUILD.bazel file that gazelle makes for that dependency, we see:

$ cat ~/.cache/bazel/_bazel_dthor/518014fa284dfa240d9c6a5ccc9044a0/execroot/_main/external/gazelle++go_deps+com_github_tree_sitter_tree_sitter_python/bindings/go/BUILD.bazel
load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test")

go_library(
    name = "go",
    srcs = ["binding.go"],
    cgo = True,
    copts = ["-std=c11 -fPIC"],
    importpath = "github.com/tree-sitter/tree-sitter-python/bindings/go",
    visibility = ["//visibility:public"],
)
...

And if we look at the project tree, those files do exist... they're just not added as deps to the go_library target 😭

$ tree ~/.cache/bazel/_bazel_dthor/518014fa284dfa240d9c6a5ccc9044a0/execroot/_main/external/gazelle++go_deps+com_github_tree_sitter_tree_sitter_python/                                                                
/usr/local/google/home/dthor/.cache/bazel/_bazel_dthor/518014fa284dfa240d9c6a5ccc9044a0/execroot/_main/external/gazelle++go_deps+com_github_tree_sitter_tree_sitter_python/
├── binding.gyp                
├── bindings              
│   ├── c                   
│   │   ├── tree-sitter-python.h
│   │   └── tree-sitter-python.pc.in
│   ├── go                                           
│   │   ├── binding.go
│   │   ├── binding_test.go   
│   │   └── BUILD.bazel       # hey this is the file I just 'cat'ed
...
├── setup.py
├── src
│   ├── grammar.json
│   ├── node-types.json
│   ├── parser.c              # would you look at that...
│   ├── scanner.c             # there they are!
│   └── tree_sitter
│       ├── alloc.h
│       ├── array.h
│       └── parser.h
├── test
...
└── tree-sitter.json

@jbedard
Copy link
Contributor

jbedard commented Jul 7, 2025

FWIW I've held back any more effort switching to https://github.com/tree-sitter/go-tree-sitter/ until I get some feedback on tree-sitter/go-tree-sitter#32

@dougthor42
Copy link
Collaborator Author

The summary thus far:

  • Tried bumping rules_go >= 0.55.0 in the hopes that gazelle would be able to make go_library target that referenced C files in parent directories, eg following #include "../array.h" in (smacker|dougthor42)/go-tree-sitter
  • We can't bump go above v1.19 because (smacker|dougthor42)/go-tree-sitter fails to build on Windows with an error ...go_tree_sitter\iter.go:17:18: undefined: Node.
  • While I have access to a Windows host, I do not know enough go to be able to update dougthor42/go-tree-sitter to work with go >= 1.22.
    • And I don't have access while at work, so my time is very limited.
  • Tried using http_archive to pull in the official tree-sitter and manually inject our own BUILD files instead of having Gazelle autogenerate them
    • This was too complex of a task, but did get me to think of the next item
  • Tried using http_archive to pull the smacker/go-tree-sitter and manually inject our own BUILD files instead of having Gazelle autogenerate them
    • So far so good...

github-merge-queue bot pushed a commit that referenced this pull request Jul 8, 2025
Finally remove the dougthor42/go-tree-sitter fork, fixing #2630.

Admittedly we could have done this sooner had I figured things
out sooner... but c'est la vie.

Instead of using the BUILD.bazel files in dougthor42/go-tree-sitter,
we basically vendor the build file via http_archive.

This is different than using patches because non-root Bazel modules
can still make use of the BUILD.bazel files we make.

Background:

The reason we migrated to dougthor42/go-tree-sitter in the first
place was to support python 3.12 grammar. smacker/go-tree-sitter
supported for python 3.12, but made a change to their file structure
that Gazelle was unable to handle. Specifically, the
python/binding.go file indirectly requires a c header file found
in a parent directory, and Gazelle doesn't know how to handle that
for `go_repository` (WORKSPACE) and `go_deps.from_file` (bzlmod).

So dougthor42/go-tree-sitter created our own BUILD.bazel files
that included the required filegroups and whatnot, thus negating
the need for Gazelle to generate BUILD.bazel files.

Future Work:

This still doesn't resolve the issues with bumping rules_go and go
seen in #2962, but it does simplify that investigation a bit as
it's just one fewer thing to account for. It also doesn't address
the desire to migrate to the official tree-sitter/go-tree-sitter
repo, but @jbedard found some perf issues with that anyway
(tree-sitter/go-tree-sitter#32).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

chore: update to rules_go v0.55.0
3 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy