As part of #60773 (tracking issue #57175) I've been working on a new parser for the execution tracer, and to save work down the line I've also been trying to come up with a nice API that would work well.

The goals of such an API would be to:

• Enable custom analyses of traces.
• Allow for streaming traces and performing on-line analyses.
• Be both backward- and forward- compatible with new Go versions and trace wire formats.

Here are the requirements I have for such an API:

• The API should be stream-based, regardless of whether the underlying trace format supports it.
• Parsing a single trace in parallel is unsupported (part of the point of parsing is to serialize the stream anyway).
• The API should expose some model of the Go scheduler explicitly. (This comes from an insight from @rhysh about the internal/trace API that doesn't actually expose e.g. the state machine goroutines go through (even though it already has to know about it), leaving users to figure it out again.)
• The API should at least allow for an extension for efficient random access from local storage (e.g. loading from an mmap'd trace file).

Any such API is likely to have a fairly large surface. Rather than try to list the full API and debate its merits, we should experiment.

https://go.dev/cl/527875 is my first attempt at such an experiment to support the work done for #60773. The implementation is not quite fully baked (there may be bugs, and there's still some validation missing) but it already seems to work for non-trivial traces produced by https://go.dev/cl/494187 (with GOEXPERIMENT=exectracer2). (I'll update this section when it's ready, including with instructions on how to try it out.)

I plan to take this API and vendor it and/or copy it into the standard library to work as the implementation for supporting new traces in internal/trace. That'll help move #60773 along but it'll also give us some experience with using the API.

Once I get this API landed in x/exp, and the first cut of the new tracer for #60773, I'd like to invite people to experiment to both shake out bugs but also to identify deficiencies in the API.

For the proposal committee: there's nothing actionable here yet, but I figured I'd create an issue to track work and start a discussion.

Related to #62461.

CC @prattmic @aclements @dominikh @felixge @nsrip-dd @bboreham @rhysh