Fix Go tracer selection for orchestrion installs

[tonyredondo]

Summary

This PR fixes the Go autoinstrumentation flow used by the Test Visibility install script when it installs orchestrion.

The original problem was that the script did not actually keep dd-trace-go aligned with the orchestrion release selected by the user. It installed orchestrion, ran orchestrion pin, and let orchestrion resolve dd-trace-go on its own. In practice, that meant projects with no existing dd-trace-go dependency could jump to the latest published tracer release, even when that release required a newer Go version than the customer project supported.

That is what happened in the customer report from orchestrion issue #806: a Go 1.24 project using orchestrion v1.8.0 drifted to dd-trace-go v2.7.0, whose orchestrion/all module requires Go 1.25.

This change keeps the installation stable, but still allows newer compatible tracer releases when the project and runner can support them.

What changed

1. Resolve the orchestrion version once per run

If DD_SET_TRACER_VERSION_GO=latest, the script now resolves latest once up front and reuses that exact tag for the rest of the flow.

This avoids mid-run drift across different network calls.

2. Choose a tracer version based on the selected orchestrion, the project Go version, and the runner Go version

The new algorithm is:

1. Resolve the selected orchestrion version.
2. Read the dd-trace-go/v2 version shipped by that orchestrion release.
3. Treat that shipped version as the minimum allowed tracer version.
4. Read the target project's go directive when possible.
5. Use the lower of the project Go version and the runner Go version as the compatibility ceiling.
6. Walk stable dd-trace-go/v2 releases from newest to oldest.
7. Pick the newest release that is not older than the shipped minimum and whose orchestrion/all module still supports that Go ceiling.
8. If no project-aware upgrade is possible, fall back to the shipped minimum instead of floating to @latest.

This gives us the behavior we actually want:

• Go 1.24 projects stay on the latest tracer line that still supports Go 1.24.
• Go 1.25 projects can use a newer tracer line when it is compatible.
• The script never upgrades beyond what the project and runner can support together.

3. Refuse to instrument projects that are below the minimum supported Go level

There are now two separate lower-bound checks:

• the runner must satisfy the selected orchestrion release
• the runner and project must also satisfy the minimum shipped tracer bundle

If the project's go.mod says something older than the minimum supported by the shipped tracer, the script now skips instrumentation cleanly instead of letting orchestrion pin silently rewrite the project to a newer Go version.

4. Support repos where the Go module is not at the repository root

The script now supports:

• DD_CIVISIBILITY_GO_MODULE_DIR as an explicit module-root override
• single nested-module auto-detection when there is exactly one go.mod
• clean skip behavior when there is no go.mod
• clean skip behavior when multiple modules exist and no override is provided

If the override is explicitly set but invalid, the script fails fast with a clear error.

5. Keep the explicit go get step versioned

The script still updates the module graph with go get github.com/DataDog/orchestrion@<resolved_tag>, but it now pins that step to the same orchestrion version already selected earlier in the run.

6. Avoid dirtying go.mod when orchestrion cannot be installed

The script now installs the orchestrion CLI before it edits the project's go.mod.

This keeps the previous failure behavior where a failed go install github.com/DataDog/orchestrion@... does not leave the module partially modified.

7. Keep the script compatible with /bin/bash 3.2 on macOS runners

The new Go-module discovery and tracer-selection logic now uses array-loading code that works on the default /bin/bash shipped on macOS GitHub runners.

This preserves:

• nested-module auto-detection on macOS runners
• newest-compatible-tracer selection on macOS runners

Why this fixes the customer issue

With orchestrion v1.8.0:

• the shipped dd-trace-go/v2 baseline is v2.6.0
• dd-trace-go/orchestrion/all/v2 v2.6.0 supports Go 1.24
• dd-trace-go/orchestrion/all/v2 v2.7.0 requires Go 1.25

So after this change:

• a Go 1.24 project stays on v2.6.0
• a Go 1.25 project can move to v2.7.0
• a Go 1.22 project is skipped instead of being silently rewritten to a newer Go level

Support matrix

Scenario	Result
Root module, project go 1.24, runner go 1.26.1	Instruments successfully. Pins dd-trace-go/v2 and orchestrion/all/v2 to v2.6.0.
Root module, project go 1.25, runner go 1.26.1	Instruments successfully. Pins dd-trace-go/v2 and orchestrion/all/v2 to v2.7.0.
Root module, project go 1.22, runner go 1.26.1	Skips cleanly. Leaves go.mod unchanged. Does not create orchestrion.tool.go.
Root module, runner below orchestrion minimum	Skips cleanly before touching the module.
Root module, runner satisfies orchestrion but is below the minimum shipped tracer bundle requirement	Skips cleanly before touching the module.
Single nested Go module, project go 1.24, runner go 1.26.1, no env override	Auto-detects the nested module and instruments it successfully. Pins dd-trace-go/v2 and orchestrion/all/v2 to v2.6.0.
Single nested Go module, project go 1.25, runner go 1.26.1, no env override	Auto-detects the nested module and instruments it successfully. Pins dd-trace-go/v2 and orchestrion/all/v2 to v2.7.0.
Multiple Go modules, no env override	Skips cleanly and tells the user to set DD_CIVISIBILITY_GO_MODULE_DIR.
Multiple Go modules, valid DD_CIVISIBILITY_GO_MODULE_DIR	Instruments only the selected module successfully.
DD_CIVISIBILITY_GO_MODULE_DIR points to a missing directory	Fails fast with an explicit error.
DD_CIVISIBILITY_GO_MODULE_DIR points to a directory without go.mod	Fails fast with an explicit error.
No go.mod in the repo	Skips cleanly.
go.mod exists but has no go directive	Falls back to the minimum tracer shipped by the selected orchestrion release and proceeds.
Stable dd-trace-go version list cannot be retrieved	Falls back to the minimum tracer shipped by the selected orchestrion release and proceeds.
DD_SET_TRACER_VERSION_GO=latest	Resolves latest once, then uses that fixed resolved tag consistently for the whole run.
go install github.com/DataDog/orchestrion@... fails after version resolution	Fails without modifying go.mod.

Verification

I verified this in two ways:

1. Real targeted repros in /tmp

   • Confirmed the original bug with a Go 1.24 repro.
   • Confirmed the fix selects v2.6.0 for Go 1.24 and v2.7.0 for Go 1.25.
   • Confirmed single nested-module auto-detection now selects v2.7.0 for a nested Go 1.25 project.
   • Confirmed Go 1.22 now skips cleanly without mutating go.mod.

2. Full scenario sweep with a lightweight harness

   • Added tests/run_go_matrix.sh to exercise the Go decision matrix without relying on full upstream orchestrion builds.
   • Verified all scenarios in the support matrix above, including the nested-module go 1.25 upgrade path and the failed-go install no-mutation path.
   • Reran the matrix successfully under the repository copy of the harness.

I also ran:

• bash -n install_test_visibility.sh
• bash -n tests/run_go_matrix.sh
• /bin/bash -n tests/run_go_matrix.sh
• tests/run_go_matrix.sh

Notes

The checked-in matrix harness uses fake go, curl, and orchestrion binaries so it can validate the script's control flow, version selection, and file mutation behavior deterministically without depending on repeated full upstream installs.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 165446b4ed

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[daniel-mohedano]

IIRC I based the Jenkins auto instrumentation logic on this script, so we might have to fix it too 🤕

[tonyredondo]

IIRC I based the Jenkins auto instrumentation logic on this script, so we might have to fix it too 🤕

I'm sure we can tell the AI to do the changes for jenkins