test: add 24 GitCode smoke scenarios + parallel runner#11
Conversation
New scenarios: branch: list, get (master) tag: list, get (skipped, requires admin) release: list (empty repo, 200 OK) commit: list, get (first commit sha) labels: list comments: list (commit comments) pulls: comments (PR #747) Result: 28 passed, 0 failed, 8 skipped Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Group scenarios by service, run service groups in parallel (up to --parallel N). Scenarios within a group still run sequentially to avoid rate-limiting the same API. Results ordered deterministically. - Runner.maxConcurrent field + --parallel flag (default 4, 0=sequential) - runParallel(): goroutine per service group, semaphore-controlled - runSequential(): kept for --parallel 0 - Fix runner tests for new NewRunner signature Speed: 36 scenarios 20s→10s (~2x) with --parallel 4 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Welcome To opensourceways CommunityHey @TommyLike , thanks for your contribution to the community. Bot Usage ManualI'm the Bot here serving you. You can find the instructions on how to interact with me at Here . That means you can comment below every pull request or issue to trigger Bot Commands. |
CLA Signature PassTommyLike, thanks for your pull request. All authors of the commits have signed the CLA. 👍 |
Linking Issue Notice@TommyLike , the pull request must be linked to at least one issue. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces parallel execution capabilities to the smoke test runner, allowing service groups to run concurrently up to a specified limit while maintaining sequential execution within each group. It also adds several new smoke test scenarios for the gitcode service. The review feedback highlights two main issues: first, the parallel runner's sorting mechanism assumes scenario names are globally unique, which could lead to non-deterministic sorting if duplicates exist; a helper struct is suggested to track original indices safely. Second, the newly added tag-get.yaml scenario contains a copy-paste error, using the list subcommand instead of get.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| // Group by service, preserving order within each group. | ||
| groups := make(map[string][]Scenario) | ||
| var serviceOrder []string | ||
| for _, s := range scenarios { | ||
| if _, ok := groups[s.Service]; !ok { | ||
| serviceOrder = append(serviceOrder, s.Service) | ||
| } | ||
| groups[s.Service] = append(groups[s.Service], s) | ||
| } | ||
|
|
||
| // Assign each scenario an index for deterministic output ordering. | ||
| origIdx := make(map[string]int) | ||
| for i, s := range scenarios { | ||
| origIdx[s.Name] = i | ||
| } | ||
|
|
||
| type indexedResult struct { | ||
| idx int | ||
| result ScenarioResult | ||
| } | ||
|
|
||
| resultCh := make(chan indexedResult, len(scenarios)) | ||
| sem := make(chan struct{}, r.maxConcurrent) | ||
| var wg sync.WaitGroup | ||
|
|
||
| for _, svc := range serviceOrder { | ||
| wg.Add(1) | ||
| go func(svcName string, svcScenarios []Scenario) { | ||
| defer wg.Done() | ||
| sem <- struct{}{} | ||
| defer func() { <-sem }() | ||
|
|
||
| for _, s := range svcScenarios { | ||
| res := r.Run(s) | ||
| resultCh <- indexedResult{idx: origIdx[s.Name], result: res} | ||
| } | ||
| }(svc, groups[svc]) | ||
| } |
There was a problem hiding this comment.
The current parallel runner groups scenarios by service and then maps each scenario's name to its original index using the origIdx map. This assumes that scenario names are globally unique, which is not enforced by the loader. If there are duplicate scenario names, this lookup will map them to the same index, leading to non-deterministic or incorrect sorting of the final report.
We can avoid this assumption and eliminate the origIdx map lookup entirely by grouping scenarios using a helper struct that stores the original index alongside each scenario.
type indexedScenario struct {
idx int
s Scenario
}
groups := make(map[string][]indexedScenario)
var serviceOrder []string
for i, s := range scenarios {
if _, ok := groups[s.Service]; !ok {
serviceOrder = append(serviceOrder, s.Service)
}
groups[s.Service] = append(groups[s.Service], indexedScenario{idx: i, s: s})
}
type indexedResult struct {
idx int
result ScenarioResult
}
resultCh := make(chan indexedResult, len(scenarios))
sem := make(chan struct{}, r.maxConcurrent)
var wg sync.WaitGroup
for _, svc := range serviceOrder {
wg.Add(1)
go func(svcName string, svcScenarios []indexedScenario) {
defer wg.Done()
sem <- struct{}{}
defer func() { <-sem }()
for _, is := range svcScenarios {
res := r.Run(is.s)
resultCh <- indexedResult{idx: is.idx, result: res}
}
}(svc, groups[svc])
}| args: | ||
| - tag | ||
| - list |
There was a problem hiding this comment.
The scenario file is named gitcode/tag-get but the arguments specify the list subcommand instead of get. This appears to be a copy-paste error from tag-list.yaml. Even though this scenario is currently skipped, it should be defined with the correct subcommand.
args:
- tag
- get
CLA Signature PassTommyLike, thanks for your pull request. All authors of the commits have signed the CLA. 👍 |
2 participants
Summary
Add 24 read-only smoke scenarios for GitCode resources + parallel smoke runner.
New GitCode Scenarios (11 added, 24 total)
Parallel Runner
--parallel Nflag (default 4, 0=sequential)Smoke Result
Generated with Claude Code