Skip to content

feat(vllm_performance): Add experiment to test geospatial endpoint with a custom dataset and remove geospatial experiments from BETA features#744

Open
christian-pinto wants to merge 8 commits intomainfrom
cp_vllm_perf_geo_updates

Conversation

@christian-pinto
Copy link
Copy Markdown
Member

No description provided.

@DRL-NextGen
Copy link
Copy Markdown
Member

DRL-NextGen commented Mar 25, 2026

No vulnerabilities found.

@AlessandroPomponio AlessandroPomponio changed the title Update the vllm-performance actuator to run experiments with geospatial models feat(vllm_performance): add experiment to test geospatial endpoint with a custom dataset Mar 27, 2026
@michael-johnston
Copy link
Copy Markdown
Member

@christian-pinto FYI plugins/actuators/vllm_performance/ado_actuators/vllm_performance/experiment_executor.py need black run on it.

@christian-pinto
Copy link
Copy Markdown
Member Author

@christian-pinto FYI plugins/actuators/vllm_performance/ado_actuators/vllm_performance/experiment_executor.py need black run on it.

Weird, the pre-commit hooks had all passed when I pushed those changes. I will give it another pass as there are more changes coming.

@christian-pinto christian-pinto changed the title feat(vllm_performance): add experiment to test geospatial endpoint with a custom dataset feat(vllm_performance): Add experiment to test geospatial endpoint with a custom dataset and remove geospatial experiments from BETA features Mar 31, 2026
christian-pinto and others added 5 commits March 31, 2026 10:53
…-custom-dataset experiment

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
…y configuration to test existing vllm geospatial endpoint

Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
…models

Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
…ts and added geospatial guidellm experiments

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
@christian-pinto christian-pinto force-pushed the cp_vllm_perf_geo_updates branch from 4a14ce3 to 9705998 Compare March 31, 2026 09:53
…endency

Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
@michael-johnston
Copy link
Copy Markdown
Member

Ready for review?

@christian-pinto
Copy link
Copy Markdown
Member Author

Ready for review?

Waiting for @mgazz to confirm he could replicate with guidellm on his side

@mgazz
Copy link
Copy Markdown
Member

mgazz commented Apr 3, 2026

Works for me. For context, the only thing I had to do to switch between vllm bench and guidellm was to swap the experiment section in the discovery space

from

experiments:
  - actuatorIdentifier: "vllm_performance"
    experimentIdentifier: "test-geospatial-deployment-custom-dataset-v1"

to

experiments:
  - actuatorIdentifier: "vllm_performance"
    experimentIdentifier: "test-geospatial-deployment-guidellm-custom-dataset-v1"

Full discovery space definition: discoveryspace_geospatial.yaml

@christian-pinto
Copy link
Copy Markdown
Member Author

@michael-johnston ready for review.

@DRL-NextGen
Copy link
Copy Markdown
Member

Checks Summary

Last run: 2026-04-07T08:22:56.796Z

Code Risk Analyzer vulnerability scan found 5 vulnerabilities:

Severity Identifier Package Details Fix
🔷 Medium CVE-2026-34756 vllm
vLLM: Unauthenticated OOM Denial of Service via Unbounded n Parameter in OpenAI API ServerGHSA-3mwp-wvh9-7528

vllm:0.18.0
0.19.0
🔷 Medium CVE-2026-34753 vllm
vLLM: Server-Side Request Forgery (SSRF) in download_bytes_from_url GHSA-pf3h-qjgv-vcpr

vllm:0.18.0
0.19.0
🔷 Medium CVE-2026-34755 vllm
vLLM: Denial of Service via Unbounded Frame Count in video/jpeg Base64 ProcessingGHSA-pq5c-rjhq-qp7p

vllm:0.18.0
0.19.0
◻ Unknown CVE-2026-34450 anthropic
Claude SDK for Python has Insecure Default File Permissions in Local Filesystem Memory ToolGHSA-q5f5-3gjm-7mfm

anthropic:0.86.0->anthropic:0.86.0,vllm:0.18.0
0.87.0
◻ Unknown CVE-2026-34452 anthropic
Claude SDK for Python: Memory Tool Path Validation Race Condition Allows Sandbox EscapeGHSA-w828-4qhx-vxx3

anthropic:0.86.0->anthropic:0.86.0,vllm:0.18.0
0.87.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants