Skip to content

feat: enable Jetson TensorRT CLI profiling#59

Open
hyeokjun32 wants to merge 1 commit intomainfrom
feat/tensorrt-jetson-cli-profile
Open

feat: enable Jetson TensorRT CLI profiling#59
hyeokjun32 wants to merge 1 commit intomainfrom
feat/tensorrt-jetson-cli-profile

Conversation

@hyeokjun32
Copy link
Copy Markdown
Contributor

Summary

  • enable TensorRT runtime execution path on Jetson
  • add TensorRT runtime buffer allocation and device pointer preparation
  • add CUDA driver/context setup for TensorRT device allocation
  • add TensorRT dummy input generation and inference execution support
  • add Python 3.10 compatible TOML loading fallback
  • validate Jetson CLI profiling flow for TensorRT engines

Test

  • poetry run python -m py_compile edgebench/engines/tensorrt.py
  • poetry run python -m py_compile edgebench/config.py
  • poetry run python -c "from edgebench.engines.tensorrt import TensorRtEngine; print('TensorRtEngine import ok')"
  • poetry run python - <<'PY'
    from edgebench.config import resolve_compare_thresholds
    print(resolve_compare_thresholds())
    PY
  • Jetson: python3 - <<'PY'
    from edgebench.engines.tensorrt import TensorRtEngine
    engine = TensorRtEngine()
    engine.load("models/resnet18.onnx", engine_path="models/resnet18.engine")
    feeds = engine.make_dummy_inputs()
    outputs = engine.run(feeds)
    print(len(outputs))
    PY
  • Jetson: python3 - <<'PY'
    from edgebench.engines.tensorrt import TensorRtEngine
    engine = TensorRtEngine()
    engine.load("models/yolov8n.onnx", engine_path="models/yolov8n.engine")
    feeds = engine.make_dummy_inputs()
    outputs = engine.run(feeds)
    print(len(outputs))
    PY
  • Jetson: python -m edgebench.cli profile models/resnet18.onnx --engine tensorrt --engine-path models/resnet18.engine --precision fp16
  • Jetson: python -m edgebench.cli profile models/yolov8n.onnx --engine tensorrt --engine-path models/yolov8n.engine --precision fp16

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant