I followed the below steps to export a TensorRT file but encountered errors:
Run the command:
sh samples/bevformer/tiny/quant_max_ptq.sh -d 0
This generated the file: bevformer_tiny_epoch_24_ptq_max.pth
Then run:
sh samples/bevformer/tiny/pth2onnx_q.sh -d 0
This generated the ONNX file: bevformer_tiny_epoch_24_ptq_max.onnx
Finally, I executed both of the following commands:
sh samples/bevformer/tiny/onnx2trt_int8_qdq.sh -d 0
sh samples/bevformer/tiny/onnx2trt_int8_fp16_qdq.sh -d 0
Both commands failed with errors.
Here is log
[12/06/2025-13:46:40] [TRT] [W] TensorRT was linked against cuDNN 8.9.0 but loaded cuDNN 8.5.0
[12/06/2025-13:46:40] [TRT] [I] BuilderFlag::kTF32 is set but hardware does not support TF32. Disabling TF32.
[12/06/2025-13:46:40] [TRT] [V] Building graph using backend strategy 2
[12/06/2025-13:46:40] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[12/06/2025-13:46:40] [TRT] [V] Constructing optimization profile number 0 [1/1].
[12/06/2025-13:46:40] [TRT] [V] Applying generic optimizations to the graph for inference.
[12/06/2025-13:46:40] [TRT] [V] Insert CopyNode after ConstantNode that produces a Myelin graph output: /decoder/layers.0/attentions.1/attention_weights/_weight_quantizer/Constant_1_output_0
[12/06/2025-13:46:40] [TRT] [E] 2: Assertion !n->candidateRequirements.empty() failed. All of the candidates were removed, which points to the node being incorrectly marked as an int8 node.
[12/06/2025-13:46:40] [TRT] [E] 2: [optimizer.cpp::filterQDQFormats::4666] Error Code 2: Internal Error (Assertion !n->candidateRequirements.empty() failed. All of the candidates were removed, which points to the node being incorrectly marked as an int8 node.)
调试时OS_PATH解析结果：/home/ll/BEVFormer_tensorrt_muxindawang/TensorRT/lib/libtensorrt_ops.so
当前工作目录cwd：/home/ll/BEVFormer_tensorrt_muxindawang
Loaded tensorrt plugins from /home/ll/BEVFormer_tensorrt_muxindawang/TensorRT/lib/libtensorrt_ops.so
WARNING!!!!, Only can be used for obtain inference speed!!!!
Loading ONNX file from path checkpoints/onnx/bevformer_tiny_epoch_24_ptq_max.onnx...
Beginning ONNX file parsing
Completed parsing of ONNX file
Building an engine from file checkpoints/onnx/bevformer_tiny_epoch_24_ptq_max.onnx; this may take a while...
Traceback (most recent call last):
File "/home/ll/BEVFormer_tensorrt_muxindawang/tools/bevformer/onnx2trt.py", line 258, in
main()
File "/home/ll/BEVFormer_tensorrt_muxindawang/tools/bevformer/onnx2trt.py", line 246, in main
build_engine(
File "/home/ll/BEVFormer_tensorrt_muxindawang/./det2trt/convert/onnx2tensorrt.py", line 63, in build_engine
engine = runtime.deserialize_cuda_engine(plan)
TypeError: deserialize_cuda_engine(): incompatible function arguments. The following argument types are supported:
1. (self: tensorrt.tensorrt.Runtime, serialized_engine: buffer) -> tensorrt.tensorrt.ICudaEngine
Invoked with: <tensorrt.tensorrt.Runtime object at 0x7f1c17ce5ef0>, None
environment:
cuda_11.7
TensorRT-8.6.1.6
onnx 1.12.0
bevformer_tiny_onnx2trt_qdq.log
I followed the below steps to export a TensorRT file but encountered errors:
Run the command:
sh samples/bevformer/tiny/quant_max_ptq.sh -d 0This generated the file: bevformer_tiny_epoch_24_ptq_max.pth
Then run:
sh samples/bevformer/tiny/pth2onnx_q.sh -d 0This generated the ONNX file: bevformer_tiny_epoch_24_ptq_max.onnx
Finally, I executed both of the following commands:
sh samples/bevformer/tiny/onnx2trt_int8_qdq.sh -d 0sh samples/bevformer/tiny/onnx2trt_int8_fp16_qdq.sh -d 0Both commands failed with errors.
Here is log
[12/06/2025-13:46:40] [TRT] [W] TensorRT was linked against cuDNN 8.9.0 but loaded cuDNN 8.5.0
[12/06/2025-13:46:40] [TRT] [I] BuilderFlag::kTF32 is set but hardware does not support TF32. Disabling TF32.
[12/06/2025-13:46:40] [TRT] [V] Building graph using backend strategy 2
[12/06/2025-13:46:40] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[12/06/2025-13:46:40] [TRT] [V] Constructing optimization profile number 0 [1/1].
[12/06/2025-13:46:40] [TRT] [V] Applying generic optimizations to the graph for inference.
[12/06/2025-13:46:40] [TRT] [V] Insert CopyNode after ConstantNode that produces a Myelin graph output: /decoder/layers.0/attentions.1/attention_weights/_weight_quantizer/Constant_1_output_0
[12/06/2025-13:46:40] [TRT] [E] 2: Assertion !n->candidateRequirements.empty() failed. All of the candidates were removed, which points to the node being incorrectly marked as an int8 node.
[12/06/2025-13:46:40] [TRT] [E] 2: [optimizer.cpp::filterQDQFormats::4666] Error Code 2: Internal Error (Assertion !n->candidateRequirements.empty() failed. All of the candidates were removed, which points to the node being incorrectly marked as an int8 node.)
调试时OS_PATH解析结果：/home/ll/BEVFormer_tensorrt_muxindawang/TensorRT/lib/libtensorrt_ops.so
当前工作目录cwd：/home/ll/BEVFormer_tensorrt_muxindawang
Loaded tensorrt plugins from /home/ll/BEVFormer_tensorrt_muxindawang/TensorRT/lib/libtensorrt_ops.so
WARNING!!!!, Only can be used for obtain inference speed!!!!
Loading ONNX file from path checkpoints/onnx/bevformer_tiny_epoch_24_ptq_max.onnx...
Beginning ONNX file parsing
Completed parsing of ONNX file
Building an engine from file checkpoints/onnx/bevformer_tiny_epoch_24_ptq_max.onnx; this may take a while...
Traceback (most recent call last):
File "/home/ll/BEVFormer_tensorrt_muxindawang/tools/bevformer/onnx2trt.py", line 258, in
main()
File "/home/ll/BEVFormer_tensorrt_muxindawang/tools/bevformer/onnx2trt.py", line 246, in main
build_engine(
File "/home/ll/BEVFormer_tensorrt_muxindawang/./det2trt/convert/onnx2tensorrt.py", line 63, in build_engine
engine = runtime.deserialize_cuda_engine(plan)
TypeError: deserialize_cuda_engine(): incompatible function arguments. The following argument types are supported:
1. (self: tensorrt.tensorrt.Runtime, serialized_engine: buffer) -> tensorrt.tensorrt.ICudaEngine
Invoked with: <tensorrt.tensorrt.Runtime object at 0x7f1c17ce5ef0>, None
environment:
cuda_11.7
TensorRT-8.6.1.6
onnx 1.12.0
bevformer_tiny_onnx2trt_qdq.log