MMDetection3d Deployment¶
MMDetection3d aka mmdet3d is an open source object detection toolbox based on PyTorch, towards the next-generation platform for general 3D detection. It is a part of the OpenMMLab project.
Install mmdet3d¶
We could install mmdet3d through mim. For other ways of installation, please refer to here
python3 -m pip install -U openmim
python3 -m mim install "mmdet3d>=1.1.0"
Convert model¶
For example, use tools/deploy.py to convert centerpoint to onnxruntime format
# cd to mmdeploy root directory
# download config and model
mim download mmdet3d --config centerpoint_pillar02_second_secfpn_head-circlenms_8xb4-cyclic-20e_nus-3d --dest .
export MODEL_CONFIG=centerpoint_pillar02_second_secfpn_head-circlenms_8xb4-cyclic-20e_nus-3d.py
export MODEL_PATH=centerpoint_02pillar_second_secfpn_circlenms_4x8_cyclic_20e_nus_20220811_031844-191a3822.pth
export TEST_DATA=tests/data/n008-2018-08-01-15-16-36-0400__LIDAR_TOP__1533151612397179.pcd.bin
python3 tools/deploy.py configs/mmdet3d/voxel-detection/voxel-detection_onnxruntime_dynamic.py $MODEL_CONFIG $MODEL_PATH $TEST_DATA --work-dir centerpoint
This step would generate end2end.onnx in work-dir
ls -lah centerpoint
..
-rw-rw-r-- 1 rg rg 87M 11月 4 19:48 end2end.onnx
Model inference¶
At present, the voxelize preprocessing and postprocessing of mmdet3d are not converted into onnx operations; the C++ SDK has not yet implemented the voxelize calculation.
The caller needs to refer to the corresponding python implementation to complete.
Supported models¶
| model | task | dataset | onnxruntime | openvino | tensorrt* |
|---|---|---|---|---|---|
| centerpoint | voxel detection | nuScenes | ✔️ | ✔️ | ✔️ |
| pointpillars | voxel detection | nuScenes | ✔️ | ✔️ | ✔️ |
| pointpillars | voxel detection | KITTI | ✔️ | ✔️ | ✔️ |
| smoke | monocular detection | KITTI | ✔️ | x | ✔️ |
Make sure trt >= 8.6 for some bug fixed, such as ScatterND, dynamic shape crash and so on.