onnxruntime 支持情况¶
Introduction of ONNX Runtime¶
ONNX Runtime is a cross-platform inference and training accelerator compatible with many popular ML/DNN frameworks. Check its github for more information.
Installation¶
Please note that only onnxruntime>=1.8.1 of on Linux platform is supported by now.
Install ONNX Runtime python package¶
CPU Version
pip install onnxruntime==1.8.1 # if you want to use cpu version
GPU Version
pip install onnxruntime-gpu==1.8.1 # if you want to use gpu version
Build custom ops¶
Download ONNXRuntime Library¶
Download onnxruntime-linux-*.tgz
library from ONNX Runtime releases, extract it, expose ONNXRUNTIME_DIR
and finally add the lib path to LD_LIBRARY_PATH
as below:
CPU Version
wget https://github.com/microsoft/onnxruntime/releases/download/v1.8.1/onnxruntime-linux-x64-1.8.1.tgz
tar -zxvf onnxruntime-linux-x64-1.8.1.tgz
cd onnxruntime-linux-x64-1.8.1
export ONNXRUNTIME_DIR=$(pwd)
export LD_LIBRARY_PATH=$ONNXRUNTIME_DIR/lib:$LD_LIBRARY_PATH
GPU Version
wget https://github.com/microsoft/onnxruntime/releases/download/v1.8.1/onnxruntime-linux-x64-gpu-1.8.1.tgz
tar -zxvf onnxruntime-linux-x64-gpu-1.8.1.tgz
cd onnxruntime-linux-x64-gpu-1.8.1
export ONNXRUNTIME_DIR=$(pwd)
export LD_LIBRARY_PATH=$ONNXRUNTIME_DIR/lib:$LD_LIBRARY_PATH
Build on Linux¶
CPU Version
cd ${MMDEPLOY_DIR} # To MMDeploy root directory
mkdir -p build && cd build
cmake -DMMDEPLOY_TARGET_DEVICES='cpu' -DMMDEPLOY_TARGET_BACKENDS=ort -DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR} ..
make -j$(nproc) && make install
GPU Version
cd ${MMDEPLOY_DIR} # To MMDeploy root directory
mkdir -p build && cd build
cmake -DMMDEPLOY_TARGET_DEVICES='cuda' -DMMDEPLOY_TARGET_BACKENDS=ort -DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR} ..
make -j$(nproc) && make install
How to convert a model¶
You could follow the instructions of tutorial How to convert model
How to add a new custom op¶
Reminder¶
The custom operator is not included in supported operator list in ONNX Runtime.
The custom operator should be able to be exported to ONNX.
Main procedures¶
Take custom operator roi_align
for example.
Create a
roi_align
directory in ONNX Runtime source directory${MMDEPLOY_DIR}/csrc/backend_ops/onnxruntime/
Add header and source file into
roi_align
directory${MMDEPLOY_DIR}/csrc/backend_ops/onnxruntime/roi_align/
Add unit test into
tests/test_ops/test_ops.py
Check here for examples.
Finally, welcome to send us PR of adding custom operators for ONNX Runtime in MMDeploy. :nerd_face: