OpenVINO Support¶

This tutorial is based on Linux systems like Ubuntu-18.04.

Installation¶

It is recommended to create a virtual environment for the project.

Install python package¶

Install OpenVINO. It is recommended to use the installer or install using pip. Installation example using pip:

pip install openvino-dev[onnx]==2022.3.0

Download OpenVINO runtime for SDK (Optional)¶

If you want to use OpenVINO in SDK, you need install OpenVINO with install_guides. Take openvino==2022.3.0 as example:

wget https://storage.openvinotoolkit.org/repositories/openvino/packages/2022.3/linux/l_openvino_toolkit_ubuntu20_2022.3.0.9052.9752fafe8eb_x86_64.tgz
tar xzf ./l_openvino_toolkit*.tgz
cd l_openvino*
export InferenceEngine_DIR=$pwd/runtime/cmake
bash ./install_dependencies/install_openvino_dependencies.sh

Build mmdeploy SDK with OpenVINO (Optional)¶

Install MMDeploy following the instructions.

cd ${MMDEPLOY_DIR} # To MMDeploy root directory
mkdir -p build && cd build
cmake -DMMDEPLOY_TARGET_DEVICES='cpu' -DMMDEPLOY_TARGET_BACKENDS=openvino -DInferenceEngine_DIR=${InferenceEngine_DIR} ..
make -j$(nproc) && make install

To work with models from MMDetection, you may need to install it additionally.

Usage¶

You could follow the instructions of tutorial How to convert model

Example:

python tools/deploy.py \
    configs/mmdet/detection/detection_openvino_static-300x300.py \
    /mmdetection_dir/mmdetection/configs/ssd/ssd300_coco.py \
    /tmp/snapshots/ssd300_coco_20210803_015428-d231a06e.pth \
    tests/data/tiger.jpeg \
    --work-dir ../deploy_result \
    --device cpu \
    --log-level INFO

List of supported models exportable to OpenVINO from MMDetection¶

The table below lists the models that are guaranteed to be exportable to OpenVINO from MMDetection.

Model name	Config	Dynamic Shape
ATSS	`configs/atss/atss_r50_fpn_1x_coco.py`	Y
Cascade Mask R-CNN	`configs/cascade_rcnn/cascade_mask_rcnn_r50_fpn_1x_coco.py`	Y
Cascade R-CNN	`configs/cascade_rcnn/cascade_rcnn_r50_fpn_1x_coco.py`	Y
Faster R-CNN	`configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py`	Y
FCOS	`configs/fcos/fcos_x101_64x4d_fpn_gn-head_mstrain_640-800_4x2_2x_coco.py`	Y
FoveaBox	`configs/foveabox/fovea_r50_fpn_4x4_1x_coco.py`	Y
FSAF	`configs/fsaf/fsaf_r50_fpn_1x_coco.py`	Y
Mask R-CNN	`configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py`	Y
RetinaNet	`configs/retinanet/retinanet_r50_fpn_1x_coco.py`	Y
SSD	`configs/ssd/ssd300_coco.py`	Y
YOLOv3	`configs/yolo/yolov3_d53_mstrain-608_273e_coco.py`	Y
YOLOX	`configs/yolox/yolox_tiny_8x8_300e_coco.py`	Y
Faster R-CNN + DCN	`configs/dcn/faster_rcnn_r50_fpn_dconv_c3-c5_1x_coco.py`	Y
VFNet	`configs/vfnet/vfnet_r50_fpn_1x_coco.py`	Y

Notes:

Custom operations from OpenVINO use the domain org.openvinotoolkit.
For faster work in OpenVINO in the Faster-RCNN, Mask-RCNN, Cascade-RCNN, Cascade-Mask-RCNN models the RoiAlign operation is replaced with the ExperimentalDetectronROIFeatureExtractor operation in the ONNX graph.
Models “VFNet” and “Faster R-CNN + DCN” use the custom “DeformableConv2D” operation.

Deployment config¶

With the deployment config, you can specify additional options for the Model Optimizer. To do this, add the necessary parameters to the backend_config.mo_options in the fields args (for parameters with values) and flags (for flags).

Example:

backend_config = dict(
    mo_options=dict(
        args=dict({
            '--mean_values': [0, 0, 0],
            '--scale_values': [255, 255, 255],
            '--data_type': 'FP32',
        }),
        flags=['--disable_fusing'],
    )
)

Information about the possible parameters for the Model Optimizer can be found in the documentation.

Troubleshooting¶

ImportError: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory

To resolve missing external dependency on Ubuntu*, execute the following command:
```
sudo apt-get install libpython3.7
```