How to write config¶

This tutorial describes how to write a config for model conversion and deployment. A deployment config includes onnx config, codebase config, backend config.

How to write config

1. How to write onnx config¶

Onnx config to describe how to export a model from pytorch to onnx.

Description of onnx config arguments¶

type: Type of config dict. Default is onnx.
export_params: If specified, all parameters will be exported. Set this to False if you want to export an untrained model.
keep_initializers_as_inputs: If True, all the initializers (typically corresponding to parameters) in the exported graph will also be added as inputs to the graph. If False, then initializers are not added as inputs to the graph, and only the non-parameter inputs are added as inputs.
opset_version: Opset_version is 11 by default.
save_file: Output onnx file.
input_names: Names to assign to the input nodes of the graph.
output_names: Names to assign to the output nodes of the graph.
input_shape: The height and width of input tensor to the model.

Example¶

onnx_config = dict(
    type='onnx',
    export_params=True,
    keep_initializers_as_inputs=False,
    opset_version=11,
    save_file='end2end.onnx',
    input_names=['input'],
    output_names=['output'],
    input_shape=None)

If you need to use dynamic axes¶

If the dynamic shape of inputs and outputs is required, you need to add dynamic_axes dict in onnx config.

dynamic_axes: Describe the dimensional information about input and output.

Example¶

    dynamic_axes={
        'input': {
            0: 'batch',
            2: 'height',
            3: 'width'
        },
        'dets': {
            0: 'batch',
            1: 'num_dets',
        },
        'labels': {
            0: 'batch',
            1: 'num_dets',
        },
    }

2. How to write codebase config¶

Codebase config part contains information like codebase type and task type.

Description of codebase config arguments¶

type: Model’s codebase, including mmpretrain, mmdet, mmseg, mmocr, mmagic.
task: Model’s task type, referring to List of tasks in all codebases.

Example¶

codebase_config = dict(type='mmpretrain', task='Classification')

3. How to write backend config¶

The backend config is mainly used to specify the backend on which model runs and provide the information needed when the model runs on the backend , referring to ONNX Runtime, TensorRT, ncnn, PPLNN.

type: Model’s backend, including onnxruntime, ncnn, pplnn, tensorrt, openvino.

Example¶

backend_config = dict(
    type='tensorrt',
    common_config=dict(
        fp16_mode=False, max_workspace_size=1 << 30),
    model_inputs=[
        dict(
            input_shapes=dict(
                input=dict(
                    min_shape=[1, 3, 512, 1024],
                    opt_shape=[1, 3, 1024, 2048],
                    max_shape=[1, 3, 2048, 2048])))
    ])

4. A complete example of mmpretrain on TensorRT¶

Here we provide a complete deployment config from mmpretrain on TensorRT.

codebase_config = dict(type='mmpretrain', task='Classification')

backend_config = dict(
    type='tensorrt',
    common_config=dict(
        fp16_mode=False,
        max_workspace_size=1 << 30),
    model_inputs=[
        dict(
            input_shapes=dict(
                input=dict(
                    min_shape=[1, 3, 224, 224],
                    opt_shape=[4, 3, 224, 224],
                    max_shape=[64, 3, 224, 224])))])

onnx_config = dict(
    type='onnx',
    dynamic_axes={
        'input': {
            0: 'batch',
            2: 'height',
            3: 'width'
        },
        'output': {
            0: 'batch'
        }
    },
    export_params=True,
    keep_initializers_as_inputs=False,
    opset_version=11,
    save_file='end2end.onnx',
    input_names=['input'],
    output_names=['output'],
    input_shape=[224, 224])

5. The name rules of our deployment config¶

There is a specific naming convention for the filename of deployment config files.

(task name)_(backend name)_(dynamic or static).py

task name: Model’s task type.
backend name: Backend’s name. Note if you use the quantization function, you need to indicate the quantization type. Just like tensorrt-int8.
dynamic or static: Dynamic or static export. Note if the backend needs explicit shape information, you need to add a description of input size with height x width format. Just like dynamic-512x1024-2048x2048, it means that the min input shape is 512x1024 and the max input shape is 2048x2048.

Example¶

detection_tensorrt-int8_dynamic-320x320-1344x1344.py

6. How to write model config¶

According to model’s codebase, write the model config file. Model’s config file is used to initialize the model, referring to MMPretrain, MMDetection, MMSegmentation, MMOCR, MMagic.