Shortcuts

Build for Linux-x86_64


Install Toolchains

  • cmake

    Make sure cmake version >= 3.14.0. The below script shows how to install cmake 3.20.0. You can find more versions here.

    wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0-linux-x86_64.tar.gz
    tar -xzvf cmake-3.20.0-linux-x86_64.tar.gz
    sudo ln -sf $(pwd)/cmake-3.20.0-linux-x86_64/bin/* /usr/bin/
    
  • GCC 7+

    MMDeploy requires compilers that support C++17.

    # Add repository if ubuntu < 18.04
    sudo add-apt-repository ppa:ubuntu-toolchain-r/test
    sudo apt-get update
    sudo apt-get install gcc-7
    sudo apt-get install g++-7
    

Install Dependencies

Install Dependencies for Model Converter

NAME INSTALLATION
conda Please install conda according to the official guide.
Create a conda virtual environment and activate it.

conda create -n mmdeploy python=3.7 -y
conda activate mmdeploy
PyTorch
(>=1.8.0)
Install PyTorch>=1.8.0 by following the official instructions. Be sure the CUDA version PyTorch requires matches that in your host.

conda install pytorch==1.8.0 torchvision==0.9.0 cudatoolkit=11.1 -c pytorch -c conda-forge
mmcv Install mmcv as follows. Refer to the guide for details.

export cu_version=cu111 # cuda 11.1
export torch_version=torch1.8
pip install -U openmim
mim install mmengine
mim install "mmcv>=2.0.0rc2"

Install Dependencies for SDK

You can skip this chapter if you are only interested in the model converter.

NAME INSTALLATION
OpenCV
(>=3.0)
On Ubuntu >=18.04,

sudo apt-get install libopencv-dev
On Ubuntu 16.04, OpenCV has to be built from the source code. Please refer to the guide.
pplcv A high-performance image processing library of openPPL.
It is optional which only be needed if cuda platform is required.

git clone https://github.com/openppl-public/ppl.cv.git
cd ppl.cv
export PPLCV_DIR=$(pwd)
git checkout tags/v0.7.0 -b v0.7.0
./build.sh cuda

Install Inference Engines for MMDeploy

Both MMDeploy’s model converter and SDK share the same inference engines.

You can select you interested inference engines and do the installation by following the given commands.

NAME PACKAGE INSTALLATION
ONNXRuntime onnxruntime
(>=1.8.1)
1. Install python package
pip install onnxruntime==1.8.1
2. Download the linux prebuilt binary package from here. Extract it and export environment variables as below:

wget https://github.com/microsoft/onnxruntime/releases/download/v1.8.1/onnxruntime-linux-x64-1.8.1.tgz
tar -zxvf onnxruntime-linux-x64-1.8.1.tgz
cd onnxruntime-linux-x64-1.8.1
export ONNXRUNTIME_DIR=$(pwd)
export LD_LIBRARY_PATH=$ONNXRUNTIME_DIR/lib:$LD_LIBRARY_PATH
TensorRT
TensorRT
1. Login NVIDIA and download the TensorRT tar file that matches the CPU architecture and CUDA version you are using from here. Follow the guide to install TensorRT.
2. Here is an example of installing TensorRT 8.2 GA Update 2 for Linux x86_64 and CUDA 11.x that you can refer to. First of all, click here to download CUDA 11.x TensorRT 8.2.3.0 and then install it and other dependency like below:

cd /the/path/of/tensorrt/tar/gz/file
tar -zxvf TensorRT-8.2.3.0.Linux.x86_64-gnu.cuda-11.4.cudnn8.2.tar.gz
pip install TensorRT-8.2.3.0/python/tensorrt-8.2.3.0-cp37-none-linux_x86_64.whl
export TENSORRT_DIR=$(pwd)/TensorRT-8.2.3.0
export LD_LIBRARY_PATH=$TENSORRT_DIR/lib:$LD_LIBRARY_PATH
pip install pycuda
cuDNN 1. Download cuDNN that matches the CPU architecture, CUDA version and TensorRT version you are using from cuDNN Archive.
In the above TensorRT's installation example, it requires cudnn8.2. Thus, you can download CUDA 11.x cuDNN 8.2
2. Extract the compressed file and set the environment variables

cd /the/path/of/cudnn/tgz/file
tar -zxvf cudnn-11.3-linux-x64-v8.2.1.32.tgz
export CUDNN_DIR=$(pwd)/cuda
export LD_LIBRARY_PATH=$CUDNN_DIR/lib64:$LD_LIBRARY_PATH
PPL.NN ppl.nn 1. Please follow the guide to build ppl.nn and install pyppl.
2. Export pplnn's root path to environment variable

cd ppl.nn
export PPLNN_DIR=$(pwd)
OpenVINO openvino 1. Install OpenVINO package

pip install openvino-dev
2. Optional. If you want to use OpenVINO in MMDeploy SDK, please install and configure it by following the guide.
ncnn ncnn 1. Download and build ncnn according to its wiki. Make sure to enable -DNCNN_PYTHON=ON in your build command.
2. Export ncnn's root path to environment variable

cd ncnn
export NCNN_DIR=$(pwd)
export LD_LIBRARY_PATH=${NCNN_DIR}/build/install/lib/:$LD_LIBRARY_PATH
3. Install pyncnn

cd ${NCNN_DIR}/python
pip install -e .
TorchScript libtorch 1. Download libtorch from here. Please note that only Pre-cxx11 ABI and version 1.8.1+ on Linux platform are supported by now. For previous versions of libtorch, you can find them in the issue comment.
2. Take Libtorch1.8.1+cu111 as an example. You can install it like this:

wget https://download.pytorch.org/libtorch/cu111/libtorch-shared-with-deps-1.8.1%2Bcu111.zip
unzip libtorch-shared-with-deps-1.8.1+cu111.zip
cd libtorch
export Torch_DIR=$(pwd)
export LD_LIBRARY_PATH=$Torch_DIR/lib:$LD_LIBRARY_PATH
   
Ascend CANN 1. Install CANN follow official guide.
2. Setup environment

export ASCEND_TOOLKIT_HOME="/usr/local/Ascend/ascend-toolkit/latest"
   
TVM TVM 1. Install TVM follow official guide.
2. Setup environment

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${TVM_HOME}/build
export PYTHONPATH=${TVM_HOME}/python:${PYTHONPATH}
  

Note:
If you want to make the above environment variables permanent, you could add them to ~/.bashrc. Take the ONNXRuntime for example,

echo '# set env for onnxruntime' >> ~/.bashrc
echo "export ONNXRUNTIME_DIR=${ONNXRUNTIME_DIR}" >> ~/.bashrc
echo "export LD_LIBRARY_PATH=$ONNXRUNTIME_DIR/lib:$LD_LIBRARY_PATH" >> ~/.bashrc
source ~/.bashrc

Build MMDeploy

cd /the/root/path/of/MMDeploy
export MMDEPLOY_DIR=$(pwd)

Build Model Converter

If one of inference engines among ONNXRuntime, TensorRT, ncnn and libtorch is selected, you have to build the corresponding custom ops.

  • ONNXRuntime Custom Ops

    cd ${MMDEPLOY_DIR}
    mkdir -p build && cd build
    cmake -DCMAKE_CXX_COMPILER=g++-7 -DMMDEPLOY_TARGET_BACKENDS=ort -DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR} ..
    make -j$(nproc) && make install
    
  • TensorRT Custom Ops

    cd ${MMDEPLOY_DIR}
    mkdir -p build && cd build
    cmake -DCMAKE_CXX_COMPILER=g++-7 -DMMDEPLOY_TARGET_BACKENDS=trt -DTENSORRT_DIR=${TENSORRT_DIR} -DCUDNN_DIR=${CUDNN_DIR} ..
    make -j$(nproc) && make install
    
  • ncnn Custom Ops

    cd ${MMDEPLOY_DIR}
    mkdir -p build && cd build
    cmake -DCMAKE_CXX_COMPILER=g++-7 -DMMDEPLOY_TARGET_BACKENDS=ncnn -Dncnn_DIR=${NCNN_DIR}/build/install/lib/cmake/ncnn ..
    make -j$(nproc) && make install
    
  • TorchScript Custom Ops

    cd ${MMDEPLOY_DIR}
    mkdir -p build && cd build
    cmake -DCMAKE_CXX_COMPILER=g++-7 -DMMDEPLOY_TARGET_BACKENDS=torchscript -DTorch_DIR=${Torch_DIR} ..
    make -j$(nproc) && make install
    

Please check cmake build option.

Install Model Converter

cd ${MMDEPLOY_DIR}
mim install -e .

Note

  • Some dependencies are optional. Simply running pip install -e . will only install the minimum runtime requirements. To use optional dependencies, install them manually with pip install -r requirements/optional.txt or specify desired extras when calling pip (e.g. pip install -e .[optional]). Valid keys for the extras field are: all, tests, build, optional.

  • It is recommended to install patch for cuda10, otherwise GEMM related errors may occur when model runs

Build SDK and Demo

MMDeploy provides two recipes as shown below for building SDK with ONNXRuntime and TensorRT as inference engines respectively. You can also activate other engines after the model.

  • cpu + ONNXRuntime

    cd ${MMDEPLOY_DIR}
    mkdir -p build && cd build
    cmake .. \
        -DCMAKE_CXX_COMPILER=g++-7 \
        -DMMDEPLOY_BUILD_SDK=ON \
        -DMMDEPLOY_BUILD_SDK_PYTHON_API=ON \
        -DMMDEPLOY_BUILD_EXAMPLES=ON \
        -DMMDEPLOY_TARGET_DEVICES=cpu \
        -DMMDEPLOY_TARGET_BACKENDS=ort \
        -DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR}
    
    make -j$(nproc) && make install
    
  • cuda + TensorRT

    cd ${MMDEPLOY_DIR}
    mkdir -p build && cd build
    cmake .. \
        -DCMAKE_CXX_COMPILER=g++-7 \
        -DMMDEPLOY_BUILD_SDK=ON \
        -DMMDEPLOY_BUILD_SDK_PYTHON_API=ON \
        -DMMDEPLOY_BUILD_EXAMPLES=ON \
        -DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
        -DMMDEPLOY_TARGET_BACKENDS=trt \
        -Dpplcv_DIR=${PPLCV_DIR}/cuda-build/install/lib/cmake/ppl \
        -DTENSORRT_DIR=${TENSORRT_DIR} \
        -DCUDNN_DIR=${CUDNN_DIR}
    
    make -j$(nproc) && make install
    
  • pplnn

    cd ${MMDEPLOY_DIR}
    mkdir -p build && cd build
    cmake .. \
        -DCMAKE_CXX_COMPILER=g++-7 \
        -DMMDEPLOY_BUILD_SDK=ON \
        -DMMDEPLOY_BUILD_EXAMPLES=ON \
        -DMMDEPLOY_BUILD_SDK_PYTHON_API=ON \
        -DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
        -DMMDEPLOY_TARGET_BACKENDS=pplnn \
        -Dpplcv_DIR=${PPLCV_DIR}/cuda-build/cuda-build/install/lib/cmake/ppl \
        -Dpplnn_DIR=${PPLNN_DIR}/pplnn-build/install/lib/cmake/ppl
    
    make -j$(nproc) && make install
    
  • cuda + TensorRT + onnxruntime + openvino + ncnn

    If the ncnn auto-install script is used, protobuf will be installed in mmdeploy-dep/pbinstall in the same directory as mmdeploy.

    export PROTO_DIR=/path/to/mmdeploy-dep/pbinstall
    cmake .. \
        -DCMAKE_CXX_COMPILER=g++-7 \
        -DMMDEPLOY_BUILD_SDK=ON \
        -DMMDEPLOY_BUILD_EXAMPLES=ON \
        -DMMDEPLOY_BUILD_SDK_PYTHON_API=ON \
        -DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
        -DMMDEPLOY_TARGET_BACKENDS="trt;ort;ncnn;openvino" \
        -Dpplcv_DIR=${PPLCV_DIR}/cuda-build/install/lib/cmake/ppl \
        -DTENSORRT_DIR=${TENSORRT_DIR} \
        -DCUDNN_DIR=${CUDNN_DIR} \
        -DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR} \
        -DInferenceEngine_DIR=${OPENVINO_DIR}/runtime/cmake \
        -Dncnn_DIR=${NCNN_DIR}/build/install/lib/cmake/ncnn \
        -DProtobuf_LIBRARIES=${PROTO_DIR}/lib/libprotobuf.so \
        -DProtobuf_PROTOC_EXECUTABLE=${PROTO_DIR}/bin/protoc \
        -DProtobuf_INCLUDE_DIR=${PROTO_DIR}/pbinstall/include
    

Read the Docs v: latest
Versions
latest
stable
1.x
v1.3.0
v1.2.0
v1.1.0
v1.0.0
0.x
v0.14.0
Downloads
pdf
html
epub
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.