Shortcuts

Quantization test result

Currently mmdeploy support ncnn quantization

Quantize with ncnn

mmcls

model dataset fp32 top-1 (%) int8 top-1 (%)
ResNet-18 Cifar10 94.82 94.83
ResNeXt-32x4d-50 ImageNet-1k 77.90 78.20*
MobileNet V2 ImageNet-1k 71.86 71.43*
HRNet-W18* ImageNet-1k 76.75 76.25*

Note:

  • Because of the large amount of imagenet-1k data and ncnn has not released Vulkan int8 version, only part of the test set (4000/50000) is used.

  • The accuracy will vary after quantization, and it is normal for the classification model to increase by less than 1%.

OCR detection

model dataset fp32 hmean int8 hmean
PANet ICDAR2015 0.795 0.792 @thr=0.9
TextSnake CTW1500 0.817 0.818

Note: mmocr Uses ‘shapely’ to compute IoU, which results in a slight difference in accuracy

Pose detection

model dataset fp32 AP int8 AP
Hourglass COCO2017 0.717 0.713
S-ViPNAS-MobileNetV3 COCO2017 0.687 0.683
S-ViPNAS-Res50 COCO2017 0.701 0.696
S-ViPNAS-MobileNetV3 COCO Wholebody 0.459 0.445
S-ViPNAS-Res50 COCO Wholebody 0.484 0.476
S-ViPNAS-MobileNetV3_dark COCO Wholebody 0.499 0.481
S-ViPNAS-Res50_dark COCO Wholebody 0.520 0.511

Note: MMPose models are tested with flip_test explicitly set to False in model configs.

Super Resolution

model dataset fp32 PSNR/SSIM int8 PSNR/SSIM
EDSRx2 Set5 35.7733/0.9365 35.4266/0.9334
EDSRx4 Set5 30.2194/0.8498 29.9340/0.8409

mmseg

model dataset fp32 mIoU int8 mIoU
Fast-SCNN cityscapes 70.96 70.24

Note:

  • Int8 models of the Fast-SCNN requires ncnnoptimize.

  • NCNN will extract 512 images from the train as a calibration dataset

Read the Docs v: latest
Versions
latest
stable
dev-1.x
Downloads
pdf
html
epub
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.