Test on embedded device¶
Here are the test conclusions of our edge devices. You can directly obtain the results of your own environment with model profiling.
Software and hardware environment¶
host OS ubuntu 18.04
backend SNPE-1.59
device Mi11 (qcom 888)
mmpretrain¶
model | dataset | spatial | fp32 top-1 (%) | snpe gpu hybrid fp32 top-1 (%) | latency (ms) |
---|---|---|---|---|---|
ShuffleNetV2 | ImageNet-1k | 224x224 | 69.55 | 69.83* | 20±7 |
MobilenetV2 | ImageNet-1k | 224x224 | 71.86 | 72.14* | 15±6 |
tips:
The ImageNet-1k dataset is too large to test, only part of the dataset is used (8000/50000)
The heating of device will downgrade the frequency, so the time consumption will actually fluctuate. Here are the stable values after running for a period of time. This result is closer to the actual demand.
mmocr detection¶
model | dataset | spatial | fp32 hmean | snpe gpu hybrid hmean | latency(ms) |
---|---|---|---|---|---|
PANet | ICDAR2015 | 1312x736 | 0.795 | 0.785 @thr=0.9 | 3100±100 |
mmpose¶
model | dataset | spatial | snpe hybrid AR@IoU=0.50 | snpe hybrid AP@IoU=0.50 | latency(ms) |
---|---|---|---|---|---|
pose_hrnet_w32 | Animalpose | 256x256 | 0.997 | 0.989 | 630±50 |
tips:
Test
pose_hrnet
using AnimalPose’s test dataset instead of val dataset.
mmseg¶
model | dataset | spatial | mIoU | latency(ms) |
---|---|---|---|---|
fcn | Cityscapes | 512x1024 | 71.11 | 4915±500 |
tips:
fcn
works fine with 512x1024 size. Cityscapes dataset uses 1024x2048 resolution which causes device to reboot.
Notes¶
We needs to manually split the mmdet model into two parts. Because
In snpe source code,
onnx_to_ir.py
can only parse onnx input whileir_to_dlc.py
does not supporttopk
operatorUDO (User Defined Operator) does not work with
snpe-onnx-to-dlc
mmagic model
srcnn
requires cubic resize which snpe does not supportesrgan
converts fine, but loading the model causes the device to reboot
mmrotate depends on e2cnn and needs to be installed manually its Python3.6 compatible branch