1. 预测部署简介与总览

本章主要介绍PP-OCRv2系统的高性能推理方法、服务化部署方法以及端侧部署方法。通过本章的学习，您可以学习到：

  • 根据不同的场景挑选合适的预测部署方法
  • PP-OCRv2系列模型在不同场景下的推理方法
  • Paddle Inference、Paddle Serving、Paddle Lite的推理部署开发方法

1.1 简介

在前面几个章节中，我们通过模型训练的方法，得到了训练好的模型，在使用它去预测的时候，我们首先需要定义好模型，然后加载训练好的模型，再将预处理之后的数据送进网络中进行预测、后处理，得到最终的结果。使用这种方法去进行预测，调试方便，但是预测效率比较低下。

针对上面的问题，对于训练得到的模型，一般有下面2种离线预测的方式。

  1. 基于训练引擎推理，也就是和训练使用同一套引擎，它调试方便，便于我们迅速定位问题，验证正确性。多为Python语言。
  2. 基于预测引擎推理，将训练得到的模型进行转换，去掉与推理无关的部分，使用这种方法可以加速预测。多为Python语言或C++。

二者具体的区别如下所示。

基于训练引擎推理 基于预测引擎推理
特点 1. 和训练使用同一套引擎
2. 预测时需要定义网络模型
3. 不适合系统集成		 1. 需要转换模型，去掉与推理无关的部分
2. 预测时无需定义网络模型3. 适合系统集成
编程语言 多为Python Python或者C++
预测步骤 1. Python端定义网络结构
2. 准备输入数据
3. 加载训练模型
4. 执行预测		 1. 准备输入数据
2. 加载模型结构和模型参数
3.执行预测

在实际离线预测部署中，更推荐基于预测引擎进行推理。

从应用场景来看，飞桨针对不同应用场景，提供了下面一些推理部署方案。

1.PP-OCRv2预测部署实践 - 图1
飞桨的不同部署方案
具体地，PaddleOCR 针对不同应用场景，提供了三种预测部署方案。

  • Inference的离线预测，这种方式主要应用在对预测响应的及时性要求不高，特别是需要大量图片预测的场景。像文档电子化、广告信息提取等。虽然不能及时响应预测请求，但是没有网络延时，计算效率比较高，数据安全性很高。
  • Serving服务化部署，这种方式主要应用在对预测响应的及时性要求很高的场景，像商业化OCR的API接口、实时拍照翻译、拍题等场景。虽然这种方式能及时对预测需求及时响应，但是网络耗时开销比较大，GPU利用率往往不高，而且存在数据安全风险。
  • Lite端侧部署，这种方式主要希望模型部署到手机、机器人等端侧设备上，主要考虑部署方便和数据安全性，像手机APP里面身份证识别、银行卡识别，工业应用场景的仪表监控识别等。这种方法，对OCR模型的大小比较敏感。虽然没有网络延时，数据安全性很高，但是由于算力限制，预测效率不高。

本章基于PP-OCRv2，介绍文本检测、识别以及系统串联预测推理与部署过程。

1.2 环境准备

体验本章节内容需要首先下载PaddleOCR代码，安装相关依赖，具体命令如下。

  1. import os
  3. os.chdir("/home/aistudio")
  4. # 下载代码
  5. !git clone https://gitee.com/paddlepaddle/PaddleOCR.git
  6. os.chdir("/home/aistudio/PaddleOCR")
  7. # 安装运行所需要的whl包
  8. !pip install -U pip
  9. !pip install -r requirements.txt
  10. # VQA任务中需要用到该库
  11. !pip install paddlenlp==2.2.1
  13. # 导入一些库
  14. import cv2
  15. import matplotlib.pyplot as plt
  16. %matplotlib inline
  17. import numpy as np
  18. import os
  1. Cloning into 'PaddleOCR'...
  2. remote: Enumerating objects: 2335, done.
  3. remote: Counting objects: 100% (2335/2335), done.
  4. remote: Compressing objects: 100% (937/937), done.
  5. remote: Total 27956 (delta 1569), reused 2078 (delta 1382), pack-reused 25621
  6. Receiving objects: 100% (27956/27956), 245.96 MiB | 4.68 MiB/s, done.
  7. Resolving deltas: 100% (19430/19430), done.
  8. Checking connectivity... done.
  9. Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
  10. Collecting pip
  11. [?25l  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a4/6d/6463d49a933f547439d6b5b98b46af8742cc03ae83543e4d7688c2420f8b/pip-21.3.1-py3-none-any.whl (1.7MB)
  12.      |████████████████████████████████| 1.7MB 17.9MB/s eta 0:00:01
  13. [?25hInstalling collected packages: pip
  14.   Found existing installation: pip 19.2.3
  15.     Uninstalling pip-19.2.3:
  16.       Successfully uninstalled pip-19.2.3
  17. Successfully installed pip-21.3.1
  18. Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
  19. Collecting shapely
  20.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ae/20/33ce377bd24d122a4d54e22ae2c445b9b1be8240edb50040b40add950cd9/Shapely-1.8.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB)
  21.      |████████████████████████████████| 1.1 MB 27.3 MB/s            
  22. [?25hCollecting scikit-image
  23.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/9a/44/8f8c7f9c9de7fde70587a656d7df7d056e6f05192a74491f7bc074a724d0/scikit_image-0.19.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (13.3 MB)
  24.      |████████████████████████████████| 13.3 MB 74.2 MB/s            
  25. [?25hCollecting imgaug==0.4.0
  26.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/66/b1/af3142c4a85cba6da9f4ebb5ff4e21e2616309552caca5e8acefe9840622/imgaug-0.4.0-py2.py3-none-any.whl (948 kB)
  27.      |████████████████████████████████| 948 kB 53.5 MB/s            
  28. [?25hCollecting pyclipper
  29.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/c5/fa/2c294127e4f88967149a68ad5b3e43636e94e3721109572f8f17ab15b772/pyclipper-1.3.0.post2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (603 kB)
  30.      |████████████████████████████████| 603 kB 55.1 MB/s            
  31. [?25hCollecting lmdb
  32.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/2e/dd/ada2fd91cd7832979069c556607903f274470c3d3d2274e0a848908272e8/lmdb-1.2.1-cp37-cp37m-manylinux2010_x86_64.whl (299 kB)
  33.      |████████████████████████████████| 299 kB 59.7 MB/s            
  34. [?25hRequirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 6)) (4.27.0)
  35. Requirement already satisfied: numpy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 7)) (1.20.3)
  36. Requirement already satisfied: visualdl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 8)) (2.2.0)
  37. Collecting python-Levenshtein
  38.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/2a/dc/97f2b63ef0fa1fd78dcb7195aca577804f6b2b51e712516cc0e902a9a201/python-Levenshtein-0.12.2.tar.gz (50 kB)
  39.      |████████████████████████████████| 50 kB 16.0 MB/s            
  40. [?25h  Preparing metadata (setup.py) ... [?25ldone
  41. [?25hCollecting opencv-contrib-python==4.4.0.46
  42.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/08/51/1e0a206dd5c70fea91084e6f43979dc13e8eb175760cc7a105083ec3eb68/opencv_contrib_python-4.4.0.46-cp37-cp37m-manylinux2014_x86_64.whl (55.7 MB)
  43.      |████████████████████████████████| 55.7 MB 49 kB/s             
  44. [?25hRequirement already satisfied: cython in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 11)) (0.29)
  45. Collecting lxml
  46.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7b/01/16a9b80c8ce4339294bb944f08e157dbfcfbb09ba9031bde4ddf7e3e5499/lxml-4.7.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (6.4 MB)
  47.      |████████████████████████████████| 6.4 MB 5.1 MB/s            
  48. [?25hCollecting premailer
  49.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/b1/07/4e8d94f94c7d41ca5ddf8a9695ad87b888104e2fd41a35546c1dc9ca74ac/premailer-3.10.0-py2.py3-none-any.whl (19 kB)
  50. Requirement already satisfied: openpyxl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 14)) (3.0.5)
  51. Collecting fasttext==0.9.1
  52.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/10/61/2e01f1397ec533756c1d893c22d9d5ed3fce3a6e4af1976e0d86bb13ea97/fasttext-0.9.1.tar.gz (57 kB)
  53.      |████████████████████████████████| 57 kB 12.8 MB/s            
  54. [?25h  Preparing metadata (setup.py) ... [?25ldone
  55. [?25hRequirement already satisfied: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (2.2.3)
  56. Requirement already satisfied: six in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (1.15.0)
  57. Requirement already satisfied: opencv-python in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (4.1.1.26)
  58. Requirement already satisfied: imageio in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (2.6.1)
  59. Requirement already satisfied: scipy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (1.6.3)
  60. Requirement already satisfied: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (7.1.2)
  61. Collecting pybind11>=2.2
  62.   Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a8/3b/fc246e1d4c7547a7a07df830128e93c6215e9b93dcb118b2a47a70726153/pybind11-2.8.1-py2.py3-none-any.whl (208 kB)
  63. Requirement already satisfied: setuptools>=0.7.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fasttext==0.9.1->-r requirements.txt (line 15)) (56.2.0)
  64. Collecting PyWavelets>=1.1.1
  65.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a1/9c/564511b6e1c4e1d835ed2d146670436036960d09339a8fa2921fe42dad08/PyWavelets-1.2.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (6.1 MB)
  66.      |████████████████████████████████| 6.1 MB 49.8 MB/s            
  67. [?25hRequirement already satisfied: packaging>=20.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r requirements.txt (line 2)) (20.9)
  68. Collecting tifffile>=2019.7.26
  69.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d8/38/85ae5ed77598ca90558c17a2f79ddaba33173b31cf8d8f545d34d9134f0d/tifffile-2021.11.2-py3-none-any.whl (178 kB)
  70.      |████████████████████████████████| 178 kB 56.8 MB/s            
  71. [?25hRequirement already satisfied: networkx>=2.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r requirements.txt (line 2)) (2.4)
  72. Requirement already satisfied: flask>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (1.1.1)
  73. Requirement already satisfied: pre-commit in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (1.21.0)
  74. Requirement already satisfied: shellcheck-py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (0.7.1.1)
  75. Requirement already satisfied: protobuf>=3.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (3.14.0)
  76. Requirement already satisfied: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (0.8.53)
  77. Requirement already satisfied: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (2.22.0)
  78. Requirement already satisfied: pandas in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (1.1.5)
  79. Requirement already satisfied: Flask-Babel>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (1.0.0)
  80. Requirement already satisfied: flake8>=3.7.9 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (3.8.2)
  81. Collecting cssutils
  82.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/24/c4/9db28fe567612896d360ab28ad02ee8ae107d0e92a22db39affd3fba6212/cssutils-2.3.0-py3-none-any.whl (404 kB)
  83.      |████████████████████████████████| 404 kB 43.6 MB/s            
  84. [?25hCollecting cssselect
  85.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/3b/d4/3b5c17f00cce85b9a1e6f91096e1cc8e8ede2e1be8e96b87ce1ed09e92c5/cssselect-1.1.0-py2.py3-none-any.whl (16 kB)
  86. Requirement already satisfied: cachetools in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from premailer->-r requirements.txt (line 13)) (4.0.0)
  87. Requirement already satisfied: jdcal in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from openpyxl->-r requirements.txt (line 14)) (1.4.1)
  88. Requirement already satisfied: et-xmlfile in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from openpyxl->-r requirements.txt (line 14)) (1.0.1)
  89. Requirement already satisfied: importlib-metadata in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (0.23)
  90. Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (0.6.1)
  91. Requirement already satisfied: pycodestyle<2.7.0,>=2.6.0a1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (2.6.0)
  92. Requirement already satisfied: pyflakes<2.3.0,>=2.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (2.2.0)
  93. Requirement already satisfied: click>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (7.0)
  94. Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (1.1.0)
  95. Requirement already satisfied: Jinja2>=2.10.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (2.11.0)
  96. Requirement already satisfied: Werkzeug>=0.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (0.16.0)
  97. Requirement already satisfied: pytz in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel>=1.0.0->visualdl->-r requirements.txt (line 8)) (2019.3)
  98. Requirement already satisfied: Babel>=2.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel>=1.0.0->visualdl->-r requirements.txt (line 8)) (2.8.0)
  99. Requirement already satisfied: decorator>=4.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from networkx>=2.2->scikit-image->-r requirements.txt (line 2)) (4.4.2)
  100. Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from packaging>=20.0->scikit-image->-r requirements.txt (line 2)) (2.4.2)
  101. Requirement already satisfied: future>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl->-r requirements.txt (line 8)) (0.18.0)
  102. Requirement already satisfied: pycryptodome>=3.8.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl->-r requirements.txt (line 8)) (3.9.9)
  103. Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->-r requirements.txt (line 3)) (2.8.0)
  104. Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->-r requirements.txt (line 3)) (0.10.0)
  105. Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->-r requirements.txt (line 3)) (1.1.0)
  106. Requirement already satisfied: identify>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (1.4.10)
  107. Requirement already satisfied: nodeenv>=0.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (1.3.4)
  108. Requirement already satisfied: virtualenv>=15.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (16.7.9)
  109. Requirement already satisfied: pyyaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (5.1.2)
  110. Requirement already satisfied: toml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (0.10.0)
  111. Requirement already satisfied: aspy.yaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (1.3.0)
  112. Requirement already satisfied: cfgv>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (2.0.1)
  113. Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl->-r requirements.txt (line 8)) (3.0.4)
  114. Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl->-r requirements.txt (line 8)) (2019.9.11)
  115. Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl->-r requirements.txt (line 8)) (2.8)
  116. Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl->-r requirements.txt (line 8)) (1.25.6)
  117. Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (1.1.1)
  118. Requirement already satisfied: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from importlib-metadata->flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (3.6.0)
  119. Building wheels for collected packages: fasttext, python-Levenshtein
  120.   Building wheel for fasttext (setup.py) ... [?25ldone
  121. [?25h  Created wheel for fasttext: filename=fasttext-0.9.1-cp37-cp37m-linux_x86_64.whl size=2585528 sha256=e9fa045840a7c64a81fede292ad670651aa5fe46380135f54ee64b17d9030594
  122.   Stored in directory: /home/aistudio/.cache/pip/wheels/a1/cb/b3/a25a8ce16c1a4ff102c1e40d6eaa4dfc9d5695b92d57331b36
  123.   Building wheel for python-Levenshtein (setup.py) ... [?25ldone
  124. [?25h  Created wheel for python-Levenshtein: filename=python_Levenshtein-0.12.2-cp37-cp37m-linux_x86_64.whl size=171687 sha256=8acb827b504424dfde1dae12b21c2d0ebbc5df8b46d73086c5c14cae7bb908fd
  125.   Stored in directory: /home/aistudio/.cache/pip/wheels/38/b9/a4/3729726160fb103833de468adb5ce019b58543ae41d0b0e446
  126. Successfully built fasttext python-Levenshtein
  127. Installing collected packages: tifffile, PyWavelets, shapely, scikit-image, pybind11, lxml, cssutils, cssselect, python-Levenshtein, pyclipper, premailer, opencv-contrib-python, lmdb, imgaug, fasttext
  128. Successfully installed PyWavelets-1.2.0 cssselect-1.1.0 cssutils-2.3.0 fasttext-0.9.1 imgaug-0.4.0 lmdb-1.2.1 lxml-4.7.1 opencv-contrib-python-4.4.0.46 premailer-3.10.0 pybind11-2.8.1 pyclipper-1.3.0.post2 python-Levenshtein-0.12.2 scikit-image-0.19.1 shapely-1.8.0 tifffile-2021.11.2
  129. Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
  130. Collecting paddlenlp==2.2.1
  131.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/dd/cb/f15e5406718079a201b869b46705ef0882be2ffb6932472e44efc3d16a75/paddlenlp-2.2.1-py3-none-any.whl (1.0 MB)
  132.      |████████████████████████████████| 1.0 MB 11.9 MB/s            
  133. [?25hRequirement already satisfied: multiprocess in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp==2.2.1) (0.70.11.1)
  134. Requirement already satisfied: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp==2.2.1) (1.2.2)
  135. Requirement already satisfied: h5py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp==2.2.1) (2.9.0)
  136. Requirement already satisfied: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp==2.2.1) (4.1.0)
  137. Requirement already satisfied: colorama in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp==2.2.1) (0.4.4)
  138. Requirement already satisfied: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp==2.2.1) (0.42.1)
  139. Requirement already satisfied: six in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from h5py->paddlenlp==2.2.1) (1.15.0)
  140. Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from h5py->paddlenlp==2.2.1) (1.20.3)
  141. Requirement already satisfied: dill>=0.3.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from multiprocess->paddlenlp==2.2.1) (0.3.3)
  142. Requirement already satisfied: scikit-learn>=0.21.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from seqeval->paddlenlp==2.2.1) (0.24.2)
  143. Requirement already satisfied: scipy>=0.19.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp==2.2.1) (1.6.3)
  144. Requirement already satisfied: joblib>=0.11 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp==2.2.1) (0.14.1)
  145. Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp==2.2.1) (2.1.0)
  146. Installing collected packages: paddlenlp
  147.   Attempting uninstall: paddlenlp
  148.     Found existing installation: paddlenlp 2.1.1
  149.     Uninstalling paddlenlp-2.1.1:
  150.       Successfully uninstalled paddlenlp-2.1.1
  151. Successfully installed paddlenlp-2.2.1

2. 基于Paddle Inference的python推理

2.1 简介

在项目中，模型的推理性能直接影响项目成本，因此我们期望一个训练好的模型的模型可以拥有更快的推理速度。直接基于训练引擎进行预测，模型中包含与训练相关的算子，因此效率一般较低；而且需要定义模型，难以与训练代码解耦。Paddle Inference应运而生。它是飞桨的原生推理库，作用于服务器端和云端，提供高性能的推理能力。由于能力直接基于飞桨的训练算子，因此Paddle Inference 可以通用支持飞桨训练出的所有模型。

考虑到大家的使用场景差异很大，Paddle Inference针对不同平台不同的应用场景进行了深度的适配优化，做到高吞吐、低时延，保证了飞桨模型在服务器端即训即用，快速部署。

本章主要介绍基于Paddle Inference的PP-OCRv2预测推理过程，更多关于Paddle Inference的介绍可以参考：Paddle Inference 介绍。

在基于Paddle Inference进行模型推理时，一般有以下几个步骤。
image.png
PP-OCRv2系统包含文字检测、方向分类器和文字识别3个模型，下面分别介绍这3个模型基于Paddle Inference的推理过程。

2.2 PP-OCRv2文字检测模型推理

PaddleOCR中，在基于文字检测模型进行推理时，需要通过参数image_dir指定单张图像或者图像集合的路径、参数det_model_dir, 指定检测的 inference 模型路径。

下面进行最新的超轻量文本检测模型推理实战，更多的模型和使用方法请参考文本检测预测教程。

更多等算法超参数的介绍可以参考PaddleOCR Inference推理相关参数介绍。

2.2.1 准备数据和环境

在最开始已经安装好了Paddle以及相应的依赖，这里环境已经准备好了。

检测示例数据在doc/imgs文件夹下面，部分数据如下所示。

  1. # 切换目录
  2. os.chdir("/home/aistudio/PaddleOCR")
  4. # 查看数据
  5. !ls doc/imgs/
  7. # 选择2张图像可视化
  8. img1 = cv2.imread("doc/imgs/00006737.jpg")
  9. img2 = cv2.imread("doc/imgs/00056221.jpg")
  10. plt.figure(figsize=(15, 6))
  11. plt.subplot(1, 2, 1)
  12. plt.imshow(img1[:,:,::-1])
  13. plt.subplot(1, 2, 2)
  14. plt.imshow(img2[:,:,::-1])
  15. plt.show()
  1. 00006737.jpg  00056221.jpg  00111002.jpg  1.jpg        japan_1.jpg
  2. 00009282.jpg  00057937.jpg  00207393.jpg  french_0.jpg    japan_2.jpg
  3. 00015504.jpg  00059985.jpg  11.jpg      ger_1.jpg    korean_1.jpg
  4. 00018069.jpg  00077949.jpg  12.jpg      ger_2.jpg    model_prod_flow_ch.png

image.png

2.2.2 准备推理模型

下载推理模型并解压，放在inference目录下面。

  1. # 下载模型
  2. !mkdir inference
  3. !cd inference && wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar -O ch_PP-OCRv2_det_infer.tar && tar -xf ch_PP-OCRv2_det_infer.tar
  4. !tree -h inference/ch_PP-OCRv2_det_infer
  1. --2021-12-25 14:55:13--  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar
  2. Resolving paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)... 100.67.200.6
  3. Connecting to paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)|100.67.200.6|:443... connected.
  4. HTTP request sent, awaiting response... 200 OK
  5. Length: 3190272 (3.0M) [application/x-tar]
  6. Saving to: ‘ch_PP-OCRv2_det_infer.tar’
  8. ch_PP-OCRv2_det_inf 100%[===================>]   3.04M  --.-KB/s    in 0.07s   
  10. 2021-12-25 14:55:13 (42.2 MB/s) - ‘ch_PP-OCRv2_det_infer.tar’ saved [3190272/3190272]
  12. inference/ch_PP-OCRv2_det_infer
  13. ├── [2.2M]  inference.pdiparams
  14. ├── [ 23K]  inference.pdiparams.info
  15. └── [845K]  inference.pdmodel
  17. 0 directories, 3 files
  • 如果您希望导出自己训练得到的模型，使用Paddle Inference部署，那么可以使用下面的命令将预训练模型使用动转静的方法，转化为推理模型。
  1. # 参考代码
  2. # https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/tools/export_model.py
  3. # 下载预训练模型
  4. !wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_distill_train.tar && tar -xf ch_PP-OCRv2_det_distill_train.tar && rm ch_PP-OCRv2_det_distill_train.tar
  5. # 导出推理模型
  6. !python tools/export_model.py -c configs/det/ch_PP-OCRv2/ch_PP-OCRv2_det_cml.yml \
  7.     -o Global.pretrained_model="ch_PP-OCRv2_det_distill_train/best_accuracy" \
  8.     Global.save_inference_dir="./my_model"
  9. # PP-OCRv2检测模型包含3个子网络：教师、学生、学生2，因此导出时，包含3个子文件，实际推理时，使用其中1个学生网络进行推理即可
  10. !tree -h my_model
  1. --2021-12-25 14:55:21--  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_distill_train.tar
  2. Resolving paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)... 100.67.200.6
  3. Connecting to paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)|100.67.200.6|:443... connected.
  4. HTTP request sent, awaiting response... 200 OK
  5. Length: 63830016 (61M) [application/x-tar]
  6. Saving to: ‘ch_PP-OCRv2_det_distill_train.tar’
  8. ch_PP-OCRv2_det_dis 100%[===================>]  60.87M  81.4MB/s    in 0.7s    
  10. 2021-12-25 14:55:22 (81.4 MB/s) - ‘ch_PP-OCRv2_det_distill_train.tar’ saved [63830016/63830016]
  12. W1225 14:55:24.746377  1078 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1
  13. W1225 14:55:24.749907  1078 device_context.cc:465] device: 0, cuDNN Version: 7.6.
  14. [2021/12/25 14:55:30] root INFO: load pretrain successful from ch_PP-OCRv2_det_distill_train/best_accuracy
  15. [2021/12/25 14:55:31] root INFO: inference model is saved to ./my_model/Teacher/inference
  16. [2021/12/25 14:55:33] root INFO: inference model is saved to ./my_model/Student/inference
  17. [2021/12/25 14:55:34] root INFO: inference model is saved to ./my_model/Student2/inference
  18. my_model
  19. ├── [4.0K]  Student
  20. │   ├── [2.2M]  inference.pdiparams
  21. │   ├── [ 23K]  inference.pdiparams.info
  22. │   └── [961K]  inference.pdmodel
  23. ├── [4.0K]  Student2
  24. │   ├── [2.2M]  inference.pdiparams
  25. │   ├── [ 23K]  inference.pdiparams.info
  26. │   └── [962K]  inference.pdmodel
  27. └── [4.0K]  Teacher
  28.     ├── [ 47M]  inference.pdiparams
  29.     ├── [ 12K]  inference.pdiparams.info
  30.     └── [568K]  inference.pdmodel
  32. 3 directories, 9 files

2.2.3 文本检测功能初探

我们先来看看加载加载推理模型预测得到的结果。

  1. # 参考代码
  2. # https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/tools/infer/predict_det.py
  3. # 预测
  4. !python tools/infer/predict_det.py --image_dir="./doc/imgs/00018069.jpg" --det_model_dir="./inference/ch_PP-OCRv2_det_infer" --use_gpu=False
  6. # 读取图像并显示出来，显示结果
  7. plt.figure(figsize=(20, 8))
  8. img_ori = cv2.imread("./doc/imgs/00018069.jpg")
  9. img_out = cv2.imread("./inference_results/det_res_00018069.jpg")
  10. plt.subplot(1, 2, 1)
  11. plt.imshow(img_ori[:,:,::-1])
  12. plt.subplot(1, 2, 2)
  13. plt.imshow(img_out[:,:,::-1])
  14. plt.show()
  1. [2021/12/25 14:56:01] root INFO: 00018069.jpg    [[[378, 249], [411, 252], [409, 266], [377, 265]], [[300, 250], [344, 250], [344, 265], [300, 265]], [[233, 250], [268, 250], [268, 265], [233, 265]], [[101, 249], [184, 249], [184, 266], [101, 266]], [[25, 250], [52, 250], [52, 266], [25, 266]], [[375, 233], [409, 233], [409, 247], [375, 247]], [[285, 233], [362, 233], [362, 247], [285, 247]], [[234, 233], [269, 233], [269, 247], [234, 247]], [[24, 234], [53, 234], [53, 247], [24, 247]], [[101, 230], [184, 230], [184, 248], [101, 248]], [[23, 214], [51, 214], [51, 229], [23, 229]], [[378, 213], [410, 213], [410, 228], [378, 228]], [[295, 211], [348, 214], [347, 229], [294, 226]], [[231, 213], [268, 213], [268, 228], [231, 228]], [[100, 212], [185, 212], [185, 229], [100, 229]], [[309, 195], [338, 195], [338, 209], [309, 209]], [[231, 194], [268, 194], [268, 209], [231, 209]], [[23, 194], [51, 194], [51, 210], [23, 210]], [[378, 193], [409, 193], [409, 209], [378, 209]], [[99, 193], [188, 193], [188, 208], [99, 208]], [[376, 175], [413, 175], [413, 190], [376, 190]], [[22, 175], [52, 175], [52, 192], [22, 192]], [[288, 173], [360, 173], [360, 191], [288, 191]], [[237, 174], [264, 174], [264, 190], [237, 190]], [[100, 173], [185, 173], [185, 191], [100, 191]], [[22, 156], [52, 156], [52, 172], [22, 172]], [[284, 154], [358, 154], [358, 172], [284, 172]], [[235, 155], [267, 155], [267, 171], [235, 171]], [[119, 155], [167, 155], [167, 170], [119, 170]], [[378, 138], [410, 135], [412, 152], [379, 155]], [[294, 136], [357, 136], [357, 153], [294, 153]], [[232, 137], [271, 137], [271, 152], [232, 152]], [[118, 137], [166, 137], [166, 151], [118, 151]], [[22, 137], [52, 137], [52, 152], [22, 152]], [[379, 119], [406, 119], [406, 135], [379, 135]], [[229, 118], [268, 118], [268, 134], [229, 134]], [[23, 118], [52, 118], [52, 134], [23, 134]], [[294, 114], [351, 117], [350, 135], [293, 132]], [[117, 113], [168, 116], [167, 135], [116, 132]], [[377, 99], [407, 99], [407, 115], [377, 115]], [[294, 99], [349, 99], [349, 114], [294, 114]], [[223, 99], [272, 99], [272, 114], [223, 114]], [[24, 99], [48, 99], [48, 115], [24, 115]], [[120, 98], [166, 98], [166, 114], [120, 114]], [[364, 80], [425, 80], [425, 95], [364, 95]], [[294, 81], [362, 81], [362, 93], [294, 93]], [[20, 80], [54, 80], [54, 95], [20, 95]], [[234, 79], [267, 79], [267, 94], [234, 94]], [[100, 76], [184, 78], [184, 96], [99, 94]], [[20, 62], [53, 62], [53, 77], [20, 77]], [[366, 62], [420, 59], [421, 75], [366, 77]], [[303, 60], [340, 60], [340, 75], [303, 75]], [[235, 60], [265, 60], [265, 76], [235, 76]], [[103, 60], [182, 60], [182, 75], [103, 75]], [[19, 43], [55, 43], [55, 58], [19, 58]], [[366, 43], [421, 40], [422, 55], [366, 58]], [[305, 42], [341, 42], [341, 57], [305, 57]], [[233, 42], [268, 42], [268, 57], [233, 57]], [[110, 42], [176, 42], [176, 56], [110, 56]], [[22, 24], [53, 24], [53, 40], [22, 40]], [[376, 23], [412, 23], [412, 38], [376, 38]], [[299, 23], [345, 23], [345, 38], [299, 38]], [[233, 23], [268, 23], [268, 39], [233, 39]], [[101, 21], [184, 21], [184, 40], [101, 40]], [[295, 3], [347, 3], [347, 20], [295, 20]], [[373, 1], [414, 1], [414, 20], [373, 20]], [[231, 1], [270, 1], [270, 20], [231, 20]], [[123, 1], [159, 1], [159, 21], [123, 21]], [[18, 1], [54, 1], [54, 21], [18, 21]]]
  3. [2021/12/25 14:56:01] root INFO: The predict time of ./doc/imgs/00018069.jpg: 0.6686491966247559
  4. [2021/12/25 14:56:01] root INFO: The visualized image saved in ./inference_results/det_res_00018069.jpg

image.png
那么具体是怎么实现的呢？下面给大家详细讲解下PP-OCRv2的推理模型加载与推理代码。

首先需要定义参数设置如下所示。更多参数的介绍可以参考：PaddleOCR推理过程参数介绍。

  1. # 参考代码
  2. # https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/tools/infer/utility.py
  3. import argparse
  4. import os
  5. import sys
  6. import cv2
  7. import numpy as np
  8. import paddle
  9. from PIL import Image, ImageDraw, ImageFont
  10. import math
  11. from paddle import inference
  12. import time
  13. from ppocr.utils.logging import get_logger
  16. def str2bool(v):
  17.     return v.lower() in ("true", "t", "1")
  20. def init_args():
  21.     parser = argparse.ArgumentParser()
  22.     # params for prediction engine
  23.     parser.add_argument("--use_gpu", type=str2bool, default=True)
  24.     parser.add_argument("--ir_optim", type=str2bool, default=True)
  25.     parser.add_argument("--use_tensorrt", type=str2bool, default=False)
  26.     parser.add_argument("--min_subgraph_size", type=int, default=15)
  27.     parser.add_argument("--precision", type=str, default="fp32")
  28.     parser.add_argument("--gpu_mem", type=int, default=500)
  30.     # params for text detector
  31.     parser.add_argument("--image_dir", type=str)
  32.     parser.add_argument("--det_algorithm", type=str, default='DB')
  33.     parser.add_argument("--det_model_dir", type=str)
  34.     parser.add_argument("--det_limit_side_len", type=float, default=960)
  35.     parser.add_argument("--det_limit_type", type=str, default='max')
  37.     # DB parmas
  38.     parser.add_argument("--det_db_thresh", type=float, default=0.3)
  39.     parser.add_argument("--det_db_box_thresh", type=float, default=0.6)
  40.     parser.add_argument("--det_db_unclip_ratio", type=float, default=1.5)
  41.     parser.add_argument("--max_batch_size", type=int, default=10)
  42.     parser.add_argument("--use_dilation", type=str2bool, default=False)
  43.     parser.add_argument("--det_db_score_mode", type=str, default="fast")
  44.     # EAST parmas
  45.     parser.add_argument("--det_east_score_thresh", type=float, default=0.8)
  46.     parser.add_argument("--det_east_cover_thresh", type=float, default=0.1)
  47.     parser.add_argument("--det_east_nms_thresh", type=float, default=0.2)
  49.     # SAST parmas
  50.     parser.add_argument("--det_sast_score_thresh", type=float, default=0.5)
  51.     parser.add_argument("--det_sast_nms_thresh", type=float, default=0.2)
  52.     parser.add_argument("--det_sast_polygon", type=str2bool, default=False)
  54.     # PSE parmas
  55.     parser.add_argument("--det_pse_thresh", type=float, default=0)
  56.     parser.add_argument("--det_pse_box_thresh", type=float, default=0.85)
  57.     parser.add_argument("--det_pse_min_area", type=float, default=16)
  58.     parser.add_argument("--det_pse_box_type", type=str, default='box')
  59.     parser.add_argument("--det_pse_scale", type=int, default=1)
  61.     # params for text recognizer
  62.     parser.add_argument("--rec_algorithm", type=str, default='CRNN')
  63.     parser.add_argument("--rec_model_dir", type=str)
  64.     parser.add_argument("--rec_image_shape", type=str, default="3, 32, 320")
  65.     parser.add_argument("--rec_batch_num", type=int, default=6)
  66.     parser.add_argument("--max_text_length", type=int, default=25)
  67.     parser.add_argument(
  68.         "--rec_char_dict_path",
  69.         type=str,
  70.         default="./ppocr/utils/ppocr_keys_v1.txt")
  71.     parser.add_argument("--use_space_char", type=str2bool, default=True)
  72.     parser.add_argument(
  73.         "--vis_font_path", type=str, default="./doc/fonts/simfang.ttf")
  74.     parser.add_argument("--drop_score", type=float, default=0.5)
  76.     # params for e2e
  77.     parser.add_argument("--e2e_algorithm", type=str, default='PGNet')
  78.     parser.add_argument("--e2e_model_dir", type=str)
  79.     parser.add_argument("--e2e_limit_side_len", type=float, default=768)
  80.     parser.add_argument("--e2e_limit_type", type=str, default='max')
  82.     # PGNet parmas
  83.     parser.add_argument("--e2e_pgnet_score_thresh", type=float, default=0.5)
  84.     parser.add_argument(
  85.         "--e2e_char_dict_path", type=str, default="./ppocr/utils/ic15_dict.txt")
  86.     parser.add_argument("--e2e_pgnet_valid_set", type=str, default='totaltext')
  87.     parser.add_argument("--e2e_pgnet_mode", type=str, default='fast')
  89.     # params for text classifier
  90.     parser.add_argument("--use_angle_cls", type=str2bool, default=False)
  91.     parser.add_argument("--cls_model_dir", type=str)
  92.     parser.add_argument("--cls_image_shape", type=str, default="3, 48, 192")
  93.     parser.add_argument("--label_list", type=list, default=['0', '180'])
  94.     parser.add_argument("--cls_batch_num", type=int, default=6)
  95.     parser.add_argument("--cls_thresh", type=float, default=0.9)
  97.     parser.add_argument("--enable_mkldnn", type=str2bool, default=False)
  98.     parser.add_argument("--cpu_threads", type=int, default=10)
  99.     parser.add_argument("--use_pdserving", type=str2bool, default=False)
  100.     parser.add_argument("--warmup", type=str2bool, default=False)
  102.     #
  103.     parser.add_argument(
  104.         "--draw_img_save_dir", type=str, default="./inference_results")
  105.     parser.add_argument("--save_crop_res", type=str2bool, default=False)
  106.     parser.add_argument("--crop_res_save_dir", type=str, default="./output")
  108.     # multi-process
  109.     parser.add_argument("--use_mp", type=str2bool, default=False)
  110.     parser.add_argument("--total_process_num", type=int, default=1)
  111.     parser.add_argument("--process_id", type=int, default=0)
  113.     parser.add_argument("--benchmark", type=str2bool, default=False)
  114.     parser.add_argument("--save_log_path", type=str, default="./log_output/")
  116.     parser.add_argument("--show_log", type=str2bool, default=True)
  117.     parser.add_argument("--use_onnx", type=str2bool, default=False)
  118.     # 这里需要注意，添加这个是因为直接在notebook中解析的话，sys.argv会在后面添加下面的内容，导致解析失败
  119.     # '-f', '/home/aistudio/.local/share/jupyter/runtime/kernel-e1221262-c656-4129-896f-1b197b6b782c.json'
  120.     parser.add_argument("-f", type=str, default=None)
  121.     return parser
  124. def parse_args():
  125.     parser = init_args()
  126.     return parser.parse_args()

下面和大家具体看下文字检测的具体代码。

  1. # 参考代码
  2. # https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/tools/infer/predict_det.py
  3. import os
  4. import sys
  5. import cv2
  6. import numpy as np
  7. import time
  9. import tools.infer.utility as utility
  10. from ppocr.utils.logging import get_logger
  11. from ppocr.utils.utility import get_image_file_list, check_and_read_gif
  12. from ppocr.data import create_operators, transform
  13. from ppocr.postprocess import build_post_process
  14. import json
  15. logger = get_logger()
  17. # 文字检测类
  18. class TextDetector(object):
  19.     def __init__(self, args):
  20.         self.args = args
  21.         self.det_algorithm = args.det_algorithm
  22.         pre_process_list = [{
  23.             'DetResizeForTest': {
  24.                 'limit_side_len': args.det_limit_side_len,
  25.                 'limit_type': args.det_limit_type,
  26.             }
  27.         }, {
  28.             'NormalizeImage': {
  29.                 'std': [0.229, 0.224, 0.225],
  30.                 'mean': [0.485, 0.456, 0.406],
  31.                 'scale': '1./255.',
  32.                 'order': 'hwc'
  33.             }
  34.         }, {
  35.             'ToCHWImage': None
  36.         }, {
  37.             'KeepKeys': {
  38.                 'keep_keys': ['image', 'shape']
  39.             }
  40.         }]
  41.         postprocess_params = {}
  42.         if self.det_algorithm == "DB":
  43.             postprocess_params['name'] = 'DBPostProcess'
  44.             postprocess_params["thresh"] = args.det_db_thresh
  45.             postprocess_params["box_thresh"] = args.det_db_box_thresh
  46.             postprocess_params["max_candidates"] = 1000
  47.             postprocess_params["unclip_ratio"] = args.det_db_unclip_ratio
  48.             postprocess_params["use_dilation"] = args.use_dilation
  49.             postprocess_params["score_mode"] = args.det_db_score_mode
  50.         else:
  51.             logger.info("unknown det_algorithm:{}".format(self.det_algorithm))
  52.             sys.exit(0)
  53.         # 初始化预测引擎
  54.         self.predictor, self.input_tensor, self.output_tensors, self.config = utility.create_predictor(
  55.             args, 'det', logger)
  56.         # 构建预处理算子
  57.         self.preprocess_op = create_operators(pre_process_list)
  58.         # 构建后处理算子
  59.         self.postprocess_op = build_post_process(postprocess_params)
  62.     def order_points_clockwise(self, pts):
  63.         """
  64.         参考: https://github.com/jrosebr1/imutils/blob/master/imutils/perspective.py
  65.         对检测出来的点进行按照顺时针排序
  66.         """
  67.         xSorted = pts[np.argsort(pts[:, 0]), :]
  69.         leftMost = xSorted[:2, :]
  70.         rightMost = xSorted[2:, :]
  72.         leftMost = leftMost[np.argsort(leftMost[:, 1]), :]
  73.         (tl, bl) = leftMost
  75.         rightMost = rightMost[np.argsort(rightMost[:, 1]), :]
  76.         (tr, br) = rightMost
  78.         rect = np.array([tl, tr, br, bl], dtype="float32")
  79.         return rect
  81.     def clip_det_res(self, points, img_height, img_width):
  82.         # 对检测结果根据宽高进行限幅，防止超出图像边界
  83.         for pno in range(points.shape[0]):
  84.             points[pno, 0] = int(min(max(points[pno, 0], 0), img_width - 1))
  85.             points[pno, 1] = int(min(max(points[pno, 1], 0), img_height - 1))
  86.         return points
  88.     def filter_tag_det_res(self, dt_boxes, image_shape):
  89.         # 去除小于特定尺寸的检测结果
  90.         img_height, img_width = image_shape[0:2]
  91.         dt_boxes_new = []
  92.         for box in dt_boxes:
  93.             box = self.order_points_clockwise(box)
  94.             box = self.clip_det_res(box, img_height, img_width)
  95.             rect_width = int(np.linalg.norm(box[0] - box[1]))
  96.             rect_height = int(np.linalg.norm(box[0] - box[3]))
  97.             if rect_width <= 3 or rect_height <= 3:
  98.                 continue
  99.             dt_boxes_new.append(box)
  100.         dt_boxes = np.array(dt_boxes_new)
  101.         return dt_boxes
  103.     def filter_tag_det_res_only_clip(self, dt_boxes, image_shape):
  104.         # 仅对检测结果的边界进行限幅
  105.         img_height, img_width = image_shape[0:2]
  106.         dt_boxes_new = []
  107.         for box in dt_boxes:
  108.             box = self.clip_det_res(box, img_height, img_width)
  109.             dt_boxes_new.append(box)
  110.         dt_boxes = np.array(dt_boxes_new)
  111.         return dt_boxes
  113.     def __call__(self, img):
  114.         ori_im = img.copy()
  115.         data = {'image': img}
  117.         st = time.time()
  119.         # 数据预处理
  120.         data = transform(data, self.preprocess_op)
  121.         img, shape_list = data
  122.         if img is None:
  123.             return None, 0
  124.         # 扩展bs维度：CHW -> NCHW
  125.         img = np.expand_dims(img, axis=0)
  126.         shape_list = np.expand_dims(shape_list, axis=0)
  127.         img = img.copy()
  128.         # 将数据拷贝到预测引擎中
  129.         self.input_tensor.copy_from_cpu(img)
  130.         # 自动推理
  131.         self.predictor.run()
  132.         outputs = []
  133.         # 将返回结果从预测引擎中拷贝回CPU
  134.         for output_tensor in self.output_tensors:
  135.             output = output_tensor.copy_to_cpu()
  136.             outputs.append(output)
  138.         preds = {}
  139.         if self.det_algorithm in ['DB', 'PSE']:
  140.             preds['maps'] = outputs[0]
  141.         else:
  142.             raise NotImplementedError
  144.         # 后处理
  145.         post_result = self.postprocess_op(preds, shape_list)
  146.         dt_boxes = post_result[0]['points']
  147.         dt_boxes = self.filter_tag_det_res(dt_boxes, ori_im.shape)
  149.         et = time.time()
  150.         return dt_boxes, et - st
  152. # 设置参数
  153. args = parse_args()
  154. args.det_model_dir = "./inference/ch_PP-OCRv2_det_infer"
  155. args.image_dir = "./doc/imgs/00018069.jpg"
  157. # 获取图片列表
  158. image_file_list = get_image_file_list(args.image_dir)
  159. # 创建文本检测器对象
  160. text_detector = TextDetector(args)
  162. count = 0
  163. total_time = 0
  164. draw_img_save = "./inference_results"
  166. if not os.path.exists(draw_img_save):
  167.     os.makedirs(draw_img_save)
  168. save_results = []
  169. for image_file in image_file_list:
  170.     img = cv2.imread(image_file)
  171.     if img is None:
  172.         logger.info("error in loading image:{}".format(image_file))
  173.         continue
  174.     st = time.time()
  175.     dt_boxes, _ = text_detector(img)
  176.     elapse = time.time() - st
  177.     if count > 0:
  178.         total_time += elapse
  179.     count += 1
  180.     save_pred = os.path.basename(image_file) + "\t" + str(
  181.         json.dumps(np.array(dt_boxes).astype(np.int32).tolist())) + "\n"
  182.     save_results.append(save_pred)
  183.     logger.info(save_pred)
  184.     logger.info("The predict time of {}: {}".format(image_file, elapse))
  185.     src_im = utility.draw_text_det_res(dt_boxes, image_file)
  186.     img_name_pure = os.path.split(image_file)[-1]
  187.     img_path = os.path.join(draw_img_save,
  188.                             "det_res_{}".format(img_name_pure))
  189.     cv2.imwrite(img_path, src_im)
  190.     logger.info("The visualized image saved in {}".format(img_path))
  192.     break
  194. with open(os.path.join(draw_img_save, "det_results.txt"), 'w') as f:
  195.     f.writelines(save_results)
  196.     f.close()
  198. plt.figure(figsize=(10, 10))
  199. plt.imshow(src_im[:, :, ::-1])
  200. plt.show()
  1. [2021/12/25 15:10:56] root INFO: 00018069.jpg    [[[378, 249], [411, 252], [409, 266], [377, 265]], [[300, 250], [344, 250], [344, 265], [300, 265]], [[233, 250], [268, 250], [268, 265], [233, 265]], [[101, 249], [184, 249], [184, 266], [101, 266]], [[25, 250], [52, 250], [52, 266], [25, 266]], [[375, 233], [409, 233], [409, 247], [375, 247]], [[285, 233], [362, 233], [362, 247], [285, 247]], [[234, 233], [269, 233], [269, 247], [234, 247]], [[24, 234], [53, 234], [53, 247], [24, 247]], [[101, 230], [184, 230], [184, 248], [101, 248]], [[23, 214], [51, 214], [51, 229], [23, 229]], [[378, 213], [410, 213], [410, 228], [378, 228]], [[295, 211], [348, 214], [347, 229], [294, 226]], [[231, 213], [268, 213], [268, 228], [231, 228]], [[100, 212], [185, 212], [185, 229], [100, 229]], [[309, 195], [338, 195], [338, 209], [309, 209]], [[231, 194], [268, 194], [268, 209], [231, 209]], [[23, 194], [51, 194], [51, 210], [23, 210]], [[378, 193], [409, 193], [409, 209], [378, 209]], [[99, 193], [188, 193], [188, 208], [99, 208]], [[376, 175], [413, 175], [413, 190], [376, 190]], [[22, 175], [52, 175], [52, 192], [22, 192]], [[288, 173], [360, 173], [360, 191], [288, 191]], [[237, 174], [264, 174], [264, 190], [237, 190]], [[100, 173], [185, 173], [185, 191], [100, 191]], [[22, 156], [52, 156], [52, 172], [22, 172]], [[284, 154], [358, 154], [358, 172], [284, 172]], [[235, 155], [267, 155], [267, 171], [235, 171]], [[119, 155], [167, 155], [167, 170], [119, 170]], [[378, 138], [410, 135], [412, 152], [379, 155]], [[294, 136], [357, 136], [357, 153], [294, 153]], [[232, 137], [271, 137], [271, 152], [232, 152]], [[118, 137], [166, 137], [166, 151], [118, 151]], [[22, 137], [52, 137], [52, 152], [22, 152]], [[379, 119], [406, 119], [406, 135], [379, 135]], [[229, 118], [268, 118], [268, 134], [229, 134]], [[23, 118], [52, 118], [52, 134], [23, 134]], [[294, 114], [351, 117], [350, 135], [293, 132]], [[117, 113], [168, 116], [167, 135], [116, 132]], [[377, 99], [407, 99], [407, 115], [377, 115]], [[294, 99], [349, 99], [349, 114], [294, 114]], [[223, 99], [272, 99], [272, 114], [223, 114]], [[24, 99], [48, 99], [48, 115], [24, 115]], [[120, 98], [166, 98], [166, 114], [120, 114]], [[364, 80], [425, 80], [425, 95], [364, 95]], [[294, 81], [362, 81], [362, 93], [294, 93]], [[20, 80], [54, 80], [54, 95], [20, 95]], [[234, 79], [267, 79], [267, 94], [234, 94]], [[100, 76], [184, 78], [184, 96], [99, 94]], [[20, 62], [53, 62], [53, 77], [20, 77]], [[366, 62], [420, 59], [421, 75], [366, 77]], [[303, 60], [340, 60], [340, 75], [303, 75]], [[235, 60], [265, 60], [265, 76], [235, 76]], [[103, 60], [182, 60], [182, 75], [103, 75]], [[19, 43], [55, 43], [55, 58], [19, 58]], [[366, 43], [421, 40], [422, 55], [366, 58]], [[305, 42], [341, 42], [341, 57], [305, 57]], [[233, 42], [268, 42], [268, 57], [233, 57]], [[110, 42], [176, 42], [176, 56], [110, 56]], [[22, 24], [53, 24], [53, 40], [22, 40]], [[376, 23], [412, 23], [412, 38], [376, 38]], [[299, 23], [345, 23], [345, 38], [299, 38]], [[233, 23], [268, 23], [268, 39], [233, 39]], [[101, 21], [184, 21], [184, 40], [101, 40]], [[295, 3], [347, 3], [347, 20], [295, 20]], [[373, 1], [414, 1], [414, 20], [373, 20]], [[231, 1], [270, 1], [270, 20], [231, 20]], [[123, 1], [159, 1], [159, 21], [123, 21]], [[18, 1], [54, 1], [54, 21], [18, 21]]]
  3. [2021/12/25 15:10:56] root INFO: The predict time of ./doc/imgs/00018069.jpg: 0.04716134071350098
  4. [2021/12/25 15:10:56] root INFO: The visualized image saved in ./inference_results/det_res_00018069.jpg

image.png
上面就完成了完整的文本检测流程。

2.3 PP-OCRv2方向分类器模型推理

同样地，对于方向分类器模型，我们也可以使用下面的命令快速体验其功能。

  1. # 下载模型
  2. !cd inference && wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar -O ch_ppocr_mobile_v2.0_cls_infer.tar && tar -xf ch_ppocr_mobile_v2.0_cls_infer.tar
  3. # 预测
  4. !python tools/infer/predict_cls.py \
  5.     --image_dir="./doc/imgs_words/ch/word_1.jpg" \
  6.     --cls_model_dir="./inference/ch_ppocr_mobile_v2.0_cls_infer" \
  7.     --use_gpu=False
  8. # 画图
  9. img = cv2.imread("./doc/imgs_words/ch/word_1.jpg")
  10. plt.imshow(img[:,:,::-1])
  11. plt.show()
  1. --2021-12-25 15:40:13--  https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar
  2. Resolving paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)... 100.67.200.6
  3. Connecting to paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)|100.67.200.6|:443... connected.
  4. HTTP request sent, awaiting response... 200 OK
  5. Length: 1454080 (1.4M) [application/x-tar]
  6. Saving to: ‘ch_ppocr_mobile_v2.0_cls_infer.tar’
  8. ch_ppocr_mobile_v2. 100%[===================>]   1.39M  --.-KB/s    in 0.04s   
  10. 2021-12-25 15:40:13 (33.6 MB/s) - ‘ch_ppocr_mobile_v2.0_cls_infer.tar’ saved [1454080/1454080]
  12. [2021/12/25 15:40:15] root INFO: Predicts of ./doc/imgs_words/ch/word_1.jpg:['0', 0.9998784]

image.png
图片的方向是正向水平文本，预测结果正确。

具体地，方向分类器的具体实现代码如下所示。

  1. # 参考代码
  2. # https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/tools/infer/predict_cls.py
  3. import copy
  5. # 方向分类器实现的类
  6. class TextClassifier(object):
  7.     def __init__(self, args):
  8.         self.cls_image_shape = [int(v) for v in args.cls_image_shape.split(",")]
  9.         self.cls_batch_num = args.cls_batch_num
  10.         self.cls_thresh = args.cls_thresh
  11.         postprocess_params = {
  12.             'name': 'ClsPostProcess',
  13.             "label_list": args.label_list,
  14.         }
  15.         # 后处理算子
  16.         self.postprocess_op = build_post_process(postprocess_params)
  17.         # 初始化预测引擎
  18.         self.predictor, self.input_tensor, self.output_tensors, _ = \
  19.             utility.create_predictor(args, 'cls', logger)
  21.     # 对图像进行resize并且normalize
  22.     def resize_norm_img(self, img):
  23.         imgC, imgH, imgW = self.cls_image_shape
  24.         h = img.shape[0]
  25.         w = img.shape[1]
  26.         ratio = w / float(h)
  27.         if math.ceil(imgH * ratio) > imgW:
  28.             resized_w = imgW
  29.         else:
  30.             resized_w = int(math.ceil(imgH * ratio))
  31.         resized_image = cv2.resize(img, (resized_w, imgH))
  32.         resized_image = resized_image.astype('float32')
  33.         if self.cls_image_shape[0] == 1:
  34.             resized_image = resized_image / 255
  35.             resized_image = resized_image[np.newaxis, :]
  36.         else:
  37.             resized_image = resized_image.transpose((2, 0, 1)) / 255
  38.         resized_image -= 0.5
  39.         resized_image /= 0.5
  40.         padding_im = np.zeros((imgC, imgH, imgW), dtype=np.float32)
  41.         padding_im[:, :, 0:resized_w] = resized_image
  42.         return padding_im
  44.     def __call__(self, img_list):
  45.         img_list = copy.deepcopy(img_list)
  46.         img_num = len(img_list)
  47.         # 记录宽高比
  48.         width_list = []
  49.         for img in img_list:
  50.             width_list.append(img.shape[1] / float(img.shape[0]))
  51.         # 排序，加速后续的预处理过程
  52.         indices = np.argsort(np.array(width_list))
  54.         cls_res = [['', 0.0]] * img_num
  55.         batch_num = self.cls_batch_num
  56.         elapse = 0
  57.         for beg_img_no in range(0, img_num, batch_num):
  58.             end_img_no = min(img_num, beg_img_no + batch_num)
  59.             norm_img_batch = []
  60.             max_wh_ratio = 0
  61.             starttime = time.time()
  62.             # 预处理数据，组batch
  63.             for ino in range(beg_img_no, end_img_no):
  64.                 h, w = img_list[indices[ino]].shape[0:2]
  65.                 wh_ratio = w * 1.0 / h
  66.                 max_wh_ratio = max(max_wh_ratio, wh_ratio)
  67.             for ino in range(beg_img_no, end_img_no):
  68.                 norm_img = self.resize_norm_img(img_list[indices[ino]])
  69.                 norm_img = norm_img[np.newaxis, :]
  70.                 norm_img_batch.append(norm_img)
  71.             norm_img_batch = np.concatenate(norm_img_batch)
  72.             norm_img_batch = norm_img_batch.copy()
  73.             # 将数据拷贝到预测引擎
  74.             self.input_tensor.copy_from_cpu(norm_img_batch)
  75.             # 自动推理过程
  76.             self.predictor.run()
  77.             # 将数据拷贝回CPU
  78.             prob_out = self.output_tensors[0].copy_to_cpu()
  79.             # 后处理
  80.             cls_result = self.postprocess_op(prob_out)
  81.             elapse += time.time() - starttime
  82.             for rno in range(len(cls_result)):
  83.                 label, score = cls_result[rno]
  84.                 cls_res[indices[beg_img_no + rno]] = [label, score]
  85.                 if '180' in label and score > self.cls_thresh:
  86.                     img_list[indices[beg_img_no + rno]] = cv2.rotate(
  87.                         img_list[indices[beg_img_no + rno]], 1)
  88.         return img_list, cls_res, elapse
  90. args = parse_args()
  91. args.cls_model_dir = "./inference/ch_ppocr_mobile_v2.0_cls_infer"
  92. args.image_dir = "./doc/imgs_words/ch/word_4.jpg"
  94. image_file_list = get_image_file_list(args.image_dir)
  95. text_classifier = TextClassifier(args)
  96. valid_image_file_list = []
  97. img_list = []
  98. for image_file in image_file_list:
  99.     img = cv2.imread(image_file)
  100.     # 预测之前对图像旋转180度
  101.     # img = cv2.rotate(img, cv2.ROTATE_180)
  102.     if img is None:
  103.         logger.info("error in loading image:{}".format(image_file))
  104.         continue
  105.     valid_image_file_list.append(image_file)
  106.     img_list.append(img)
  107. img_list, cls_res, predict_time = text_classifier(img_list)
  108. for ino in range(len(img_list)):
  109.     logger.info("Predicts of {}:{}".format(valid_image_file_list[ino],
  110.                                             cls_res[ino]))
  112. plt.imshow(img[:,:,::-1])
  113. plt.show()
  1. [2021/12/25 15:43:28] root INFO: Predicts of ./doc/imgs_words/ch/word_4.jpg:['0', 0.9999982]

image.png
这里我们也可以将图像旋转180度之后，看下方向分类器的分类效果。

上面就完成了完整的方向分类器的推理过程。

2.4 PP-OCRv2文字识别模型推理

对于文字识别模型，我们也可以使用下面的命令快速体验其功能。

  1. # 下载模型
  2. !cd inference && wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar -O ch_PP-OCRv2_rec_infer.tar &&  tar -xf ch_PP-OCRv2_rec_infer.tar
  3. # 预测
  4. !python tools/infer/predict_rec.py \
  5.     --image_dir="./doc/imgs_words/ch/word_4.jpg" \
  6.     --rec_model_dir="./inference/ch_PP-OCRv2_rec_infer" \
  7.     --use_gpu=False
  9. # 读取图像并显示
  10. img = cv2.imread("./doc/imgs_words/ch/word_4.jpg")
  11. plt.imshow(img[:,:,::-1])
  12. plt.show()
  1. --2021-12-25 15:43:40--  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar
  2. Resolving paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)... 100.67.200.6
  3. Connecting to paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)|100.67.200.6|:443... connected.
  4. HTTP request sent, awaiting response... 200 OK
  5. Length: 8875520 (8.5M) [application/x-tar]
  6. Saving to: ‘ch_PP-OCRv2_rec_infer.tar’
  8. ch_PP-OCRv2_rec_inf 100%[===================>]   8.46M  --.-KB/s    in 0.1s    
  10. 2021-12-25 15:43:40 (64.5 MB/s) - ‘ch_PP-OCRv2_rec_infer.tar’ saved [8875520/8875520]
  12. [2021/12/25 15:43:42] root INFO: Predicts of ./doc/imgs_words/ch/word_4.jpg:('实力活力', 0.9409585)

image.png
文字识别的具体代码如下所示。

  1. # 参考代码
  2. # https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/tools/infer/predict_rec.py
  3. class TextRecognizer(object):
  4.     def __init__(self, args):
  5.         self.rec_image_shape = [int(v) for v in args.rec_image_shape.split(",")]
  6.         self.rec_batch_num = args.rec_batch_num
  7.         self.rec_algorithm = args.rec_algorithm
  8.         postprocess_params = {
  9.             'name': 'CTCLabelDecode',
  10.             "character_dict_path": args.rec_char_dict_path,
  11.             "use_space_char": args.use_space_char
  12.         }
  13.         # 初始化预测引擎
  14.         self.predictor, self.input_tensor, self.output_tensors, self.config = \
  15.             utility.create_predictor(args, 'rec', logger)
  16.         # 初始化后处理过程
  17.         self.postprocess_op = build_post_process(postprocess_params)
  19.     # 预处理核心逻辑
  20.     def resize_norm_img(self, img, max_wh_ratio):
  21.         imgC, imgH, imgW = self.rec_image_shape
  22.         assert imgC == img.shape[2]
  23.         imgW = int((32 * max_wh_ratio))
  24.         h, w = img.shape[:2]
  25.         ratio = w / float(h)
  26.         if math.ceil(imgH * ratio) > imgW:
  27.             resized_w = imgW
  28.         else:
  29.             resized_w = int(math.ceil(imgH * ratio))
  30.         resized_image = cv2.resize(img, (resized_w, imgH))
  31.         resized_image = resized_image.astype('float32')
  32.         # [0, 255] -> [0, 1]
  33.         resized_image = resized_image.transpose((2, 0, 1)) / 255
  34.         # [0, 1] -> [-0.5, 0.5]
  35.         resized_image -= 0.5
  36.         # [-0.5, 0.5] -> [-1, 1]
  37.         resized_image /= 0.5
  38.         padding_im = np.zeros((imgC, imgH, imgW), dtype=np.float32)
  39.         padding_im[:, :, 0:resized_w] = resized_image
  40.         return padding_im
  42.     # 对图像列表进行处理
  43.     def __call__(self, img_list):
  44.         img_num = len(img_list)
  45.         # 记录宽高比
  46.         width_list = []
  47.         for img in img_list:
  48.             width_list.append(img.shape[1] / float(img.shape[0]))
  49.         # 排序，加速处理过程
  50.         indices = np.argsort(np.array(width_list))
  51.         rec_res = [['', 0.0]] * img_num
  52.         batch_num = self.rec_batch_num
  53.         st = time.time()
  54.         for beg_img_no in range(0, img_num, batch_num):
  55.             end_img_no = min(img_num, beg_img_no + batch_num)
  56.             norm_img_batch = []
  57.             max_wh_ratio = 0
  58.             for ino in range(beg_img_no, end_img_no):
  59.                 h, w = img_list[indices[ino]].shape[0:2]
  60.                 wh_ratio = w * 1.0 / h
  61.                 max_wh_ratio = max(max_wh_ratio, wh_ratio)
  62.             # 调用预处理方法并组batch
  63.             for ino in range(beg_img_no, end_img_no):
  64.                 norm_img = self.resize_norm_img(img_list[indices[ino]],
  65.                                                 max_wh_ratio)
  66.                 norm_img = norm_img[np.newaxis, :]
  67.                 norm_img_batch.append(norm_img)
  68.             norm_img_batch = np.concatenate(norm_img_batch)
  69.             norm_img_batch = norm_img_batch.copy()
  71.             # 将数据拷贝到预测引擎中
  72.             self.input_tensor.copy_from_cpu(norm_img_batch)
  73.             # 自动化推理过程
  74.             self.predictor.run()
  75.             outputs = []
  76.             # 将数据拷贝到CPU
  77.             for output_tensor in self.output_tensors:
  78.                 output = output_tensor.copy_to_cpu()
  79.                 outputs.append(output)
  80.             if len(outputs) != 1:
  81.                 preds = outputs
  82.             else:
  83.                 preds = outputs[0]
  84.             # 后处理
  85.             rec_result = self.postprocess_op(preds)
  86.             for rno in range(len(rec_result)):
  87.                 rec_res[indices[beg_img_no + rno]] = rec_result[rno]
  88.         return rec_res, time.time() - st
  91. # 定义参数
  92. args = parse_args()
  93. args.rec_model_dir = "./inference/ch_PP-OCRv2_rec_infer"
  94. args.image_dir = "./doc/imgs_words/ch/word_4.jpg"
  95. img_list = []
  97. image_file_list = get_image_file_list(args.image_dir)
  98. text_recognizer = TextRecognizer(args)
  99. valid_image_file_list = []
  100. for image_file in image_file_list:
  101.     img = cv2.imread(image_file)
  102.     if img is None:
  103.         logger.info("error in loading image:{}".format(image_file))
  104.         continue
  105.     valid_image_file_list.append(image_file)
  106.     img_list.append(img)
  107. rec_res, _ = text_recognizer(img_list)
  108. for ino in range(len(img_list)):
  109.     logger.info("Predicts of {}:{}".format(valid_image_file_list[ino],
  110.                                             rec_res[ino]))
  1. [2021/12/25 15:51:51] root INFO: Predicts of ./doc/imgs_words/ch/word_4.jpg:('实力活力', 0.9409561)

2.5 PP-OCRv2 系统串联预测推理

前面的内容给大家介绍了PP-OCRv2系统中，检测、方向分类器、识别模型的单独推理过程。为了方便大家端到端地使用，我们将这三个模块串联起来，组成了PP-OCRv2系统，并提供了相应的预测脚本。

在执行PP-OCRv2的系统推理时，需要通过参数image_dir指定单张图像或者图像集合的路径、参数det_model_dir, cls_model_dir 和 rec_model_dir 分别指定检测、方向分类和识别的 inference 模型路径。参数 use_angle_cls 用于控制是否启用方向分类模型。use_mp 表示是否使用多进程。total_process_num 表示在使用多进程时的进程数。

以图像文件./doc/imgs/00018069.jpg为例，预测的原始图像如下。
image.png
如果串联预测时使用方向分类器，则可以使用下面的命令进行预测。

  1. # 使用方向分类器，运行PP-OCRv2系统
  2. !python tools/infer/predict_system.py \
  3.     --image_dir="./doc/imgs/00018069.jpg" \
  4.     --det_model_dir="./inference/ch_PP-OCRv2_det_infer/" \
  5.     --cls_model_dir="./inference/ch_ppocr_mobile_v2.0_cls_infer/" \
  6.     --rec_model_dir="./inference/ch_PP-OCRv2_rec_infer/" \
  7.     --use_angle_cls=True
  9. # 可视化
  10. img = cv2.imread("./inference_results/00018069.jpg")
  11. plt.figure(figsize=(20, 8))
  12. plt.imshow(img[..., ::-1])
  13. plt.show()
  1. [2021/12/25 15:53:46] root DEBUG: dt_boxes num : 69, elapse : 2.86064076423645
  2. [2021/12/25 15:53:46] root DEBUG: cls num  : 69, elapse : 0.05832171440124512
  3. [2021/12/25 15:53:46] root DEBUG: rec_res num  : 69, elapse : 0.11343073844909668
  4. [2021/12/25 15:53:46] root DEBUG: 0  Predict time of ./doc/imgs/00018069.jpg: 3.039s
  5. [2021/12/25 15:53:46] root DEBUG: 代号, 0.951
  6. [2021/12/25 15:53:46] root DEBUG: 项目, 0.981
  7. [2021/12/25 15:53:46] root DEBUG: 结果, 0.994
  8. [2021/12/25 15:53:46] root DEBUG: 参考值, 0.999
  9. [2021/12/25 15:53:46] root DEBUG: 单位, 0.994
  10. [2021/12/25 15:53:46] root DEBUG: 谷丙转氨酶, 0.982
  11. [2021/12/25 15:53:46] root DEBUG: 25.6, 0.936
  12. [2021/12/25 15:53:46] root DEBUG: 0--40, 0.993
  13. [2021/12/25 15:53:46] root DEBUG: ALT, 0.999
  14. [2021/12/25 15:53:46] root DEBUG: U/L, 0.961
  15. [2021/12/25 15:53:46] root DEBUG: 总胆红素, 0.970
  16. [2021/12/25 15:53:46] root DEBUG: 11.2, 0.980
  17. [2021/12/25 15:53:46] root DEBUG: TBIL, 0.987
  18. [2021/12/25 15:53:46] root DEBUG: <20, 0.976
  19. [2021/12/25 15:53:46] root DEBUG: UMOL/L, 0.754
  20. [2021/12/25 15:53:46] root DEBUG: 直接胆红素, 0.954
  21. [2021/12/25 15:53:46] root DEBUG: 3.3, 0.954
  22. [2021/12/25 15:53:46] root DEBUG: DBIL, 0.993
  23. [2021/12/25 15:53:46] root DEBUG: 01-7, 0.839
  24. [2021/12/25 15:53:46] root DEBUG: UMOL/L, 0.786
  25. [2021/12/25 15:53:46] root DEBUG: 间接胆红素, 0.961
  26. [2021/12/25 15:53:46] root DEBUG: IBIL, 0.996
  27. [2021/12/25 15:53:46] root DEBUG: 7.9, 0.868
  28. [2021/12/25 15:53:46] root DEBUG: 1.5--15, 0.887
  29. [2021/12/25 15:53:46] root DEBUG: UMOL/L, 0.864
  30. [2021/12/25 15:53:46] root DEBUG: TP, 0.927
  31. [2021/12/25 15:53:46] root DEBUG: 总蛋白, 0.983
  32. [2021/12/25 15:53:46] root DEBUG: +6 89, 0.801
  33. [2021/12/25 15:53:46] root DEBUG: 60--80, 0.996
  34. [2021/12/25 15:53:46] root DEBUG: 8/L, 0.991
  35. [2021/12/25 15:53:46] root DEBUG: 白蛋白, 0.984
  36. [2021/12/25 15:53:46] root DEBUG: ALB, 0.998
  37. [2021/12/25 15:53:46] root DEBUG: 35.1, 0.956
  38. [2021/12/25 15:53:46] root DEBUG: 33--55, 0.898
  39. [2021/12/25 15:53:46] root DEBUG: 8/L, 0.962
  40. [2021/12/25 15:53:46] root DEBUG: GLO, 0.985
  41. [2021/12/25 15:53:46] root DEBUG: 球蛋白, 0.981
  42. [2021/12/25 15:53:46] root DEBUG: 23.8, 0.962
  43. [2021/12/25 15:53:46] root DEBUG: 20--30, 0.986
  44. [2021/12/25 15:53:46] root DEBUG: 8/L, 0.821
  45. [2021/12/25 15:53:46] root DEBUG: 白球比, 0.992
  46. [2021/12/25 15:53:46] root DEBUG: 1.5, 0.993
  47. [2021/12/25 15:53:46] root DEBUG: A/G, 0.978
  48. [2021/12/25 15:53:46] root DEBUG: 11.5--2.5, 0.920
  49. [2021/12/25 15:53:46] root DEBUG: 碱性磷酸酶, 0.965
  50. [2021/12/25 15:53:46] root DEBUG: 93, 0.940
  51. [2021/12/25 15:53:46] root DEBUG: ALP, 0.998
  52. [2021/12/25 15:53:46] root DEBUG: 15--112, 0.964
  53. [2021/12/25 15:53:46] root DEBUG: IU/L, 0.895
  54. [2021/12/25 15:53:46] root DEBUG: 谷氨酰转肽酶, 0.902
  55. [2021/12/25 15:53:46] root DEBUG: GGT, 0.858
  56. [2021/12/25 15:53:46] root DEBUG: 14.3, 0.957
  57. [2021/12/25 15:53:46] root DEBUG: <50, 0.938
  58. [2021/12/25 15:53:46] root DEBUG: U/L, 0.928
  59. [2021/12/25 15:53:46] root DEBUG: 谷草转氨酶, 0.975
  60. [2021/12/25 15:53:46] root DEBUG: 16.3, 0.991
  61. [2021/12/25 15:53:46] root DEBUG: 8--40, 0.978
  62. [2021/12/25 15:53:46] root DEBUG: AST, 0.996
  63. [2021/12/25 15:53:46] root DEBUG: U/L, 0.872
  64. [2021/12/25 15:53:46] root DEBUG: 乳酸脱氢酶, 0.939
  65. [2021/12/25 15:53:46] root DEBUG: 167, 0.984
  66. [2021/12/25 15:53:46] root DEBUG: 114--240, 0.980
  67. [2021/12/25 15:53:46] root DEBUG: LDH, 0.990
  68. [2021/12/25 15:53:46] root DEBUG: U/L, 0.989
  69. [2021/12/25 15:53:46] root DEBUG: 腺甘脱氨酶, 0.975
  70. [2021/12/25 15:53:46] root DEBUG: ADA, 0.990
  71. [2021/12/25 15:53:46] root DEBUG: 12.6, 0.984
  72. [2021/12/25 15:53:46] root DEBUG: 4--24, 0.896
  73. [2021/12/25 15:53:46] root DEBUG: U/L, 0.877
  74. [2021/12/25 15:53:46] root DEBUG: The visualized image saved in ./inference_results/00018069.jpg
  75. [2021/12/25 15:53:46] root INFO: The predict total time is 3.074248790740967

image.png
可视化识别结果默认保存到 ./inference_results 文件夹里面。

在图象中可视化出了检测框和识别结果，在上面的notebook中也打印出了具体的识别文件以及文件读取路径信息。

如果希望保存裁剪后的识别结果，可以将save_crop_res参数设置为True，最终结果保存在output目录下，其中部分裁剪后图像如下所示。保存的结果可以用于后续的识别模型标注与训练。

  1. # 裁剪文字检测的结果图像并保存
  2. !python tools/infer/predict_system.py \
  3.     --image_dir="./doc/imgs/00018069.jpg" \
  4.     --det_model_dir="./inference/ch_PP-OCRv2_det_infer/" \
  5.     --cls_model_dir="./inference/ch_ppocr_mobile_v2.0_cls_infer/" \
  6.     --rec_model_dir="./inference/ch_PP-OCRv2_rec_infer/" \
  7.     --use_angle_cls=True \
  8.     --save_crop_res=True
  10. !ls output
  12. plt.figure(figsize=(8, 8))
  13. plt.imshow(cv2.imread("./doc/imgs/00018069.jpg")[:, :, ::-1])
  14. plt.show()
  15. plt.figure(figsize=(14, 4))
  16. plt.subplot(1,3,1)
  17. plt.imshow(cv2.imread("output/mg_crop_0.jpg")[:, :, ::-1])
  18. plt.subplot(1,3,2)
  19. plt.imshow(cv2.imread("output/mg_crop_1.jpg")[:, :, ::-1])
  20. plt.subplot(1,3,3)
  21. plt.imshow(cv2.imread("output/mg_crop_2.jpg")[:, :, ::-1])
  22. plt.show()
  1. [2021/12/22 00:23:26] root DEBUG: dt_boxes num : 69, elapse : 1.6756877899169922
  2. [2021/12/22 00:23:26] root DEBUG: cls num  : 69, elapse : 0.07216906547546387
  3. [2021/12/22 00:23:26] root DEBUG: rec_res num  : 69, elapse : 0.10237336158752441
  4. [2021/12/22 00:23:26] root DEBUG: 0, ('代号', 0.9509918)
  5. [2021/12/22 00:23:26] root DEBUG: 1, ('项目', 0.98095936)
  6. [2021/12/22 00:23:26] root DEBUG: 2, ('结果', 0.99443346)
  7. [2021/12/22 00:23:26] root DEBUG: 3, ('参考值', 0.99930096)
  8. [2021/12/22 00:23:26] root DEBUG: 4, ('单位', 0.99435925)
  9. [2021/12/22 00:23:26] root DEBUG: 5, ('谷丙转氨酶', 0.9817702)
  10. [2021/12/22 00:23:26] root DEBUG: 6, ('25.6', 0.9356477)
  11. [2021/12/22 00:23:26] root DEBUG: 7, ('0--40', 0.99340856)
  12. [2021/12/22 00:23:26] root DEBUG: 8, ('ALT', 0.9987676)
  13. [2021/12/22 00:23:26] root DEBUG: 9, ('U/L', 0.96127224)
  14. [2021/12/22 00:23:26] root DEBUG: 10, ('总胆红素', 0.9696524)
  15. [2021/12/22 00:23:26] root DEBUG: 11, ('11.2', 0.98009694)
  16. [2021/12/22 00:23:26] root DEBUG: 12, ('TBIL', 0.9866674)
  17. [2021/12/22 00:23:26] root DEBUG: 13, ('<20', 0.97562915)
  18. [2021/12/22 00:23:26] root DEBUG: 14, ('UMOL/L', 0.753596)
  19. [2021/12/22 00:23:26] root DEBUG: 15, ('直接胆红素', 0.95369786)
  20. [2021/12/22 00:23:26] root DEBUG: 16, ('3.3', 0.9539189)
  21. [2021/12/22 00:23:26] root DEBUG: 17, ('DBIL', 0.992847)
  22. [2021/12/22 00:23:26] root DEBUG: 18, ('01-7', 0.8393893)
  23. [2021/12/22 00:23:26] root DEBUG: 19, ('UMOL/L', 0.78601867)
  24. [2021/12/22 00:23:26] root DEBUG: 20, ('间接胆红素', 0.9609308)
  25. [2021/12/22 00:23:26] root DEBUG: 21, ('IBIL', 0.9956264)
  26. [2021/12/22 00:23:26] root DEBUG: 22, ('7.9', 0.8684632)
  27. [2021/12/22 00:23:26] root DEBUG: 23, ('1.5--15', 0.8868044)
  28. [2021/12/22 00:23:26] root DEBUG: 24, ('UMOL/L', 0.86376244)
  29. [2021/12/22 00:23:26] root DEBUG: 25, ('TP', 0.92697346)
  30. [2021/12/22 00:23:26] root DEBUG: 26, ('总蛋白', 0.9828675)
  31. [2021/12/22 00:23:26] root DEBUG: 27, ('+6 89', 0.8005167)
  32. [2021/12/22 00:23:26] root DEBUG: 28, ('60--80', 0.9963381)
  33. [2021/12/22 00:23:26] root DEBUG: 29, ('8/L', 0.9908847)
  34. [2021/12/22 00:23:26] root DEBUG: 30, ('白蛋白', 0.9840359)
  35. [2021/12/22 00:23:26] root DEBUG: 31, ('ALB', 0.997701)
  36. [2021/12/22 00:23:26] root DEBUG: 32, ('35.1', 0.9561434)
  37. [2021/12/22 00:23:26] root DEBUG: 33, ('33--55', 0.89755243)
  38. [2021/12/22 00:23:26] root DEBUG: 34, ('8/L', 0.96184856)
  39. [2021/12/22 00:23:26] root DEBUG: 35, ('GLO', 0.98490185)
  40. [2021/12/22 00:23:26] root DEBUG: 36, ('球蛋白', 0.9813283)
  41. [2021/12/22 00:23:26] root DEBUG: 37, ('23.8', 0.9624181)
  42. [2021/12/22 00:23:26] root DEBUG: 38, ('20--30', 0.98609096)
  43. [2021/12/22 00:23:26] root DEBUG: 39, ('8/L', 0.821223)
  44. [2021/12/22 00:23:26] root DEBUG: 40, ('白球比', 0.9924216)
  45. [2021/12/22 00:23:26] root DEBUG: 41, ('1.5', 0.9929995)
  46. [2021/12/22 00:23:26] root DEBUG: 42, ('A/G', 0.9776535)
  47. [2021/12/22 00:23:26] root DEBUG: 43, ('11.5--2.5', 0.92048424)
  48. [2021/12/22 00:23:26] root DEBUG: 44, ('碱性磷酸酶', 0.9653308)
  49. [2021/12/22 00:23:26] root DEBUG: 45, ('93', 0.9396907)
  50. [2021/12/22 00:23:26] root DEBUG: 46, ('ALP', 0.9981435)
  51. [2021/12/22 00:23:26] root DEBUG: 47, ('15--112', 0.96404344)
  52. [2021/12/22 00:23:26] root DEBUG: 48, ('IU/L', 0.89462817)
  53. [2021/12/22 00:23:26] root DEBUG: 49, ('谷氨酰转肽酶', 0.90209746)
  54. [2021/12/22 00:23:26] root DEBUG: 50, ('GGT', 0.85804963)
  55. [2021/12/22 00:23:26] root DEBUG: 51, ('14.3', 0.9572656)
  56. [2021/12/22 00:23:26] root DEBUG: 52, ('<50', 0.9381611)
  57. [2021/12/22 00:23:26] root DEBUG: 53, ('U/L', 0.9277277)
  58. [2021/12/22 00:23:26] root DEBUG: 54, ('谷草转氨酶', 0.9747366)
  59. [2021/12/22 00:23:26] root DEBUG: 55, ('16.3', 0.99106205)
  60. [2021/12/22 00:23:26] root DEBUG: 56, ('8--40', 0.9781853)
  61. [2021/12/22 00:23:26] root DEBUG: 57, ('AST', 0.99589247)
  62. [2021/12/22 00:23:26] root DEBUG: 58, ('U/L', 0.87197894)
  63. [2021/12/22 00:23:26] root DEBUG: 59, ('乳酸脱氢酶', 0.938639)
  64. [2021/12/22 00:23:26] root DEBUG: 60, ('167', 0.9842878)
  65. [2021/12/22 00:23:26] root DEBUG: 61, ('114--240', 0.980333)
  66. [2021/12/22 00:23:26] root DEBUG: 62, ('LDH', 0.99032784)
  67. [2021/12/22 00:23:26] root DEBUG: 63, ('U/L', 0.9886821)
  68. [2021/12/22 00:23:26] root DEBUG: 64, ('腺甘脱氨酶', 0.9749187)
  69. [2021/12/22 00:23:26] root DEBUG: 65, ('ADA', 0.99024695)
  70. [2021/12/22 00:23:26] root DEBUG: 66, ('12.6', 0.98363185)
  71. [2021/12/22 00:23:26] root DEBUG: 67, ('4--24', 0.8959759)
  72. [2021/12/22 00:23:26] root DEBUG: 68, ('U/L', 0.8774794)
  73. [2021/12/22 00:23:26] root DEBUG: 0  Predict time of ./doc/imgs/00018069.jpg: 1.868s
  74. [2021/12/22 00:23:26] root DEBUG: 代号, 0.951
  75. [2021/12/22 00:23:26] root DEBUG: 项目, 0.981
  76. [2021/12/22 00:23:26] root DEBUG: 结果, 0.994
  77. [2021/12/22 00:23:26] root DEBUG: 参考值, 0.999
  78. [2021/12/22 00:23:26] root DEBUG: 单位, 0.994
  79. [2021/12/22 00:23:26] root DEBUG: 谷丙转氨酶, 0.982
  80. [2021/12/22 00:23:26] root DEBUG: 25.6, 0.936
  81. [2021/12/22 00:23:26] root DEBUG: 0--40, 0.993
  82. [2021/12/22 00:23:26] root DEBUG: ALT, 0.999
  83. [2021/12/22 00:23:26] root DEBUG: U/L, 0.961
  84. [2021/12/22 00:23:26] root DEBUG: 总胆红素, 0.970
  85. [2021/12/22 00:23:26] root DEBUG: 11.2, 0.980
  86. [2021/12/22 00:23:26] root DEBUG: TBIL, 0.987
  87. [2021/12/22 00:23:26] root DEBUG: <20, 0.976
  88. [2021/12/22 00:23:26] root DEBUG: UMOL/L, 0.754
  89. [2021/12/22 00:23:26] root DEBUG: 直接胆红素, 0.954
  90. [2021/12/22 00:23:26] root DEBUG: 3.3, 0.954
  91. [2021/12/22 00:23:26] root DEBUG: DBIL, 0.993
  92. [2021/12/22 00:23:26] root DEBUG: 01-7, 0.839
  93. [2021/12/22 00:23:26] root DEBUG: UMOL/L, 0.786
  94. [2021/12/22 00:23:26] root DEBUG: 间接胆红素, 0.961
  95. [2021/12/22 00:23:26] root DEBUG: IBIL, 0.996
  96. [2021/12/22 00:23:26] root DEBUG: 7.9, 0.868
  97. [2021/12/22 00:23:26] root DEBUG: 1.5--15, 0.887
  98. [2021/12/22 00:23:26] root DEBUG: UMOL/L, 0.864
  99. [2021/12/22 00:23:26] root DEBUG: TP, 0.927
  100. [2021/12/22 00:23:26] root DEBUG: 总蛋白, 0.983
  101. [2021/12/22 00:23:26] root DEBUG: +6 89, 0.801
  102. [2021/12/22 00:23:26] root DEBUG: 60--80, 0.996
  103. [2021/12/22 00:23:26] root DEBUG: 8/L, 0.991
  104. [2021/12/22 00:23:26] root DEBUG: 白蛋白, 0.984
  105. [2021/12/22 00:23:26] root DEBUG: ALB, 0.998
  106. [2021/12/22 00:23:26] root DEBUG: 35.1, 0.956
  107. [2021/12/22 00:23:26] root DEBUG: 33--55, 0.898
  108. [2021/12/22 00:23:26] root DEBUG: 8/L, 0.962
  109. [2021/12/22 00:23:26] root DEBUG: GLO, 0.985
  110. [2021/12/22 00:23:26] root DEBUG: 球蛋白, 0.981
  111. [2021/12/22 00:23:26] root DEBUG: 23.8, 0.962
  112. [2021/12/22 00:23:26] root DEBUG: 20--30, 0.986
  113. [2021/12/22 00:23:26] root DEBUG: 8/L, 0.821
  114. [2021/12/22 00:23:26] root DEBUG: 白球比, 0.992
  115. [2021/12/22 00:23:26] root DEBUG: 1.5, 0.993
  116. [2021/12/22 00:23:26] root DEBUG: A/G, 0.978
  117. [2021/12/22 00:23:26] root DEBUG: 11.5--2.5, 0.920
  118. [2021/12/22 00:23:26] root DEBUG: 碱性磷酸酶, 0.965
  119. [2021/12/22 00:23:26] root DEBUG: 93, 0.940
  120. [2021/12/22 00:23:26] root DEBUG: ALP, 0.998
  121. [2021/12/22 00:23:26] root DEBUG: 15--112, 0.964
  122. [2021/12/22 00:23:26] root DEBUG: IU/L, 0.895
  123. [2021/12/22 00:23:26] root DEBUG: 谷氨酰转肽酶, 0.902
  124. [2021/12/22 00:23:26] root DEBUG: GGT, 0.858
  125. [2021/12/22 00:23:26] root DEBUG: 14.3, 0.957
  126. [2021/12/22 00:23:26] root DEBUG: <50, 0.938
  127. [2021/12/22 00:23:26] root DEBUG: U/L, 0.928
  128. [2021/12/22 00:23:26] root DEBUG: 谷草转氨酶, 0.975
  129. [2021/12/22 00:23:26] root DEBUG: 16.3, 0.991
  130. [2021/12/22 00:23:26] root DEBUG: 8--40, 0.978
  131. [2021/12/22 00:23:26] root DEBUG: AST, 0.996
  132. [2021/12/22 00:23:26] root DEBUG: U/L, 0.872
  133. [2021/12/22 00:23:26] root DEBUG: 乳酸脱氢酶, 0.939
  134. [2021/12/22 00:23:26] root DEBUG: 167, 0.984
  135. [2021/12/22 00:23:26] root DEBUG: 114--240, 0.980
  136. [2021/12/22 00:23:26] root DEBUG: LDH, 0.990
  137. [2021/12/22 00:23:26] root DEBUG: U/L, 0.989
  138. [2021/12/22 00:23:26] root DEBUG: 腺甘脱氨酶, 0.975
  139. [2021/12/22 00:23:26] root DEBUG: ADA, 0.990
  140. [2021/12/22 00:23:26] root DEBUG: 12.6, 0.984
  141. [2021/12/22 00:23:26] root DEBUG: 4--24, 0.896
  142. [2021/12/22 00:23:26] root DEBUG: U/L, 0.877
  143. [2021/12/22 00:23:26] root DEBUG: The visualized image saved in ./inference_results/00018069.jpg
  144. [2021/12/22 00:23:26] root INFO: The predict total time is 1.9217827320098877
  145. mg_crop_0.jpg    mg_crop_22.jpg    mg_crop_35.jpg    mg_crop_48.jpg    mg_crop_60.jpg
  146. mg_crop_10.jpg    mg_crop_23.jpg    mg_crop_36.jpg    mg_crop_49.jpg    mg_crop_61.jpg
  147. mg_crop_11.jpg    mg_crop_24.jpg    mg_crop_37.jpg    mg_crop_4.jpg    mg_crop_62.jpg
  148. mg_crop_12.jpg    mg_crop_25.jpg    mg_crop_38.jpg    mg_crop_50.jpg    mg_crop_63.jpg
  149. mg_crop_13.jpg    mg_crop_26.jpg    mg_crop_39.jpg    mg_crop_51.jpg    mg_crop_64.jpg
  150. mg_crop_14.jpg    mg_crop_27.jpg    mg_crop_3.jpg    mg_crop_52.jpg    mg_crop_65.jpg
  151. mg_crop_15.jpg    mg_crop_28.jpg    mg_crop_40.jpg    mg_crop_53.jpg    mg_crop_66.jpg
  152. mg_crop_16.jpg    mg_crop_29.jpg    mg_crop_41.jpg    mg_crop_54.jpg    mg_crop_67.jpg
  153. mg_crop_17.jpg    mg_crop_2.jpg    mg_crop_42.jpg    mg_crop_55.jpg    mg_crop_68.jpg
  154. mg_crop_18.jpg    mg_crop_30.jpg    mg_crop_43.jpg    mg_crop_56.jpg    mg_crop_6.jpg
  155. mg_crop_19.jpg    mg_crop_31.jpg    mg_crop_44.jpg    mg_crop_57.jpg    mg_crop_7.jpg
  156. mg_crop_1.jpg    mg_crop_32.jpg    mg_crop_45.jpg    mg_crop_58.jpg    mg_crop_8.jpg
  157. mg_crop_20.jpg    mg_crop_33.jpg    mg_crop_46.jpg    mg_crop_59.jpg    mg_crop_9.jpg
  158. mg_crop_21.jpg    mg_crop_34.jpg    mg_crop_47.jpg    mg_crop_5.jpg

image.png
串联预测通过TextSystem类进行实现，其具体实现过程与函数定义如下。

  1. # 参考代码：https://github.com/PaddlePaddle/PaddleOCR/blob/release%2F2.4/tools/infer/predict_system.py
  2. from tools.infer.utility import draw_ocr_box_txt, get_rotate_crop_image
  3. from ppocr.utils.utility import get_image_file_list
  5. class TextSystem(object):
  6.     # 初始化函数
  7.     def __init__(self, args):
  8.         self.args = args
  9.           # 如果不希望显示log，可以将show_log设置为False
  10.         if not args.show_log:
  11.             logger.setLevel(logging.INFO)
  12.         # 定义文本检测模型预测引擎
  13.         self.text_detector = TextDetector(args)
  14.         # 定义文本识别模型预测引擎
  15.         self.text_recognizer = TextRecognizer(args)
  16.         # 是否使用方向分类器
  17.         self.use_angle_cls = args.use_angle_cls
  18.         # 得分阈值，根据该阈值判断检测与识别结果是否需要进行可视化或者返回
  19.         self.drop_score = args.drop_score
  20.         # 定义方向分类器预测引擎
  21.         if self.use_angle_cls:
  22.             self.text_classifier = TextClassifier(args)
  24.       # 保存文本检测结果图像
  25.     def draw_crop_rec_res(self, output_dir, img_crop_list, rec_res):
  26.         os.makedirs(output_dir, exist_ok=True)
  27.         bbox_num = len(img_crop_list)
  28.         for bno in range(bbox_num):
  29.             cv2.imwrite(
  30.                 os.path.join(output_dir,
  31.                              f"mg_crop_{bno+self.crop_image_res_index}.jpg"),
  32.                 img_crop_list[bno])
  33.             logger.debug(f"{bno}, {rec_res[bno]}")
  34.         self.crop_image_res_index += bbox_num
  36.     # 核心预测函数
  37.     def __call__(self, img, cls=True):
  38.         ori_im = img.copy()
  39.         # 获取检测文本检测结果
  40.         dt_boxes, elapse = self.text_detector(img)
  41.         logger.debug("dt_boxes num : {}, elapse : {}".format(
  42.             len(dt_boxes), elapse))
  43.         if dt_boxes is None:
  44.             return None, None
  45.         img_crop_list = []
  46.         # 对检测框进行排序，顺序为：优先从上到下，其次从左到右
  47.         dt_boxes = sorted_boxes(dt_boxes)
  48.         # 对检测结果进行透视变换与校正
  49.         for bno in range(len(dt_boxes)):
  50.             tmp_box = copy.deepcopy(dt_boxes[bno])
  51.             img_crop = get_rotate_crop_image(ori_im, tmp_box)
  52.             img_crop_list.append(img_crop)
  53.         # 使用方向分类器对检测结果进行转正
  54.         if self.use_angle_cls and cls:
  55.             img_crop_list, angle_list, elapse = self.text_classifier(
  56.                 img_crop_list)
  57.             logger.debug("cls num  : {}, elapse : {}".format(
  58.                 len(img_crop_list), elapse))
  59.         # 获取文本识别结果
  60.         rec_res, elapse = self.text_recognizer(img_crop_list)
  61.         logger.debug("rec_res num  : {}, elapse : {}".format(
  62.             len(rec_res), elapse))
  63.         # 保存经过校正之后的文本检测图像
  64.         if self.args.save_crop_res:
  65.             self.draw_crop_rec_res(self.args.crop_res_save_dir, img_crop_list,
  66.                                    rec_res)
  67.         filter_boxes, filter_rec_res = [], []
  68.         # 根据识别得分的阈值对结果进行过滤，如果得分小于阈值，就过滤掉
  69.         for box, rec_reuslt in zip(dt_boxes, rec_res):
  70.             text, score = rec_reuslt
  71.             if score >= self.drop_score:
  72.                 filter_boxes.append(box)
  73.                 filter_rec_res.append(rec_reuslt)
  74.         return filter_boxes, filter_rec_res
  76. def sorted_boxes(dt_boxes):
  77.     # 对检测框进行排序：优先从上到下，其次从左到右
  78.     num_boxes = dt_boxes.shape[0]
  79.     sorted_boxes = sorted(dt_boxes, key=lambda x: (x[0][1], x[0][0]))
  80.     _boxes = list(sorted_boxes)
  82.     for i in range(num_boxes - 1):
  83.         if abs(_boxes[i + 1][0][1] - _boxes[i][0][1]) < 10 and \
  84.                 (_boxes[i + 1][0][0] < _boxes[i][0][0]):
  85.             tmp = _boxes[i]
  86.             _boxes[i] = _boxes[i + 1]
  87.             _boxes[i + 1] = tmp
  88.     return _boxes
  90. args = parse_args()
  91. args.cls_model_dir = "./inference/ch_ppocr_mobile_v2.0_cls_infer"
  92. args.det_model_dir="./inference/ch_PP-OCRv2_det_infer/"
  93. args.rec_model_dir="./inference/ch_PP-OCRv2_rec_infer/"
  94. args.image_dir = "./doc/imgs/00018069.jpg"
  95. args.use_angle_cls=True
  96. args.use_gpu=True
  98. image_file_list = get_image_file_list(args.image_dir)
  99. image_file_list = image_file_list[args.process_id::args.total_process_num]
  100. text_sys = TextSystem(args)
  101. is_visualize = True
  102. font_path = args.vis_font_path
  103. drop_score = args.drop_score
  105. total_time = 0
  106. cpu_mem, gpu_mem, gpu_util = 0, 0, 0
  107. _st = time.time()
  108. count = 0
  109. for idx, image_file in enumerate(image_file_list):
  110.     img = cv2.imread(image_file)
  111.     if img is None:
  112.         logger.debug("error in loading image:{}".format(image_file))
  113.         continue
  114.     starttime = time.time()
  115.     dt_boxes, rec_res = text_sys(img)
  116.     elapse = time.time() - starttime
  117.     total_time += elapse
  119.     logger.debug(
  120.         str(idx) + "  Predict time of %s: %.3fs" % (image_file, elapse))
  121.     for text, score in rec_res:
  122.         logger.debug("{}, {:.3f}".format(text, score))
  124.     if is_visualize:
  125.         image = Image.fromarray(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
  126.         boxes = dt_boxes
  127.         txts = [rec_res[i][0] for i in range(len(rec_res))]
  128.         scores = [rec_res[i][1] for i in range(len(rec_res))]
  130.         draw_img = draw_ocr_box_txt(
  131.             image,
  132.             boxes,
  133.             txts,
  134.             scores,
  135.             drop_score=drop_score,
  136.             font_path=font_path)
  137.         draw_img_save_dir = args.draw_img_save_dir
  138.         os.makedirs(draw_img_save_dir, exist_ok=True)
  139.         cv2.imwrite(
  140.             os.path.join(draw_img_save_dir, os.path.basename(image_file)),
  141.             draw_img[:, :, ::-1])
  142.         logger.debug("The visualized image saved in {}".format(
  143.             os.path.join(draw_img_save_dir, os.path.basename(image_file))))
  145. logger.info("The predict total time is {}".format(time.time() - _st))
  147. plt.figure(figsize=(8, 8))
  148. plt.imshow(image)
  149. plt.show()
  150. plt.figure(figsize=(16, 8))
  151. plt.imshow(draw_img)
  152. plt.show()
  1. [2021/12/22 00:23:29] root DEBUG: dt_boxes num : 69, elapse : 0.04321765899658203
  2. [2021/12/22 00:23:29] root DEBUG: cls num  : 69, elapse : 0.04562950134277344
  3. [2021/12/22 00:23:29] root DEBUG: rec_res num  : 69, elapse : 0.09602046012878418
  4. [2021/12/22 00:23:29] root DEBUG: 0  Predict time of ./doc/imgs/00018069.jpg: 0.195s
  5. [2021/12/22 00:23:29] root DEBUG: 代号, 0.951
  6. [2021/12/22 00:23:29] root DEBUG: 项目, 0.981
  7. [2021/12/22 00:23:29] root DEBUG: 结果, 0.994
  8. [2021/12/22 00:23:29] root DEBUG: 参考值, 0.999
  9. [2021/12/22 00:23:29] root DEBUG: 单位, 0.994
  10. [2021/12/22 00:23:29] root DEBUG: 谷丙转氨酶, 0.982
  11. [2021/12/22 00:23:29] root DEBUG: 25.6, 0.936
  12. [2021/12/22 00:23:29] root DEBUG: 0--40, 0.993
  13. [2021/12/22 00:23:29] root DEBUG: ALT, 0.999
  14. [2021/12/22 00:23:29] root DEBUG: U/L, 0.961
  15. [2021/12/22 00:23:29] root DEBUG: 总胆红素, 0.970
  16. [2021/12/22 00:23:29] root DEBUG: 11.2, 0.980
  17. [2021/12/22 00:23:29] root DEBUG: TBIL, 0.987
  18. [2021/12/22 00:23:29] root DEBUG: <20, 0.976
  19. [2021/12/22 00:23:29] root DEBUG: UMOL/L, 0.754
  20. [2021/12/22 00:23:29] root DEBUG: 直接胆红素, 0.954
  21. [2021/12/22 00:23:29] root DEBUG: 3.3, 0.954
  22. [2021/12/22 00:23:29] root DEBUG: DBIL, 0.993
  23. [2021/12/22 00:23:29] root DEBUG: 01-7, 0.839
  24. [2021/12/22 00:23:29] root DEBUG: UMOL/L, 0.786
  25. [2021/12/22 00:23:29] root DEBUG: 间接胆红素, 0.961
  26. [2021/12/22 00:23:29] root DEBUG: IBIL, 0.996
  27. [2021/12/22 00:23:29] root DEBUG: 7.9, 0.868
  28. [2021/12/22 00:23:29] root DEBUG: 1.5--15, 0.887
  29. [2021/12/22 00:23:29] root DEBUG: UMOL/L, 0.864
  30. [2021/12/22 00:23:29] root DEBUG: TP, 0.927
  31. [2021/12/22 00:23:29] root DEBUG: 总蛋白, 0.983
  32. [2021/12/22 00:23:29] root DEBUG: +6 89, 0.801
  33. [2021/12/22 00:23:29] root DEBUG: 60--80, 0.996
  34. [2021/12/22 00:23:29] root DEBUG: 8/L, 0.991
  35. [2021/12/22 00:23:29] root DEBUG: 白蛋白, 0.984
  36. [2021/12/22 00:23:29] root DEBUG: ALB, 0.998
  37. [2021/12/22 00:23:29] root DEBUG: 35.1, 0.956
  38. [2021/12/22 00:23:29] root DEBUG: 33--55, 0.898
  39. [2021/12/22 00:23:29] root DEBUG: 8/L, 0.962
  40. [2021/12/22 00:23:29] root DEBUG: GLO, 0.985
  41. [2021/12/22 00:23:29] root DEBUG: 球蛋白, 0.981
  42. [2021/12/22 00:23:29] root DEBUG: 23.8, 0.962
  43. [2021/12/22 00:23:29] root DEBUG: 20--30, 0.986
  44. [2021/12/22 00:23:29] root DEBUG: 8/L, 0.821
  45. [2021/12/22 00:23:29] root DEBUG: 白球比, 0.992
  46. [2021/12/22 00:23:29] root DEBUG: 1.5, 0.993
  47. [2021/12/22 00:23:29] root DEBUG: A/G, 0.978
  48. [2021/12/22 00:23:29] root DEBUG: 11.5--2.5, 0.920
  49. [2021/12/22 00:23:29] root DEBUG: 碱性磷酸酶, 0.965
  50. [2021/12/22 00:23:29] root DEBUG: 93, 0.940
  51. [2021/12/22 00:23:29] root DEBUG: ALP, 0.998
  52. [2021/12/22 00:23:29] root DEBUG: 15--112, 0.964
  53. [2021/12/22 00:23:29] root DEBUG: IU/L, 0.895
  54. [2021/12/22 00:23:29] root DEBUG: 谷氨酰转肽酶, 0.902
  55. [2021/12/22 00:23:29] root DEBUG: GGT, 0.858
  56. [2021/12/22 00:23:29] root DEBUG: 14.3, 0.957
  57. [2021/12/22 00:23:29] root DEBUG: <50, 0.938
  58. [2021/12/22 00:23:29] root DEBUG: U/L, 0.928
  59. [2021/12/22 00:23:29] root DEBUG: 谷草转氨酶, 0.975
  60. [2021/12/22 00:23:29] root DEBUG: 16.3, 0.991
  61. [2021/12/22 00:23:29] root DEBUG: 8--40, 0.978
  62. [2021/12/22 00:23:29] root DEBUG: AST, 0.996
  63. [2021/12/22 00:23:29] root DEBUG: U/L, 0.872
  64. [2021/12/22 00:23:29] root DEBUG: 乳酸脱氢酶, 0.939
  65. [2021/12/22 00:23:29] root DEBUG: 167, 0.984
  66. [2021/12/22 00:23:29] root DEBUG: 114--240, 0.980
  67. [2021/12/22 00:23:29] root DEBUG: LDH, 0.990
  68. [2021/12/22 00:23:29] root DEBUG: U/L, 0.989
  69. [2021/12/22 00:23:29] root DEBUG: 腺甘脱氨酶, 0.975
  70. [2021/12/22 00:23:29] root DEBUG: ADA, 0.990
  71. [2021/12/22 00:23:29] root DEBUG: 12.6, 0.984
  72. [2021/12/22 00:23:29] root DEBUG: 4--24, 0.896
  73. [2021/12/22 00:23:29] root DEBUG: U/L, 0.877
  74. [2021/12/22 00:23:29] root DEBUG: The visualized image saved in ./inference_results/00018069.jpg
  75. [2021/12/22 00:23:29] root INFO: The predict total time is 0.28207850456237793

image.png

2.6 PP-OCRv2 使用whl包预测推理

为了更加方便快速体验OCR文本检测与识别模型，PaddleOCR提供了基于Paddle Inference预测引擎的whl包，方便您一键安装，体验PaddleOCR。

2.6.1 安装whl包

使用pip安装paddleocr的whl包，命令如下。

  1. !pip install "paddleocr==2.3.0.2"
  3. # 如果希望获取最新特性，可以基于源码编译安装
  4. #     python3 setup.py bdist_wheel
  5. #     pip3 install dist/paddleocr-x.x.x-py3-none-any.whl # x.x.x是paddleocr的版本号
  1. Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
  2. Collecting paddleocr==2.3.0.2
  3.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/e1/b6/5486e674ce096667dff247b58bf0fb789c2ce17a10e546c2686a2bb07aec/paddleocr-2.3.0.2-py3-none-any.whl (250 kB)
  4.      |████████████████████████████████| 250 kB 13.6 MB/s            
  5. [?25hRequirement already satisfied: imgaug==0.4.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (0.4.0)
  6. Requirement already satisfied: opencv-contrib-python==4.4.0.46 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (4.4.0.46)
  7. Requirement already satisfied: python-Levenshtein in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (0.12.2)
  8. Requirement already satisfied: pyclipper in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (1.3.0.post2)
  9. Requirement already satisfied: numpy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (1.20.3)
  10. Requirement already satisfied: visualdl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (2.2.0)
  11. Requirement already satisfied: lmdb in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (1.2.1)
  12. Requirement already satisfied: scikit-image in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (0.19.1)
  13. Requirement already satisfied: shapely in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (1.8.0)
  14. Requirement already satisfied: cython in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (0.29)
  15. Requirement already satisfied: openpyxl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (3.0.5)
  16. Requirement already satisfied: lxml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (4.7.1)
  17. Requirement already satisfied: premailer in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (3.10.0)
  18. Requirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (4.27.0)
  19. Requirement already satisfied: fasttext==0.9.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (0.9.1)
  20. Requirement already satisfied: setuptools>=0.7.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fasttext==0.9.1->paddleocr==2.3.0.2) (56.2.0)
  21. Requirement already satisfied: pybind11>=2.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fasttext==0.9.1->paddleocr==2.3.0.2) (2.8.1)
  22. Requirement already satisfied: scipy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->paddleocr==2.3.0.2) (1.6.3)
  23. Requirement already satisfied: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->paddleocr==2.3.0.2) (7.1.2)
  24. Requirement already satisfied: imageio in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->paddleocr==2.3.0.2) (2.6.1)
  25. Requirement already satisfied: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->paddleocr==2.3.0.2) (2.2.3)
  26. Requirement already satisfied: six in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->paddleocr==2.3.0.2) (1.15.0)
  27. Requirement already satisfied: opencv-python in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->paddleocr==2.3.0.2) (4.1.1.26)
  28. Requirement already satisfied: tifffile>=2019.7.26 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->paddleocr==2.3.0.2) (2021.11.2)
  29. Requirement already satisfied: networkx>=2.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->paddleocr==2.3.0.2) (2.4)
  30. Requirement already satisfied: PyWavelets>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->paddleocr==2.3.0.2) (1.2.0)
  31. Requirement already satisfied: packaging>=20.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->paddleocr==2.3.0.2) (20.9)
  32. Requirement already satisfied: et-xmlfile in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from openpyxl->paddleocr==2.3.0.2) (1.0.1)
  33. Requirement already satisfied: jdcal in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from openpyxl->paddleocr==2.3.0.2) (1.4.1)
  34. Requirement already satisfied: cssutils in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from premailer->paddleocr==2.3.0.2) (2.3.0)
  35. Requirement already satisfied: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from premailer->paddleocr==2.3.0.2) (2.22.0)
  36. Requirement already satisfied: cssselect in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from premailer->paddleocr==2.3.0.2) (1.1.0)
  37. Requirement already satisfied: cachetools in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from premailer->paddleocr==2.3.0.2) (4.0.0)
  38. Requirement already satisfied: pre-commit in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (1.21.0)
  39. Requirement already satisfied: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (0.8.53)
  40. Requirement already satisfied: shellcheck-py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (0.7.1.1)
  41. Requirement already satisfied: pandas in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (1.1.5)
  42. Requirement already satisfied: flask>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (1.1.1)
  43. Requirement already satisfied: flake8>=3.7.9 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (3.8.2)
  44. Requirement already satisfied: protobuf>=3.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (3.14.0)
  45. Requirement already satisfied: Flask-Babel>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (1.0.0)
  46. Requirement already satisfied: importlib-metadata in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->paddleocr==2.3.0.2) (0.23)
  47. Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->paddleocr==2.3.0.2) (0.6.1)
  48. Requirement already satisfied: pyflakes<2.3.0,>=2.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->paddleocr==2.3.0.2) (2.2.0)
  49. Requirement already satisfied: pycodestyle<2.7.0,>=2.6.0a1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->paddleocr==2.3.0.2) (2.6.0)
  50. Requirement already satisfied: click>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->paddleocr==2.3.0.2) (7.0)
  51. Requirement already satisfied: Jinja2>=2.10.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->paddleocr==2.3.0.2) (2.11.0)
  52. Requirement already satisfied: Werkzeug>=0.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->paddleocr==2.3.0.2) (0.16.0)
  53. Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->paddleocr==2.3.0.2) (1.1.0)
  54. Requirement already satisfied: pytz in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel>=1.0.0->visualdl->paddleocr==2.3.0.2) (2019.3)
  55. Requirement already satisfied: Babel>=2.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel>=1.0.0->visualdl->paddleocr==2.3.0.2) (2.8.0)
  56. Requirement already satisfied: decorator>=4.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from networkx>=2.2->scikit-image->paddleocr==2.3.0.2) (4.4.2)
  57. Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from packaging>=20.0->scikit-image->paddleocr==2.3.0.2) (2.4.2)
  58. Requirement already satisfied: future>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl->paddleocr==2.3.0.2) (0.18.0)
  59. Requirement already satisfied: pycryptodome>=3.8.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl->paddleocr==2.3.0.2) (3.9.9)
  60. Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->paddleocr==2.3.0.2) (2.8.0)
  61. Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->paddleocr==2.3.0.2) (0.10.0)
  62. Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->paddleocr==2.3.0.2) (1.1.0)
  63. Requirement already satisfied: cfgv>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (2.0.1)
  64. Requirement already satisfied: toml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (0.10.0)
  65. Requirement already satisfied: nodeenv>=0.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (1.3.4)
  66. Requirement already satisfied: aspy.yaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (1.3.0)
  67. Requirement already satisfied: virtualenv>=15.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (16.7.9)
  68. Requirement already satisfied: identify>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (1.4.10)
  69. Requirement already satisfied: pyyaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (5.1.2)
  70. Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->premailer->paddleocr==2.3.0.2) (3.0.4)
  71. Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->premailer->paddleocr==2.3.0.2) (2019.9.11)
  72. Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->premailer->paddleocr==2.3.0.2) (2.8)
  73. Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->premailer->paddleocr==2.3.0.2) (1.25.6)
  74. Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask>=1.1.1->visualdl->paddleocr==2.3.0.2) (1.1.1)
  75. Requirement already satisfied: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from importlib-metadata->flake8>=3.7.9->visualdl->paddleocr==2.3.0.2) (3.6.0)
  76. Installing collected packages: paddleocr
  77. Successfully installed paddleocr-2.3.0.2

2.6.2 使用whl包预测推理

paddleocr whl包会自动下载PP-OCRv2超轻量模型作为默认模型，也支持自定义模型路径、预测配置等参数，参数名称与基于Paddle Inference的python预测中参数相同。

  • 单独执行检测。

运行下面的代码，可快速体验文本检测模型的预测与效果。

  1. from paddleocr import PaddleOCR, draw_ocr
  3. ocr = PaddleOCR(use_gpu=False)  # need to run only once to download and load model into memory
  4. img_path = '/home/aistudio/PaddleOCR/doc/imgs/11.jpg'
  5. result = ocr.ocr(img_path, rec=False)
  6. for line in result:
  7.     print(line)
  9. # 显示结果
  10. from PIL import Image
  12. image = Image.open(img_path).convert('RGB')
  13. im_show = draw_ocr(image, result, txts=None, scores=None, font_path='/home/aistudio/PaddleOCR/doc/fonts/simfang.ttf')
  14. plt.figure(figsize=(15, 8))
  15. plt.imshow(im_show)
  16. plt.show()
  1. [2021/12/25 16:04:56] root WARNING: version PP-OCRv2 not support cls models, auto switch to version PP-OCR
  2. download https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar to /home/aistudio/.paddleocr/2.3.0.2/ocr/det/ch/ch_PP-OCRv2_det_infer/ch_PP-OCRv2_det_infer.tar
  3. download https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar to /home/aistudio/.paddleocr/2.3.0.2/ocr/rec/ch/ch_PP-OCRv2_rec_infer/ch_PP-OCRv2_rec_infer.tar
  6. 100%|██████████| 3.19M/3.19M [00:00<00:00, 47.1MiB/s]
  7. 100%|██████████| 8.88M/8.88M [00:00<00:00, 54.8MiB/s]
  8. 100%|██████████| 1.45M/1.45M [00:00<00:00, 35.4MiB/s]
  11. download https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar to /home/aistudio/.paddleocr/2.3.0.2/ocr/cls/ch_ppocr_mobile_v2.0_cls_infer/ch_ppocr_mobile_v2.0_cls_infer.tar
  12. Namespace(benchmark=False, cls_batch_num=6, cls_image_shape='3, 48, 192', cls_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_thresh=0.9, cpu_threads=10, crop_res_save_dir='./output', det=True, det_algorithm='DB', det_db_box_thresh=0.6, det_db_score_mode='fast', det_db_thresh=0.3, det_db_unclip_ratio=1.5, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_east_score_thresh=0.8, det_limit_side_len=960, det_limit_type='max', det_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/det/ch/ch_PP-OCRv2_det_infer', det_pse_box_thresh=0.85, det_pse_box_type='box', det_pse_min_area=16, det_pse_scale=1, det_pse_thresh=0, det_sast_nms_thresh=0.2, det_sast_polygon=False, det_sast_score_thresh=0.5, draw_img_save_dir='./inference_results', drop_score=0.5, e2e_algorithm='PGNet', e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_limit_side_len=768, e2e_limit_type='max', e2e_model_dir=None, e2e_pgnet_mode='fast', e2e_pgnet_score_thresh=0.5, e2e_pgnet_valid_set='totaltext', enable_mkldnn=False, gpu_mem=500, help='==SUPPRESS==', image_dir=None, ir_optim=True, label_list=['0', '180'], label_map_path='./vqa/labels/labels_ser.txt', lang='ch', layout_path_model='lp://PubLayNet/ppyolov2_r50vd_dcn_365e_publaynet/config', max_batch_size=10, max_seq_length=512, max_text_length=25, min_subgraph_size=15, mode='structure', model_name_or_path=None, ocr_version='PP-OCRv2', output='./output', precision='fp32', process_id=0, rec=True, rec_algorithm='CRNN', rec_batch_num=6, rec_char_dict_path='/home/aistudio/PaddleOCR/ppocr/utils/ppocr_keys_v1.txt', rec_image_shape='3, 32, 320', rec_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/rec/ch/ch_PP-OCRv2_rec_infer', save_crop_res=False, save_log_path='./log_output/', show_log=True, structure_version='STRUCTURE', table_char_dict_path=None, table_char_type='en', table_max_len=488, table_model_dir=None, total_process_num=1, type='ocr', use_angle_cls=False, use_dilation=False, use_gpu=False, use_mp=False, use_onnx=False, use_pdserving=False, use_space_char=True, use_tensorrt=False, vis_font_path='./doc/fonts/simfang.ttf', warmup=False)
  13. [2021/12/25 16:04:57] root WARNING: Since the angle classifier is not initialized, the angle classifier will not be uesd during the forward process
  14. [[27.0, 459.0], [135.0, 459.0], [135.0, 479.0], [27.0, 479.0]]
  15. [[29.0, 431.0], [369.0, 431.0], [369.0, 444.0], [29.0, 444.0]]
  16. [[26.0, 397.0], [361.0, 397.0], [361.0, 414.0], [26.0, 414.0]]
  17. [[369.0, 368.0], [475.0, 368.0], [475.0, 389.0], [369.0, 389.0]]
  18. [[28.0, 367.0], [279.0, 367.0], [279.0, 381.0], [28.0, 381.0]]
  19. [[27.0, 335.0], [343.0, 335.0], [343.0, 352.0], [27.0, 352.0]]
  20. [[26.0, 304.0], [251.0, 304.0], [251.0, 320.0], [26.0, 320.0]]
  21. [[24.0, 271.0], [180.0, 269.0], [180.0, 289.0], [24.0, 290.0]]
  22. [[24.0, 239.0], [241.0, 238.0], [241.0, 258.0], [24.0, 259.0]]
  23. [[413.0, 233.0], [430.0, 233.0], [430.0, 303.0], [413.0, 303.0]]
  24. [[25.0, 208.0], [234.0, 208.0], [234.0, 228.0], [25.0, 228.0]]
  25. [[25.0, 177.0], [300.0, 177.0], [300.0, 196.0], [25.0, 196.0]]
  26. [[26.0, 143.0], [281.0, 144.0], [281.0, 164.0], [26.0, 163.0]]
  27. [[28.0, 113.0], [330.0, 113.0], [330.0, 133.0], [28.0, 133.0]]
  28. [[27.0, 82.0], [172.0, 82.0], [172.0, 103.0], [27.0, 103.0]]
  29. [[28.0, 37.0], [302.0, 39.0], [302.0, 72.0], [28.0, 70.0]]

image.png

  • 单独执行识别

可以指定det=False，仅运行单独的识别模块。

  1. from paddleocr import PaddleOCR
  3. ocr = PaddleOCR(use_gpu=False)  # need to run only once to download and load model into memory
  4. img_path = '/home/aistudio/PaddleOCR/doc/imgs_words/ch/word_1.jpg'
  5. result = ocr.ocr(img_path, det=False)
  6. for line in result:
  7.     print(line)
  1. [2021/12/25 16:05:03] root WARNING: version PP-OCRv2 not support cls models, auto switch to version PP-OCR
  2. Namespace(benchmark=False, cls_batch_num=6, cls_image_shape='3, 48, 192', cls_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_thresh=0.9, cpu_threads=10, crop_res_save_dir='./output', det=True, det_algorithm='DB', det_db_box_thresh=0.6, det_db_score_mode='fast', det_db_thresh=0.3, det_db_unclip_ratio=1.5, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_east_score_thresh=0.8, det_limit_side_len=960, det_limit_type='max', det_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/det/ch/ch_PP-OCRv2_det_infer', det_pse_box_thresh=0.85, det_pse_box_type='box', det_pse_min_area=16, det_pse_scale=1, det_pse_thresh=0, det_sast_nms_thresh=0.2, det_sast_polygon=False, det_sast_score_thresh=0.5, draw_img_save_dir='./inference_results', drop_score=0.5, e2e_algorithm='PGNet', e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_limit_side_len=768, e2e_limit_type='max', e2e_model_dir=None, e2e_pgnet_mode='fast', e2e_pgnet_score_thresh=0.5, e2e_pgnet_valid_set='totaltext', enable_mkldnn=False, gpu_mem=500, help='==SUPPRESS==', image_dir=None, ir_optim=True, label_list=['0', '180'], label_map_path='./vqa/labels/labels_ser.txt', lang='ch', layout_path_model='lp://PubLayNet/ppyolov2_r50vd_dcn_365e_publaynet/config', max_batch_size=10, max_seq_length=512, max_text_length=25, min_subgraph_size=15, mode='structure', model_name_or_path=None, ocr_version='PP-OCRv2', output='./output', precision='fp32', process_id=0, rec=True, rec_algorithm='CRNN', rec_batch_num=6, rec_char_dict_path='/home/aistudio/PaddleOCR/ppocr/utils/ppocr_keys_v1.txt', rec_image_shape='3, 32, 320', rec_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/rec/ch/ch_PP-OCRv2_rec_infer', save_crop_res=False, save_log_path='./log_output/', show_log=True, structure_version='STRUCTURE', table_char_dict_path=None, table_char_type='en', table_max_len=488, table_model_dir=None, total_process_num=1, type='ocr', use_angle_cls=False, use_dilation=False, use_gpu=False, use_mp=False, use_onnx=False, use_pdserving=False, use_space_char=True, use_tensorrt=False, vis_font_path='./doc/fonts/simfang.ttf', warmup=False)
  3. [2021/12/25 16:05:03] root WARNING: Since the angle classifier is not initialized, the angle classifier will not be uesd during the forward process
  4. ('韩国小馆', 0.9967349)
  • 单独执行方向分类器

可以指定det=False, rec=False, cls=True，仅运行方向分类器。

  1. from paddleocr import PaddleOCR
  3. ocr = PaddleOCR(use_angle_cls=True, use_gpu=False)  # need to run only once to download and load model into memory
  4. img_path = '/home/aistudio/PaddleOCR/doc/imgs_words/ch/word_1.jpg'
  5. result = ocr.ocr(img_path, det=False, rec=False, cls=True)
  6. for line in result:
  7.     print(line)
  9. img = cv2.imread(img_path)
  10. plt.imshow(img[...,::-1])
  11. plt.show()
  1. [2021/12/25 16:06:27] root WARNING: version PP-OCRv2 not support cls models, auto switch to version PP-OCR
  2. Namespace(benchmark=False, cls_batch_num=6, cls_image_shape='3, 48, 192', cls_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_thresh=0.9, cpu_threads=10, crop_res_save_dir='./output', det=True, det_algorithm='DB', det_db_box_thresh=0.6, det_db_score_mode='fast', det_db_thresh=0.3, det_db_unclip_ratio=1.5, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_east_score_thresh=0.8, det_limit_side_len=960, det_limit_type='max', det_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/det/ch/ch_PP-OCRv2_det_infer', det_pse_box_thresh=0.85, det_pse_box_type='box', det_pse_min_area=16, det_pse_scale=1, det_pse_thresh=0, det_sast_nms_thresh=0.2, det_sast_polygon=False, det_sast_score_thresh=0.5, draw_img_save_dir='./inference_results', drop_score=0.5, e2e_algorithm='PGNet', e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_limit_side_len=768, e2e_limit_type='max', e2e_model_dir=None, e2e_pgnet_mode='fast', e2e_pgnet_score_thresh=0.5, e2e_pgnet_valid_set='totaltext', enable_mkldnn=False, gpu_mem=500, help='==SUPPRESS==', image_dir=None, ir_optim=True, label_list=['0', '180'], label_map_path='./vqa/labels/labels_ser.txt', lang='ch', layout_path_model='lp://PubLayNet/ppyolov2_r50vd_dcn_365e_publaynet/config', max_batch_size=10, max_seq_length=512, max_text_length=25, min_subgraph_size=15, mode='structure', model_name_or_path=None, ocr_version='PP-OCRv2', output='./output', precision='fp32', process_id=0, rec=True, rec_algorithm='CRNN', rec_batch_num=6, rec_char_dict_path='/home/aistudio/PaddleOCR/ppocr/utils/ppocr_keys_v1.txt', rec_image_shape='3, 32, 320', rec_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/rec/ch/ch_PP-OCRv2_rec_infer', save_crop_res=False, save_log_path='./log_output/', show_log=True, structure_version='STRUCTURE', table_char_dict_path=None, table_char_type='en', table_max_len=488, table_model_dir=None, total_process_num=1, type='ocr', use_angle_cls=True, use_dilation=False, use_gpu=False, use_mp=False, use_onnx=False, use_pdserving=False, use_space_char=True, use_tensorrt=False, vis_font_path='./doc/fonts/simfang.ttf', warmup=False)
  3. ['0', 0.9998784]

image.png

  • 检测+方向分类器+识别全流程体验
  1. from paddleocr import PaddleOCR, draw_ocr
  2. import matplotlib.pyplot as plt
  3. %matplotlib inline
  5. # PaddleOCR目前支持中英文、英文、法语、德语、韩语、日语，可以通过修改lang参数进行切换
  6. # 参数依次为`ch`, `en`, `french`, `german`, `korean`, `japan`。
  7. ocr = PaddleOCR(use_angle_cls=True, lang="ch", use_gpu=False)  # need to run only once to download and load model into memory
  8. img_path = '/home/aistudio/PaddleOCR/doc/imgs/11.jpg'
  9. result = ocr.ocr(img_path, cls=True)
  10. for line in result:
  11.     print(line)
  13. # 显示结果
  14. from PIL import Image
  16. image = Image.open(img_path).convert('RGB')
  17. boxes = [line[0] for line in result]
  18. txts = [line[1][0] for line in result]
  19. scores = [line[1][1] for line in result]
  20. im_show = draw_ocr(image, boxes, txts, scores, font_path='/home/aistudio/PaddleOCR/doc/fonts/simfang.ttf')
  21. plt.figure(figsize=(15, 8))
  22. plt.imshow(im_show)
  23. plt.show()
  1. [2021/12/25 16:07:58] root WARNING: version PP-OCRv2 not support cls models, auto switch to version PP-OCR
  2. Namespace(benchmark=False, cls_batch_num=6, cls_image_shape='3, 48, 192', cls_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_thresh=0.9, cpu_threads=10, crop_res_save_dir='./output', det=True, det_algorithm='DB', det_db_box_thresh=0.6, det_db_score_mode='fast', det_db_thresh=0.3, det_db_unclip_ratio=1.5, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_east_score_thresh=0.8, det_limit_side_len=960, det_limit_type='max', det_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/det/ch/ch_PP-OCRv2_det_infer', det_pse_box_thresh=0.85, det_pse_box_type='box', det_pse_min_area=16, det_pse_scale=1, det_pse_thresh=0, det_sast_nms_thresh=0.2, det_sast_polygon=False, det_sast_score_thresh=0.5, draw_img_save_dir='./inference_results', drop_score=0.5, e2e_algorithm='PGNet', e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_limit_side_len=768, e2e_limit_type='max', e2e_model_dir=None, e2e_pgnet_mode='fast', e2e_pgnet_score_thresh=0.5, e2e_pgnet_valid_set='totaltext', enable_mkldnn=False, gpu_mem=500, help='==SUPPRESS==', image_dir=None, ir_optim=True, label_list=['0', '180'], label_map_path='./vqa/labels/labels_ser.txt', lang='ch', layout_path_model='lp://PubLayNet/ppyolov2_r50vd_dcn_365e_publaynet/config', max_batch_size=10, max_seq_length=512, max_text_length=25, min_subgraph_size=15, mode='structure', model_name_or_path=None, ocr_version='PP-OCRv2', output='./output', precision='fp32', process_id=0, rec=True, rec_algorithm='CRNN', rec_batch_num=6, rec_char_dict_path='/home/aistudio/PaddleOCR/ppocr/utils/ppocr_keys_v1.txt', rec_image_shape='3, 32, 320', rec_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/rec/ch/ch_PP-OCRv2_rec_infer', save_crop_res=False, save_log_path='./log_output/', show_log=True, structure_version='STRUCTURE', table_char_dict_path=None, table_char_type='en', table_max_len=488, table_model_dir=None, total_process_num=1, type='ocr', use_angle_cls=True, use_dilation=False, use_gpu=False, use_mp=False, use_onnx=False, use_pdserving=False, use_space_char=True, use_tensorrt=False, vis_font_path='./doc/fonts/simfang.ttf', warmup=False)
  3. [2021/12/25 16:08:00] root DEBUG: dt_boxes num : 16, elapse : 1.1848454475402832
  4. [2021/12/25 16:08:01] root DEBUG: cls num  : 16, elapse : 0.7081763744354248
  5. [2021/12/25 16:08:08] root DEBUG: rec_res num  : 16, elapse : 7.406725883483887
  6. [[[28.0, 37.0], [302.0, 39.0], [302.0, 72.0], [28.0, 70.0]], ('纯臻营养护发素', 0.9924613)]
  7. [[[27.0, 82.0], [172.0, 82.0], [172.0, 103.0], [27.0, 103.0]], ('产品信息/参数', 0.992365)]
  8. [[[28.0, 113.0], [330.0, 113.0], [330.0, 133.0], [28.0, 133.0]], ('（45元/每公斤，100公斤起订）', 0.90023524)]
  9. [[[26.0, 143.0], [281.0, 144.0], [281.0, 164.0], [26.0, 163.0]], ('每瓶22元，1000瓶起订）', 0.9793598)]
  10. [[[25.0, 177.0], [300.0, 177.0], [300.0, 196.0], [25.0, 196.0]], ('【品牌】：代加工方式/OEMODM', 0.98173994)]
  11. [[[25.0, 208.0], [234.0, 208.0], [234.0, 228.0], [25.0, 228.0]], ('【品名】：纯臻营养护发素', 0.9742198)]
  12. [[[24.0, 239.0], [241.0, 238.0], [241.0, 258.0], [24.0, 259.0]], ('【产品编号】：YM-X-3011', 0.9428464)]
  13. [[[413.0, 233.0], [430.0, 233.0], [430.0, 303.0], [413.0, 303.0]], ('ODM OEM', 0.91271496)]
  14. [[[24.0, 271.0], [180.0, 269.0], [180.0, 289.0], [24.0, 290.0]], ('【净含量】：220ml', 0.97186875)]
  15. [[[26.0, 304.0], [251.0, 304.0], [251.0, 320.0], [26.0, 320.0]], ('【适用人群】：适合所有肤质', 0.94716364)]
  16. [[[27.0, 335.0], [343.0, 335.0], [343.0, 352.0], [27.0, 352.0]], ('（主要成分】：鲸蜡硬脂醇、燕麦B-葡聚', 0.89800084)]
  17. [[[28.0, 367.0], [279.0, 367.0], [279.0, 381.0], [28.0, 381.0]], ('糖、椰油酰胺丙基甜菜碱泛醒', 0.8583736)]
  18. [[[369.0, 368.0], [475.0, 368.0], [475.0, 389.0], [369.0, 389.0]], ('（成品包材）', 0.971282)]
  19. [[[26.0, 397.0], [361.0, 397.0], [361.0, 414.0], [26.0, 414.0]], ('主要功能】：可紧致头发磷层，从而达到', 0.97077614)]
  20. [[[29.0, 431.0], [369.0, 431.0], [369.0, 444.0], [29.0, 444.0]], ('即时持久改善头发光泽的效果，给干燥的头', 0.9430727)]
  21. [[[27.0, 459.0], [135.0, 459.0], [135.0, 479.0], [27.0, 479.0]], ('发足够的滋养', 0.97956467)]

image.png
结果是一个list，每个item包含了文本框，文字和识别置信度。

3. 基于Paddle Inference的C++推理

在推理部署过程中，相比于Python，C的性能一般会更好一些，因此很多实际推理场景中会考虑使用C作为开发语言进行推理。

上一小节给大家介绍的Paddle Inference也支持C的推理过程，本节主要介绍C的PP-OCRv2推理过程。

在基于Paddle Inference，对PP-OCRv2系统使用C++推理时，有以下几个步骤。

（1）准备模型

（2）编译opencv库

（3）获取Paddle Inference预测库

（4）编译PaddleOCR C++推理代码

（5）运行PP-OCRv2系统

由于AiStudio上版本限制，这里不做具体演示过程，仅给大家介绍具体的流程。建议您在本地体验PP-OCRv2的C++推理过程。

关于本小节更加详细的内容可以参考：PP-OCRv2 C++推理教程。

3.1 准备模型

使用下面的命令，准备PP-OCRv2的推理模型。

  1. cd deploy/cpp_infer
  2. wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar -O ch_PP-OCRv2_det_infer.tar && tar -xf ch_PP-OCRv2_det_infer.tar
  3. wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar -O ch_PP-OCRv2_rec_infer.tar &&  tar -xf ch_PP-OCRv2_rec_infer.tar
  4. wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar -O ch_ppocr_mobile_v2.0_cls_infer.tar && tar -xf ch_ppocr_mobile_v2.0_cls_infer.tar

3.2 编译OpenCV库

  • 首先需要从opencv官网上下载在Linux环境下源码编译的包，以opencv3.4.7为例，下载命令如下。
  1. wget https://paddleocr.bj.bcebos.com/libs/opencv/opencv-3.4.7.tar.gz
  2. tar -xf opencv-3.4.7.tar.gz

最终可以在当前目录下看到opencv-3.4.7/的文件夹。

  • 编译OpenCV，设置OpenCV源码路径(root_path)以及安装路径(install_path)。进入opencv源码路径下，按照下面的方式进行编译。
  1. root_path="your_opencv_root_path"
  2. install_path=${root_path}/opencv3
  3. build_dir=${root_path}/build
  5. rm -rf ${build_dir}
  6. mkdir ${build_dir}
  7. cd ${build_dir}
  9. cmake .. \
  10.     -DCMAKE_INSTALL_PREFIX=${install_path} \
  11.     -DCMAKE_BUILD_TYPE=Release \
  12.     -DBUILD_SHARED_LIBS=OFF \
  13.     -DWITH_IPP=OFF \
  14.     -DBUILD_IPP_IW=OFF \
  15.     -DWITH_LAPACK=OFF \
  16.     -DWITH_EIGEN=OFF \
  17.     -DCMAKE_INSTALL_LIBDIR=lib64 \
  18.     -DWITH_ZLIB=ON \
  19.     -DBUILD_ZLIB=ON \
  20.     -DWITH_JPEG=ON \
  21.     -DBUILD_JPEG=ON \
  22.     -DWITH_PNG=ON \
  23.     -DBUILD_PNG=ON \
  24.     -DWITH_TIFF=ON \
  25.     -DBUILD_TIFF=ON
  27. make -j
  28. make install

也可以直接修改tools/build_opencv.sh的内容，然后直接运行下面的命令进行编译。

  1. sh tools/build_opencv.sh

其中root_path为下载的opencv源码路径，install_path为opencv的安装路径，make install完成之后，会在该文件夹下生成opencv头文件和库文件，用于后面的OCR代码编译。

最终在安装路径下的文件结构如下所示。

  1. opencv3/
  2. |-- bin
  3. |-- include
  4. |-- lib
  5. |-- lib64
  6. |-- share

3.3 获取Paddle Inference预测库

  • Paddle预测库官网 上提供了不同cuda版本的Linux预测库，可以在官网根据自己的环境选择合适的预测库版本。
  • 下载之后使用下面的方法解压。 
  1. wget https://paddle-inference-lib.bj.bcebos.com/2.2.1/cxx_c/Linux/GPU/x86-64_gcc8.2_avx_mkl_cuda10.2_cudnn8.1.1_trt7.2.3.4/paddle_inference.tgz -O paddle_inference.tgz
  2. tar -xf paddle_inference.tgz

最终会在当前的文件夹中生成paddle_inference/的子文件夹。

3.4 编译PaddleOCR的推理代码

编译命令如下，其中Paddle C++预测库、opencv等其他依赖库的地址需要换成自己机器上的实际地址。

  1. sh tools/build.sh
  • 具体的，需要修改tools/build.sh中环境路径，相关内容如下：
  1. OPENCV_DIR=your_opencv_dir
  2. LIB_DIR=your_paddle_inference_dir
  3. CUDA_LIB_DIR=your_cuda_lib_dir
  4. CUDNN_LIB_DIR=/your_cudnn_lib_dir

其中，OPENCV_DIR为opencv编译安装的地址；LIB_DIR为下载(paddle_inference文件夹)或者编译生成的Paddle预测库地址(build/paddle_inference_install_dir文件夹)；CUDA_LIB_DIR为cuda库文件地址，在docker中为/usr/local/cuda/lib64；CUDNN_LIB_DIR为cudnn库文件地址，在docker中为/usr/lib/x86_64-linux-gnu/。注意：以上路径都写绝对路径，不要写相对路径。

  • 编译完成之后，会在build文件夹下生成一个名为ppocr的可执行文件。

3.5 运行PP-OCRv2系统

运行方式：

  1. ./build/ppocr <mode> [--param1] [--param2] [...]

其中，mode为必选参数，表示选择的功能，取值范围[‘det’, ‘rec’, ‘system’]，分别表示调用检测、识别、检测识别串联（包括方向分类器）。具体命令如下：

  • 只运行文字检测模型
  1. ./build/ppocr det \
  2.     --det_model_dir=./ch_PP-OCRv2_det_infer/ \
  3.     --image_dir=../../doc/imgs/12.jpg
  • 只运行文字识别模型
  1. ./build/ppocr rec \
  2.     --rec_model_dir=./ch_PP-OCRv2_rec_infer/ \
  3.     --image_dir=../../doc/imgs_words/ch/
  • 运行PP-OCRv2系统
  1. # 不使用方向分类器
  2. ./build/ppocr system \
  3.     --det_model_dir=./ch_PP-OCRv2_det_infer/ \
  4.     --rec_model_dir=./ch_PP-OCRv2_rec_infer/ \
  5.     --image_dir=../../doc/imgs/12.jpg
  7. # 使用方向分类器
  8. ./build/ppocr system \
  9.     --det_model_dir=./ch_PP-OCRv2_det_infer/ \
  10.     --rec_model_dir=./ch_PP-OCRv2_rec_infer/ \
  11.     --use_angle_cls=true \
  12.     --cls_model_dir=./ch_ppocr_mobile_v2.0_cls_infer \
  13.     --image_dir=../../doc/imgs/12.jpg

4. Paddle Serving服务化部署实战

在第2和第3节内容中，我们详细介绍了基于Paddle Inference的PP-OCRv2系统推理，它属于离线推理，即在特定机器上部署的代码只能在这台机器上使用，无法通过其他机器进行访问。因此模型服务化部署的需求也就衍生出来。

服务化部署指的是，将模型以服务的形式进行部署，其他的设备可以通过发送请求的形式去访问服务，从而获取模型服务的推理结果。服务化部署示意图如下所示。
image.png
在模型部署成功后，不同用户都可以通过客户端，以发送网络请求的方式获得推理服务。

Paddle Serving是飞桨为方便开发者进行服务化部署而打造的工具，本节主要介绍基于Paddle Serving的PP-OCRv2系统服务化部署过程。

4.1 Padde Serving简介

Paddle Serving作为飞桨（PaddlePaddle）开源的服务化部署框架，长期目标就是围绕着人工智能落地的最后一公里提供越来越专业、可靠、易用的服务。Paddle Serving目前提供了两套框架C++ Serving和Python Pipeline。Python Pipeline框架倾向于二次开发的便捷性，C++ Serving框架更倾向于追求极致性能。

基于Paddle Serving进行PP-OCRv2模型的服务化部署时，流程如下所示。
image.png

4.2 准备预测数据和部署环境

数据与模型推理所用数据一致。

运行Paddle Serving，需要安装Paddle Serving三个安装包：paddle-serving-server、paddle-serving-client 和 paddle-serving-app，命令如下。

  1. !wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_server_gpu-0.7.0.post102-py3-none-any.whl
  2. !pip install paddle_serving_server_gpu-0.7.0.post102-py3-none-any.whl
  4. !wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_client-0.7.0-cp37-none-any.whl
  5. !pip install paddle_serving_client-0.7.0-cp37-none-any.whl
  7. !wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_app-0.7.0-py3-none-any.whl
  8. !pip install paddle_serving_app-0.7.0-py3-none-any.whl
  10. !rm ./*.whl
  1. --2021-12-25 16:25:12--  https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_server_gpu-0.7.0.post102-py3-none-any.whl
  2. Resolving paddle-serving.bj.bcebos.com (paddle-serving.bj.bcebos.com)... 100.67.200.6
  3. Connecting to paddle-serving.bj.bcebos.com (paddle-serving.bj.bcebos.com)|100.67.200.6|:443... connected.
  4. HTTP request sent, awaiting response... 200 OK
  5. Length: 7146257 (6.8M) [application/octet-stream]
  6. Saving to: ‘paddle_serving_server_gpu-0.7.0.post102-py3-none-any.whl’
  8. paddle_serving_serv 100%[===================>]   6.81M  --.-KB/s    in 0.1s    
  10. 2021-12-25 16:25:12 (57.6 MB/s) - ‘paddle_serving_server_gpu-0.7.0.post102-py3-none-any.whl’ saved [7146257/7146257]
  12. Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
  13. Processing ./paddle_serving_server_gpu-0.7.0.post102-py3-none-any.whl
  14. Collecting Jinja2==2.11.3
  15.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7e/c2/1eece8c95ddbc9b1aeb64f5783a9e07a286de42191b7204d67b7496ddf35/Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)
  16.      |████████████████████████████████| 125 kB 14.9 MB/s            
  17. [?25hCollecting func-timeout
  18.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/b3/0d/bf0567477f7281d9a3926c582bfef21bff7498fc0ffd3e9de21811896a0b/func_timeout-4.3.5.tar.gz (44 kB)
  19.      |████████████████████████████████| 44 kB 6.3 MB/s             
  20. [?25h  Preparing metadata (setup.py) ... [?25ldone
  21. [?25hRequirement already satisfied: six>=1.10.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-server-gpu==0.7.0.post102) (1.15.0)
  22. Requirement already satisfied: itsdangerous==1.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-server-gpu==0.7.0.post102) (1.1.0)
  23. Requirement already satisfied: MarkupSafe==1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-server-gpu==0.7.0.post102) (1.1.1)
  24. Collecting Werkzeug==1.0.1
  25.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/cc/94/5f7079a0e00bd6863ef8f1da638721e9da21e5bacee597595b318f71d62e/Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
  26.      |████████████████████████████████| 298 kB 20.6 MB/s            
  27. [?25hRequirement already satisfied: protobuf>=3.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-server-gpu==0.7.0.post102) (3.14.0)
  28. Collecting grpcio<=1.33.2
  29.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7a/46/d08d8a5d0e0449f541fe9e7a226854019a41a4fa41fd14332e55b0e4394f/grpcio-1.33.2-cp37-cp37m-manylinux2014_x86_64.whl (3.8 MB)
  30.      |████████████████████████████████| 3.8 MB 82.7 MB/s            
  31. [?25hCollecting grpcio-tools<=1.33.2
  32.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/77/1e/91eaee901589ebee04c21df2f551502e7ba946bab99338f77a1f8a4237e1/grpcio_tools-1.33.2-cp37-cp37m-manylinux2014_x86_64.whl (2.5 MB)
  33.      |████████████████████████████████| 2.5 MB 2.7 MB/s            
  34. [?25hCollecting click==7.1.2
  35.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/click-7.1.2-py2.py3-none-any.whl (82 kB)
  36.      |████████████████████████████████| 82 kB 2.6 MB/s             
  37. [?25hRequirement already satisfied: flask<2.0.0,>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-server-gpu==0.7.0.post102) (1.1.1)
  38. Requirement already satisfied: pyyaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-server-gpu==0.7.0.post102) (5.1.2)
  39. Building wheels for collected packages: func-timeout
  40.   Building wheel for func-timeout (setup.py) ... [?25ldone
  41. [?25h  Created wheel for func-timeout: filename=func_timeout-4.3.5-py3-none-any.whl size=15077 sha256=787d26c33bea8918d151100cf0cd6c5953b1b065c427075198f1610f8788274d
  42.   Stored in directory: /home/aistudio/.cache/pip/wheels/ea/6b/56/b7bcacbd1bd8cd29883e7f7a29cbd98b87b2d789b25bae5563
  43. Successfully built func-timeout
  44. Installing collected packages: Werkzeug, Jinja2, grpcio, click, grpcio-tools, func-timeout, paddle-serving-server-gpu
  45.   Attempting uninstall: Werkzeug
  46.     Found existing installation: Werkzeug 0.16.0
  47.     Uninstalling Werkzeug-0.16.0:
  48.       Successfully uninstalled Werkzeug-0.16.0
  49.   Attempting uninstall: Jinja2
  50.     Found existing installation: Jinja2 2.11.0
  51.     Uninstalling Jinja2-2.11.0:
  52.       Successfully uninstalled Jinja2-2.11.0
  53.   Attempting uninstall: grpcio
  54.     Found existing installation: grpcio 1.35.0
  55.     Uninstalling grpcio-1.35.0:
  56.       Successfully uninstalled grpcio-1.35.0
  57.   Attempting uninstall: click
  58.     Found existing installation: Click 7.0
  59.     Uninstalling Click-7.0:
  60.       Successfully uninstalled Click-7.0
  61. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
  62. blackhole 1.0.1 requires numpy<=1.19.5, but you have numpy 1.20.3 which is incompatible.
  63. Successfully installed Jinja2-2.11.3 Werkzeug-1.0.1 click-7.1.2 func-timeout-4.3.5 grpcio-1.33.2 grpcio-tools-1.33.2 paddle-serving-server-gpu-0.7.0.post102
  64. --2021-12-25 16:25:19--  https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_client-0.7.0-cp37-none-any.whl
  65. Resolving paddle-serving.bj.bcebos.com (paddle-serving.bj.bcebos.com)... 100.67.200.6
  66. Connecting to paddle-serving.bj.bcebos.com (paddle-serving.bj.bcebos.com)|100.67.200.6|:443... connected.
  67. HTTP request sent, awaiting response... 200 OK
  68. Length: 44311645 (42M) [application/octet-stream]
  69. Saving to: ‘paddle_serving_client-0.7.0-cp37-none-any.whl’
  71. paddle_serving_clie 100%[===================>]  42.26M  66.9MB/s    in 0.6s    
  73. 2021-12-25 16:25:20 (66.9 MB/s) - ‘paddle_serving_client-0.7.0-cp37-none-any.whl’ saved [44311645/44311645]
  75. Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
  76. Processing ./paddle_serving_client-0.7.0-cp37-none-any.whl
  77. Requirement already satisfied: six>=1.10.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-client==0.7.0) (1.15.0)
  78. Requirement already satisfied: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-client==0.7.0) (2.22.0)
  79. Requirement already satisfied: grpcio-tools<=1.33.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-client==0.7.0) (1.33.2)
  80. Requirement already satisfied: grpcio<=1.33.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-client==0.7.0) (1.33.2)
  81. Requirement already satisfied: protobuf>=3.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-client==0.7.0) (3.14.0)
  82. Requirement already satisfied: numpy>=1.12 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-client==0.7.0) (1.20.3)
  83. Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddle-serving-client==0.7.0) (1.25.6)
  84. Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddle-serving-client==0.7.0) (2019.9.11)
  85. Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddle-serving-client==0.7.0) (3.0.4)
  86. Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddle-serving-client==0.7.0) (2.8)
  87. Installing collected packages: paddle-serving-client
  88. Successfully installed paddle-serving-client-0.7.0
  89. --2021-12-25 16:25:23--  https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_app-0.7.0-py3-none-any.whl
  90. Resolving paddle-serving.bj.bcebos.com (paddle-serving.bj.bcebos.com)... 100.67.200.6
  91. Connecting to paddle-serving.bj.bcebos.com (paddle-serving.bj.bcebos.com)|100.67.200.6|:443... connected.
  92. HTTP request sent, awaiting response... 200 OK
  93. Length: 54130 (53K) [application/octet-stream]
  94. Saving to: ‘paddle_serving_app-0.7.0-py3-none-any.whl’
  96. paddle_serving_app- 100%[===================>]  52.86K  --.-KB/s    in 0.002s  
  98. 2021-12-25 16:25:23 (27.6 MB/s) - ‘paddle_serving_app-0.7.0-py3-none-any.whl’ saved [54130/54130]
  100. Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
  101. Processing ./paddle_serving_app-0.7.0-py3-none-any.whl
  102. Collecting sentencepiece<=0.1.92
  103.   Downloading https://pypi.tuna.tsinghua.edu.cn/packages/1e/05/8d8d8fc7ef89d36bbdc9b4ff808def3db0dd52658e67f16779eb05ddb558/sentencepiece-0.1.92-cp37-cp37m-manylinux1_x86_64.whl (1.2 MB)
  104.      |████████████████████████████████| 1.2 MB 25.9 MB/s            
  105. [?25hRequirement already satisfied: opencv-python<=4.2.0.32 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-app==0.7.0) (4.1.1.26)
  106. Requirement already satisfied: six>=1.10.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-app==0.7.0) (1.15.0)
  107. Requirement already satisfied: pyclipper in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-app==0.7.0) (1.3.0.post2)
  108. Requirement already satisfied: pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-app==0.7.0) (7.1.2)
  109. Requirement already satisfied: shapely in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-app==0.7.0) (1.8.0)
  110. Requirement already satisfied: numpy>=1.14.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from opencv-python<=4.2.0.32->paddle-serving-app==0.7.0) (1.20.3)
  111. Installing collected packages: sentencepiece, paddle-serving-app
  112.   Attempting uninstall: sentencepiece
  113.     Found existing installation: sentencepiece 0.1.96
  114.     Uninstalling sentencepiece-0.1.96:
  115.       Successfully uninstalled sentencepiece-0.1.96
  116. Successfully installed paddle-serving-app-0.7.0 sentencepiece-0.1.92

4.3 准备部署模型

在进行模型服务化部署时，首先需要将推理模型转为用户服务化部署的模型。

首先运行下面的命令下载推理模型。

  1. os.chdir("/home/aistudio/PaddleOCR/deploy/pdserving/")
  3. # 下载并解压 OCR 文本检测模型
  4. !wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar -O ch_PP-OCRv2_det_infer.tar && tar -xf ch_PP-OCRv2_det_infer.tar && rm ch_PP-OCRv2_det_infer.tar
  5. # 下载并解压 OCR 文本识别模型
  6. !wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar -O ch_PP-OCRv2_rec_infer.tar &&  tar -xf ch_PP-OCRv2_rec_infer.tar && rm ch_PP-OCRv2_rec_infer.tar
  1. --2021-12-25 16:25:32--  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar
  2. Resolving paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)... 100.67.200.6
  3. Connecting to paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)|100.67.200.6|:443... connected.
  4. HTTP request sent, awaiting response... 200 OK
  5. Length: 3190272 (3.0M) [application/x-tar]
  6. Saving to: ‘ch_PP-OCRv2_det_infer.tar’
  8. ch_PP-OCRv2_det_inf 100%[===================>]   3.04M  --.-KB/s    in 0.09s   
  10. 2021-12-25 16:25:32 (35.0 MB/s) - ‘ch_PP-OCRv2_det_infer.tar’ saved [3190272/3190272]
  12. --2021-12-25 16:25:33--  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar
  13. Resolving paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)... 100.67.200.6
  14. Connecting to paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)|100.67.200.6|:443... connected.
  15. HTTP request sent, awaiting response... 200 OK
  16. Length: 8875520 (8.5M) [application/x-tar]
  17. Saving to: ‘ch_PP-OCRv2_rec_infer.tar’
  19. ch_PP-OCRv2_rec_inf 100%[===================>]   8.46M  --.-KB/s    in 0.1s    
  21. 2021-12-25 16:25:33 (69.0 MB/s) - ‘ch_PP-OCRv2_rec_infer.tar’ saved [8875520/8875520]

运行下面的命令进行模型转换。

  1. # 转换检测模型
  2. !python -m paddle_serving_client.convert --dirname ./ch_PP-OCRv2_det_infer/ \
  3.                                          --model_filename inference.pdmodel          \
  4.                                          --params_filename inference.pdiparams       \
  5.                                          --serving_server ./ppocrv2_det_serving/ \
  6.                                          --serving_client ./ppocrv2_det_client/
  8. # 转换识别模型
  9. !python -m paddle_serving_client.convert --dirname ./ch_PP-OCRv2_rec_infer/ \
  10.                                          --model_filename inference.pdmodel          \
  11.                                          --params_filename inference.pdiparams       \
  12.                                          --serving_server ./ppocrv2_rec_serving/  \
  13.                                          --serving_client ./ppocrv2_rec_client/
  15. # 查看文件夹
  16. !tree -h *_client *_serving
  1. ppocrv2_det_client
  2. ├── [ 296]  serving_client_conf.prototxt
  3. └── [  98]  serving_client_conf.stream.prototxt
  4. ppocrv2_rec_client
  5. ├── [ 284]  serving_client_conf.prototxt
  6. └── [  93]  serving_client_conf.stream.prototxt
  7. ppocrv2_det_serving
  8. ├── [2.2M]  inference.pdiparams
  9. ├── [842K]  inference.pdmodel
  10. ├── [ 296]  serving_server_conf.prototxt
  11. └── [  98]  serving_server_conf.stream.prototxt
  12. ppocrv2_rec_serving
  13. ├── [7.9M]  inference.pdiparams
  14. ├── [527K]  inference.pdmodel
  15. ├── [ 284]  serving_server_conf.prototxt
  16. └── [  93]  serving_server_conf.stream.prototxt
  18. 0 directories, 12 files

检测模型转换完成后，会在当前文件夹多出ppocrv2_det_mobile_serving 和ppocrv2_det_mobile_client的文件夹，具备如下格式：

  1. |- ppocrv2_det_mobile_serving/
  2.   |- __model__  
  3.   |- __params__
  4.   |- serving_server_conf.prototxt  
  5.   |- serving_server_conf.stream.prototxt
  7. |- ppocrv2_det_mobile_client
  8.   |- serving_client_conf.prototxt  
  9.   |- serving_client_conf.stream.prototxt

识别模型同理。

4.4 Paddle Serving pipeline部署

注意： 将PaddleOCR/deploy/pdserving/config.yml文件中的两个model_config字段分别修改为ppocrv2_det_mobile_serving、ppocrv2_rec_mobile_serving，对应模型转换的文件夹。

  1. pdserving目录包含启动pipeline服务和发送预测请求的代码，包括：

init.py config.yml # 启动服务的配置文件 ocr_reader.py # OCR模型预处理和后处理的代码实现 pipeline_http_client.py # 发送预测请求的脚本 web_service.py # 启动服务端的脚本

  1. ```
  3. <a name="0aa08cc8"></a>
  4. ### 4.4.1 启动服务
  6. 运行如下命令启动服务：
  8. **开启新的终端运行下列启动服务的命令**
  1. # 启动服务，运行日志保存在web_serving_log.txt
  2. cd PaddleOCR/deploy/pdserving/
  3. nohup python web_service.py &>web_serving_log.txt &
  2. 成功启动服务后，web_serving_log.txt中会打印类似如下日志<br />![image.png](https://cdn.nlark.com/yuque/0/2022/png/10371642/1645538124712-ef6439d5-d547-449e-ba26-819c6434cb42.png#clientId=uc709038f-e6c9-4&from=paste&height=286&id=u2a4ada30&margin=%5Bobject%20Object%5D&name=image.png&originHeight=571&originWidth=819&originalType=binary&ratio=1&size=313743&status=done&style=none&taskId=uefb012eb-930a-4dd9-98e2-0972fde9707&width=409.5)
  3. <a name="f5cb50bd"></a>
  4. ### 4.4.2 发送服务请求：
  6. ```python
  7. !python pipeline_http_client.py
  1. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['土地整治与土壤修复研究中心', '华南农业大学|东图']"], 'tensors': []}
  2. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['纯臻营养护发素', '产品信息/参数', '（45元/每公斤，100公斤起订', '每瓶22元，1000瓶起订）', '【品牌】：代加工方式/OEMODM', '【品名】：纯臻营养护发素', '【产品编号】：YM-X-3011', 'ODMOEM', '（净含量】：220ml', '【适用人群）：适合所有肤质', '［主要成分】：鲸蜡硬脂醇、燕麦B-葡聚', '糖、椰油酰胺内基甜菜碱、泛醒', '（成品包材）', '主要功能】：可紧致头发磷层，从而达到', '即时持久改善头发光泽的效果，给干燥的头', '发足够的滋养']"], 'tensors': []}
  3. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['Mairiedu|er|农', '*|Palais du LOUVRE', '公 LES ARTS DECORATIFS', '＊ Musée du LOUVRE', 'Theatre', 'du PALAIS-ROYAL L&']"], 'tensors': []}
  4. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['2', '证#', '', '##', '孔叫叫E2号动']"], 'tensors': []}
  5. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['合格证', '品名：纯棉单面背心套', '执行标准：FZ/T73025-2006', '安全类别：GB18401-2010', 'A类婴幼儿用品', '货号：B2390', 'DREA', '等级：一等品检验员：08', 'WOTHERAL', '成份：面料：100%棉', '小晶晶母婴', '品名：纯棉单面背心套', '上装：66/48', '零售价：47.50元', '下装：66/47']"], 'tensors': []}
  6. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['中国农业银行', 'AGRICULTURAL BANK OF CHINA', '美太网点办', '理务贵全融华务']"], 'tensors': []}
  7. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['Halten', 'Sie', 'Sie', 'was', 'versprechen', 'Und', 'geben', 'Sie noch', 'was dazu']"], 'tensors': []}
  8. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['上海斯格威铂尔受大酒店', '一打浦路15号', '绿洲仕格维花园公寓', '打浦路252935号']"], 'tensors': []}
  9. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['小学数学第八册第三单元测试题', '（小数乘法）', '姓名：', '班别：', '分数：', '计算题.(共38分）', '1.直接写出得数。（10分）', '0.8X25=', '0X4.23=', '0.57+0.43=', '2.4X0.2=', '0.02', 'X500=', '2.8X10=', '8.29X100=', '2.5X0.4=', '4.87-1.7=', '8.07', 'X1000=', '2.用坚式计算。（12分）', '2.97X3.8=', '7.05X0.84=']"], 'tensors': []}
  10. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['Paddle.jit.saveO', ': best accuracy.pdopt', ': inference.pdmodel', 'ich det mv3', '模型训练', 'i best accuracy.pdparams', '模型转换', 'opt优化模型', 'i inference.pdiparams', 'idb opt.nb', ': best accuracy.states']"], 'tensors': []}
  11. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['CreditCord', '中国工商银行', 'INDUSTRIL AD COMMERCIA BAHKSLUHIN', '持卡人存根', '商户名称&编号(MERCHANT NAME&CODE)', '工行浣纱支行积分兑换', '120214020408', '检索参考号(SYSTEMINOJ）', '终端号(TERMINAL)', '60128018', '001', '卡号（CARD NUMBER):', '622210512', 'S', '卡别（CARDTYPE)', '有效期(EXPIRY', '牡丹准货记卡', '交易日期DATE)', '交易时间(TIMEE)', '2012-10-23', '12:43:15', '批次号(BATCHNO.）', '流水号（TRACEHOJ', '000016', '积分兑奖', '信用卡积分：50000', '持卡人姓名（CARDHOLDERNAME）', 'MR.1', '持卡人签名', '(CARDHOLDER S', 'SIGNATURE:']"], 'tensors': []}
  12. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['1', 'EPPING', 'Twinned with', 'Eppingen，']"], 'tensors': []}
  13. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['订007778', 'PASS', '登机牌', 'BOARDING', '座位号', '序号', 'SERIALNO.', '舱位 CLASS', 'SEAT NO.', '航班 FLIGHT', '日期 DATE', 'W', '03DEC', '035', 'MU2379', 'GATE', '登机口', '始发地', 'FROM', '登机时间BDT', '自的地 TO', '福州', 'TAIYUAN', 'G11', 'FUZHOU', '身份识别ID NO.', '姓名NAME', 'ZHANGQIWET', '票号TKT NO', '张祺伟', '票价 FARE', 'ETKT7813699238489/1', '登机口于起飞前10分钟关闭 GATES CLOSE 10 MINUTES BEFORE DEPARTURE TIME']"], 'tensors': []}
  14. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['右', '天然の', '#', '味の占', '濃厚衣', '梦少久', '本勇5办', '味\\\\深', '燒', '深办の苏石', '苏扣子布', '5味の', '3-9', '警', '鲍の己品', '匕辛', '节办', '永书', '香', '梦之', '二の两石', '脂の秉龙', '力力', 'N', '乙龙の揚', '', '出来龙了', ':', '新鲜', '少手', '炊吉大了', '風味豊', '熟成L力', '二人办', '美味', '芳醇', '口の.', 'C', '二の石', '史方办套', '旬', '産地直送', '久七', '食人二', '秘の', '<<', '淹机个一卜', '>1一>一', '本場の', '朝探', '俊味吉', '七3寸子']"], 'tensors': []}
  15. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['7788.com', 'Z57A001950', '杭州东售', '2013年07月07日13:39开', '06车12B号', '二等座', '杭州东', 'G7512次', '上海虹桥', 'HangZhouDong', 'Shang HaiHongQiao', '￥73.00元', '限乘当日当次车', '余友红', '检票口16', '3623301993****0941', '9004-1300-5707-08A0-0195-0', '和谐号', 'C0400']"], 'tensors': []}
  16. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['Meters/bonwe', '洗涤说明', '淘X口Ne', '最高洗涤温度30°C', '常规程序', '不可漂白', '悬挂晾干', '熨斗底板最高温度110°C', '常规干洗', '注童事项：']"], 'tensors': []}
  17. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['3504811', '属瑞金医院', '标本号：OA', '支吉单（临床内分泌', '标本种类：血清', '标志提示：', '门诊号：E05535540', '门诊科别：门诊内分泌', '条码号：10080603139 临床诊断：', '申请医生：孙首悦', '检测日期：2010', '产', '龄：31', '检验备注：', '结果', '参考范围', '项目', '3.12', 'FT3', '2.62--6.49pmol/L', '15.40', 'FT4', '9.01--19.04pmol/L', '1.31', '0.35--4.94uIU/ml', 'STSH', '0.15', '<4.1110/ml', 'TGAB', '0.1', '25.0U/L', 'TRAb', '0.17', '<5.611U/ml', 'TPOAb', '0.1', '0.1--10.0pg/ml', '降钙素', '报告时间：10-08-0711:28', '打印时间：1', '采样时间：10-08-0611:23', '收到时间：10-08-0611:44', '核', '检验：李纪平', '对：保', '以上7项检验结果仅供临床医师参考']"], 'tensors': []}
  18. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['栏L。未来味元氨']"], 'tensors': []}
  19. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['代号', '项目', '结果', '参考值', '单位', '谷丙转氨酶', '25.6', '0--40', 'ALT', 'U/L', 'TBIL', '总胆红素', '11.2', '<20', 'UMO1/L', '直接胆红素', '3.3', '0--7', 'DBIL', 'UMOL/L', '间接胆红素', '7.9', 'IBIL', '1.5--15', 'UMOL/L', '总蛋白', '58.9V', '60--80', 'TP', 'S/L', '白蛋白', '35.1', '33--55', 'ALB', ':/L', '球蛋白', '23.8', '20--30', 'GLO', '8/L', '白球比', 'A/G', '1.5', '1.5--2.5', 'ALP', '可性磷酉酯', '93', '15--112', 'IU/L', '谷氨酰转肽酸', '<50', '14.3', 'GGT', 'U/L', '谷草转氨酶', 'AST', '16.3', '8--40', 'U/L', '乳酸脱氢酶', 'LDH', '167', '114--240', 'U/L', '腺甘脱氨酶', '12.6', '4--24', 'ADA', 'U/L']"], 'tensors': []}
  20. {'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': ["['股股东的持股情况', '单位：股', '股东总数（户）', '75.103', '前十名股东持股情况', '持有有限售条', '质押或冻结情况', '股东名称', '期末持股数量', '比例', '件股份数量', '股东性质', '股份状态', '数量', '国有法人', 'HONG Leong Bank', '境外法人', '浩海产业投资基金管理', '240000000', '境内非国有法人', '0', '国有法人', '公司', '16000000', '16000000', '公司', '国有法人', '成都欣天颐投资有限责', '0', '国有法人', '上海东昌投资发展有限', '3.3220%', '冻结', '公司', '50.000.00境内非国有法人', '新华文轩出版传媒股份', '0', '80000.000', '国有法人', '四川新华发行集团有限', '71243800', '1.9723%', '公司', '国有法人', '71.154.900', '限责任公司', '国有法人']"], 'tensors': []}
  21. ==> total number of test imgs:  20

进一步可以调整 config.yml 中的并发数，这里仅演示跑通效果，在这里默认都设置为1

  1.     det:
  2.         #并发数，is_thread_op=True时，为线程并发；否则为进程并发
  3.         concurrency: 1
  4.         ...
  5.     rec:
  6.         #并发数，is_thread_op=True时，为线程并发；否则为进程并发
  7.         concurrency: 1
  8.         ...

预测性能数据会被自动写入 PipelineServingLogs/pipeline.tracer 文件中。

4.5 FAQ

Q1： 发送请求后没有结果返回或者提示输出解码报错

A1： 启动服务和发送请求时不要设置代理，可以在启动服务前和发送请求前关闭代理，关闭代理的命令是：

  1. unset https_proxy
  2. unset http_proxy

5. 基于Paddle Lite的端侧推理实战

随时移动互联网的普及，越来越多的手机、嵌入式设备被使用，同时考虑到数据安全性以及模型运行成本，越来越多的模型被直接运行在端侧设备中。

Paddle Lite是飞桨轻量化推理引擎，为手机、IOT端提供高效推理能力，并广泛整合跨平台硬件，为端侧部署及应用落地问题提供轻量化的部署方案。

本章介绍基于Paddle Lite 在移动端部署PaddleOCR超轻量中文检测、识别模型的步骤。

下面给大家演示一下Android上运行PP-OCRv2系列模型的demo。

Android demo演示链接

由于在这里无法演示，在这里给大家介绍一下基于Paddle Lite开发PP-OCRv2系统运行程序的流程。

如果希望实操，可以参考PaddleOCR基于Paddle Lite的部署文档。

5.1 准备环境

这里需要同时准备交叉编译的环境以及Paddle Lite预测库，交叉编译环境用于生成可以在端侧设备上使用的可执行文件。推荐使用docker作为交叉编译环境。

5.2 准备模型

在使用Paddle Lite进行模型预测时，需要首先将推理模型转化为用于Paddle Lite预测的优化模型（后缀一般为nb），它采取了多种策略来自动优化原始的模型，其中包括量化、子图融合、混合调度、Kernel优选等方法。优化后的模型更轻量，速度更快。

5.3 编译

运行make -j执行编译，得到可执行文件, 第一次执行此命令会下载opencv等依赖库，等待下载完成之后再次运行make -j即可。

5.4 上传至手机等移动端

使用adb等工具，将可执行文件、模型文件、配置文件等传到手机等移动端上。

5.5 运行

在移动端中运行可执行文件即可得到结果，输出结果示例如下。
image.png

5.6 FAQ

Q1：如果想更换模型怎么办，需要重新按照流程走一遍吗？

A1：如果已经走通了上述步骤，更换模型只需要替换 .nb 模型文件即可，同时要注意更新字典

Q2：换一个图测试怎么做？

A2：替换debug下的.jpg测试图像为你想要测试的图像，adb push 到手机上即可

Q3：如何封装到手机APP中？

A3：此demo旨在提供能在手机上运行OCR的核心算法部分，PaddleOCR/deploy/android_demo是将这个demo封装到手机app的示例，供参考

6. 作业

请参考预测部署客观题与预测部署实战题部分。