Merge branch 'develop' of https://github.com/PaddlePaddle/PaddleX into develop_jf

dygraph/deploy/cpp/docs/manufacture_sdk/README.md

@@ -0,0 +1,103 @@
+# 工业级多端多平台预编译部署开发包
+
+PaddleX-Deploy全面升级，支持飞桨视觉套件PaddleX、PaddleDetection、PaddleClas、PaddleSeg的统一部署能力，端到端打通PaddleInference、PaddleLite、OpenVINO、Triton等多种高性能预测引擎，如果需要从**源码编译使用**，可至目录[PaddlePaddle模型C++部署](https://github.com/PaddlePaddle/PaddleX/tree/develop/dygraph/deploy/cpp)。
+
+在工业部署的开发过程中，常常因环境问题导致在部署代码编译环节中耗费较多的时间和人力成本。如果产线上的业务逻辑稍微复杂一点，尤其是串联多个模型时，则需要在模型推理前插入预处理、中间结果处理等操作，如此复杂的逻辑对应的部署代码开发工程量是很大的。
+
+为更进一步地提升部署效率，**:heart:PaddleX部署全新发布Manufacture SDK，提供工业级多端多平台部署加速的预编译飞桨部署开发包（SDK），通过配置业务逻辑流程文件即可以低代码方式快速完成推理部署。:heart:**
+
+
+## 目录
+* [1 Manufactue SDK简介](#1)
+* [2 下载安装Manufacture SDK](#2)
+* [3 Pipeline配置文件说明](#3)
+* [4 Pipeline Node说明](#4)
+* [5 多模型串联的工业表计读数部署](#5)
+
+## <h2 id="1">1 Manufactue SDK简介</h2>
+
+PaddleX Manufacture基于[PaddleX-Deploy](https://github.com/PaddlePaddle/PaddleX/tree/develop/dygraph/deploy/cpp)的端到端高性能部署能力，将应用深度学习模型的业务逻辑抽象成Pipeline，而接入深度学习模型前的数据前处理、模型预测、模型串联时的中间结果处理等操作都对应于Pipeline中的节点PipelineNode，用户只需在Pipeline配置文件中编排好各节点的前后关系，就可以给Pipeline发送数据并快速地获取相应的推理结果。Manufacture SDK的架构设计如下图所示：
+
+<div align="center">
+<img src="images/pipeline_arch.png"  width = "500" />              </div>
+
+* **PaddleX Pipeline**：完成业务逻辑的推理预测，同时提供向其发送数据和获取结果的接口。
+* **PaddleX Config**: 定义Pipeline的节点类型和前后关系。
+* **PaddleX Node**：功能节点，例如图像解码节点、图像缩放节点、兴趣区域提取节点、模型推理节点。
+* **PaddleX Deploy**：高性能模型推理能力。
+
+以PaddleX导出后的单一检测模型（无数据前处理、无中间结果处理）为例，配置好流程配置文件后使用简单几行代码就可以完成预测：
+
+<div align="center">
+<img src="images/pipeline_det.png"  width = "600" />              </div>
+
+## <h2 id="2">2 下载安装Manufacture SDK</h2>
+
+Manufature SDK的文件夹结构如下所示：
+
+```
+```
+
+### 下载安装Windows Manufacture SDK
+
+| 版本说明 | Manufacture SDK | 编译器 | 构建工具 | cuDNN | CUDA |
+| -- | -- | -- | -- | -- | -- |
+| | | | | | |
+
+### 下载安装Linux Manufacture SDK
+
+| 版本说明 | Manufacture SDK |
+| -- | -- |
+| | |
+
+## <h2 id="3">3 Pipeline配置文件说明</h2>
+
+PaddleX的模型导出后都会在模型文件夹中自动生成一个名为`pipeline.yml`流程编排文件，下面展示单一检测模型的流程配置文件：
+
+```
+pipeline_name: detector
+pipeline_nodes:
+- src0:
+    next: decode0
+    type: Source
+- decode0:
+    next: predict0
+    type: Decode
+- predict0:
+    init_params:
+      gpu_id: 0
+      model_dir: test_inference/inference_model
+      use_gpu: false
+      use_trt: false
+    next: sink0
+    type: Predict
+- sink0:
+    type: Sink
+version: 1.0.0
+```
+
+从上面示例可以看出，Pipeline配置文件是一个存储字典的`yaml`文件，文件中**必须包含`pipeline_name`和`pipeline_nodes`两个关键字**。每个关键词和键值说明如下：
+
+| 关键字 | 键值 |
+| -- | -- |
+| pipeline_name | Pipeline的名称 |
+| pipeline_nodes | Pipeline的节点列表。列表中**必须包含输入节点（Source）和输出节点（Sink）**。**列表中每个节点还是一个字典，关键字是该节点的名字，键值用于定义节点类型(type)、节点初始化参数(init_params)、连接的下一个节点的名字(next)，需要注意的是，每个节点的名字是独语无二的**。|
+
+## <h2 id="4">4 Pipeline Node说明</h2>
+
+目前支持的功能节点有：输入、图像解码、图像缩放、感兴趣区域提取、模型推理、检测框过滤、检测/分割结果可视化、输出。各功能节点的类型、初始化参数说明如下：
+
+| 功能类型 type | 功能作用 |初始化参数 init_params | 下一个节点 next | 上一个节点 |
+| -- | -- | -- | -- | -- |
+| Source | 接收Pipeline所需的输入数据 | 无 | `str/List(str)`: 可以是单个节点名字或多个节点名字组成的列表 | 无 |
+| Decode | 图像解码 | 无 | `str/List(str)`: 可以是单个节点名字或多个节点名字组成的列表 | 只能有一个 |
+| Resize | 图像大小缩放 | `width (int)`: 目标宽；<br>`height (int)`: 目标高；<br>`interp (int)`：差值类型，默认为`1`；| str/List(str): 可以是单个节点名字或多个节点名字组成的列表 | 只能有一个 |
+| Predict | PaddleX导出的分类/检测/分割模型预测 | `model_dir (str)`: PaddleX导出后的模型文件夹所在路径；<br>`use_gpu (bool)`: 是否使用GPU，默认为`false`；<br>`gpu_id (int)`：GPU卡号，在`use_gpu`为`true`时有效；<br>`use_trt (bool)`: 是否开启TensorRT加速GPU端预测 | `str/List(str)`: 可以是单个节点名字或多个节点名字组成的列表 | 只能有一个 |
+| FilterBbox | 过滤置信度低于阈值的检测框 | `score_thresh (float)`: 置信度阈值 | `str/List(str)`: 可以是单个节点名字或多个节点名字组成的列表 | 只能有一个 |
+| RoiCrop | 感兴趣区域提取 | 无 | `str/List(str)`: 可以是单个节点名字或多个节点名字组成的列表 | 必须有两个：能给出图像数据的节点、能给出检测模型预测结果的节点 |
+| Visualize | 目标检测/实例分割/语义分割模型预测结果可视化 | `save_dir (str)`: 存储可视化结果的文件夹路径 | 无 (可视化结果只能本地保存) | 必须有两个：能给出图像数据的节点、能给目标检测/实例分割/语义分割模型预测结果的节点 |
+| Sink | 获取Pipeline的输出数据 | 无 | 无 | 只能有一个 |
+
+**注意：上一个节点不需要在Pipeline配置文件中指定，只需要指定下一个节点即可，实际运行时程序解析连接至上一个节点**。
+
+## <h2 id="5">多模型串联的工业表计读数部署</h2>

dygraph/deploy/cpp/model_deploy/utils/include/visualize.h

@@ -0,0 +1,43 @@
+// Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#pragma once
+
+#include <iostream>
+#include <map>
+#include <vector>
+#include <string>
+
+#include <opencv2/core/core.hpp>
+#include <opencv2/highgui/highgui.hpp>
+#include <opencv2/imgproc/imgproc.hpp>
+
+#include "model_deploy/common/include/output_struct.h"
+
+namespace PaddleDeploy {
+
+bool GenerateColorMap(const int &num_classes,
+                      std::vector<int> *color_map);
+
+bool Visualize(const cv::Mat& img,
+               const DetResult& results,
+               cv::Mat* vis_img,
+               const int& num_classes = 81);
+
+bool Visualize(const cv::Mat& img,
+               const SegResult& result,
+               cv::Mat* vis_img,
+               const int& num_classes = 81);
+
+}  // namespace PaddleDeploy

dygraph/examples/meter_reader/README.md

@@ -102,7 +102,7 @@ meter_test/
 
 PaddleX提供了丰富的视觉模型，在目标检测中提供了RCNN和YOLO系列模型，在语义分割中提供了DeepLabV3P和BiSeNetV2等模型。
 
-因最终部署场景是本地化的服务器GPU端，算力相对充足，因此在本项目中采用精度和预测性能的PPYOLOV2进行表计检测。
+因最终部署场景是本地化的服务器GPU端，算力相对充足，因此在本项目中采用精度和预测性能皆优的PPYOLOV2进行表计检测。
 
 考虑到指针和刻度均为细小区域，我们采用精度更优的DeepLabV3P进行指针和刻度的分割。
 
@@ -221,7 +221,7 @@ eval_transforms = T.Compose([
 * 定义数据集路径
 
 ```python
-# 下载和解压指针刻度分割数据集，如果已经预先下载，可注视掉下面两行
+# 下载和解压指针刻度分割数据集，如果已经预先下载，可注释掉下面两行
 meter_seg_dataset = 'https://bj.bcebos.com/paddlex/examples/meter_reader/datasets/meter_seg.tar.gz'
 pdx.utils.download_and_decompress(meter_seg_dataset, path='./')
 
@@ -295,8 +295,7 @@ def predict(self,
             erode_kernel=4,
             score_threshold=0.5,
             seg_batch_size=2):
-    """检测图像中的表盘，而后分割出各表盘中的指针和刻度，对分割结果进行读数后厨后得到各表盘的读数。
-
+    """检测图像中的表盘，而后分割出各表盘中的指针和刻度，对分割结果进行读数后处理后得到各表盘的读数。
 
         参数：
             img_file (str)：待预测的图片路径。

dygraph/examples/meter_reader/deploy/cpp/meter_reader/include/meter_config.h

@@ -0,0 +1,67 @@
+// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#pragma once
+
+#include <vector>
+#include <string>
+#include <map>
+
+struct MeterConfig {
+  float scale_interval_value_;
+  float range_;
+  std::string unit_;
+
+  MeterConfig() {}
+
+  MeterConfig(const float &scale_interval_value,
+              const float &range,
+              const std::string &unit) :
+    scale_interval_value_(scale_interval_value),
+    range_(range), unit_(unit) {}
+};
+
+struct MeterResult {
+  // the number of scales
+  int num_scales_;
+  // the pointer location relative to the scales
+  float pointed_scale_;
+
+  MeterResult() {}
+
+  MeterResult(const int &num_scales, const float &pointed_scale) :
+    num_scales_(num_scales), pointed_scale_(pointed_scale) {}
+};
+
+// The size of inputting images of the segmenter.
+extern std::vector<int> METER_SHAPE;  // height x width
+// Center of a circular meter
+extern std::vector<int> CIRCLE_CENTER;  // height x width
+// Radius of a circular meter
+extern int CIRCLE_RADIUS;
+extern float PI;
+
+// During the postprocess phase, annulus formed by the radius from
+// 130 to 250 of a circular meter will be converted to a rectangle.
+// So the height of the rectangle is 120.
+extern int RECTANGLE_HEIGHT;
+// The width of the rectangle is 1570, that is to say the perimeter
+// of a circular meter.
+extern int RECTANGLE_WIDTH;
+
+// The configuration information of a meter,
+// composed of scale value, range, unit
+extern int TYPE_THRESHOLD;
+extern std::vector<MeterConfig> METER_CONFIG;
+extern std::map<std::string, uint8_t> SEG_CNAME2CLSID;

dygraph/examples/meter_reader/deploy/cpp/meter_reader/include/reader_postprocess.h

@@ -0,0 +1,61 @@
+// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+
+#pragma once
+
+#include <vector>
+
+#include "pipeline/include/pipeline.h"
+#include "meter_reader/include/meter_config.h"
+
+bool Erode(const int32_t &kernel_size,
+           const std::vector<PaddleDeploy::Result> &seg_results,
+           std::vector<std::vector<uint8_t>> *seg_label_maps);
+
+bool CircleToRectangle(
+  const std::vector<uint8_t> &seg_label_map,
+  std::vector<uint8_t> *rectangle_meter);
+
+bool RectangleToLine(const std::vector<uint8_t> &rectangle_meter,
+                     std::vector<int> *line_scale,
+                     std::vector<int> *line_pointer);
+
+bool MeanBinarization(const std::vector<int> &data,
+                      std::vector<int> *binaried_data);
+
+bool LocateScale(const std::vector<int> &scale,
+                 std::vector<float> *scale_location);
+
+bool LocatePointer(const std::vector<int> &pointer,
+                   float *pointer_location);
+
+bool GetRelativeLocation(
+  const std::vector<float> &scale_location,
+  const float &pointer_location,
+  MeterResult *result);
+
+bool CalculateReading(const MeterResult &result,
+                      float *reading);
+
+bool PrintMeterReading(const std::vector<float> &readings);
+
+bool Visualize(const cv::Mat& img,
+               const PaddleDeploy::Result &det_result,
+               const std::vector<float> &reading,
+               cv::Mat* vis_img);
+
+bool GetMeterReading(
+  const std::vector<std::vector<uint8_t>> &seg_label_maps,
+  std::vector<float> *readings);

dygraph/examples/meter_reader/deploy/cpp/meter_reader/meter_pipeline.yml

@@ -0,0 +1,63 @@
+pipeline_name: meter_reader
+pipeline_nodes:
+- src0:
+    type: Source
+    next: decode0
+- decode0:
+    type: Decode
+    next:
+    - modelpredict0
+    - visualize0
+    - roicrop0
+- modelpredict0:
+    type: Predict
+    init_params:
+      model_dir: /paddle/PaddleX/dygraph/examples/meter_reader/det_inference/inference_model
+      gpu_id: 0
+      use_gpu: true
+      use_trt: false
+    next: filter0
+- filter0:
+    type: FilterBbox
+    init_params:
+        score_thresh: 0.3
+    next:
+    - sink0
+    - visualize0
+    - roicrop0
+- roicrop0:
+    type: RoiCrop
+    next:
+    - resize0
+- resize0:
+    type: Resize
+    init_params:
+      height: 512
+      width: 512
+      interp: 0
+    next:
+    - modelpredict1
+    - visualize1
+- modelpredict1:
+    type: Predict
+    init_params:
+      model_dir: /paddle/PaddleX/dygraph/examples/meter_reader/seg_inference/inference_model
+      gpu_id: 0
+      use_gpu: true
+      use_trt: false
+    next:
+    - sink1
+    - visualize1
+- visualize0:
+    type: Visualize
+    init_params:
+      save_dir: ./output_det
+- visualize1:
+    type: Visualize
+    init_params:
+      save_dir: ./output_seg
+- sink0:
+    type: Sink
+- sink1:
+    type: Sink
+version: 2.0.0

dygraph/examples/meter_reader/deploy/cpp/meter_reader/meter_reader.cpp

@@ -0,0 +1,78 @@
+// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include <gflags/gflags.h>
+#include <string>
+#include <iostream>
+#include <vector>
+#include <utility>
+#include <limits>
+
+#include <opencv2/opencv.hpp>
+#include <opencv2/highgui.hpp>
+#include <opencv2/core/core.hpp>
+
+#include "pipeline/include/pipeline.h"
+#include "meter_reader/include/reader_postprocess.h"
+
+DEFINE_string(pipeline_cfg, "", "Path of pipeline config file");
+DEFINE_bool(use_erode, true, "Eroding predicted label map");
+DEFINE_int32(erode_kernel, 4, "Eroding kernel size");
+DEFINE_string(image, "", "Path of test image file");
+DEFINE_string(save_dir, "", "Path to save visualized results");
+
+int main(int argc, char **argv) {
+  google::ParseCommandLineFlags(&argc, &argv, true);
+  if (FLAGS_pipeline_cfg == "") {
+    std::cerr << "--pipeline_cfg need to be defined" << std::endl;
+    return -1;
+  }
+  if (FLAGS_image == "") {
+    std::cerr << "--image need to be defined "
+              << "when the camera is not been used" << std::endl;
+    return -1;
+  }
+
+  std::vector<std::string> image_paths = {FLAGS_image};
+  PaddleXPipeline::Pipeline pipeline;
+  if (pipeline.Init(FLAGS_pipeline_cfg)) {
+    pipeline.SetInput("src0", image_paths);
+    pipeline.Run();
+    std::vector<PaddleDeploy::Result> det_results;
+    std::vector<PaddleDeploy::Result> seg_results;
+    pipeline.GetOutput("sink0", &det_results);
+    pipeline.GetOutput("sink1", &seg_results);
+
+    // Do image erosion for the predicted label map of each meter
+    std::vector<std::vector<uint8_t>> seg_label_maps;
+    Erode(FLAGS_erode_kernel, seg_results, &seg_label_maps);
+
+    // The postprocess are done to get the reading or each meter
+    std::vector<float> readings;
+    GetMeterReading(seg_label_maps, &readings);
+    PrintMeterReading(readings);
+    if (FLAGS_save_dir != "") {
+      cv::Mat img = cv::imread(FLAGS_image);
+      cv::Mat vis_img;
+      Visualize(img, det_results[0], readings, &vis_img);
+      std::string save_path;
+      if (PaddleXPipeline::GenerateSavePath(
+          FLAGS_save_dir, FLAGS_image, &save_path)) {
+         cv::imwrite(save_path, vis_img);
+      }
+    }
+  }
+
+  return 0;
+}

dygraph/examples/meter_reader/deploy/cpp/meter_reader/src/meter_config.cpp

@@ -0,0 +1,33 @@
+// Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+
+#include "meter_reader/include/meter_config.h"
+
+std::vector<int> METER_SHAPE = {512, 512};  // height x width
+std::vector<int> CIRCLE_CENTER = {256, 256};
+int CIRCLE_RADIUS = 250;
+float PI = 3.1415926536;
+int RECTANGLE_HEIGHT = 120;
+int RECTANGLE_WIDTH = 1570;
+
+int TYPE_THRESHOLD = 40;
+std::vector<MeterConfig> METER_CONFIG = {
+  MeterConfig(25.0f/50.0f, 25.0f, "(MPa)"),
+  MeterConfig(1.6f/32.0f,  1.6f,   "(MPa)")
+};
+
+std::map<std::string, uint8_t> SEG_CNAME2CLSID = {
+  {"background", 0}, {"pointer", 1}, {"scale", 2}
+};

dygraph/examples/meter_reader/deploy/cpp/meter_reader/src/reader_postprocess.cpp

@@ -0,0 +1,264 @@
+// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+
+#include <iostream>
+#include <vector>
+#include <utility>
+#include <limits>
+#include <cmath>
+#include <string>
+
+#include <opencv2/opencv.hpp>
+#include <opencv2/highgui.hpp>
+#include <opencv2/core/core.hpp>
+
+#include "meter_reader/include/reader_postprocess.h"
+
+bool Erode(const int32_t &kernel_size,
+           const std::vector<PaddleDeploy::Result> &seg_results,
+           std::vector<std::vector<uint8_t>> *seg_label_maps) {
+  cv::Mat kernel(kernel_size, kernel_size, CV_8U, cv::Scalar(1));
+  for (auto result : seg_results) {
+    std::vector<uint8_t> label_map(result.seg_result->label_map.data.begin(),
+                                   result.seg_result->label_map.data.end());
+    cv::Mat mask(result.seg_result->label_map.shape[0],
+                 result.seg_result->label_map.shape[1],
+                 CV_8UC1,
+                 label_map.data());
+    cv::erode(mask, mask, kernel);
+    std::vector<uint8_t> map;
+    if (mask.isContinuous()) {
+        map.assign(mask.data, mask.data + mask.total() * mask.channels());
+    } else {
+      for (int r = 0; r < mask.rows; r++) {
+        map.insert(map.end(),
+                   mask.ptr<int64_t>(r),
+                   mask.ptr<int64_t>(r) + mask.cols * mask.channels());
+      }
+    }
+    seg_label_maps->push_back(map);
+  }
+  return true;
+}
+
+
+bool CircleToRectangle(
+  const std::vector<uint8_t> &seg_label_map,
+  std::vector<uint8_t> *rectangle_meter) {
+  float theta;
+  int rho;
+  int image_x;
+  int image_y;
+
+  // The minimum scale value is at the bottom left, the maximum scale value
+  // is at the bottom right, so the vertical down axis is the starting axis and
+  // rotates around the meter ceneter counterclockwise.
+  *rectangle_meter =
+    std::vector<uint8_t> (RECTANGLE_WIDTH * RECTANGLE_HEIGHT, 0);
+  for (int row = 0; row < RECTANGLE_HEIGHT; row++) {
+    for (int col = 0; col < RECTANGLE_WIDTH; col++) {
+      theta = PI * 2 / RECTANGLE_WIDTH * (col + 1);
+      rho = CIRCLE_RADIUS - row - 1;
+      int y = static_cast<int>(CIRCLE_CENTER[0] + rho * cos(theta) + 0.5);
+      int x = static_cast<int>(CIRCLE_CENTER[1] - rho * sin(theta) + 0.5);
+      (*rectangle_meter)[row * RECTANGLE_WIDTH + col] =
+        seg_label_map[y * METER_SHAPE[1] + x];
+    }
+  }
+
+  return true;
+}
+
+bool RectangleToLine(const std::vector<uint8_t> &rectangle_meter,
+                     std::vector<int> *line_scale,
+                     std::vector<int> *line_pointer) {
+  // Accumulte the number of positions whose label is 1 along the height axis.
+  // Accumulte the number of positions whose label is 2 along the height axis.
+  (*line_scale) = std::vector<int> (RECTANGLE_WIDTH, 0);
+  (*line_pointer) = std::vector<int> (RECTANGLE_WIDTH, 0);
+  for (int col = 0; col < RECTANGLE_WIDTH; col++) {
+    for (int row = 0; row < RECTANGLE_HEIGHT; row++) {
+        if (rectangle_meter[row * RECTANGLE_WIDTH + col] ==
+          static_cast<uint8_t>(SEG_CNAME2CLSID["pointer"])) {
+            (*line_pointer)[col]++;
+        } else if (rectangle_meter[row * RECTANGLE_WIDTH + col] ==
+          static_cast<uint8_t>(SEG_CNAME2CLSID["scale"])) {
+            (*line_scale)[col]++;
+        }
+    }
+  }
+  return true;
+}
+
+bool MeanBinarization(const std::vector<int> &data,
+                      std::vector<int> *binaried_data) {
+  int sum = 0;
+  float mean = 0;
+  for (auto i = 0; i < data.size(); i++) {
+    sum = sum + data[i];
+  }
+  mean = static_cast<float>(sum) / static_cast<float>(data.size());
+
+  for (auto i = 0; i < data.size(); i++) {
+    if (static_cast<float>(data[i]) >= mean) {
+      binaried_data->push_back(1);
+    } else {
+      binaried_data->push_back(0);
+    }
+  }
+  return  true;
+}
+
+bool LocateScale(const std::vector<int> &scale,
+                 std::vector<float> *scale_location) {
+  float one_scale_location = 0;
+  bool find_start = false;
+  int one_scale_start = 0;
+  int one_scale_end = 0;
+
+  for (int i = 0; i < RECTANGLE_WIDTH; i++) {
+    // scale location
+    if (scale[i] > 0 && scale[i + 1] > 0) {
+      if (!find_start) {
+        one_scale_start = i;
+        find_start = true;
+      }
+    }
+    if (find_start) {
+      if (scale[i] == 0 && scale[i + 1] == 0) {
+          one_scale_end = i - 1;
+          one_scale_location = (one_scale_start + one_scale_end) / 2.;
+          scale_location->push_back(one_scale_location);
+          one_scale_start = 0;
+          one_scale_end = 0;
+          find_start = false;
+      }
+    }
+  }
+  return true;
+}
+
+bool LocatePointer(const std::vector<int> &pointer,
+                   float *pointer_location) {
+  bool find_start = false;
+  int one_pointer_start = 0;
+  int one_pointer_end = 0;
+
+  for (int i = 0; i < RECTANGLE_WIDTH; i++) {
+    // pointer location
+    if (pointer[i] > 0 && pointer[i + 1] > 0) {
+      if (!find_start) {
+        one_pointer_start = i;
+        find_start = true;
+      }
+    }
+    if (find_start) {
+      if ((pointer[i] == 0) && (pointer[i+1] == 0)) {
+        one_pointer_end = i - 1;
+        *pointer_location = (one_pointer_start + one_pointer_end) / 2.;
+        one_pointer_start = 0;
+        one_pointer_end = 0;
+        find_start = false;
+        break;
+      }
+    }
+  }
+  return true;
+}
+
+bool GetRelativeLocation(
+  const std::vector<float> &scale_location,
+  const float &pointer_location,
+  MeterResult *result) {
+  int num_scales = static_cast<int>(scale_location.size());
+  result->num_scales_ = num_scales;
+  result->pointed_scale_ = -1;
+  if (num_scales > 0) {
+    for (auto i = 0; i < num_scales - 1; i++) {
+      if (scale_location[i] <= pointer_location &&
+            pointer_location < scale_location[i + 1]) {
+        result->pointed_scale_ = i + 1 +
+          (pointer_location-scale_location[i]) /
+          (scale_location[i+1]-scale_location[i] + 1e-05);
+      }
+    }
+  }
+  return true;
+}
+
+bool CalculateReading(const MeterResult &result,
+                      float *reading) {
+  // Provide a digital readout according to point location relative
+  // to the scales
+  if (result.num_scales_ > TYPE_THRESHOLD) {
+    *reading = result.pointed_scale_ * METER_CONFIG[0].scale_interval_value_;
+  } else {
+    *reading = result.pointed_scale_ * METER_CONFIG[1].scale_interval_value_;
+  }
+  return true;
+}
+
+bool PrintMeterReading(const std::vector<float> &readings) {
+  for (auto i = 0; i < readings.size(); ++i) {
+    std::cout << "Meter " << i + 1 << ": " << readings[i] << std::endl;
+  }
+  return true;
+}
+
+bool Visualize(const cv::Mat& img,
+               const PaddleDeploy::Result &det_result,
+               const std::vector<float> &reading,
+               cv::Mat* vis_img) {
+  for (auto i = 0; i < det_result.det_result->boxes.size(); ++i) {
+     std::string category = std::to_string(reading[i]);
+     det_result.det_result->boxes[i].category = category;
+  }
+
+  PaddleDeploy::Visualize(img, *(det_result.det_result), vis_img);
+  return true;
+}
+
+bool GetMeterReading(
+  const std::vector<std::vector<uint8_t>> &seg_label_maps,
+  std::vector<float> *readings) {
+  for (auto i = 0; i < seg_label_maps.size(); i++) {
+    std::vector<uint8_t> rectangle_meter;
+    CircleToRectangle(seg_label_maps[i], &rectangle_meter);
+
+    std::vector<int> line_scale;
+    std::vector<int> line_pointer;
+    RectangleToLine(rectangle_meter, &line_scale, &line_pointer);
+
+    std::vector<int> binaried_scale;
+    MeanBinarization(line_scale, &binaried_scale);
+    std::vector<int> binaried_pointer;
+    MeanBinarization(line_pointer, &binaried_pointer);
+
+    std::vector<float> scale_location;
+    LocateScale(binaried_scale, &scale_location);
+
+    float pointer_location;
+    LocatePointer(binaried_pointer, &pointer_location);
+
+    MeterResult result;
+    GetRelativeLocation(
+      scale_location, pointer_location, &result);
+
+    float reading;
+    CalculateReading(result, &reading);
+    readings->push_back(reading);
+  }
+  return true;
+}

dygraph/examples/meter_reader/reader_infer.py

@@ -528,22 +528,7 @@ class MeterReader:
                 erode_kernel=4,
                 score_threshold=0.5,
                 seg_batch_size=2):
-        """Detect meters in a image, segment scales and points in these meters, the postprocess are
-        done to provide a digital readout according to scale and point location.
-
-        Args:
-            im_file (str):  the path of a image to be predicted.
-            save_dir (str): the directory to save the visual prediction. Default: './'.
-            use_erode (bool, optional): whether to do image erosion by using a specific structuring element for
-                the label map output from the segmenter. Default: True.
-            erode_kernel (int, optional): structuring element used for erosion. Default: 4.
-            score_threshold (float, optional): detected meters whose scores are not lower than `score_threshold`
-                will be fed into the following segmenter. Default: 0.5.
-            seg_batch_size (int, optional): batch size of meters when do segmentation. Default: 2.
-
-        """
-        """检测图像中的表盘，而后分割出各表盘中的指针和刻度，对分割结果进行读数后厨后得到各表盘的读数。
-
+        """检测图像中的表盘，而后分割出各表盘中的指针和刻度，对分割结果进行读数后处理后得到各表盘的读数。
 
         参数：
             img_file (str)：待预测的图片路径。

dygraph/examples/meter_reader/train_segmentation.py

@@ -16,7 +16,7 @@ eval_transforms = T.Compose([
         mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]),
 ])
 
-# 下载和解压指针刻度分割数据集，如果已经预先下载，可注视掉下面两行
+# 下载和解压指针刻度分割数据集，如果已经预先下载，可注释掉下面两行
 meter_seg_dataset = 'https://bj.bcebos.com/paddlex/examples/meter_reader/datasets/meter_seg.tar.gz'
 pdx.utils.download_and_decompress(meter_seg_dataset, path='./')