!9 [cherry-pick] fix the iteration method (#4576)

Merge pull request !9 from zhch158/release/3.2

deploy/hps/README.md

@@ -10,9 +10,9 @@ comments: true
 
 **请注意，本项目依赖于如下环境配置：**
 
+- **CPU 架构**：x86-64
 - **操作系统**：Linux
-- **Docker 版本**：`>= 20.10.0`，用于镜像构建和部署
-- **CPU 架构**：x86-64 
+- **Docker Engine 版本**：`>= 20.10.0`，用于镜像构建和部署
 
 本文档主要介绍如何基于本项目提供的脚本完成高稳定性服务化部署环境搭建与物料打包。整体流程分为两个阶段：
 
@@ -48,7 +48,7 @@ comments: true
 为了使构建结果的可重现性更强，本步骤将依赖锁定到精确版本。请切换至 `server_env` 目录执行如下脚本：
 
 ```bash
-./script/freeze_requirements.sh
+./scripts/freeze_requirements.sh
 ```
 
 该脚本调用 `pip-tools compile` 解析依赖源文件，并最终生成一系列 `.txt` 文件（如 `requirements/gpu.txt`、`requirements/cpu.txt` 等），这些文件将为 [1.3 镜像构建](./README.md#13-镜像构建) 提供依赖版本约束。
@@ -85,7 +85,7 @@ comments: true
 对于 Triton Server，项目使用预先编译好的版本，将在构建镜像时自动下载，无需手动下载。以构建 GPU 镜像为例，在 `server_env` 目录下执行以下命令：
 
 ```bash
-./scripts/build_deployment_image.sh -k gpu -t latest-gpu 
+./scripts/build_deployment_image.sh -k gpu -t latest-gpu
 ```
 
 构建镜像的参数配置项包括
@@ -118,10 +118,10 @@ comments: true
 执行成功后，命令行会输出以下提示信息：
 
 ```text
- => => exporting to image                                                         
- => => exporting layers                                                      
- => => writing image  sha256:ba3d0b2b079d63ee0239a99043fec7e25f17bf2a7772ec2fc80503c1582b3459   
- => => naming to ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:latest-gpu   
+ => => exporting to image
+ => => exporting layers
+ => => writing image  sha256:ba3d0b2b079d63ee0239a99043fec7e25f17bf2a7772ec2fc80503c1582b3459
+ => => naming to ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:latest-gpu
 ```
 
 如需批量构建 GPU 和 CPU 镜像，可以执行以下命令：
@@ -172,7 +172,7 @@ comments: true
 </tbody>
 </table>
 
-调用后存储到当前目录 `/output` 路径下。
+调用后存储到当前目录 `output` 路径下。
 
 ## 3.FAQ
 

deploy/hps/README_en.md

@@ -11,9 +11,9 @@ This project provides a high-stability serving solution, consisting of two main
 
 **Note: This project relies on the following environment configurations:**
 
-- **Operating System**: Linux
-- **Docker Version**: `>= 20.10.0` (Used for image building and deployment)
 - **CPU Architecture**: x86-64
+- **Operating System**: Linux
+- **Docker Engine Version**: `>= 20.10.0` (Used for image building and deployment)
 
 This  document  mainly introduces how to set up a high stability serving environment and package related materials using the scripts provided by this project. The overall process consists of two main stages:
 
@@ -32,13 +32,13 @@ Image Building Steps:
 
 1. Build a requirement collection image. (Optional)
 2. Freeze requirement versions to improve the reproducibility of deployment image building. (Optional)
-3. Build the deployment image based on the frozen requirement information to generate the final deployment image and provide image support for subsequent pipeline execution. 
+3. Build the deployment image based on the frozen requirement information to generate the final deployment image and provide image support for subsequent pipeline execution.
 
 **If you do not need to modify requirement-related information, you can skip to [1.3 Building Image](./README_en.md#13-building-image) to build the deployment image using cached requirement information.**
 
 ## 1.1 Build the Requirement Collection Image (Optional)
 
-Navigate to the `server_env` directory and run follow script for building the requirement collection image in this directory. 
+Navigate to the `server_env` directory and run follow script for building the requirement collection image in this directory.
 
 ```bash
 ./scripts/prepare_rc_image.sh
@@ -121,10 +121,10 @@ If the basic image cannot be pulled, please refer to the solutions in the [FAQ](
 After run successfully, the command line will display the following message:
 
 ```text
- => => exporting to image                                                         
- => => exporting layers                                                      
- => => writing image  sha256:ba3d0b2b079d63ee0239a99043fec7e25f17bf2a7772ec2fc80503c1582b3459   
- => => naming to ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:latest-gpu   
+ => => exporting to image
+ => => exporting layers
+ => => writing image  sha256:ba3d0b2b079d63ee0239a99043fec7e25f17bf2a7772ec2fc80503c1582b3459
+ => => naming to ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:latest-gpu
 ```
 
 To build both GPU and CPU images  run the following command:
@@ -140,7 +140,7 @@ This stage mainly introduces how to package pipeline materials. This function is
 - `client`: Responsible for invoking the model services.
 - `server`: Deployed using the images built in [1. Image Building](./README_en.md#1-image-building), serving as the runtime environment for model services.
 
-Before packaging the pipeline materials, you need to switch to the `sdk` directory and run the `scripts/assemble.sh` script in this directory for  packaging. For example, to package the general OCR pipeline, run:
+Before packaging the pipeline materials, you need to switch to the `sdk` directory and run the `scripts/assemble.sh` script in this directory for packaging. For example, to package the general OCR pipeline, run:
 
 ```bash
 ./scripts/assemble.sh OCR
@@ -175,7 +175,7 @@ The parameters for the packaging script are described as follows:
 </tbody>
 </table>
 
-After run successfully, the packaged  will be stored in the `/output` directory.
+After run successfully, the packaged  will be stored in the `output` directory.
 
 ## 3. FAQ
 
@@ -191,4 +191,4 @@ When running the image build scripts, you can use the `-p` parameter to specify
 
 ```bash
 ./scripts/prepare_rc_image.sh -p  https://pypi.tuna.tsinghua.edu.cn/simple
-```
+```

deploy/hps/server_env/scripts/remove_images.sh

@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 
-paddlex_version="$(cat ../../paddlex/.version)"
+paddlex_version="$(cat ../../../paddlex/.version)"
 
 for device_type in 'gpu' 'cpu'; do
     version="$(cat "${device_type}_version.txt")"

deploy/hps/server_env/scripts/tag_and_push_images.sh

@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 
-paddlex_version="$(cat ../../paddlex/.version)"
+paddlex_version="$(cat ../../../paddlex/.version)"
 
 for device_type in 'gpu' 'cpu'; do
     version="$(cat "${device_type}_version.txt")"

paddlex/.version

@@ -1 +1 @@
-3.2.0
+3.2.1

paddlex/inference/models/common/static_infer.py

@@ -445,6 +445,12 @@ class PaddleInfer(StaticInfer):
                     # Delete unsupported passes in dcu
                     config.delete_pass("conv2d_add_act_fuse_pass")
                     config.delete_pass("conv2d_add_fuse_pass")
+            elif self._option.device_type == "iluvatar_gpu":
+                config.enable_custom_device("iluvatar_gpu", int(self._option.device_id))
+                if hasattr(config, "enable_new_ir"):
+                    config.enable_new_ir(self._option.enable_new_ir)
+                if hasattr(config, "enable_new_executor"):
+                    config.enable_new_executor()
             else:
                 assert self._option.device_type == "cpu"
                 config.disable_gpu()

paddlex/inference/models/common/vlm/fusion_ops.py

@@ -49,6 +49,8 @@ def get_env_device():
         return "gcu"
     elif "intel_hpu" in paddle.device.get_all_custom_device_type():
         return "intel_hpu"
+    elif "iluvatar_gpu" in paddle.device.get_all_custom_device_type():
+        return "iluvatar_gpu"
     elif paddle.is_compiled_with_rocm():
         return "rocm"
     elif paddle.is_compiled_with_xpu():
@@ -61,7 +63,7 @@ try:
 except ImportError:
     fused_rotary_position_embedding = None
 try:
-    if get_env_device() in ["npu", "mlu", "gcu"]:
+    if get_env_device() in ["npu", "mlu", "gcu", "iluvatar_gpu"]:
         from paddle.base import core
 
         for lib in os.listdir(os.getenv("CUSTOM_DEVICE_ROOT")):
@@ -84,7 +86,7 @@ def fusion_rope(
     rotary_emb,
     context_parallel_degree=-1,
 ):
-    if get_env_device() not in ["gcu", "intel_hpu"]:
+    if get_env_device() not in ["gcu", "intel_hpu", "iluvatar_gpu"]:
         assert past_key_value is None, "fuse rotary not support cache kv for now"
     batch_size, seq_length, num_heads, head_dim = query_states.shape
     _, kv_seq_len, num_key_value_heads, _ = key_states.shape
@@ -93,7 +95,7 @@ def fusion_rope(
             get_env_device() == "gpu"
         ), "context parallel only support cuda device for now"
         kv_seq_len *= context_parallel_degree
-    if get_env_device() not in ["gcu", "intel_hpu"]:
+    if get_env_device() not in ["gcu", "intel_hpu", "iluvatar_gpu"]:
         cos, sin = rotary_emb(value_states, seq_len=kv_seq_len)
     if get_env_device() == "npu":
         query_states = core.eager._run_custom_op("fused_rope", query_states, cos, sin)[

paddlex/inference/models/formula_recognition/processors.py

@@ -337,8 +337,8 @@ class LaTeXOCRDecode(object):
             str: The post-processed LaTeX string.
         """
         text_reg = r"(\\(operatorname|mathrm|text|mathbf)\s?\*? {.*?})"
-        letter = "[a-zA-Z]"
-        noletter = "[\W_^\d]"
+        letter = r"[a-zA-Z]"
+        noletter = r"[\W_^\d]"
         names = [x[0].replace(" ", "") for x in re.findall(text_reg, s)]
         s = re.sub(text_reg, lambda match: str(names.pop(0)), s)
         news = s
@@ -840,8 +840,8 @@ class UniMERNetDecode(object):
             str: Normalized string.
         """
         text_reg = r"(\\(operatorname|mathrm|text|mathbf)\s?\*? {.*?})"
-        letter = "[a-zA-Z]"
-        noletter = "[\W_^\d]"
+        letter = r"[a-zA-Z]"
+        noletter = r"[\W_^\d]"
         names = []
         for x in re.findall(text_reg, s):
             pattern = r"\\[a-zA-Z]+"
@@ -874,7 +874,7 @@ class UniMERNetDecode(object):
         return s.replace("XXXXXXX", " ")
 
     def remove_chinese_text_wrapping(self, formula):
-        pattern = re.compile(r"\\text\s*{\s*([^}]*?[\u4e00-\u9fff]+[^}]*?)\s*}")
+        pattern = re.compile(r"\\text\s*{([^{}]*[\u4e00-\u9fff]+[^{}]*)}")
 
         def replacer(match):
             return match.group(1)

paddlex/inference/models/formula_recognition/result.py

@@ -164,18 +164,39 @@ def generate_tex_file(tex_file_path: str, equation: str) -> None:
         equation (str): The LaTeX equation to be written into the file.
     """
     with custom_open(tex_file_path, "w") as fp:
-        start_template = (
-            r"\documentclass[varwidth]{standalone}" + "\n"
-            r"\usepackage{cite}" + "\n"
-            r"\usepackage{amsmath,amssymb,amsfonts,upgreek}" + "\n"
-            r"\usepackage{graphicx}" + "\n"
-            r"\usepackage{textcomp}" + "\n"
-            r"\usepackage{xeCJK}" + "\n"
-            r"\DeclareMathSizes{14}{14}{9.8}{7}" + "\n"
-            r"\pagestyle{empty}" + "\n"
-            r"\begin{document}" + "\n"
-            r"\begin{large}" + "\n"
-        )
+        start_template = r"""
+            \documentclass[varwidth]{standalone}
+            \usepackage{cite}
+            \usepackage{amsmath,amssymb,amsfonts,upgreek}
+            \usepackage{graphicx}
+            \usepackage{textcomp}
+            \usepackage{xeCJK}
+            \DeclareMathSizes{14}{14}{9.8}{7}
+            \pagestyle{empty}
+            \makeatletter
+            \def\x@arrow{\DOTSB\Relbar}
+            \def\xlongequalsignfill@{\arrowfill@\x@arrow\Relbar\x@arrow}
+            \newcommand{\xlongequal}[2][]{\ext@arrow 0099\xlongequalsignfill@{#1}{#2}}
+            \def\xLongleftrightarrowfill@{\arrowfill@\Longleftarrow\Relbar\Longrightarrow}
+            \newcommand{\xLongleftrightarrow}[2][]{\ext@arrow 0099\xLongleftrightarrowfill@{#1}{#2}}
+            \def\xlongleftrightarrowfill@{\arrowfill@\longleftarrow\relbar\longrightarrow}
+            \newcommand{\xlongleftrightarrow}[2][]{\ext@arrow 0099\xlongleftrightarrowfill@{#1}{#2}}
+            \def\xLeftrightarrowfill@{\arrowfill@\Leftarrow\Relbar\Rightarrow}
+            \newcommand{\xLeftrightarrow}[2][]{\ext@arrow 0099\xLeftrightarrowfill@{#1}{#2}}
+            \def\xleftrightarrowfill@{\arrowfill@\leftarrow\relbar\rightarrow}
+            \newcommand{\xleftrightarrow}[2][]{\ext@arrow 0099\xleftrightarrowfill@{#1}{#2}}
+            \def\xLongleftarrowfill@{\arrowfill@\Longleftarrow\Relbar\Relbar}
+            \newcommand{\xLongleftarrow}[2][]{\ext@arrow 0099\xLongleftarrowfill@{#1}{#2}}
+            \def\xLongrightarrowfill@{\arrowfill@\Relbar\Relbar\Longrightarrow}
+            \newcommand{\xLongrightarrow}[2][]{\ext@arrow 0099\xLongrightarrowfill@{#1}{#2}}
+            \def\xlongleftarrowfill@{\arrowfill@\longleftarrow\relbar\relbar}
+            \newcommand{\xlongleftarrow}[2][]{\ext@arrow 0099\xlongleftarrowfill@{#1}{#2}}
+            \def\xlongrightarrowfill@{\arrowfill@\relbar\relbar\longrightarrow}
+            \newcommand{\xlongrightarrow}[2][]{\ext@arrow 0099\xlongrightarrowfill@{#1}{#2}}
+            \makeatother
+            \begin{document}
+            \begin{large}
+        """
         fp.write(start_template)
         equation = add_text_for_zh_formula(equation)
         equation = get_align_equation(equation)

paddlex/inference/pipelines/_parallel.py

@@ -44,7 +44,7 @@ class MultiDeviceSimpleInferenceExecutor(object):
                 input_future_pairs = []
                 for pipeline in self._pipelines:
                     try:
-                        input_batch = list(input_batches)[0]
+                        input_batch = next(input_batches)
                     except StopIteration:
                         out_of_data = True
                         break

paddlex/inference/pipelines/formula_recognition/pipeline.py

@@ -268,7 +268,7 @@ class _FormulaRecognitionPipeline(BasePipeline):
                     layout_det_results = []
                     for _ in doc_preprocessor_images:
                         try:
-                            layout_det_res = list(external_layout_det_results)[0]
+                            layout_det_res = next(external_layout_det_results)
                         except StopIteration:
                             raise ValueError("No more layout det results")
                         layout_det_results.append(layout_det_res)

paddlex/inference/pipelines/seal_recognition/pipeline.py

@@ -259,7 +259,7 @@ class _SealRecognitionPipeline(BasePipeline):
                     layout_det_results = []
                     for _ in doc_preprocessor_images:
                         try:
-                            layout_det_res = list(external_layout_det_results)[0]
+                            layout_det_res = next(external_layout_det_results)
                         except StopIteration:
                             raise ValueError("No more layout det results")
                         layout_det_results.append(layout_det_res)

paddlex/inference/utils/official_models.py

@@ -405,7 +405,15 @@ class _BaseModelHoster(ABC):
             model_name in self.model_list
         ), f"The model {model_name} is not supported on hosting {self.__class__.__name__}!"
         model_dir = self._save_dir / f"{model_name}"
-        self._download(model_name, model_dir)
+        if os.path.exists(model_dir):
+            logging.info(
+                f"Model files already exist. Using cached files. To redownload, please delete the directory manually: `{model_dir}`."
+            )
+        else:
+            logging.info(
+                f"Using official model ({model_name}), the model files will be automatically downloaded and saved in `{model_dir}`."
+            )
+            self._download(model_name, model_dir)
         return model_dir
 
     @abstractmethod
@@ -543,9 +551,6 @@ Otherwise, only local models can be used."""
         return hosters
 
     def _get_model_local_path(self, model_name):
-        logging.info(
-            f"Using official model ({model_name}), the model files will be automatically downloaded and saved in {self._save_dir}."
-        )
         if len(self._hosters) == 0:
             msg = "No available model hosting platforms detected. Please check your network connection."
             logging.error(msg)

paddlex/inference/utils/pp_option.py

@@ -54,7 +54,7 @@ class PaddlePredictorOption(object):
         "mkldnn",
         "mkldnn_bf16",
     )
-    SUPPORT_DEVICE = ("gpu", "cpu", "npu", "xpu", "mlu", "dcu", "gcu")
+    SUPPORT_DEVICE = ("gpu", "cpu", "npu", "xpu", "mlu", "dcu", "gcu", "iluvatar_gpu")
 
     def __init__(self, **kwargs):
         super().__init__()

paddlex/repo_apis/PaddleOCR_api/text_rec/config.py

@@ -249,6 +249,7 @@ class TextRecConfig(BaseConfig):
             "Global.use_npu": False,
             "Global.use_mlu": False,
             "Global.use_gcu": False,
+            "Global.use_iluvatar_gpu": False,
         }
 
         device_cfg = {
@@ -258,6 +259,7 @@ class TextRecConfig(BaseConfig):
             "mlu": {"Global.use_mlu": True},
             "npu": {"Global.use_npu": True},
             "gcu": {"Global.use_gcu": True},
+            "iluvatar_gpu": {"Global.use_iluvatar_gpu": True},
         }
         default_cfg.update(device_cfg[device])
         self.update(default_cfg)

paddlex/utils/device.py

@@ -25,7 +25,7 @@ from .custom_device_list import (
 )
 from .flags import DISABLE_DEV_MODEL_WL
 
-SUPPORTED_DEVICE_TYPE = ["cpu", "gpu", "xpu", "npu", "mlu", "gcu", "dcu"]
+SUPPORTED_DEVICE_TYPE = ["cpu", "gpu", "xpu", "npu", "mlu", "gcu", "dcu", "iluvatar_gpu"]
 
 
 def constr_device(device_type, device_ids):