|
|
@@ -0,0 +1,324 @@
|
|
|
+```bash
|
|
|
+pip install onnx2pytorch
|
|
|
+paddlex \
|
|
|
+ --paddle2onnx \
|
|
|
+ --paddle_model_dir /Users/zhch158/.paddlex/official_models/RT-DETR-H_layout_17cls \
|
|
|
+ --onnx_model_dir ./unified_pytorch_models/Layout
|
|
|
+
|
|
|
+paddlex \
|
|
|
+ --paddle2onnx \
|
|
|
+ --paddle_model_dir /Users/zhch158/.paddlex/official_models/PP-LCNet_x1_0_doc_ori \
|
|
|
+ --onnx_model_dir ./unified_pytorch_models/Layout
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+# onnx方向检测算法说明
|
|
|
+
|
|
|
+**mineru/model/ori_cls/paddle_ori_cls.py**
|
|
|
+
|
|
|
+## 方法概览
|
|
|
+
|
|
|
+这个方法将输入图像预处理为适合方向分类模型的格式,包含三个主要步骤:**缩放**、**裁剪**和**归一化**。
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 步骤 1: 图像缩放
|
|
|
+
|
|
|
+```python
|
|
|
+scale = 256 / min(h, w)
|
|
|
+h_resize = round(h * scale)
|
|
|
+w_resize = round(w * scale)
|
|
|
+img = cv2.resize(input_img, (w_resize, h_resize), interpolation=1)
|
|
|
+```
|
|
|
+
|
|
|
+**`256` 的含义:**
|
|
|
+
|
|
|
+- 这是**目标最短边长度**
|
|
|
+- 确保图像有足够的分辨率进行后续处理
|
|
|
+- 例如:如果原图是 100×200,缩放后变为 256×512
|
|
|
+
|
|
|
+**`interpolation=1`:**
|
|
|
+
|
|
|
+- 对应 `cv2.INTER_LINEAR`,双线性插值方法
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 步骤 2: 中心裁剪为正方形
|
|
|
+
|
|
|
+```python
|
|
|
+cw, ch = 224, 224
|
|
|
+x1 = max(0, (w - cw) // 2)
|
|
|
+y1 = max(0, (h - ch) // 2)
|
|
|
+x2 = min(w, x1 + cw)
|
|
|
+y2 = min(h, y1 + ch)
|
|
|
+```
|
|
|
+
|
|
|
+**`224×224` 的含义:**
|
|
|
+
|
|
|
+- 这是模型的**标准输入尺寸**
|
|
|
+- 大多数图像分类模型(如 ResNet、VGG)使用此尺寸
|
|
|
+- 从图像中心裁剪出 224×224 的区域
|
|
|
+
|
|
|
+**裁剪逻辑:**
|
|
|
+
|
|
|
+- `(w - cw) // 2`:计算水平方向的起始位置(居中)
|
|
|
+- `(h - ch) // 2`:计算垂直方向的起始位置(居中)
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 步骤 3: 归一化
|
|
|
+
|
|
|
+```python
|
|
|
+std = [0.229, 0.224, 0.225]
|
|
|
+scale = 0.00392156862745098 # 1/255
|
|
|
+mean = [0.485, 0.456, 0.406]
|
|
|
+alpha = [scale / std[i] for i in range(len(std))]
|
|
|
+beta = [-mean[i] / std[i] for i in range(len(std))]
|
|
|
+```
|
|
|
+
|
|
|
+**这些"魔法数字"的含义:**
|
|
|
+
|
|
|
+### `scale = 0.00392156862745098`
|
|
|
+
|
|
|
+- 等于 $\frac{1}{255}$
|
|
|
+- 将像素值从 $[0, 255]$ 范围归一化到 $[0, 1]$
|
|
|
+
|
|
|
+### `mean = [0.485, 0.456, 0.406]` 和 `std = [0.229, 0.224, 0.225]`
|
|
|
+
|
|
|
+- 这是 **ImageNet 数据集的统计值**(RGB 三个通道)
|
|
|
+- 用于标准化图像,使其分布与训练数据一致
|
|
|
+- 归一化公式:
|
|
|
+
|
|
|
+$$
|
|
|
+\text{normalized\_pixel} = \frac{\frac{\text{pixel}}{255} - \text{mean}}{\text{std}}
|
|
|
+$$
|
|
|
+
|
|
|
+**代码中的优化实现:**
|
|
|
+
|
|
|
+```python
|
|
|
+alpha = [scale / std[i] for i in range(len(std))] # [1/255/std[0], 1/255/std[1], 1/255/std[2]]
|
|
|
+beta = [-mean[i] / std[i] for i in range(len(std))]
|
|
|
+```
|
|
|
+
|
|
|
+- 预计算系数以提高效率
|
|
|
+- 每个通道: `pixel * alpha[c] + beta[c]`
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 潜在问题与改进建议
|
|
|
+
|
|
|
+### ⚠️ 问题 1: 重复定义变量
|
|
|
+
|
|
|
+```python
|
|
|
+# 类中已定义
|
|
|
+self.std = [0.229, 0.224, 0.225]
|
|
|
+self.mean = [0.485, 0.456, 0.406]
|
|
|
+self.cw, self.ch = 224, 224
|
|
|
+
|
|
|
+# 但方法中又重新定义了局部变量
|
|
|
+std = [0.229, 0.224, 0.225]
|
|
|
+cw, ch = 224, 224
|
|
|
+```
|
|
|
+
|
|
|
+**建议:**使用类属性以提高代码可维护性:
|
|
|
+
|
|
|
+````python
|
|
|
+def preprocess(self, input_img):
|
|
|
+ # 放大图片,使其最短边长为256
|
|
|
+ h, w = input_img.shape[:2]
|
|
|
+ scale = self.less_length / min(h, w) # 使用 self.less_length (256)
|
|
|
+ h_resize = round(h * scale)
|
|
|
+ w_resize = round(w * scale)
|
|
|
+ img = cv2.resize(input_img, (w_resize, h_resize), interpolation=1)
|
|
|
+
|
|
|
+ # 调整为224*224的正方形
|
|
|
+ h, w = img.shape[:2]
|
|
|
+ x1 = max(0, (w - self.cw) // 2) # 使用 self.cw
|
|
|
+ y1 = max(0, (h - self.ch) // 2) # 使用 self.ch
|
|
|
+ x2 = min(w, x1 + self.cw)
|
|
|
+ y2 = min(h, y1 + self.ch)
|
|
|
+
|
|
|
+ if w < self.cw or h < self.ch:
|
|
|
+ raise ValueError(
|
|
|
+ f"Input image ({w}, {h}) smaller than target ({self.cw}, {self.ch})."
|
|
|
+ )
|
|
|
+ img = img[y1:y2, x1:x2, ...]
|
|
|
+
|
|
|
+ # 正则化
|
|
|
+ split_im = list(cv2.split(img))
|
|
|
+ alpha = [self.scale / self.std[i] for i in range(len(self.std))] # 使用 self.std
|
|
|
+ beta = [-self.mean[i] / self.std[i] for i in range(len(self.std))] # 使用 self.mean
|
|
|
+
|
|
|
+ for c in range(img.shape[2]):
|
|
|
+ split_im[c] = split_im[c].astype(np.float32)
|
|
|
+ split_im[c] *= alpha[c]
|
|
|
+ split_im[c] += beta[c]
|
|
|
+
|
|
|
+ img = cv2.merge(split_im)
|
|
|
+ img = img.transpose((2, 0, 1)) # CHW 格式
|
|
|
+ imgs = [img]
|
|
|
+ x = np.stack(imgs, axis=0).astype(dtype=np.float32, copy=False)
|
|
|
+ return x
|
|
|
+````
|
|
|
+
|
|
|
+### ⚠️ 问题 2: 不必要的列表包装
|
|
|
+
|
|
|
+```python
|
|
|
+imgs = [img]
|
|
|
+x = np.stack(imgs, axis=0)
|
|
|
+```
|
|
|
+
|
|
|
+可以简化为:
|
|
|
+
|
|
|
+```python
|
|
|
+x = np.expand_dims(img, axis=0).astype(dtype=np.float32, copy=False)
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 总结
|
|
|
+
|
|
|
+| 数值 | 含义 | 作用 |
|
|
|
+| ------------ | ---------------- | ----------------------- |
|
|
|
+| `256` | 缩放后最短边长度 | 保证图像有足够分辨率 |
|
|
|
+| `224` | 模型输入尺寸 | 标准分类模型输入 |
|
|
|
+| `1/255` | 像素归一化因子 | 将 [0,255] 映射到 [0,1] |
|
|
|
+| `mean/std` | ImageNet 统计值 | 标准化图像分布 |
|
|
|
+
|
|
|
+这些数值都是计算机视觉领域的**标准配置**,源于在 ImageNet 数据集上训练的经典模型。
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+# 什么是正则化(Normalization)?
|
|
|
+
|
|
|
+在这个代码中的"正则化"实际上是**标准化(Standardization)**,是深度学习中的一种**数据预处理技术**。
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 1️⃣ 核心概念
|
|
|
+
|
|
|
+### 数学公式
|
|
|
+
|
|
|
+```python
|
|
|
+normalized_value = (pixel_value / 255 - mean) / std
|
|
|
+```
|
|
|
+
|
|
|
+对应代码中的实现:
|
|
|
+
|
|
|
+```python
|
|
|
+alpha = [scale / std[i] for i in range(len(std))] # scale = 1/255
|
|
|
+beta = [-mean[i] / std[i] for i in range(len(std))]
|
|
|
+
|
|
|
+# 对每个像素: pixel * alpha + beta
|
|
|
+split_im[c] *= alpha[c] # pixel/255/std
|
|
|
+split_im[c] += beta[c] # -mean/std
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 2️⃣ 为什么需要标准化?
|
|
|
+
|
|
|
+### **原因 1: 统一数据分布**
|
|
|
+
|
|
|
+```python
|
|
|
+# 原始像素值范围: [0, 255]
|
|
|
+# 不同图像的分布差异很大
|
|
|
+
|
|
|
+# 标准化后: 均值≈0, 标准差≈1
|
|
|
+# 使所有图像具有相似的统计特性
|
|
|
+```
|
|
|
+
|
|
|
+### **原因 2: 加速模型训练**
|
|
|
+
|
|
|
+- 梯度下降更稳定
|
|
|
+- 学习率更容易设置
|
|
|
+- 避免某些特征主导训练
|
|
|
+
|
|
|
+### **原因 3: 匹配训练数据分布**
|
|
|
+
|
|
|
+```python
|
|
|
+mean = [0.485, 0.456, 0.406] # ImageNet 数据集的均值
|
|
|
+std = [0.229, 0.224, 0.225] # ImageNet 数据集的标准差
|
|
|
+```
|
|
|
+
|
|
|
+这些值是从 ImageNet 百万张图像计算得出的,使用相同的标准化确保推理时的输入分布与训练时一致。
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 3️⃣ 代码逐步解析
|
|
|
+
|
|
|
+### **步骤 1: 分离通道**
|
|
|
+
|
|
|
+```python
|
|
|
+split_im = list(cv2.split(img))
|
|
|
+# 将 BGR 图像分离为 [B, G, R] 三个通道
|
|
|
+```
|
|
|
+
|
|
|
+### **步骤 2: 计算标准化系数**
|
|
|
+
|
|
|
+```python
|
|
|
+alpha = [scale / std[i] for i in range(len(std))]
|
|
|
+# alpha = [1/255/0.229, 1/255/0.224, 1/255/0.225]
|
|
|
+# alpha ≈ [0.01709, 0.01752, 0.01741]
|
|
|
+
|
|
|
+beta = [-mean[i] / std[i] for i in range(len(std))]
|
|
|
+# beta = [-0.485/0.229, -0.456/0.224, -0.406/0.225]
|
|
|
+# beta ≈ [-2.118, -2.036, -1.804]
|
|
|
+```
|
|
|
+
|
|
|
+### **步骤 3: 应用标准化**
|
|
|
+
|
|
|
+```python
|
|
|
+for c in range(img.shape[2]): # 对每个通道 (B, G, R)
|
|
|
+ split_im[c] = split_im[c].astype(np.float32)
|
|
|
+ split_im[c] *= alpha[c] # (pixel / 255) / std
|
|
|
+ split_im[c] += beta[c] # - mean / std
|
|
|
+```
|
|
|
+
|
|
|
+### **步骤 4: 合并通道**
|
|
|
+
|
|
|
+```python
|
|
|
+img = cv2.merge(split_im)
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 4️⃣ 实际效果对比
|
|
|
+
|
|
|
+```python
|
|
|
+# 示例:蓝色通道的一个像素值为 128
|
|
|
+
|
|
|
+# 原始值
|
|
|
+pixel = 128
|
|
|
+
|
|
|
+# 标准化后
|
|
|
+normalized = (128 / 255 - 0.485) / 0.229
|
|
|
+ = (0.502 - 0.485) / 0.229
|
|
|
+ ≈ 0.074
|
|
|
+
|
|
|
+# 代码中的计算
|
|
|
+result = 128 * alpha[0] + beta[0]
|
|
|
+ = 128 * 0.01709 - 2.118
|
|
|
+ ≈ 0.069 # 与上面结果一致(有浮点误差)
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 5️⃣ 术语澄清
|
|
|
+
|
|
|
+| 中文术语 | 英文术语 | 代码中的用法 |
|
|
|
+| ---------------- | --------------- | ---------------------------- |
|
|
|
+| **归一化** | Normalization | 将数据缩放到 [0,1] 或 [-1,1] |
|
|
|
+| **标准化** | Standardization | 使数据均值=0,标准差=1 |
|
|
|
+| **正则化** | Regularization | 防止过拟合的技术(L1/L2) |
|
|
|
+
|
|
|
+**代码注释有误:**
|
|
|
+
|
|
|
+```python
|
|
|
+# 正则化 ← 这里应该写"标准化"
|
|
|
+split_im = list(cv2.split(img))
|
|
|
+```
|
|
|
+
|
|
|
+---
|