mineru 命令执行后,除了输出主要的 markdown 文件外,还会生成多个辅助文件用于调试、质检和进一步处理。这些文件包括:
下面将详细介绍每个文件的作用和格式。
文件命名格式:{原文件名}_layout.pdf
功能说明:
使用场景:
[!NOTE] 仅适用于 pipeline 后端
文件命名格式:{原文件名}_spans.pdf
功能说明:
使用场景:
[!IMPORTANT] 2.5版本vlm后端的输出存在较大变化,与pipeline版本存在不兼容情况,如需基于结构化输出进行二次开发,请仔细阅读本文档内容。
文件命名格式:{原文件名}_model.json
from pydantic import BaseModel, Field
from enum import IntEnum
class CategoryType(IntEnum):
"""内容类别枚举"""
title = 0 # 标题
plain_text = 1 # 文本
abandon = 2 # 包括页眉页脚页码和页面注释
figure = 3 # 图片
figure_caption = 4 # 图片描述
table = 5 # 表格
table_caption = 6 # 表格描述
table_footnote = 7 # 表格注释
isolate_formula = 8 # 行间公式
formula_caption = 9 # 行间公式的标号
embedding = 13 # 行内公式
isolated = 14 # 行间公式
text = 15 # OCR 识别结果
class PageInfo(BaseModel):
"""页面信息"""
page_no: int = Field(description="页码序号,第一页的序号是 0", ge=0)
height: int = Field(description="页面高度", gt=0)
width: int = Field(description="页面宽度", ge=0)
class ObjectInferenceResult(BaseModel):
"""对象识别结果"""
category_id: CategoryType = Field(description="类别", ge=0)
poly: list[float] = Field(description="四边形坐标,格式为 [x0,y0,x1,y1,x2,y2,x3,y3]")
score: float = Field(description="推理结果的置信度")
latex: str | None = Field(description="LaTeX 解析结果", default=None)
html: str | None = Field(description="HTML 解析结果", default=None)
class PageInferenceResults(BaseModel):
"""页面推理结果"""
layout_dets: list[ObjectInferenceResult] = Field(description="页面识别结果")
page_info: PageInfo = Field(description="页面元信息")
# 完整的推理结果
inference_result: list[PageInferenceResults] = []
poly 坐标格式:[x0, y0, x1, y1, x2, y2, x3, y3]
[
{
"layout_dets": [
{
"category_id": 2,
"poly": [
99.1906967163086,
100.3119125366211,
730.3707885742188,
100.3119125366211,
730.3707885742188,
245.81326293945312,
99.1906967163086,
245.81326293945312
],
"score": 0.9999997615814209
}
],
"page_info": {
"page_no": 0,
"height": 2339,
"width": 1654
}
},
{
"layout_dets": [
{
"category_id": 5,
"poly": [
99.13092803955078,
2210.680419921875,
497.3183898925781,
2210.680419921875,
497.3183898925781,
2264.78076171875,
99.13092803955078,
2264.78076171875
],
"score": 0.9999997019767761
}
],
"page_info": {
"page_no": 1,
"height": 2339,
"width": 1654
}
}
]
文件命名格式:{原文件名}_middle.json
| 字段名 | 类型 | 说明 |
|---|---|---|
pdf_info |
list[dict] |
每一页的解析结果数组 |
_backend |
string |
解析模式:pipeline 或 vlm |
_version_name |
string |
MinerU 版本号 |
| 字段名 | 说明 |
|---|---|
preproc_blocks |
PDF 预处理后的未分段中间结果 |
page_idx |
页码,从 0 开始 |
page_size |
页面的宽度和高度 [width, height] |
images |
图片块信息列表 |
tables |
表格块信息列表 |
interline_equations |
行间公式块信息列表 |
discarded_blocks |
需要丢弃的块信息 |
para_blocks |
分段后的内容块结果 |
一级块 (table | image)
└── 二级块
└── 行 (line)
└── 片段 (span)
| 字段名 | 说明 |
|---|---|
type |
块类型:table 或 image |
bbox |
块的矩形框坐标 [x0, y0, x1, y1] |
blocks |
包含的二级块列表 |
| 字段名 | 说明 |
|---|---|
type |
块类型(详见下表) |
bbox |
块的矩形框坐标 |
lines |
包含的行信息列表 |
| 类型 | 说明 |
|---|---|
image_body |
图像本体 |
image_caption |
图像描述文本 |
image_footnote |
图像脚注 |
table_body |
表格本体 |
table_caption |
表格描述文本 |
table_footnote |
表格脚注 |
text |
文本块 |
title |
标题块 |
index |
目录块 |
list |
列表块 |
interline_equation |
行间公式块 |
行 (line) 字段:
bbox:行的矩形框坐标spans:包含的片段列表片段 (span) 字段:
bbox:片段的矩形框坐标type:片段类型(image、table、text、inline_equation、interline_equation)content | img_path:文本内容或图片路径{
"pdf_info": [
{
"preproc_blocks": [
{
"type": "text",
"bbox": [
52,
61.956024169921875,
294,
82.99800872802734
],
"lines": [
{
"bbox": [
52,
61.956024169921875,
294,
72.0000228881836
],
"spans": [
{
"bbox": [
54.0,
61.956024169921875,
296.2261657714844,
72.0000228881836
],
"content": "dependent on the service headway and the reliability of the departure ",
"type": "text",
"score": 1.0
}
]
}
]
}
],
"layout_bboxes": [
{
"layout_bbox": [
52,
61,
294,
731
],
"layout_label": "V",
"sub_layout": []
}
],
"page_idx": 0,
"page_size": [
612.0,
792.0
],
"_layout_tree": [],
"images": [],
"tables": [],
"interline_equations": [],
"discarded_blocks": [],
"para_blocks": [
{
"type": "text",
"bbox": [
52,
61.956024169921875,
294,
82.99800872802734
],
"lines": [
{
"bbox": [
52,
61.956024169921875,
294,
72.0000228881836
],
"spans": [
{
"bbox": [
54.0,
61.956024169921875,
296.2261657714844,
72.0000228881836
],
"content": "dependent on the service headway and the reliability of the departure ",
"type": "text",
"score": 1.0
}
]
}
]
}
]
}
],
"_backend": "pipeline",
"_version_name": "0.6.1"
}
文件命名格式:{原文件名}_content_list.json
这是一个简化版的 middle.json,按阅读顺序平铺存储所有可读内容块,去除了复杂的布局信息,便于后续处理。
| 类型 | 说明 |
|---|---|
image |
图片 |
table |
表格 |
text |
文本/标题 |
equation |
行间公式 |
通过 text_level 字段区分文本层级:
text_level 或 text_level: 0:正文文本text_level: 1:一级标题text_level: 2:二级标题page_idx 字段,表示所在页码(从 0 开始)。bbox 字段,表示内容块的边界框坐标 [x0, y0, x1, y1] 映射在0-1000范围内的结果。[
{
"type": "text",
"text": "The response of flow duration curves to afforestation ",
"text_level": 1,
"bbox": [
62,
480,
946,
904
],
"page_idx": 0
},
{
"type": "image",
"img_path": "images/a8ecda1c69b27e4f79fce1589175a9d721cbdc1cf78b4cc06a015f3746f6b9d8.jpg",
"img_caption": [
"Fig. 1. Annual flow duration curves of daily flows from Pine Creek, Australia, 1989–2000. "
],
"img_footnote": [],
"bbox": [
62,
480,
946,
904
],
"page_idx": 1
},
{
"type": "equation",
"img_path": "images/181ea56ef185060d04bf4e274685f3e072e922e7b839f093d482c29bf89b71e8.jpg",
"text": "$$\nQ _ { \\% } = f ( P ) + g ( T )\n$$",
"text_format": "latex",
"bbox": [
62,
480,
946,
904
],
"page_idx": 2
},
{
"type": "table",
"img_path": "images/e3cb413394a475e555807ffdad913435940ec637873d673ee1b039e3bc3496d0.jpg",
"table_caption": [
"Table 2 Significance of the rainfall and time terms "
],
"table_footnote": [
"indicates that the rainfall term was significant at the $5 \\%$ level, $T$ indicates that the time term was significant at the $5 \\%$ level, \\* represents significance at the $10 \\%$ level, and na denotes too few data points for meaningful analysis. "
],
"table_body": "<html><body><table><tr><td rowspan=\"2\">Site</td><td colspan=\"10\">Percentile</td></tr><tr><td>10</td><td>20</td><td>30</td><td>40</td><td>50</td><td>60</td><td>70</td><td>80</td><td>90</td><td>100</td></tr><tr><td>Traralgon Ck</td><td>P</td><td>P,*</td><td>P</td><td>P</td><td>P,</td><td>P,</td><td>P,</td><td>P,</td><td>P</td><td>P</td></tr><tr><td>Redhill</td><td>P,T</td><td>P,T</td><td>,*</td><td>**</td><td>P.T</td><td>P,*</td><td>P*</td><td>P*</td><td>*</td><td>,*</td></tr><tr><td>Pine Ck</td><td></td><td>P,T</td><td>P,T</td><td>P,T</td><td>P,T</td><td>T</td><td>T</td><td>T</td><td>na</td><td>na</td></tr><tr><td>Stewarts Ck 5</td><td>P,T</td><td>P,T</td><td>P,T</td><td>P,T</td><td>P.T</td><td>P.T</td><td>P,T</td><td>na</td><td>na</td><td>na</td></tr><tr><td>Glendhu 2</td><td>P</td><td>P,T</td><td>P,*</td><td>P,T</td><td>P.T</td><td>P,ns</td><td>P,T</td><td>P,T</td><td>P,T</td><td>P,T</td></tr><tr><td>Cathedral Peak 2</td><td>P,T</td><td>P,T</td><td>P,T</td><td>P,T</td><td>P,T</td><td>*,T</td><td>P,T</td><td>P,T</td><td>P,T</td><td>T</td></tr><tr><td>Cathedral Peak 3</td><td>P.T</td><td>P.T</td><td>P,T</td><td>P,T</td><td>P,T</td><td>T</td><td>P,T</td><td>P,T</td><td>P,T</td><td>T</td></tr><tr><td>Lambrechtsbos A</td><td>P,T</td><td>P</td><td>P</td><td>P,T</td><td>*,T</td><td>*,T</td><td>*,T</td><td>*,T</td><td>*,T</td><td>T</td></tr><tr><td>Lambrechtsbos B</td><td>P,T</td><td>P,T</td><td>P,T</td><td>P,T</td><td>P,T</td><td>P,T</td><td>P,T</td><td>P,T</td><td>T</td><td>T</td></tr><tr><td>Biesievlei</td><td>P,T</td><td>P.T</td><td>P,T</td><td>P,T</td><td>*,T</td><td>*,T</td><td>T</td><td>T</td><td>P,T</td><td>P,T</td></tr></table></body></html>",
"bbox": [
62,
480,
946,
904
],
"page_idx": 5
}
]
文件命名格式:{原文件名}_model.json
type、bbox、angle、content 字段{
"text": "文本",
"title": "标题",
"equation": "行间公式",
"image": "图片",
"image_caption": "图片描述",
"image_footnote": "图片脚注",
"table": "表格",
"table_caption": "表格描述",
"table_footnote": "表格脚注",
"phonetic": "拼音",
"code": "代码块",
"code_caption": "代码描述",
"ref_text": "参考文献",
"algorithm": "算法块",
"list": "列表",
"header": "页眉",
"footer": "页脚",
"page_number": "页码",
"aside_text": "装订线旁注",
"page_footnote": "页面脚注"
}
bbox 坐标格式:[x0, y0, x1, y1]
[
[
{
"type": "header",
"bbox": [
0.077,
0.095,
0.18,
0.181
],
"angle": 0,
"score": null,
"block_tags": null,
"content": "ELSEVIER",
"format": null,
"content_tags": null
},
{
"type": "title",
"bbox": [
0.157,
0.228,
0.833,
0.253
],
"angle": 0,
"score": null,
"block_tags": null,
"content": "The response of flow duration curves to afforestation",
"format": null,
"content_tags": null
}
]
]
文件命名格式:{原文件名}_middle.json
vlm 后端的 middle.json 文件结构与 pipeline 后端类似,但存在以下差异:
sub_type字段区分list类型:
text(文本类型)ref_text(引用类型)discarded_blocks内元素type增加以下类型:
header(页眉)footer(页脚)page_number(页码)aside_text(装订线文本)page_footnote(脚注)angle字段,用来表示旋转角度,0,90,180,270list block 示例
{
"bbox": [
174,
155,
818,
333
],
"type": "list",
"angle": 0,
"index": 11,
"blocks": [
{
"bbox": [
174,
157,
311,
175
],
"type": "text",
"angle": 0,
"lines": [
{
"bbox": [
174,
157,
311,
175
],
"spans": [
{
"bbox": [
174,
157,
311,
175
],
"type": "text",
"content": "H.1 Introduction"
}
]
}
],
"index": 3
},
{
"bbox": [
175,
182,
464,
229
],
"type": "text",
"angle": 0,
"lines": [
{
"bbox": [
175,
182,
464,
229
],
"spans": [
{
"bbox": [
175,
182,
464,
229
],
"type": "text",
"content": "H.2 Example: Divide by Zero without Exception Handling"
}
]
}
],
"index": 4
}
],
"sub_type": "text"
}
code block 示例
{
"type": "code",
"bbox": [
114,
780,
885,
1231
],
"blocks": [
{
"bbox": [
114,
780,
885,
1231
],
"lines": [
{
"bbox": [
114,
780,
885,
1231
],
"spans": [
{
"bbox": [
114,
780,
885,
1231
],
"type": "text",
"content": "1 // Fig. H.1: DivideByZeroNoExceptionHandling.java \n2 // Integer division without exception handling. \n3 import java.util.Scanner; \n4 \n5 public class DivideByZeroNoExceptionHandling \n6 { \n7 // demonstrates throwing an exception when a divide-by-zero occurs \n8 public static int quotient( int numerator, int denominator ) \n9 { \n10 return numerator / denominator; // possible division by zero \n11 } // end method quotient \n12 \n13 public static void main(String[] args) \n14 { \n15 Scanner scanner = new Scanner(System.in); // scanner for input \n16 \n17 System.out.print(\"Please enter an integer numerator: \"); \n18 int numerator = scanner.nextInt(); \n19 System.out.print(\"Please enter an integer denominator: \"); \n20 int denominator = scanner.nextInt(); \n21"
}
]
}
],
"index": 17,
"angle": 0,
"type": "code_body"
},
{
"bbox": [
867,
160,
1280,
189
],
"lines": [
{
"bbox": [
867,
160,
1280,
189
],
"spans": [
{
"bbox": [
867,
160,
1280,
189
],
"type": "text",
"content": "Algorithm 1 Modules for MCTSteg"
}
]
}
],
"index": 19,
"angle": 0,
"type": "code_caption"
}
],
"index": 17,
"sub_type": "code"
}
文件命名格式:{原文件名}_content_list.json
vlm 后端的 content_list.json 文件结构与 pipeline 后端类似,伴随本次middle.json的变化,做了以下调整:
code类型,code类型包含两种"sub_type":
list类型,list类型包含两种"sub_type":
textref_textdiscarded_blocks的输出内容
headerfooterpage_numberaside_textpage_footnotecode 类型 content
{
"type": "code",
"sub_type": "algorithm",
"code_caption": [
"Algorithm 1 Modules for MCTSteg"
],
"code_body": "1: function GETCOORDINATE(d) \n2: $x \\gets d / l$ , $y \\gets d$ mod $l$ \n3: return $(x, y)$ \n4: end function \n5: function BESTCHILD(v) \n6: $C \\gets$ child set of $v$ \n7: $v' \\gets \\arg \\max_{c \\in C} \\mathrm{UCTScore}(c)$ \n8: $v'.n \\gets v'.n + 1$ \n9: return $v'$ \n10: end function \n11: function BACK PROPAGATE(v) \n12: Calculate $R$ using Equation 11 \n13: while $v$ is not a root node do \n14: $v.r \\gets v.r + R$ , $v \\gets v.p$ \n15: end while \n16: end function \n17: function RANDOMSEARCH(v) \n18: while $v$ is not a leaf node do \n19: Randomly select an untried action $a \\in A(v)$ \n20: Create a new node $v'$ \n21: $(x, y) \\gets \\mathrm{GETCOORDINATE}(v'.d)$ \n22: $v'.p \\gets v$ , $v'.d \\gets v.d + 1$ , $v'.\\Gamma \\gets v.\\Gamma$ \n23: $v'.\\gamma_{x,y} \\gets a$ \n24: if $a = -1$ then \n25: $v.lc \\gets v'$ \n26: else if $a = 0$ then \n27: $v.mc \\gets v'$ \n28: else \n29: $v.rc \\gets v'$ \n30: end if \n31: $v \\gets v'$ \n32: end while \n33: return $v$ \n34: end function \n35: function SEARCH(v) \n36: while $v$ is fully expanded do \n37: $v \\gets$ BESTCHILD(v) \n38: end while \n39: if $v$ is not a leaf node then \n40: $v \\gets$ RANDOMSEARCH(v) \n41: end if \n42: return $v$ \n43: end function",
"bbox": [
510,
87,
881,
740
],
"page_idx": 0
}
list 类型 content
{
"type": "list",
"sub_type": "text",
"list_items": [
"H.1 Introduction",
"H.2 Example: Divide by Zero without Exception Handling",
"H.3 Example: Divide by Zero with Exception Handling",
"H.4 Summary"
],
"bbox": [
174,
155,
818,
333
],
"page_idx": 0
}
discarded 类型 content
[{
"type": "header",
"text": "Journal of Hydrology 310 (2005) 253-265",
"bbox": [
363,
164,
623,
177
],
"page_idx": 0
},
{
"type": "page_footnote",
"text": "* Corresponding author. Address: Forest Science Centre, Department of Sustainability and Environment, P.O. Box 137, Heidelberg, Vic. 3084, Australia. Tel.: +61 3 9450 8719; fax: +61 3 9450 8644.",
"bbox": [
71,
815,
915,
841
],
"page_idx": 0
}]
以上文件为 MinerU 的完整输出结果,用户可根据需要选择合适的文件进行后续处理:
模型输出:
调试和验证:
内容提取:
二次开发: