KIND解析器
|KIND项目解析器
✨ 特性
📄 Single-Pass PDF Parsing: Leverages docling---{Mu, Layout Model, Tableformer, EasyOCR} to extract all elements—text(to MD.), layout, and images.
🤖 VLM-Powered Table Recognition: Analyzes high-resolution table images with a VLM to accurately convert tables with complex merged cells into clean HTML.
🧠 Hierrarchical Structure Generation: Automatically builds a folder structure that mirrors the parent-child relationships defined in the MCP.
🧩 Modular Pipeline: Decouples the PDF Parsing, VLM Table Processing, and Content Structuring steps, allowing them to be run independently.
🚀 High-Performance Asynchronous Processing: Utilizes asyncio to efficiently process with concurrent processing.⚙️ 详细工艺流程
该项目执行三个核心步骤，可以按顺序或独立运行以生成最终输出。
graph LR
A[Step 1: Conversion] --> B[Step 2: VLM Processing] --> C[Step 3: Structuring]
subgraph A
A1(Input: PDF) --> A2{Docling Parse};
A2 --> A3(Output: MD + Images);
end
subgraph B
B1(Input: Table Images) --> B2{VLM HTML Parsing};
B2 --> B3{Fix, Merge, Replace};
B3 --> B4(Output: Final MD);
end
subgraph C
C1(Input: Final MD) --> C2{Hierarchical Sectioning};
C2 --> C3(Output: Structured Data);
end🚀 快速开始
- 需求
- Python 3.12+
- 文档2.46.0
- pymupdf4llm
- 简易光学字符识别
- 模型\[布局模型、Tableformer（准确、快速）、EasyOCR（ko）、VLM（Nanonets OCR）\]
- 安装
# 1. Clone the repository
git clone
cd kind_parser
# 2. Install dependencies
pip install -r requirements.txt
# 3. Prepare Models
# Place the model artifacts in the directory specified by the --model_path argument.- 运行管道
要按顺序运行整个管道，请使用提供的shell脚本。
# Execute the full pipeline
bash pdf_parsing.sh🔧 用法
您可以使用main.py中的--step参数独立运行每个步骤。这对于调试或重新处理特定阶段非常有用。
- 步骤1:pdf转换
解析源PDF文件以生成Markdown和高分辨率表格图像。
python main.py --steps pdf_conversion- 步骤2:vlm_处理
使用VLM处理步骤1中的表图像，然后替换并合并原始Markdown中的表。
python main.py --steps vlm_processing- 第三步：结构化
从步骤2中获取最终的Markdown，并将其拆分为“Sections”（Gwan）和“Articles”（Jo），以创建最终的文件夹结构。
python main.py --steps structuring- 命令行参数
关键参数在utils/arg_parser.py中定义，可以在运行时动态更改。
python main.py \
--accelerator_thread 128 \
--model_path "$MODEL_PATH" \
--data_dir "$DATA_DIR" \
--output_dir "$OUTPUT_DIR" \
--file_list_path "$FILE_LIST_PATH" \
--pdf_parsing_num_workers 10 \
--image_resolution 4.0 \
--vlm_base_url "http://50.50.79.151:8000/v1" \
--vlm_model_name "Nanonets-OCR-s" \
--vlm_concurrency_limit 300 \
--steps "pdf_conversion"
...🏗️ 项目结构
parser/
├── main.py # CLI Entrypoint
├── pdf_parsing.sh # Pipeline Execution Script
├── core/ # Core components (Orchestrator, Interfaces)
├── process/ # Independent pipeline steps
├── service-object/ # Business logic services (PDFConverter, VLMProcessor, etc.)
├── models/ # Data classes and models (FileInfo, DocumentTree)
└── utils/ # Utility modules (ArgParser, Constants)🙏 致谢
如果没有文档库强大的文档分析功能，这个项目是不可能的。
📝 待办事项
[ ✅ ] Parallel Processing (Auto-calculate process num)
[ ✅ ] Constants to paths and parameters.
[ 🟢 ] Unit and integration tests for each service and step.
[ 🔴 ] Table postprocess Algorithm.