86 lines
2.9 KiB
Markdown
86 lines
2.9 KiB
Markdown
# VinBigData 结构图
|
|
|
|
这份结构图描述当前仓库的主流程、各 notebook 职责，以及它们之间的输入输出关系。
|
|
|
|
## 1. 总体流水线
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
A[官方比赛数据<br/>train.csv / test dicom] --> B[图像预处理数据集<br/>256 / 512 / 1024 PNG]
|
|
B --> C[yolov5-chest-512.ipynb<br/>主检测流程]
|
|
A --> C
|
|
B --> D[vinbigdata-2-class-classifier-complete-pipeline.ipynb<br/>normal / abnormal 二分类]
|
|
A --> D
|
|
|
|
C --> E[14 类异常检测结果]
|
|
D --> F[test_pred.csv / valid_pred.csv<br/>整图 normal 概率]
|
|
|
|
E --> G[后处理逻辑<br/>Keep / Add / Replace]
|
|
F --> G
|
|
G --> H[submission.csv]
|
|
|
|
H --> I[ensemble-of-best-public-notebooks.ipynb<br/>轻量 2-class 再过滤]
|
|
F --> I
|
|
I --> J[postprocessed submission.csv]
|
|
|
|
H --> K[ensembling-approach.ipynb<br/>多 submission 融合]
|
|
J --> K
|
|
K --> L[final submission.csv]
|
|
```
|
|
|
|
## 2. Notebook 职责图
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
A[yolov5-chest-512.ipynb] --> A1[读取比赛数据]
|
|
A --> A2[MultilabelStratifiedKFold 分 fold]
|
|
A --> A3[生成 YOLO 标签]
|
|
A --> A4[调用检测模型推理]
|
|
A --> A5[结合整图分类做后处理]
|
|
A --> A6[导出 submission.csv]
|
|
|
|
B[vinbigdata-cxr-ad-yolov5-14-class-infer.ipynb] --> B1[加载 best.pt]
|
|
B --> B2[运行 detect.py]
|
|
B --> B3[YOLO 坐标转回比赛格式]
|
|
B --> B4[纯检测 baseline submission]
|
|
|
|
C[vinbigdata-2-class-classifier-complete-pipeline.ipynb] --> C1[构造 normal / abnormal 标签]
|
|
C --> C2[StratifiedKFold 训练]
|
|
C --> C3[导出 valid_pred.csv]
|
|
C --> C4[导出 test_pred.csv]
|
|
C --> C5[按阈值修正检测 submission]
|
|
|
|
D[ensemble-of-best-public-notebooks.ipynb] --> D1[读取已有 submission]
|
|
D --> D2[读取 2-class 概率]
|
|
D --> D3[Keep / Add / Replace]
|
|
|
|
E[ensembling-approach.ipynb] --> E1[读取多个 submission]
|
|
E --> E2[按图片拆分 PredictionString]
|
|
E --> E3[top-n 保留]
|
|
E --> E4[同类框平均融合]
|
|
```
|
|
|
|
## 3. 关键后处理决策
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
A[输入: 一张图的检测结果 + class0 概率] --> B{normal 概率阈值}
|
|
B -->|p < low_threshold| C[Keep<br/>保留检测结果]
|
|
B -->|low_threshold <= p < high_threshold| D[Add<br/>保留检测结果并追加 class 14]
|
|
B -->|p >= high_threshold| E[Replace<br/>替换为 14 1 0 0 1 1]
|
|
```
|
|
|
|
## 4. 代码层面的对应关系
|
|
|
|
- 主检测流程：`yolov5-chest-512.ipynb`
|
|
- 纯检测推理：`vinbigdata-cxr-ad-yolov5-14-class-infer.ipynb`
|
|
- 二分类与后处理：`vinbigdata-2-class-classifier-complete-pipeline.ipynb`
|
|
- 轻量再过滤：`ensemble-of-best-public-notebooks.ipynb`
|
|
- 多结果融合：`ensembling-approach.ipynb`
|
|
|
|
## 5. 适合怎么用
|
|
|
|
- 想先理解全局：先看“总体流水线”
|
|
- 想找每个 notebook 的职责：看“Notebook 职责图”
|
|
- 想单独复现提分逻辑：看“关键后处理决策”
|