vinbigdata-cxr-kaggle-pipeline/VINBIGDATA_ARCHITECTURE.md

VinBigData 结构图

这份结构图描述当前仓库的主流程、各 notebook 职责，以及它们之间的输入输出关系。

1. 总体流水线

flowchart LR
    A[官方比赛数据<br/>train.csv / test dicom] --> B[图像预处理数据集<br/>256 / 512 / 1024 PNG]
    B --> C[yolov5-chest-512.ipynb<br/>主检测流程]
    A --> C
    B --> D[vinbigdata-2-class-classifier-complete-pipeline.ipynb<br/>normal / abnormal 二分类]
    A --> D

    C --> E[14 类异常检测结果]
    D --> F[test_pred.csv / valid_pred.csv<br/>整图 normal 概率]

    E --> G[后处理逻辑<br/>Keep / Add / Replace]
    F --> G
    G --> H[submission.csv]

    H --> I[ensemble-of-best-public-notebooks.ipynb<br/>轻量 2-class 再过滤]
    F --> I
    I --> J[postprocessed submission.csv]

    H --> K[ensembling-approach.ipynb<br/>多 submission 融合]
    J --> K
    K --> L[final submission.csv]

2. Notebook 职责图

flowchart TD
    A[yolov5-chest-512.ipynb] --> A1[读取比赛数据]
    A --> A2[MultilabelStratifiedKFold 分 fold]
    A --> A3[生成 YOLO 标签]
    A --> A4[调用检测模型推理]
    A --> A5[结合整图分类做后处理]
    A --> A6[导出 submission.csv]

    B[vinbigdata-cxr-ad-yolov5-14-class-infer.ipynb] --> B1[加载 best.pt]
    B --> B2[运行 detect.py]
    B --> B3[YOLO 坐标转回比赛格式]
    B --> B4[纯检测 baseline submission]

    C[vinbigdata-2-class-classifier-complete-pipeline.ipynb] --> C1[构造 normal / abnormal 标签]
    C --> C2[StratifiedKFold 训练]
    C --> C3[导出 valid_pred.csv]
    C --> C4[导出 test_pred.csv]
    C --> C5[按阈值修正检测 submission]

    D[ensemble-of-best-public-notebooks.ipynb] --> D1[读取已有 submission]
    D --> D2[读取 2-class 概率]
    D --> D3[Keep / Add / Replace]

    E[ensembling-approach.ipynb] --> E1[读取多个 submission]
    E --> E2[按图片拆分 PredictionString]
    E --> E3[top-n 保留]
    E --> E4[同类框平均融合]

3. 关键后处理决策

flowchart TD
    A[输入: 一张图的检测结果 + class0 概率] --> B{normal 概率阈值}
    B -->|p < low_threshold| C[Keep<br/>保留检测结果]
    B -->|low_threshold <= p < high_threshold| D[Add<br/>保留检测结果并追加 class 14]
    B -->|p >= high_threshold| E[Replace<br/>替换为 14 1 0 0 1 1]

4. 代码层面的对应关系

• 主检测流程：yolov5-chest-512.ipynb
• 纯检测推理：vinbigdata-cxr-ad-yolov5-14-class-infer.ipynb
• 二分类与后处理：vinbigdata-2-class-classifier-complete-pipeline.ipynb
• 轻量再过滤：ensemble-of-best-public-notebooks.ipynb
• 多结果融合：ensembling-approach.ipynb

5. 适合怎么用

• 想先理解全局：先看“总体流水线”
• 想找每个 notebook 的职责：看“Notebook 职责图”
• 想单独复现提分逻辑：看“关键后处理决策”