489 KiB
489 KiB
Image from https://www.pexels.com/ja-jp/photo/4989186/
VinBigData 2-class classifier complete pipeline
This competition is object detection task to find a class and location of thoracic abnormalities from chest x-ray image (radiographs).
I explained how to train detection model in the kernel 📸VinBigData detectron2 train.
However, it is mentioned that training 2 class classifier to understand which is the normal image is important to get high score.
- Kernel: VinBigData 🌟2 Class Filter🌟
- Discussion: [LB0.155] baseline solution
Here, I will introduce complete EDA, Training (with 5-fold cross validation) and Prediction pipeline for training 2-class classifier.
You can learn the usage of following tools to accelerate deep learning tasks in computer vision!
- pytorch: Deep learning framework, it's popular among researchers for its flexible usage. no need to explain detail!
- albumentations: Image augmentation library, developed by famous kagglers!
- timm: pytorch-image-models, it provides a lot of popular SoTA CNN models with pretrained weights.
- pytorch ignite: Traning/Evaluation abstraction framework on top of pytorch.
- pytorch pfn extras: It is used to add more feature-rich functionality on Ignite.
[UPDATE 2021/2/6] I added submission procedure using 2-class classification prediction result.
[UPDATE 2021/2/10] I added mixup augmentation and label smoothing explanation.
Also, further improvement using ChExpert dataset is mentioned at "Next step" section
Also, further improvement using ChExpert dataset is mentioned at "Next step" section
[Note] Actually, there is on-going discussion 1-step training & prediction, which suggests the way only using detection model while keeping performance almost same with this kernel.
In [1]:
import gc
import os
from pathlib import Path
import random
import sys
from tqdm.notebook import tqdm
import numpy as np
import pandas as pd
import scipy as sp
import matplotlib.pyplot as plt
import seaborn as sns
from IPython.core.display import display, HTML
# --- plotly ---
from plotly import tools, subplots
import plotly.offline as py
py.init_notebook_mode(connected=True)
import plotly.graph_objs as go
import plotly.express as px
import plotly.figure_factory as ff
import plotly.io as pio
pio.templates.default = "plotly_dark"
# --- models ---
from sklearn import preprocessing
from sklearn.model_selection import KFold
import lightgbm as lgb
import xgboost as xgb
import catboost as cb
import torch
# --- setup ---
pd.set_option('max_columns', 50)
In [2]:
!pip install detectron2 -f \
https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.7/index.html
!pip install pytorch-pfn-extras timmLooking in links: https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.7/index.html Collecting detectron2 Downloading https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.7/detectron2-0.4%2Bcu102-cp37-cp37m-linux_x86_64.whl (6.0 MB) [K |████████████████████████████████| 6.0 MB 1.7 MB/s [?25hRequirement already satisfied: tensorboard in /opt/conda/lib/python3.7/site-packages (from detectron2) (2.4.0) Requirement already satisfied: termcolor>=1.1 in /opt/conda/lib/python3.7/site-packages (from detectron2) (1.1.0) Requirement already satisfied: matplotlib in /opt/conda/lib/python3.7/site-packages (from detectron2) (3.2.1) Requirement already satisfied: pydot in /opt/conda/lib/python3.7/site-packages (from detectron2) (1.4.1) Requirement already satisfied: yacs>=0.1.6 in /opt/conda/lib/python3.7/site-packages (from detectron2) (0.1.8) Requirement already satisfied: future in /opt/conda/lib/python3.7/site-packages (from detectron2) (0.18.2) Requirement already satisfied: cloudpickle in /opt/conda/lib/python3.7/site-packages (from detectron2) (1.6.0) Requirement already satisfied: tqdm>4.29.0 in /opt/conda/lib/python3.7/site-packages (from detectron2) (4.45.0) Requirement already satisfied: tabulate in /opt/conda/lib/python3.7/site-packages (from detectron2) (0.8.7) Requirement already satisfied: Pillow>=7.1 in /opt/conda/lib/python3.7/site-packages (from detectron2) (8.0.1) Collecting fvcore<0.1.4,>=0.1.3 Downloading fvcore-0.1.3.post20210317.tar.gz (47 kB) [K |████████████████████████████████| 47 kB 528 kB/s [?25hRequirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from fvcore<0.1.4,>=0.1.3->detectron2) (1.18.5) Requirement already satisfied: yacs>=0.1.6 in /opt/conda/lib/python3.7/site-packages (from detectron2) (0.1.8) Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.7/site-packages (from fvcore<0.1.4,>=0.1.3->detectron2) (5.3.1) Requirement already satisfied: tqdm>4.29.0 in /opt/conda/lib/python3.7/site-packages (from detectron2) (4.45.0) Requirement already satisfied: termcolor>=1.1 in /opt/conda/lib/python3.7/site-packages (from detectron2) (1.1.0) Requirement already satisfied: Pillow>=7.1 in /opt/conda/lib/python3.7/site-packages (from detectron2) (8.0.1) Requirement already satisfied: tabulate in /opt/conda/lib/python3.7/site-packages (from detectron2) (0.8.7) Collecting iopath>=0.1.2 Downloading iopath-0.1.6.tar.gz (16 kB) Requirement already satisfied: tqdm>4.29.0 in /opt/conda/lib/python3.7/site-packages (from detectron2) (4.45.0) Requirement already satisfied: portalocker in /opt/conda/lib/python3.7/site-packages (from iopath>=0.1.2->detectron2) (2.0.0) Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.7/site-packages (from matplotlib->detectron2) (1.2.0) Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.7/site-packages (from matplotlib->detectron2) (0.10.0) Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from fvcore<0.1.4,>=0.1.3->detectron2) (1.18.5) Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/lib/python3.7/site-packages (from matplotlib->detectron2) (2.8.1) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /opt/conda/lib/python3.7/site-packages (from matplotlib->detectron2) (2.4.7) Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from cycler>=0.10->matplotlib->detectron2) (1.14.0) Collecting omegaconf>=2 Downloading omegaconf-2.0.6-py3-none-any.whl (36 kB) Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.7/site-packages (from omegaconf>=2->detectron2) (3.7.4.1) Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.7/site-packages (from fvcore<0.1.4,>=0.1.3->detectron2) (5.3.1) Collecting pycocotools>=2.0.2 Downloading pycocotools-2.0.2.tar.gz (23 kB) Requirement already satisfied: setuptools>=18.0 in /opt/conda/lib/python3.7/site-packages (from pycocotools>=2.0.2->detectron2) (46.1.3.post20200325) Requirement already satisfied: cython>=0.27.3 in /opt/conda/lib/python3.7/site-packages (from pycocotools>=2.0.2->detectron2) (0.29.21) Requirement already satisfied: matplotlib in /opt/conda/lib/python3.7/site-packages (from detectron2) (3.2.1) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /opt/conda/lib/python3.7/site-packages (from matplotlib->detectron2) (2.4.7) Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from cycler>=0.10->matplotlib->detectron2) (1.14.0) Requirement already satisfied: protobuf>=3.6.0 in /opt/conda/lib/python3.7/site-packages (from tensorboard->detectron2) (3.14.0) Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /opt/conda/lib/python3.7/site-packages (from tensorboard->detectron2) (0.4.1) Requirement already satisfied: werkzeug>=0.11.15 in /opt/conda/lib/python3.7/site-packages (from tensorboard->detectron2) (1.0.1) Requirement already satisfied: markdown>=2.6.8 in /opt/conda/lib/python3.7/site-packages (from tensorboard->detectron2) (3.2.1) Requirement already satisfied: requests<3,>=2.21.0 in /opt/conda/lib/python3.7/site-packages (from tensorboard->detectron2) (2.23.0) Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /opt/conda/lib/python3.7/site-packages (from tensorboard->detectron2) (1.7.0) Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from fvcore<0.1.4,>=0.1.3->detectron2) (1.18.5) Requirement already satisfied: google-auth<2,>=1.6.3 in /opt/conda/lib/python3.7/site-packages (from tensorboard->detectron2) (1.23.0) Requirement already satisfied: grpcio>=1.24.3 in /opt/conda/lib/python3.7/site-packages (from tensorboard->detectron2) (1.34.0) Requirement already satisfied: absl-py>=0.4 in /opt/conda/lib/python3.7/site-packages (from tensorboard->detectron2) (0.10.0) Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from cycler>=0.10->matplotlib->detectron2) (1.14.0) Requirement already satisfied: setuptools>=18.0 in /opt/conda/lib/python3.7/site-packages (from pycocotools>=2.0.2->detectron2) (46.1.3.post20200325) Requirement already satisfied: wheel>=0.26 in /opt/conda/lib/python3.7/site-packages (from tensorboard->detectron2) (0.34.2) Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from cycler>=0.10->matplotlib->detectron2) (1.14.0) Requirement already satisfied: cachetools<5.0,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard->detectron2) (3.1.1) Requirement already satisfied: rsa<5,>=3.1.4 in /opt/conda/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard->detectron2) (4.0) Requirement already satisfied: setuptools>=18.0 in /opt/conda/lib/python3.7/site-packages (from pycocotools>=2.0.2->detectron2) (46.1.3.post20200325) Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from cycler>=0.10->matplotlib->detectron2) (1.14.0) Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard->detectron2) (0.2.7) Requirement already satisfied: google-auth<2,>=1.6.3 in /opt/conda/lib/python3.7/site-packages (from tensorboard->detectron2) (1.23.0) Requirement already satisfied: requests-oauthlib>=0.7.0 in /opt/conda/lib/python3.7/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard->detectron2) (1.2.0) Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from cycler>=0.10->matplotlib->detectron2) (1.14.0) Requirement already satisfied: setuptools>=18.0 in /opt/conda/lib/python3.7/site-packages (from pycocotools>=2.0.2->detectron2) (46.1.3.post20200325) Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from cycler>=0.10->matplotlib->detectron2) (1.14.0) Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /opt/conda/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard->detectron2) (0.4.8) Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard->detectron2) (2020.12.5) Requirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard->detectron2) (2.9) Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard->detectron2) (3.0.4) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard->detectron2) (1.25.9) Requirement already satisfied: oauthlib>=3.0.0 in /opt/conda/lib/python3.7/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard->detectron2) (3.0.1) Requirement already satisfied: requests<3,>=2.21.0 in /opt/conda/lib/python3.7/site-packages (from tensorboard->detectron2) (2.23.0) Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /opt/conda/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard->detectron2) (0.4.8) Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.7/site-packages (from fvcore<0.1.4,>=0.1.3->detectron2) (5.3.1) Building wheels for collected packages: fvcore, iopath, pycocotools Building wheel for fvcore (setup.py) ... [?25l- \ done [?25h Created wheel for fvcore: filename=fvcore-0.1.3.post20210317-py3-none-any.whl size=58540 sha256=92acdf60118715ef0df29c40cfb80231eeb10b806bbae9b4405db1b006363d09 Stored in directory: /root/.cache/pip/wheels/a6/02/09/10e3a0150eb92e5ecbee3677a813bffc32a8ec6f876bfe4adf Building wheel for iopath (setup.py) ... [?25l- done [?25h Created wheel for iopath: filename=iopath-0.1.6-py3-none-any.whl size=18268 sha256=ddb8af38f87ee3f41a4cb561b2ec827ddb53ba8f1e90f755649758ea5262b3fe Stored in directory: /root/.cache/pip/wheels/f4/07/64/6aceaa162e955df9c8b6f7aa432b34cab88ef669e7883a632f Building wheel for pycocotools (setup.py) ... [?25l- \ | / - \ | / done [?25h Created wheel for pycocotools: filename=pycocotools-2.0.2-cp37-cp37m-linux_x86_64.whl size=273761 sha256=f0681ceb4e565d4ea883c74deb80bffd4748e3e3c64e70b4f5be3c851d3fb7bb Stored in directory: /root/.cache/pip/wheels/bc/cf/1b/e95c99c5f9d1648be3f500ca55e7ce55f24818b0f48336adaf Successfully built fvcore iopath pycocotools Installing collected packages: iopath, pycocotools, omegaconf, fvcore, detectron2 Successfully installed detectron2-0.4+cu102 fvcore-0.1.3.post20210317 iopath-0.1.6 omegaconf-2.0.6 pycocotools-2.0.2 [33mWARNING: You are using pip version 20.3.1; however, version 21.0.1 is available. You should consider upgrading via the '/opt/conda/bin/python3.7 -m pip install --upgrade pip' command.[0m Collecting pytorch-pfn-extras Downloading pytorch-pfn-extras-0.3.2.tar.gz (94 kB) [K |████████████████████████████████| 94 kB 946 kB/s [?25hRequirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from pytorch-pfn-extras) (1.18.5) Requirement already satisfied: torch in /opt/conda/lib/python3.7/site-packages (from pytorch-pfn-extras) (1.7.0) Collecting timm Downloading timm-0.4.5-py3-none-any.whl (287 kB) [K |████████████████████████████████| 287 kB 4.3 MB/s [?25hRequirement already satisfied: torch in /opt/conda/lib/python3.7/site-packages (from pytorch-pfn-extras) (1.7.0) Requirement already satisfied: torchvision in /opt/conda/lib/python3.7/site-packages (from timm) (0.8.1) Requirement already satisfied: future in /opt/conda/lib/python3.7/site-packages (from torch->pytorch-pfn-extras) (0.18.2) Requirement already satisfied: typing_extensions in /opt/conda/lib/python3.7/site-packages (from torch->pytorch-pfn-extras) (3.7.4.1) Requirement already satisfied: dataclasses in /opt/conda/lib/python3.7/site-packages (from torch->pytorch-pfn-extras) (0.6) Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from pytorch-pfn-extras) (1.18.5) Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from pytorch-pfn-extras) (1.18.5) Requirement already satisfied: torch in /opt/conda/lib/python3.7/site-packages (from pytorch-pfn-extras) (1.7.0) Requirement already satisfied: pillow>=4.1.1 in /opt/conda/lib/python3.7/site-packages (from torchvision->timm) (8.0.1) Building wheels for collected packages: pytorch-pfn-extras Building wheel for pytorch-pfn-extras (setup.py) ... [?25l- \ done [?25h Created wheel for pytorch-pfn-extras: filename=pytorch_pfn_extras-0.3.2-py3-none-any.whl size=104301 sha256=8e2827a42bc2c505879ef24a8b52d2ffc2d02d32afa9312c8738af0527ef2860 Stored in directory: /root/.cache/pip/wheels/86/97/78/8eff42f17b564da55e7b81ec3f273ff4618ca143b8b646f25d Successfully built pytorch-pfn-extras Installing collected packages: timm, pytorch-pfn-extras Successfully installed pytorch-pfn-extras-0.3.2 timm-0.4.5 [33mWARNING: You are using pip version 20.3.1; however, version 21.0.1 is available. You should consider upgrading via the '/opt/conda/bin/python3.7 -m pip install --upgrade pip' command.[0m
In [3]:
from typing import Any
import yaml
def save_yaml(filepath: str, content: Any, width: int = 120):
with open(filepath, "w") as f:
yaml.dump(content, f, width=width)In [4]:
from dataclasses import dataclass, field
from typing import Dict, Any, Tuple, Union, List
@dataclass
class Flags:
# General
debug: bool = True
outdir: str = "results/det"
device: str = "cuda:0"
# Data config
imgdir_name: str = "vinbigdata-chest-xray-resized-png-256x256"
# split_mode: str = "all_train" # all_train or valid20
seed: int = 111
target_fold: int = 0 # 0~4
label_smoothing: float = 0.0
# Model config
model_name: str = "resnet18"
model_mode: str = "normal" # normal, cnn_fixed supported
# Training config
epoch: int = 20
batchsize: int = 8
valid_batchsize: int = 16
num_workers: int = 4
snapshot_freq: int = 5
ema_decay: float = 0.999 # negative value is to inactivate ema.
scheduler_type: str = ""
scheduler_kwargs: Dict[str, Any] = field(default_factory=lambda: {})
scheduler_trigger: List[Union[int, str]] = field(default_factory=lambda: [1, "iteration"])
aug_kwargs: Dict[str, Dict[str, Any]] = field(default_factory=lambda: {})
mixup_prob: float = -1.0 # Apply mixup augmentation when positive value is set.
def update(self, param_dict: Dict) -> "Flags":
# Overwrite by `param_dict`
for key, value in param_dict.items():
if not hasattr(self, key):
raise ValueError(f"[ERROR] Unexpected key for flag = {key}")
setattr(self, key, value)
return selfIn [5]:
flags_dict = {
"debug": False, # Change to True for fast debug run!
"outdir": "results/tmp_debug",
# Data
"imgdir_name": "vinbigdata-chest-xray-resized-png-256x256",
# Model
"model_name": "resnet18",
# Training
"num_workers": 4,
"epoch": 15,
"batchsize": 8,
"scheduler_type": "CosineAnnealingWarmRestarts",
"scheduler_kwargs": {"T_0": 28125}, # 15000 * 15 epoch // (batchsize=8)
"scheduler_trigger": [1, "iteration"],
"aug_kwargs": {
"HorizontalFlip": {"p": 0.5},
"ShiftScaleRotate": {"scale_limit": 0.15, "rotate_limit": 10, "p": 0.5},
"RandomBrightnessContrast": {"p": 0.5},
"CoarseDropout": {"max_holes": 8, "max_height": 25, "max_width": 25, "p": 0.5},
"Blur": {"blur_limit": [3, 7], "p": 0.5},
"Downscale": {"scale_min": 0.25, "scale_max": 0.9, "p": 0.3},
"RandomGamma": {"gamma_limit": [80, 120], "p": 0.6},
}
}In [6]:
import dataclasses
# args = parse()
print("torch", torch.__version__)
flags = Flags().update(flags_dict)
print("flags", flags)
debug = flags.debug
outdir = Path(flags.outdir)
os.makedirs(str(outdir), exist_ok=True)
flags_dict = dataclasses.asdict(flags)
save_yaml(str(outdir / "flags.yaml"), flags_dict)
# --- Read data ---
inputdir = Path("/kaggle/input")
datadir = inputdir / "vinbigdata-chest-xray-abnormalities-detection"
imgdir = inputdir / flags.imgdir_name
# Read in the data CSV files
train = pd.read_csv(datadir / "train.csv")
# sample_submission = pd.read_csv(datadir / 'sample_submission.csv')torch 1.7.0
flags Flags(debug=False, outdir='results/tmp_debug', device='cuda:0', imgdir_name='vinbigdata-chest-xray-resized-png-256x256', seed=111, target_fold=0, label_smoothing=0.0, model_name='resnet18', model_mode='normal', epoch=15, batchsize=8, valid_batchsize=16, num_workers=4, snapshot_freq=5, ema_decay=0.999, scheduler_type='CosineAnnealingWarmRestarts', scheduler_kwargs={'T_0': 28125}, scheduler_trigger=[1, 'iteration'], aug_kwargs={'HorizontalFlip': {'p': 0.5}, 'ShiftScaleRotate': {'scale_limit': 0.15, 'rotate_limit': 10, 'p': 0.5}, 'RandomBrightnessContrast': {'p': 0.5}, 'CoarseDropout': {'max_holes': 8, 'max_height': 25, 'max_width': 25, 'p': 0.5}, 'Blur': {'blur_limit': [3, 7], 'p': 0.5}, 'Downscale': {'scale_min': 0.25, 'scale_max': 0.9, 'p': 0.3}, 'RandomGamma': {'gamma_limit': [80, 120], 'p': 0.6}}, mixup_prob=-1.0)
In [7]:
train.query("image_id == '50a418190bc3fb1ef1633bf9678929b3'")Out [7]:
| image_id | class_name | class_id | rad_id | x_min | y_min | x_max | y_max | |
|---|---|---|---|---|---|---|---|---|
| 0 | 50a418190bc3fb1ef1633bf9678929b3 | No finding | 14 | R11 | NaN | NaN | NaN | NaN |
| 45863 | 50a418190bc3fb1ef1633bf9678929b3 | No finding | 14 | R15 | NaN | NaN | NaN | NaN |
| 57424 | 50a418190bc3fb1ef1633bf9678929b3 | No finding | 14 | R16 | NaN | NaN | NaN | NaN |
In [8]:
is_normal_df = train.groupby("image_id")["class_id"].agg(lambda s: (s == 14).sum()).reset_index().rename({"class_id": "num_normal_annotations"}, axis=1)
is_normal_df.head()Out [8]:
| image_id | num_normal_annotations | |
|---|---|---|
| 0 | 000434271f63a053c4128a0ba6352c7f | 3 |
| 1 | 00053190460d56c53cc3e57321387478 | 3 |
| 2 | 0005e8e3701dfb1dd93d53e2ff537b6e | 0 |
| 3 | 0006e0a85696f6bb578e84fafa9a5607 | 3 |
| 4 | 0007d316f756b3fa0baea2ff514ce945 | 0 |
In [9]:
num_normal_anno_counts = is_normal_df["num_normal_annotations"].value_counts()
num_normal_anno_counts.plot(kind="bar")
plt.title("The number of 'No finding' annotations in each image")Out [9]:
Text(0.5, 1.0, "The number of 'No finding' annotations in each image")
In [10]:
num_normal_anno_counts_df = num_normal_anno_counts.reset_index()
num_normal_anno_counts_df["name"] = num_normal_anno_counts_df["index"].map({0: "Abnormal", 3: "Normal"})
num_normal_anno_counts_dfOut [10]:
| index | num_normal_annotations | name | |
|---|---|---|---|
| 0 | 3 | 10606 | Normal |
| 1 | 0 | 4394 | Abnormal |
In [11]:
px.pie(num_normal_anno_counts_df, values="num_normal_annotations", names="name", title="Normal/Abnormal ratio")In [12]:
import pickle
from pathlib import Path
from typing import Optional
import cv2
import numpy as np
import pandas as pd
from detectron2.structures import BoxMode
from tqdm import tqdm
def get_vinbigdata_dicts(
imgdir: Path,
train_df: pd.DataFrame,
train_data_type: str = "original",
use_cache: bool = True,
debug: bool = True,
target_indices: Optional[np.ndarray] = None,
):
debug_str = f"_debug{int(debug)}"
train_data_type_str = f"_{train_data_type}"
cache_path = Path(".") / f"dataset_dicts_cache{train_data_type_str}{debug_str}.pkl"
if not use_cache or not cache_path.exists():
print("Creating data...")
train_meta = pd.read_csv(imgdir / "train_meta.csv")
if debug:
train_meta = train_meta.iloc[:500] # For debug....
# Load 1 image to get image size.
image_id = train_meta.loc[0, "image_id"]
image_path = str(imgdir / "train" / f"{image_id}.png")
image = cv2.imread(image_path)
resized_height, resized_width, ch = image.shape
print(f"image shape: {image.shape}")
dataset_dicts = []
for index, train_meta_row in tqdm(train_meta.iterrows(), total=len(train_meta)):
record = {}
image_id, height, width = train_meta_row.values
filename = str(imgdir / "train" / f"{image_id}.png")
record["file_name"] = filename
record["image_id"] = image_id
record["height"] = resized_height
record["width"] = resized_width
objs = []
for index2, row in train_df.query("image_id == @image_id").iterrows():
# print(row)
# print(row["class_name"])
# class_name = row["class_name"]
class_id = row["class_id"]
if class_id == 14:
# It is "No finding"
# This annotator does not find anything, skip.
pass
else:
# bbox_original = [int(row["x_min"]), int(row["y_min"]), int(row["x_max"]), int(row["y_max"])]
h_ratio = resized_height / height
w_ratio = resized_width / width
bbox_resized = [
int(row["x_min"]) * w_ratio,
int(row["y_min"]) * h_ratio,
int(row["x_max"]) * w_ratio,
int(row["y_max"]) * h_ratio,
]
obj = {
"bbox": bbox_resized,
"bbox_mode": BoxMode.XYXY_ABS,
"category_id": class_id,
}
objs.append(obj)
record["annotations"] = objs
dataset_dicts.append(record)
with open(cache_path, mode="wb") as f:
pickle.dump(dataset_dicts, f)
print(f"Load from cache {cache_path}")
with open(cache_path, mode="rb") as f:
dataset_dicts = pickle.load(f)
if target_indices is not None:
dataset_dicts = [dataset_dicts[i] for i in target_indices]
return dataset_dicts
def get_vinbigdata_dicts_test(
imgdir: Path, test_meta: pd.DataFrame, use_cache: bool = True, debug: bool = True,
):
debug_str = f"_debug{int(debug)}"
cache_path = Path(".") / f"dataset_dicts_cache_test{debug_str}.pkl"
if not use_cache or not cache_path.exists():
print("Creating data...")
# test_meta = pd.read_csv(imgdir / "test_meta.csv")
if debug:
test_meta = test_meta.iloc[:500] # For debug....
# Load 1 image to get image size.
image_id = test_meta.loc[0, "image_id"]
image_path = str(imgdir / "test" / f"{image_id}.png")
image = cv2.imread(image_path)
resized_height, resized_width, ch = image.shape
print(f"image shape: {image.shape}")
dataset_dicts = []
for index, test_meta_row in tqdm(test_meta.iterrows(), total=len(test_meta)):
record = {}
image_id, height, width = test_meta_row.values
filename = str(imgdir / "test" / f"{image_id}.png")
record["file_name"] = filename
# record["image_id"] = index
record["image_id"] = image_id
record["height"] = resized_height
record["width"] = resized_width
# objs = []
# record["annotations"] = objs
dataset_dicts.append(record)
with open(cache_path, mode="wb") as f:
pickle.dump(dataset_dicts, f)
print(f"Load from cache {cache_path}")
with open(cache_path, mode="rb") as f:
dataset_dicts = pickle.load(f)
return dataset_dicts
In [13]:
"""
Referenced `chainer.dataset.DatasetMixin` to work with pytorch Dataset.
"""
import numpy
import six
import torch
from torch.utils.data.dataset import Dataset
class DatasetMixin(Dataset):
def __init__(self, transform=None):
self.transform = transform
def __getitem__(self, index):
"""Returns an example or a sequence of examples."""
if torch.is_tensor(index):
index = index.tolist()
if isinstance(index, slice):
current, stop, step = index.indices(len(self))
return [self.get_example_wrapper(i) for i in
six.moves.range(current, stop, step)]
elif isinstance(index, list) or isinstance(index, numpy.ndarray):
return [self.get_example_wrapper(i) for i in index]
else:
return self.get_example_wrapper(index)
def __len__(self):
"""Returns the number of data points."""
raise NotImplementedError
def get_example_wrapper(self, i):
"""Wrapper of `get_example`, to apply `transform` if necessary"""
example = self.get_example(i)
if self.transform:
example = self.transform(example)
return example
def get_example(self, i):
"""Returns the i-th example.
Implementations should override it. It should raise :class:`IndexError`
if the index is invalid.
Args:
i (int): The index of the example.
Returns:
The i-th example.
"""
raise NotImplementedError
In [14]:
import cv2
import numpy as np
class VinbigdataTwoClassDataset(DatasetMixin):
def __init__(self, dataset_dicts, image_transform=None, transform=None, train: bool = True,
mixup_prob: float = -1.0, label_smoothing: float = 0.0):
super(VinbigdataTwoClassDataset, self).__init__(transform=transform)
self.dataset_dicts = dataset_dicts
self.image_transform = image_transform
self.train = train
self.mixup_prob = mixup_prob
self.label_smoothing = label_smoothing
def _get_single_example(self, i):
d = self.dataset_dicts[i]
filename = d["file_name"]
img = cv2.imread(filename)
if self.image_transform:
img = self.image_transform(img)
img = torch.tensor(np.transpose(img, (2, 0, 1)).astype(np.float32))
if self.train:
label = int(len(d["annotations"]) > 0) # 0 normal, 1 abnormal
if self.label_smoothing > 0:
if label == 0:
return img, float(label) + self.label_smoothing
else:
return img, float(label) - self.label_smoothing
else:
return img, float(label)
else:
# Only return img
return img, None
def get_example(self, i):
img, label = self._get_single_example(i)
if self.mixup_prob > 0. and np.random.uniform() < self.mixup_prob:
j = np.random.randint(0, len(self.dataset_dicts))
p = np.random.uniform()
img2, label2 = self._get_single_example(j)
img = img * p + img2 * (1 - p)
if self.train:
label = label * p + label2 * (1 - p)
if self.train:
label_logit = torch.tensor([1 - label, label], dtype=torch.float32)
return img, label_logit
else:
# Only return img
return img
def __len__(self):
return len(self.dataset_dicts)In [15]:
dataset_dicts = get_vinbigdata_dicts(imgdir, train, debug=debug)
dataset = VinbigdataTwoClassDataset(dataset_dicts)0%| | 20/15000 [00:00<01:16, 195.90it/s]
Creating data... image shape: (256, 256, 3)
100%|██████████| 15000/15000 [01:14<00:00, 201.10it/s]
Load from cache dataset_dicts_cache_original_debug0.pkl
In [16]:
index = 0
img, label = dataset[index]
plt.imshow(img.cpu().numpy().transpose((1, 2, 0)) / 255.)
plt.title(f"{index}-th image: label {label}")Out [16]:
Text(0.5, 1.0, '0-th image: label tensor([1., 0.])')
In [17]:
import albumentations as A
class Transform:
def __init__(
self, hflip_prob: float = 0.5, ssr_prob: float = 0.5, random_bc_prob: float = 0.5
):
self.transform = A.Compose(
[
A.HorizontalFlip(p=hflip_prob),
A.ShiftScaleRotate(
shift_limit=0.0625, scale_limit=0.1, rotate_limit=10, p=ssr_prob
),
A.RandomBrightnessContrast(p=random_bc_prob),
]
)
def __call__(self, image):
image = self.transform(image=image)["image"]
return image
In [18]:
aug_dataset = VinbigdataTwoClassDataset(dataset_dicts, image_transform=Transform())In [19]:
index = 0
n_images = 4
fig, axes = plt.subplots(1, n_images, figsize=(16, 5))
for i in range(n_images):
# Each time the data is accessed, the result is different due to random augmentation!
img, label = aug_dataset[index]
ax = axes[i]
ax.imshow(img.cpu().numpy().transpose((1, 2, 0)) / 255.)
ax.set_title(f"{index}-th image: label {label}")
plt.show()
In [20]:
from typing import Dict
import albumentations as A
class Transform:
def __init__(self, aug_kwargs: Dict):
self.transform = A.Compose(
[getattr(A, name)(**kwargs) for name, kwargs in aug_kwargs.items()]
)
def __call__(self, image):
image = self.transform(image=image)["image"]
return imageIn [21]:
from torch import nn
from torch.nn import Linear
class CNNFixedPredictor(nn.Module):
def __init__(self, cnn: nn.Module, num_classes: int = 2):
super(CNNFixedPredictor, self).__init__()
self.cnn = cnn
self.lin = Linear(cnn.num_features, num_classes)
print("cnn.num_features", cnn.num_features)
# We do not learn CNN parameters.
# https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html
for param in self.cnn.parameters():
param.requires_grad = False
def forward(self, x):
feat = self.cnn(x)
return self.lin(feat)
In [22]:
import timm
def build_predictor(model_name: str, model_mode: str = "normal"):
if model_mode == "normal":
# normal configuration. train all parameters.
return timm.create_model(model_name, pretrained=True, num_classes=2, in_chans=3)
elif model_mode == "cnn_fixed":
# normal configuration. train all parameters.
# https://rwightman.github.io/pytorch-image-models/feature_extraction/
timm_model = timm.create_model(model_name, pretrained=True, num_classes=0, in_chans=3)
return CNNFixedPredictor(timm_model, num_classes=2)
else:
raise ValueError(f"[ERROR] Unexpected value model_mode={model_mode}")
In [23]:
import torch
def accuracy(y: torch.Tensor, t: torch.Tensor) -> torch.Tensor:
"""Computes multi-class classification accuracy"""
assert y.shape[:-1] == t.shape, f"y {y.shape}, t {t.shape} is inconsistent."
pred_label = torch.max(y.detach(), dim=-1)[1]
count = t.nelement()
correct = (pred_label == t).sum().float()
acc = correct / count
return acc
def accuracy_with_logits(y: torch.Tensor, t: torch.Tensor) -> torch.Tensor:
"""Computes multi-class classification accuracy"""
assert y.shape == t.shape
gt_label = torch.max(t.detach(), dim=-1)[1]
return accuracy(y, gt_label)In [24]:
import torch
import torch.nn.functional as F
def cross_entropy_with_logits(input, target, dim=-1):
loss = torch.sum(- target * F.log_softmax(input, dim), dim)
return loss.mean()
In [25]:
import torch
import torch.nn.functional as F
from torch import nn
import pytorch_pfn_extras as ppe
class Classifier(nn.Module):
"""two class classfication"""
def __init__(self, predictor, lossfun=cross_entropy_with_logits):
super().__init__()
self.predictor = predictor
self.lossfun = lossfun
self.prefix = ""
def forward(self, image, targets):
outputs = self.predictor(image)
loss = self.lossfun(outputs, targets)
metrics = {
f"{self.prefix}loss": loss.item(),
f"{self.prefix}acc": accuracy_with_logits(outputs, targets).item()
}
ppe.reporting.report(metrics, self)
return loss, metrics
def predict(self, data_loader):
pred = self.predict_proba(data_loader)
label = torch.argmax(pred, dim=1)
return label
def predict_proba(self, data_loader):
device: torch.device = next(self.parameters()).device
y_list = []
self.eval()
with torch.no_grad():
for batch in data_loader:
if isinstance(batch, (tuple, list)):
# Assumes first argument is "image"
batch = batch[0].to(device)
else:
batch = batch.to(device)
y = self.predictor(batch)
y = torch.softmax(y, dim=-1)
y_list.append(y)
pred = torch.cat(y_list)
return pred
In [26]:
supported_models = timm.list_models()
print(f"{len(supported_models)} models are supported in timm.")
print(supported_models)434 models are supported in timm. ['adv_inception_v3', 'cspdarknet53', 'cspdarknet53_iabn', 'cspresnet50', 'cspresnet50d', 'cspresnet50w', 'cspresnext50', 'cspresnext50_iabn', 'darknet53', 'densenet121', 'densenet121d', 'densenet161', 'densenet169', 'densenet201', 'densenet264', 'densenet264d_iabn', 'densenetblur121d', 'dla34', 'dla46_c', 'dla46x_c', 'dla60', 'dla60_res2net', 'dla60_res2next', 'dla60x', 'dla60x_c', 'dla102', 'dla102x', 'dla102x2', 'dla169', 'dm_nfnet_f0', 'dm_nfnet_f1', 'dm_nfnet_f2', 'dm_nfnet_f3', 'dm_nfnet_f4', 'dm_nfnet_f5', 'dm_nfnet_f6', 'dpn68', 'dpn68b', 'dpn92', 'dpn98', 'dpn107', 'dpn131', 'eca_vovnet39b', 'ecaresnet26t', 'ecaresnet50d', 'ecaresnet50d_pruned', 'ecaresnet50t', 'ecaresnet101d', 'ecaresnet101d_pruned', 'ecaresnet200d', 'ecaresnet269d', 'ecaresnetlight', 'ecaresnext26t_32x4d', 'ecaresnext50t_32x4d', 'efficientnet_b0', 'efficientnet_b1', 'efficientnet_b1_pruned', 'efficientnet_b2', 'efficientnet_b2_pruned', 'efficientnet_b2a', 'efficientnet_b3', 'efficientnet_b3_pruned', 'efficientnet_b3a', 'efficientnet_b4', 'efficientnet_b5', 'efficientnet_b6', 'efficientnet_b7', 'efficientnet_b8', 'efficientnet_cc_b0_4e', 'efficientnet_cc_b0_8e', 'efficientnet_cc_b1_8e', 'efficientnet_el', 'efficientnet_em', 'efficientnet_es', 'efficientnet_l2', 'efficientnet_lite0', 'efficientnet_lite1', 'efficientnet_lite2', 'efficientnet_lite3', 'efficientnet_lite4', 'ens_adv_inception_resnet_v2', 'ese_vovnet19b_dw', 'ese_vovnet19b_slim', 'ese_vovnet19b_slim_dw', 'ese_vovnet39b', 'ese_vovnet39b_evos', 'ese_vovnet57b', 'ese_vovnet99b', 'ese_vovnet99b_iabn', 'fbnetc_100', 'gernet_l', 'gernet_m', 'gernet_s', 'gluon_inception_v3', 'gluon_resnet18_v1b', 'gluon_resnet34_v1b', 'gluon_resnet50_v1b', 'gluon_resnet50_v1c', 'gluon_resnet50_v1d', 'gluon_resnet50_v1s', 'gluon_resnet101_v1b', 'gluon_resnet101_v1c', 'gluon_resnet101_v1d', 'gluon_resnet101_v1s', 'gluon_resnet152_v1b', 'gluon_resnet152_v1c', 'gluon_resnet152_v1d', 'gluon_resnet152_v1s', 'gluon_resnext50_32x4d', 'gluon_resnext101_32x4d', 'gluon_resnext101_64x4d', 'gluon_senet154', 'gluon_seresnext50_32x4d', 'gluon_seresnext101_32x4d', 'gluon_seresnext101_64x4d', 'gluon_xception65', 'hrnet_w18', 'hrnet_w18_small', 'hrnet_w18_small_v2', 'hrnet_w30', 'hrnet_w32', 'hrnet_w40', 'hrnet_w44', 'hrnet_w48', 'hrnet_w64', 'ig_resnext101_32x8d', 'ig_resnext101_32x16d', 'ig_resnext101_32x32d', 'ig_resnext101_32x48d', 'inception_resnet_v2', 'inception_v3', 'inception_v4', 'legacy_senet154', 'legacy_seresnet18', 'legacy_seresnet34', 'legacy_seresnet50', 'legacy_seresnet101', 'legacy_seresnet152', 'legacy_seresnext26_32x4d', 'legacy_seresnext50_32x4d', 'legacy_seresnext101_32x4d', 'mixnet_l', 'mixnet_m', 'mixnet_s', 'mixnet_xl', 'mixnet_xxl', 'mnasnet_050', 'mnasnet_075', 'mnasnet_100', 'mnasnet_140', 'mnasnet_a1', 'mnasnet_b1', 'mnasnet_small', 'mobilenetv2_100', 'mobilenetv2_110d', 'mobilenetv2_120d', 'mobilenetv2_140', 'mobilenetv3_large_075', 'mobilenetv3_large_100', 'mobilenetv3_rw', 'mobilenetv3_small_075', 'mobilenetv3_small_100', 'nasnetalarge', 'nf_ecaresnet26', 'nf_ecaresnet50', 'nf_ecaresnet101', 'nf_regnet_b0', 'nf_regnet_b1', 'nf_regnet_b2', 'nf_regnet_b3', 'nf_regnet_b4', 'nf_regnet_b5', 'nf_resnet26', 'nf_resnet50', 'nf_resnet101', 'nf_seresnet26', 'nf_seresnet50', 'nf_seresnet101', 'nfnet_f0', 'nfnet_f0s', 'nfnet_f1', 'nfnet_f1s', 'nfnet_f2', 'nfnet_f2s', 'nfnet_f3', 'nfnet_f3s', 'nfnet_f4', 'nfnet_f4s', 'nfnet_f5', 'nfnet_f5s', 'nfnet_f6', 'nfnet_f6s', 'nfnet_f7', 'nfnet_f7s', 'nfnet_l0a', 'nfnet_l0b', 'nfnet_l0c', 'pnasnet5large', 'regnetx_002', 'regnetx_004', 'regnetx_006', 'regnetx_008', 'regnetx_016', 'regnetx_032', 'regnetx_040', 'regnetx_064', 'regnetx_080', 'regnetx_120', 'regnetx_160', 'regnetx_320', 'regnety_002', 'regnety_004', 'regnety_006', 'regnety_008', 'regnety_016', 'regnety_032', 'regnety_040', 'regnety_064', 'regnety_080', 'regnety_120', 'regnety_160', 'regnety_320', 'repvgg_a2', 'repvgg_b0', 'repvgg_b1', 'repvgg_b1g4', 'repvgg_b2', 'repvgg_b2g4', 'repvgg_b3', 'repvgg_b3g4', 'res2net50_14w_8s', 'res2net50_26w_4s', 'res2net50_26w_6s', 'res2net50_26w_8s', 'res2net50_48w_2s', 'res2net101_26w_4s', 'res2next50', 'resnest14d', 'resnest26d', 'resnest50d', 'resnest50d_1s4x24d', 'resnest50d_4s2x40d', 'resnest101e', 'resnest200e', 'resnest269e', 'resnet18', 'resnet18d', 'resnet26', 'resnet26d', 'resnet34', 'resnet34d', 'resnet50', 'resnet50d', 'resnet101', 'resnet101d', 'resnet152', 'resnet152d', 'resnet200', 'resnet200d', 'resnetblur18', 'resnetblur50', 'resnetv2_50x1_bitm', 'resnetv2_50x1_bitm_in21k', 'resnetv2_50x3_bitm', 'resnetv2_50x3_bitm_in21k', 'resnetv2_101x1_bitm', 'resnetv2_101x1_bitm_in21k', 'resnetv2_101x3_bitm', 'resnetv2_101x3_bitm_in21k', 'resnetv2_152x2_bitm', 'resnetv2_152x2_bitm_in21k', 'resnetv2_152x4_bitm', 'resnetv2_152x4_bitm_in21k', 'resnext50_32x4d', 'resnext50d_32x4d', 'resnext101_32x4d', 'resnext101_32x8d', 'resnext101_64x4d', 'rexnet_100', 'rexnet_130', 'rexnet_150', 'rexnet_200', 'rexnetr_100', 'rexnetr_130', 'rexnetr_150', 'rexnetr_200', 'selecsls42', 'selecsls42b', 'selecsls60', 'selecsls60b', 'selecsls84', 'semnasnet_050', 'semnasnet_075', 'semnasnet_100', 'semnasnet_140', 'senet154', 'seresnet18', 'seresnet34', 'seresnet50', 'seresnet50t', 'seresnet101', 'seresnet152', 'seresnet152d', 'seresnet200d', 'seresnet269d', 'seresnext26d_32x4d', 'seresnext26t_32x4d', 'seresnext26tn_32x4d', 'seresnext50_32x4d', 'seresnext101_32x4d', 'seresnext101_32x8d', 'skresnet18', 'skresnet34', 'skresnet50', 'skresnet50d', 'skresnext50_32x4d', 'spnasnet_100', 'ssl_resnet18', 'ssl_resnet50', 'ssl_resnext50_32x4d', 'ssl_resnext101_32x4d', 'ssl_resnext101_32x8d', 'ssl_resnext101_32x16d', 'swsl_resnet18', 'swsl_resnet50', 'swsl_resnext50_32x4d', 'swsl_resnext101_32x4d', 'swsl_resnext101_32x8d', 'swsl_resnext101_32x16d', 'tf_efficientnet_b0', 'tf_efficientnet_b0_ap', 'tf_efficientnet_b0_ns', 'tf_efficientnet_b1', 'tf_efficientnet_b1_ap', 'tf_efficientnet_b1_ns', 'tf_efficientnet_b2', 'tf_efficientnet_b2_ap', 'tf_efficientnet_b2_ns', 'tf_efficientnet_b3', 'tf_efficientnet_b3_ap', 'tf_efficientnet_b3_ns', 'tf_efficientnet_b4', 'tf_efficientnet_b4_ap', 'tf_efficientnet_b4_ns', 'tf_efficientnet_b5', 'tf_efficientnet_b5_ap', 'tf_efficientnet_b5_ns', 'tf_efficientnet_b6', 'tf_efficientnet_b6_ap', 'tf_efficientnet_b6_ns', 'tf_efficientnet_b7', 'tf_efficientnet_b7_ap', 'tf_efficientnet_b7_ns', 'tf_efficientnet_b8', 'tf_efficientnet_b8_ap', 'tf_efficientnet_cc_b0_4e', 'tf_efficientnet_cc_b0_8e', 'tf_efficientnet_cc_b1_8e', 'tf_efficientnet_el', 'tf_efficientnet_em', 'tf_efficientnet_es', 'tf_efficientnet_l2_ns', 'tf_efficientnet_l2_ns_475', 'tf_efficientnet_lite0', 'tf_efficientnet_lite1', 'tf_efficientnet_lite2', 'tf_efficientnet_lite3', 'tf_efficientnet_lite4', 'tf_inception_v3', 'tf_mixnet_l', 'tf_mixnet_m', 'tf_mixnet_s', 'tf_mobilenetv3_large_075', 'tf_mobilenetv3_large_100', 'tf_mobilenetv3_large_minimal_100', 'tf_mobilenetv3_small_075', 'tf_mobilenetv3_small_100', 'tf_mobilenetv3_small_minimal_100', 'tresnet_l', 'tresnet_l_448', 'tresnet_m', 'tresnet_m_448', 'tresnet_xl', 'tresnet_xl_448', 'tv_densenet121', 'tv_resnet34', 'tv_resnet50', 'tv_resnet101', 'tv_resnet152', 'tv_resnext50_32x4d', 'vgg11', 'vgg11_bn', 'vgg13', 'vgg13_bn', 'vgg16', 'vgg16_bn', 'vgg19', 'vgg19_bn', 'vit_base_patch16_224', 'vit_base_patch16_224_in21k', 'vit_base_patch16_384', 'vit_base_patch32_224', 'vit_base_patch32_224_in21k', 'vit_base_patch32_384', 'vit_base_resnet26d_224', 'vit_base_resnet50_224_in21k', 'vit_base_resnet50_384', 'vit_base_resnet50d_224', 'vit_deit_base_distilled_patch16_224', 'vit_deit_base_distilled_patch16_384', 'vit_deit_base_patch16_224', 'vit_deit_base_patch16_384', 'vit_deit_small_distilled_patch16_224', 'vit_deit_small_patch16_224', 'vit_deit_tiny_distilled_patch16_224', 'vit_deit_tiny_patch16_224', 'vit_huge_patch14_224_in21k', 'vit_large_patch16_224', 'vit_large_patch16_224_in21k', 'vit_large_patch16_384', 'vit_large_patch32_224', 'vit_large_patch32_224_in21k', 'vit_large_patch32_384', 'vit_small_patch16_224', 'vit_small_resnet26d_224', 'vit_small_resnet50d_s3_224', 'vovnet39a', 'vovnet57a', 'wide_resnet50_2', 'wide_resnet101_2', 'xception', 'xception41', 'xception65', 'xception71']
In [27]:
"""
From https://github.com/pfnet-research/kaggle-lyft-motion-prediction-4th-place-solution
"""
from logging import getLogger
from torch import nn
class EMA(object):
"""Exponential moving average of model parameters.
Ref
- https://github.com/tensorflow/addons/blob/v0.10.0/tensorflow_addons/optimizers/moving_average.py#L26-L103
- https://anmoljoshi.com/Pytorch-Dicussions/
Args:
model (nn.Module): Model with parameters whose EMA will be kept.
decay (float): Decay rate for exponential moving average.
strict (bool): Apply strict check for `assign` & `resume`.
use_dynamic_decay (bool): Dynamically change decay rate. If `True`, small decay rate is
used at the beginning of training to move moving average faster.
""" # NOQA
def __init__(
self,
model: nn.Module,
decay: float,
strict: bool = True,
use_dynamic_decay: bool = True,
):
self.decay = decay
self.model = model
self.strict = strict
self.use_dynamic_decay = use_dynamic_decay
self.logger = getLogger(__name__)
self.n_step = 0
self.shadow = {}
self.original = {}
# Flag to manage which parameter is assigned.
# When `False`, original model's parameter is used.
# When `True` (`assign` method is called), `shadow` parameter (ema param) is used.
self._assigned = False
# Register model parameters
for name, param in model.named_parameters():
if param.requires_grad:
self.shadow[name] = param.data.clone()
def step(self):
self.n_step += 1
if self.use_dynamic_decay:
_n_step = float(self.n_step)
decay = min(self.decay, (1.0 + _n_step) / (10.0 + _n_step))
else:
decay = self.decay
for name, param in self.model.named_parameters():
if param.requires_grad:
assert name in self.shadow
new_average = (1.0 - decay) * param.data + decay * self.shadow[name]
self.shadow[name] = new_average.clone()
# alias
__call__ = step
def assign(self):
"""Assign exponential moving average of parameter values to the respective parameters."""
if self._assigned:
if self.strict:
raise ValueError("[ERROR] `assign` is called again before `resume`.")
else:
self.logger.warning(
"`assign` is called again before `resume`."
"shadow parameter is already assigned, skip."
)
return
for name, param in self.model.named_parameters():
if param.requires_grad:
assert name in self.shadow
self.original[name] = param.data.clone()
param.data = self.shadow[name]
self._assigned = True
def resume(self):
"""Restore original parameters to a model.
That is, put back the values that were in each parameter at the last call to `assign`.
"""
if not self._assigned:
if self.strict:
raise ValueError("[ERROR] `resume` is called before `assign`.")
else:
self.logger.warning("`resume` is called before `assign`, skip.")
return
for name, param in self.model.named_parameters():
if param.requires_grad:
assert name in self.shadow
param.data = self.original[name]
self._assigned = False
In [28]:
"""
From https://github.com/pfnet-research/kaggle-lyft-motion-prediction-4th-place-solution
"""
from typing import Mapping, Any
from torch import optim
from pytorch_pfn_extras.training.extension import Extension, PRIORITY_READER
from pytorch_pfn_extras.training.manager import ExtensionsManager
class LRScheduler(Extension):
"""A thin wrapper to resume the lr_scheduler"""
trigger = 1, 'iteration'
priority = PRIORITY_READER
name = None
def __init__(self, optimizer: optim.Optimizer, scheduler_type: str, scheduler_kwargs: Mapping[str, Any]) -> None:
super().__init__()
self.scheduler = getattr(optim.lr_scheduler, scheduler_type)(optimizer, **scheduler_kwargs)
def __call__(self, manager: ExtensionsManager) -> None:
self.scheduler.step()
def state_dict(self) -> None:
return self.scheduler.state_dict()
def load_state_dict(self, to_load) -> None:
self.scheduler.load_state_dict(to_load)
In [29]:
from ignite.engine import Engine
def create_trainer(model, optimizer, device) -> Engine:
model.to(device)
def update_fn(engine, batch):
model.train()
optimizer.zero_grad()
loss, metrics = model(*[elem.to(device) for elem in batch])
loss.backward()
optimizer.step()
return metrics
trainer = Engine(update_fn)
return trainer
In [30]:
import dataclasses
import os
import sys
from pathlib import Path
import numpy as np
import pandas as pd
import pytorch_pfn_extras.training.extensions as E
import torch
from ignite.engine import Events
from pytorch_pfn_extras.training import IgniteExtensionsManager
from sklearn.model_selection import StratifiedKFold
from torch import nn, optim
from torch.utils.data.dataloader import DataLoaderIn [31]:
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=flags.seed)
# skf.get_n_splits(None, None)
y = np.array([int(len(d["annotations"]) > 0) for d in dataset_dicts])
split_inds = list(skf.split(dataset_dicts, y))
train_inds, valid_inds = split_inds[flags.target_fold] # 0th fold
train_dataset = VinbigdataTwoClassDataset(
[dataset_dicts[i] for i in train_inds],
image_transform=Transform(flags.aug_kwargs),
mixup_prob=flags.mixup_prob,
label_smoothing=flags.label_smoothing,
)
valid_dataset = VinbigdataTwoClassDataset([dataset_dicts[i] for i in valid_inds])
In [32]:
train_loader = DataLoader(
train_dataset,
batch_size=flags.batchsize,
num_workers=flags.num_workers,
shuffle=True,
pin_memory=True,
)
valid_loader = DataLoader(
valid_dataset,
batch_size=flags.valid_batchsize,
num_workers=flags.num_workers,
shuffle=False,
pin_memory=True,
)
device = torch.device(flags.device)
predictor = build_predictor(model_name=flags.model_name, model_mode=flags.model_mode)
classifier = Classifier(predictor)
model = classifier
# optimizer = optim.Adam(model.parameters(), lr=1e-3)
optimizer = optim.Adam([param for param in model.parameters() if param.requires_grad], lr=1e-3)
# Train setup
trainer = create_trainer(model, optimizer, device)
ema = EMA(predictor, decay=flags.ema_decay)
def eval_func(*batch):
loss, metrics = model(*[elem.to(device) for elem in batch])
# HACKING: report ema value with prefix.
if flags.ema_decay > 0:
classifier.prefix = "ema_"
ema.assign()
loss, metrics = model(*[elem.to(device) for elem in batch])
ema.resume()
classifier.prefix = ""
valid_evaluator = E.Evaluator(
valid_loader, model, progress_bar=False, eval_func=eval_func, device=device
)
# log_trigger = (10 if debug else 1000, "iteration")
log_trigger = (1, "epoch")
log_report = E.LogReport(trigger=log_trigger)
extensions = [
log_report,
E.ProgressBarNotebook(update_interval=10 if debug else 100), # Show progress bar during training
E.PrintReportNotebook(), # Show "log" on jupyter notebook
# E.ProgressBar(update_interval=10 if debug else 100), # Show progress bar during training
# E.PrintReport(), # Print "log" to terminal
E.FailOnNonNumber(), # Stop training when nan is detected.
]
epoch = flags.epoch
models = {"main": model}
optimizers = {"main": optimizer}
manager = IgniteExtensionsManager(
trainer, models, optimizers, epoch, extensions=extensions, out_dir=str(outdir),
)
# Run evaluation for valid dataset in each epoch.
manager.extend(valid_evaluator)
# Save predictor.pt every epoch
manager.extend(
E.snapshot_object(predictor, "predictor.pt"), trigger=(flags.snapshot_freq, "epoch")
)
# Check & Save best validation predictor.pt every epoch
# manager.extend(E.snapshot_object(predictor, "best_predictor.pt"),
# trigger=MinValueTrigger("validation/module/nll",
# trigger=(flags.snapshot_freq, "iteration")))
# --- lr scheduler ---
if flags.scheduler_type != "":
scheduler_type = flags.scheduler_type
print(f"using {scheduler_type} scheduler with kwargs {flags.scheduler_kwargs}")
manager.extend(
LRScheduler(optimizer, scheduler_type, flags.scheduler_kwargs),
trigger=flags.scheduler_trigger,
)
manager.extend(E.observe_lr(optimizer=optimizer), trigger=log_trigger)
if flags.ema_decay > 0:
# Exponential moving average
manager.extend(lambda manager: ema(), trigger=(1, "iteration"))
def save_ema_model(manager):
ema.assign()
torch.save(predictor.state_dict(), outdir / "predictor_ema.pt")
ema.resume()
manager.extend(save_ema_model, trigger=(flags.snapshot_freq, "epoch"))
_ = trainer.run(train_loader, max_epochs=epoch)Downloading: "https://download.pytorch.org/models/resnet18-5c106cde.pth" to /root/.cache/torch/hub/checkpoints/resnet18-5c106cde.pth
using CosineAnnealingWarmRestarts scheduler with kwargs {'T_0': 28125}
VBox(children=(HBox(children=(FloatProgress(value=0.0, bar_style='info', description='total', max=1.0), HTML(v…
HTML(value='')
In [33]:
torch.save(predictor.state_dict(), outdir / "predictor_last.pt")
df = log_report.to_dataframe()
df.to_csv(outdir / "log.csv", index=False)
dfOut [33]:
| main/loss | main/acc | validation/main/loss | validation/main/acc | validation/main/ema_loss | validation/main/ema_acc | lr | epoch | iteration | elapsed_time | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0.528381 | 0.747083 | 0.588147 | 0.718750 | 0.393282 | 0.816157 | 0.000993 | 1 | 1500 | 114.799011 |
| 1 | 0.412800 | 0.817500 | 0.347529 | 0.856051 | 0.615470 | 0.632314 | 0.000972 | 2 | 3000 | 221.793034 |
| 2 | 0.370459 | 0.842083 | 0.407633 | 0.845412 | 0.320290 | 0.881981 | 0.000938 | 3 | 4500 | 332.510463 |
| 3 | 0.351097 | 0.850917 | 0.336031 | 0.854056 | 0.290663 | 0.896941 | 0.000892 | 4 | 6000 | 440.133601 |
| 4 | 0.321259 | 0.867333 | 0.239500 | 0.898936 | 0.378596 | 0.865691 | 0.000835 | 5 | 7500 | 549.527412 |
| 5 | 0.302923 | 0.875250 | 0.297767 | 0.865027 | 0.312154 | 0.874668 | 0.000768 | 6 | 9000 | 657.854909 |
| 6 | 0.285813 | 0.883583 | 0.223919 | 0.913896 | 0.250231 | 0.907247 | 0.000694 | 7 | 10500 | 767.562975 |
| 7 | 0.265541 | 0.892667 | 0.357997 | 0.856715 | 0.296358 | 0.874668 | 0.000614 | 8 | 12000 | 879.002276 |
| 8 | 0.253445 | 0.898167 | 0.211335 | 0.916888 | 0.211629 | 0.918218 | 0.000531 | 9 | 13500 | 991.472235 |
| 9 | 0.236711 | 0.902667 | 0.197833 | 0.923205 | 0.271394 | 0.885306 | 0.000448 | 10 | 15000 | 1104.456698 |
| 10 | 0.222535 | 0.908833 | 0.199566 | 0.920545 | 0.199515 | 0.922540 | 0.000366 | 11 | 16500 | 1217.035449 |
| 11 | 0.207314 | 0.916833 | 0.182642 | 0.932181 | 0.189772 | 0.927194 | 0.000287 | 12 | 18000 | 1329.338816 |
| 12 | 0.194512 | 0.921167 | 0.174983 | 0.938165 | 0.195822 | 0.924202 | 0.000215 | 13 | 19500 | 1442.329682 |
| 13 | 0.187734 | 0.923667 | 0.169579 | 0.934840 | 0.172055 | 0.933511 | 0.000150 | 14 | 21000 | 1554.886963 |
| 14 | 0.173698 | 0.929917 | 0.167541 | 0.937500 | 0.173820 | 0.934508 | 0.000096 | 15 | 22500 | 1666.066400 |
In [34]:
# --- Prediction ---
print("Training done! Start prediction...")
# valid data
valid_pred = classifier.predict_proba(valid_loader).cpu().numpy()
valid_pred_df = pd.DataFrame({
"image_id": [dataset_dicts[i]["image_id"] for i in valid_inds],
"class0": valid_pred[:, 0],
"class1": valid_pred[:, 1]
})
valid_pred_df.to_csv(outdir/"valid_pred.csv", index=False)
# test data
test_meta = pd.read_csv(inputdir / "vinbigdata-testmeta" / "test_meta.csv")
dataset_dicts_test = get_vinbigdata_dicts_test(imgdir, test_meta, debug=debug)
test_dataset = VinbigdataTwoClassDataset(dataset_dicts_test, train=False)
test_loader = DataLoader(
test_dataset,
batch_size=flags.valid_batchsize,
num_workers=flags.num_workers,
shuffle=False,
pin_memory=True,
)
test_pred = classifier.predict_proba(test_loader).cpu().numpy()
test_pred_df = pd.DataFrame({
"image_id": [d["image_id"] for d in dataset_dicts_test],
"class0": test_pred[:, 0],
"class1": test_pred[:, 1]
})
test_pred_df.to_csv(outdir/"test_pred.csv", index=False)Training done! Start prediction...
36%|███▌ | 1078/3000 [00:00<00:00, 10770.64it/s]
Creating data... image shape: (256, 256, 3)
100%|██████████| 3000/3000 [00:00<00:00, 10812.96it/s]
Load from cache dataset_dicts_cache_test_debug0.pkl
In [35]:
# --- Test dataset prediction result ---
test_pred_dfOut [35]:
| image_id | class0 | class1 | |
|---|---|---|---|
| 0 | 8dec5497ecc246766acfba5a4be4e619 | 0.999959 | 0.000041 |
| 1 | 287422bed1d9d153387361889619abed | 0.220325 | 0.779675 |
| 2 | 1d12b94b7acbeadef7d7700b50aa90d4 | 0.994647 | 0.005353 |
| 3 | 6b872791e23742f6c33a08fc24f77365 | 0.705537 | 0.294463 |
| 4 | d0d2addff91ad7beb1d92126ff74d621 | 0.997452 | 0.002548 |
| ... | ... | ... | ... |
| 2995 | 78b44b96b121d6075d7ae27135278e03 | 0.999906 | 0.000094 |
| 2996 | afee8ff90f29b8827d0eb78774d25324 | 0.999739 | 0.000261 |
| 2997 | 6e07fab2014be723250f7897ab6e3df2 | 0.919132 | 0.080868 |
| 2998 | 690bb572300ef08bbbb7ebf4196099cf | 0.973054 | 0.026946 |
| 2999 | 0a08191a658edb1327e7282045ec71cf | 0.982506 | 0.017494 |
3000 rows × 3 columns
In [36]:
sns.distplot(valid_pred_df["class0"].values, color='green', label='valid pred')
sns.distplot(test_pred_df["class0"].values, color='orange', label='test pred')
plt.title("Prediction results histogram")
plt.xlim([0., 1.])
plt.legend()Out [36]:
<matplotlib.legend.Legend at 0x7fde3027b9d0>
In [37]:
# pred_2class = pd.read_csv(inputdir/"vinbigdata-2class-prediction/2-cls test pred.csv") # LB 0.230
# low_threshold = 0.0
# high_threshold = 0.95
pred_2class = pd.read_csv(inputdir/"vinbigdata2classpred/test_pred.csv")
low_threshold = 0.0
high_threshold = 0.976
pred_2classOut [37]:
| image_id | class0 | class1 | |
|---|---|---|---|
| 0 | 8dec5497ecc246766acfba5a4be4e619 | 0.976988 | 0.023012 |
| 1 | 287422bed1d9d153387361889619abed | 0.950402 | 0.049598 |
| 2 | 1d12b94b7acbeadef7d7700b50aa90d4 | 0.995952 | 0.004048 |
| 3 | 6b872791e23742f6c33a08fc24f77365 | 0.874948 | 0.125052 |
| 4 | d0d2addff91ad7beb1d92126ff74d621 | 0.997519 | 0.002481 |
| ... | ... | ... | ... |
| 2995 | 78b44b96b121d6075d7ae27135278e03 | 0.991775 | 0.008225 |
| 2996 | afee8ff90f29b8827d0eb78774d25324 | 0.998331 | 0.001669 |
| 2997 | 6e07fab2014be723250f7897ab6e3df2 | 0.990037 | 0.009963 |
| 2998 | 690bb572300ef08bbbb7ebf4196099cf | 0.975643 | 0.024356 |
| 2999 | 0a08191a658edb1327e7282045ec71cf | 0.993029 | 0.006971 |
3000 rows × 3 columns
In [38]:
NORMAL = "14 1 0 0 1 1"
pred_det_df = pd.read_csv(inputdir/"vinbigdata-detectron2-prediction/results/20210125_all_alb_aug_512_cos/submission.csv") # You can load from another submission.csv here too.
n_normal_before = len(pred_det_df.query("PredictionString == @NORMAL"))
merged_df = pd.merge(pred_det_df, pred_2class, on="image_id", how="left")
# 1. p < low_threshold -> "Keep": Do nothing, Keep det prediction.
# 2. low_threshold <= p < high_threshold -> "Add": Just "Add" Normal prediction
# 3. high_threshold <= p -> "Replace": Replace with Normal prediction
if "target" in merged_df.columns:
merged_df["class0"] = 1 - merged_df["target"]
c0, c1, c2 = 0, 0, 0
for i in range(len(merged_df)):
p0 = merged_df.loc[i, "class0"]
if p0 < low_threshold:
# Keep, do nothing.
c0 += 1
elif low_threshold <= p0 and p0 < high_threshold:
# Add, keep "det" preds and add normal pred.
merged_df.loc[i, "PredictionString"] += f" 14 {p0} 0 0 1 1"
c1 += 1
else:
# Replace, remove all "det" preds.
merged_df.loc[i, "PredictionString"] = NORMAL
c2 += 1
n_normal_after = len(merged_df.query("PredictionString == @NORMAL"))
print(
f"n_normal: {n_normal_before} -> {n_normal_after} with threshold {low_threshold} & {high_threshold}"
)
print(f"Keep {c0} Add {c1} Replace {c2}")
submission_filepath = str(outdir / "submission.csv")
submission_df = merged_df[["image_id", "PredictionString"]]
submission_df.to_csv(submission_filepath, index=False)
print(f"Saved to {submission_filepath}")n_normal: 0 -> 1713 with threshold 0.0 & 0.976 Keep 0 Add 1287 Replace 1713 Saved to results/tmp_debug/submission.csv
Using other datasets?
There are several other X-ray images in this research field.
The paper of this competition dataset "VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations”.
summarizes the existing public datasets.
CheXpert is one of the biggest dataset in this area.
Even if local label (for detection) is not available, it helps to create more accurate model by combining to use this dataset.
You can register your name and E-mail address to Download this Dataset!
In [ ]: