{ "cells": [ { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.009537, "end_time": "2021-03-19T01:19:31.724839", "exception": false, "start_time": "2021-03-19T01:19:31.715302", "status": "completed" }, "tags": [] }, "source": [ "
Chest-X-ray
\n", "
How to deal with models averaging ?
\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.007744, "end_time": "2021-03-19T01:19:31.740829", "exception": false, "start_time": "2021-03-19T01:19:31.733085", "status": "completed" }, "tags": [] }, "source": [ "The objective of this notebook is to aproach different methods for Ensembling\n", "\n", "* OR method (Affirmative): A box is considered if it’s generated by at least one of the models.\n", "* AND method (Unanimous): A box is considered if all of the models generate the same box (the box is considered the same if IOU > 0.5).\n", "* Consensus method: A box is considered if the majority of the models generate the same box (ie) if there are m models and (m/2 +1) models generate the same box, that box is considered as valid.\n", "* Weighted Fusion: This is a novel method which was created to replace NMS and it’s shortcomings." ] }, { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.007818, "end_time": "2021-03-19T01:19:31.756653", "exception": false, "start_time": "2021-03-19T01:19:31.748835", "status": "completed" }, "tags": [] }, "source": [ "#
Summary
\n", "\n", "**1. Load libraries and dataframes with predictions**\n", "**
2. Helper functions**\n", "**
3. Run ensembling with appropriate strategy**\n", "**
4. Save results**" ] }, { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.00764, "end_time": "2021-03-19T01:19:31.772375", "exception": false, "start_time": "2021-03-19T01:19:31.764735", "status": "completed" }, "tags": [] }, "source": [ "#
1. Load libraries and dataframes with predictions" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19", "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5", "execution": { "iopub.execute_input": "2021-03-19T01:19:31.792647Z", "iopub.status.busy": "2021-03-19T01:19:31.791596Z", "iopub.status.idle": "2021-03-19T01:19:31.797664Z", "shell.execute_reply": "2021-03-19T01:19:31.796907Z" }, "papermill": { "duration": 0.017416, "end_time": "2021-03-19T01:19:31.797873", "exception": false, "start_time": "2021-03-19T01:19:31.780457", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "import warnings\n", "warnings.filterwarnings('ignore')\n", "\n", "import numpy as np\n", "import pandas as pd\n", "import os\n", "from tqdm import tqdm" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2021-03-19T01:19:31.820376Z", "iopub.status.busy": "2021-03-19T01:19:31.819460Z", "iopub.status.idle": "2021-03-19T01:19:31.859156Z", "shell.execute_reply": "2021-03-19T01:19:31.859859Z" }, "papermill": { "duration": 0.053733, "end_time": "2021-03-19T01:19:31.860047", "exception": false, "start_time": "2021-03-19T01:19:31.806314", "status": "completed" }, "tags": [] }, "outputs": [ { "ename": "FileNotFoundError", "evalue": "[Errno 2] No such file or directory: '../input/ensample/submission0.175.csv'", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mFileNotFoundError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;31m# detectron = pd.read_csv('../input/vinbigdatastack/detectron2.csv')\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0;31m# fasterrcnn = pd.read_csv('../input/vinbigdatastack/fasterrcnn.csv')\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0myolo\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'../input/ensample/submission0.175.csv'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 6\u001b[0m \u001b[0mdetectron\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'../input/ensample/submission_0.2.csv'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 7\u001b[0m \u001b[0mfasterrcnn\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'../input/ensample/submission_2class filter.csv'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36mread_csv\u001b[0;34m(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision, storage_options)\u001b[0m\n\u001b[1;32m 603\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkwds_defaults\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 604\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 605\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0m_read\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath_or_buffer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 606\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 607\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_read\u001b[0;34m(filepath_or_buffer, kwds)\u001b[0m\n\u001b[1;32m 455\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 456\u001b[0m \u001b[0;31m# Create the parser.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 457\u001b[0;31m \u001b[0mparser\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTextFileReader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath_or_buffer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 458\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 459\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mchunksize\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0miterator\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, f, engine, **kwds)\u001b[0m\n\u001b[1;32m 812\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"has_index_names\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"has_index_names\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 813\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 814\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_engine\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_make_engine\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mengine\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 815\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 816\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mclose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_make_engine\u001b[0;34m(self, engine)\u001b[0m\n\u001b[1;32m 1043\u001b[0m )\n\u001b[1;32m 1044\u001b[0m \u001b[0;31m# error: Too many arguments for \"ParserBase\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1045\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mmapping\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mengine\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# type: ignore[call-arg]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1046\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1047\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_failover_to_python\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, src, **kwds)\u001b[0m\n\u001b[1;32m 1860\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1861\u001b[0m \u001b[0;31m# open handles\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1862\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_open_handles\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msrc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1863\u001b[0m \u001b[0;32massert\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhandles\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1864\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mkey\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m\"storage_options\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"encoding\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"memory_map\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"compression\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_open_handles\u001b[0;34m(self, src, kwds)\u001b[0m\n\u001b[1;32m 1361\u001b[0m \u001b[0mcompression\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"compression\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1362\u001b[0m \u001b[0mmemory_map\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"memory_map\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1363\u001b[0;31m \u001b[0mstorage_options\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"storage_options\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1364\u001b[0m )\n\u001b[1;32m 1365\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/common.py\u001b[0m in \u001b[0;36mget_handle\u001b[0;34m(path_or_buf, mode, encoding, compression, memory_map, is_text, errors, storage_options)\u001b[0m\n\u001b[1;32m 642\u001b[0m \u001b[0mencoding\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mioargs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mencoding\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 643\u001b[0m \u001b[0merrors\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0merrors\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 644\u001b[0;31m \u001b[0mnewline\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 645\u001b[0m )\n\u001b[1;32m 646\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: '../input/ensample/submission0.175.csv'" ] } ], "source": [ "# Import outputs of each selected models\n", "# yolo = pd.read_csv('../input/vinbigdatastack/yolov5.csv')\n", "# detectron = pd.read_csv('../input/vinbigdatastack/detectron2.csv')\n", "# fasterrcnn = pd.read_csv('../input/vinbigdatastack/fasterrcnn.csv')\n", "yolo = pd.read_csv('../input/ensample/submission0.175.csv')\n", "detectron = pd.read_csv('../input/ensample/submission_0.2.csv')\n", "fasterrcnn = pd.read_csv('../input/ensample/submission_2class filter.csv')\n", "image_ids = yolo.image_id.values" ] }, { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.008483, "end_time": "2021-03-19T01:19:31.877751", "exception": false, "start_time": "2021-03-19T01:19:31.869268", "status": "completed" }, "tags": [] }, "source": [ "#
2. Helper functions" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2021-03-19T01:19:31.899085Z", "iopub.status.busy": "2021-03-19T01:19:31.898351Z", "iopub.status.idle": "2021-03-19T01:19:31.915873Z", "shell.execute_reply": "2021-03-19T01:19:31.916363Z" }, "papermill": { "duration": 0.029965, "end_time": "2021-03-19T01:19:31.916530", "exception": false, "start_time": "2021-03-19T01:19:31.886565", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "def getitem(dataframe, img_id):\n", " \n", " \"\"\"\n", " Parameters\n", " ----------\n", " dataframe : pd.DataFrame\n", " img_id : str\n", " \n", " Returns\n", " -------\n", " Dictionary of radiographic observations\n", " \"\"\"\n", "\n", " pred = list(dataframe.loc[dataframe.image_id == img_id, \"PredictionString\"])[0].split(' ')\n", " nb_elm = len(pred)//6\n", " output = {}\n", " \n", " for elm in range(nb_elm):\n", " output[f'elm_{elm}'] = pred[elm*6 : (elm+1)*6]\n", " \n", " return output\n", "\n", "\n", "def sortDictByProba(dict_):\n", " \n", " \"\"\"\n", " Parameters\n", " ----------\n", " dict_ : dict, Dictionary of radiographic observations\n", " \n", " Returns\n", " -------\n", " Dictionary of radiographic observations sorted by probabilities \n", " \"\"\"\n", " \n", " for key in dict_.keys():\n", " dict_[key] = list(map(lambda x: float(x), dict_[key]))\n", " \n", " # item[1][1] corresponds to the second element of the value (the confidence of the class identified)\n", " return {k: v for k, v in sorted(dict_.items(), key=lambda item: item[1][1], reverse = True)}\n", "\n", "\n", "def getHighestProba(*list_of_dicts, n=3):\n", " \n", " \"\"\"\n", " Parameters\n", " ----------\n", " list_of_dicts : list[dict], List of dictionaries containing radiographic observations\n", " n : int, keep n highest elements of each list_of_dicts at most\n", " \n", " Returns\n", " -------\n", " Dict of merged top3 confidence interval in each dict of list_of_dicts\n", " \"\"\"\n", " \n", " output = {}\n", " for index, dict_ in enumerate(list_of_dicts):\n", " dict_length = len(dict_)\n", " for i in range(dict_length):\n", " if i < n:\n", " output[f\"elm_{i}_dict_{index}\"] =list(dict_.values())[i]\n", " \n", " return output\n", "\n", "\n", "def getUnique(dict_):\n", " \n", " \"\"\"\n", " Parameters\n", " ----------\n", " dict_ : dict, Dictionary of radiographic observations\n", " \n", " Returns\n", " -------\n", " List of unique class_id, list of duplicates class_id\n", " \"\"\"\n", " \n", " dict_length = len(dict_)\n", " \n", " classes_non_unique = [list(dict_.values())[index][0] for index in range(dict_length)]\n", " classes_unique = list(set(classes_non_unique))\n", " \n", " uniques, counts = np.unique(classes_non_unique, return_counts=True)\n", " duplicates = uniques[counts > 1]\n", " singles = np.setdiff1d(classes_unique, duplicates)\n", " \n", " return singles, duplicates\n", "\n", "\n", "def getKeysByValue(dictOfElements, valueToFind):\n", " \n", " \"\"\"\n", " Parameters\n", " ----------\n", " dictOfElements : dict, Dictionary of radiographic observations\n", " valueToFind : int, corresponds to class_id\n", " \n", " Returns\n", " -------\n", " List of keys of dictOfElements that contain valueToFind\n", " \"\"\"\n", " \n", " output = list()\n", " listOfItems = dictOfElements.items()\n", " \n", " for item in listOfItems:\n", " if item[1][0] == valueToFind:\n", " output.append(item[0])\n", " \n", " return output\n", "\n", "\n", "def getListKeysByValue(dictOfElements, valuesToFind):\n", " \n", " \"\"\"\n", " Parameters\n", " ----------\n", " dictOfElements : dict, Dictionary of radiographic observations\n", " valuesToFind : list[int], list of class_id\n", " \n", " Returns\n", " -------\n", " List of lists of keys of dictOfElements for each value in valuesToFind\n", " \"\"\"\n", " \n", " output = []\n", " \n", " for value in valuesToFind:\n", " output.append(getKeysByValue(dictOfElements, value))\n", " \n", " return output\n", "\n", "\n", "def averaging(from_dict, single_keys, dupl_keys):\n", " \n", " \"\"\"\n", " Parameters\n", " ----------\n", " from_dict : dict, dictionary to be filtered\n", " single_keys : list[str], list of keys that should be infered\n", " dupl_keys : list[str], list of class_id\n", " \n", " Returns\n", " -------\n", " A filtered dictionary with averaged probs and boxes\n", " \"\"\"\n", " \n", " output = {}\n", " \n", " # Infer single keys\n", " if len(np.ravel(single_keys)) != 0:\n", " for single in np.ravel(single_keys):\n", " output[single] = from_dict[single]\n", "\n", " # For each duplicates, get index of all occurences and average boxing\n", " if len(np.ravel(dupl_keys)) != 0:\n", " for index, list_of_duplicate_class in enumerate(dupl_keys):\n", " probs = [] \n", " boxing1 = []\n", " boxing2 = []\n", " boxing3 = []\n", " boxing4 = []\n", " \n", " for elm in list_of_duplicate_class:\n", " probs.append(from_dict[elm][1])\n", " boxing1.append(from_dict[elm][2])\n", " boxing2.append(from_dict[elm][3])\n", " boxing3.append(from_dict[elm][4])\n", " boxing4.append(from_dict[elm][5])\n", " \n", " output[f\"elm_{index}\"] = [from_dict[list_of_duplicate_class[0]][0],\n", " np.mean(probs),\n", " np.mean(boxing1),\n", " np.mean(boxing2),\n", " np.mean(boxing3),\n", " np.mean(boxing4)]\n", " \n", " return output\n", "\n", "\n", "def toString(pred_list):\n", " \n", " \"\"\"\n", " Parameters\n", " ----------\n", " list_final : list[int], list of all radiographic observations\n", " \n", " Returns\n", " -------\n", " A string which fits with the expected output\n", " \"\"\"\n", " \n", " castedList = []\n", " for index, elm in enumerate(pred_list):\n", " if index%6 == 0:\n", " castedList.append(str(int(elm)))\n", " else:\n", " castedList.append(str(elm))\n", " \n", " output = \" \".join(castedList)\n", " \n", " return output" ] }, { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.008647, "end_time": "2021-03-19T01:19:31.934073", "exception": false, "start_time": "2021-03-19T01:19:31.925426", "status": "completed" }, "tags": [] }, "source": [ "--------\n", "\n", "**Back to summary**" ] }, { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.008675, "end_time": "2021-03-19T01:19:31.951695", "exception": false, "start_time": "2021-03-19T01:19:31.943020", "status": "completed" }, "tags": [] }, "source": [ "#
3. Run ensembling with appropriate strategy" ] }, { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.008744, "end_time": "2021-03-19T01:19:31.969329", "exception": false, "start_time": "2021-03-19T01:19:31.960585", "status": "completed" }, "tags": [] }, "source": [ "My strategy here consists in averaging observations that have at least one dupplicate among all models. Some filtering about boxing areas should be added. This will come in a future release" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2021-03-19T01:19:31.992666Z", "iopub.status.busy": "2021-03-19T01:19:31.992043Z", "iopub.status.idle": "2021-03-19T01:19:31.999240Z", "shell.execute_reply": "2021-03-19T01:19:31.999739Z" }, "papermill": { "duration": 0.021755, "end_time": "2021-03-19T01:19:31.999915", "exception": false, "start_time": "2021-03-19T01:19:31.978160", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "def main():\n", " \n", " output = pd.DataFrame(columns = [\"image_id\", \"PredictionString\"])\n", " \n", " for image_id in tqdm(image_ids):\n", " \n", " # For each model, get PredictionString of image_id as a dict\n", " fasterrcnn_pred = getitem(fasterrcnn, image_id)\n", " detectron_pred = getitem(detectron, image_id)\n", " yolo_pred = getitem(yolo, image_id) \n", " \n", " # Sort dicts by proba\n", " sorted_fasterrcnn = sortDictByProba(fasterrcnn_pred)\n", " sorted_detectron = sortDictByProba(detectron_pred)\n", " sorted_yolo = sortDictByProba(yolo_pred)\n", "\n", " # Filter dicts into one dict with at most top n probs\n", " highest_probs = getHighestProba(sorted_fasterrcnn, \n", " sorted_detectron, \n", " sorted_yolo,\n", " n = 3)\n", " \n", " # Get keys of unique and duplicates values in the filtered dict\n", " singles, duplicates = getUnique(highest_probs)\n", " single_keys = getListKeysByValue(highest_probs, singles)\n", " dupl_keys = getListKeysByValue(highest_probs, duplicates)\n", " \n", " # Apply averaging strategy\n", " stacked_dict = averaging(highest_probs, single_keys, dupl_keys)\n", " \n", " # Put string in right format\n", " prediction_int = np.ravel(list(stacked_dict.values()))\n", " prediction_string = toString(prediction_int)\n", " \n", " output = output.append({\"image_id\": image_id, \n", " \"PredictionString\": prediction_string},\n", " ignore_index=True)\n", " \n", " return output" ] }, { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.008891, "end_time": "2021-03-19T01:19:32.018555", "exception": false, "start_time": "2021-03-19T01:19:32.009664", "status": "completed" }, "tags": [] }, "source": [ "Some other strategies will be tested in a future release:\n", "* OR method \n", "* AND method\n", "* Consensus method\n", "* Weighted Fusion" ] }, { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.009073, "end_time": "2021-03-19T01:19:32.036791", "exception": false, "start_time": "2021-03-19T01:19:32.027718", "status": "completed" }, "tags": [] }, "source": [ "In the meantime, if you found this notebook usefull and you do have some suggestions on how this could be better implemented, do not hesitate to contribute, i'd really appreciate !" ] }, { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.008758, "end_time": "2021-03-19T01:19:32.054697", "exception": false, "start_time": "2021-03-19T01:19:32.045939", "status": "completed" }, "tags": [] }, "source": [ "--------\n", "\n", "**Back to summary**" ] }, { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.00879, "end_time": "2021-03-19T01:19:32.072694", "exception": false, "start_time": "2021-03-19T01:19:32.063904", "status": "completed" }, "tags": [] }, "source": [ "#
4. Save results" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2021-03-19T01:19:32.094093Z", "iopub.status.busy": "2021-03-19T01:19:32.093456Z", "iopub.status.idle": "2021-03-19T01:19:32.119598Z", "shell.execute_reply": "2021-03-19T01:19:32.119006Z" }, "papermill": { "duration": 0.038032, "end_time": "2021-03-19T01:19:32.119771", "exception": false, "start_time": "2021-03-19T01:19:32.081739", "status": "completed" }, "tags": [] }, "outputs": [ { "ename": "NameError", "evalue": "name 'image_ids' is not defined", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mfinal_sub\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmain\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 2\u001b[0m \u001b[0mfinal_sub\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"submission.csv\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m\u001b[0m in \u001b[0;36mmain\u001b[0;34m()\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDataFrame\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcolumns\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m\"image_id\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"PredictionString\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0;32mfor\u001b[0m \u001b[0mimage_id\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtqdm\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mimage_ids\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 6\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 7\u001b[0m \u001b[0;31m# For each model, get PredictionString of image_id as a dict\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;31mNameError\u001b[0m: name 'image_ids' is not defined" ] } ], "source": [ "final_sub = main()\n", "final_sub.to_csv(\"submission.csv\", index=False)" ] }, { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.009203, "end_time": "2021-03-19T01:19:32.138645", "exception": false, "start_time": "2021-03-19T01:19:32.129442", "status": "completed" }, "tags": [] }, "source": [ "# References\n", "\n", "* Article on object detection through ensemble of models\n", "* detectron2 : https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/code?competitionId=24800&sortBy=scoreDescending\n", "* fasterrcnn : https://www.kaggle.com/awsaf49/vinbigdata-cxr-ad-yolov5-14-class-infer\n", "* yolov5 : https://www.kaggle.com/basu369victor/chest-x-ray-abnormalities-detection-submission" ] }, { "cell_type": "markdown", "metadata": { "papermill": { "duration": 0.009058, "end_time": "2021-03-19T01:19:32.157045", "exception": false, "start_time": "2021-03-19T01:19:32.147987", "status": "completed" }, "tags": [] }, "source": [ "
\n", "
Thank you for taking the time to read this notebook. I hope that I was able to answer your questions or your curiosity and that it was quite understandable. any constructive comments are welcome. They help me progress and motivate me to share better quality content. I am above all a passionate person who tries to advance my knowledge but also that of others. If you liked it, feel free to upvote and share my work.
\n", "
\n", "
Thank you and may passion guide you.
" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.9" }, "papermill": { "default_parameters": {}, "duration": 6.790273, "end_time": "2021-03-19T01:19:32.779060", "environment_variables": {}, "exception": null, "input_path": "__notebook__.ipynb", "output_path": "__notebook__.ipynb", "parameters": {}, "start_time": "2021-03-19T01:19:25.988787", "version": "2.2.2" } }, "nbformat": 4, "nbformat_minor": 4 }