698 lines
33 KiB
Plaintext
698 lines
33 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.009537,
|
||
"end_time": "2021-03-19T01:19:31.724839",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:31.715302",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"<div align='center'><font size=\"5\" color='#353B47'>Chest-X-ray</font></div>\n",
|
||
"<div align='center'><font size=\"4\" color=\"#353B47\">How to deal with models averaging ?</font></div>\n",
|
||
"<br>\n",
|
||
"<hr>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.007744,
|
||
"end_time": "2021-03-19T01:19:31.740829",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:31.733085",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"The objective of this notebook is to aproach different methods for Ensembling\n",
|
||
"\n",
|
||
"* OR method (Affirmative): A box is considered if it’s generated by at least one of the models.\n",
|
||
"* AND method (Unanimous): A box is considered if all of the models generate the same box (the box is considered the same if IOU > 0.5).\n",
|
||
"* Consensus method: A box is considered if the majority of the models generate the same box (ie) if there are m models and (m/2 +1) models generate the same box, that box is considered as valid.\n",
|
||
"* Weighted Fusion: This is a novel method which was created to replace NMS and it’s shortcomings."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.007818,
|
||
"end_time": "2021-03-19T01:19:31.756653",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:31.748835",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"# <div id=\"summary\">Summary</div>\n",
|
||
"\n",
|
||
"**<font size=\"2\"><a href=\"#chap1\">1. Load libraries and dataframes with predictions</a></font>**\n",
|
||
"**<br><font size=\"2\"><a href=\"#chap2\">2. Helper functions</a></font>**\n",
|
||
"**<br><font size=\"2\"><a href=\"#chap3\">3. Run ensembling with appropriate strategy</a></font>**\n",
|
||
"**<br><font size=\"2\"><a href=\"#chap4\">4. Save results</a></font>**"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.00764,
|
||
"end_time": "2021-03-19T01:19:31.772375",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:31.764735",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"# <div id=\"chap1\">1. Load libraries and dataframes with predictions"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"metadata": {
|
||
"_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
|
||
"_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
|
||
"execution": {
|
||
"iopub.execute_input": "2021-03-19T01:19:31.792647Z",
|
||
"iopub.status.busy": "2021-03-19T01:19:31.791596Z",
|
||
"iopub.status.idle": "2021-03-19T01:19:31.797664Z",
|
||
"shell.execute_reply": "2021-03-19T01:19:31.796907Z"
|
||
},
|
||
"papermill": {
|
||
"duration": 0.017416,
|
||
"end_time": "2021-03-19T01:19:31.797873",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:31.780457",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"import warnings\n",
|
||
"warnings.filterwarnings('ignore')\n",
|
||
"\n",
|
||
"import numpy as np\n",
|
||
"import pandas as pd\n",
|
||
"import os\n",
|
||
"from tqdm import tqdm"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2021-03-19T01:19:31.820376Z",
|
||
"iopub.status.busy": "2021-03-19T01:19:31.819460Z",
|
||
"iopub.status.idle": "2021-03-19T01:19:31.859156Z",
|
||
"shell.execute_reply": "2021-03-19T01:19:31.859859Z"
|
||
},
|
||
"papermill": {
|
||
"duration": 0.053733,
|
||
"end_time": "2021-03-19T01:19:31.860047",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:31.806314",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [
|
||
{
|
||
"ename": "FileNotFoundError",
|
||
"evalue": "[Errno 2] No such file or directory: '../input/ensample/submission0.175.csv'",
|
||
"output_type": "error",
|
||
"traceback": [
|
||
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
||
"\u001b[0;31mFileNotFoundError\u001b[0m Traceback (most recent call last)",
|
||
"\u001b[0;32m<ipython-input-2-7dcdf4de33b8>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;31m# detectron = pd.read_csv('../input/vinbigdatastack/detectron2.csv')\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0;31m# fasterrcnn = pd.read_csv('../input/vinbigdatastack/fasterrcnn.csv')\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0myolo\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'../input/ensample/submission0.175.csv'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 6\u001b[0m \u001b[0mdetectron\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'../input/ensample/submission_0.2.csv'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 7\u001b[0m \u001b[0mfasterrcnn\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'../input/ensample/submission_2class filter.csv'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
||
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36mread_csv\u001b[0;34m(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision, storage_options)\u001b[0m\n\u001b[1;32m 603\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkwds_defaults\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 604\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 605\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0m_read\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath_or_buffer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 606\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 607\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
|
||
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_read\u001b[0;34m(filepath_or_buffer, kwds)\u001b[0m\n\u001b[1;32m 455\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 456\u001b[0m \u001b[0;31m# Create the parser.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 457\u001b[0;31m \u001b[0mparser\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTextFileReader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath_or_buffer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 458\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 459\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mchunksize\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0miterator\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
||
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, f, engine, **kwds)\u001b[0m\n\u001b[1;32m 812\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"has_index_names\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"has_index_names\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 813\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 814\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_engine\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_make_engine\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mengine\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 815\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 816\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mclose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
||
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_make_engine\u001b[0;34m(self, engine)\u001b[0m\n\u001b[1;32m 1043\u001b[0m )\n\u001b[1;32m 1044\u001b[0m \u001b[0;31m# error: Too many arguments for \"ParserBase\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1045\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mmapping\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mengine\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# type: ignore[call-arg]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1046\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1047\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_failover_to_python\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
||
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, src, **kwds)\u001b[0m\n\u001b[1;32m 1860\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1861\u001b[0m \u001b[0;31m# open handles\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1862\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_open_handles\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msrc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1863\u001b[0m \u001b[0;32massert\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhandles\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1864\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mkey\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m\"storage_options\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"encoding\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"memory_map\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"compression\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
||
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_open_handles\u001b[0;34m(self, src, kwds)\u001b[0m\n\u001b[1;32m 1361\u001b[0m \u001b[0mcompression\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"compression\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1362\u001b[0m \u001b[0mmemory_map\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"memory_map\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1363\u001b[0;31m \u001b[0mstorage_options\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"storage_options\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1364\u001b[0m )\n\u001b[1;32m 1365\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
|
||
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/pandas/io/common.py\u001b[0m in \u001b[0;36mget_handle\u001b[0;34m(path_or_buf, mode, encoding, compression, memory_map, is_text, errors, storage_options)\u001b[0m\n\u001b[1;32m 642\u001b[0m \u001b[0mencoding\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mioargs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mencoding\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 643\u001b[0m \u001b[0merrors\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0merrors\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 644\u001b[0;31m \u001b[0mnewline\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 645\u001b[0m )\n\u001b[1;32m 646\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
||
"\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: '../input/ensample/submission0.175.csv'"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Import outputs of each selected models\n",
|
||
"# yolo = pd.read_csv('../input/vinbigdatastack/yolov5.csv')\n",
|
||
"# detectron = pd.read_csv('../input/vinbigdatastack/detectron2.csv')\n",
|
||
"# fasterrcnn = pd.read_csv('../input/vinbigdatastack/fasterrcnn.csv')\n",
|
||
"yolo = pd.read_csv('../input/ensample/submission0.175.csv')\n",
|
||
"detectron = pd.read_csv('../input/ensample/submission_0.2.csv')\n",
|
||
"fasterrcnn = pd.read_csv('../input/ensample/submission_2class filter.csv')\n",
|
||
"image_ids = yolo.image_id.values"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.008483,
|
||
"end_time": "2021-03-19T01:19:31.877751",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:31.869268",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"# <div id=\"chap2\">2. Helper functions"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2021-03-19T01:19:31.899085Z",
|
||
"iopub.status.busy": "2021-03-19T01:19:31.898351Z",
|
||
"iopub.status.idle": "2021-03-19T01:19:31.915873Z",
|
||
"shell.execute_reply": "2021-03-19T01:19:31.916363Z"
|
||
},
|
||
"papermill": {
|
||
"duration": 0.029965,
|
||
"end_time": "2021-03-19T01:19:31.916530",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:31.886565",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def getitem(dataframe, img_id):\n",
|
||
" \n",
|
||
" \"\"\"\n",
|
||
" Parameters\n",
|
||
" ----------\n",
|
||
" dataframe : pd.DataFrame\n",
|
||
" img_id : str\n",
|
||
" \n",
|
||
" Returns\n",
|
||
" -------\n",
|
||
" Dictionary of radiographic observations\n",
|
||
" \"\"\"\n",
|
||
"\n",
|
||
" pred = list(dataframe.loc[dataframe.image_id == img_id, \"PredictionString\"])[0].split(' ')\n",
|
||
" nb_elm = len(pred)//6\n",
|
||
" output = {}\n",
|
||
" \n",
|
||
" for elm in range(nb_elm):\n",
|
||
" output[f'elm_{elm}'] = pred[elm*6 : (elm+1)*6]\n",
|
||
" \n",
|
||
" return output\n",
|
||
"\n",
|
||
"\n",
|
||
"def sortDictByProba(dict_):\n",
|
||
" \n",
|
||
" \"\"\"\n",
|
||
" Parameters\n",
|
||
" ----------\n",
|
||
" dict_ : dict, Dictionary of radiographic observations\n",
|
||
" \n",
|
||
" Returns\n",
|
||
" -------\n",
|
||
" Dictionary of radiographic observations sorted by probabilities \n",
|
||
" \"\"\"\n",
|
||
" \n",
|
||
" for key in dict_.keys():\n",
|
||
" dict_[key] = list(map(lambda x: float(x), dict_[key]))\n",
|
||
" \n",
|
||
" # item[1][1] corresponds to the second element of the value (the confidence of the class identified)\n",
|
||
" return {k: v for k, v in sorted(dict_.items(), key=lambda item: item[1][1], reverse = True)}\n",
|
||
"\n",
|
||
"\n",
|
||
"def getHighestProba(*list_of_dicts, n=3):\n",
|
||
" \n",
|
||
" \"\"\"\n",
|
||
" Parameters\n",
|
||
" ----------\n",
|
||
" list_of_dicts : list[dict], List of dictionaries containing radiographic observations\n",
|
||
" n : int, keep n highest elements of each list_of_dicts at most\n",
|
||
" \n",
|
||
" Returns\n",
|
||
" -------\n",
|
||
" Dict of merged top3 confidence interval in each dict of list_of_dicts\n",
|
||
" \"\"\"\n",
|
||
" \n",
|
||
" output = {}\n",
|
||
" for index, dict_ in enumerate(list_of_dicts):\n",
|
||
" dict_length = len(dict_)\n",
|
||
" for i in range(dict_length):\n",
|
||
" if i < n:\n",
|
||
" output[f\"elm_{i}_dict_{index}\"] =list(dict_.values())[i]\n",
|
||
" \n",
|
||
" return output\n",
|
||
"\n",
|
||
"\n",
|
||
"def getUnique(dict_):\n",
|
||
" \n",
|
||
" \"\"\"\n",
|
||
" Parameters\n",
|
||
" ----------\n",
|
||
" dict_ : dict, Dictionary of radiographic observations\n",
|
||
" \n",
|
||
" Returns\n",
|
||
" -------\n",
|
||
" List of unique class_id, list of duplicates class_id\n",
|
||
" \"\"\"\n",
|
||
" \n",
|
||
" dict_length = len(dict_)\n",
|
||
" \n",
|
||
" classes_non_unique = [list(dict_.values())[index][0] for index in range(dict_length)]\n",
|
||
" classes_unique = list(set(classes_non_unique))\n",
|
||
" \n",
|
||
" uniques, counts = np.unique(classes_non_unique, return_counts=True)\n",
|
||
" duplicates = uniques[counts > 1]\n",
|
||
" singles = np.setdiff1d(classes_unique, duplicates)\n",
|
||
" \n",
|
||
" return singles, duplicates\n",
|
||
"\n",
|
||
"\n",
|
||
"def getKeysByValue(dictOfElements, valueToFind):\n",
|
||
" \n",
|
||
" \"\"\"\n",
|
||
" Parameters\n",
|
||
" ----------\n",
|
||
" dictOfElements : dict, Dictionary of radiographic observations\n",
|
||
" valueToFind : int, corresponds to class_id\n",
|
||
" \n",
|
||
" Returns\n",
|
||
" -------\n",
|
||
" List of keys of dictOfElements that contain valueToFind\n",
|
||
" \"\"\"\n",
|
||
" \n",
|
||
" output = list()\n",
|
||
" listOfItems = dictOfElements.items()\n",
|
||
" \n",
|
||
" for item in listOfItems:\n",
|
||
" if item[1][0] == valueToFind:\n",
|
||
" output.append(item[0])\n",
|
||
" \n",
|
||
" return output\n",
|
||
"\n",
|
||
"\n",
|
||
"def getListKeysByValue(dictOfElements, valuesToFind):\n",
|
||
" \n",
|
||
" \"\"\"\n",
|
||
" Parameters\n",
|
||
" ----------\n",
|
||
" dictOfElements : dict, Dictionary of radiographic observations\n",
|
||
" valuesToFind : list[int], list of class_id\n",
|
||
" \n",
|
||
" Returns\n",
|
||
" -------\n",
|
||
" List of lists of keys of dictOfElements for each value in valuesToFind\n",
|
||
" \"\"\"\n",
|
||
" \n",
|
||
" output = []\n",
|
||
" \n",
|
||
" for value in valuesToFind:\n",
|
||
" output.append(getKeysByValue(dictOfElements, value))\n",
|
||
" \n",
|
||
" return output\n",
|
||
"\n",
|
||
"\n",
|
||
"def averaging(from_dict, single_keys, dupl_keys):\n",
|
||
" \n",
|
||
" \"\"\"\n",
|
||
" Parameters\n",
|
||
" ----------\n",
|
||
" from_dict : dict, dictionary to be filtered\n",
|
||
" single_keys : list[str], list of keys that should be infered\n",
|
||
" dupl_keys : list[str], list of class_id\n",
|
||
" \n",
|
||
" Returns\n",
|
||
" -------\n",
|
||
" A filtered dictionary with averaged probs and boxes\n",
|
||
" \"\"\"\n",
|
||
" \n",
|
||
" output = {}\n",
|
||
" \n",
|
||
" # Infer single keys\n",
|
||
" if len(np.ravel(single_keys)) != 0:\n",
|
||
" for single in np.ravel(single_keys):\n",
|
||
" output[single] = from_dict[single]\n",
|
||
"\n",
|
||
" # For each duplicates, get index of all occurences and average boxing\n",
|
||
" if len(np.ravel(dupl_keys)) != 0:\n",
|
||
" for index, list_of_duplicate_class in enumerate(dupl_keys):\n",
|
||
" probs = [] \n",
|
||
" boxing1 = []\n",
|
||
" boxing2 = []\n",
|
||
" boxing3 = []\n",
|
||
" boxing4 = []\n",
|
||
" \n",
|
||
" for elm in list_of_duplicate_class:\n",
|
||
" probs.append(from_dict[elm][1])\n",
|
||
" boxing1.append(from_dict[elm][2])\n",
|
||
" boxing2.append(from_dict[elm][3])\n",
|
||
" boxing3.append(from_dict[elm][4])\n",
|
||
" boxing4.append(from_dict[elm][5])\n",
|
||
" \n",
|
||
" output[f\"elm_{index}\"] = [from_dict[list_of_duplicate_class[0]][0],\n",
|
||
" np.mean(probs),\n",
|
||
" np.mean(boxing1),\n",
|
||
" np.mean(boxing2),\n",
|
||
" np.mean(boxing3),\n",
|
||
" np.mean(boxing4)]\n",
|
||
" \n",
|
||
" return output\n",
|
||
"\n",
|
||
"\n",
|
||
"def toString(pred_list):\n",
|
||
" \n",
|
||
" \"\"\"\n",
|
||
" Parameters\n",
|
||
" ----------\n",
|
||
" list_final : list[int], list of all radiographic observations\n",
|
||
" \n",
|
||
" Returns\n",
|
||
" -------\n",
|
||
" A string which fits with the expected output\n",
|
||
" \"\"\"\n",
|
||
" \n",
|
||
" castedList = []\n",
|
||
" for index, elm in enumerate(pred_list):\n",
|
||
" if index%6 == 0:\n",
|
||
" castedList.append(str(int(elm)))\n",
|
||
" else:\n",
|
||
" castedList.append(str(elm))\n",
|
||
" \n",
|
||
" output = \" \".join(castedList)\n",
|
||
" \n",
|
||
" return output"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.008647,
|
||
"end_time": "2021-03-19T01:19:31.934073",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:31.925426",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"--------\n",
|
||
"\n",
|
||
"**<font size=\"2\"><a href=\"#summary\">Back to summary</a></font>**"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.008675,
|
||
"end_time": "2021-03-19T01:19:31.951695",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:31.943020",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"# <div id=\"chap3\">3. Run ensembling with appropriate strategy"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.008744,
|
||
"end_time": "2021-03-19T01:19:31.969329",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:31.960585",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"My strategy here consists in averaging observations that have at least one dupplicate among all models. Some filtering about boxing areas should be added. This will come in a future release"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2021-03-19T01:19:31.992666Z",
|
||
"iopub.status.busy": "2021-03-19T01:19:31.992043Z",
|
||
"iopub.status.idle": "2021-03-19T01:19:31.999240Z",
|
||
"shell.execute_reply": "2021-03-19T01:19:31.999739Z"
|
||
},
|
||
"papermill": {
|
||
"duration": 0.021755,
|
||
"end_time": "2021-03-19T01:19:31.999915",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:31.978160",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def main():\n",
|
||
" \n",
|
||
" output = pd.DataFrame(columns = [\"image_id\", \"PredictionString\"])\n",
|
||
" \n",
|
||
" for image_id in tqdm(image_ids):\n",
|
||
" \n",
|
||
" # For each model, get PredictionString of image_id as a dict\n",
|
||
" fasterrcnn_pred = getitem(fasterrcnn, image_id)\n",
|
||
" detectron_pred = getitem(detectron, image_id)\n",
|
||
" yolo_pred = getitem(yolo, image_id) \n",
|
||
" \n",
|
||
" # Sort dicts by proba\n",
|
||
" sorted_fasterrcnn = sortDictByProba(fasterrcnn_pred)\n",
|
||
" sorted_detectron = sortDictByProba(detectron_pred)\n",
|
||
" sorted_yolo = sortDictByProba(yolo_pred)\n",
|
||
"\n",
|
||
" # Filter dicts into one dict with at most top n probs\n",
|
||
" highest_probs = getHighestProba(sorted_fasterrcnn, \n",
|
||
" sorted_detectron, \n",
|
||
" sorted_yolo,\n",
|
||
" n = 3)\n",
|
||
" \n",
|
||
" # Get keys of unique and duplicates values in the filtered dict\n",
|
||
" singles, duplicates = getUnique(highest_probs)\n",
|
||
" single_keys = getListKeysByValue(highest_probs, singles)\n",
|
||
" dupl_keys = getListKeysByValue(highest_probs, duplicates)\n",
|
||
" \n",
|
||
" # Apply averaging strategy\n",
|
||
" stacked_dict = averaging(highest_probs, single_keys, dupl_keys)\n",
|
||
" \n",
|
||
" # Put string in right format\n",
|
||
" prediction_int = np.ravel(list(stacked_dict.values()))\n",
|
||
" prediction_string = toString(prediction_int)\n",
|
||
" \n",
|
||
" output = output.append({\"image_id\": image_id, \n",
|
||
" \"PredictionString\": prediction_string},\n",
|
||
" ignore_index=True)\n",
|
||
" \n",
|
||
" return output"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.008891,
|
||
"end_time": "2021-03-19T01:19:32.018555",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:32.009664",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"Some other strategies will be tested in a future release:\n",
|
||
"* OR method \n",
|
||
"* AND method\n",
|
||
"* Consensus method\n",
|
||
"* Weighted Fusion"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.009073,
|
||
"end_time": "2021-03-19T01:19:32.036791",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:32.027718",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"In the meantime, if you found this notebook usefull and you do have some suggestions on how this could be better implemented, do not hesitate to contribute, i'd really appreciate !"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.008758,
|
||
"end_time": "2021-03-19T01:19:32.054697",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:32.045939",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"--------\n",
|
||
"\n",
|
||
"**<font size=\"2\"><a href=\"#summary\">Back to summary</a></font>**"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.00879,
|
||
"end_time": "2021-03-19T01:19:32.072694",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:32.063904",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"# <div id=\"chap4\">4. Save results"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2021-03-19T01:19:32.094093Z",
|
||
"iopub.status.busy": "2021-03-19T01:19:32.093456Z",
|
||
"iopub.status.idle": "2021-03-19T01:19:32.119598Z",
|
||
"shell.execute_reply": "2021-03-19T01:19:32.119006Z"
|
||
},
|
||
"papermill": {
|
||
"duration": 0.038032,
|
||
"end_time": "2021-03-19T01:19:32.119771",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:32.081739",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [
|
||
{
|
||
"ename": "NameError",
|
||
"evalue": "name 'image_ids' is not defined",
|
||
"output_type": "error",
|
||
"traceback": [
|
||
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
||
"\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
|
||
"\u001b[0;32m<ipython-input-5-27cf38391b15>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mfinal_sub\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmain\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 2\u001b[0m \u001b[0mfinal_sub\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"submission.csv\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
||
"\u001b[0;32m<ipython-input-4-dec6ed9bcdff>\u001b[0m in \u001b[0;36mmain\u001b[0;34m()\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDataFrame\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcolumns\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m\"image_id\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"PredictionString\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0;32mfor\u001b[0m \u001b[0mimage_id\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtqdm\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mimage_ids\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 6\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 7\u001b[0m \u001b[0;31m# For each model, get PredictionString of image_id as a dict\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
||
"\u001b[0;31mNameError\u001b[0m: name 'image_ids' is not defined"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"final_sub = main()\n",
|
||
"final_sub.to_csv(\"submission.csv\", index=False)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.009203,
|
||
"end_time": "2021-03-19T01:19:32.138645",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:32.129442",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"# References\n",
|
||
"\n",
|
||
"* <a href = \"https://medium.com/inspiredbrilliance/object-detection-through-ensemble-of-models-fed015bc1ee0\">Article on object detection through ensemble of models</a>\n",
|
||
"* detectron2 : https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/code?competitionId=24800&sortBy=scoreDescending\n",
|
||
"* fasterrcnn : https://www.kaggle.com/awsaf49/vinbigdata-cxr-ad-yolov5-14-class-infer\n",
|
||
"* yolov5 : https://www.kaggle.com/basu369victor/chest-x-ray-abnormalities-detection-submission"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.009058,
|
||
"end_time": "2021-03-19T01:19:32.157045",
|
||
"exception": false,
|
||
"start_time": "2021-03-19T01:19:32.147987",
|
||
"status": "completed"
|
||
},
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"<hr>\n",
|
||
"<div align='justify'><font color=\"#353B47\" size=\"4\">Thank you for taking the time to read this notebook. I hope that I was able to answer your questions or your curiosity and that it was quite understandable. <u>any constructive comments are welcome</u>. They help me progress and motivate me to share better quality content. I am above all a passionate person who tries to advance my knowledge but also that of others. If you liked it, feel free to <u>upvote and share my work.</u> </font></div>\n",
|
||
"<br>\n",
|
||
"<div align='center'><font color=\"#353B47\" size=\"3\">Thank you and may passion guide you.</font></div>"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.7.9"
|
||
},
|
||
"papermill": {
|
||
"default_parameters": {},
|
||
"duration": 6.790273,
|
||
"end_time": "2021-03-19T01:19:32.779060",
|
||
"environment_variables": {},
|
||
"exception": null,
|
||
"input_path": "__notebook__.ipynb",
|
||
"output_path": "__notebook__.ipynb",
|
||
"parameters": {},
|
||
"start_time": "2021-03-19T01:19:25.988787",
|
||
"version": "2.2.2"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 4
|
||
}
|