{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Image summary and visual question answering"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebooks shows how to generate image captions and use the visual question answering with [LAVIS](https://github.com/salesforce/LAVIS). \n",
    "\n",
    "The first cell is only run on google colab and installs the [ammico](https://github.com/ssciwr/AMMICO) package.\n",
    "\n",
    "After that, we can import `ammico` and read in the files given a folder path."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:34:20.124150Z",
     "iopub.status.busy": "2023-05-26T09:34:20.123919Z",
     "iopub.status.idle": "2023-05-26T09:34:20.132420Z",
     "shell.execute_reply": "2023-05-26T09:34:20.131778Z"
    }
   },
   "outputs": [],
   "source": [
    "# if running on google colab\n",
    "# flake8-noqa-cell\n",
    "import os\n",
    "\n",
    "if \"google.colab\" in str(get_ipython()):\n",
    "    # update python version\n",
    "    # install setuptools\n",
    "    # %pip install setuptools==61 -qqq\n",
    "    # install ammico\n",
    "    %pip install git+https://github.com/ssciwr/ammico.git -qqq\n",
    "    # mount google drive for data and API key\n",
    "    from google.colab import drive\n",
    "\n",
    "    drive.mount(\"/content/drive\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:34:20.135371Z",
     "iopub.status.busy": "2023-05-26T09:34:20.134938Z",
     "iopub.status.idle": "2023-05-26T09:34:31.164137Z",
     "shell.execute_reply": "2023-05-26T09:34:31.163398Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import ammico\n",
    "from ammico import utils as mutils\n",
    "from ammico import display as mdisplay\n",
    "import ammico.summary as sm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:34:31.168295Z",
     "iopub.status.busy": "2023-05-26T09:34:31.167332Z",
     "iopub.status.idle": "2023-05-26T09:34:31.171678Z",
     "shell.execute_reply": "2023-05-26T09:34:31.170972Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Here you need to provide the path to your google drive folder\n",
    "# or local folder containing the images\n",
    "images = mutils.find_files(\n",
    "    path=\"data/\",\n",
    "    limit=10,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:34:31.174681Z",
     "iopub.status.busy": "2023-05-26T09:34:31.174079Z",
     "iopub.status.idle": "2023-05-26T09:34:31.177445Z",
     "shell.execute_reply": "2023-05-26T09:34:31.176797Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "mydict = mutils.initialize_dict(images)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create captions for images and directly write to csv"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here you can choose between two models: \"base\" or \"large\". This will generate the caption for each image and directly put the results in a dataframe. This dataframe can be exported as a csv file.\n",
    "\n",
    "The results are written into the columns `const_image_summary` - this will always be the same result (as always the same seed will be used). The column `3_non-deterministic summary` displays three different answers generated with different seeds, these are most likely different when you run the analysis again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:34:31.180851Z",
     "iopub.status.busy": "2023-05-26T09:34:31.180314Z",
     "iopub.status.idle": "2023-05-26T09:35:08.210237Z",
     "shell.execute_reply": "2023-05-26T09:35:08.209403Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  0%|          | 0.00/2.50G [00:00<?, ?B/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  0%|          | 208k/2.50G [00:00<21:21, 2.10MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  0%|          | 2.72M/2.50G [00:00<02:44, 16.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  0%|          | 12.2M/2.50G [00:00<00:53, 50.2MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  1%|          | 21.2M/2.50G [00:00<00:41, 64.8MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  1%|          | 31.6M/2.50G [00:00<00:33, 80.0MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  2%|▏         | 40.8M/2.50G [00:00<00:30, 85.5MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  2%|▏         | 49.0M/2.50G [00:00<00:31, 84.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  2%|▏         | 58.5M/2.50G [00:00<00:29, 88.8MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  3%|▎         | 67.1M/2.50G [00:00<00:29, 89.4MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  3%|▎         | 76.1M/2.50G [00:01<00:28, 90.7MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  3%|▎         | 85.9M/2.50G [00:01<00:27, 94.4MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  4%|▎         | 95.1M/2.50G [00:01<00:27, 95.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  4%|▍         | 106M/2.50G [00:01<00:25, 101MB/s]  "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  5%|▍         | 117M/2.50G [00:01<00:24, 106MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  5%|▌         | 128M/2.50G [00:01<00:23, 108MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  5%|▌         | 139M/2.50G [00:01<00:23, 110MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  6%|▌         | 150M/2.50G [00:01<00:22, 112MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  6%|▋         | 162M/2.50G [00:01<00:22, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  7%|▋         | 172M/2.50G [00:01<00:22, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  7%|▋         | 183M/2.50G [00:02<00:21, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  8%|▊         | 195M/2.50G [00:02<00:21, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  8%|▊         | 206M/2.50G [00:02<00:21, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  8%|▊         | 217M/2.50G [00:02<00:21, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  9%|▉         | 228M/2.50G [00:02<00:21, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  9%|▉         | 239M/2.50G [00:02<00:21, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 10%|▉         | 251M/2.50G [00:02<00:20, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 10%|█         | 262M/2.50G [00:02<00:20, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 11%|█         | 273M/2.50G [00:02<00:20, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 11%|█         | 285M/2.50G [00:02<00:20, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 12%|█▏        | 296M/2.50G [00:03<00:20, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 12%|█▏        | 307M/2.50G [00:03<00:20, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 12%|█▏        | 319M/2.50G [00:03<00:19, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 13%|█▎        | 330M/2.50G [00:03<00:19, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 13%|█▎        | 341M/2.50G [00:03<00:19, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 14%|█▎        | 352M/2.50G [00:03<00:19, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 14%|█▍        | 363M/2.50G [00:03<00:19, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 15%|█▍        | 375M/2.50G [00:03<00:19, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 15%|█▌        | 386M/2.50G [00:04<00:28, 79.2MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 15%|█▌        | 395M/2.50G [00:04<00:26, 84.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 16%|█▌        | 406M/2.50G [00:04<00:24, 91.8MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 16%|█▋        | 418M/2.50G [00:04<00:22, 98.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 17%|█▋        | 429M/2.50G [00:04<00:21, 104MB/s] "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 17%|█▋        | 440M/2.50G [00:04<00:20, 107MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 18%|█▊        | 451M/2.50G [00:04<00:20, 110MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 18%|█▊        | 463M/2.50G [00:04<00:19, 113MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 18%|█▊        | 474M/2.50G [00:04<00:19, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 19%|█▉        | 485M/2.50G [00:04<00:18, 115MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 19%|█▉        | 496M/2.50G [00:05<00:18, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 20%|█▉        | 507M/2.50G [00:05<00:18, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 20%|██        | 519M/2.50G [00:05<00:18, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 21%|██        | 530M/2.50G [00:05<00:18, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 21%|██        | 541M/2.50G [00:05<00:18, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 22%|██▏       | 552M/2.50G [00:05<00:18, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 22%|██▏       | 563M/2.50G [00:05<00:17, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 22%|██▏       | 575M/2.50G [00:05<00:17, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 23%|██▎       | 586M/2.50G [00:05<00:17, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 23%|██▎       | 597M/2.50G [00:05<00:17, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 24%|██▎       | 608M/2.50G [00:06<00:17, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 24%|██▍       | 620M/2.50G [00:06<00:17, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 25%|██▍       | 631M/2.50G [00:06<00:17, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 25%|██▌       | 642M/2.50G [00:06<00:17, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 25%|██▌       | 653M/2.50G [00:06<00:17, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 26%|██▌       | 664M/2.50G [00:06<00:17, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 26%|██▋       | 676M/2.50G [00:06<00:17, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 27%|██▋       | 687M/2.50G [00:07<00:46, 42.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 27%|██▋       | 698M/2.50G [00:07<00:37, 52.0MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 28%|██▊       | 707M/2.50G [00:07<00:35, 55.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 28%|██▊       | 718M/2.50G [00:07<00:29, 66.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 28%|██▊       | 729M/2.50G [00:07<00:25, 76.9MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 29%|██▉       | 739M/2.50G [00:08<00:34, 55.0MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 29%|██▉       | 749M/2.50G [00:08<00:30, 63.2MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 30%|██▉       | 760M/2.50G [00:08<00:25, 74.0MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 30%|███       | 771M/2.50G [00:08<00:22, 83.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 31%|███       | 782M/2.50G [00:08<00:20, 91.4MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 31%|███       | 792M/2.50G [00:08<00:24, 76.4MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 31%|███▏      | 804M/2.50G [00:08<00:21, 85.8MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 32%|███▏      | 815M/2.50G [00:08<00:19, 93.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 32%|███▏      | 826M/2.50G [00:08<00:18, 99.8MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 33%|███▎      | 837M/2.50G [00:09<00:17, 105MB/s] "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 33%|███▎      | 848M/2.50G [00:09<00:16, 106MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 34%|███▎      | 860M/2.50G [00:09<00:16, 111MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 34%|███▍      | 871M/2.50G [00:09<00:15, 112MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 34%|███▍      | 882M/2.50G [00:09<00:15, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 35%|███▍      | 893M/2.50G [00:09<00:15, 115MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 35%|███▌      | 905M/2.50G [00:09<00:15, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 36%|███▌      | 916M/2.50G [00:09<00:14, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 36%|███▌      | 927M/2.50G [00:09<00:14, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 37%|███▋      | 938M/2.50G [00:09<00:14, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 37%|███▋      | 949M/2.50G [00:10<00:14, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 37%|███▋      | 960M/2.50G [00:10<00:14, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 38%|███▊      | 972M/2.50G [00:10<00:14, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 38%|███▊      | 983M/2.50G [00:10<00:14, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 39%|███▉      | 994M/2.50G [00:10<00:14, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 39%|███▉      | 0.98G/2.50G [00:10<00:13, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 40%|███▉      | 0.99G/2.50G [00:10<00:13, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 40%|████      | 1.00G/2.50G [00:10<00:13, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 41%|████      | 1.02G/2.50G [00:10<00:13, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 41%|████      | 1.03G/2.50G [00:10<00:13, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 41%|████▏     | 1.04G/2.50G [00:11<00:13, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 42%|████▏     | 1.05G/2.50G [00:11<00:13, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 42%|████▏     | 1.06G/2.50G [00:11<00:13, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 43%|████▎     | 1.07G/2.50G [00:11<00:13, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 43%|████▎     | 1.08G/2.50G [00:11<00:13, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 44%|████▎     | 1.09G/2.50G [00:11<00:12, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 44%|████▍     | 1.10G/2.50G [00:11<00:12, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 45%|████▍     | 1.11G/2.50G [00:11<00:12, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 45%|████▍     | 1.13G/2.50G [00:11<00:12, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 45%|████▌     | 1.14G/2.50G [00:11<00:12, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 46%|████▌     | 1.15G/2.50G [00:12<00:12, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 46%|████▋     | 1.16G/2.50G [00:12<00:12, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 47%|████▋     | 1.17G/2.50G [00:12<00:12, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 47%|████▋     | 1.18G/2.50G [00:12<00:12, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 48%|████▊     | 1.19G/2.50G [00:12<00:12, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 48%|████▊     | 1.20G/2.50G [00:12<00:12, 112MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 48%|████▊     | 1.21G/2.50G [00:12<00:12, 113MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 49%|████▉     | 1.22G/2.50G [00:12<00:12, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 49%|████▉     | 1.24G/2.50G [00:12<00:11, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 50%|████▉     | 1.25G/2.50G [00:13<00:11, 115MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 50%|█████     | 1.26G/2.50G [00:13<00:11, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 51%|█████     | 1.27G/2.50G [00:13<00:11, 115MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 51%|█████     | 1.28G/2.50G [00:13<00:11, 115MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 52%|█████▏    | 1.29G/2.50G [00:13<00:17, 75.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 52%|█████▏    | 1.30G/2.50G [00:13<00:15, 84.0MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 52%|█████▏    | 1.31G/2.50G [00:13<00:13, 91.5MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 53%|█████▎    | 1.32G/2.50G [00:13<00:15, 84.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 53%|█████▎    | 1.33G/2.50G [00:14<00:13, 93.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 54%|█████▎    | 1.34G/2.50G [00:14<00:22, 54.8MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 54%|█████▍    | 1.35G/2.50G [00:14<00:19, 63.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 54%|█████▍    | 1.36G/2.50G [00:14<00:16, 73.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 55%|█████▍    | 1.37G/2.50G [00:14<00:14, 83.5MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 55%|█████▌    | 1.38G/2.50G [00:14<00:13, 91.5MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 56%|█████▌    | 1.39G/2.50G [00:15<00:16, 73.8MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 56%|█████▌    | 1.41G/2.50G [00:15<00:13, 85.9MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 57%|█████▋    | 1.42G/2.50G [00:15<00:12, 92.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 57%|█████▋    | 1.43G/2.50G [00:15<00:11, 97.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 57%|█████▋    | 1.44G/2.50G [00:15<00:19, 58.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 58%|█████▊    | 1.45G/2.50G [00:15<00:17, 66.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 58%|█████▊    | 1.46G/2.50G [00:15<00:15, 74.5MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 59%|█████▊    | 1.47G/2.50G [00:16<00:13, 84.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 59%|█████▉    | 1.48G/2.50G [00:16<00:11, 92.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 59%|█████▉    | 1.49G/2.50G [00:16<00:11, 98.7MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 60%|█████▉    | 1.50G/2.50G [00:16<00:10, 104MB/s] "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 60%|██████    | 1.51G/2.50G [00:16<00:09, 108MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 61%|██████    | 1.52G/2.50G [00:16<00:09, 109MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 61%|██████    | 1.53G/2.50G [00:16<00:09, 111MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 62%|██████▏   | 1.54G/2.50G [00:16<00:09, 112MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 62%|██████▏   | 1.55G/2.50G [00:16<00:08, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 63%|██████▎   | 1.57G/2.50G [00:16<00:08, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 63%|██████▎   | 1.58G/2.50G [00:17<00:08, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 63%|██████▎   | 1.59G/2.50G [00:17<00:08, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 64%|██████▍   | 1.60G/2.50G [00:17<00:08, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 64%|██████▍   | 1.61G/2.50G [00:17<00:08, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 65%|██████▍   | 1.62G/2.50G [00:17<00:08, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 65%|██████▌   | 1.63G/2.50G [00:17<00:08, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 66%|██████▌   | 1.64G/2.50G [00:17<00:08, 115MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 66%|██████▌   | 1.65G/2.50G [00:17<00:07, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 66%|██████▋   | 1.66G/2.50G [00:17<00:07, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 67%|██████▋   | 1.67G/2.50G [00:17<00:07, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 67%|██████▋   | 1.69G/2.50G [00:18<00:07, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 68%|██████▊   | 1.70G/2.50G [00:18<00:07, 115MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 68%|██████▊   | 1.71G/2.50G [00:18<00:07, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 69%|██████▊   | 1.72G/2.50G [00:18<00:07, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 69%|██████▉   | 1.73G/2.50G [00:18<00:07, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 70%|██████▉   | 1.74G/2.50G [00:18<00:06, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 70%|██████▉   | 1.75G/2.50G [00:18<00:06, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 70%|███████   | 1.76G/2.50G [00:18<00:06, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 71%|███████   | 1.77G/2.50G [00:18<00:06, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 71%|███████▏  | 1.78G/2.50G [00:18<00:06, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 72%|███████▏  | 1.79G/2.50G [00:19<00:06, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 72%|███████▏  | 1.81G/2.50G [00:19<00:06, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 73%|███████▎  | 1.82G/2.50G [00:19<00:06, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 73%|███████▎  | 1.83G/2.50G [00:19<00:06, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 73%|███████▎  | 1.84G/2.50G [00:19<00:06, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 74%|███████▍  | 1.85G/2.50G [00:19<00:06, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 74%|███████▍  | 1.86G/2.50G [00:19<00:05, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 75%|███████▍  | 1.87G/2.50G [00:19<00:05, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 75%|███████▌  | 1.88G/2.50G [00:19<00:05, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 76%|███████▌  | 1.89G/2.50G [00:19<00:05, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 76%|███████▌  | 1.90G/2.50G [00:20<00:05, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 76%|███████▋  | 1.92G/2.50G [00:20<00:05, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 77%|███████▋  | 1.93G/2.50G [00:20<00:10, 59.4MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 77%|███████▋  | 1.94G/2.50G [00:20<00:08, 69.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 78%|███████▊  | 1.95G/2.50G [00:20<00:07, 79.4MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 78%|███████▊  | 1.96G/2.50G [00:20<00:06, 84.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 79%|███████▊  | 1.97G/2.50G [00:21<00:09, 62.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 79%|███████▉  | 1.98G/2.50G [00:21<00:08, 63.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 79%|███████▉  | 1.98G/2.50G [00:21<00:08, 68.8MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 80%|███████▉  | 1.99G/2.50G [00:21<00:07, 75.0MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 80%|███████▉  | 2.00G/2.50G [00:21<00:06, 83.7MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 80%|████████  | 2.01G/2.50G [00:21<00:08, 60.4MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 81%|████████  | 2.02G/2.50G [00:21<00:07, 71.4MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 81%|████████  | 2.03G/2.50G [00:22<00:06, 81.9MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 82%|████████▏ | 2.04G/2.50G [00:22<00:05, 89.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 82%|████████▏ | 2.05G/2.50G [00:22<00:07, 60.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 82%|████████▏ | 2.06G/2.50G [00:22<00:06, 68.2MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 83%|████████▎ | 2.07G/2.50G [00:22<00:05, 78.9MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 83%|████████▎ | 2.08G/2.50G [00:22<00:05, 87.8MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 84%|████████▎ | 2.09G/2.50G [00:22<00:04, 95.0MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 84%|████████▍ | 2.11G/2.50G [00:22<00:04, 101MB/s] "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 85%|████████▍ | 2.12G/2.50G [00:23<00:03, 105MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 85%|████████▍ | 2.13G/2.50G [00:23<00:03, 108MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 85%|████████▌ | 2.14G/2.50G [00:23<00:03, 111MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 86%|████████▌ | 2.15G/2.50G [00:23<00:03, 113MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 86%|████████▋ | 2.16G/2.50G [00:23<00:03, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 87%|████████▋ | 2.17G/2.50G [00:23<00:03, 115MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 87%|████████▋ | 2.18G/2.50G [00:23<00:02, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 88%|████████▊ | 2.19G/2.50G [00:23<00:02, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 88%|████████▊ | 2.20G/2.50G [00:23<00:02, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 88%|████████▊ | 2.22G/2.50G [00:23<00:02, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 89%|████████▉ | 2.23G/2.50G [00:24<00:02, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 89%|████████▉ | 2.24G/2.50G [00:24<00:02, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 90%|████████▉ | 2.25G/2.50G [00:24<00:02, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 90%|█████████ | 2.26G/2.50G [00:24<00:02, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 91%|█████████ | 2.27G/2.50G [00:24<00:02, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 91%|█████████ | 2.28G/2.50G [00:24<00:02, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 92%|█████████▏| 2.29G/2.50G [00:24<00:01, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 92%|█████████▏| 2.30G/2.50G [00:24<00:01, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 92%|█████████▏| 2.31G/2.50G [00:24<00:01, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 93%|█████████▎| 2.32G/2.50G [00:24<00:01, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 93%|█████████▎| 2.34G/2.50G [00:25<00:01, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 94%|█████████▎| 2.35G/2.50G [00:25<00:01, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 94%|█████████▍| 2.36G/2.50G [00:25<00:01, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 95%|█████████▍| 2.37G/2.50G [00:25<00:01, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 95%|█████████▌| 2.38G/2.50G [00:25<00:01, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 96%|█████████▌| 2.39G/2.50G [00:25<00:01, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 96%|█████████▌| 2.40G/2.50G [00:25<00:00, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 96%|█████████▋| 2.41G/2.50G [00:25<00:00, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 97%|█████████▋| 2.42G/2.50G [00:25<00:00, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 97%|█████████▋| 2.43G/2.50G [00:25<00:00, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 98%|█████████▊| 2.45G/2.50G [00:26<00:00, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 98%|█████████▊| 2.46G/2.50G [00:26<00:00, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 99%|█████████▊| 2.47G/2.50G [00:26<00:00, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 99%|█████████▉| 2.48G/2.50G [00:26<00:00, 116MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 99%|█████████▉| 2.49G/2.50G [00:26<00:00, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "100%|█████████▉| 2.50G/2.50G [00:26<00:00, 109MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "100%|██████████| 2.50G/2.50G [00:26<00:00, 101MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "obj = sm.SummaryDetector(mydict)\n",
    "summary_model, summary_vis_processors = obj.load_model(model_type=\"base\")\n",
    "# summary_model, summary_vis_processors = mutils.load_model(\"large\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:35:08.214704Z",
     "iopub.status.busy": "2023-05-26T09:35:08.214273Z",
     "iopub.status.idle": "2023-05-26T09:35:41.235078Z",
     "shell.execute_reply": "2023-05-26T09:35:41.234360Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "for key in mydict:\n",
    "    mydict[key] = sm.SummaryDetector(mydict[key]).analyse_image(\n",
    "        summary_model=summary_model, summary_vis_processors=summary_vis_processors\n",
    "    )"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "Convert the dictionary of dictionarys into a dictionary with lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:35:41.238872Z",
     "iopub.status.busy": "2023-05-26T09:35:41.238162Z",
     "iopub.status.idle": "2023-05-26T09:35:41.242202Z",
     "shell.execute_reply": "2023-05-26T09:35:41.241687Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "outdict = mutils.append_data_to_dict(mydict)\n",
    "df = mutils.dump_df(outdict)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:35:41.245065Z",
     "iopub.status.busy": "2023-05-26T09:35:41.244505Z",
     "iopub.status.idle": "2023-05-26T09:35:41.256710Z",
     "shell.execute_reply": "2023-05-26T09:35:41.256049Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>filename</th>\n",
       "      <th>const_image_summary</th>\n",
       "      <th>3_non-deterministic summary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>data/102730_eng.png</td>\n",
       "      <td>two people in blue coats spray disinfection a van</td>\n",
       "      <td>[two people in blue robes spraying geranis, tw...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>data/102141_2_eng.png</td>\n",
       "      <td>a collage of images including a corona sign, a...</td>\n",
       "      <td>[the corona vaccine is being released from cor...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>data/106349S_por.png</td>\n",
       "      <td>a man wearing a face mask while looking at a c...</td>\n",
       "      <td>[a man in a white shirt is on a news desk, a m...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                filename                                const_image_summary   \n",
       "0    data/102730_eng.png  two people in blue coats spray disinfection a van  \\\n",
       "1  data/102141_2_eng.png  a collage of images including a corona sign, a...   \n",
       "2   data/106349S_por.png  a man wearing a face mask while looking at a c...   \n",
       "\n",
       "                         3_non-deterministic summary  \n",
       "0  [two people in blue robes spraying geranis, tw...  \n",
       "1  [the corona vaccine is being released from cor...  \n",
       "2  [a man in a white shirt is on a news desk, a m...  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(10)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Write the csv file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:35:41.259680Z",
     "iopub.status.busy": "2023-05-26T09:35:41.259247Z",
     "iopub.status.idle": "2023-05-26T09:35:41.264127Z",
     "shell.execute_reply": "2023-05-26T09:35:41.263539Z"
    }
   },
   "outputs": [],
   "source": [
    "df.to_csv(\"data_out.csv\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Manually inspect the summaries\n",
    "\n",
    "To check the analysis, you can inspect the analyzed elements here. Loading the results takes a moment, so please be patient. If you are sure of what you are doing.\n",
    "\n",
    "`const_image_summary` - the permanent summarys, which does not change from run to run (analyse_image).\n",
    "\n",
    "`3_non-deterministic summary` - 3 different summarys examples that change from run to run (analyse_image). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:35:41.266914Z",
     "iopub.status.busy": "2023-05-26T09:35:41.266476Z",
     "iopub.status.idle": "2023-05-26T09:35:41.292442Z",
     "shell.execute_reply": "2023-05-26T09:35:41.291849Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dash is running on http://127.0.0.1:8055/\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:dash.dash:Dash is running on http://127.0.0.1:8055/\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"100%\"\n",
       "            height=\"650\"\n",
       "            src=\"http://127.0.0.1:8055/\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.IFrame at 0x7fb1519bd730>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "analysis_explorer = mdisplay.AnalysisExplorer(mydict, identify=\"summary\")\n",
    "analysis_explorer.run_server(port=8055)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate answers to free-form questions about images written in natural language. "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the list of questions as a list of strings:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:35:41.295577Z",
     "iopub.status.busy": "2023-05-26T09:35:41.295144Z",
     "iopub.status.idle": "2023-05-26T09:35:41.298652Z",
     "shell.execute_reply": "2023-05-26T09:35:41.297666Z"
    }
   },
   "outputs": [],
   "source": [
    "list_of_questions = [\n",
    "    \"How many persons on the picture?\",\n",
    "    \"Are there any politicians in the picture?\",\n",
    "    \"Does the picture show something from medicine?\",\n",
    "]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Explore the analysis using the interface:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:35:41.301756Z",
     "iopub.status.busy": "2023-05-26T09:35:41.301328Z",
     "iopub.status.idle": "2023-05-26T09:35:41.824478Z",
     "shell.execute_reply": "2023-05-26T09:35:41.823781Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dash is running on http://127.0.0.1:8055/\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:dash.dash:Dash is running on http://127.0.0.1:8055/\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"100%\"\n",
       "            height=\"650\"\n",
       "            src=\"http://127.0.0.1:8055/\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.IFrame at 0x7fb14c512790>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "analysis_explorer = mdisplay.AnalysisExplorer(mydict, identify=\"summary\")\n",
    "analysis_explorer.run_server(port=8055)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Or directly analyze for further processing\n",
    "Instead of inspecting each of the images, you can also directly carry out the analysis and export the result into a csv. This may take a while depending on how many images you have loaded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:35:41.828140Z",
     "iopub.status.busy": "2023-05-26T09:35:41.827666Z",
     "iopub.status.idle": "2023-05-26T09:36:39.251261Z",
     "shell.execute_reply": "2023-05-26T09:36:39.250458Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  0%|          | 0.00/1.35G [00:00<?, ?B/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  0%|          | 288k/1.35G [00:00<08:15, 2.92MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  0%|          | 5.23M/1.35G [00:00<00:45, 31.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  1%|          | 10.7M/1.35G [00:00<00:33, 43.2MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  1%|          | 16.1M/1.35G [00:00<00:29, 48.7MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  2%|▏         | 20.8M/1.35G [00:00<00:30, 47.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  2%|▏         | 27.2M/1.35G [00:00<00:26, 53.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  3%|▎         | 35.5M/1.35G [00:00<00:28, 50.2MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  3%|▎         | 42.3M/1.35G [00:00<00:26, 53.4MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  4%|▎         | 51.2M/1.35G [00:01<00:21, 64.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  4%|▍         | 57.6M/1.35G [00:01<00:22, 62.9MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  5%|▍         | 64.0M/1.35G [00:01<00:26, 52.0MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  5%|▌         | 72.0M/1.35G [00:01<00:28, 48.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  6%|▌         | 80.0M/1.35G [00:01<00:29, 46.2MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  6%|▋         | 88.0M/1.35G [00:01<00:29, 46.2MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  7%|▋         | 97.2M/1.35G [00:01<00:23, 56.2MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  8%|▊         | 106M/1.35G [00:02<00:21, 63.3MB/s] "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  8%|▊         | 116M/1.35G [00:02<00:17, 74.0MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  9%|▉         | 124M/1.35G [00:02<00:17, 73.2MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 10%|▉         | 131M/1.35G [00:02<00:18, 72.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 10%|█         | 143M/1.35G [00:02<00:15, 85.9MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 11%|█▏        | 155M/1.35G [00:02<00:13, 98.5MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 12%|█▏        | 168M/1.35G [00:02<00:11, 107MB/s] "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 13%|█▎        | 180M/1.35G [00:02<00:11, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 14%|█▍        | 192M/1.35G [00:02<00:11, 112MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 15%|█▍        | 203M/1.35G [00:03<00:11, 108MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 15%|█▌        | 213M/1.35G [00:03<00:12, 99.5MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 16%|█▌        | 224M/1.35G [00:03<00:17, 69.5MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 17%|█▋        | 232M/1.35G [00:03<00:24, 49.4MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 18%|█▊        | 245M/1.35G [00:03<00:18, 63.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 18%|█▊        | 253M/1.35G [00:04<00:18, 64.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 19%|█▉        | 265M/1.35G [00:04<00:14, 78.0MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 20%|██        | 278M/1.35G [00:04<00:12, 90.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 21%|██        | 288M/1.35G [00:04<00:18, 60.8MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 22%|██▏       | 301M/1.35G [00:04<00:15, 73.9MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 22%|██▏       | 310M/1.35G [00:04<00:19, 58.7MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 23%|██▎       | 318M/1.35G [00:05<00:19, 57.8MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 24%|██▍       | 329M/1.35G [00:05<00:16, 68.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 25%|██▍       | 341M/1.35G [00:05<00:13, 80.9MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 26%|██▌       | 354M/1.35G [00:05<00:11, 92.7MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 27%|██▋       | 367M/1.35G [00:05<00:10, 103MB/s] "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 27%|██▋       | 379M/1.35G [00:05<00:09, 110MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 28%|██▊       | 390M/1.35G [00:05<00:09, 106MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 29%|██▉       | 403M/1.35G [00:05<00:09, 112MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 30%|███       | 415M/1.35G [00:05<00:08, 118MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 31%|███       | 428M/1.35G [00:05<00:08, 121MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 32%|███▏      | 440M/1.35G [00:06<00:07, 124MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 33%|███▎      | 453M/1.35G [00:06<00:07, 126MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 34%|███▍      | 466M/1.35G [00:06<00:07, 128MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 35%|███▍      | 478M/1.35G [00:06<00:07, 129MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 36%|███▌      | 490M/1.35G [00:06<00:08, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 36%|███▋      | 502M/1.35G [00:06<00:07, 115MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 37%|███▋      | 514M/1.35G [00:06<00:08, 106MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 38%|███▊      | 526M/1.35G [00:06<00:08, 111MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 39%|███▉      | 538M/1.35G [00:06<00:07, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 40%|███▉      | 551M/1.35G [00:07<00:07, 121MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 41%|████      | 563M/1.35G [00:07<00:06, 123MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 42%|████▏     | 576M/1.35G [00:07<00:06, 126MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 43%|████▎     | 589M/1.35G [00:07<00:06, 128MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 44%|████▎     | 601M/1.35G [00:07<00:07, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 44%|████▍     | 612M/1.35G [00:07<00:08, 91.2MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 45%|████▌     | 624M/1.35G [00:07<00:10, 72.5MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 46%|████▌     | 636M/1.35G [00:08<00:09, 82.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 47%|████▋     | 645M/1.35G [00:08<00:09, 77.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 47%|████▋     | 655M/1.35G [00:08<00:09, 81.9MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 48%|████▊     | 667M/1.35G [00:08<00:07, 94.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 49%|████▉     | 680M/1.35G [00:08<00:07, 103MB/s] "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 50%|█████     | 690M/1.35G [00:08<00:07, 93.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 51%|█████     | 703M/1.35G [00:08<00:06, 103MB/s] "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 52%|█████▏    | 715M/1.35G [00:08<00:06, 111MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 53%|█████▎    | 728M/1.35G [00:08<00:05, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 54%|█████▎    | 741M/1.35G [00:09<00:05, 121MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 55%|█████▍    | 753M/1.35G [00:09<00:05, 124MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 56%|█████▌    | 766M/1.35G [00:09<00:05, 126MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 56%|█████▋    | 778M/1.35G [00:09<00:04, 127MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 57%|█████▋    | 790M/1.35G [00:09<00:04, 128MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 58%|█████▊    | 803M/1.35G [00:09<00:05, 120MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 59%|█████▉    | 815M/1.35G [00:09<00:04, 123MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 60%|██████    | 828M/1.35G [00:09<00:04, 126MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 61%|██████    | 840M/1.35G [00:09<00:05, 99.7MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 62%|██████▏   | 850M/1.35G [00:10<00:06, 85.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 62%|██████▏   | 859M/1.35G [00:10<00:08, 63.8MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 63%|██████▎   | 872M/1.35G [00:10<00:06, 77.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 64%|██████▍   | 885M/1.35G [00:10<00:05, 89.4MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 65%|██████▌   | 897M/1.35G [00:10<00:05, 99.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 66%|██████▌   | 910M/1.35G [00:10<00:04, 108MB/s] "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 67%|██████▋   | 921M/1.35G [00:11<00:07, 66.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 68%|██████▊   | 933M/1.35G [00:11<00:06, 76.7MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 69%|██████▊   | 945M/1.35G [00:11<00:05, 86.4MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 69%|██████▉   | 957M/1.35G [00:11<00:04, 96.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 70%|███████   | 970M/1.35G [00:11<00:04, 105MB/s] "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 71%|███████   | 981M/1.35G [00:11<00:05, 72.3MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 72%|███████▏  | 994M/1.35G [00:11<00:04, 83.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 73%|███████▎  | 0.98G/1.35G [00:12<00:04, 94.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 74%|███████▍  | 1.00G/1.35G [00:12<00:03, 104MB/s] "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 75%|███████▍  | 1.01G/1.35G [00:12<00:03, 112MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 76%|███████▌  | 1.02G/1.35G [00:12<00:03, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 77%|███████▋  | 1.03G/1.35G [00:12<00:02, 117MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 78%|███████▊  | 1.04G/1.35G [00:12<00:02, 121MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 78%|███████▊  | 1.06G/1.35G [00:12<00:02, 124MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 79%|███████▉  | 1.07G/1.35G [00:12<00:02, 127MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 80%|████████  | 1.08G/1.35G [00:12<00:02, 128MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 81%|████████  | 1.09G/1.35G [00:12<00:02, 129MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 82%|████████▏ | 1.11G/1.35G [00:13<00:02, 122MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 83%|████████▎ | 1.12G/1.35G [00:13<00:01, 125MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 84%|████████▍ | 1.13G/1.35G [00:13<00:01, 126MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 85%|████████▍ | 1.14G/1.35G [00:13<00:01, 127MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 86%|████████▌ | 1.15G/1.35G [00:13<00:01, 129MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 87%|████████▋ | 1.17G/1.35G [00:13<00:01, 128MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 87%|████████▋ | 1.18G/1.35G [00:13<00:01, 129MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 88%|████████▊ | 1.19G/1.35G [00:13<00:01, 130MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 89%|████████▉ | 1.20G/1.35G [00:13<00:01, 131MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 90%|█████████ | 1.22G/1.35G [00:14<00:01, 126MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 91%|█████████ | 1.23G/1.35G [00:14<00:01, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 92%|█████████▏| 1.24G/1.35G [00:14<00:00, 119MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 93%|█████████▎| 1.25G/1.35G [00:14<00:01, 85.8MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 94%|█████████▎| 1.26G/1.35G [00:14<00:00, 95.7MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 94%|█████████▍| 1.27G/1.35G [00:14<00:01, 79.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 95%|█████████▌| 1.29G/1.35G [00:14<00:00, 90.6MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 96%|█████████▋| 1.30G/1.35G [00:14<00:00, 98.1MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 97%|█████████▋| 1.31G/1.35G [00:15<00:00, 101MB/s] "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 98%|█████████▊| 1.32G/1.35G [00:15<00:00, 107MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 99%|█████████▉| 1.33G/1.35G [00:15<00:00, 114MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "100%|█████████▉| 1.34G/1.35G [00:15<00:00, 119MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "100%|██████████| 1.35G/1.35G [00:15<00:00, 93.7MB/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "for key in mydict:\n",
    "    mydict[key] = sm.SummaryDetector(mydict[key]).analyse_questions(list_of_questions)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convert to dataframe and write csv\n",
    "These steps are required to convert the dictionary of dictionarys into a dictionary with lists, that can be converted into a pandas dataframe and exported to a csv file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:36:39.255865Z",
     "iopub.status.busy": "2023-05-26T09:36:39.255097Z",
     "iopub.status.idle": "2023-05-26T09:36:39.260950Z",
     "shell.execute_reply": "2023-05-26T09:36:39.260274Z"
    }
   },
   "outputs": [],
   "source": [
    "outdict2 = mutils.append_data_to_dict(mydict)\n",
    "df2 = mutils.dump_df(outdict2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:36:39.264624Z",
     "iopub.status.busy": "2023-05-26T09:36:39.264087Z",
     "iopub.status.idle": "2023-05-26T09:36:39.274051Z",
     "shell.execute_reply": "2023-05-26T09:36:39.273379Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>filename</th>\n",
       "      <th>const_image_summary</th>\n",
       "      <th>3_non-deterministic summary</th>\n",
       "      <th>How many persons on the picture?</th>\n",
       "      <th>Are there any politicians in the picture?</th>\n",
       "      <th>Does the picture show something from medicine?</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>data/102730_eng.png</td>\n",
       "      <td>two people in blue coats spray disinfection a van</td>\n",
       "      <td>[two people in blue robes spraying geranis, tw...</td>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>data/102141_2_eng.png</td>\n",
       "      <td>a collage of images including a corona sign, a...</td>\n",
       "      <td>[the corona vaccine is being released from cor...</td>\n",
       "      <td>1</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>data/106349S_por.png</td>\n",
       "      <td>a man wearing a face mask while looking at a c...</td>\n",
       "      <td>[a man in a white shirt is on a news desk, a m...</td>\n",
       "      <td>1</td>\n",
       "      <td>yes</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                filename                                const_image_summary   \n",
       "0    data/102730_eng.png  two people in blue coats spray disinfection a van  \\\n",
       "1  data/102141_2_eng.png  a collage of images including a corona sign, a...   \n",
       "2   data/106349S_por.png  a man wearing a face mask while looking at a c...   \n",
       "\n",
       "                         3_non-deterministic summary   \n",
       "0  [two people in blue robes spraying geranis, tw...  \\\n",
       "1  [the corona vaccine is being released from cor...   \n",
       "2  [a man in a white shirt is on a news desk, a m...   \n",
       "\n",
       "  How many persons on the picture? Are there any politicians in the picture?   \n",
       "0                                2                                        no  \\\n",
       "1                                1                                        no   \n",
       "2                                1                                       yes   \n",
       "\n",
       "  Does the picture show something from medicine?  \n",
       "0                                            yes  \n",
       "1                                            yes  \n",
       "2                                            yes  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-26T09:36:39.277136Z",
     "iopub.status.busy": "2023-05-26T09:36:39.276608Z",
     "iopub.status.idle": "2023-05-26T09:36:39.281540Z",
     "shell.execute_reply": "2023-05-26T09:36:39.280961Z"
    }
   },
   "outputs": [],
   "source": [
    "df2.to_csv(\"data_out2.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  },
  "vscode": {
   "interpreter": {
    "hash": "f1142466f556ab37fe2d38e2897a16796906208adb09fea90ba58bdf8a56f0ba"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}