locsec/AMMICO
1
Форкнуть 0
зеркало из https://github.com/ssciwr/AMMICO.git synced 2025-10-30 21:46:04 +02:00
Код Задачи Пакеты Проекты Релизы Вики Активность
AMMICO/build/html/notebooks/Example multimodal.ipynb

3140 строки
88 KiB
Plaintext
Исходник Ответственный История

{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "22df2297-0629-45aa-b88c-6c61f1544db6",
"metadata": {},
"source": [
"# Image Multimodal Search"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "9eeeb302-296e-48dc-86c7-254aa02f2b3a",
"metadata": {},
"source": [
"This notebooks shows how to carry out an image multimodal search with the [LAVIS](https://github.com/salesforce/LAVIS) library. \n",
"\n",
"The first cell is only run on google colab and installs the [ammico](https://github.com/ssciwr/AMMICO) package.\n",
"\n",
"After that, we can import `ammico` and read in the files given a folder path."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "0b0a6bdf",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:38:41.724056Z",
"iopub.status.busy": "2023-09-20T09:38:41.723186Z",
"iopub.status.idle": "2023-09-20T09:38:41.733620Z",
"shell.execute_reply": "2023-09-20T09:38:41.732902Z"
}
},
"outputs": [],
"source": [
"# if running on google colab\n",
"# flake8-noqa-cell\n",
"import os\n",
"\n",
"if \"google.colab\" in str(get_ipython()):\n",
" # update python version\n",
" # install setuptools\n",
" # %pip install setuptools==61 -qqq\n",
" # install ammico\n",
" %pip install git+https://github.com/ssciwr/ammico.git -qqq\n",
" # mount google drive for data and API key\n",
" from google.colab import drive\n",
"\n",
" drive.mount(\"/content/drive\")"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "f10ad6c9-b1a0-4043-8c5d-ed660d77be37",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:38:41.737259Z",
"iopub.status.busy": "2023-09-20T09:38:41.736631Z",
"iopub.status.idle": "2023-09-20T09:38:54.598500Z",
"shell.execute_reply": "2023-09-20T09:38:54.597595Z"
},
"tags": []
},
"outputs": [],
"source": [
"import ammico.utils as mutils\n",
"import ammico.multimodal_search as ms"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "8d3fe589-ff3c-4575-b8f5-650db85596bc",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:38:54.602969Z",
"iopub.status.busy": "2023-09-20T09:38:54.602048Z",
"iopub.status.idle": "2023-09-20T09:38:54.609315Z",
"shell.execute_reply": "2023-09-20T09:38:54.608641Z"
},
"tags": []
},
"outputs": [],
"source": [
"images = mutils.find_files(\n",
" path=\"data/\",\n",
" limit=10,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "a08bd3a9-e954-4a0e-ad64-6817abd3a25a",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:38:54.613215Z",
"iopub.status.busy": "2023-09-20T09:38:54.612580Z",
"iopub.status.idle": "2023-09-20T09:38:54.620427Z",
"shell.execute_reply": "2023-09-20T09:38:54.619791Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"{'102141_2_eng': {'filename': 'data/102141_2_eng.png'},\n",
" '102730_eng': {'filename': 'data/102730_eng.png'},\n",
" '106349S_por': {'filename': 'data/106349S_por.png'}}"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"images"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "adf3db21-1f8b-4d44-bbef-ef0acf4623a0",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:38:54.624158Z",
"iopub.status.busy": "2023-09-20T09:38:54.623515Z",
"iopub.status.idle": "2023-09-20T09:38:54.628579Z",
"shell.execute_reply": "2023-09-20T09:38:54.627870Z"
},
"tags": []
},
"outputs": [],
"source": [
"mydict = mutils.initialize_dict(images)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "4c091f95-07cf-42c3-82c8-5f3a3c5929f8",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:38:54.632349Z",
"iopub.status.busy": "2023-09-20T09:38:54.631695Z",
"iopub.status.idle": "2023-09-20T09:38:54.637009Z",
"shell.execute_reply": "2023-09-20T09:38:54.636190Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"{'102141_2_eng': {'filename': '102141_2_eng'},\n",
" '102730_eng': {'filename': '102730_eng'},\n",
" '106349S_por': {'filename': '106349S_por'}}"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mydict"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "987540a8-d800-4c70-a76b-7bfabaf123fa",
"metadata": {},
"source": [
"## Indexing and extracting features from images in selected folder"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "66d6ede4-00bc-4aeb-9a36-e52d7de33fe5",
"metadata": {},
"source": [
"First you need to select a model. You can choose one of the following models: \n",
"- [blip](https://github.com/salesforce/BLIP)\n",
"- [blip2](https://huggingface.co/docs/transformers/main/model_doc/blip-2) \n",
"- [albef](https://github.com/salesforce/ALBEF) \n",
"- [clip_base](https://github.com/openai/CLIP/blob/main/model-card.md)\n",
"- [clip_vitl14](https://github.com/mlfoundations/open_clip) \n",
"- [clip_vitl14_336](https://github.com/mlfoundations/open_clip)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "7bbca1f0-d4b0-43cd-8e05-ee39d37c328e",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:38:54.640495Z",
"iopub.status.busy": "2023-09-20T09:38:54.639893Z",
"iopub.status.idle": "2023-09-20T09:38:54.643574Z",
"shell.execute_reply": "2023-09-20T09:38:54.642805Z"
},
"tags": []
},
"outputs": [],
"source": [
"model_type = \"blip\"\n",
"# model_type = \"blip2\"\n",
"# model_type = \"albef\"\n",
"# model_type = \"clip_base\"\n",
"# model_type = \"clip_vitl14\"\n",
"# model_type = \"clip_vitl14_336\""
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "357828c9",
"metadata": {},
"source": [
"To process the loaded images using the selected model, use the below code:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "f6f2c9b1-4a91-47cb-86b5-2c9c67e4837b",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:38:54.647044Z",
"iopub.status.busy": "2023-09-20T09:38:54.646533Z",
"iopub.status.idle": "2023-09-20T09:38:54.651422Z",
"shell.execute_reply": "2023-09-20T09:38:54.650713Z"
}
},
"outputs": [],
"source": [
"my_obj = ms.MultimodalSearch(mydict)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "16603ded-078e-4362-847b-57ad76829327",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:38:54.654984Z",
"iopub.status.busy": "2023-09-20T09:38:54.654311Z",
"iopub.status.idle": "2023-09-20T09:38:54.661028Z",
"shell.execute_reply": "2023-09-20T09:38:54.660265Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"{'102141_2_eng': {'filename': '102141_2_eng'},\n",
" '102730_eng': {'filename': '102730_eng'},\n",
" '106349S_por': {'filename': '106349S_por'}}"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"my_obj.subdict"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "ca095404-57d0-4f5d-aeb0-38c232252b17",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:38:54.664556Z",
"iopub.status.busy": "2023-09-20T09:38:54.664031Z",
"iopub.status.idle": "2023-09-20T09:39:35.235304Z",
"shell.execute_reply": "2023-09-20T09:39:35.234295Z"
},
"tags": []
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
"Downloading (…)solve/main/vocab.txt: 0%| | 0.00/232k [00:00<?, ?B/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
"Downloading (…)solve/main/vocab.txt: 100%|██████████| 232k/232k [00:00<00:00, 4.02MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
"Downloading (…)okenizer_config.json: 0%| | 0.00/28.0 [00:00<?, ?B/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
"Downloading (…)okenizer_config.json: 100%|██████████| 28.0/28.0 [00:00<00:00, 3.32kB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
"Downloading (…)lve/main/config.json: 0%| | 0.00/570 [00:00<?, ?B/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
"Downloading (…)lve/main/config.json: 100%|██████████| 570/570 [00:00<00:00, 279kB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 0%| | 0.00/1.97G [00:00<?, ?B/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 0%| | 4.30M/1.97G [00:00<00:46, 44.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 0%| | 8.98M/1.97G [00:00<00:44, 47.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 1%| | 16.0M/1.97G [00:00<00:44, 47.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 1%| | 24.0M/1.97G [00:00<00:37, 55.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 2%|▏ | 31.7M/1.97G [00:00<00:33, 63.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 2%|▏ | 37.8M/1.97G [00:00<00:34, 59.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 2%|▏ | 45.0M/1.97G [00:00<00:32, 63.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 3%|▎ | 51.2M/1.97G [00:00<00:33, 60.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 3%|▎ | 57.1M/1.97G [00:01<00:34, 60.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 3%|▎ | 64.1M/1.97G [00:01<00:32, 63.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 3%|▎ | 70.3M/1.97G [00:01<00:32, 63.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 4%|▍ | 76.4M/1.97G [00:01<00:34, 59.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 4%|▍ | 82.2M/1.97G [00:01<00:37, 54.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 4%|▍ | 88.1M/1.97G [00:01<00:35, 56.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 5%|▍ | 93.6M/1.97G [00:01<00:38, 52.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 5%|▍ | 100M/1.97G [00:01<00:35, 56.5MB/s] "
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 5%|▌ | 108M/1.97G [00:01<00:31, 62.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 6%|▌ | 114M/1.97G [00:02<00:34, 57.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 6%|▌ | 120M/1.97G [00:02<00:32, 60.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 6%|▋ | 128M/1.97G [00:02<00:29, 66.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 7%|▋ | 136M/1.97G [00:02<00:28, 69.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 7%|▋ | 142M/1.97G [00:02<00:30, 64.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 7%|▋ | 149M/1.97G [00:02<00:29, 66.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 8%|▊ | 155M/1.97G [00:02<00:32, 59.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 8%|▊ | 164M/1.97G [00:02<00:29, 66.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 9%|▊ | 172M/1.97G [00:02<00:26, 72.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 9%|▉ | 180M/1.97G [00:03<00:25, 74.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 9%|▉ | 189M/1.97G [00:03<00:24, 79.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 10%|▉ | 197M/1.97G [00:03<00:23, 80.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 10%|█ | 204M/1.97G [00:03<00:25, 75.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 11%|█ | 213M/1.97G [00:03<00:23, 80.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 11%|█ | 221M/1.97G [00:03<00:24, 78.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 11%|█▏ | 229M/1.97G [00:03<00:25, 75.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 12%|█▏ | 237M/1.97G [00:03<00:23, 78.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 12%|█▏ | 245M/1.97G [00:03<00:27, 67.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 13%|█▎ | 253M/1.97G [00:04<00:25, 71.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 13%|█▎ | 261M/1.97G [00:04<00:23, 76.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 13%|█▎ | 269M/1.97G [00:04<00:24, 75.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 14%|█▍ | 278M/1.97G [00:04<00:22, 80.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 14%|█▍ | 286M/1.97G [00:04<00:23, 77.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 15%|█▍ | 295M/1.97G [00:04<00:21, 82.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 15%|█▌ | 303M/1.97G [00:04<00:24, 73.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 15%|█▌ | 310M/1.97G [00:04<00:24, 73.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 16%|█▌ | 319M/1.97G [00:04<00:22, 78.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 16%|█▌ | 326M/1.97G [00:05<00:23, 76.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 17%|█▋ | 334M/1.97G [00:05<00:23, 75.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 17%|█▋ | 341M/1.97G [00:05<00:22, 77.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 17%|█▋ | 349M/1.97G [00:05<00:23, 75.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 18%|█▊ | 357M/1.97G [00:05<00:21, 79.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 18%|█▊ | 366M/1.97G [00:05<00:21, 81.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 19%|█▊ | 374M/1.97G [00:05<00:22, 78.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 19%|█▉ | 381M/1.97G [00:05<00:25, 67.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 19%|█▉ | 389M/1.97G [00:05<00:23, 72.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 20%|█▉ | 396M/1.97G [00:06<00:24, 69.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 20%|█▉ | 403M/1.97G [00:06<00:25, 67.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 20%|██ | 410M/1.97G [00:06<00:24, 68.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 21%|██ | 418M/1.97G [00:06<00:23, 72.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 21%|██ | 425M/1.97G [00:06<00:24, 67.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 21%|██▏ | 432M/1.97G [00:06<00:24, 67.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 22%|██▏ | 439M/1.97G [00:06<00:24, 68.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 22%|██▏ | 446M/1.97G [00:06<00:23, 71.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 22%|██▏ | 453M/1.97G [00:06<00:23, 69.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 23%|██▎ | 460M/1.97G [00:06<00:23, 70.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 23%|██▎ | 469M/1.97G [00:07<00:21, 76.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 24%|██▎ | 476M/1.97G [00:07<00:21, 74.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 24%|██▍ | 483M/1.97G [00:07<00:22, 71.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 24%|██▍ | 492M/1.97G [00:07<00:20, 77.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 25%|██▍ | 500M/1.97G [00:07<00:20, 78.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 25%|██▌ | 507M/1.97G [00:07<00:21, 74.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 25%|██▌ | 515M/1.97G [00:07<00:21, 74.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 26%|██▌ | 522M/1.97G [00:07<00:22, 70.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 26%|██▋ | 531M/1.97G [00:07<00:19, 78.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 27%|██▋ | 540M/1.97G [00:08<00:18, 83.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 27%|██▋ | 548M/1.97G [00:08<00:20, 74.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 28%|██▊ | 556M/1.97G [00:08<00:20, 75.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 28%|██▊ | 563M/1.97G [00:08<00:20, 74.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 28%|██▊ | 570M/1.97G [00:08<00:21, 71.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 29%|██▊ | 578M/1.97G [00:08<00:20, 73.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 29%|██▉ | 585M/1.97G [00:08<00:20, 73.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 29%|██▉ | 592M/1.97G [00:08<00:20, 71.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 30%|██▉ | 600M/1.97G [00:08<00:20, 73.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 30%|███ | 607M/1.97G [00:09<00:21, 69.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 30%|███ | 615M/1.97G [00:09<00:20, 73.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 31%|███ | 622M/1.97G [00:09<00:20, 70.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 31%|███ | 629M/1.97G [00:09<00:20, 72.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 32%|███▏ | 636M/1.97G [00:09<00:20, 69.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 32%|███▏ | 645M/1.97G [00:09<00:18, 76.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 32%|███▏ | 654M/1.97G [00:09<00:17, 80.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 33%|███▎ | 662M/1.97G [00:09<00:17, 82.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 33%|███▎ | 670M/1.97G [00:09<00:17, 79.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 34%|███▎ | 679M/1.97G [00:10<00:17, 82.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 34%|███▍ | 686M/1.97G [00:10<00:17, 79.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 34%|███▍ | 696M/1.97G [00:10<00:16, 84.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 35%|███▍ | 704M/1.97G [00:10<00:16, 84.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 35%|███▌ | 712M/1.97G [00:10<00:19, 68.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 36%|███▌ | 720M/1.97G [00:10<00:18, 71.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 36%|███▌ | 728M/1.97G [00:10<00:19, 71.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 36%|███▋ | 736M/1.97G [00:10<00:17, 75.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 37%|███▋ | 744M/1.97G [00:10<00:17, 74.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 37%|███▋ | 751M/1.97G [00:11<00:18, 71.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 38%|███▊ | 758M/1.97G [00:11<00:51, 25.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 38%|███▊ | 771M/1.97G [00:11<00:33, 38.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 39%|███▊ | 778M/1.97G [00:12<00:31, 41.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 39%|███▉ | 785M/1.97G [00:12<00:29, 43.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 39%|███▉ | 793M/1.97G [00:12<00:24, 52.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 40%|███▉ | 800M/1.97G [00:12<00:23, 55.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 40%|████ | 808M/1.97G [00:12<00:20, 61.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 40%|████ | 815M/1.97G [00:12<00:20, 60.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 41%|████ | 822M/1.97G [00:12<00:19, 63.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 41%|████ | 829M/1.97G [00:12<00:19, 65.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 41%|████▏ | 835M/1.97G [00:12<00:18, 66.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 42%|████▏ | 842M/1.97G [00:13<00:19, 63.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 42%|████▏ | 849M/1.97G [00:13<00:18, 65.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 42%|████▏ | 856M/1.97G [00:13<00:17, 68.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 43%|████▎ | 864M/1.97G [00:13<00:17, 68.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 43%|████▎ | 872M/1.97G [00:13<00:16, 70.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 44%|████▎ | 879M/1.97G [00:13<00:16, 70.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 44%|████▍ | 888M/1.97G [00:13<00:15, 77.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 44%|████▍ | 896M/1.97G [00:13<00:16, 69.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 45%|████▍ | 904M/1.97G [00:13<00:16, 72.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 45%|████▌ | 912M/1.97G [00:14<00:15, 75.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 46%|████▌ | 919M/1.97G [00:14<00:15, 76.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 46%|████▌ | 928M/1.97G [00:14<00:13, 82.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 46%|████▋ | 937M/1.97G [00:14<00:13, 83.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 47%|████▋ | 945M/1.97G [00:14<00:13, 81.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 47%|████▋ | 953M/1.97G [00:14<00:14, 79.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 48%|████▊ | 960M/1.97G [00:14<00:13, 79.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 48%|████▊ | 968M/1.97G [00:14<00:13, 78.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 48%|████▊ | 977M/1.97G [00:14<00:12, 84.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 49%|████▉ | 985M/1.97G [00:14<00:13, 82.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 49%|████▉ | 993M/1.97G [00:15<00:12, 83.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 50%|████▉ | 0.98G/1.97G [00:15<00:14, 75.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 50%|█████ | 0.99G/1.97G [00:15<00:13, 80.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 50%|█████ | 0.99G/1.97G [00:15<00:13, 75.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 51%|█████ | 1.00G/1.97G [00:15<00:13, 79.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 51%|█████ | 1.01G/1.97G [00:15<00:14, 72.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 52%|█████▏ | 1.02G/1.97G [00:15<00:13, 76.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 52%|█████▏ | 1.03G/1.97G [00:15<00:13, 77.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 52%|█████▏ | 1.03G/1.97G [00:15<00:12, 81.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 53%|█████▎ | 1.04G/1.97G [00:16<00:11, 84.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 53%|█████▎ | 1.05G/1.97G [00:16<00:11, 85.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 54%|█████▎ | 1.06G/1.97G [00:16<00:12, 81.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 54%|█████▍ | 1.07G/1.97G [00:16<00:11, 82.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 55%|█████▍ | 1.07G/1.97G [00:16<00:11, 81.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 55%|█████▍ | 1.08G/1.97G [00:16<00:11, 82.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 55%|█████▌ | 1.09G/1.97G [00:16<00:12, 77.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 56%|█████▌ | 1.10G/1.97G [00:16<00:12, 77.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 56%|█████▌ | 1.11G/1.97G [00:16<00:11, 79.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 57%|█████▋ | 1.11G/1.97G [00:17<00:11, 82.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 57%|█████▋ | 1.12G/1.97G [00:17<00:13, 66.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 57%|█████▋ | 1.13G/1.97G [00:17<00:13, 68.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 58%|█████▊ | 1.14G/1.97G [00:17<00:14, 63.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 58%|█████▊ | 1.14G/1.97G [00:17<00:14, 62.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 58%|█████▊ | 1.15G/1.97G [00:17<00:13, 66.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 59%|█████▊ | 1.16G/1.97G [00:17<00:12, 70.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 59%|█████▉ | 1.16G/1.97G [00:17<00:11, 75.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 59%|█████▉ | 1.17G/1.97G [00:17<00:11, 74.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 60%|█████▉ | 1.18G/1.97G [00:18<00:10, 77.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 60%|██████ | 1.19G/1.97G [00:18<00:10, 79.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 61%|██████ | 1.20G/1.97G [00:18<00:10, 79.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 61%|██████ | 1.20G/1.97G [00:18<00:11, 72.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 61%|██████▏ | 1.21G/1.97G [00:18<00:10, 76.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 62%|██████▏ | 1.22G/1.97G [00:18<00:10, 78.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 62%|██████▏ | 1.23G/1.97G [00:18<00:10, 72.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 63%|██████▎ | 1.23G/1.97G [00:18<00:11, 71.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 63%|██████▎ | 1.24G/1.97G [00:18<00:10, 72.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 63%|██████▎ | 1.25G/1.97G [00:19<00:11, 69.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 64%|██████▍ | 1.26G/1.97G [00:19<00:10, 74.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 64%|██████▍ | 1.26G/1.97G [00:19<00:10, 73.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 64%|██████▍ | 1.27G/1.97G [00:19<00:10, 73.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 65%|██████▍ | 1.28G/1.97G [00:19<00:09, 74.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 65%|██████▌ | 1.28G/1.97G [00:19<00:09, 75.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 66%|██████▌ | 1.29G/1.97G [00:19<00:10, 72.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 66%|██████▌ | 1.30G/1.97G [00:19<00:09, 75.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 66%|██████▋ | 1.31G/1.97G [00:19<00:09, 75.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 67%|██████▋ | 1.31G/1.97G [00:19<00:09, 76.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 67%|██████▋ | 1.32G/1.97G [00:20<00:08, 78.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 67%|██████▋ | 1.33G/1.97G [00:20<00:08, 78.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 68%|██████▊ | 1.34G/1.97G [00:20<00:09, 75.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 68%|██████▊ | 1.34G/1.97G [00:20<00:08, 77.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 69%|██████▊ | 1.35G/1.97G [00:20<00:08, 78.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 69%|██████▉ | 1.36G/1.97G [00:20<00:08, 81.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 69%|██████▉ | 1.37G/1.97G [00:20<00:07, 84.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 70%|██████▉ | 1.38G/1.97G [00:20<00:07, 84.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 70%|███████ | 1.38G/1.97G [00:20<00:08, 74.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 71%|███████ | 1.39G/1.97G [00:21<00:07, 78.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 71%|███████ | 1.40G/1.97G [00:21<00:07, 81.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 72%|███████▏ | 1.41G/1.97G [00:21<00:07, 79.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 72%|███████▏ | 1.42G/1.97G [00:21<00:07, 82.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 72%|███████▏ | 1.43G/1.97G [00:21<00:07, 81.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 73%|███████▎ | 1.43G/1.97G [00:21<00:07, 81.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 73%|███████▎ | 1.44G/1.97G [00:21<00:07, 73.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 73%|███████▎ | 1.45G/1.97G [00:21<00:07, 73.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 74%|███████▍ | 1.46G/1.97G [00:21<00:07, 75.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 74%|███████▍ | 1.46G/1.97G [00:22<00:07, 71.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 75%|███████▍ | 1.47G/1.97G [00:22<00:07, 72.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 75%|███████▌ | 1.48G/1.97G [00:22<00:06, 79.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 75%|███████▌ | 1.49G/1.97G [00:22<00:07, 69.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 76%|███████▌ | 1.50G/1.97G [00:22<00:06, 76.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 76%|███████▋ | 1.50G/1.97G [00:22<00:06, 78.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 77%|███████▋ | 1.51G/1.97G [00:22<00:06, 80.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 77%|███████▋ | 1.52G/1.97G [00:22<00:05, 83.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 77%|███████▋ | 1.53G/1.97G [00:22<00:06, 77.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 78%|███████▊ | 1.54G/1.97G [00:23<00:13, 34.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 79%|███████▊ | 1.55G/1.97G [00:23<00:09, 48.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 79%|███████▉ | 1.56G/1.97G [00:23<00:08, 51.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 79%|███████▉ | 1.56G/1.97G [00:23<00:07, 54.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 80%|███████▉ | 1.57G/1.97G [00:24<00:08, 52.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 80%|████████ | 1.58G/1.97G [00:24<00:07, 58.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 80%|████████ | 1.59G/1.97G [00:24<00:06, 61.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 81%|████████ | 1.59G/1.97G [00:24<00:06, 64.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 81%|████████ | 1.60G/1.97G [00:24<00:06, 63.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 81%|████████▏ | 1.60G/1.97G [00:24<00:06, 63.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 82%|████████▏ | 1.61G/1.97G [00:24<00:05, 65.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 82%|████████▏ | 1.62G/1.97G [00:24<00:05, 72.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 83%|████████▎ | 1.63G/1.97G [00:24<00:05, 72.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 83%|████████▎ | 1.63G/1.97G [00:24<00:05, 70.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 83%|████████▎ | 1.64G/1.97G [00:25<00:04, 71.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 84%|████████▎ | 1.65G/1.97G [00:25<00:04, 72.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 84%|████████▍ | 1.66G/1.97G [00:25<00:04, 72.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 84%|████████▍ | 1.66G/1.97G [00:25<00:04, 68.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 85%|████████▍ | 1.67G/1.97G [00:25<00:04, 75.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 85%|████████▌ | 1.68G/1.97G [00:25<00:03, 78.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 86%|████████▌ | 1.69G/1.97G [00:25<00:04, 76.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 86%|████████▌ | 1.69G/1.97G [00:25<00:03, 77.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 86%|████████▋ | 1.70G/1.97G [00:25<00:03, 78.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 87%|████████▋ | 1.71G/1.97G [00:26<00:04, 67.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 87%|████████▋ | 1.72G/1.97G [00:26<00:03, 72.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 87%|████████▋ | 1.72G/1.97G [00:26<00:03, 74.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 88%|████████▊ | 1.73G/1.97G [00:26<00:03, 71.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 88%|████████▊ | 1.74G/1.97G [00:26<00:03, 68.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 89%|████████▊ | 1.75G/1.97G [00:26<00:03, 72.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 89%|████████▉ | 1.75G/1.97G [00:26<00:03, 75.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 89%|████████▉ | 1.76G/1.97G [00:26<00:02, 76.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 90%|████████▉ | 1.77G/1.97G [00:26<00:02, 74.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 90%|█████████ | 1.78G/1.97G [00:27<00:02, 77.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 91%|█████████ | 1.78G/1.97G [00:27<00:02, 78.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 91%|█████████ | 1.79G/1.97G [00:27<00:02, 74.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 91%|█████████▏| 1.80G/1.97G [00:27<00:02, 79.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 92%|█████████▏| 1.81G/1.97G [00:27<00:02, 74.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 92%|█████████▏| 1.81G/1.97G [00:27<00:02, 70.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 92%|█████████▏| 1.82G/1.97G [00:27<00:02, 72.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 93%|█████████▎| 1.83G/1.97G [00:27<00:02, 75.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 93%|█████████▎| 1.84G/1.97G [00:27<00:01, 76.6MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 94%|█████████▎| 1.84G/1.97G [00:27<00:01, 77.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 94%|█████████▍| 1.85G/1.97G [00:28<00:01, 80.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 94%|█████████▍| 1.86G/1.97G [00:28<00:01, 79.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 95%|█████████▍| 1.87G/1.97G [00:28<00:01, 79.8MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 95%|█████████▌| 1.88G/1.97G [00:28<00:01, 85.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 96%|█████████▌| 1.88G/1.97G [00:28<00:01, 85.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 96%|█████████▌| 1.89G/1.97G [00:28<00:00, 87.4MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 96%|█████████▋| 1.90G/1.97G [00:28<00:00, 84.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 97%|█████████▋| 1.91G/1.97G [00:28<00:00, 77.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 97%|█████████▋| 1.92G/1.97G [00:28<00:00, 75.7MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 98%|█████████▊| 1.92G/1.97G [00:29<00:00, 79.1MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 98%|█████████▊| 1.93G/1.97G [00:29<00:00, 79.2MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 98%|█████████▊| 1.94G/1.97G [00:29<00:00, 80.0MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 99%|█████████▉| 1.95G/1.97G [00:29<00:00, 83.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
" 99%|█████████▉| 1.96G/1.97G [00:29<00:00, 84.5MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
"100%|█████████▉| 1.96G/1.97G [00:29<00:00, 83.9MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
"100%|██████████| 1.97G/1.97G [00:29<00:00, 71.3MB/s]"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n"
]
},
{
"ename": "FileNotFoundError",
"evalue": "[Errno 2] No such file or directory: '102141_2_eng'",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mFileNotFoundError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[10], line 8\u001b[0m\n\u001b[1;32m 1\u001b[0m (\n\u001b[1;32m 2\u001b[0m model,\n\u001b[1;32m 3\u001b[0m vis_processors,\n\u001b[1;32m 4\u001b[0m txt_processors,\n\u001b[1;32m 5\u001b[0m image_keys,\n\u001b[1;32m 6\u001b[0m image_names,\n\u001b[1;32m 7\u001b[0m features_image_stacked,\n\u001b[0;32m----> 8\u001b[0m ) \u001b[38;5;241m=\u001b[39m \u001b[43mmy_obj\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mparsing_images\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 9\u001b[0m \u001b[43m \u001b[49m\u001b[43mmodel_type\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\n\u001b[1;32m 10\u001b[0m \u001b[43m \u001b[49m\u001b[43mpath_to_save_tensors\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mdata/\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m 11\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n",
"File \u001b[0;32m~/work/AMMICO/AMMICO/ammico/multimodal_search.py:371\u001b[0m, in \u001b[0;36mMultimodalSearch.parsing_images\u001b[0;34m(self, model_type, path_to_save_tensors, path_to_load_tensors)\u001b[0m\n\u001b[1;32m 366\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 367\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mSyntaxError\u001b[39;00m(\n\u001b[1;32m 368\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mPlease, use one of the following models: blip2, blip, albef, clip_base, clip_vitl14, clip_vitl14_336\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 369\u001b[0m )\n\u001b[0;32m--> 371\u001b[0m _, images_tensors \u001b[38;5;241m=\u001b[39m \u001b[43mMultimodalSearch\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mread_and_process_images\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 372\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mimage_names\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mvis_processors\u001b[49m\n\u001b[1;32m 373\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 374\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m path_to_load_tensors \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m 375\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m torch\u001b[38;5;241m.\u001b[39mno_grad():\n",
"File \u001b[0;32m~/work/AMMICO/AMMICO/ammico/multimodal_search.py:168\u001b[0m, in \u001b[0;36mMultimodalSearch.read_and_process_images\u001b[0;34m(self, image_paths, vis_processor)\u001b[0m\n\u001b[1;32m 156\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mread_and_process_images\u001b[39m(\u001b[38;5;28mself\u001b[39m, image_paths: \u001b[38;5;28mlist\u001b[39m, vis_processor) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m \u001b[38;5;28mtuple\u001b[39m:\n\u001b[1;32m 157\u001b[0m \u001b[38;5;250m \u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m 158\u001b[0m \u001b[38;5;124;03m Read and process images with vis_processor.\u001b[39;00m\n\u001b[1;32m 159\u001b[0m \n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 166\u001b[0m \u001b[38;5;124;03m images_tensors (torch.Tensor): tensors of images stacked in device.\u001b[39;00m\n\u001b[1;32m 167\u001b[0m \u001b[38;5;124;03m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 168\u001b[0m raw_images \u001b[38;5;241m=\u001b[39m [MultimodalSearch\u001b[38;5;241m.\u001b[39mread_img(\u001b[38;5;28mself\u001b[39m, path) \u001b[38;5;28;01mfor\u001b[39;00m path \u001b[38;5;129;01min\u001b[39;00m image_paths]\n\u001b[1;32m 169\u001b[0m images \u001b[38;5;241m=\u001b[39m [\n\u001b[1;32m 170\u001b[0m vis_processor[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124meval\u001b[39m\u001b[38;5;124m\"\u001b[39m](r_img)\n\u001b[1;32m 171\u001b[0m \u001b[38;5;241m.\u001b[39munsqueeze(\u001b[38;5;241m0\u001b[39m)\n\u001b[1;32m 172\u001b[0m \u001b[38;5;241m.\u001b[39mto(MultimodalSearch\u001b[38;5;241m.\u001b[39mmultimodal_device)\n\u001b[1;32m 173\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m r_img \u001b[38;5;129;01min\u001b[39;00m raw_images\n\u001b[1;32m 174\u001b[0m ]\n\u001b[1;32m 175\u001b[0m images_tensors \u001b[38;5;241m=\u001b[39m torch\u001b[38;5;241m.\u001b[39mstack(images)\n",
"File \u001b[0;32m~/work/AMMICO/AMMICO/ammico/multimodal_search.py:168\u001b[0m, in \u001b[0;36m<listcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m 156\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mread_and_process_images\u001b[39m(\u001b[38;5;28mself\u001b[39m, image_paths: \u001b[38;5;28mlist\u001b[39m, vis_processor) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m \u001b[38;5;28mtuple\u001b[39m:\n\u001b[1;32m 157\u001b[0m \u001b[38;5;250m \u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m 158\u001b[0m \u001b[38;5;124;03m Read and process images with vis_processor.\u001b[39;00m\n\u001b[1;32m 159\u001b[0m \n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 166\u001b[0m \u001b[38;5;124;03m images_tensors (torch.Tensor): tensors of images stacked in device.\u001b[39;00m\n\u001b[1;32m 167\u001b[0m \u001b[38;5;124;03m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 168\u001b[0m raw_images \u001b[38;5;241m=\u001b[39m [\u001b[43mMultimodalSearch\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mread_img\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mpath\u001b[49m\u001b[43m)\u001b[49m \u001b[38;5;28;01mfor\u001b[39;00m path \u001b[38;5;129;01min\u001b[39;00m image_paths]\n\u001b[1;32m 169\u001b[0m images \u001b[38;5;241m=\u001b[39m [\n\u001b[1;32m 170\u001b[0m vis_processor[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124meval\u001b[39m\u001b[38;5;124m\"\u001b[39m](r_img)\n\u001b[1;32m 171\u001b[0m \u001b[38;5;241m.\u001b[39munsqueeze(\u001b[38;5;241m0\u001b[39m)\n\u001b[1;32m 172\u001b[0m \u001b[38;5;241m.\u001b[39mto(MultimodalSearch\u001b[38;5;241m.\u001b[39mmultimodal_device)\n\u001b[1;32m 173\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m r_img \u001b[38;5;129;01min\u001b[39;00m raw_images\n\u001b[1;32m 174\u001b[0m ]\n\u001b[1;32m 175\u001b[0m images_tensors \u001b[38;5;241m=\u001b[39m torch\u001b[38;5;241m.\u001b[39mstack(images)\n",
"File \u001b[0;32m~/work/AMMICO/AMMICO/ammico/multimodal_search.py:153\u001b[0m, in \u001b[0;36mMultimodalSearch.read_img\u001b[0;34m(self, filepath)\u001b[0m\n\u001b[1;32m 143\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mread_img\u001b[39m(\u001b[38;5;28mself\u001b[39m, filepath: \u001b[38;5;28mstr\u001b[39m) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m Image:\n\u001b[1;32m 144\u001b[0m \u001b[38;5;250m \u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m 145\u001b[0m \u001b[38;5;124;03m Load Image from filepath.\u001b[39;00m\n\u001b[1;32m 146\u001b[0m \n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 151\u001b[0m \u001b[38;5;124;03m raw_image (PIL.Image): image.\u001b[39;00m\n\u001b[1;32m 152\u001b[0m \u001b[38;5;124;03m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 153\u001b[0m raw_image \u001b[38;5;241m=\u001b[39m \u001b[43mImage\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mopen\u001b[49m\u001b[43m(\u001b[49m\u001b[43mfilepath\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241m.\u001b[39mconvert(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mRGB\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m 154\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m raw_image\n",
"File \u001b[0;32m/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/PIL/Image.py:3236\u001b[0m, in \u001b[0;36mopen\u001b[0;34m(fp, mode, formats)\u001b[0m\n\u001b[1;32m 3233\u001b[0m filename \u001b[38;5;241m=\u001b[39m fp\n\u001b[1;32m 3235\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m filename:\n\u001b[0;32m-> 3236\u001b[0m fp \u001b[38;5;241m=\u001b[39m \u001b[43mbuiltins\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mopen\u001b[49m\u001b[43m(\u001b[49m\u001b[43mfilename\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mrb\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[1;32m 3237\u001b[0m exclusive_fp \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mTrue\u001b[39;00m\n\u001b[1;32m 3239\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n",
"\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: '102141_2_eng'"
]
}
],
"source": [
"(\n",
" model,\n",
" vis_processors,\n",
" txt_processors,\n",
" image_keys,\n",
" image_names,\n",
" features_image_stacked,\n",
") = my_obj.parsing_images(\n",
" model_type, \n",
" path_to_save_tensors=\"data/\",\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "f236c3b1-c3a6-471a-9fc5-ef831b675286",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.239567Z",
"iopub.status.busy": "2023-09-20T09:39:35.238846Z",
"iopub.status.idle": "2023-09-20T09:39:35.281225Z",
"shell.execute_reply": "2023-09-20T09:39:35.280440Z"
}
},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'features_image_stacked' is not defined",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[11], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mfeatures_image_stacked\u001b[49m\n",
"\u001b[0;31mNameError\u001b[0m: name 'features_image_stacked' is not defined"
]
}
],
"source": [
"features_image_stacked"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "9ff8a894-566b-4c4f-acca-21c50b5b1f52",
"metadata": {},
"source": [
"The images are then processed and stored in a numerical representation, a tensor. These tensors do not change for the same image and same model - so if you run this analysis once, and save the tensors giving a path with the keyword `path_to_save_tensors`, a file with filename `.<Number_of_images>_<model_name>_saved_features_image.pt` will be placed there.\n",
"\n",
"This will save you a lot of time if you want to analyse same images with the same model but different questions. To run using the saved tensors, execute the below code giving the path and name of the tensor file."
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "56c6d488-f093-4661-835a-5c73a329c874",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.284919Z",
"iopub.status.busy": "2023-09-20T09:39:35.284396Z",
"iopub.status.idle": "2023-09-20T09:39:35.288153Z",
"shell.execute_reply": "2023-09-20T09:39:35.287328Z"
},
"tags": []
},
"outputs": [],
"source": [
"# (\n",
"# model,\n",
"# vis_processors,\n",
"# txt_processors,\n",
"# image_keys,\n",
"# image_names,\n",
"# features_image_stacked,\n",
"# ) = my_obj.parsing_images(\n",
"# model_type,\n",
"# path_to_load_tensors=\"/content/drive/MyDrive/misinformation-data/5_clip_base_saved_features_image.pt\",\n",
"# )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "309923c1-d6f8-4424-8fca-bde5f3a98b38",
"metadata": {},
"source": [
"Here we already processed our image folder with 5 images and the `clip_base` model. So you need just to write the name `5_clip_base_saved_features_image.pt` of the saved file that consists of tensors of all images as keyword argument for `path_to_load_tensors`. "
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "162a52e8-6652-4897-b92e-645cab07aaef",
"metadata": {},
"source": [
"## Formulate your search queries\n",
"\n",
"Next, you need to form search queries. You can search either by image or by text. You can search for a single query, or you can search for several queries at once, the computational time should not be much different. The format of the queries is as follows:"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "c4196a52-d01e-42e4-8674-5712f7d6f792",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.292030Z",
"iopub.status.busy": "2023-09-20T09:39:35.291351Z",
"iopub.status.idle": "2023-09-20T09:39:35.295311Z",
"shell.execute_reply": "2023-09-20T09:39:35.294707Z"
},
"tags": []
},
"outputs": [],
"source": [
"search_query3 = [\n",
" {\"text_input\": \"politician press conference\"},\n",
" {\"text_input\": \"a world map\"},\n",
" {\"text_input\": \"a dog\"},\n",
"]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "8bcf3127-3dfd-4ff4-b9e7-a043099b1418",
"metadata": {},
"source": [
"You can filter your results in 3 different ways:\n",
"- `filter_number_of_images` limits the number of images found. That is, if the parameter `filter_number_of_images = 10`, then the first 10 images that best match the query will be shown. The other images ranks will be set to `None` and the similarity value to `0`.\n",
"- `filter_val_limit` limits the output of images with a similarity value not bigger than `filter_val_limit`. That is, if the parameter `filter_val_limit = 0.2`, all images with similarity less than 0.2 will be discarded.\n",
"- `filter_rel_error` (percentage) limits the output of images with a similarity value not bigger than `100 * abs(current_simularity_value - best_simularity_value_in_current_search)/best_simularity_value_in_current_search < filter_rel_error`. That is, if we set filter_rel_error = 30, it means that if the top1 image have 0.5 similarity value, we discard all image with similarity less than 0.35."
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "7f7dc52f-7ee9-4590-96b7-e0d9d3b82378",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.298729Z",
"iopub.status.busy": "2023-09-20T09:39:35.298254Z",
"iopub.status.idle": "2023-09-20T09:39:35.342887Z",
"shell.execute_reply": "2023-09-20T09:39:35.342077Z"
},
"tags": []
},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'model' is not defined",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[14], line 2\u001b[0m\n\u001b[1;32m 1\u001b[0m similarity, sorted_lists \u001b[38;5;241m=\u001b[39m my_obj\u001b[38;5;241m.\u001b[39mmultimodal_search(\n\u001b[0;32m----> 2\u001b[0m \u001b[43mmodel\u001b[49m,\n\u001b[1;32m 3\u001b[0m vis_processors,\n\u001b[1;32m 4\u001b[0m txt_processors,\n\u001b[1;32m 5\u001b[0m model_type,\n\u001b[1;32m 6\u001b[0m image_keys,\n\u001b[1;32m 7\u001b[0m features_image_stacked,\n\u001b[1;32m 8\u001b[0m search_query3,\n\u001b[1;32m 9\u001b[0m filter_number_of_images\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m20\u001b[39m,\n\u001b[1;32m 10\u001b[0m )\n",
"\u001b[0;31mNameError\u001b[0m: name 'model' is not defined"
]
}
],
"source": [
"similarity, sorted_lists = my_obj.multimodal_search(\n",
" model,\n",
" vis_processors,\n",
" txt_processors,\n",
" model_type,\n",
" image_keys,\n",
" features_image_stacked,\n",
" search_query3,\n",
" filter_number_of_images=20,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "65210ca2-b674-44bd-807a-4165e14bad74",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.346453Z",
"iopub.status.busy": "2023-09-20T09:39:35.345934Z",
"iopub.status.idle": "2023-09-20T09:39:35.386572Z",
"shell.execute_reply": "2023-09-20T09:39:35.385800Z"
}
},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'similarity' is not defined",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[15], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43msimilarity\u001b[49m\n",
"\u001b[0;31mNameError\u001b[0m: name 'similarity' is not defined"
]
}
],
"source": [
"similarity"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "557473df-e2b9-4ef0-9439-3daadf6741ac",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.390395Z",
"iopub.status.busy": "2023-09-20T09:39:35.389705Z",
"iopub.status.idle": "2023-09-20T09:39:35.431137Z",
"shell.execute_reply": "2023-09-20T09:39:35.430348Z"
}
},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'sorted_lists' is not defined",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[16], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43msorted_lists\u001b[49m\n",
"\u001b[0;31mNameError\u001b[0m: name 'sorted_lists' is not defined"
]
}
],
"source": [
"sorted_lists"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "c93d7e88-594d-4095-b5f2-7bf01210dc61",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.435275Z",
"iopub.status.busy": "2023-09-20T09:39:35.434615Z",
"iopub.status.idle": "2023-09-20T09:39:35.442173Z",
"shell.execute_reply": "2023-09-20T09:39:35.441460Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"{'102141_2_eng': {'filename': '102141_2_eng'},\n",
" '102730_eng': {'filename': '102730_eng'},\n",
" '106349S_por': {'filename': '106349S_por'}}"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mydict"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "e1cf7e46-0c2c-4fb2-b89a-ef585ccb9339",
"metadata": {},
"source": [
"After launching `multimodal_search` function, the results of each query will be added to the source dictionary. "
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "9ad74b21-6187-4a58-9ed8-fd3e80f5a4ed",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.445758Z",
"iopub.status.busy": "2023-09-20T09:39:35.445099Z",
"iopub.status.idle": "2023-09-20T09:39:35.450251Z",
"shell.execute_reply": "2023-09-20T09:39:35.449532Z"
},
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"{'filename': '106349S_por'}"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mydict[\"106349S_por\"]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "cd3ee120-8561-482b-a76a-e8f996783325",
"metadata": {},
"source": [
"A special function was written to present the search results conveniently. "
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "4324e4fd-e9aa-4933-bb12-074d54e0c510",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.453728Z",
"iopub.status.busy": "2023-09-20T09:39:35.453134Z",
"iopub.status.idle": "2023-09-20T09:39:35.561263Z",
"shell.execute_reply": "2023-09-20T09:39:35.560304Z"
},
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"'Your search query: politician press conference'"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"'--------------------------------------------------'"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"'Results:'"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"ename": "KeyError",
"evalue": "'politician press conference'",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[19], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mmy_obj\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mshow_results\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 2\u001b[0m \u001b[43m \u001b[49m\u001b[43msearch_query3\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;241;43m0\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 3\u001b[0m \u001b[43m)\u001b[49m\n",
"File \u001b[0;32m~/work/AMMICO/AMMICO/ammico/multimodal_search.py:970\u001b[0m, in \u001b[0;36mMultimodalSearch.show_results\u001b[0;34m(self, query, itm, image_gradcam_with_itm)\u001b[0m\n\u001b[1;32m 967\u001b[0m current_querry_val \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mlist\u001b[39m(query\u001b[38;5;241m.\u001b[39mvalues())[\u001b[38;5;241m0\u001b[39m]\n\u001b[1;32m 968\u001b[0m current_querry_rank \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrank \u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;241m+\u001b[39m \u001b[38;5;28mlist\u001b[39m(query\u001b[38;5;241m.\u001b[39mvalues())[\u001b[38;5;241m0\u001b[39m]\n\u001b[0;32m--> 970\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m s \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28;43msorted\u001b[39;49m\u001b[43m(\u001b[49m\n\u001b[1;32m 971\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msubdict\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mitems\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkey\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mlambda\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mt\u001b[49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mt\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;241;43m1\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m[\u001b[49m\u001b[43mcurrent_querry_val\u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mreverse\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\n\u001b[1;32m 972\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m:\n\u001b[1;32m 973\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m s[\u001b[38;5;241m1\u001b[39m][current_querry_rank] \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m 974\u001b[0m \u001b[38;5;28;01mbreak\u001b[39;00m\n",
"File \u001b[0;32m~/work/AMMICO/AMMICO/ammico/multimodal_search.py:971\u001b[0m, in \u001b[0;36mMultimodalSearch.show_results.<locals>.<lambda>\u001b[0;34m(t)\u001b[0m\n\u001b[1;32m 967\u001b[0m current_querry_val \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mlist\u001b[39m(query\u001b[38;5;241m.\u001b[39mvalues())[\u001b[38;5;241m0\u001b[39m]\n\u001b[1;32m 968\u001b[0m current_querry_rank \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrank \u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;241m+\u001b[39m \u001b[38;5;28mlist\u001b[39m(query\u001b[38;5;241m.\u001b[39mvalues())[\u001b[38;5;241m0\u001b[39m]\n\u001b[1;32m 970\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m s \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28msorted\u001b[39m(\n\u001b[0;32m--> 971\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39msubdict\u001b[38;5;241m.\u001b[39mitems(), key\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mlambda\u001b[39;00m t: \u001b[43mt\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;241;43m1\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m[\u001b[49m\u001b[43mcurrent_querry_val\u001b[49m\u001b[43m]\u001b[49m, reverse\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m\n\u001b[1;32m 972\u001b[0m ):\n\u001b[1;32m 973\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m s[\u001b[38;5;241m1\u001b[39m][current_querry_rank] \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m 974\u001b[0m \u001b[38;5;28;01mbreak\u001b[39;00m\n",
"\u001b[0;31mKeyError\u001b[0m: 'politician press conference'"
]
}
],
"source": [
"my_obj.show_results(\n",
" search_query3[0],\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "0b750e9f-fe64-4028-9caf-52d7187462f1",
"metadata": {},
"source": [
"## Improve the search results\n",
"\n",
"For even better results, a slightly different approach has been prepared that can improve search results. It is quite resource-intensive, so it is applied after the main algorithm has found the most relevant images. This approach works only with text queries. Among the parameters you can choose 3 models: `\"blip_base\"`, `\"blip_large\"`, `\"blip2_coco\"`. If you get an `Out of Memory` error, try reducing the batch_size value (minimum = 1), which is the number of images being processed simultaneously. With the parameter `need_grad_cam = True/False` you can enable the calculation of the heat map of each image to be processed. Thus the `image_text_match_reordering` function calculates new similarity values and new ranks for each image. The resulting values are added to the general dictionary."
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "b3af7b39-6d0d-4da3-9b8f-7dfd3f5779be",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.565172Z",
"iopub.status.busy": "2023-09-20T09:39:35.564893Z",
"iopub.status.idle": "2023-09-20T09:39:35.568993Z",
"shell.execute_reply": "2023-09-20T09:39:35.568282Z"
},
"tags": []
},
"outputs": [],
"source": [
"itm_model = \"blip_base\"\n",
"# itm_model = \"blip_large\"\n",
"# itm_model = \"blip2_coco\""
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "caf1f4ae-4b37-4954-800e-7120f0419de5",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.572921Z",
"iopub.status.busy": "2023-09-20T09:39:35.572391Z",
"iopub.status.idle": "2023-09-20T09:39:35.618177Z",
"shell.execute_reply": "2023-09-20T09:39:35.617297Z"
},
"tags": []
},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'image_keys' is not defined",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[21], line 4\u001b[0m\n\u001b[1;32m 1\u001b[0m itm_scores, image_gradcam_with_itm \u001b[38;5;241m=\u001b[39m my_obj\u001b[38;5;241m.\u001b[39mimage_text_match_reordering(\n\u001b[1;32m 2\u001b[0m search_query3,\n\u001b[1;32m 3\u001b[0m itm_model,\n\u001b[0;32m----> 4\u001b[0m \u001b[43mimage_keys\u001b[49m,\n\u001b[1;32m 5\u001b[0m sorted_lists,\n\u001b[1;32m 6\u001b[0m batch_size\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m1\u001b[39m,\n\u001b[1;32m 7\u001b[0m need_grad_cam\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m,\n\u001b[1;32m 8\u001b[0m )\n",
"\u001b[0;31mNameError\u001b[0m: name 'image_keys' is not defined"
]
}
],
"source": [
"itm_scores, image_gradcam_with_itm = my_obj.image_text_match_reordering(\n",
" search_query3,\n",
" itm_model,\n",
" image_keys,\n",
" sorted_lists,\n",
" batch_size=1,\n",
" need_grad_cam=True,\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "9e98c150-5fab-4251-bce7-0d8fc7b385b9",
"metadata": {},
"source": [
"Then using the same output function you can add the `ITM=True` arguments to output the new image order. You can also add the `image_gradcam_with_itm` argument to output the heat maps of the calculated images. "
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "6a829b99-5230-463a-8b11-30ffbb67fc3a",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.622429Z",
"iopub.status.busy": "2023-09-20T09:39:35.621843Z",
"iopub.status.idle": "2023-09-20T09:39:35.664625Z",
"shell.execute_reply": "2023-09-20T09:39:35.663703Z"
},
"tags": []
},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'image_gradcam_with_itm' is not defined",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[22], line 2\u001b[0m\n\u001b[1;32m 1\u001b[0m my_obj\u001b[38;5;241m.\u001b[39mshow_results(\n\u001b[0;32m----> 2\u001b[0m search_query3[\u001b[38;5;241m0\u001b[39m], itm\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m, image_gradcam_with_itm\u001b[38;5;241m=\u001b[39m\u001b[43mimage_gradcam_with_itm\u001b[49m\n\u001b[1;32m 3\u001b[0m )\n",
"\u001b[0;31mNameError\u001b[0m: name 'image_gradcam_with_itm' is not defined"
]
}
],
"source": [
"my_obj.show_results(\n",
" search_query3[0], itm=True, image_gradcam_with_itm=image_gradcam_with_itm\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d86ab96b-1907-4b7f-a78e-3983b516d781",
"metadata": {
"tags": []
},
"source": [
"## Save search results to csv"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "4bdbc4d4-695d-4751-ab7c-d2d98e2917d7",
"metadata": {
"tags": []
},
"source": [
"Convert the dictionary of dictionarys into a dictionary with lists:"
]
},
{
"cell_type": "code",
"execution_count": 23,
"id": "6c6ddd83-bc87-48f2-a8d6-1bd3f4201ff7",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.669193Z",
"iopub.status.busy": "2023-09-20T09:39:35.668501Z",
"iopub.status.idle": "2023-09-20T09:39:35.675371Z",
"shell.execute_reply": "2023-09-20T09:39:35.674640Z"
},
"tags": []
},
"outputs": [],
"source": [
"outdict = mutils.append_data_to_dict(mydict)\n",
"df = mutils.dump_df(outdict)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "ea2675d5-604c-45e7-86d2-080b1f4559a0",
"metadata": {
"tags": []
},
"source": [
"Check the dataframe:"
]
},
{
"cell_type": "code",
"execution_count": 24,
"id": "e78646d6-80be-4d3e-8123-3360957bcaa8",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.679083Z",
"iopub.status.busy": "2023-09-20T09:39:35.678363Z",
"iopub.status.idle": "2023-09-20T09:39:35.694719Z",
"shell.execute_reply": "2023-09-20T09:39:35.693954Z"
},
"tags": []
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>filename</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>102141_2_eng</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>102730_eng</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>106349S_por</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" filename\n",
"0 102141_2_eng\n",
"1 102730_eng\n",
"2 106349S_por"
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df.head(10)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "05546d99-afab-4565-8f30-f14e1426abcf",
"metadata": {},
"source": [
"Write the csv file:"
]
},
{
"cell_type": "code",
"execution_count": 25,
"id": "185f7dde-20dc-44d8-9ab0-de41f9b5734d",
"metadata": {
"execution": {
"iopub.execute_input": "2023-09-20T09:39:35.698809Z",
"iopub.status.busy": "2023-09-20T09:39:35.698062Z",
"iopub.status.idle": "2023-09-20T09:39:35.705143Z",
"shell.execute_reply": "2023-09-20T09:39:35.704420Z"
},
"tags": []
},
"outputs": [],
"source": [
"df.to_csv(\"data/data_out.csv\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b6a79201-7c17-496c-a6a1-b8ecfd3dd1e8",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.18"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Ссылка в новой задаче Показать git blame Копировать постоянную ссылку