feat: create space in demo notebook for multiple image search

Этот коммит содержится в:
Inga Ulusoy 2025-03-31 09:46:46 +02:00
родитель 1a629836db
Коммит a55ee5aa3f
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: BDC64F2E85CF8272

Просмотреть файл

@ -1037,6 +1037,26 @@
"To process the loaded images using the selected model, use the below code:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"image_dict = ammico.find_files(\n",
" path = \"../../../test-images-politicians\",\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"image_dict"
]
},
{
"cell_type": "code",
"execution_count": null,
@ -1061,7 +1081,7 @@
" features_image_stacked,\n",
") = my_obj.parsing_images(\n",
" model_type, \n",
" path_to_save_tensors=\"/content/drive/MyDrive/misinformation-data/\",\n",
" path_to_save_tensors=\"../../../test-images-politicians\",\n",
" )"
]
},
@ -1118,12 +1138,10 @@
"outputs": [],
"source": [
"import importlib_resources # only require for image query example\n",
"image_example_query = str(importlib_resources.files(\"ammico\") / \"data\" / \"test-crop-image.png\") # creating the path to the image for the image query example\n",
"# image_example_query = str(importlib_resources.files(\"ammico\") / \"data\" / \"test-crop-image.png\") # creating the path to the image for the image query example\n",
"image_example_query = \"../../../test-images-politicians/5e4c7722-d620-42ab-915e-9b1a048e2992Original.jpg\" # creating the path to the image for the image query example\n",
"\n",
"search_query = [\n",
" {\"text_input\": \"politician press conference\"}, \n",
" {\"text_input\": \"a world map\"},\n",
" {\"text_input\": \"a dog\"}, # This is how looks text query\n",
" {\"image\": image_example_query}, # This is how looks image query, here `image_example_path` is the path to query image like \"data/test-crop-image.png\"\n",
"]"
]
@ -1208,22 +1226,29 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Formulate your search queries: Search for the best match using multiple reference images, for example, of a person"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"my_obj.show_results(\n",
" search_query[3], # you can change the index to see the results for other queries\n",
")"
"# Here goes the code that reads in multiple images as reference\n",
"# then you will loop over these multiple images and find the best matches\n",
"# in the end, the best matches will be averaged over for each picture and a list of averaged best matches will be provided"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Improve the search results\n",
"### Improve the search results: Use only for text queries, not image search\n",
"\n",
"For even better results, a slightly different approach has been prepared that can improve search results. It is quite resource-intensive, so it is applied after the main algorithm has found the most relevant images. This approach works only with text queries and it skips image queries. Among the parameters you can choose 3 models: `\"blip_base\"`, `\"blip_large\"`, `\"blip2_coco\"`. If you get an `Out of Memory` error, try reducing the batch_size value (minimum = 1), which is the number of images being processed simultaneously. With the parameter `need_grad_cam = True/False` you can enable the calculation of the heat map of each image to be processed and save them in `image_gradcam_with_itm`. Thus the `image_text_match_reordering()` function calculates new similarity values and new ranks for each image. The resulting values are added to the general dictionary."
]
@ -1438,7 +1463,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "ammico",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
@ -1452,7 +1477,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
"version": "3.11.11"
}
},
"nbformat": 4,