зеркало из
https://github.com/ssciwr/AMMICO.git
synced 2025-10-29 21:16:06 +02:00
feat: create space in demo notebook for multiple image search
Этот коммит содержится в:
родитель
1a629836db
Коммит
a55ee5aa3f
@ -1037,6 +1037,26 @@
|
||||
"To process the loaded images using the selected model, use the below code:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"image_dict = ammico.find_files(\n",
|
||||
" path = \"../../../test-images-politicians\",\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"image_dict"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
@ -1061,7 +1081,7 @@
|
||||
" features_image_stacked,\n",
|
||||
") = my_obj.parsing_images(\n",
|
||||
" model_type, \n",
|
||||
" path_to_save_tensors=\"/content/drive/MyDrive/misinformation-data/\",\n",
|
||||
" path_to_save_tensors=\"../../../test-images-politicians\",\n",
|
||||
" )"
|
||||
]
|
||||
},
|
||||
@ -1118,12 +1138,10 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import importlib_resources # only require for image query example\n",
|
||||
"image_example_query = str(importlib_resources.files(\"ammico\") / \"data\" / \"test-crop-image.png\") # creating the path to the image for the image query example\n",
|
||||
"# image_example_query = str(importlib_resources.files(\"ammico\") / \"data\" / \"test-crop-image.png\") # creating the path to the image for the image query example\n",
|
||||
"image_example_query = \"../../../test-images-politicians/5e4c7722-d620-42ab-915e-9b1a048e2992Original.jpg\" # creating the path to the image for the image query example\n",
|
||||
"\n",
|
||||
"search_query = [\n",
|
||||
" {\"text_input\": \"politician press conference\"}, \n",
|
||||
" {\"text_input\": \"a world map\"},\n",
|
||||
" {\"text_input\": \"a dog\"}, # This is how looks text query\n",
|
||||
" {\"image\": image_example_query}, # This is how looks image query, here `image_example_path` is the path to query image like \"data/test-crop-image.png\"\n",
|
||||
"]"
|
||||
]
|
||||
@ -1208,22 +1226,29 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Formulate your search queries: Search for the best match using multiple reference images, for example, of a person"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"my_obj.show_results(\n",
|
||||
" search_query[3], # you can change the index to see the results for other queries\n",
|
||||
")"
|
||||
"# Here goes the code that reads in multiple images as reference\n",
|
||||
"# then you will loop over these multiple images and find the best matches\n",
|
||||
"# in the end, the best matches will be averaged over for each picture and a list of averaged best matches will be provided"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Improve the search results\n",
|
||||
"### Improve the search results: Use only for text queries, not image search\n",
|
||||
"\n",
|
||||
"For even better results, a slightly different approach has been prepared that can improve search results. It is quite resource-intensive, so it is applied after the main algorithm has found the most relevant images. This approach works only with text queries and it skips image queries. Among the parameters you can choose 3 models: `\"blip_base\"`, `\"blip_large\"`, `\"blip2_coco\"`. If you get an `Out of Memory` error, try reducing the batch_size value (minimum = 1), which is the number of images being processed simultaneously. With the parameter `need_grad_cam = True/False` you can enable the calculation of the heat map of each image to be processed and save them in `image_gradcam_with_itm`. Thus the `image_text_match_reordering()` function calculates new similarity values and new ranks for each image. The resulting values are added to the general dictionary."
|
||||
]
|
||||
@ -1438,7 +1463,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "ammico",
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
@ -1452,7 +1477,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.9"
|
||||
"version": "3.11.11"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
Загрузка…
x
Ссылка в новой задаче
Block a user