Этот коммит содержится в:
Inga Ulusoy 2025-04-10 21:30:40 +02:00
родитель a47ecb0575
Коммит 5b529a8105
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: BDC64F2E85CF8272

Просмотреть файл

@ -145,9 +145,7 @@
},
{
"cell_type": "markdown",
"metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"metadata": {},
"source": [
"# Step 1: Read your data into AMMICO\n",
"The ammico package reads in one or several input files given in a folder for processing. The user can select to read in all image files in a folder, to include subfolders via the `recursive` option, and can select the file extension that should be considered (for example, only \"jpg\" files, or both \"jpg\" and \"png\" files). For reading in the files, the ammico function `find_files` is used, with optional keywords:\n",
@ -984,6 +982,13 @@
"| `emotion (category)` | `list[str]` | the detected emotion category (positive, negative, or neutral), up to three faces, if above the confidence threshold |"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Image Multimodal Search"
]
},
{
"cell_type": "markdown",
"metadata": {},
@ -1039,7 +1044,7 @@
"outputs": [],
"source": [
"image_dict = ammico.find_files(\n",
" path = \"../../../test-images-politicians\",\n",
" path=str(data_path),\n",
")"
]
},
@ -1076,7 +1081,7 @@
" features_image_stacked,\n",
") = my_obj.parsing_images(\n",
" model_type, \n",
" path_to_save_tensors=\"../../../test-images-politicians\",\n",
" path_to_save_tensors=str(data_path),\n",
" )"
]
},
@ -1133,8 +1138,7 @@
"outputs": [],
"source": [
"import importlib_resources # only require for image query example\n",
"# image_example_query = str(importlib_resources.files(\"ammico\") / \"data\" / \"test-crop-image.png\") # creating the path to the image for the image query example\n",
"image_example_query = \"../../../test-images-politicians/5e4c7722-d620-42ab-915e-9b1a048e2992Original.jpg\" \n",
"image_example_query = str(importlib_resources.files(\"ammico\") / \"data\" / \"test-crop-image.png\") # creating the path to the image for the image query example\n",
"\n",
"search_query = [\n",
" {\"image\": image_example_query}, # This is how looks image query, here `image_example_path` is the path to query image like \"data/test-crop-image.png\"\n",
@ -1184,7 +1188,7 @@
"metadata": {},
"outputs": [],
"source": [
"sorted_lists # rank sorted by similarity\n"
"sorted_lists"
]
},
{
@ -1218,8 +1222,14 @@
"source": [
"my_obj.show_results(\n",
" search_query[0], # you can change the index to see the results for other queries\n",
")\n",
"# possible to show the averages?"
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Formulate your search queries: Search for the best match using multiple reference images, for example, of a person"
]
},
{
@ -1233,74 +1243,6 @@
"# in the end, the best matches will be averaged over for each picture and a list of averaged best matches will be provided"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import importlib_resources # only require for image query example\n",
"# image_example_query = str(importlib_resources.files(\"ammico\") / \"data\" / \"test-crop-image.png\") # creating the path to the image for the image query example\n",
"image_example_query = \"../../../test-images-politicians/5e4c7722-d620-42ab-915e-9b1a048e2992Original.jpg\" # creating the path to the image for the image query example\n",
"image_example_query2 = \"../../../test-images-politicians/service-pnp-ppbd-00600-00608_150px.jpg\"\n",
"\n",
"search_query = [\n",
" {\"image\": image_example_query}, # This is how looks image query, here `image_example_path` is the path to query image like \"data/test-crop-image.png\"\n",
" {\"image\": image_example_query2},\n",
"]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"similarity, sorted_lists = my_obj.multimodal_search(\n",
" model,\n",
" vis_processors,\n",
" txt_processors,\n",
" model_type,\n",
" image_keys,\n",
" features_image_stacked,\n",
" search_query,\n",
" filter_number_of_images=20,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"similarity # now a 2D tensor"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# average similarities\n",
"print(similarity.mean(dim = 1)) "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# add the similarity average to the image_dict\n",
"for key in image_dict.keys():\n",
" # find the similarities for each image in the search query\n",
" similarities = [image_dict[key][query[\"image\"]] for query in search_query]\n",
" image_dict[key][\"similarity_average\"] = sum(similarities)/len(similarities)\n",
" image_dict[key][\"rank similarity_average\"] = \"some_value\""
]
},
{
"cell_type": "markdown",
"metadata": {},
@ -1520,7 +1462,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": "ammico",
"language": "python",
"name": "python3"
},
@ -1534,7 +1476,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.21"
"version": "3.11.11"
}
},
"nbformat": 4,