зеркало из
https://github.com/ssciwr/AMMICO.git
synced 2025-10-29 21:16:06 +02:00
update paths in demo notebook
Этот коммит содержится в:
родитель
a47ecb0575
Коммит
5b529a8105
@ -145,9 +145,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"jp-MarkdownHeadingCollapsed": true
|
||||
},
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Step 1: Read your data into AMMICO\n",
|
||||
"The ammico package reads in one or several input files given in a folder for processing. The user can select to read in all image files in a folder, to include subfolders via the `recursive` option, and can select the file extension that should be considered (for example, only \"jpg\" files, or both \"jpg\" and \"png\" files). For reading in the files, the ammico function `find_files` is used, with optional keywords:\n",
|
||||
@ -984,6 +982,13 @@
|
||||
"| `emotion (category)` | `list[str]` | the detected emotion category (positive, negative, or neutral), up to three faces, if above the confidence threshold |"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Image Multimodal Search"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@ -1039,7 +1044,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"image_dict = ammico.find_files(\n",
|
||||
" path = \"../../../test-images-politicians\",\n",
|
||||
" path=str(data_path),\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
@ -1076,7 +1081,7 @@
|
||||
" features_image_stacked,\n",
|
||||
") = my_obj.parsing_images(\n",
|
||||
" model_type, \n",
|
||||
" path_to_save_tensors=\"../../../test-images-politicians\",\n",
|
||||
" path_to_save_tensors=str(data_path),\n",
|
||||
" )"
|
||||
]
|
||||
},
|
||||
@ -1133,8 +1138,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import importlib_resources # only require for image query example\n",
|
||||
"# image_example_query = str(importlib_resources.files(\"ammico\") / \"data\" / \"test-crop-image.png\") # creating the path to the image for the image query example\n",
|
||||
"image_example_query = \"../../../test-images-politicians/5e4c7722-d620-42ab-915e-9b1a048e2992Original.jpg\" \n",
|
||||
"image_example_query = str(importlib_resources.files(\"ammico\") / \"data\" / \"test-crop-image.png\") # creating the path to the image for the image query example\n",
|
||||
"\n",
|
||||
"search_query = [\n",
|
||||
" {\"image\": image_example_query}, # This is how looks image query, here `image_example_path` is the path to query image like \"data/test-crop-image.png\"\n",
|
||||
@ -1184,7 +1188,7 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"sorted_lists # rank sorted by similarity\n"
|
||||
"sorted_lists"
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -1218,8 +1222,14 @@
|
||||
"source": [
|
||||
"my_obj.show_results(\n",
|
||||
" search_query[0], # you can change the index to see the results for other queries\n",
|
||||
")\n",
|
||||
"# possible to show the averages?"
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Formulate your search queries: Search for the best match using multiple reference images, for example, of a person"
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -1233,74 +1243,6 @@
|
||||
"# in the end, the best matches will be averaged over for each picture and a list of averaged best matches will be provided"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import importlib_resources # only require for image query example\n",
|
||||
"# image_example_query = str(importlib_resources.files(\"ammico\") / \"data\" / \"test-crop-image.png\") # creating the path to the image for the image query example\n",
|
||||
"image_example_query = \"../../../test-images-politicians/5e4c7722-d620-42ab-915e-9b1a048e2992Original.jpg\" # creating the path to the image for the image query example\n",
|
||||
"image_example_query2 = \"../../../test-images-politicians/service-pnp-ppbd-00600-00608_150px.jpg\"\n",
|
||||
"\n",
|
||||
"search_query = [\n",
|
||||
" {\"image\": image_example_query}, # This is how looks image query, here `image_example_path` is the path to query image like \"data/test-crop-image.png\"\n",
|
||||
" {\"image\": image_example_query2},\n",
|
||||
"]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"similarity, sorted_lists = my_obj.multimodal_search(\n",
|
||||
" model,\n",
|
||||
" vis_processors,\n",
|
||||
" txt_processors,\n",
|
||||
" model_type,\n",
|
||||
" image_keys,\n",
|
||||
" features_image_stacked,\n",
|
||||
" search_query,\n",
|
||||
" filter_number_of_images=20,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"similarity # now a 2D tensor"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# average similarities\n",
|
||||
"print(similarity.mean(dim = 1)) "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# add the similarity average to the image_dict\n",
|
||||
"for key in image_dict.keys():\n",
|
||||
" # find the similarities for each image in the search query\n",
|
||||
" similarities = [image_dict[key][query[\"image\"]] for query in search_query]\n",
|
||||
" image_dict[key][\"similarity_average\"] = sum(similarities)/len(similarities)\n",
|
||||
" image_dict[key][\"rank similarity_average\"] = \"some_value\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@ -1520,7 +1462,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"display_name": "ammico",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
@ -1534,7 +1476,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.21"
|
||||
"version": "3.11.11"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
Загрузка…
x
Ссылка в новой задаче
Block a user