зеркало из
https://github.com/ssciwr/AMMICO.git
synced 2025-10-29 13:06:04 +02:00
fixing the absence of a vqa model (#150)
* fixing the absence of a VQA model
Этот коммит содержится в:
родитель
a57e67be32
Коммит
32d1321a1a
@ -84,7 +84,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"Here you can choose between two models: `\"base\"` or `\"large\"`. This will generate the caption for each image and directly put the results in your dictionary `mydict`. Then you can transform it into the dataframe and this dataframe can be exported as a .csv file.\n",
|
"Here you can choose between models: `\"base\"` or `\"large\"`. This will generate the caption for each image and directly put the results in your dictionary `mydict`. Then you can transform it into the dataframe and this dataframe can be exported as a .csv file.\n",
|
||||||
"\n",
|
"\n",
|
||||||
"The results are written in the columns: \n",
|
"The results are written in the columns: \n",
|
||||||
"- `const_image_summary` - the permanent summaries, which do not change from run to run (analyse_image).\n",
|
"- `const_image_summary` - the permanent summaries, which do not change from run to run (analyse_image).\n",
|
||||||
@ -201,7 +201,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"Set the list of questions as a list of strings `list_of_questions`, load the models to the memory and pass them to the function"
|
"Load the model to the memory through object creation with parameters `analysis_type=\"questions\"` and `model_type=\"vqa\"`. Set the list of questions as a list of strings `list_of_questions`, and pass them to the function"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -223,7 +223,7 @@
|
|||||||
"metadata": {},
|
"metadata": {},
|
||||||
"outputs": [],
|
"outputs": [],
|
||||||
"source": [
|
"source": [
|
||||||
"obj = ammico.SummaryDetector(mydict, analysis_type=\"questions\", list_of_questions = list_of_questions)"
|
"obj = ammico.SummaryDetector(mydict, analysis_type=\"questions\", model_type=\"vqa\")"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|||||||
@ -9,6 +9,7 @@ class SummaryDetector(AnalysisMethod):
|
|||||||
allowed_model_types = [
|
allowed_model_types = [
|
||||||
"base",
|
"base",
|
||||||
"large",
|
"large",
|
||||||
|
"vqa",
|
||||||
]
|
]
|
||||||
allowed_new_model_types = [
|
allowed_new_model_types = [
|
||||||
"blip2_t5_pretrain_flant5xxl",
|
"blip2_t5_pretrain_flant5xxl",
|
||||||
@ -44,7 +45,7 @@ class SummaryDetector(AnalysisMethod):
|
|||||||
Args:
|
Args:
|
||||||
subdict (dict, optional): Dictionary containing the image to be analysed. Defaults to {}.
|
subdict (dict, optional): Dictionary containing the image to be analysed. Defaults to {}.
|
||||||
|
|
||||||
model_type (str, optional): Type of model to use. Can be "base" or "large" for blip_caption. Or can be one of the new models:
|
model_type (str, optional): Type of model to use. Can be "base" or "large" or "vqa" for blip_caption and VQA. Or can be one of the new models:
|
||||||
"blip2_t5_pretrain_flant5xxl",
|
"blip2_t5_pretrain_flant5xxl",
|
||||||
"blip2_t5_pretrain_flant5xl",
|
"blip2_t5_pretrain_flant5xl",
|
||||||
"blip2_t5_caption_coco_flant5xl",
|
"blip2_t5_caption_coco_flant5xl",
|
||||||
|
|||||||
Загрузка…
x
Ссылка в новой задаче
Block a user