fixing the absence of a vqa model (#150)

* fixing the absence of a VQA model
Этот коммит содержится в:
Petr Andriushchenko 2023-09-26 13:23:49 +02:00 коммит произвёл GitHub
родитель a57e67be32
Коммит 32d1321a1a
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
2 изменённых файлов: 5 добавлений и 4 удалений

Просмотреть файл

@ -84,7 +84,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Here you can choose between two models: `\"base\"` or `\"large\"`. This will generate the caption for each image and directly put the results in your dictionary `mydict`. Then you can transform it into the dataframe and this dataframe can be exported as a .csv file.\n",
"Here you can choose between models: `\"base\"` or `\"large\"`. This will generate the caption for each image and directly put the results in your dictionary `mydict`. Then you can transform it into the dataframe and this dataframe can be exported as a .csv file.\n",
"\n",
"The results are written in the columns: \n",
"- `const_image_summary` - the permanent summaries, which do not change from run to run (analyse_image).\n",
@ -201,7 +201,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Set the list of questions as a list of strings `list_of_questions`, load the models to the memory and pass them to the function"
"Load the model to the memory through object creation with parameters `analysis_type=\"questions\"` and `model_type=\"vqa\"`. Set the list of questions as a list of strings `list_of_questions`, and pass them to the function"
]
},
{
@ -223,7 +223,7 @@
"metadata": {},
"outputs": [],
"source": [
"obj = ammico.SummaryDetector(mydict, analysis_type=\"questions\", list_of_questions = list_of_questions)"
"obj = ammico.SummaryDetector(mydict, analysis_type=\"questions\", model_type=\"vqa\")"
]
},
{

Просмотреть файл

@ -9,6 +9,7 @@ class SummaryDetector(AnalysisMethod):
allowed_model_types = [
"base",
"large",
"vqa",
]
allowed_new_model_types = [
"blip2_t5_pretrain_flant5xxl",
@ -44,7 +45,7 @@ class SummaryDetector(AnalysisMethod):
Args:
subdict (dict, optional): Dictionary containing the image to be analysed. Defaults to {}.
model_type (str, optional): Type of model to use. Can be "base" or "large" for blip_caption. Or can be one of the new models:
model_type (str, optional): Type of model to use. Can be "base" or "large" or "vqa" for blip_caption and VQA. Or can be one of the new models:
"blip2_t5_pretrain_flant5xxl",
"blip2_t5_pretrain_flant5xl",
"blip2_t5_caption_coco_flant5xl",