fixing the absence of a vqa model (#150)

* fixing the absence of a VQA model
Этот коммит содержится в:
Petr Andriushchenko 2023-09-26 13:23:49 +02:00 коммит произвёл GitHub
родитель a57e67be32
Коммит 32d1321a1a
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
2 изменённых файлов: 5 добавлений и 4 удалений

Просмотреть файл

@ -84,7 +84,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Here you can choose between two models: `\"base\"` or `\"large\"`. This will generate the caption for each image and directly put the results in your dictionary `mydict`. Then you can transform it into the dataframe and this dataframe can be exported as a .csv file.\n", "Here you can choose between models: `\"base\"` or `\"large\"`. This will generate the caption for each image and directly put the results in your dictionary `mydict`. Then you can transform it into the dataframe and this dataframe can be exported as a .csv file.\n",
"\n", "\n",
"The results are written in the columns: \n", "The results are written in the columns: \n",
"- `const_image_summary` - the permanent summaries, which do not change from run to run (analyse_image).\n", "- `const_image_summary` - the permanent summaries, which do not change from run to run (analyse_image).\n",
@ -201,7 +201,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Set the list of questions as a list of strings `list_of_questions`, load the models to the memory and pass them to the function" "Load the model to the memory through object creation with parameters `analysis_type=\"questions\"` and `model_type=\"vqa\"`. Set the list of questions as a list of strings `list_of_questions`, and pass them to the function"
] ]
}, },
{ {
@ -223,7 +223,7 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"obj = ammico.SummaryDetector(mydict, analysis_type=\"questions\", list_of_questions = list_of_questions)" "obj = ammico.SummaryDetector(mydict, analysis_type=\"questions\", model_type=\"vqa\")"
] ]
}, },
{ {

Просмотреть файл

@ -9,6 +9,7 @@ class SummaryDetector(AnalysisMethod):
allowed_model_types = [ allowed_model_types = [
"base", "base",
"large", "large",
"vqa",
] ]
allowed_new_model_types = [ allowed_new_model_types = [
"blip2_t5_pretrain_flant5xxl", "blip2_t5_pretrain_flant5xxl",
@ -44,7 +45,7 @@ class SummaryDetector(AnalysisMethod):
Args: Args:
subdict (dict, optional): Dictionary containing the image to be analysed. Defaults to {}. subdict (dict, optional): Dictionary containing the image to be analysed. Defaults to {}.
model_type (str, optional): Type of model to use. Can be "base" or "large" for blip_caption. Or can be one of the new models: model_type (str, optional): Type of model to use. Can be "base" or "large" or "vqa" for blip_caption and VQA. Or can be one of the new models:
"blip2_t5_pretrain_flant5xxl", "blip2_t5_pretrain_flant5xxl",
"blip2_t5_pretrain_flant5xl", "blip2_t5_pretrain_flant5xl",
"blip2_t5_caption_coco_flant5xl", "blip2_t5_caption_coco_flant5xl",