AMMICO/misinformation/objects_cvlib.py
Petr Andriushchenko 2891c8a6ed
add image summary notebook (#57)
* add image summary notebook

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* pin deepface version to avoid bug with progress bar after update

* update actions version for checkout and python

* test ci without lavis

* no lavis for ci test

* merging

* return lavis

* change lavis to salesforce-lavis

* change pycocotools install method

* change pycocotools install method

* fix_pycocotools

* Downgrade Python

* back to 3.9 and remove pycocotools dependance

* instrucctions for windows

* missing comma after merge

* lavis only for ubuntu

* use lavis package name in install instead of git

* adding multimodal searching py and notebook

* exclude lavis on windows

* skip import on windows

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* reactivate lavis

* Revert "reactivate lavis"

This reverts commit ecdaf9d316e4b08816ba62da5e0482c8ff15b14e.

* Change input format for multimodal search

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix clip models

* account for new interface in init imports

* changed imports bec of lavis/windows

* fix if-else, added clip ViT-L-14=336 model

* fix code smells

* add model change function to summary

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixed new model in summary.py

* fixed summary windget

* moved some function to utils

* fixed imort torch in utils

* added test_summary.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixed opencv version

* added first test of multimodal_search.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixed test

* removed windows in CI and added test in multimodal search

* change lavis from dependencies from pip ro git

* fixed blip2 model in test_multimodal_search.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixed test multimodal search on cpu and gpu machines

* added test, fixed dependencies

* add -vv to pytest command in CI

* added test_multimodal_search tests

* fixed tests in test_multimodal_search.py

* fixed tests in test_summary

* changed CI and fixed test_multimodel search

* fixed ci

* fixed error in test multimodal search, changed ci

* added multimodal search test, added windows CI, added picture in test data

* CI debuging

* fixing tests in CI

* fixing test in CI 2

* fixing CI 3

* fixing CI

* added filtering function

* Brought back all tests after CI fixing

* changed CI one pytest by individual tests

* fixed opencv problem

* fix path for text, adjust result for new gcv

* remove opencv

* fixing cv2 error

* added opencv-contrib, change objects_cvlib

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixing tests in CI

* fixing CI testing

* cleanup objects

* fixing codecov in CI

* fixing codecov in CI

* run tests together; install opencv last

* update requirements for opencv dependencies

* moved lavis functions from utils to summary

* Remove lavis from utils.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add missing jupyter

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: iulusoy <inga.ulusoy@uni-heidelberg.de>
2023-03-22 10:28:09 +01:00

84 строки
2.1 KiB
Python

import cv2
import cvlib as cv
import numpy as np
from PIL import Image
def objects_from_cvlib(objects_list: list) -> dict:
objects = init_default_objects()
for key in objects:
if key in objects_list:
objects[key] = "yes"
return objects
def init_default_objects():
objects = {
"person": "no",
"bicycle": "no",
"car": "no",
"motorcycle": "no",
"airplane": "no",
"bus": "no",
"train": "no",
"truck": "no",
"boat": "no",
"traffic light": "no",
"cell phone": "no",
}
return objects
class ObjectsMethod:
"""Base class to be inherited by all objects methods."""
def __init__(self):
# initialize in child class
pass
def analyse_image(self, subdict):
raise NotImplementedError()
class ObjectCVLib(ObjectsMethod):
def __init__(self, client_type=1):
# as long as imageai is not activated this remains empty
pass
def detect_objects_cvlib(self, image_path):
"""Localize objects in the local image.
Args:
image_path: The path to the local file.
"""
img = cv2.imread(image_path)
# preimg = Image.open(image_path).convert("RGB")
# preimg2 = np.asarray(preimg)
# img = cv2.cvtColor(preimg2, cv2.COLOR_BGR2RGB)
_, label, _ = cv.detect_common_objects(img)
# output_image = draw_bbox(im, bbox, label, conf)
objects = objects_from_cvlib(label)
return objects
def analyse_image_from_file(self, image_path):
"""Localize objects in the local image.
Args:
image_path: The path to the local file.
"""
objects = self.detect_objects_cvlib(image_path)
return objects
def analyse_image(self, subdict):
"""Localize objects in the local image.
Args:
subdict: The dictionary for an image expression instance.
"""
objects = self.analyse_image_from_file(subdict["filename"])
for key in objects:
subdict[key] = objects[key]
return subdict