text module
-summary module
+text module
-
-
- -class summary.SummaryDetector(subdict: dict = {}, model_type: str = 'base', analysis_type: str = 'summary_and_questions', list_of_questions: list[str] | None = None, summary_model=None, summary_vis_processors=None, summary_vqa_model=None, summary_vqa_vis_processors=None, summary_vqa_txt_processors=None, summary_vqa_model_new=None, summary_vqa_vis_processors_new=None, summary_vqa_txt_processors_new=None, device_type: str | None = None) +
- +class text.TextAnalyzer(csv_path: str, column_key: str = None, csv_encoding: str = 'utf-8') +
Bases:
+objectUsed to get text from a csv and then run the TextDetector on it.
+-
+
- +read_csv() dict +
Read the CSV file and return the dictionary with the text entries.
+-
+
- Returns: +
dict – The dictionary with the text entries.
+
+
-
+
- +class text.TextDetector(subdict: dict, analyse_text: bool = False, skip_extraction: bool = False, accept_privacy: str = 'PRIVACY_AMMICO')
Bases:
-AnalysisMethod-
-
- -all_allowed_model_types = ['base', 'large', 'vqa', 'blip2_t5_pretrain_flant5xxl', 'blip2_t5_pretrain_flant5xl', 'blip2_t5_caption_coco_flant5xl', 'blip2_opt_pretrain_opt2.7b', 'blip2_opt_pretrain_opt6.7b', 'blip2_opt_caption_coco_opt2.7b', 'blip2_opt_caption_coco_opt6.7b'] -
-
-
- -allowed_analysis_types = ['summary', 'questions', 'summary_and_questions'] -
-
-
- -allowed_model_types = ['base', 'large', 'vqa'] -
-
-
- -allowed_new_model_types = ['blip2_t5_pretrain_flant5xxl', 'blip2_t5_pretrain_flant5xl', 'blip2_t5_caption_coco_flant5xl', 'blip2_opt_pretrain_opt2.7b', 'blip2_opt_pretrain_opt6.7b', 'blip2_opt_caption_coco_opt2.7b', 'blip2_opt_caption_coco_opt6.7b'] -
-
+
- +analyse_image() dict +
Perform text extraction and analysis of the text.
+-
+
- Returns: +
dict – The updated dictionary with text analysis results.
+
+
-
-
- -analyse_image(subdict: dict | None = None, analysis_type: str | None = None, list_of_questions: list[str] | None = None, consequential_questions: bool = False) -
Analyse image with blip_caption model.
+- +get_text_from_image() +
Detect text on the image using Google Cloud Vision API.
+
-
+
- +remove_linebreaks() +
Remove linebreaks from original and translated text.
+
-
+
- +set_keys() dict +
Set the default keys for text analysis.
+-
+
- Returns: +
dict – The dictionary with default text keys.
+
+
-
+
- +translate_text() +
Translate the detected text to English using the Translator object.
+
-
+
- +text.privacy_disclosure(accept_privacy: str = 'PRIVACY_AMMICO') +
Asks the user to accept the privacy statement.
- Parameters: -
-
-
analysis_type (str) – type of the analysis.
-subdict (dict) – dictionary with analising pictures.
-list_of_questions (list[str]) – list of questions.
-consequential_questions (bool) – whether to ask consequential questions. Works only for new BLIP2 models.
-
-- Returns: -
self.subdict (dict) – dictionary with analysis results.
+accept_privacy (str) – The name of the disclosure variable (default: “PRIVACY_AMMICO”).
-
-
- -analyse_questions(list_of_questions: list[str], consequential_questions: bool = False) dict -
Generate answers to free-form questions about image written in natural language.
--
-
- Parameters: -
-
-
list_of_questions (list[str]) – list of questions.
-consequential_questions (bool) – whether to ask consequential questions. Works only for new BLIP2 models.
-
-- Returns: -
self.subdict (dict) – dictionary with answers to questions.
-
-
-
-
- -analyse_summary(nondeterministic_summaries: bool = True) -
Create 1 constant and 3 non deterministic captions for image.
--
-
- Parameters: -
nondeterministic_summaries (bool) – whether to create 3 non deterministic captions.
-
-- Returns: -
self.subdict (dict) – dictionary with analysis results.
-
-
-
-
- -check_model() -
Check model type and return appropriate model and preprocessors.
-Args:
--
-
- Returns: -
-
-
model (nn.Module) – model.
-vis_processors (dict) – visual preprocessor.
-txt_processors (dict) – text preprocessor.
-model_old (bool) – whether model is old or new.
-
-
-
-
- -load_model(model_type: str) -
Load blip_caption model and preprocessors for visual inputs from lavis.models.
--
-
- Parameters: -
model_type (str) – type of the model.
-
-- Returns: -
-
-
summary_model (torch.nn.Module) – model.
-summary_vis_processors (dict) – preprocessors for visual inputs.
-
-
-
-
- -load_model_base() -
Load base_coco blip_caption model and preprocessors for visual inputs from lavis.models.
-Args:
--
-
- Returns: -
-
-
summary_model (torch.nn.Module) – model.
-summary_vis_processors (dict) – preprocessors for visual inputs.
-
-
-
-
- -load_model_base_blip2_opt_caption_coco_opt67b() -
Load BLIP2 model with caption_coco_opt6.7b architecture.
-Args:
--
-
- Returns: -
-
-
model (torch.nn.Module) – model.
-vis_processors (dict) – preprocessors for visual inputs.
-txt_processors (dict) – preprocessors for text inputs.
-
-
-
-
- -load_model_base_blip2_opt_pretrain_opt67b() -
Load BLIP2 model with pretrain_opt6.7b architecture.
-Args:
--
-
- Returns: -
-
-
model (torch.nn.Module) – model.
-vis_processors (dict) – preprocessors for visual inputs.
-txt_processors (dict) – preprocessors for text inputs.
-
-
-
-
- -load_model_blip2_opt_caption_coco_opt27b() -
Load BLIP2 model with caption_coco_opt2.7b architecture.
-Args:
--
-
- Returns: -
-
-
model (torch.nn.Module) – model.
-vis_processors (dict) – preprocessors for visual inputs.
-txt_processors (dict) – preprocessors for text inputs.
-
-
-
-
- -load_model_blip2_opt_pretrain_opt27b() -
Load BLIP2 model with pretrain_opt2 architecture.
-Args:
--
-
- Returns: -
-
-
model (torch.nn.Module) – model.
-vis_processors (dict) – preprocessors for visual inputs.
-txt_processors (dict) – preprocessors for text inputs.
-
-
-
-
- -load_model_blip2_t5_caption_coco_flant5xl() -
Load BLIP2 model with caption_coco_flant5xl architecture.
-Args:
--
-
- Returns: -
-
-
model (torch.nn.Module) – model.
-vis_processors (dict) – preprocessors for visual inputs.
-txt_processors (dict) – preprocessors for text inputs.
-
-
-
-
- -load_model_blip2_t5_pretrain_flant5xl() -
Load BLIP2 model with FLAN-T5 XL architecture.
-Args:
--
-
- Returns: -
-
-
model (torch.nn.Module) – model.
-vis_processors (dict) – preprocessors for visual inputs.
-txt_processors (dict) – preprocessors for text inputs.
-
-
-
-
- -load_model_blip2_t5_pretrain_flant5xxl() -
Load BLIP2 model with FLAN-T5 XXL architecture.
-Args:
--
-
- Returns: -
-
-
model (torch.nn.Module) – model.
-vis_processors (dict) – preprocessors for visual inputs.
-txt_processors (dict) – preprocessors for text inputs.
-
-
-
-
- -load_model_large() -
Load large_coco blip_caption model and preprocessors for visual inputs from lavis.models.
-Args:
--
-
- Returns: -
-
-
summary_model (torch.nn.Module) – model.
-summary_vis_processors (dict) – preprocessors for visual inputs.
-
-
-
-
- -load_new_model(model_type: str) -
Load new BLIP2 models.
--
-
- Parameters: -
model_type (str) – type of the model.
-
-- Returns: -
-
-
model (torch.nn.Module) – model.
-vis_processors (dict) – preprocessors for visual inputs.
-txt_processors (dict) – preprocessors for text inputs.
-
-
-
-
- -load_vqa_model() -
Load blip_vqa model and preprocessors for visual and text inputs from lavis.models.
-Args:
--
-
- Returns: -
-
-
summary_vqa_model (torch.nn.Module) – model.
-summary_vqa_vis_processors (dict) – preprocessors for visual inputs.
-summary_vqa_txt_processors (dict) – preprocessors for text inputs.
-
-
summary module
multimodal search module
@@ -448,7 +243,7 @@faces module
- -class faces.EmotionDetector(subdict: dict, emotion_threshold: float = 50.0, race_threshold: float = 50.0, gender_threshold: float = 50.0, accept_disclosure: str = 'DISCLOSURE_AMMICO') +class faces.EmotionDetector(subdict: dict, emotion_threshold: float = 50.0, race_threshold: float = 50.0, gender_threshold: float = 50.0, accept_disclosure: str = 'DISCLOSURE_AMMICO')
Bases:
AnalysisMethod-
@@ -548,7 +343,7 @@
color_analysis module
-
@@ -548,7 +343,7 @@
cropposts module
+cropposts module
+utils module
+-
+
- +class utils.AnalysisMethod(subdict: dict) +
Bases:
+objectBase class to be inherited by all analysis methods.
+-
+
- +analyse_image() +
-
+
- +set_keys() +
-
+
- +class utils.DownloadResource(**kwargs) +
Bases:
+objectA remote resource that needs on demand downloading.
+We use this as a wrapper to the pooch library. The wrapper registers +each data file and allows prefetching through the CLI entry point +ammico_prefetch_models.
+-
+
- +get() +
-
+
- +resources = [] +
-
-
- -cropposts.compute_crop_corner(matches: DMatch, kp1: ndarray, kp2: ndarray, region: int = 30, h_margin: int = 0, v_margin: int = 5, min_match: int = 6) Tuple[int, int] | None -
Estimate the position on the image from where to crop.
+- +utils.ammico_prefetch_models() +
Prefetch all the download resources
+
-
+
- +utils.append_data_to_dict(mydict: dict) dict +
Append entries from nested dictionaries to keys in a global dict.
+
-
+
- +utils.dump_df(mydict: dict) DataFrame +
Utility to dump the dictionary into a dataframe.
+
-
+
- +utils.find_files(path: str = None, pattern=['png', 'jpg', 'jpeg', 'gif', 'webp', 'avif', 'tiff'], recursive: bool = True, limit=20, random_seed: int = None) dict +
Find image files on the file system.
- Parameters:
-
-
matches (cv2.DMatch) – The matched objects on the image.
-kp1 (np.ndarray) – Key points of the matches for the reference image.
-kp2 (np.ndarray) – Key points of the matches for the social media posts.
-region (int, optional) – Area to consider around the keypoints. -Defaults to 30.
-h_margin (int, optional) – Horizontal margin to subtract from the minimum -horizontal position. Defaults to 0.
-v_margin (int, optional) – Vertical margin to subtract from the minimum -vertical position. Defaults to 5.
-min_match – Minimum number of matches required. Defaults to 6.
+path (str, optional) – The base directory where we are looking for the images. Defaults +to None, which uses the ammico data directory if set or the current +working directory otherwise.
+pattern (str|list, optional) – The naming pattern that the filename should match. +Use either ‘.ext’ or just ‘ext’ +Defaults to [“png”, “jpg”, “jpeg”, “gif”, “webp”, “avif”,”tiff”]. Can be used to allow other patterns or to only include +specific prefixes or suffixes.
+recursive (bool, optional) – Whether to recurse into subdirectories. Default is set to True.
+limit (int/list, optional) – The maximum number of images to be found. +Provide a list or tuple of length 2 to batch the images. +Defaults to 20. To return all images, set to None or -1.
+random_seed (int, optional) – The random seed to use for shuffling the images. +If None is provided the data will not be shuffeled. Defaults to None.
- Returns: -
tuple, optional – Tuple of vertical and horizontal crop corner coordinates.
+dict – A nested dictionary with file ids and all filenames including the path.
-
-
- -cropposts.crop_image_from_post(view: ndarray, final_h: int) ndarray -
Crop the image part from the social media post.
+- +utils.get_color_table() +
-
+
- +utils.get_dataframe(mydict: dict) DataFrame +
-
+
- +utils.initialize_dict(filelist: list) dict +
Initialize the nested dictionary for all the found images.
- Parameters: -
-
-
view (np.ndarray) – The image to be cropped.
-final_h – The horizontal position up to which should be cropped.
-
filelist (list) – The list of files to be analyzed, including their paths.
- Returns: -
np.ndarray – The cropped image part.
+dict – The nested dictionary with all image ids and their paths.
-
-
- -cropposts.crop_media_posts(files, ref_files, save_crop_dir, plt_match=False, plt_crop=False, plt_image=False) None -
Crop social media posts so that comments beyond the first comment/post are cut off.
--
-
- Parameters: -
-
-
files (list) – List of all the files to be cropped.
-ref_files (list) – List of all the reference images that signify -below which regions should be cropped.
-save_crop_dir (str) – Directory where to write the cropped social media posts to.
-plt_match (Bool, optional) – Display the matched areas on the social media post. -Defaults to False.
-plt_crop (Bool, optional) – Display the cropped text part of the social media post. -Defaults to False.
-plt_image (Bool, optional) – Display the image part of the social media post. -Defaults to False.
-
-
- +utils.is_interactive() +
Check if we are running in an interactive environment.
-
-
- -cropposts.crop_posts_from_refs(ref_views: List, view: ndarray, plt_match: bool = False, plt_crop: bool = False, plt_image: bool = False) ndarray -
Crop the social media post comments from the image.
--
-
- Parameters: -
-
-
ref_views (list) – List of all the reference images (as numpy arrays) that signify -below which regions should be cropped.
-view (np.ndarray) – The image to crop.
-
-- Returns: -
np.ndarray – The cropped social media post.
-
-
-
-
- -cropposts.crop_posts_image(ref_view: List, view: ndarray) None | Tuple[ndarray, int, int, int] -
Crop the social media post to exclude additional comments. Sometimes also crops the -image part of the post - this is put back in later.
--
-
- Parameters: -
-
-
ref_views (list) – List of all the reference images (as numpy arrays) that signify -below which regions should be cropped.
-view (np.ndarray) – The image to crop.
-
-- Returns: -
np.ndarray – The cropped social media post.
-
-
-
-
- -cropposts.draw_matches(matches: List, img1: ndarray, img2: ndarray, kp1: List[KeyPoint], kp2: List[KeyPoint]) None -
Visualize the matches from SIFT.
--
-
- Parameters: -
-
-
matches (list[cv2.Match]) – List of cv2.Match matches on the image.
-img1 (np.ndarray) – The reference image.
-img2 (np.ndarray) – The social media post.
-kp1 (list[cv2.KeyPoint]) – List of keypoints from the first image.
-kp2 (list[cv2.KeyPoint]) – List of keypoints from the second image.
-
-
-
-
- -cropposts.kp_from_matches(matches, kp1: ndarray, kp2: ndarray) Tuple[Tuple, Tuple] -
Extract the match indices from the keypoints.
--
-
- Parameters: -
-
-
kp1 (np.ndarray) – Key points of the matches,
-kp2 (np.ndarray) – Key points of the matches,
-
-- Returns: -
-
-
tuple – Index of the descriptor in the list of train descriptors.
-tuple – index of the descriptor in the list of query descriptors.
-
-
-
-
- -cropposts.matching_points(img1: ndarray, img2: ndarray) Tuple[DMatch, List[KeyPoint], List[KeyPoint]] -
Computes keypoint matches using the SIFT algorithm between two images.
--
-
- Parameters: -
-
-
img1 (np.ndarray) – The reference image.
-img2 (np.ndarray) – The social media post.
-
-- Returns: -
-
-
cv2.DMatch – List of filtered keypoint matches.
-cv2.KeyPoint – List of keypoints from the first image.
-cv2.KeyPoint – List of keypoints from the second image.
-
-
-
-
- -cropposts.paste_image_and_comment(crop_post: ndarray, crop_view: ndarray) ndarray -
Paste the image part and the text part together without the unecessary comments.
--
-
- Parameters: -
-
-
crop_post (np.ndarray) – The cropped image part of the social media post.
-crop_view (np.ndarray) – The cropped text part of the social media post.
-
-- Returns: -
np.ndarray – The image and text part of the social media post in one image.
-
-
utils module
-display module
+display module
+-
+
- +class display.AnalysisExplorer(mydict: dict) +
Bases:
+object-
+
- +run_server(port: int = 8050) None +
Run the Dash server to start the analysis explorer.
+-
+
- Parameters: +
port (int, optional) – The port number to run the server on (default: 8050).
+
+
-
+
- +update_picture(img_path: str) +
Callback function to update the displayed image.
+-
+
- Parameters: +
img_path (str) – The path of the selected image.
+
+- Returns: +
Union[PIL.PngImagePlugin, None] – The image object to be displayed +or None if the image path is
+
+
