more changes to doc (#188)

* more changes to doc

* write output after each detector

README.md

@@ -104,16 +104,12 @@ Be careful, it requires around 7 GB of disk space.
 
 ## Usage
 
-The main demonstration notebook can be found in the `notebooks` folder and also on [google colab](https://colab.research.google.com/github/ssciwr/ammico/blob/main/ammico/notebooks/DemoNotebook_ammico.ipynb)
+The main demonstration notebook can be found in the `notebooks` folder and also on [google colab](https://colab.research.google.com/github/ssciwr/ammico/blob/main/ammico/notebooks/DemoNotebook_ammico.ipynb).
 
 There are further sample notebooks in the `notebooks` folder for the more experimental features:
 1. Topic analysis: Use the notebook `get-text-from-image.ipynb` to analyse the topics of the extraced text.\
 **You can run this notebook on google colab: [Here](https://colab.research.google.com/github/ssciwr/ammico/blob/main/ammico/notebooks/get-text-from-image.ipynb)**  
 Place the data files and google cloud vision API key in your google drive to access the data.
-1. Multimodal content: Use the notebook `multimodal_search.ipynb` to find the best fitting images to an image or text query.
-**You can run this notebook on google colab: [Here](https://colab.research.google.com/github/ssciwr/ammico/blob/main/ammico/notebooks/multimodal_search.ipynb)**
-1. Color analysis: Use the notebook `color_analysis.ipynb` to identify colors the image. The colors are then classified into the main named colors in the English language.
-**You can run this notebook on google colab: [Here](https://colab.research.google.com/github/ssciwr/ammico/blob/main/ammico/notebooks/colors_analysis.ipynb)**
 1. To crop social media posts use the `cropposts.ipynb` notebook. 
 **You can run this notebook on google colab: [Here](https://colab.research.google.com/github/ssciwr/ammico/blob/main/ammico/notebooks/cropposts.ipynb)**
 

ammico/notebooks/DemoNotebook_ammico.ipynb

@@ -18,7 +18,10 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# if running on google colab\n",
+    "# if running on google colab\\\n",
+    "# PLEASE RUN THIS ONLY AS CPU RUNTIME\n",
+    "# for a GPU runtime, there are conflicts with pre-installed packages - \n",
+    "# you first need to uninstall them (prepare a clean environment with no pre-installs) and then install ammico\n",
     "# flake8-noqa-cell\n",
     "\n",
     "if \"google.colab\" in str(get_ipython()):\n",
@@ -26,7 +29,7 @@
     "    # install setuptools\n",
     "    # %pip install setuptools==61 -qqq\n",
     "    # uninstall some pre-installed packages due to incompatibility\n",
-    "    %pip uninstall tensorflow-probability dopamine-rl lida pandas-gbq torchaudio torchdata torchtext orbax-checkpoint flex-y -qqq\n",
+    "    %pip uninstall --yes tensorflow-probability dopamine-rl lida pandas-gbq torchaudio torchdata torchtext orbax-checkpoint flex-y -qqq\n",
     "    # install ammico\n",
     "    %pip install git+https://github.com/ssciwr/ammico.git -qqq\n",
     "    # mount google drive for data and API key\n",
@@ -150,7 +153,9 @@
     "\n",
     "The `find_files` function returns a nested dict that contains the file ids and the paths to the files and is empty otherwise. This dict is filled step by step with more data as each detector class is run on the data (see below).\n",
     "\n",
-    "If you downloaded the test dataset above, you can directly provide the path you already set for the test directory, `data_path`."
+    "If you downloaded the test dataset above, you can directly provide the path you already set for the test directory, `data_path`. The below cell is already set up for the test dataset.\n",
+    "\n",
+    "If you use your own dataset, you need to toggle the path and provide the directory where you have saved your data."
    ]
   },
   {
@@ -267,7 +272,6 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# clear memory on cuda first? Faces seems to always not release GPU\n",
     "# initialize the models\n",
     "image_summary_detector = ammico.SummaryDetector(subdict = image_dict, analysis_type=\"summary\", model_type=\"base\")\n",
     "\n",
@@ -414,6 +418,17 @@
     "        image_df.to_csv(dump_file)"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write output to csv\n",
+    "image_df = ammico.get_dataframe(image_dict)\n",
+    "image_df.to_csv(\"/content/drive/MyDrive/misinformation-data/data_out.csv\")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -584,7 +599,7 @@
    "metadata": {},
    "source": [
     "### BLIP2 models\n",
-    "This is very heavy models. They requare approx 60GB of RAM and they can use more than 20GB GPUs memory."
+    "The BLIP2 models are computationally very heavy models, and require approximately 60GB of RAM. These models can easily use more than 20GB GPU memory."
    ]
   },
   {
@@ -610,30 +625,6 @@
     "#also you can perform all calculation on cpu if you set device_type= \"cpu\" or gpu if you set device_type= \"cuda\""
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "for key in image_dict:\n",
-    "    image_dict[key] = obj.analyse_image(subdict = image_dict[key], analysis_type=\"summary_and_questions\")\n",
-    "\n",
-    "# analysis_type can be \n",
-    "# \"summary\",\n",
-    "# \"questions\",\n",
-    "# \"summary_and_questions\"."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "image_dict"
-   ]
-  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -662,7 +653,12 @@
    "outputs": [],
    "source": [
     "for key in image_dict:\n",
-    "    image_dict[key] = obj.analyse_image(subdict = image_dict[key], analysis_type=\"questions\", list_of_questions=list_of_questions)"
+    "    image_dict[key] = obj.analyse_image(subdict = image_dict[key], analysis_type=\"questions\", list_of_questions=list_of_questions)\n",
+    "\n",
+    "# analysis_type can be \n",
+    "# \"summary\",\n",
+    "# \"questions\",\n",
+    "# \"summary_and_questions\"."
    ]
   },
   {
@@ -743,6 +739,17 @@
     "image_dict"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write output to csv\n",
+    "image_df = ammico.get_dataframe(image_dict)\n",
+    "image_df.to_csv(\"/content/drive/MyDrive/misinformation-data/data_out.csv\")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -774,6 +781,17 @@
     "    image_dict[key] = ammico.EmotionDetector(image_dict[key], emotion_threshold=50, race_threshold=50).analyse_image()"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write output to csv\n",
+    "image_df = ammico.get_dataframe(image_dict)\n",
+    "image_df.to_csv(\"/content/drive/MyDrive/misinformation-data/data_out.csv\")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -884,7 +902,7 @@
    "source": [
     "The images are then processed and stored in a numerical representation, a tensor. These tensors do not change for the same image and same model - so if you run this analysis once, and save the tensors giving a path with the keyword `path_to_save_tensors`, a file with filename `.<Number_of_images>_<model_name>_saved_features_image.pt` will be placed there.\n",
     "\n",
-    "This can save you time if you want to analyse same images with the same model but different questions. To run using the saved tensors, execute the below code giving the path and name of the tensor file."
+    "This can save you time if you want to analyse the same images with the same model but different questions. To run using the saved tensors, execute the below code giving the path and name of the tensor file. Any subsequent query of the model will run in a fraction of the time than it run in initially."
    ]
   },
   {
@@ -893,6 +911,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "# uncomment the code below if you want to load the tensors from the drive\n",
+    "# and just want to ask different questions for the same set of images\n",
     "# (\n",
     "#     model,\n",
     "#     vis_processors,\n",

docs/source/notebooks/DemoNotebook_ammico.ipynb

@@ -26,7 +26,7 @@
     "    # install setuptools\n",
     "    # %pip install setuptools==61 -qqq\n",
     "    # uninstall some pre-installed packages due to incompatibility\n",
-    "    %pip uninstall tensorflow-probability dopamine-rl lida pandas-gbq torchaudio torchdata torchtext orbax-checkpoint flex-y -qqq\n",
+    "    %pip uninstall --yes tensorflow-probability dopamine-rl lida pandas-gbq torchaudio torchdata torchtext orbax-checkpoint flex-y -qqq\n",
     "    # install ammico\n",
     "    %pip install git+https://github.com/ssciwr/ammico.git -qqq\n",
     "    # mount google drive for data and API key\n",
@@ -150,7 +150,9 @@
     "\n",
     "The `find_files` function returns a nested dict that contains the file ids and the paths to the files and is empty otherwise. This dict is filled step by step with more data as each detector class is run on the data (see below).\n",
     "\n",
-    "If you downloaded the test dataset above, you can directly provide the path you already set for the test directory, `data_path`."
+    "If you downloaded the test dataset above, you can directly provide the path you already set for the test directory, `data_path`. The below cell is already set up for the test dataset.\n",
+    "\n",
+    "If you use your own dataset, you need to toggle the path and provide the directory where you have saved your data."
    ]
   },
   {
@@ -267,7 +269,6 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# clear memory on cuda first? Faces seems to always not release GPU\n",
     "# initialize the models\n",
     "image_summary_detector = ammico.SummaryDetector(subdict = image_dict, analysis_type=\"summary\", model_type=\"base\")\n",
     "\n",
@@ -584,7 +585,7 @@
    "metadata": {},
    "source": [
     "### BLIP2 models\n",
-    "This is very heavy models. They requare approx 60GB of RAM and they can use more than 20GB GPUs memory."
+    "The BLIP2 models are computationally very heavy models, and require approximately 60GB of RAM. These models can easily use more than 20GB GPU memory."
    ]
   },
   {
@@ -610,30 +611,6 @@
     "#also you can perform all calculation on cpu if you set device_type= \"cpu\" or gpu if you set device_type= \"cuda\""
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "for key in image_dict:\n",
-    "    image_dict[key] = obj.analyse_image(subdict = image_dict[key], analysis_type=\"summary_and_questions\")\n",
-    "\n",
-    "# analysis_type can be \n",
-    "# \"summary\",\n",
-    "# \"questions\",\n",
-    "# \"summary_and_questions\"."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "image_dict"
-   ]
-  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -662,7 +639,12 @@
    "outputs": [],
    "source": [
     "for key in image_dict:\n",
-    "    image_dict[key] = obj.analyse_image(subdict = image_dict[key], analysis_type=\"questions\", list_of_questions=list_of_questions)"
+    "    image_dict[key] = obj.analyse_image(subdict = image_dict[key], analysis_type=\"questions\", list_of_questions=list_of_questions)\n",
+    "\n",
+    "# analysis_type can be \n",
+    "# \"summary\",\n",
+    "# \"questions\",\n",
+    "# \"summary_and_questions\".\n"
    ]
   },
   {
@@ -884,7 +866,7 @@
    "source": [
     "The images are then processed and stored in a numerical representation, a tensor. These tensors do not change for the same image and same model - so if you run this analysis once, and save the tensors giving a path with the keyword `path_to_save_tensors`, a file with filename `.<Number_of_images>_<model_name>_saved_features_image.pt` will be placed there.\n",
     "\n",
-    "This can save you time if you want to analyse same images with the same model but different questions. To run using the saved tensors, execute the below code giving the path and name of the tensor file."
+    "This can save you time if you want to analyse same images with the same model but different questions. To run using the saved tensors, execute the below code giving the path and name of the tensor file. Any subsequent query of the model will run in a fraction of the time than it run in initially."
    ]
   },
   {
@@ -893,6 +875,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "# uncomment the code below if you want to load the tensors from the drive\n",
+    "# and just want to ask different questions for the same set of images\n",
     "# (\n",
     "#     model,\n",
     "#     vis_processors,\n",

docs/source/set_up_credentials.md

@@ -41,4 +41,4 @@
 20. Google Cloud Vision API is now enabled for your key.
 ![img19](img19.png)
 21. Place the JSON key in a selected folder on your computer and reference this key in your Jupyter Notebook / Python console when running ammico. Or, upload it to your google Drive to use it on google Colaboratory.
-
+22. Make sure that [billing is enabled](https://support.google.com/googleapi/answer/6158867?hl=en) for your google account. You can get the first three month for free; after that, you will be charged if processing more than 1000 images / month (currently $1.50 per 1000 images, see [here](https://cloud.google.com/vision/pricing/)).