{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# Objects recognition" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "This notebooks shows how to detect objects quickly using [cvlib](https://github.com/arunponnusamy/cvlib) and the [YOLOv4](https://github.com/AlexeyAB/darknet) model. This library detects faces, people, and several inanimate objects; we currently have restricted the output to person, bicycle, car, motorcycle, airplane, bus, train, truck, boat, traffic light, cell phone.\n", "\n", "The first cell is only run on google colab and installs the [ammico](https://github.com/ssciwr/AMMICO) package.\n", "\n", "After that, we can import `ammico` and read in the files given a folder path." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2023-06-28T06:55:52.977606Z", "iopub.status.busy": "2023-06-28T06:55:52.977235Z", "iopub.status.idle": "2023-06-28T06:55:52.985673Z", "shell.execute_reply": "2023-06-28T06:55:52.984772Z" } }, "outputs": [], "source": [ "# if running on google colab\n", "# flake8-noqa-cell\n", "import os\n", "\n", "if \"google.colab\" in str(get_ipython()):\n", " # update python version\n", " # install setuptools\n", " # %pip install setuptools==61 -qqq\n", " # install ammico\n", " %pip install git+https://github.com/ssciwr/ammico.git -qqq\n", " # mount google drive for data and API key\n", " from google.colab import drive\n", "\n", " drive.mount(\"/content/drive\")" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2023-06-28T06:55:52.988359Z", "iopub.status.busy": "2023-06-28T06:55:52.988145Z", "iopub.status.idle": "2023-06-28T06:56:05.900716Z", "shell.execute_reply": "2023-06-28T06:56:05.900069Z" } }, "outputs": [], "source": [ "import ammico\n", "from ammico import utils as mutils\n", "from ammico import display as mdisplay\n", "import ammico.objects as ob" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Set an image path as input file path." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2023-06-28T06:56:05.905170Z", "iopub.status.busy": "2023-06-28T06:56:05.903583Z", "iopub.status.idle": "2023-06-28T06:56:05.909863Z", "shell.execute_reply": "2023-06-28T06:56:05.909290Z" } }, "outputs": [], "source": [ "# Here you need to provide the path to your google drive folder\n", "# or local folder containing the images\n", "images = mutils.find_files(\n", " path=\"data/\",\n", " limit=10,\n", ")" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2023-06-28T06:56:05.912879Z", "iopub.status.busy": "2023-06-28T06:56:05.912550Z", "iopub.status.idle": "2023-06-28T06:56:05.916852Z", "shell.execute_reply": "2023-06-28T06:56:05.916259Z" } }, "outputs": [], "source": [ "mydict = mutils.initialize_dict(images)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Detect objects and directly write to csv\n", "You can directly carry out the analysis and export the result into a csv. This may take a while depending on how many images you have loaded." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2023-06-28T06:56:05.919860Z", "iopub.status.busy": "2023-06-28T06:56:05.919411Z", "iopub.status.idle": "2023-06-28T06:56:13.454060Z", "shell.execute_reply": "2023-06-28T06:56:13.453412Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ " 0% | |\r" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Downloading yolov4.cfg from https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4.cfg\n", "Downloading yolov4.weights from https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.weights\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 0% | |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 1% | |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 2% |# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 3% |## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 4% |## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 5% |### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 6% |#### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 7% |##### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 8% |##### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 9% |###### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 10% |####### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 11% |####### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 12% |######## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 13% |######### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 14% |########## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 15% |########## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 16% |########### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 17% |############ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 18% |############ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 19% |############# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 20% |############## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 21% |############### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 22% |############### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 23% |################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 24% |################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 25% |################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 26% |################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 27% |################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 28% |#################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 29% |#################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 30% |##################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 31% |###################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 32% |####################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 33% |####################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 34% |######################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 35% |######################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 36% |######################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 37% |########################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 38% |########################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 39% |############################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 40% |############################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 41% |############################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 42% |############################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 43% |############################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 44% |############################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 45% |################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 46% |################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 47% |################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 48% |################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 49% |################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 50% |#################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 51% |#################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 52% |##################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 53% |###################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 54% |###################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 55% |####################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 56% |######################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 57% |######################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 58% |######################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 59% |########################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 60% |########################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 61% |########################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 62% |############################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 63% |############################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 64% |############################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 65% |############################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 66% |############################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 67% |################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 68% |################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 69% |################################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 70% |################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 71% |################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 72% |################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 73% |#################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 74% |##################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 75% |###################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 76% |###################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 77% |####################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 78% |######################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 79% |######################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 80% |######################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 81% |########################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 82% |########################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 83% |########################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 84% |############################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 85% |############################################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 86% |############################################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 87% |############################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 88% |############################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 89% |################################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 90% |################################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 91% |################################################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 92% |################################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 93% |################################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 94% |################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 95% |#################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 96% |##################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 97% |##################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 98% |###################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 99% |####################################################################### |\r" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Downloading yolov3_classes.txt from https://github.com/arunponnusamy/object-detection-opencv/raw/master/yolov3.txt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100% | |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100% | |\r" ] } ], "source": [ "for key in mydict:\n", " mydict[key] = ob.ObjectDetector(mydict[key]).analyse_image()" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Convert the dictionary of dictionarys into a dictionary with lists:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2023-06-28T06:56:13.457836Z", "iopub.status.busy": "2023-06-28T06:56:13.457210Z", "iopub.status.idle": "2023-06-28T06:56:13.461136Z", "shell.execute_reply": "2023-06-28T06:56:13.460631Z" } }, "outputs": [], "source": [ "outdict = mutils.append_data_to_dict(mydict)\n", "df = mutils.dump_df(outdict)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Check the dataframe:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2023-06-28T06:56:13.463949Z", "iopub.status.busy": "2023-06-28T06:56:13.463377Z", "iopub.status.idle": "2023-06-28T06:56:13.476816Z", "shell.execute_reply": "2023-06-28T06:56:13.476320Z" } }, "outputs": [ { "data": { "text/html": [ "
| \n", " | filename | \n", "person | \n", "bicycle | \n", "car | \n", "motorcycle | \n", "airplane | \n", "bus | \n", "train | \n", "truck | \n", "boat | \n", "traffic light | \n", "cell phone | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "data/106349S_por.png | \n", "yes | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "yes | \n", "
| 1 | \n", "data/102141_2_eng.png | \n", "yes | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "
| 2 | \n", "data/102730_eng.png | \n", "yes | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "yes | \n", "no | \n", "no | \n", "no | \n", "