{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# Objects recognition" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "This notebooks shows how to detect objects quickly using [cvlib](https://github.com/arunponnusamy/cvlib) and the [YOLOv4](https://github.com/AlexeyAB/darknet) model. This library detects faces, people, and several inanimate objects; we currently have restricted the output to person, bicycle, car, motorcycle, airplane, bus, train, truck, boat, traffic light, cell phone.\n", "\n", "The first cell is only run on google colab and installs the [ammico](https://github.com/ssciwr/AMMICO) package.\n", "\n", "After that, we can import `ammico` and read in the files given a folder path." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2023-06-23T12:20:17.146895Z", "iopub.status.busy": "2023-06-23T12:20:17.146513Z", "iopub.status.idle": "2023-06-23T12:20:17.155906Z", "shell.execute_reply": "2023-06-23T12:20:17.155333Z" } }, "outputs": [], "source": [ "# if running on google colab\n", "# flake8-noqa-cell\n", "import os\n", "\n", "if \"google.colab\" in str(get_ipython()):\n", " # update python version\n", " # install setuptools\n", " # %pip install setuptools==61 -qqq\n", " # install ammico\n", " %pip install git+https://github.com/ssciwr/ammico.git -qqq\n", " # mount google drive for data and API key\n", " from google.colab import drive\n", "\n", " drive.mount(\"/content/drive\")" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2023-06-23T12:20:17.158651Z", "iopub.status.busy": "2023-06-23T12:20:17.158320Z", "iopub.status.idle": "2023-06-23T12:20:31.039101Z", "shell.execute_reply": "2023-06-23T12:20:31.038463Z" } }, "outputs": [], "source": [ "import ammico\n", "from ammico import utils as mutils\n", "from ammico import display as mdisplay\n", "import ammico.objects as ob" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Set an image path as input file path." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2023-06-23T12:20:31.042879Z", "iopub.status.busy": "2023-06-23T12:20:31.042154Z", "iopub.status.idle": "2023-06-23T12:20:31.046405Z", "shell.execute_reply": "2023-06-23T12:20:31.045745Z" } }, "outputs": [], "source": [ "# Here you need to provide the path to your google drive folder\n", "# or local folder containing the images\n", "images = mutils.find_files(\n", " path=\"data/\",\n", " limit=10,\n", ")" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2023-06-23T12:20:31.049117Z", "iopub.status.busy": "2023-06-23T12:20:31.048785Z", "iopub.status.idle": "2023-06-23T12:20:31.051944Z", "shell.execute_reply": "2023-06-23T12:20:31.051301Z" } }, "outputs": [], "source": [ "mydict = mutils.initialize_dict(images)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Detect objects and directly write to csv\n", "You can directly carry out the analysis and export the result into a csv. This may take a while depending on how many images you have loaded." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2023-06-23T12:20:31.054874Z", "iopub.status.busy": "2023-06-23T12:20:31.054359Z", "iopub.status.idle": "2023-06-23T12:20:38.916763Z", "shell.execute_reply": "2023-06-23T12:20:38.916053Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloading yolov4.cfg from https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4.cfg\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 0% | |\r" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Downloading yolov4.weights from https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.weights\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 0% | |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 1% | |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 2% |# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 3% |## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 4% |## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 5% |### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 6% |#### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 7% |##### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 8% |##### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 9% |###### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 10% |####### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 11% |####### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 12% |######## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 13% |######### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 14% |########## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 15% |########## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 16% |########### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 17% |############ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 18% |############ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 19% |############# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 20% |############## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 21% |############### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 22% |############### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 23% |################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 24% |################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 25% |################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 26% |################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 27% |################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 28% |#################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 29% |#################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 30% |##################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 31% |###################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 32% |####################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 33% |####################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 34% |######################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 35% |######################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 36% |######################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 37% |########################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 38% |########################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 39% |############################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 40% |############################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 41% |############################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 42% |############################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 43% |############################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 44% |############################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 45% |################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 46% |################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 47% |################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 48% |################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 49% |################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 50% |#################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 51% |#################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 52% |##################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 53% |###################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 54% |###################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 55% |####################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 56% |######################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 57% |######################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 58% |######################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 59% |########################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 60% |########################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 61% |########################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 62% |############################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 63% |############################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 64% |############################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 65% |############################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 66% |############################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 67% |################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 68% |################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 69% |################################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 70% |################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 71% |################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 72% |################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 73% |#################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 74% |##################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 75% |###################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 76% |###################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 77% |####################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 78% |######################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 79% |######################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 80% |######################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 81% |########################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 82% |########################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 83% |########################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 84% |############################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 85% |############################################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 86% |############################################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 87% |############################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 88% |############################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 89% |################################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 90% |################################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 91% |################################################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 92% |################################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 93% |################################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 94% |################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 95% |#################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 96% |##################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 97% |##################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 98% |###################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 99% |####################################################################### |\r" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Downloading yolov3_classes.txt from https://github.com/arunponnusamy/object-detection-opencv/raw/master/yolov3.txt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100% | |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100% | |\r" ] } ], "source": [ "for key in mydict:\n", " mydict[key] = ob.ObjectDetector(mydict[key]).analyse_image()" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Convert the dictionary of dictionarys into a dictionary with lists:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2023-06-23T12:20:38.921021Z", "iopub.status.busy": "2023-06-23T12:20:38.920460Z", "iopub.status.idle": "2023-06-23T12:20:38.925166Z", "shell.execute_reply": "2023-06-23T12:20:38.924589Z" } }, "outputs": [], "source": [ "outdict = mutils.append_data_to_dict(mydict)\n", "df = mutils.dump_df(outdict)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Check the dataframe:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2023-06-23T12:20:38.927656Z", "iopub.status.busy": "2023-06-23T12:20:38.927443Z", "iopub.status.idle": "2023-06-23T12:20:38.941909Z", "shell.execute_reply": "2023-06-23T12:20:38.941288Z" } }, "outputs": [ { "data": { "text/html": [ "
| \n", " | filename | \n", "person | \n", "bicycle | \n", "car | \n", "motorcycle | \n", "airplane | \n", "bus | \n", "train | \n", "truck | \n", "boat | \n", "traffic light | \n", "cell phone | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "data/106349S_por.png | \n", "yes | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "yes | \n", "
| 1 | \n", "data/102141_2_eng.png | \n", "yes | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "
| 2 | \n", "data/102730_eng.png | \n", "yes | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "yes | \n", "no | \n", "no | \n", "no | \n", "