{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Objects Expression recognition" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "This notebooks shows some preliminary work on detecting objects expressions with cvlib. It is mainly meant to explore its capabilities and to decide on future research directions. We package our code into a `ammico` package that is imported here:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2023-05-17T07:56:38.075081Z", "iopub.status.busy": "2023-05-17T07:56:38.074628Z", "iopub.status.idle": "2023-05-17T07:56:51.738080Z", "shell.execute_reply": "2023-05-17T07:56:51.737175Z" }, "tags": [] }, "outputs": [], "source": [ "import ammico\n", "from ammico import utils as mutils\n", "from ammico import display as mdisplay\n", "import ammico.objects as ob" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Set an image path as input file path." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2023-05-17T07:56:51.742895Z", "iopub.status.busy": "2023-05-17T07:56:51.741831Z", "iopub.status.idle": "2023-05-17T07:56:51.746630Z", "shell.execute_reply": "2023-05-17T07:56:51.745796Z" }, "tags": [] }, "outputs": [], "source": [ "images = mutils.find_files(\n", " path=\"data/\",\n", " limit=10,\n", ")" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2023-05-17T07:56:51.750268Z", "iopub.status.busy": "2023-05-17T07:56:51.749997Z", "iopub.status.idle": "2023-05-17T07:56:51.754088Z", "shell.execute_reply": "2023-05-17T07:56:51.753041Z" }, "tags": [] }, "outputs": [], "source": [ "mydict = mutils.initialize_dict(images)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Manually inspect what was detected\n", "\n", "To check the analysis, you can inspect the analyzed elements here. Loading the results takes a moment, so please be patient. If you are sure of what you are doing." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2023-05-17T07:56:51.759766Z", "iopub.status.busy": "2023-05-17T07:56:51.759262Z", "iopub.status.idle": "2023-05-17T07:56:53.079877Z", "shell.execute_reply": "2023-05-17T07:56:53.078969Z" }, "tags": [] }, "outputs": [ { "ename": "AttributeError", "evalue": "module 'ammico.display' has no attribute 'explore_analysis'", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[4], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mmdisplay\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mexplore_analysis\u001b[49m(mydict, identify\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mobjects\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n", "\u001b[0;31mAttributeError\u001b[0m: module 'ammico.display' has no attribute 'explore_analysis'" ] } ], "source": [ "mdisplay.explore_analysis(mydict, identify=\"objects\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Detect objects and directly write to csv" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2023-05-17T07:56:53.085011Z", "iopub.status.busy": "2023-05-17T07:56:53.084350Z", "iopub.status.idle": "2023-05-17T07:57:02.014454Z", "shell.execute_reply": "2023-05-17T07:57:02.013565Z" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloading yolov4.cfg from https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4.cfg\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 0% | |\r" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Downloading yolov4.weights from https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.weights\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 0% | |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 1% | |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 2% |# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 3% |## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 4% |## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 5% |### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 6% |#### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 7% |##### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 8% |##### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 9% |###### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 10% |####### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 11% |####### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 12% |######## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 13% |######### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 14% |########## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 15% |########## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 16% |########### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 17% |############ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 18% |############ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 19% |############# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 20% |############## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 21% |############### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 22% |############### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 23% |################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 24% |################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 25% |################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 26% |################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 27% |################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 28% |#################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 29% |#################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 30% |##################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 31% |###################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 32% |####################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 33% |####################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 34% |######################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 35% |######################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 36% |######################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 37% |########################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 38% |########################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 39% |############################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 40% |############################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 41% |############################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 42% |############################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 43% |############################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 44% |############################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 45% |################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 46% |################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 47% |################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 48% |################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 49% |################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 50% |#################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 51% |#################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 52% |##################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 53% |###################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 54% |###################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 55% |####################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 56% |######################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 57% |######################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 58% |######################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 59% |########################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 60% |########################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 61% |########################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 62% |############################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 63% |############################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 64% |############################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 65% |############################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 66% |############################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 67% |################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 68% |################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 69% |################################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 70% |################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 71% |################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 72% |################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 73% |#################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 74% |##################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 75% |###################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 76% |###################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 77% |####################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 78% |######################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 79% |######################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 80% |######################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 81% |########################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 82% |########################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 83% |########################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 84% |############################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 85% |############################################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 86% |############################################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 87% |############################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 88% |############################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 89% |################################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 90% |################################################################ |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 91% |################################################################# |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 92% |################################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 93% |################################################################## |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 94% |################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 95% |#################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 96% |##################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 97% |##################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 98% |###################################################################### |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 99% |####################################################################### |\r" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Downloading yolov3_classes.txt from https://github.com/arunponnusamy/object-detection-opencv/raw/master/yolov3.txt\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100% | |\r" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100% | |\r" ] } ], "source": [ "for key in mydict:\n", " mydict[key] = ob.ObjectDetector(mydict[key]).analyse_image()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Convert the dictionary of dictionarys into a dictionary with lists:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2023-05-17T07:57:02.019115Z", "iopub.status.busy": "2023-05-17T07:57:02.018450Z", "iopub.status.idle": "2023-05-17T07:57:02.023922Z", "shell.execute_reply": "2023-05-17T07:57:02.023227Z" }, "tags": [] }, "outputs": [], "source": [ "outdict = mutils.append_data_to_dict(mydict)\n", "df = mutils.dump_df(outdict)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Check the dataframe:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2023-05-17T07:57:02.027079Z", "iopub.status.busy": "2023-05-17T07:57:02.026812Z", "iopub.status.idle": "2023-05-17T07:57:02.044114Z", "shell.execute_reply": "2023-05-17T07:57:02.043348Z" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
| \n", " | filename | \n", "person | \n", "bicycle | \n", "car | \n", "motorcycle | \n", "airplane | \n", "bus | \n", "train | \n", "truck | \n", "boat | \n", "traffic light | \n", "cell phone | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "data/102730_eng.png | \n", "yes | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "yes | \n", "no | \n", "no | \n", "no | \n", "
| 1 | \n", "data/102141_2_eng.png | \n", "yes | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "
| 2 | \n", "data/106349S_por.png | \n", "yes | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "no | \n", "yes | \n", "