GSoc/2023/StatusReports/QuocHungTran
Add Automatic Tags Assignment Tools and Improve Face Recognition Engine for digiKam
digiKam is an advanced open-source digital photo management application that runs on Linux, Windows, and macOS. The application provides a comprehensive set of tools for importing, managing, editing, and sharing photos and raw files.
The goal of this project is to develop a deep learning model that can recognize various categories of objects, scenes, and events in digital photos, and generate corresponding keywords that can be stored in Digikam's database and assigned to each photo automatically. The model should be able to recognize objects such as animals, plants, and vehicles, scenes such as beaches, mountains, and cities,... The model should also be able to handle photos taken in various lighting conditions and from different angles.
Mentors : Gilles Caulier, Maik Qualmann, Thanh Trung Dinh
Project Proposal
Automatic Tags Assignment Tools and Improve Face Recognition Engine for digiKam Proposal
GitLab development branch
Contacts
Email: [email protected]
Github: quochungtran
Invent KDE: quochungtran
LinkedIn: https://www.linkedin.com/in/tran-quoc-hung-6362821b3/
Project goals
Links to Blogs and other writing
Main merge request
KDE repository for object detection and face recognition researching
Issue tracker
My blog for GSoC
My entire blog :
May 29 to June 11 (Week 1 - 2) - Experimentation on COCO dataset
In this phase, I focus mainly on offline analysis, this analysis aims to create a Deep learning pipeline for object detection model
DONE
- Constructing data sets (training dataset, validation dataset and testing dataset) firsly in some common kind of objects as person, bicycle, car.
- Preprocessing data, studying about construct of COCO dataset which is used for training dataset and validation dataset.
- Research and create model pipeline for all YOLO version in python.
- Evaluate performance of YOLO methode by considering some evaluated metrics.
TODO
Construct of COCO dataset format
The Common Object in Context (COCO) is one of the most popular large-scale labeled image datasets available for public use. It represents a handful of objects we encounter on a daily basis and contains image annotations in 80 categories, with over 1.5 million object instances. You can explore COCO dataset by visiting SuperAnnotate’s respective dataset section.
COCO stores data in a JSON file formatted by info, licenses, categories, images, and annotations. For downloading COCO dataset reason, I used the instances_train2017.json and instances_val2017.json files.
"info": { "year": "2021", "version": "1.0", "description": "Exported from FiftyOne", "contributor": "Voxel51", "url": "https://fiftyone.ai", "date_created": "2021-01-19T09:48:27" }, "licenses": [ { "url": "http://creativecommons.org/licenses/by-nc-sa/2.0/", "id": 1, "name": "Attribution-NonCommercial-ShareAlike License" }, ... ], "categories": [ ... { "id": 2, "name": "cat", "supercategory": "animal" }, ... ], "images": [ { "id": 0, "license": 1, "file_name": "<filename0>.<ext>", "height": 480, "width": 640, "date_captured": null }, ... ], "annotations": [ { "id": 0, "image_id": 0, "category_id": 2, "bbox": [260, 177, 231, 199], "segmentation": [...], "area": 45969, "iscrowd": 0 }, ... ]
So to extract necessary information, I have used the COCO API who assists in loading, parsing, and visualizing annotations in COCO. The API supports multiple annotation formats
APIs | Description |
---|---|
getImgIdsGet | Get img ids that satisfy given filter conditions. |
getCatIdsGet | Get cat ids that satisfy given filter condition |
getAnnIdsGet | Get ann ids that satisfy given filter conditions. |
Firstly, I focus on some common kind of objects need to be used for bench marking the model including person, bicycle and car. In term of these subcategories, currently there are 1101 training images, wheres there are 45 validation images.
For testing dataset, I would like to labeling manually by utilizing customing dataset from user. This use case will be the real case.
You can find some samples in training dataset below. In fact in each image there are plenty of objects annotaions format under the form of bounding box x, y, w, h where (x, y) is coordinate of the top left corner of the box and w, h the width and the height of the box.
YOLO model pipeline
So we can see that YOLO — You Only Look Once — is an extremely fast multi object detection algorithm which uses convolutional neural network (CNN) to detect and identify objects.
I would like to build a pipeline for YOLO detection in Python first