GSoC/2020/StatusReports/KartikRamesh

Digikam : Face Management Workflow Improvements

DigiKam is a KDE Desktop Application for Photo Management. Apart from the standard functionality of being able to view photos, DigiKam provides the user with a lot of added features such as Image Tagging, Photo Editing, Image Metadata viewing/editing. At the heart of DigiKam's commendable functionality is the FaceEngine. DigiKam can detect faces in Photos, and recognize faces in new photos based on prior information. This allows for a great personalized experience for the user.

A major breakthrough in the FaceEngine came last year when Thanh Trung Dinh implemented OpenCV's DNN module to bring great improvements to performance. Igor Antropov implemented many changes to the workflow Interface, to make the overall experience much comfortable for the user.

This project is in essence an extension to the work that Igor did last summer. As such, this project does not intend to implement one Major feature.Instead, it aims to rectify issues in the current workflow, as well as introduce new features in an effort to improve the user experience.

Mentors : Gilles Caulier, Maik Qualmann, Thanh Trung Dinh

Important Links

Project Proposal

DigiKam Face Engine Workflow Improvements

GitLab development branch

gsoc-20-facesengine-workflow

Project Goals

This project aims to :

• Provide a Help Box to aid first time users of Facial Recognition.
• Provide notification about results of a Facial Recognition.
• Order People Sidebar, to show tags of Priority first.
• Order Face Item View, to display Unconfirmed Faces before Confirmed Faces.
• Provide new “Ignored” Category for Face Tags.
• Automatically Group Results in Unconfirmed Tag.
• Provide Functionality to reject Face Suggestions.
• Automatically add Icons to newly created face tags.

Week 1 : May 11 to May 18

NOTE: Due to the current global situation, there's some uncertainty regarding when my college final term exams will be scheduled. After having discussed this with the mentors, we've decided that starting early with the project is the best course of action.

The first issue I intend to tackle is that of Rejecting Face Suggestions in DigiKam. Face Suggestions are a key part of the Facial Recognition process, and allow the User to categorize their album according to People Identities, while training the Facial Recognition algorithm.

The user interacts with these Facial Rejections by means of the Assign Name Overlay. This overlay appears on hovering over the Face Suggestion and allows the User to confirm (✅) or reject(⛔) the suggestion.

However, in the present version there's really no way to "Reject" a Face Suggestion. Pressing the ⛔ does exactly what pressing ✖ does, it deletes the Face Region from the Database.

Currently the ✖ and ⛔ perform the same function

This is not ideal. The ⛔ button should technically do the opposite of what ✅ does. It should be the user's way of telling the Facial Recognition Algorithm that it's incorrect.

Present Scenario

• Facial Recognition outputs incorrect suggestion.
• User intuitively presses ⛔, hoping the algorithm realizes the mistake.
• Instead the Face is deleted from the Database.
• To recover the Face, the User re-runs Face Detection and Face Recognition.
• Since the Algorithm was not provided any inputs, it repeats the Incorrect Suggestion.

Desired Scenario

• Facial Recognition outputs incorrect suggestion.
• User intuitively presses ⛔, hoping the algorithm realizes the mistake.
• Face gets moved to "Unknown" Tag.
• If user re-runs Face Recognition immediately, in all likelihood the mistake will be repeated, as it will perform again on identical conditions.
• If user provides some input, in the form of assigning Faces to other photos of the same person, then the mistake will be reconciled.

Implementation

The Confirmation/Rejection happens through an interplay between 3 classes.

• AssignNameWidget : The actual widget which controls the ✅, ⛔ buttons, LineEdit etc.
• AssignNameOverlay : The overlay which controls how it's drawn when user hovers over a face etc. It includes the AssignNameWidget as a member variable.
• DigiKamItemView : Responsible for the main view of DigiKam.

The current workflow is as follows:

• The user presses on the ⛔ button.
• AssignNameWidget sends a reject signal to AssignNameOverlay.
• AssignNameOverlay sends a reject signal to DigiKamItemView.
• DigiKamItemView calls appropriate methods of FaceTagsEditor to delete this face.

The issue is that the Rejection Overlay (✖ Button) is also connect to the same DigiKamItemView::removeFaces(), hence the identical functionality.

I started off with connecting the Reject Signal of AssignNameOverlay to a new function DigiKamItemView::rejectFaces(). The implementation for this function came to be after a discussion with the developers on the mailing list.

In essence, I'm treating Rejection of Face Suggestion as a change of Tag from the current Unconfirmed tag, to an Unknown Tag. To do this I've implemented FaceTagsEditor::changeTags().

Results

Here's how Rejection of Face Suggestions now works:

https://imgur.com/a/eaFFDG4

First I proceed to Reject the Face Suggestion. Notice how this leads to an increment of the count of Unknown Faces, and the Face does show up in the Unknown Category. Hence the Face is not being deleted!

Next, I demonstrate the deletion of face using the (✖) button. This does not change the Unknown Counter, as the Face has been deleted.

Related Links

Blog Post