GSoC/2017/StatusReports/YingjieLiu

digiKam: Face Management Improvements

Face recognition in digiKam is now implemented using Local Binary Patterns Histograms(LBPH) from OpenCV. However, since the performance is not as expected, the algorithm needs to be improved. Another two face recognition algorithms named Eigenfaces and Fisherfaces are partially implemented in the face engine and the two algorithms will be finalized. One sufficient reason that recognition cannot work is that we don’t have any code for pose estimation and normalization. To improve face recognition, new face algorithm with pose estimation and normalization will be added. The algorithms will be selectable in GUI by users. Besides the algorithms, the face region in database should be synchronized when the image is transformed.

Proposal

Complete Proposal

Work Report

1. Community Bonding: Eigenfaces Algorithm Finished For DigiKam

In my work I have added Eigenfaces module in digiKam for face recognition enhencement. My work including 4 parts: (1) UI modification. (2) Algorithm selection. (3) Database modification. (4) Eigenfaces algorithm. The details and screenshots can be seen in my blog post.

2. May 30-June 30 (1): Fisherfaces Algorithm Finished For DigiKam

I added Fisherfaces algorithm as an independent module in digiKam for face recognition enhencement. My work including 3 parts: (1) UI modification. (2) Algorithm selection. (3) Fisherfaces algorithm. The details and screenshots can be seen in my work blog post.

3. May 30-June 30 (2): UI Modification and Face Recognition Accuracy Improved to 99%

I changed the UI and corresponding code for user to choose among the face recognition algorithms. I tested the deep learning method for face recognition in dlib. There has been a big improve for face recognition using deep learning method from dlib. The Accuracy can reach 99.2% with only one training image for each class, and even 100% with more trining images. My work including 2 parts: (1) UI modification for users to choose algorithms. (2) Face recognition experiments with deep learning method from dlib. The details and screenshots can be seen in my work blog post.

4. May 30-June 30 (3): Deep Learning Model Investigation

After testing deep learning module in dlib last week, I am starting to investigate dnn module in opencv contrib. The sample code caffe_googlenet.cpp can load a pretrained caffe model and finish image classification task in opencv just like using caffe. I think it is better to use dnn module rather than rewrite dlib code for face recognition.

I investigated a lot of pre-trained caffe models for face recognition, such as vgg, blvc model zoo, LightenedCNN_A.

Feed back from my mentor Gilles Caulier says that not all Linux Distro have switched from OpenCV2 to OpenCV3. There is no dnn module in OpenCV2. So I have to give up my plan for using dnn module. And I should rewrite dlib face recognition part, port the module into DigiKam. And now I am working on it.

The details can be seen in my work blog post.

5. July 1-July 31 (1): Progress of Importing Code From Dlib

I have been keeping on importing deep learning code from dlib, the details can be seen here. There left two problems in last report: reading model file and how to input data.The two part has been solved, and I am writing the "forward propagation" part of the neural network.

The details can be seen in my work blog post.

6. July 1-July 31 (2): Neural Network is Ok for Face Recognition

These days I keep on importing code and fix the bugs from dlib. Now, the neural network can calculate the 128d vector for each face image, and the ouput vector is the same as in dlib. The code can be seen in the test project.

The details can be seen in my work blog post.

6. July 1-July 31 (3): Face Recognition Accuracy in DigiKam Has Reached 99%

The dnn module has been added into digiKam. I have finished the code about algorithm selection, database modification and recognition classes. I met a lot of compilation errors which is caused by the incompatible between the new code and digikam. But the errors are solved with my mentors' help.

The details can be seen in my work blog post.

Blog Post

2017/06/01 Community Bonding Eigenfaces Algorithm Finished For DigiKam

2017/06/13 May 30-June 30 (1): Fisherfaces Algorithm Finished For DigiKam

2017/06/18 May 30-June 30 (2): UI Modification and Face Recognition Accuracy Improved to 99%

2017/06/25 May 30-June 30 (3): Deep Learning Model Investigation

2017/07/03 July 1-July 31 (1): Progress of Importing Code From Dlib

2017/07/09 July 1-July 31 (2): Neural Network is Ok for Face Recognition

2017/07/26 July 1-July 31 (3): Face Recognition Accuracy in DigiKam Has Reached 99%

Final Work Report