How to Build a Custom Webcam Facial Recognition System in Python

Facial Recognition is one of the most well-researched problems in the field of Computer Vision. Long having been a part of futuristic portrayals of tech in movies and TV shows, Facial Recognition Technology has now matured into production-ready software, and is already being used for authentication and surveillance purposes. Silicon Valley Tech giants such as Facebook & Apple have led the way with bringing Facial Recognition into the mainstream, with their Photo Tagging & FaceID features respectively, and now other companies have started following suit. Facial Recognition is increasingly also becoming a part of the security software suite, and can help identify persons of interest in crowded public areas such as shopping malls and airports, where it might be difficult to do so otherwise. As discussed in a previous post, Facial Recognition has plenty of potential use cases to businesses across industry verticals.

Image result for facial recognition
Facial Recognition has use cases across industry verticals, such as security, authentication and automatic photo tagging.

This post follows up from the earlier tutorial on how to use your computer’s webcam for Face Detection, and goes one step further to help you build an actual Facial Recognition classifier that can be trained on images of people and taught to recognize faces on your computer’s live webcam feed. With the simplicity of programming using open-source packages available to today’s data scientists, this solution simply uses the OpenCV and Face-Recognition libraries to build your own custom Facial Recognition tool.

Installation (Mac/Linux ONLY)

This solution makes use of the custom Face-Recognition Python library, open-sourced on GitHub here. Face-Recognition is one of the simplest Facial Recognition APIs available for Python and the command line. Installing this package, however, requires you to have dlib already installed with Python bindings. dlib is a modern C++ toolkit with machine learning algorithms written to create software for real-world tasks, and it possesses robust algorithms for facial recognition tasks.

The installation steps required are:

Navigate to the directory for this project on the Terminal. Create a new folder if necessary:

cd Projects
mkdir Facial-Recognition
cd Facial-Recognition

Create a Python 3.6 Virtual Environment inside this directory, activate it, upgrade ‘pip’ and set it up with libraries such as NumPy, Pandas, Matpotlib, Scikit-Learn & OpenCV:

python3.6 -m venv env
source env/bin/activate
python3.6 -m pip install --upgrade pip
python3.6 -m pip install numpy
python3.6 -m pip install pandas
python3.6 -m pip install matplotlib
python3.6 -m pip install sklearn
python3.6 -m pip install opencv-python

Now, clone the code for dlib from GitHub inside this directory:

git clone https://github.com/davisking/dlib.git

Within this directory, you should now see the dlib folder appear. Navigate to within this directory and create a folder called ‘build’ for setting up the main dlib library:

cd dlib
mkdir build
cd build

The Brew package manager is necessary to build the dlib library. So first, ensure you install brew by copy-pasting this command onto your terminal and following through:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Brew should be installed on your system now. Then, use brew to install cmake:

brew install cmake

Now use cmake to build the dlib library:

cmake ..
cmake --build

Go back up one level in the directory and install the Python extensions:

cd ..
python3.6 setup.py install

At this point, you should be able to open Python3.6 and import dlib successfully.

Now, you can finally install the main face_recognition Python library on your system. Ensure the virtual environment is activated, and simply type:

python3.6 -m pip install face_recognition

The setup is complete and now ready for use to write a Facial Recognition script.

Python Implementation

Within Python, import the NumPy, OpenCV & Face-Recognition libraries:

import numpy as np;
import cv2;
import face_recognition;

Get a reference to the webcam stream by:

video_capture = cv2.VideoCapture(0);

Here, we will be loading two sample pictures to train the machine to recognize those faces. I created a directory called Known-Pics with these images, but that of course, is unnecessary. Simply ensure you provide the right directory for these images for the script to find them. The two images I chose are one of myself and one of Barack Obama:

obama_image = face_recognition.load_image_file('Known-Pics/barack-obama.jpg');
vishnu_image = face_recognition.load_image_file('Known-Pics/vishnu.jpg');

Now to teach the machine to recognize the unique face encodings inside these images:

obama_face_encoding = face_recognition.face_encodings(obama_image)[0];
vishnu_face_encoding = face_recognition.face_encodings(trump_image)[0];

We will now create arrays of known face encodings and their names:

known_face_encodings = [obama_face_encoding, vishnu_face_encoding];
known_face_names = ['Barack Obama', 'Vishnu'];

Now, we shall loop over every image frame captured by the webcam video feed and apply the operations necessary to detect and recognize faces:

while True:

    ret, frame = video_capture.read();

    # Converting the frame from OpenCV's BGR format to the RGB format
    rgb_frame = frame[:, :, ::-1];

    # Finding the face locations and encodings in each frame
    face_locations = face_recognition.face_locations(rgb_frame);
    face_encodings = face_recognition.face_encodings(rgb_frame, face_locations);

    # Now to loop through each face in this frame
    for (top, right, bottom, left), face_encoding in zip(face_locations, face_encodings):

        # Checking if the face is a match for known faces
        matches = face_recognition.compare_faces(known_face_encodings, face_encoding);

        name = 'Unknown';

        # Use the known face with the smallest vector distance to the new face
        face_distances = face_recognition.face_distance(known_face_encodings, face_encoding);
        best_match_index = np.argmin(face_distances);

        if matches[best_match_index]:
            name = known_face_names[best_match_index];

        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (255, 0, 0), 2);

        # Draw a label with the name below the face
        cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (255, 0, 0), cv2.FILLED);
        font = cv2.FONT_HERSHEY_DUPLEX;
        cv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1);

    # Display the image
    cv2.imshow('Video', frame);

    # Hit 'q' on the keyboard to quit
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break;

Finally, we will need to release the handle to the webcam and destroy all OpenCV windows as soon as the ‘q’ button is pressed:

video_capture.release();
cv2.destroyAllWindows();

Although this facial recognition solution is simple and hence has a slow frame-rate, it tends to work quite well in recognizing faces, thanks to the robustness of dlib’s machine learning solution.

And there you have it! Using open-source third party libraries like OpenCV & Face-Recognition greatly simplifies the training process and allows you to build your own custom Facial Recognition software in just around 60 lines of code. Of course, as seen from the slow frame rates, a program like this is nowhere near ready for deployment in a real-world environment, but that is just a matter of increasing the performance and the efficiency of the code. Computer Vision has advanced by leaps and bounds over the last decade, and Facial Recognition Technology is but one example of the huge potential that exists in teaching machines to see and recognize objects around them in the world.

Skcript https://www.artelliq.com/blog/how-to-create-a-custom-webcam-facial-recognition-system-in-python/ https://www.artelliq.com/media/heroes/Screen%20Shot%202019-07-08%20at%201.01.26%20PM.png