Facial Emotion Recognition with DeepFace¶

The history of face detection is now a few decades and involves significant advancements in computer vision, pattern recognition, machine/deep learning and other.

History of face detection¶

Here’s an approximative timeline of the key developments in the field of face detection since the 60's to our days.

Early Research and Concepts (1960s-1980s)¶

• 1960s: The idea of automated face recognition was introduced, focusing on manually extracting facial features and using statistical analysis to differentiate faces.
• 1970s: Turk and Pentland developed the Eigenfaces method, representing faces as vectors in high-dimensional space and using Principal Component Analysis (PCA) to reduce dimensionality, laying the groundwork for modern face recognition.

Development of Statistical Models (1990s)¶

• 1991: Turk and Pentland’s Eigenfaces method was formally published, utilizing PCA to create a low-dimensional subspace for face representation and detection.
• 1994: Cootes and colleagues introduced the Active Shape Model (ASM) and Active Appearance Model (AAM), allowing for flexible modeling of facial shapes and appearances using statistical methods.
• 1997: Rowley, Baluja, and Kanade developed the first successful neural network-based face detection system, employing a sliding window approach to detect faces at various scales and positions.

The Advent of Haar Cascade Classifiers (2000s)¶

Like for the pop music the 2000's have been a good years in the computer science field 😂

In 2001 Paul Viola and Michael Jones revolutionized face detection with their Haar-like feature-based method, outlined in "Rapid Object Detection using a Boosted Cascade of Simple Features":

• Haar-like Features: Detected structural patterns in images inspired by Haar wavelets.
• Integral Image: Allowed for rapid calculation of Haar-like features, increasing efficiency.
• Adaboost Algorithm: Selected and combined the most important features into a strong classifier.
• Cascade Classifier: Used a series of increasingly complex classifiers to quickly eliminate non-face regions, significantly improving detection speed.

Advancements with Machine Learning and Deep Learning (2010s-Present)¶

The year of 2010s is the year of Machine learning and deep learning whre they significantly enhanced face detection accuracy and speed with :

• Convolutional Neural Networks (CNNs): Became popular for their ability to automatically learn features from data, with CNN-based detectors like R-CNN and SSD achieving state-of-the-art performance.
• DeepFace (2014): Facebook's deep learning model reached near-human accuracy in face recognition and detection.
• FaceNet (2015): Google introduced FaceNet, embedding faces in high-dimensional space and using deep learning for high accuracy in recognition and verification tasks.
• Multi-task Cascaded Convolutional Networks (MTCNN): Combined face detection, alignment, and recognition in one framework, enhancing robustness and efficiency.

Modern Techniques and Applications¶

Now face detection technology has become ubiquitous, with applications in security, surveillance, biometrics, and social media:

• Real-time Detection: Advances in hardware and optimization techniques have enabled real-time face detection on mobile devices and embedded systems.
• Robustness: Modern face detectors handle challenges like occlusions, varying lighting conditions, and different facial orientations effectively.
• Ethical and Privacy Concerns: The widespread use of face detection has raised ethical and privacy issues, prompting discussions and regulations to ensure responsible use of the technology.

The journey of face detection¶

The journey of face detection has been marked by continuous innovation and breakthroughs. From the early days of manual feature extraction to the sophisticated deep learning models of today, each advancement has contributed to making face detection faster, more accurate, and more accessible.

As the field continues to evolve, researchers and practitioners are focused on addressing the challenges and ensuring the technology is used ethically and responsibly.

Environment setup¶

In this article we will be focusing on deepface library. Deepface is a lightweight face recognition and facial attribute analysis (age, gender, emotion and race) framework for python. It is a hybrid face recognition framework wrapping state-of-the-art models.

We love this lib because it is very simple to use and to mix it up with opencv 😇

In [1]:

!pip install deepface

!pip install deepface

Collecting deepface
  Downloading deepface-0.0.91-py3-none-any.whl (97 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.3/97.3 kB 980.1 kB/s eta 0:00:00
Requirement already satisfied: requests>=2.27.1 in /usr/local/lib/python3.10/dist-packages (from deepface) (2.31.0)
Requirement already satisfied: numpy>=1.14.0 in /usr/local/lib/python3.10/dist-packages (from deepface) (1.25.2)
Requirement already satisfied: pandas>=0.23.4 in /usr/local/lib/python3.10/dist-packages (from deepface) (2.0.3)
Requirement already satisfied: gdown>=3.10.1 in /usr/local/lib/python3.10/dist-packages (from deepface) (5.1.0)
Requirement already satisfied: tqdm>=4.30.0 in /usr/local/lib/python3.10/dist-packages (from deepface) (4.66.4)
Requirement already satisfied: Pillow>=5.2.0 in /usr/local/lib/python3.10/dist-packages (from deepface) (9.4.0)
Requirement already satisfied: opencv-python>=4.5.5.64 in /usr/local/lib/python3.10/dist-packages (from deepface) (4.8.0.76)
Requirement already satisfied: tensorflow>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from deepface) (2.15.0)
Requirement already satisfied: keras>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from deepface) (2.15.0)
Requirement already satisfied: Flask>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from deepface) (2.2.5)
Collecting mtcnn>=0.1.0 (from deepface)
  Downloading mtcnn-0.1.1-py3-none-any.whl (2.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 MB 37.2 MB/s eta 0:00:00
Collecting retina-face>=0.0.1 (from deepface)
  Downloading retina_face-0.0.17-py3-none-any.whl (25 kB)
Collecting fire>=0.4.0 (from deepface)
  Downloading fire-0.6.0.tar.gz (88 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 88.4/88.4 kB 8.3 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting gunicorn>=20.1.0 (from deepface)
  Downloading gunicorn-22.0.0-py3-none-any.whl (84 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.4/84.4 kB 9.6 MB/s eta 0:00:00
Requirement already satisfied: six in /usr/local/lib/python3.10/dist-packages (from fire>=0.4.0->deepface) (1.16.0)
Requirement already satisfied: termcolor in /usr/local/lib/python3.10/dist-packages (from fire>=0.4.0->deepface) (2.4.0)
Requirement already satisfied: Werkzeug>=2.2.2 in /usr/local/lib/python3.10/dist-packages (from Flask>=1.1.2->deepface) (3.0.3)
Requirement already satisfied: Jinja2>=3.0 in /usr/local/lib/python3.10/dist-packages (from Flask>=1.1.2->deepface) (3.1.4)
Requirement already satisfied: itsdangerous>=2.0 in /usr/local/lib/python3.10/dist-packages (from Flask>=1.1.2->deepface) (2.2.0)
Requirement already satisfied: click>=8.0 in /usr/local/lib/python3.10/dist-packages (from Flask>=1.1.2->deepface) (8.1.7)
Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.10/dist-packages (from gdown>=3.10.1->deepface) (4.12.3)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from gdown>=3.10.1->deepface) (3.14.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from gunicorn>=20.1.0->deepface) (24.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas>=0.23.4->deepface) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=0.23.4->deepface) (2023.4)
Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=0.23.4->deepface) (2024.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.27.1->deepface) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.27.1->deepface) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.27.1->deepface) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.27.1->deepface) (2024.2.2)
Requirement already satisfied: absl-py>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (1.4.0)
Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (1.6.3)
Requirement already satisfied: flatbuffers>=23.5.26 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (24.3.25)
Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (0.5.4)
Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (0.2.0)
Requirement already satisfied: h5py>=2.9.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (3.9.0)
Requirement already satisfied: libclang>=13.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (18.1.1)
Requirement already satisfied: ml-dtypes~=0.2.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (0.2.0)
Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (3.3.0)
Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (3.20.3)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (67.7.2)
Requirement already satisfied: typing-extensions>=3.6.6 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (4.11.0)
Requirement already satisfied: wrapt<1.15,>=1.11.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (1.14.1)
Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (0.37.0)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (1.64.0)
Requirement already satisfied: tensorboard<2.16,>=2.15 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (2.15.2)
Requirement already satisfied: tensorflow-estimator<2.16,>=2.15.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow>=1.9.0->deepface) (2.15.0)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from astunparse>=1.6.0->tensorflow>=1.9.0->deepface) (0.43.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from Jinja2>=3.0->Flask>=1.1.2->deepface) (2.1.5)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.16,>=2.15->tensorflow>=1.9.0->deepface) (2.27.0)
Requirement already satisfied: google-auth-oauthlib<2,>=0.5 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.16,>=2.15->tensorflow>=1.9.0->deepface) (1.2.0)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.16,>=2.15->tensorflow>=1.9.0->deepface) (3.6)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.16,>=2.15->tensorflow>=1.9.0->deepface) (0.7.2)
Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4->gdown>=3.10.1->deepface) (2.5)
Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /usr/local/lib/python3.10/dist-packages (from requests>=2.27.1->deepface) (1.7.1)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow>=1.9.0->deepface) (5.3.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow>=1.9.0->deepface) (0.4.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow>=1.9.0->deepface) (4.9)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib<2,>=0.5->tensorboard<2.16,>=2.15->tensorflow>=1.9.0->deepface) (1.3.1)
Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow>=1.9.0->deepface) (0.6.0)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<2,>=0.5->tensorboard<2.16,>=2.15->tensorflow>=1.9.0->deepface) (3.2.2)
Building wheels for collected packages: fire
  Building wheel for fire (setup.py) ... done
  Created wheel for fire: filename=fire-0.6.0-py2.py3-none-any.whl size=117029 sha256=094268846aa5ccec2bef8554b24d5bb2d78fec0bf4743abcd7f534690b5cc683
  Stored in directory: /root/.cache/pip/wheels/d6/6d/5d/5b73fa0f46d01a793713f8859201361e9e581ced8c75e5c6a3
Successfully built fire
Installing collected packages: gunicorn, fire, mtcnn, retina-face, deepface
Successfully installed deepface-0.0.91 fire-0.6.0 gunicorn-22.0.0 mtcnn-0.1.1 retina-face-0.0.17

In [2]:

from deepface import DeepFace

from deepface import DeepFace

24-06-02 17:24:21 - Directory /root/.deepface created
24-06-02 17:24:21 - Directory /root/.deepface/weights created

In [4]:

!pip install opencv-python tf_keras

!pip install opencv-python tf_keras

Requirement already satisfied: opencv-python in /usr/local/lib/python3.10/dist-packages (4.8.0.76)
Requirement already satisfied: tf_keras in /usr/local/lib/python3.10/dist-packages (2.15.1)
Requirement already satisfied: numpy>=1.21.2 in /usr/local/lib/python3.10/dist-packages (from opencv-python) (1.25.2)
Requirement already satisfied: tensorflow<2.16,>=2.15 in /usr/local/lib/python3.10/dist-packages (from tf_keras) (2.15.0)
Requirement already satisfied: absl-py>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (1.4.0)
Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (1.6.3)
Requirement already satisfied: flatbuffers>=23.5.26 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (24.3.25)
Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (0.5.4)
Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (0.2.0)
Requirement already satisfied: h5py>=2.9.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (3.9.0)
Requirement already satisfied: libclang>=13.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (18.1.1)
Requirement already satisfied: ml-dtypes~=0.2.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (0.2.0)
Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (3.3.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (24.0)
Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (3.20.3)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (67.7.2)
Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (1.16.0)
Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (2.4.0)
Requirement already satisfied: typing-extensions>=3.6.6 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (4.11.0)
Requirement already satisfied: wrapt<1.15,>=1.11.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (1.14.1)
Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (0.37.0)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (1.64.0)
Requirement already satisfied: tensorboard<2.16,>=2.15 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (2.15.2)
Requirement already satisfied: tensorflow-estimator<2.16,>=2.15.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (2.15.0)
Requirement already satisfied: keras<2.16,>=2.15.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow<2.16,>=2.15->tf_keras) (2.15.0)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from astunparse>=1.6.0->tensorflow<2.16,>=2.15->tf_keras) (0.43.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (2.27.0)
Requirement already satisfied: google-auth-oauthlib<2,>=0.5 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (1.2.0)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (3.6)
Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (2.31.0)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (0.7.2)
Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (3.0.3)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (5.3.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (0.4.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (4.9)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib<2,>=0.5->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (1.3.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (2024.2.2)
Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/local/lib/python3.10/dist-packages (from werkzeug>=1.0.1->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (2.1.5)
Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (0.6.0)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<2,>=0.5->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->tf_keras) (3.2.2)

Build our classifier¶

In this part we will be build our facial emotion classifier leveraging the deepface library 🤓

Our mission¶

Now let's resume our mission in a to do list form in order to be understood easily !

1. Importing Necessary Libraries: No need to talk much about it, opencv and deepfaces are required as you guessed lol.
2. Load a pre-trained Haar cascade classifier for frontal face detection (this classifier is a part of OpenCV’s data package)
3. Reading and Preparing the Image: Converted image to grayscale because the Haar cascade classifier operates more effectively on grayscale images.We also need to converted back the image to RGB format to prepare it for emotion analysis, as DeepFace expects RGB images.
4. Detecting Faces: With the Haar cascade classifier. This method will scans the image with a sliding window at multiple scales to locate faces. It returns a list of rectangles, each representing a detected face.
5. Analyzing Emotions: For each detected face we need to filter the region of interest (ROI) corresponding to the face is extracted from the RGB image. This ROI will then be passed to the DeepFace.analyze method, which will perform emotion analysis and identifies the dominant emotion for that face.
6. Displaying the Annotated Image: Finally, the annotated image, with faces and their corresponding emotions.

In [13]:

import cv2
from deepface import DeepFace
from google.colab.patches import cv2_imshow

#load face cascade classifier
face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')

#load the image
image_path = './angelina.jpeg'  # Change this to your image path
frame = cv2.imread(image_path)

#convert frame to grayscale
gray_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

#convert image to grayscale frame to RGB format
rgb_frame = cv2.cvtColor(gray_frame, cv2.COLOR_GRAY2RGB)

#detect faces in the frame
faces = face_cascade.detectMultiScale(gray_frame, scaleFactor=1.1, minNeighbors=5, minSize=(30, 30))

for (x, y, w, h) in faces:
    #filter to extract the face ROI (Region of Interest)
    face_roi = rgb_frame[y:y + h, x:x + w]

    #perform emotion analysis on the face ROI
    result = DeepFace.analyze(face_roi, actions=['emotion'], enforce_detection=False)

    #determine the dominant emotion
    emotion = result[0]['dominant_emotion']

    #draw rectangle around face and label with predicted emotion
    cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 0, 255), 2)
    cv2.putText(frame, emotion, (x, y - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (0, 0, 255), 2)

#display the resulting frame
cv2_imshow(frame)

#wait for a key press and close the window
cv2.waitKey(0)
cv2.destroyAllWindows()

import cv2 from deepface import DeepFace from google.colab.patches import cv2_imshow #load face cascade classifier face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml') #load the image image_path = './angelina.jpeg' # Change this to your image path frame = cv2.imread(image_path) #convert frame to grayscale gray_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) #convert image to grayscale frame to RGB format rgb_frame = cv2.cvtColor(gray_frame, cv2.COLOR_GRAY2RGB) #detect faces in the frame faces = face_cascade.detectMultiScale(gray_frame, scaleFactor=1.1, minNeighbors=5, minSize=(30, 30)) for (x, y, w, h) in faces: #filter to extract the face ROI (Region of Interest) face_roi = rgb_frame[y:y + h, x:x + w] #perform emotion analysis on the face ROI result = DeepFace.analyze(face_roi, actions=['emotion'], enforce_detection=False) #determine the dominant emotion emotion = result[0]['dominant_emotion'] #draw rectangle around face and label with predicted emotion cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 0, 255), 2) cv2.putText(frame, emotion, (x, y - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (0, 0, 255), 2) #display the resulting frame cv2_imshow(frame) #wait for a key press and close the window cv2.waitKey(0) cv2.destroyAllWindows()

In [15]:

#see the results form from deepface api
result

#see the results form from deepface api result

Out[15]:

[{'emotion': {'angry': 7.247548474721959e-13,
   'disgust': 8.370559677950369e-22,
   'fear': 2.012128956938744e-18,
   'happy': 100.0,
   'sad': 6.646035989339022e-12,
   'surprise': 3.3129949373376206e-14,
   'neutral': 1.6046157735871702e-07},
  'dominant_emotion': 'happy',
  'region': {'x': 0,
   'y': 0,
   'w': 93,
   'h': 93,
   'left_eye': None,
   'right_eye': None},
  'face_confidence': 0}]