To help with animal conservation efforts, University of Oxford researchers developed a deep learning-based model that can identify individual chimpanzees with 93% accuracy and correctly classify their sex with 96% accuracy.
“Automating the process of individual identification could represent a step change in our use of large image databases from the wild to open up vast amounts of data available for ethologists to analyze behavior for research and conservation in the wildlife sciences,” the researchers stated in their paper.
The model, trained on NVIDIA TITAN X GPUs, with the cuDNN-accelerated MATLAB deep learning framework, relies on a deep convolutional neural network to correctly classify the chimps.
For the training dataset, the researchers took approximately 50 hours of chimpanzee footage from 14 years to train a deep convolutional neural network. This allowed the team to extract over 10 million face images to use during training.
“Using the identified faces, we generated co-occurrence matrices to trace changes in the social network structure of an aging population,” the researchers explained.
Fully unified pipeline for wild chimpanzee face tracking and recognition from raw video footage. The pipeline consists of the following stages: (A) Frames are extracted from raw video. (B) Detection of faces is performed using a deep CNN single-shot detector (SSD) model. (C) Face tracking, which is implemented using a Kanade-Lucas-Tomasi (KLT) tracker (25) to group detections into face tracks. (D) Facial identity and sex recognition, which are achieved through the training of deep CNN models. (E) The system only requires the raw video as input and produces labeled face tracks and metadata as temporal and spatial information. (F) This output from the pipeline can then be used to support, for example, social network analysis. (Photo credit: Kyoto University, Primate Research Institute)
When comparing the model to human observers, it took experts an estimated 55 minutes and novices 130 minutes to correctly complete the experiment. The model achieved 84 percent accuracy in just 60 milliseconds, outperforming the human observers in speed and accuracy.
The TITAN X GPUs were also used for inference.
Face recognition results demonstrating the CNN model’s robustness to variations in pose, lighting, scale, and age over time. (A) Example of a correctly labeled face track. The first two faces (nonfrontal) were initially labeled incorrectly by the model but were corrected automatically by recognition of the other faces in the track, demonstrating the benefit of our face track aggregation approach. (B) Examples of chimpanzee face detections and recognition results in frames extracted from raw video. Note how the system has achieved invariance to scale and is able to perform identification despite extreme poses and occlusions from vegetation and other individuals. (C) Examples of correctly identified faces for two individuals. The individuals age 12 years from left to right (top row: from 41 to 53 years; bottom row: from 2 to 14 years). Note how the model can recognize extreme profiles, as well as faces with motion blur and lighting variations. (Photo credit: Kyoto University, Primate Research Institute)
“The tools we developed enable easy processing and annotation of video datasets, including those from other species. Such automated analysis unveils the future potential of largescale longitudinal video archives to address fundamental questions in behavior and conservation,” the researchers said.
