Facial Recognition Software Helping Caterpillar Identify Sleepy Operators

Operator fatigue can potentially be a fatal problem for Caterpillar employees driving the massive mine trucks on long, repetitive shifts throughout the night.
Caterpillar recognized this and joined forces with Seeing Machines to install their fatigue detection software in thousands of mining trucks worldwide. Using NVIDIA TITAN X and GTX 1080 GPUs along with the cuDNN-accelerated Theano, TensorFlow and Caffe deep learning frameworks, the Australian-based tech company trained their software for face tracking, gaze tracking, driver attention region estimation, facial recognition, and fatigue detection.

On-board the truck, a camera, speaker and light system are used to monitor the driver and once a potential “fatigue event” is detected, an alarm sounds in the truck and a video clip of the driver is sent to a 24-hour “sleep fatigue center” at the Caterpillar headquarters.
“This system automatically scans for the characteristics of microsleep in a driver,” Sal Angelone, a fatigue consultant at the company, told The Huffington Post, referencing the brief, involuntary pockets of unconsciousness that are highly dangerous to drivers. “But this is verified by a human working at our headquarters in Peoria.”
In the past year, Caterpillar referenced two instances – in one, a driver had three fatique events within four hours and he was contacted onsite and forced to take a nap. In another, a night shift truck driver who experienced a fatique event realized it was a sign of sleep disorder and asked his management for medical assistance.
It’s a matter of time before this technology is incorporated into every car on the road.