Using the Power of AI to Make Factories Safer

As industrial automation increases, safety becomes a greater challenge and top priority for enterprises.

Safety encompasses multiple aspects:

System safety: A rational pursuit of acceptable mishap risk within a systems perspective.
Safety of the intended functionality: The absence of unreasonable risk due to hazards resulting from functional insufficiencies of the intended functionality or by reasonably foreseeable misuse by persons.
Functional safety: Part of the overall safety that depends on the correct functioning of the safety-related systems and external risk reduction facilities.
Worker safety: Providing a safe working environment for employees by incorporating safe equipment and safe procedures at the workplace.
And other, more general types of safety.

The same technological solution that’s driving automation can be used to also address safety: artificial intelligence.

AI-powered stationary outside-in safety platforms, which monitor activity across many distributed machines or robots, can predictively and proactively orchestrate consistent safety policies. Machines and robots equipped with inside-out reactive safety can detect any specific interaction within their workspace and take appropriate measures.

Diagram shows orchestrating distributed mobile machines with stationary safety platforms. — *Figure 1. Inside-out and outside-in safety*

IEC 61508 is an international standard published by the International Electrotechnical Commission (IEC). It consists of methods to apply, design, deploy, and maintain safety-related systems.

Using IEC 61508 general architecture, AI can either be used to implement the safety function or to assist functional safety to provide additional risk reduction measures.

Workflow diagram shows an EUC control system assisting reactive functional safety with proactive safety functions. — *Figure 2. Reactive and proactive safety*

The use of AI in safety-related systems is being analyzed in upcoming international standardization activities such as ISO/IEC TR 5469 and ISO/IEC AWI TS 22440 being developed by ISO/IEC joint working group JWG 4.

The examples in Table 1 describe some typical safety use cases that can benefit from AI.

Use case	Surveilled machines
Detecting position and movement of humans toward a machine safety zone.	conveyor belt, high-speed door, collaborative robot, autonomous mobile robot, forklift
Detecting machines moving in the direction of a forbidden area or in the direction of other machines; machines moving at excessive speed or excessive strength.	autonomous mobile robot, forklift
Detecting defects or material that could cause safety hazards based on norms such as ANSI/RIA R15.08, and so on.	autonomous mobile robot, train, airplane
Detecting and classifying worker or object poses and movement.	Factory, warehouses

Table 1. Use cases and machines that can benefit from AI proactive safety functions

The following examples of responses to those proactive safety functions are ordered by increasing level of intervention:

Updating statistics to later optimize worker-machine or machine-machine interactions, change the factory floor plan, or improve worker training.
Informing a safety supervisor.
Generating visible and audible alerts to workers, through machine local alerts or personal devices.
Slowing down or stopping a machine (acting as a speed or force limiter).
Changing the position or trajectory of the machine.
Preventing the start of the equipment mission.
Activating hazard protection devices.

NVIDIA IGX Orin combines industrial-grade hardware with enterprise-level software and support to implement both inside-out reactive and outside-in proactive safety in the same platform:

AI safety framework
- Framework to provide proactive safety through AI.
Safety Extension Package (SEP)
- Safety services for fault avoidance, detection, and control leveraging hardware safety extensions
- Edge Safety Link, a safety communication protocol.
Hardware safety extensions:
- Hundreds of diagnostic mechanisms implemented in NVIDIA Orin SoC elements (CPU, GPU, accelerators, interfaces, ECC, and so on), In-System-Test, clock, temperature and voltage monitors, and so on.
- Functional Safety Island (FSI): An independent, redundant set of processors with onboard memory that is used as a monitoring and safety processor working beside other SoC engines.
- Safety MCU: A third-party, safety-certified microcontroller, collaborating with FSI and Orin SoC, and controlling board-level voltage and temperature monitors.
Safety documentation
- Set of documents (safety application note, safety manual, and FMEDA) to support customer safety cases.

Safety in action example

Here’s an example of proactive and reactive safety in action, implemented in collaboration with FORT Robotics and Protex.AI.

Workflow diagram shows warn zone, stop zone, and communication channels to safety protocol elements. — *Figure 3. Proactive and reactive safety in action*

In the example, an HD IP camera connected with Ethernet to NVIDIA IGX running Protex AI stack detects a worker moving toward an industrial mobile machine.

Photo of FORT robotics warn and stop zones labeled. — *Figure 4. Overall setting with camera and safety zones*

If a worker crosses the warning safety zone, IGX commands switch on a visible alarm (proactive safety).
If the worker continues or crosses the stop safety zone, the FORT safety stack on IGX commands (WiFi) the FORT endpoint (embedded in the machine) to activate the emergency stop (reactive safety).

Still of the camera feed in the safety stack. — *Figure 5. Setting safety zones in Protex.AI user interface*

The Protex edge computer vision app uses NVIDIA DeepStream 6.2 and the Protex model has been trained with NVIDIA TAO Toolkit and quantized with NVIDIA TensorRT.