HPD Secures Your Platform While Lowering Power

February 24, 2023

Sponsored Story

Human presence detection (HPD), an application of visual sensing, is the art of knowing when a person is present, determining their status, and their environment. If implemented effectively, it has the potential to reap many benefits, particularly regarding security and lowering power consumption, when compared to traditional methods.

This technology has been with us for some time in various forms. It is now getting to the point where the accuracy has become good enough for use in a laptop, PC, or automotive applications such as driver alertness detection. Other areas that can make use of this technology include the smart-home and industrial safety.

Traditional cameras with computer vision algorithms were often used for this application, but such a setup can be overkill, providing far more information than is necessary and consuming too much power for edge IoT applications. Integrated sensor modules can also be a significant cost reducer as the bandwidth is greatly reduced allowing for the use of a low-cost MCU, just enough to handle some simple AI algorithms. The deployment can then be handled with tinyML code.

The module determines where a person, or thing, is present and where they are relative to the laptop. It should supply enough information for real-time decisions to be made directly at the device level.

HPD In a Module

One available HPD solution for the IoT comes from Synaptics in the form of Emza Visual Sense. Optimized machine learning (ML) computer-vision algorithms run on an ultra-low-power edge AI SoC. A typical application requires less than 1 Mbyte of memory.

The algorithms work on data from a low-power CMOS sensor that keeps power consumption typically in the range of 5 to 15 mW. This can be further reduced based on the actual need. These power levels are approaching the power noise floor in a modern laptop. All three elements—the Emza Visual Sense algorithms, the low-power edge AI processor, and the CMOS sensor—could reside within the same module.

In the case of a laptop, this module would turn on immediately when the laptop is opened and remain on as long as the laptop is running. When the user walks away, the lack of presence is detected, putting the laptop into a locked sleep or suspend mode until the user returns.

Between powering on and powering down, there are more interesting options to provide both security and extended battery life. For example, when someone nearby is looking at the laptop screen, the user is alerted to a security threat (referred to as “on-looker detect”) and the display visibility can be reduced until the threat passes.

To save power, Synaptics’ HPD can dim the display when the user looks away (called “look-away detect”). Because the ML algorithms can be updated with more accurate models of users’ habits. The HPD function continues to improve over time in terms of speed, accuracy, responsiveness, and usefulness.

Rather than a sensor module, a similar configuration can be designed using a time-of-flight (ToF) sensor. But such a device can’t perform facial recognition or determine the difference between a person and a nearby pet.

In practice, the CMOS sensor output is not constantly recorded. Rather, the processor takes two snapshots or frames every second. That information is stored, then compared to other frames.

That’s enough information to sense the environment and determine human presence and movement, as well as eye-tracking, which in-turn could trigger another algorithm to handle the facial recognition. This is a little more compute intensive.

Given the nature of the information being detected, implementations must perform the necessary processing at the edge to ensure that no personally identifiable information (PII) is sent upstream to the cloud.

A Complex Software Design

While the hardware to handle this application was readily available, the algorithm development aspect is non-trivial. Google’s AutoML was a step forward, simplifying the process. AutoML lets developers train their AI models for specific needs in an efficient manner.

(The evaluation kit for the Katana Edge AI SoC can be used to rapidly develop visual sensing models using Edge Impulse’s Embedded ML platform.)

The Synaptics Emza Visual Sense HPD technology is already implemented in laptops from Dell and Panasonic. Solutions are also being built around the company’s Katana AI SoC platform, allowing HPD benefits to be brought to home appliances and consumer electronics. For example, a television could know who is watching and serve up the appropriate content without being asked, or even lock out under-age or other unauthorized users.

When the Synaptics design team first embarked on this project, everything was planned to meet the very stringent ultra-low-power needs of edge AI design. The model builds all the necessary hooks into the silicon and software including building IP for image processing, which avoids the need to spin up the main CPU.

The algorithms then need to shrink, making them as efficient as possible by reducing the layers involved. And because a relatively low resolution is employed by the sensor, the proper training must take place, including acquiring the necessary data sets.

It is important that the technology continues to evolve as new data sets become available.

If you need further evidence of Synaptics HPD platform or want to see HPD and other edge AI applications—such as intelligent digital signage—in action, stop by the company’s booth at the embedded world trade show in Nuremberg, Germany, March 14-16 (Booth 4A-259).