The peace of mind a home security camera gives its owner is worth more than the cost of the equipment, installation, and monitoring fees combined. Taking a quick peek around the house while they are away lets homeowners know that everything is safe and secure whether they are around the corner, or around the world. However, these Internet-connected cameras do come with some hidden privacy risks that owners must be aware of. Furthermore, privacy aside, there is room for these cameras to do more to keep us and our homes safe.
Like all IoT devices, security cameras are at risk of having vulnerabilities that can be exploited by bad actors. The cloud computing environments, which commonly store private images and video footage from inside people’s homes, are also a valuable target for malicious hackers. No matter what security measures device manufacturers put in place, you know that you are only one email away from finding out all of your private data has been stolen.
Beyond privacy issues, traditional security cameras are also lacking in certain other respects. When an intruder is detected, they generally do nothing more than send you an alert. That is undoubtedly very valuable information, but wouldn’t it be even better if they could take action to make that intruder think twice about setting foot in your home before they make off with your possessions?
Smart security with a bite
Manivannan Sivan thinks so. But unfortunately, he was unable to find any commercial devices on the market that met all of his requirements. When faced with the decision to whine about it or fix the problem, I am happy to report that Sivan worked out a solution. With the help of Edge Impulse, he came up with a security camera design that preserves privacy by working entirely offline. What’s more, it has a few tricks up its aperture that will send any uninvited guests running away as fast as their feet will carry them.
For his project, Sivan took inspiration from the movie Home Alone, in which the young protagonist defends his house from burglars by rigging it with an array of hilarious yet effective booby traps. No, there are no flamethrowers in Sivan’s build. But when the offline person detection algorithm recognizes an intruder, it will play the sound of a barking dog over a speaker. Then moments later, it will turn lights on inside the home, giving the appearance of the homeowner being present.
The heart of this custom security system is an Arduino Portenta H7 development board. With a Portenta Vision Shield attached, the hardware can capture images and has enough computational horsepower to run a highly optimized computer vision algorithm developed with Edge Impulse. The Portenta H7 is wired to a Raspberry Pi Pico microcontroller, which is given the job of playing an audio clip of a dog barking through a speaker, and of controlling a relay that turns lamps on and off on command.
The key to this device’s operation is an object detection model developed with Edge Impulse. This machine learning algorithm is trained to recognize the distinctive features of humans so that it can trigger its deterrents when it finds someone where they should not be. This training requires a dataset from which to learn, so as a first step, Sivan collected images of people and uploaded them to a new project in Edge Impulse Studio.
To make data collection (and everything else) as simple as possible, Sivan installed custom Edge Impulse firmware on the Portenta H7. This allows the hardware to be directly linked to an Edge Impulse project. In this way, data collected by the hardware’s sensors — such as the camera — can flow right into the project like magic.
Training the digital watchdog
Object detection models also need to know exactly what it is that they are looking for in each image. This is specified by drawing bounding boxes, each of which is assigned a label, around every object of interest — in this case, people. That can be a very tedious and time-consuming process when annotating a large dataset, but the Labeling Queue tool offers an AI-powered assist that does most of the work for you.
With a dataset to work with, Sivan turned his attention to building the impulse. This is the software pipeline that specifies how images will be processed, from the time they are captured until the object detection model makes its prediction. For preprocessing, the resolution of the images was reduced, which is an important step to take on resource-constrained hardware like a microcontroller. That reduces the computational complexity of the next step, which is Edge Impulse’s own FOMO object detection model, which is ideal for use when memory and processing power are in short supply.
This impulse was trained using the dataset that was previously collected. When complete, an accuracy score of better than 92% was recorded. This result was verified by another tool that tests the model using data that was not included in the training process. Everything looked good, so Sivan deployed the impulse to the hardware.
Have it your way
There was still a lot of customization to do, however. Based on the model predictions, Sivan would need to trigger audio playback and control a relay. For this reason, the impulse was deployed as an Arduino library. This is one of the most flexible options, because it enables the developer to insert any arbitrary code into the project that is needed to accomplish their goals. For this project, that involved sending signals to the Portenta H7’s GPIO pins, which were then received by the Pico. The Pico initiated the deterrents when specific signals were given.
If you were to create your own AI-powered security camera, what additional capabilities would you build into it? Whatever you have in mind, the best starting point is reading through Sivan’s project write-up for some tips. You can even “borrow” his public Edge Impulse project to get a running start.