Over the past decade, advancements in machine learning have led to the development of techniques that are phenomenally successful at classifying images. This is great news for those that want to make the world a little bit cleaner, keep drivers safe behind the wheel, or improve health and physical fitness. But what can you do if you are not exactly sure what you are looking for, but instead want to look for anything that is out of the ordinary?
This is the position that developer Rob Lauer found himself in when he decided that he wanted to monitor a system of pipes in his basement that feed into a boiler system to heat his house. These pipes get very hot, and any problems with the system could lead to a dangerous situation, so Lauer wants a way to catch potential problems before they have a chance to become serious. Since it is not clear what any possible problem looks like a priori, he developed a method that recognizes what normal heat signatures look like for the piping, then triggers alerts when anomalous conditions are observed.
The device prototype consists of a Raspberry Pi Zero 2 W single-board computer to handle data processing and interactions with peripherals, and a Pimoroni MLX90640 thermal camera breakout board to capture environmental temperature measurements. A Blues Wireless Notecard was added to allow data collected by the device to be wirelessly transmitted to the cloud via cellular networks. A Ubidots dashboard aggregates the data to give a nice, clean view of what is going on with the monitored system via a cloud-based interface.
To detect anomalies, Lauer built a tinyML image classification model with Edge Impulse. A dataset of thermal images was built up, consisting of normal states of the heating system — off, heating up, and actively heating the house. To simulate anomalies, images were manually manipulated in a number of ways to show various abnormal heat signatures, and these manipulated images formed the final class that represents unknown system states. Approximately 120 images were captured with the Raspberry Pi, then uploaded to Edge Impulse along with the modified images, where labels were applied to examples of each class. Next, Lauer designed an impulse to handle data preprocessing and the design of a neural network, which he said was the easiest part of the project because of the simple interface he was working with.
After training the model, he found it to perform remarkably well on the test dataset that was set aside from training data. Then after a look at the Feature Explorer to better understand the few cases that did not classify as expected, it was time to deploy this model to the Raspberry Pi. After running a few commands to connect the Raspberry Pi to Edge Impulse, the trained model file was downloaded locally. A handful of Python libraries were installed to allow for programmatic interactions with the thermal camera and Blues Wireless Notecard, then a simple script was developed to handle the workflow of capturing a thermal image, classifying it, and sending that data to the cloud.
The cloud-based dashboard is a great way to check the historical status of the boiler system, but what if there might be a real problem? You would want to be notified of that right away, not the next time you happen to check the dashboard. So for such cases, Lauer used Twilio to send SMS messages to his phone so that he would be immediately notified of any strange situations.
This device looks like it could be very useful in the home, but Lauer also notes that the same basic technique could be used in industrial settings where the stakes are much higher. If you are interested in making your own home a bit smarter and safer, check out the project write-up for all the details.
Want to see Edge Impulse in action? Schedule a demo today.