Earlier this year, we reported on Solomon Githu’s environmental audio monitoring wearable that was designed to assist the hearing impaired. As a refresher, studies have shown that deaf individuals are three times more likely to be hit by a car while crossing the street than their hearing counterparts, in large part because they miss important audible cues. Githu was determined to overcome this problem by creating a wrist-mounted device that could provide a warning, via haptic feedback, when important sounds, like those of emergency vehicle sirens and car horns, were detected.
The prototype system was very successful. A neural network classifier was built and trained using Edge Impulse Studio, and it achieved an average classification accuracy of better than 97%. And by deploying the model to Syntiant’s TinyML Board, Githu had a platform that was well-suited for a wearable. The onboard NDP101 Neural Decision Processor enables ultrafast predictions to be made by neural networks, and the TinyML Board was specially designed with low-power applications in mind, which stretches battery life to the max.
Since the TinyML Board also has a microphone onboard, and a very tiny form factor, it is ideal for being built into a wearable product. But Githu was in full prototype mode when he first built the device, focusing on function over form, which explains the bare circuit board being strapped to the wrist, not to mention the USB cable tether that was supplying power. Though now, he is ready to transition into the next phase of development.
Turning prototypes into finished products is a crucial step in the innovation and development process, as it bridges the gap between conceptualization and practical application. Prototypes serve as preliminary models that allow creators to visualize and test their ideas in a tangible form. However, it’s the transformation of these prototypes into polished, market-ready products that truly unlocks their potential and delivers value to both creators and end users.
Since Githu did not build the device for himself, he wanted to finish the job and create a practical device that could actually provide help to those that need it. The classification pipeline was already in tip-top shape, so it was deployed, as-is, from Edge Impulse Studio as an optimized Syntiant NDP 101 library. A bit of code was added to this template in Arduino IDE to control the vibration motor when sirens or horns are recognized, then the program was flashed to the TinyML Board.
The device would fit very nicely into a wristwatch-sized package, so Githu came up with a plan and started collecting the parts to make it happen. To fit the components, a custom case was 3D printed, as well as a watch wristband that was attached to it to make it wearable. A 500 mAh rechargeable LiPo battery was added to the design to eliminate the USB tether. The TinyML board comes standard with a battery connector, so this was as simple as plugging in a connector.
The wristband was printed with a TPU material to give it the flexibility it needs to comfortably secure the device around the wrist. The main case was printed with a PLA material to give it the rigidity needed to protect the components inside. Shelves were included inside the case so that each component would have its own place, nicely organized and also protected from coming into contact with other components, which might otherwise cause a short circuit. A cover was separately printed to provide an easy way to quickly access the TinyML Board. The casing is on the large side, however, there is a good deal of free space inside of it. It would be possible to refine the design in the future to make it a bit more aesthetically pleasing without having to alter the hardware design.
After finishing the build, Githu put the new design through a series of tests to make sure that it would still function as intended. Fortunately, he found that the casing did not impact the ability of the board to detect environmental sounds, and the vibration motor was still very noticeable to the wearer. And unlike before, the device was now self-contained and ready to be used by a non-technical individual.
An interesting aspect of this work is that the TinyML Board is very versatile. In addition to a microphone, it also has a motion sensor onboard. And with the GPIO pins, virtually any other hardware can be interfaced with it. As such, it would be fairly simple to modify Githu’s work to create an entirely different machine learning-powered wearable device. Got some ideas? Great! Gheck out the first write-up to build your machine learning model with Edge Impulse Studio, then move on to the second write-up to learn how to polish it up for real-world use.
Want to see Edge Impulse in action? Schedule a demo today.