As anyone who has children knows, having a baby does not exactly do wonders for your sleep schedule. In fact, parents lose an average of nearly two hours of sleep each night in the first year of their child’s life. There is not enough coffee in the world to counter this level of sustained sleep loss. And it is not enough to just power through that persistent feeling of tiredness — lack of sleep can have a major effect on both one’s physical and mental health. The additional responsibilities and worries that come along with caring for the new little bundle of joy only serve to further amplify these detrimental effects.
Volumes have been written on the subject of getting babies to sleep through the night, but as the dark circles under the eyes of any parent of a young child will tell you, the problem has not been solved. Unsatisfied by books and blog posts touting “10 simple steps to get your baby to sleep through the night” or “secrets to sleep training,” engineer Manivannan Sivan took matters into his own hands. Sivan noticed that when his baby was on a rocker swing, it helped the baby to calm down and go to sleep. But the swing needed to be turned on manually, so it was not very useful for heading off crying in the night.
Momentarily emerging from a delirium brought on by sleep deprivation, Sivan had the idea to use machine learning to recognize the sound of a baby crying, and to use that as a signal to automatically turn on the rocker swing. Starting the swing immediately would help the baby to calm down before the crying got completely out of hand, and would also save Sivan from having to get up to comfort the child. Hoping to get a good night’s sleep for a change, Sivan turned to Edge Impulse to help build the machine learning algorithm quickly.
To keep things simple, an Arduino Portenta H7 development board was chosen as the hardware platform. With a dual-core Arm Cortex-M7/M4 MCU and 8 MB of SDRAM, the Portenta H7 is ideal for machine learning on the edge. A Portenta Vision Shield was added to the build for the pair of high quality MP34DT05 microphones that it contains. The Arduino also controls a relay that switches the DC motor in the rocker swing on and off based on the output of the machine learning algorithm.
Before the classification algorithm could recognize the sound of a baby crying, it needed to be provided with sample audio data. Sivan downloaded a custom firmware image that linked his Portenta H7 directly to his Edge Impulse project, which allowed him to capture data with the device’s microphone and have it automatically uploaded to Edge Impulse Studio. In this way he collected a small, custom dataset consisting of just under three minutes of audio. The samples fell into two classes — “baby crying” and “normal background noises.”
To process this data, an impulse was created that began with a step to convert the audio data into a spectrogram. This preprocessing step allows the audio data to be analyzed by algorithms that are designed for dealing with image data, and also helps to bring the most relevant features to light. The spectrogram was then passed on to a convolutional neural network that can classify audio into one of the two classes present in the training data. After the training process completed, an average classification accuracy rate of 98.2% was reported. This is an exceptional result considering that a very small amount of training data was supplied.
To confirm that the algorithm would work in the real world as well as the testing indicated it would, Sivan deployed a custom firmware image containing the full machine learning pipeline to the Portenta H7. This was a quick way to give the device a test drive. After he was satisfied that everything was working as expected, Sivan then deployed the model as an Arduino library, which gave him the flexibility to add his own code into the pipeline. In this way, he built in support for turning the rocker swing on and off, depending on the classifications made by the model.
Before you start winding down for bed tonight, you may want to look over the documentation for this project to see if it might be able to help you catch some more Zs. Even if you happen to be in a different stage of life, you can still clone Sivan’s public Edge Impulse project and modify it to solve any other problem that you can dream up — no pun intended.
Want to see Edge Impulse in action? Schedule a demo today.