Short video clips of people falling down on television shows like America’s Funniest Home Videos have kept us laughing for over 30 years, but for the elderly and workers in certain fields, such as construction and manufacturing, falls are no laughing matter. A fall can result in a wide range of injuries, including fractures, head injuries, or even death.
According to the Centers for Disease Control and Prevention, one out of every four adults age 65 or older falls each year, and falls are the leading cause of both fatal and non-fatal injuries for this age group. And it’s not just older people that are at risk — the Occupational Safety and Health Administration reports that falls are one of the leading causes of workplace injuries and deaths, with over 800 deaths and 300,000 injuries caused by falls in the workplace each year.
Falls also have a significant economic impact. In 2015, they resulted in an estimated $50 billion in medical costs in the United States alone. This includes the cost of hospitalizations, rehabilitation, and ongoing care for those who are injured. Moreover, employers find that productivity suffers due to the missed work days that result from an employee being injured.
Certainly the ideal situation is one in which falls are prevented from ever happening in the first place, but despite our best efforts to take proactive measures like eliminating tripping hazards or improving lighting, accidents still happen. Since preventing all falls is not a realistic goal, we need to look at ways that we can improve our response when an incident does occur.
One important area to consider is fall detection. When the unavoidable does happen, an accurate fall detector can make an immediate call for help to prevent a bad situation from becoming an even worse situation. A fall detector may sound like it would be a complicated or expensive piece of equipment, but as engineer Thomas Vikstrom recently demonstrated, that need not be the case.
Vikstrom’s idea was to leverage the Arduino’s collaboration with K-Way. This choice makes using the fall detector as easy as putting on a jacket. It also provides the necessary computational resources in the form of a tiny Arduino Nicla Sense ME that is integrated with the jacket. Designed with intelligent sensing solutions in mind, the Nicla Sense ME also comes standard with lots of high-quality sensors, including an accelerometer, pressure sensor, gas sensor, and more.
To keep with the theme of simplicity, the intelligence for the device was provided by a machine learning analysis pipeline created with Edge Impulse Studio. Using data from the accelerometer, the plan was to develop an anomaly detector. After training this algorithm with data collected while carrying out normal daily activities, it should be capable of detecting anything outside of the ordinary, such as a fall.
To complete the design, the hackable, nRF52840-based Bangle.js 2 smartwatch was used to simulate making calls to an emergency service. By communicating with the Arduino via Bluetooth Low Energy, the watch can be notified when the wearer of the device has fallen. It can then jump into action to immediately send for help, even if the fall victim is unconscious. Vikstrom notes that it would also be possible to do away with the smartwatch and have the Arduino connect to a smartphone to provide the same functionality.
The Arduino x K-Way smart jacket provided a complete hardware solution for the fall detector, so there was little to do to get the physical device set up, aside from slipping on the jacket. With that out of the way, Vikstrom turned his attention to data collection. Before the model can recognize normal behavior — and warn when anomalous activity is seen — it needs to be shown samples of normal behavior. Vikstrom put on his jacket and did the sort of things that he normally does, like sitting, walking, running, driving, and cycling.
After a small dataset was collected to prove the concept, it was uploaded to Edge Impulse Studio using the data acquisition tool. Next, an impulse was created to process the data all the way from raw sensor measurements to predictions from a machine learning model. This process was initiated by slicing the accelerometer data into 500 millisecond windows, then performing a spectral analysis to extract frequency and power characteristics from the signal. This is an excellent way to prepare sensor data containing many repetitive patterns, as is often the case with accelerometer measurements.
The spectral features were then fed into a K-means anomaly detector. As previously mentioned, this algorithm learns to recognize what is considered to be normal, using the training data as a baseline. It can then predict when something out of the ordinary is happening. If you are not familiar with anomaly detection algorithms, there is no need to be concerned. Vikstrom was not either, so he simply used the reasonable default settings and started the model training process.
After training had finished, the Anomaly Explorer tool was used to assess how well the model was performing. It was found that each of the distinct activities in the training data clustered together quite nicely, which indicated that the model had learned to recognize the relationship between them and the raw sensor data. That should mean that it will also be very capable at detecting unusual activity.
Since everything checked out, it was time to move on to the next phase of the project — deployment of the impulse to the physical hardware. Using the deployment tools in Edge Impulse Studio, Vikstrom prepared and downloaded an Arduino library containing the full anomaly detection pipeline. Using Arduino IDE, the downloaded code was flashed to the Nicla Sense ME. Some testing was conducted to determine what value was an appropriate threshold to distinguish between normal activities and falls.
To finish up the project, Vikstrom developed an app for his smartwatch that could receive predictions from the model running on the Nicla Sense ME via Bluetooth Low Energy. When the anomaly scores exceeded the previously determined threshold, the watch would simulate making a call to emergency services for help. As an additional safeguard against false alerts, the watch counts for 10 seconds before making the call, and the wearer can cancel the process by tapping the screen.
Moving forward, Vikstrom is planning to collect a larger dataset to ensure that the anomaly detector is well generalized and can be used in any situation. He is also planning to create a smartphone app to eliminate the need for a smartwatch, since relatively few people wear a smartwatch daily. If you had created this device, what would your next steps be? Hey, that sounds pretty good! Why not check out the project documentation for all the details that you need to build it?
Want to see Edge Impulse in action? Schedule a demo today.