It is frequently said that we can learn a lot about a person by the way that they walk. While this statement is generally referring to personality traits and should probably be taken with a grain of salt, in certain other respects it is demonstrably true. In medicine, for example, assessing an individual’s walking gait is an invaluable diagnostic tool. Abnormalities or changes in walking patterns can be one of the first visible signs of a number of neurodegenerative diseases, like Parkinson's disease or multiple sclerosis.
Many of these conditions are not accompanied by any known biomarkers that can definitively diagnose them, so gait may also be the only diagnostic tool available to medical professionals in the early stages of disease progression. Since early diagnosis is a crucial factor in initiating an appropriate treatment plan and improving patient outcomes, walking gait is regularly assessed during routine check-ups. Unfortunately, since these assessments happen infrequently, and under unnatural, clinical conditions, many abnormalities are able to slip through the cracks for far too long.
In addition to being infrequent, these types of assessments also involve a great deal of subjectivity. What one physician considers to be concerning might not cause another physician to bat an eyelash. In order to consistently detect unusual gait patterns, and do so as early as possible, better diagnostic tools are needed.
Engineer and machine learning enthusiast Samuel Alexander recently had an idea that could be just what the doctor ordered. By using Edge Impulse in conjunction with edge ML-capable hardware, Alexander reasoned that it should be possible to watch for objective signs of walking gait abnormalities around the clock, under everyday conditions.
Alexander’s plan was to use an inertial measurement unit (IMU) attached to a shoe to collect motion data as the wearer walks. With the help of some machine learning algorithms trained and deployed by Edge Impulse, this data can be used to look for abnormalities in walking gait. This continuous, objective assessment could alert a user if anything unusual is noticed, allowing them to seek out medical advice.
In order to build a shoe-based device, the hardware platform must be small enough as to be unobtrusive, and it must also be energy efficient such that it can run for long periods of time on battery power. This platform must also be powerful enough to run machine learning models. That is a tall order, but Alexander realized that the Nordic Thingy:53 IoT prototyping platform would be a great fit. Built around Nordic Semiconductor’s flagship dual-core wireless nRF5340 SoC, this device has the power and memory to run the latest edge ML models. It is also known for slowly sipping energy, and it has a Bluetooth Low Energy transceiver for communicating with other devices. And with an onboard IMU, it contains everything that is needed for the project in a single, tiny package.
Alexander wanted to be able to detect anomalous gait patterns, and also to be able to associate them with at particular activity (E.g., walking, running, standing), so two machine learning models would be needed — a K-means anomaly detector and a neural network classifier. Both of these models would need the same sort of data to learn from: measurements from an IMU attached to a shoe as a person walked, ran, and stood still. Alexander utilized the nRF Connect app to collect this data and transfer it to an Edge Impulse project. A relatively small dataset of 13 minutes was collected, which was split into training and testing portions.
Next, Alexander designed an impulse to manage the flow of data from the time that it is collected until predictions are made by the machine learning algorithms. In this case, after IMU data is captured, it is split into short segments. A spectral analysis is then performed on these segments. This transforms raw accelerometer data from the time domain into the frequency domain, which reveals hidden patterns in gait data, such as stride frequency, step regularity, and harmonic components of movement patterns. The extracted features are then forwarded into both a neural network classifier (to determine if the individual is walking, running, or standing), and a K-means anomaly detector (to flag anything that does not look like the normal gait pattern seen in the training data).
The training process was initiated so that the models could learn from the information hidden in the training dataset. The classifier reached an accuracy of 100% initially, and that excellent result was confirmed by the model testing tool, which leverages data samples that were not included in the training process.
The Nordic Thingy:53 is fully supported by Edge Impulse, so the simplest deployment option is to choose a pre-built binary firmware download, which was the path that Alexander chose. After downloading the firmware, which contained the full machine learning pipeline, it was flashed to the hardware using the platform’s standard tools. For more advanced use cases, other deployment options are available that enable developers to customize the pipeline and add in their own logic.
After programming the device and installing it on a shoe, with the help of a 3D-printed case, Alexander confirmed that the anomaly detector was able to detect different gaits, as evidenced by the results shown in the nRF Edge Impulse application successfully matching his running, walking, standing, and abnormal strides. In this situation, the abnormal walking patterns were intentional, but, in theory, the detector should work just as well when they are authentic. Of course this would need to be validated clinically before the system could be used as a diagnostic tool in the real world.
Alexander’s proof of concept device shows just how simple and inexpensive it can be to build a diagnostic tool that fills a presently unmet medical need. This power is now in the hands of individual developers, and could prove to be transformative in the months and years to come. Has this work sparked some creativity within you? If so, a great way to get started is by cloning Alexander’s public Edge Impulse project, then modifying it to meet your own project’s needs. And make sure you read the project write-up as well for some extra details to help you on your way.