A keyboard and mouse, a touchscreen, voice commands — these are just a few of the most popular ways to interact with machines today. However, we should stop and ask ourselves every so often if these are the best, most efficient interfaces to support our everyday workflows. After all, just because they are the most popular options does not mean that they are the best.
Many of these tools are legacy solutions designed for the hardware of the past. As we move toward a future of augmented reality and ambient computing, the “friction” of physical clicking, typing, and tapping may soon give way to more fluid forms of interaction that feel less like operating a tool and more like an extension of our own thoughts.
After reflecting on recent technological advances in embedded computing systems and edge machine learning, Solomon Githu came to the conclusion that the next frontier of human-computer interaction may involve AI-powered wearables that enable simple, context-dependent gestures to control the world around us. By leveraging TinyML to process sensor data directly on the device, Githu envisions a reality where a flick of the wrist or a tap of a finger can navigate complex interfaces or manage smart environments.
An Edge-AI Wearable Takes Shape
To demonstrate what’s possible, Githu developed a proof-of-concept wearable gesture control system. It consists of a wrist-worn device that measures hand motion and runs a machine learning algorithm to detect when certain gestures have been made, as well as a wireless receiving unit that turns those gestures into actions, such as switching a lamp on. To keep latency low and protect the user’s privacy, the system runs an Edge Impulse model on-device.
The wearable was built around Seeed Studio’s XIAO nRF52840 Sense, a tiny yet capable microcontroller board. Despite its small footprint, the board packs in Bluetooth Low Energy support and ample flash and RAM for embedded machine learning workloads. Crucially for this project, it also includes a built-in six-degree-of-freedom inertial measurement unit (IMU). This IMU provides precise data that can capture subtle changes in hand motion while consuming very little power.
To make the hardware wearable, Githu designed a simple 3D-printed enclosure sized to house the XIAO board and a small LiPo battery. The case includes a USB-C cutout, allowing the device to be connected to a computer for data collection and programming without removing it from the enclosure. Flexible TPU wrist straps were also custom-designed and printed, ensuring that the wearable could be securely attached to the user’s wrist. This attention to physical design is important, as gesture recognition systems are highly sensitive to sensor placement and orientation.
On the receiving end of the system is an Arduino Nano 33 BLE Sense that acts as a Bluetooth peripheral. While it does not run any machine learning itself, it listens for commands sent from the wearable and uses them to control a GPIO pin. In Githu’s demonstration setup, that GPIO pin drives a relay module, which in turn switches mains-powered devices such as lights on and off. This separation of concerns — intelligence on the wearable, actuation on a nearby controller — keeps the overall system flexible and easy to expand.
Building a Dataset That Reflects Reality
Before any machine learning could take place, the system needed data. Gesture recognition models are only as good as the datasets used to train them, and collecting realistic motion data is often one of the most challenging aspects of TinyML projects. To address this, the XIAO board was worn on the wrist during data collection, mirroring how it would be used in practice. An Arduino sketch streamed raw accelerometer readings over USB serial, providing continuous three-axis motion data as the wearer performed various hand movements.
This serial data was forwarded to Edge Impulse Studio using the Edge Impulse CLI and its data forwarder tool. Acting as a bridge between the microcontroller and the cloud-based development environment, the data forwarder automatically uploaded labeled samples to the project. Four gesture classes were defined: circular motion, left-right motion, random motion, and idle. By including both deliberate gestures and non-gesture movement, the dataset taught the model not only what to recognize, but also what to ignore.
In total, approximately six minutes of motion data were collected, evenly distributed across the four classes. Each recording was then segmented into one-second windows, a length chosen to balance responsiveness with sufficient temporal context for classification. Finally, the dataset was split into training and testing sets using the standard 80/20 rule, ensuring that the model would be evaluated on data it had never seen before.
Finding Patterns in Motion
With the dataset prepared, the focus shifted to model creation and training. In Edge Impulse, this process begins by defining an impulse, which specifies how raw sensor data is processed and which machine learning algorithm is used. For this project, Githu selected a one-second window size with no overlap, feeding the accelerometer data into a spectral analysis processing block. Spectral features are particularly effective for repetitive motions, as they capture frequency-domain characteristics that are often more discriminative than raw time-series values.
The extracted features were then passed to a classification block implemented as a simple feedforward neural network. This network consists of an input layer corresponding to the generated features, two dense hidden layers, and an output layer producing probability scores for each gesture class. Training was performed over 100 epochs with a modest learning rate, using a CPU-based training backend. Thanks to the relatively small dataset and efficient feature extraction, training completed in under ten minutes.
The results were impressive for a first iteration. The trained model achieved approximately 96 percent accuracy on the validation set, with a low loss value indicating good generalization. More importantly, when tested against the reserved 20 percent of unseen data, the model maintained similar performance. This consistency suggested that the classifier had learned meaningful patterns rather than simply memorizing the training samples.
Deploying Intelligence at the Edge
Once the model’s performance was deemed acceptable, it was time to deploy it to the wearable. Edge Impulse simplifies this step by packaging the entire impulse — including signal processing and inference code — into an Arduino-compatible library. Given the memory and processing constraints of microcontrollers, optimization is critical at this stage. By enabling the EON Compiler and quantizing the model to int8 precision, Githu was able to significantly reduce RAM and flash usage without sacrificing accuracy.
After downloading and installing the generated library, the inference sketch was flashed onto the XIAO nRF52840 Sense. In operation, the device continuously samples one second of IMU data, runs it through the optimized model, and determines the most likely gesture. When a circular or left-right motion is detected with sufficient confidence, the wearable transmits a simple character command over Bluetooth Low Energy.
From Motion to Action
The final piece of the puzzle is the IoT control board. Programmed as a BLE peripheral, the Arduino Nano 33 BLE Sense listens for incoming data from the wearable. When it receives a command, it toggles the state of the relay connected to its GPIO pin. In Githu’s demonstration, this relay controlled a string of Christmas lights, turning them on or off in response to hand gestures.
While the current system recognizes only a small set of gestures, it clearly illustrates the potential of edge AI-enabled wearables as intuitive human–machine interfaces. What use cases would you apply gesture recognition to?
Now that you know how straightforward the process can be, why not get started today? If you need a few more pointers, Githu’s project write-up is a great place to start.