The percentage of retail sales that take place online has steadily risen over the past decade, from about 5% of total retail sales to almost 15% today. In the US, nearly one trillion dollars is now being spent annually at online retailers according to recent figures from the US Census Bureau. As you might expect, this surge in online shopping is creating both opportunities and strains in the shipping industry that is having to continually adapt to meet the increased demand. In the worst case, these pressures could lead to situations where packages are hastily handled in the rush to meet strict deadlines.
This is a problem for retailers and shoppers alike. No one wants to open up their long-awaited package only to find that the contents have been damaged. That makes for unhappy customers, less repeat business for online retailers, and expensive, time-consuming return processes. Moreover, there is rarely a clear way of determining who is at fault so that the affected parties can be fairly compensated — were the contents damaged in the retailer’s warehouse, in transit, or by the end customer? To put an end to the blame game that can ensue, engineer Shebin Jose Jacob decided to create a simple, yet powerful, tool that can keep an eye on precious cargo from the moment it is placed in a box until it reaches the customer’s hands.
Jacob sketched out an idea that would make use of a small device that could be placed in a package during shipment. This device collects data from a nine-axis inertial measurement unit (IMU), which provides information about the movement and acceleration forces placed on the box. The measurements are then analyzed by an algorithm created with the Edge Impulse machine learning platform that can predict if, and how, a package has been mishandled. Any potential cases of mishandling are wirelessly reported to a cloud database.
An Arduino Nano 33 BLE Sense development board was selected for the prototype because of its small form factor in conjunction with the onboard IMU and processing power that is capable of running the necessary machine learning algorithms. The addition of a GSM module allows the device to communicate with a cloud database via cellular networks while the package is still in transit. A rechargeable battery supplies the device with power.
A batch of example data would be needed to teach the model the conditions that it should recognize, so Jacob assembled the device and started collecting IMU measurements. Initially he wanted to be able to recognize five conditions — hard fall, vigorous shaking, throwing, normal carrying, and idle. Through a simple process, the Arduino was connected to his project in Edge Impulse Studio, after which any collected data was automatically uploaded to the project. After shaking, throwing, and generally abusing the package for 28 minutes to collect data, Jacob was ready to move on to the next step — creating an impulse.
The impulse consists of a spectral analysis to extract the most meaningful components from the data, which are then passed into a neural network classifier. A few hyperparameters were tweaked to optimize the training process for this particular task. Jacob noted that he could have also included an anomaly detector in the impulse to detect any type of unusual activity, but for this initial proof of concept, he decided to stick with classification of known event types. With the impulse complete, the training process was initiated.
Upon completion, Jacob found that the training classification accuracy was a very impressive 95.4%. He also wanted to validate that on data that had not been included in the training process, so he used the model testing tool as another check. This revealed a very respectable average classification accuracy rate of 91.3%, which is quite high considering the limited amount of training data that was supplied.
Since this device goes along for the ride on shipments, the algorithm needs to run locally on the hardware. Fortunately, Edge Impulse fully supports the Arduino Nano 33 BLE Sense, which made deployment to the physical hardware as simple as downloading a custom firmware image and flashing it to the board. The EON Compiler was enabled during deployment to optimize the model for resource-constrained devices, without sacrificing model accuracy.
When a package equipped with Jacob’s device detects either a hard fall, vigorous shaking, or throwing, a record is inserted into a cloud database. That information can be incorporated into the normal tracking information, and it can also be used in the event of a problem to determine where damage was most likely to have occurred. In this way, finger pointing can be replaced with hard facts, and issues can be resolved more quickly.
Do you have a similar problem that you are trying to solve? No need to go it alone, Jacob’s project documentation is a great resource that will bring you up to speed in no time.
Want to see Edge Impulse in action? Schedule a demo today.