In the United States, there is a network of sewer pipes in place that extends more than 1.2 million miles to support the country’s 329 million residents. Maintaining this infrastructure is a herculean feat, with up to 75,000 sewer pipe failures being reported annually. Aside from the inconveniences this causes to people serviced by these pipes, it also leads to billions of gallons of untreated sewage being released into the environment.
Under normal circumstances, these pipes are regularly inspected by experts who manually operate remote control robots to visually check for faults. With over a million miles of pipes to inspect, this is a time-consuming and expensive process. The monotonous nature of the work also makes it error-prone, and faults can be missed, which may lead to larger problems in the future..
Machine learning enthusiast Huy Mai saw an opportunity to automate the pipe inspection process, and designed a low-power, vision-based device called TinySewer to take some of the load off of inspectors. By installing TinySewer on an existing robotics platform, an operator can simply drive the vehicle slowly through a pipe and await a notification from the device that a fault has been detected. At that point, the inspector can manually check the area to confirm that assessment. This reduces the incidence of errors, and also allows less experienced inspectors to perform accurate inspections.
The device consists of an Arduino Portenta H7 board and a Portenta Vision Shield to capture images during inspections. High brightness LEDs are used to illuminate the area for the camera. The casing for the device is 3D printed, and design files are available in the project write-up. To simplify and streamline the process of incorporating a machine learning classification algorithm, the Edge Impulse platform was leveraged.
Mai downloaded an existing dataset of sewer pipe images, which included labels for a number of types of common faults. This data was loaded into Edge Impulse, where it was preprocessed for optimal inference performance, as well as to match with the type of images produced by the Portenta Vision Shield. The next step was to use the “Transfer Learning” tab to retrain a MobileNetV2 neural network with the sewer fault data. After training for fifty epochs in Edge Impulse, an overall model accuracy of 94 percent was achieved.
Deploying the model to the Portenta was a breeze. Edge Impulse packages up everything that is needed to run the classifications into a zip file, and the zip file contents can be copy-and-pasted into the Portenta’s internal storage. The model is then loaded by a MicroPython script developed by Mai that streams video and inference results from TinySewer to a client application used by the robot operator. Through a GUI application the inspector can control the illumination level, watch a live stream of the robot’s view, and be notified of any faults detected by the machine learning model. The video stream is also saved as an MP4 file that can be used for later analysis, which would allow for fully autonomous operation of the robot, with the option for an operator to confirm fault predictions at a later time.
TinySewer is a fantastic example of the ability of tinyML to automate repetitive tasks, and simultaneously to improve the results. That is a very nice combination, especially when considering that TinySewer only costs about $150 USD in parts, which is far less than the inferior, existing solutions for pipe inspection.
Want to see Edge Impulse in action? Schedule a demo today.