Pharmaceutical manufacturing is a complex process that involves the production of medications for treating various medical conditions. The production of these drugs is a highly regulated industry, with strict guidelines in place to ensure that the end product is safe and effective for patients. However, despite these regulations, errors can still occur during the manufacturing process. And when errors do occur, they can result in very serious consequences for patients, including allergic reactions, a wide array of side effects, or even death.
Considering the high stakes involved, pharmaceutical manufacturers must have strict quality control procedures in place. This includes regular testing and inspection of raw materials and finished products, as well as strict protocols for handling, storage, and packaging of drugs. The technologies involved in these inspections are often very costly, so new innovations that can reduce costs, while still offering accurate, high-quality evaluations of the medications would be very welcome in the industry.
Engineer Naveen Kumar believes that with recent advancements in both hardware and software tools, there is an opportunity to produce simpler inspection systems that will not blow the budget. He created a prototype device that demonstrates how Edge Impulse Studio and a Seeed Studio reComputer can be put to work for complex computer vision tasks. In particular, he showed how defects in capsules can automatically be detected. And using the same basic methods, it would in theory be possible to detect many more types of manufacturing defects.
Kumar’s plan was to use an object detection algorithm to identify certain types of capsule defects that can occur during manufacturing. To handle the computations, a Seeed Studio reComputer with a Jetson Xavier NX module was selected. Capable of performing up to 21 trillion operations per second, and sporting 8 GB of RAM, this little powerhouse packs a lot more punch than is required to run a computer vision pipeline that has been highly optimized for edge devices by Edge Impulse Studio, so excellent performance would be expected. A webcam was also included in the build to give the device the ability to capture images of the capsules.
Running a one-liner on the command line is all it takes to set up the Edge Impulse CLI on the Jetson. After doing so, the device will be directly linked to an Edge Impulse Studio project, which makes it very easy to collect data and deploy models. Speaking of data collection, Kumar needed to collect sample images of the types of defects he wanted his system to be able to detect, so he got together a wide assortment of capsule shells. He cracked some of them, and drew spots on others, then collected just over 200 images to train the model.
After the images were uploaded to the project, the defects needed to be labeled by drawing bounding boxes around them so that the algorithm could learn exactly what they look like. That could take quite a lot of time to do, even for a small dataset like this. But fortunately, the labeling queue tool gives users an AI-powered assist by helping to draw these bounding boxes. Typically it is only necessary to verify that the boxes look correct, and occasionally make a small tweak.
The next step in the process was to design the impulse that defines the machine learning analysis pipeline. This is where the real work happens in the project, but nevertheless, Kumar found creating the impulse to be the easiest part of the entire build. Using a simple, web-based interface, preprocessing steps were added to reduce the size of the input images and extract the most important features from them. These steps both serve to reduce the computational complexity of the next step — the object detection model. Edge Impulse’s FOMO algorithm was selected because Kumar found that traditional object detection models performed poorly in detecting tiny defects on small capsules.
Using the previously collected dataset, the model training process was initiated. Upon completion, it was discovered that the model’s accuracy rate was topping 98%. To ensure this excellent result was not just a case of a model overfitting itself to the training data, the more stringent model testing tool was also used. This is another test that uses data that was not included in the training process. This showed an accuracy rate of nearly 90%, which is quite good considering the very small size of the training dataset. This accuracy would be expected to improve by providing a larger amount of sample data.
The algorithm was ready to go at this point, so Kumar pivoted to the Deployment tab, then selected the “NVIDIA TensorRT Library” option for deployment to the Seeed Studio reComputer. He then grabbed a copy of a standalone inferencing example from GitHub created by Edge Impulse, which can be modified as needed for one’s own needs. The model that was downloaded from the deployment tab was added to the example code, then the device was up and running, able to detect defects in capsules.
When we said that the Jetson Xavier NX module packed a lot of power, we were not kidding. The inferencing rate was topping out at around 165 frames per second. It was so snappy that it had no problems detecting defects as the capsules were in motion, as might be the case on a conveyor belt during manufacturing. So while the cost of this system is already very low, it could likely be reduced even further by scaling back the hardware resources a bit.
For those in the pharmaceutical manufacturing industry, being told that product inspections can be done inexpensively may be a hard pill to swallow, but this project has proven it to be true. Take a look at the full documentation of Kumar’s project to see for yourself.
Want to see Edge Impulse in action? Schedule a demo today.