Image Defect Classification & Detection Using AutoML

Asian Paints Ltd, a leader in the paint manufacturing industry in India since 1967, is taking a groundbreaking step towards automating defect classification in painted surfaces. Facing the challenge of enhancing efficiency and accuracy in detecting and classifying paint defects, the company has now embarked on integrating advanced machine learning technologies. This blog explores how Asian Paints is revolutionizing its defect detection process by leveraging AutoML for image classification.

The autoML implementation has been the next step in their modernization journey, significantly reducing manual effort and streamlining operations to better serve its customers.

Business Objectives and Machine Learning Implementation

The process of defect classification has been rather  labor-intensive , for which the business wanted to build a machine learning solution model specifically  capable of image defect classification. Their traditional method was not only time-consuming but also prone to human error, impacting overall efficiency and customer satisfaction.

The Challenge – Solving for Cumbersome Manual Tasks 

Whenever the Asian Paints team received a defect request, they used to visit the client’s location, perform all necessary procedures, and consult with a paint expert to detect the defect present in the wall. They would then provide the appropriate solution to the user. This was a time-consuming process, and these activities often took several days to weeks.

The Asian Paints team wanted to reduce manual efforts and make the entire process autonomous.

How the Machine Learning Models Addresses the Business Goal

GCP’s AutoML Image multi-label classification provided a low-code solution that could be searched for through a zoo of ML algorithms. This could possibly be used as a single model for multiple defect types that were detected with respective confidence thresholds. You can check the detailed implementation in our case study, Asian Paints’ Automated Defect Detection using Image Processing.

This AutoML service was a part of the Vertex AI platform, an end-to-end ML platform, which was used for the deployment of the AutoML models on managed GKE clusters. AutoML supported all four data types, namely tabular data, text data, image data, and videos. It enabled the flexibility required to train the ML model, view the evaluation metrics, and deploy different versions of the model.

Data Exploration Based on the Hygiene List

There are a total 16 defect images contained in separate respective folders. In the realm of data exploration, a meticulous adherence to data hygiene standards is paramount for ensuring the integrity and utility of the information at hand. The prescribed criteria delineate a rigorous framework, wherein images must occupy a substantial 60% of the surface cover to adequately represent the underlying defects. Blurriness, an arch-nemesis of clarity, is rigorously forbidden, ensuring that each image possesses crisp, discernible features. Similarly, the prohibition of glare and flashes safeguards against distortions that could obfuscate crucial details. 

Moreover, a steadfast commitment to high-quality imagery underscores the commitment to precision and reliability, elevating the exploration process to a realm of excellence where each visual artifact serves as a faithful representation of its subject matter. 

Adherence to such hygiene standards not only fortifies the foundations of data exploration but also imbues it with a sense of trustworthiness and accuracy indispensable for informed decision-making.

The Decisions Influenced by Data Exploration

Initially, the 16 defects (including no defects) were categorized into two broad categories/datasets as internal and external defects as specified by client’s field expertise. The categories are grouped as follows: 

Internal defects: 

• Blisters & Bubbles
• Chalking
• Efflorescence
• Fungus
• Patchiness
• Sand Like Particles
• Peeling
• Shade Variation
• Poor Hiding
• Poor Adhesion
• Rough Finish
• No defects (Internal)

External defects:

• Algae
• Blisters & Bubbles
• Chalking
• Efflorescence
• Lumps
• Patchiness
• Peeling
• Poor Adhesion
• Poor Hiding
• Rough Finish
• Shade Fading
• Share Variation
• No defects (External – Plain and Textured)

The decision was made to divide it into 2 categories (internal and external) based on data exploration as both have different characteristics. For example, external defects usually cover low surface area. There would be lots of sunlight, the outside texture or finishing is not that smooth and perfect, Sky coverage, all of which impact the paint. Whereas in internal defects it covers more surface area, low sunlight, etc.

Pre-Processing and Data Pipeline

As we have used AutoML which itself performs the data pre-processing, there is no need to explicitly pre-process the dataset. 

Which AutoML Product were Chosen

As mentioned above in the business use case and after analyzing the data, we have decided to use Image Classification (single-label) where for a given image it will detect the top most defect.

Data Sampling Methods Used

In our analysis of defect categories, a notable observation surfaced regarding the relatively low volume of image data associated with certain defect categories. In response to this discrepancy, a strategic decision was made to augment the existing data set. 

This augmentation process encompasses various transformative techniques, including but not limited to 90-degree rotations, zooming in, and applying 30-degree tilts to the images. By implementing such augmentations, we aim to bolster the diversity and richness of the data pool, thereby enhancing the robustness and generalization capabilities of our machine learning models.

Code snap 1 : Data augmentation with tensorflow

This proactive approach underscores our commitment to optimizing model performance and efficacy, ensuring that our algorithms are equipped to discern and address the business scenarios with accuracy and reliability.We have trained AutoML models with different numbers of images, categories (internal and external defects) and number of node hours. We have started the training with a lesser number of images and few node hours, With each iteration we have increased the image size and node hours. We have observed data augmentation works really well and an image set with augmented technique gives better results. The final model has an accuracy of 93.4%.

Model Iteration	Total Images	Node Hours	Accuracy
1	453	8	76.3
2	453	16	75.9
3	941	80	65.1
4	1895	60	79.8
5	3007	60	92.1
6	5270	30	93.4

Table 1 : Result of all the iterations

Machine Learning Model Evaluation

Online prediction helps in providing predictions to the real-time data that would be helpful for timely made decisions.

Steps for real time prediction are:

• Once we trained the final AutoML model we deployed it to Vertex AI endpoint which gives you the deployment ID.
• We have written python scripts which will call that endpoint id and give the prediction for the input image.
• We have also used docker image and deployed it to artifact registry
• At last on cloud run the model gets deployed which give the https API

For evaluation we have used average precision which is 93.4% for final iteration. The higher average precision means the higher confidence score for the top most defect detected by the AutoML model.

Image 1 : Confusion matrix for final model iteration

Conclusion:

Asian Paints Ltd. has taken a significant leap forward by integrating AutoML technology into their defect detection process. This transition to an automated, no-code solution not only streamlines operations but also enhances accuracy and efficiency in identifying defects. By reducing the dependency on manual labor and expediting response times, Asian Paints is not only ensuring higher customer satisfaction but also reinforcing its position as an innovative leader in the paint industry. As the company continues to embrace technological advancements, it sets a benchmark in the industry, showcasing how blending traditional business with modern technology can result in substantial operational benefits. This initiative marks a transformative step in their journey, promising even greater efficiency and effectiveness in future endeavors.