Case Study
Automated Label Data Extraction
for Data Fleet
Company |
Industry |
 |
Facility Management |
Client
Data Fleet is an asset information management company located in the US. With their proprietary software platform, Property Echo, they help companies manage their facility's assets by combining a skilled field workforce with advanced digital capture tools. This gives companies greater insight and awareness of their operational assets, improves operating costs, reduces the time to respond to issues, unlocks hidden cost-saving opportunities, and more.
Impact
Data Fleet's objective was to speed up and improve the process of inputting data from the labels found on the refrigerators maintained by their field workers. The ready-to-use AI optical character recognition (OCR) tools proved insufficient for accurately extracting the specific data required in the specific format. However, our algorithm enabled field workers to simply take a photo of the refrigerator label, and the relevant data was automatically extracted, post-processed, and saved. This AI technology not only improved the speed and efficiency of field workers but also elevated the quality of the collected data, thereby enhancing the overall reliability and utility of their platform.
Problem
After Data Fleet's field experts complete their assessment and maintenance of a particular asset or equipment, they are required to take a photo of the asset and input data such as:
- Manufacturer's name
- Model number
- Serial number
- Manufacturing date
- Additional data specific to the asset
Challenges
Different labels
Different assets, even from the same manufacturer, have different labels, which means that relevant information is written in different places and different formats.
Partially visible labels
Suppose the label is placed in some hard-to-reach place (imagine a label on the side of a fridge close to a wall or inside the bottom of a fridge). In that case, the angle of the photo can be a problem, or only a part of the label is visible, and overall photo quality is not ideal. However, even if only part of the label is visible in the image, our algorithm is still functioning effectively.
Including new labels and data
We need a way to add new labels and data to be extracted easily. Also, for all labels, we need an easy and flexible way to add new data, even if it's specific to a particular label.
Inference speed
The algorithm must work in real time. After a field expert takes a photo, the extraction of necessary data should take no more than a few seconds.
Solution
Our solution combines Azure OCR, a matching algorithm like SIFT, and our custom logic. This combined algorithm takes an input image containing a label, aligns it, sends it to the OCR, and then extracts only the relevant data. We have developed a custom data post-processing step that ensures data quality and fixes common AI OCR tool errors. Additionally, we have established criteria for evaluating the accuracy of results. Results that meet the criteria are automatically accepted, while those that don't are flagged for manual review by a field worker. These components are integrated into a production API, making it easy for our clients to use and integrate into their systems.
Tools and Technologies
Python
OpenCV
Azure
Numpy
Pandas
PyTorch
Results
Our API enabled easy integration into Data Fleet Property Echo software. This integration has facilitated fast and accurate data collection from Data Fleet field workers, which eliminates the need for laborious manual data entry, allowing them to carry out their tasks more efficiently and quickly. Consequently, the gathered data attains a higher quality level, significantly improving their platform's overall reliability and utility.
Discover More Case Studies
Discover the impact of our custom AI solutions on business success through customer stories
