The RedAI app is an artificial intelligence software solution that enables FMCG/CPG companies in Retail and HoReCa to detect, classify and control their own and their competition’s SKUs (products) with a series of photos taken from a mobile device or tablet. It was created as a result of the partnership between a company that provides custom AI and machine learning systems and Serengeti.
This semi-custom application provides full automation of the in-store execution process. It generates real-time results and delivers complete business intelligence reports. The main purpose of RedAI is to perform visual perception and brand recognition. The results and reports will allow the user to get a clear insight into what is happening in stores and effectively act on it while optimizing time spent per store.
How RedAI Works
RedAI’s artificial intelligence neural network scans the photo based on brand, packaging or SKU and detects, classifies, and delivers a complete analysis of market share per category, planogram compliance and custom KPI metrics. It helps automatize daily business, track market activities and stock status in real-time, for every single location.
The first step in using the RedAI application is taking a picture of a shelf or a fridge with a mobile or tablet device. The application offers simple incorporated guidelines, so the process is simple to perform.
After the photos are taken, the AI software detects, processes, and classifies every SKU which has priorly been defined.
Within minutes, the AI delivers results.
The application consists of three key components: a semi-custom-made application, an AI algorithm (deep learning agents) and Big Data analysis output for business intelligence.
Serengeti’s team started with gathering, processing and analysing data required for the machine learning process. After all the necessary data was processed and analysed, together with our partner we’ve started with development and evaluation of models and specific learning procedures.
The architecture of the RedAI application consists of PostgreSQL, ASP MVC Core API&Pages, AI Workers, Web Browser, and Mobile App. Serengeti’s task in this project was focused on AI Workers, i.e., the part related to the use of machine learning technologies with emphasis on deep learning. This part is key for the project delivery and data processing as it enables detection and classification of SKUs, which are the main components of business process automation (BPA).
The RedAI application is based on image processing. It uses computer vision for:
The input image is processed through the deep learning concept.
After the input, the picture goes through reconstruction and inference. The first step in the deep learning concept of image processing is reconstruction, which consists of features, match, reconstruct, and dense reconstruct. In this part, the picture goes through a 3D reconstruction of the shelf. After the completion of the 3D reconstruction, the inference phase follows.
The inference has three parts – detection, classification, and segmentation – and it represents the most important part of the application.
For inference detection, we have used Faster R-CNN, one of the three main models for object detection. Faster R-CNN is a part of a series of models that use region proposals for object detection. It uses a pre-trained CNN generating a convolutional feature map while using it as part of the Region Proposal Network (RPN) that finds region proposals. We have set this model to be trained and optimized in accordance with the number of product classes on the shelves that need to be detected.
For inference classification, we have used a pre-trained neural network based on convolutional neural networks with several convolutional and one fully-connected layer.
For the third step, inference segmentation, we have used a deep learning model called a fully convolutional network. This model is trained and optimized for one segmentation class. In this case, it was trained for a single shelf.
The final part of image processing is the output image result, which gives a reconstructed shelf. During this phase, outliners are removed, and the image is down sampled. It is necessary to find the shelf plane and detect and contour the back project, after which the reconstruction scale is estimated. Using NMS RedAI, the application then merges shelf plane detections and creates its outputs through extended discovery (dots, contours, and endpoints) and coordinates normalization.
After merging shelf plan detections, the application merges shelf plane contours into shelves and creates planogram data. It assigns the product to shelves and calculates shelf shares.
The next feature that Serengeti’s team is working on is price recognition and 3D product mapping, which requires further application of advanced and modern technologies.
Serengeti’s team, in cooperation with our partner, successfully created a product that leverages retail AI image recognition technology to get ahead. This partnership resulted with a user-friendly app that delivers immediate results on the device in correlation with your previously defined SKUs and KPIs. This time-saving application with the ability to analyse large amount of data fast, increases sales force productivity and gives a clear insight into what is happening in stores, including an analysis of the competition.
RedAI reduces the time required for sales rep to visit stores by 40%. The application is also a web-based app that permits other departments to access the collected data for better data analysis and compliance planning.