Computer Vision Application for Retail Industry

Retailers and brands have to deal with a common pain point: keeping track of all the merchandise. Auditing products in stores is the most challenging because it requires a lot of time and attention, while do not add value for sales visits. Sales reps and merchandisers are required to track and audit various KPIs – SKUs, facings, shelf level, competitors, promo, etc. For brands having large product portfolio in various categories, this becomes even more channeling and may lead to mistakes. 

Customertimes (CT), a leading software publisher and systems integrator in the Salesforce ecosystem, is focusing on providing applications for consumer goods and consumer health companies. CT products include mobile apps for sales force effectiveness and computer vision for AI-powered store checks. CT products – CT Mobile, CT Presenter, and CT Vision – are widely adopted by leading consumer goods and consumer health brands such as Bayer, Danone, L’Oréal, Pernod Ricard, Abbott, etc. Customertimes is recognized by the Promotion Optimization Institute (POI) as one of the leaders of the retail execution market.

As part of their innovation stream, Customertimes have decided to add Augmented Reality to their CT Vision product to demonstrate the future of retail audit and retail execution. CT Vision is already used by field sales teams of companies such as Danone and L’Oréal on their tablets. The idea was to take a step further – to empower the reps by equipping them with wearable AR devices that could help them do their work efficiently.

By having product information – product availability, actual shelf vs. the target one, missing SKUs, etc. – literally in front of their eyes, reps can be much more efficient and focus on fixing the deviations rather than just reporting those. Moreover, such technology can help to gain the time of the store check and thus let the reps spend more time on selling new offers, promotions, etc.

The open architecture of CT Vision and various APIs made possible direct integration with AR wearable goggles. As Customertimes did not have in-house expertise in AR, they have interviewed several service providers and have selected Skywell for AR application development.  

Skywell utilized its deep expertise in retail software solutions to realize this project. Only based on this competence, we could develop the AR app, integrating with CT Vision backend and enabling the end to end process from image capture to image recognition and visualization using AR.

As you can imagine, creating such an advanced retail image recognition solution involves using a combination of various technologies. For the pilot project, Customertimes have provided the EPSON Moverio BT-300 AR-glasses. This device is much lighter than those of the competition, and its cost was much lower as well.

On our end, we used Unity and SDK Vuforia to achieve the most flexible solution possible, i.e., one that can work on both AR-powered glasses as well as on mobile devices. This played a significant role in choosing the tech stack because one of the main requirements was that the solution must support both visualizations mentioned above. But first things first.

Choosing the proper SDK, we narrowed down our decision to Vuforia and Wikitude. However, after conducting extensive research, we discovered that Wikitude could only support visualization on mobile apps, despite their website claiming that headgear was supported as well.

So, to visualize the information in AR, we chose Vuforia basic SDK that allows for quality object tracking and would enable us to fully actualize the inventory management software that client had in mind. SDK Vuforia turned out to be a universal solution for the computer vision project, and the Unity engine gave us the ability to adapt the solution for all platforms. As of today, the platform was built only for Android, but it is meant to be deployed for other platforms in the future.

While choosing the right tech stack and further programming, it was vital to consider the array of requirements for the solution.

For Customertimes, it was essential that the solution was tested & set up correctly. Important stages in the development were gaining control of the gyroscope and autofocus of the camera, adjusting the transmission and preparation of data.

It was also crucial that everything was done within the short timeframe it was supposed to be demoed at the Customertimes’ customer event in Paris. 

It was also necessary to ensure the flexibility of the team. Since the deadlines were very tight, the team that worked on the project had to be as accessible as possible for the client, flexible as far as implementation was concerned, and rapid in overcoming the difficulties.

Customertimes have paid particular attention to the scalability of the solution, so it could be easily scaled in the future, and new modules could be added, as well as some new features. 

To create a universal solution, we had to choose a technology stack that would ensure the retail shelving solution could work on all platforms (iOS, Windows, etc.) and different devices. And it became possible thanks to Unity and Vuforia SDK bundles.

Customertimes have chosen the Skywell Software team due to the expertise and insider know-how. This includes timeboxing — we were the ones who took the responsibility to develop the solution in a short time and made under the short deadlines. Besides, we provided the client with timely, accurate communications of the findings that were discovered during the research. So, all of the added values played a crucial role in the client’s decision to choose us as their research & development partner.