If a picture is worth a thousand words, what might retailers learn about their products, customers, and stores from hundreds of high-quality, 3D images stitched together?
Computer vision, sometimes called the “eyes” of artificial intelligence (AI), turns captured images into valuable pieces of data. Thanks to advancements in both camera hardware and AI, computer vision and AI can inform new strategies for managing inventory, reducing out-of-stocks, and repurposing labor time spent on restocking.
A full 58% of retailers have definite plans to implement computer vision into their operations. And it’s no surprise why: accurate computer vision not only improves existing operations in-store—it’s also a critical stepping stone for unlocking 5% operating margin and 4.5% sales, per a Coresight report on the state of in-store retailing.
Realizing such ROI across the retail business can be tricky, however. The use of computer vision in retail is still relatively new, which means stores in the buying process will be challenged to separate best-in-class solutions from vaporware. Leaders have to ensure the technology is actually available, tested, and effective in practice. Several store chains have already begun to withdraw their deployments of computer vision technology because they lacked confidence in the viability of their solutions.
This asset intends to help IT and operations teams feel confident choosing a solution that will actually deliver business results.
Behind the lens: what gives the computer its "vision"
The two components that make computer vision for AI work are the cameras that capture images, and the software that turns these images into data with AI. The type and position of the cameras is key to capturing quality photos—and by extension, generating quality data and insights.
In retail stores, cameras need to capture products of all sizes, located at different heights, and in all kinds of lighting, from the bright lights of a grocery aisle to the shadowy spaces at the back of the shelf. They can’t rely on a flash, because many products are located behind freezer doors (not to mention, constant flashes throughout the store would be pretty distracting to customers!) They also need to capture minute details, like barcode numbers, item descriptions, and price.
We built Tally, our autonomous inventory robot, to specifically overcome all these challenges. Tally is tall enough to capture products located on high shelves and is the only robot able to capture precise images through freezers and coolers.
We built Tally, our autonomous inventory robot, to overcome all these challenges. Tally is tall enough to capture products located on high shelves, and takes photos without using a flash. Two types of cameras work together to take in different pieces of information. High resolution cameras, similar to those found in smartphones, capture color and shape, while 3D cameras capture depth, ultimately creating an output closer to what human eyes see in real life versus a flat, 2D image. Each robot is equipped with more than 5x the cameras of any existing solution on the market today. Because of the ability to maneuver around the store, there's no limit to the number of angles (and therefore, the quality of data) Tally can capture—a severe limitation of fixed cameras.
Tally can fully scan a typical grocery store in about three hours, capturing approximately 400 images per aisle. The results? Cohesive views of entire shelves as well as precise details about each item.
How pictures become pieces of data
The computer then steps in to make sense of all of these pictures, as well as the time and location data captured with each shot.
First, it looks to see what’s in the image using neural network processing to analyze different shapes within images and break them down into “areas.” Object detection techniques then identify what each of these areas actually are, categorizing and labeling key elements of the photos, such as price tags, promotion tags, shelf edges, and products themselves. Overlaying multiple shots from the color and depth cameras allows Tally to easily and consistently distinguish between empty, low, and full shelf areas.
Specialized optical character recognition systems target text with higher resolution to extract information about regular and promotional prices, for example. AI also enhances the readability of price tags and barcodes: Tally is even able to tell the difference between a regular and a fat-free variety of soup from up to 40 feet away.
Using AI computer vision data to improve processes
Once processed by the computer, retailers gain real-time visibility into products in their stores. These insights can drive operational efficiency and the bottom line. As with any AI model, the more precise the data inputs, the more valuable the outputs. Precise, clear images are a key component of seeing ROI from computer vision technology.
For example, today, 60% of products that store teams believe are out of stock are actually somewhere in the store. Data about a store inventory can be used to generate daily task lists for employees to precisely pinpoint and address out-of-stocks items, as well as dramatically improve online inventory accuracy and order fulfillment. The data can also help stores figure out the best restocking times for fast-moving products—minimizing out-of-stock situations while maximizing potential sales.
Data about a shelf’s contents can be used to generate daily task lists for employees to precisely pinpoint and address out-of-stocks items—minimizing out-of-stock situations while maximizing potential sales.
Precise computer vision for AI can also ensure accurate pricing. With manual audits, up to 10% of price tags might be incorrect. With up-to-date inventory count data, teams can ensure pricing and promotional tags are applied correctly—which is especially important in instances where customers that purchase mispriced items are entitled to discounts or money back. Pricing and promotion errors are also a legal liability and can even put relationships with CPG brands at risk.
Computer vision, alongside the robot’s navigation data, also allows for creating a physical map of the store—allowing a customer to easily find the aisle and shelf of a product they search for. Stores can also use the data to update (or potentially replace) planograms or optimize pick paths for both ecommerce fulfillers and store staff.
As Krystyna Kostka, SVP of Store Operations at BJ’s Wholesale Club told Retail Customer Experience about how Tally the robot is helping BJ’s boost the customer experience: “The BJ's brand and mission are all about creating an exceptional member experience. Tally is an amazing robot that allows us, with computer vision, to see exactly where our stock is every single day in every place in the store.” The BJ’s case study and BJ’s video testimonial about Simbe share further details.
Customizing computer vision AI solutions to your store needs
Using AI computer vision systems, mobile cameras can routinely collect terabytes of data, opening up a world of information for stores to capitalize on. However, many of these computer vision systems rely on third-party companies to handle the data they collect. This slows down how long it takes customers to get these crucial insights and often means the analysis of the data is limited by what the partner can offer.
More companies—up to 70%—want technology partners to be able to provide end-to-end support when implementing AI initiatives. At Simbe, we own both our hardware and software. This tight integration between our robots and our cloud-based app means we can always provide our customers with timely data. We can even build and run customized code for different needs, such as pallet optimization, validating bottom-of-cart items, and seeing to the top of upper steel in wholesale clubs.
Computer vision serves as the foundation of every aspect of our technology stack, from capturing images of shelves to processing vast amounts of data extracted from those pictures. With the ability to seamlessly navigate data capture, processing, and analysis, retailers can transform data to streamline operations, boost revenue, and enhance the overall store experience for customers and workers alike.
As you assess computer vision and AI companies, however, it’s important to understand all the nuances across your business, vendor claims, and available technology. This IHL Report shares an evaluation checklist for computer vision technology.
