Pick n Pay, a South African retailer competing in the local online grocery sector, released an artificial intelligence shopping assistant within its delivery application. The deployment utilizes Gemini models from Google, a technology vendor supplying global enterprise software systems, to process multimodal inputs. The rollout begins July 6 and will reach all application users during the month.
The assistant, branded as Penny, operates inside the asap! application to interpret natural language requests and generate shopping baskets. Rather than navigating manual search menus, shoppers submit voice commands, text prompts or photographs to locate products. The system analyzes images of handwritten lists and existing refrigerator inventory to calculate meal plans based on defined budgets. Through the Google integration, the voice search function processes commands across multiple languages.
The application connects the assistant with the Smart Shopper loyalty program to calculate recommendations. This integration surfaces suggestions based on previous purchases, preferred brands, item quantities and price points. Users operate the conversational interface to reload previous orders and execute repeat transactions.
“Penny allows customers to upload handwritten shopping lists, photograph ingredients from recipes, snap products they want to buy or even take photos of ingredients already in their fridge to receive meal suggestions,” noted Enrico Ferigolli, retail executive: omni-channel at Pick n Pay. “The AI assistant can also recommend recipes, suggest ingredient substitutions, help plan meals within a specific budget and instantly add the required ingredients to a customer's shopping basket. Is Penny going to be 100% perfect? Absolutely not. But it will continue improving over time."
Kabelo Makwane, country director for Google South Africa, addressed the mechanics of the multimodal integration during the launch event in Johannesburg.
"We are now in the era of AI assistants, where AI is moving beyond simply answering questions, to helping people get things done," Makwane said. "The power of multimodal AI is that people don't have to adapt to technology – the technology adapts to them. Whether someone speaks, types or shares a photo, AI understands their intent. When these capabilities are built into products people already know and trust, they solve real everyday problems in a simple, intuitive way."
The software release follows a platform modernization project executed last year across the front and back ends of the asap! service. The retailer implemented the architectural changes as the domestic e-commerce market intensified, with rival organizations Checkers Sixty60, Woolworths Dash and SPAR2U deploying rapid delivery and tracking features.
"Last year, we re-platformed asap! across both the front- and back-end, significantly accelerating our speed of development. That gives us the ability to bring new AI-powered innovations to customers much faster, with more enhancements rolling out over the coming months. This is only the beginning of what AI can unlock for everyday grocery shopping, and Penny is the first step in that journey," Ferigolli concluded.
