Work & case studies · Retail & E-commerce
Multimodal AI catalogue system for a home-and-garden retailer
Vision-language models that turn supplier photos and spec sheets into structured, searchable product listings, with attribute extraction, image tagging and visual search.
A home-and-garden retailer received supplier photos, PDFs and half-filled spreadsheets for tens of thousands of products and turned them into listings by hand. We built a multimodal-ai catalogue pipeline that reads images and documents together, extracts attributes, writes consistent descriptions, tags images and powers visual search. New products now go live roughly ten times faster.
Challenge
Merchandisers spent most of their time transcribing dimensions, materials and colours from photos and spec sheets, and still produced inconsistent listings, with the same shade of green filed under four names. Missing attributes broke filters, and customers who arrived with a photo of a garden chair had no way to find its equivalent. Suppliers were not going to improve their data, and previous text-only extraction had failed because the crucial details were in the images.
What we built
- An ingestion service that pairs every SKU's images with its documents and runs a vision-language pass (GPT-4o, with Qwen2-VL evaluated as a self-hosted alternative) to extract dimensions, materials, colours, style and use-case attributes into the retailer's taxonomy.
- Cross-checks between sources: when the spec sheet and the image disagree on colour or size, the item is flagged rather than guessed.
- A description generator (
generative-ai-llm) with tone and length rules per category and a mandatory review step for the first ninety days. - CLIP-based image embeddings in Qdrant for "find similar" on the storefront and for de-duplicating near-identical supplier photos (
computer-vision). - A merchandiser workbench showing extracted attributes beside the source image, with one-click corrections that feed the evaluation set.
Results
Listing a new product fell from about two hours to roughly twelve minutes of merchandiser time ; attribute accuracy on the weekly QA sample sits around 93% , with the flagged-disagreement queue catching most of the rest. Consistent attributes fixed the filters, and search-to-product-page click-through rose by around 9% after visual search launched. The retailer cleared its backlog of unlisted SKUs within a quarter.
What made the difference
We built the disagreement flag before the description generator. Merchandisers stopped fearing the automation the moment they saw it admit uncertainty, and the review workbench turned their corrections into training signal instead of lost work.
Ready to chart your AI course?
A free 30-minute consultation: we listen, ask hard questions and tell you honestly where AI will and won't help.