ShopTalk: Independent Blind Shopping Through Verbal Route Directions and Barcode Scans

ABSTRACT

Independent shopping in modern grocery stores that carry thousands of products is a great challenge for people with visual impairments. ShopTalk is a proof-of-concept wearable system designed to assist visually impaired shoppers with finding shelved products in grocery stores. Using synthetic verbal route directions and descriptions of the store layout, ShopTalk leverages the everyday orientation and mobility skills of independent visually impaired travelers to direct them to aisles with target products. Inside aisles, an off-the-shelf barcode scanner is used in conjunction with a software data structure, called a barcode connectivity matrix, to locate target product on shelves. Two experiments were performed at a real world supermarket. A successful earlier single-subject experiment is summarized and a new experiment involving ten visually impaired participants is presented. In both experiments, ShopTalk was successfully used to guide visually impaired shoppers to multiple products located in aisles on shelves. ShopTalk is a feasible system for guiding visually impaired shoppers who are skilled, independent travelers. Its design does not require any hardware instrumentation of the store and leads to low installation and maintenance costs.