Smartphone App Mobilio Aids Blind and Low-Vision Navigation

Engineers at Harvard's John A. Paulson School of Engineering and Applied Sciences (SEAS) have created Mobilio, a smartphone app designed to assist blind and low-vision users in navigating their environment more safely and efficiently. The system integrates machine learning, algorithms that process data from multiple sensors, and personalized audio cues to deliver turn-by-turn directions, path guidance, and obstacle avoidance.

Development of Mobilio was informed by surveys with over 100 blind or low-vision individuals, who identified reliable turn-by-turn directions, continuous path guidance, and obstacle detection as critical needs. Existing assistive tools often address only some of these requirements and can be costly. While GPS navigation exists, its accuracy is insufficient for safe independent travel by visually impaired users.

Mobilio leverages a smartphone's built-in sensors, including the camera, GPS, inertial measurement unit, and LiDAR (if available). A key innovation is a custom computer vision model trained on pedestrian-perspective images to accurately identify walkable surfaces like sidewalks and crosswalks. This addresses limitations of standard models trained on car-mounted camera data.

Ensuring all computational processes run in real-time on a standard smartphone was a significant challenge. Mobilio provides continuous directional audio cues, like beeps, that indicate steering adjustments. The system employs human-in-the-loop optimization, learning from how accurately a user follows the cues and adjusting audio patterns for personalized guidance.

Evaluations with 14 participants from the Carroll Center for the Blind demonstrated Mobilio's effectiveness. Users employing Mobilio and a cane completed an outdoor route approximately 13% faster and reduced obstacle contact by about 41% on an indoor course compared to using Google Maps with a cane. The app's reliability was found to be comparable to that of a human guide.