---
type: "post"
id: "paper-published-at-uist-2025-with-a-google-d7a2ef"
slug: "paper-published-at-uist-2025-with-a-google-d7a2ef"
title: "Paper published at UIST 2025, with a Google Research writeup available."
description: "Google Research introduced StreetReaderAI, a proof-of-concept that combines Street View panoramas, geographic context and Gemini to help blind users explore streets virtually."
retrieval_nugget: "Google Research introduced StreetReaderAI, a proof-of-concept that combines Street View panoramas, geographic context and Gemini to help blind users explore streets virtually. In a lab study, participants valued its conversational interface, but the measured errors and orientation difficulties show why accessibility features powered by AI need clear limits and careful evaluation."
published_at: "2026-11-17"
updated_at: "2026-11-17"
record_date: "2026-11-17"
date_kind: "published_at"
topics: ["google","gemini"]
entities: ["Google Research","StreetReaderAI","Gemini","Google Multimodal Live API","UIST 2025"]
related_patterns: []
novelty_label: "new_fact"
what_changed: "Google Research published StreetReaderAI, a UIST 2025 paper exploring context-aware multimodal AI to make Street View accessible."
prior_refs: []
source_url: "https://research.google/blog/streetreaderai-towards-making-street-view-accessible-via-context-aware-multimodal-ai"
source_urls: ["https://research.google/blog/streetreaderai-towards-making-street-view-accessible-via-context-aware-multimodal-ai"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:paper-published-at-uist-2025-with-a-google-d7a2ef"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/paper-published-at-uist-2025-with-a-google-d7a2ef/","markdown":"https://newruntime.com/posts/paper-published-at-uist-2025-with-a-google-d7a2ef.md","json":"https://newruntime.com/posts/paper-published-at-uist-2025-with-a-google-d7a2ef.json"}
markdown_url: "https://newruntime.com/posts/paper-published-at-uist-2025-with-a-google-d7a2ef.md"
---

# Paper published at UIST 2025, with a Google Research writeup available.

## Retrieval answer

Google Research introduced StreetReaderAI, a proof-of-concept that combines Street View panoramas, geographic context and Gemini to help blind users explore streets virtually. In a lab study, participants valued its conversational interface, but the measured errors and orientation difficulties show why accessibility features powered by AI need clear limits and careful evaluation.

## Summary
Google Research introduced StreetReaderAI, a proof-of-concept that combines Street View panoramas, geographic context and Gemini to help blind users explore streets virtually. In a lab study, participants valued its conversational interface, but the measured errors and orientation difficulties show why accessibility features powered by AI need clear limits and careful evaluation.

## What happened
Google Research’s October 29, 2025 announcement describes StreetReaderAI as an accessible street-view prototype presented at UIST ’25. The project was developed iteratively by blind and sighted accessibility researchers. Its premise is that screen readers cannot interpret Street View imagery and that ordinary alt text is unavailable for this immersive visual experience. Google notes the potential scale of the problem: Street View contains more than 220 billion images across over 110 countries and territories. The announcement presents the prototype as a proof of concept, not a newly released feature in Google Maps or Street View.

The system has two Gemini-backed subsystems. AI Describer combines analysis of the current Street View image with dynamic geographic information, including nearby places, road information and the user’s current field of view, to generate an audio description. It offers a default mode emphasizing navigation and safety for blind pedestrians, and a tour-guide mode that adds tourism context such as historical or architectural information. Gemini also predicts possible follow-up questions about the scene and local geography.

AI Chat builds on that description flow and lets people ask about the current view, previous views and nearby geography. It uses Google’s Multimodal Live API for real-time interaction, function calling and temporary memory within a session. The system sends the user’s view, location and geographic context as the user pans or moves. The announcement says the chat context window is set to a maximum of 1,048,576 input tokens, described as roughly equivalent to more than 4,000 input images. A user could, for instance, pass a bus stop and later ask where it was; the agent can use the session context to answer with its location relative to the user.

People navigate with voice commands or keyboard shortcuts. Left and right arrows pan, with audio feedback for cardinal or intercardinal heading; up and down arrows move forward or backward in virtual steps. The system reports distance traveled and nearby geographic information, and includes jump or teleport controls for moving quickly to new locations. The announcement does not state a public price or general access route.

Google evaluated the prototype in an in-person lab study with 11 blind screen-reader users who explored locations and considered potential walking routes. They visited over 350 panoramas and made more than 1,000 AI requests. Participants rated overall usefulness 6.4 out of 7 (median 7; standard deviation 0.9). Of 816 AI Chat questions, 703 (86.3%) were judged correctly answered, 32 (3.9%) incorrect, 26 (3.2%) partially correct and 54 (6.6%) refused. The most common question categories were spatial orientation (27.0%), object existence (26.5%), general description (18.4%) and object or place location (14.9%).

## What the archive adds
New Runtime’s earlier coverage of [A2UI 1.0](https://newruntime.com/posts/a2ui-1-0-build-delightful-fast-reliable-agentic-456dee/) described an open protocol for agentic experiences across platforms. StreetReaderAI is a new research event, not a new fact about that protocol: it explores accessibility through a particular conversational interface to geographic imagery, rather than proposing an interoperability standard.

## Trend signal
This does not yet fit a pattern New Runtime tracks. StreetReaderAI shows a task-specific conversational interface around visual and geographic context, but a research prototype and one lab evaluation do not establish a broader shift in how model answers become task interfaces.

## Why it matters
The study offers product teams a more useful design target than “describe the image”: people asked where objects were, whether features existed, and how far away things might be. Combining a live view with location context and short-term interaction history can support questions that a static caption cannot answer. The six-to-one preference for AI Chat over AI Describer suggests that some users may value a dialogue they can steer more than a single generated description.

But accessibility raises the cost of confident mistakes. A false negative about a bike rack or a misidentified speed bump can distort someone’s understanding of a route. The team also reports that participants sometimes struggled to orient themselves and judge the truth of answers. For builders, the lesson is to evaluate task-level interaction and failure modes with intended users—not just whether a model can produce fluent scene descriptions. A compelling interface still needs to communicate uncertainty and avoid implying that virtual exploration is equivalent to safe real-world navigation.

## Limits and uncertainty
The evaluation involved 11 blind screen-reader users in an in-person lab, so these results do not establish performance across a wider population, locations, devices or real-world travel. Google says more work is needed beyond the lab. The reported accuracy is based on 816 questions in this study; the announcement does not describe a separate independent evaluation or establish how often failures would affect real route decisions. The prototype does not yet support full origin-to-destination routing. It also leaves unresolved how users should assess the limits of its knowledge, a difficulty participants encountered during testing. No public launch, production availability or price is announced.

## Reader takeaway
StreetReaderAI is a research prototype, not a service to rely on for navigation. Its strongest design signal is that accessibility can come from a context-aware conversation—not only from image captions—while its study makes clear that orientation, uncertainty and errors need to be treated as core product requirements.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Google](https://newruntime.com/topics/google.md): Explore the google topic hub.
- topic hub: [Gemini](https://newruntime.com/topics/gemini.md): Explore the gemini topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)
