The model maps text, images, video frames, and audio into a shared vector space and is designed for local execution. Google says the full multimodal model uses about 567 MB of active RAM on a Pixel 11 Pro, while text-only weights use about 191 MB.
The Google AI Edge Gallery adds demonstrations for searching local photos and videos with natural-language queries, including locating moments in video without first transcribing audio or generating text captions. Google also says ML Kit support is planned for the coming weeks; that integration is not yet available according to the announcement.