Google teaches its medical AI to read a room — literally
AMIE now handles live video consultations, catching visual and verbal cues the way a physician would
Written by OutOfToken AI
August 12, 2026 · 4 min read · Synthesized from reporting by AI News · How this works
Google Research and Google DeepMind have pushed AMIE, the company's research medical AI system, into new territory: real-time video consultations. In evaluations using trained patient actors, clinical evaluators rated AMIE's performance on par with primary care physicians across several core measures — the first demonstration of expert-level AI performance in this live, audiovisual format.
From text chat to live video
AMIE — short for Articulate Medical Intelligence Explorer — first made headlines for text-based diagnostic conversations, where it matched or exceeded primary care clinicians in simulated consultations. But text chat strips away most of what happens in an actual visit: a physician clocking a patient's cough, labored breathing, or grimace while asking questions. Google says that gap has kept expert-level AI performance out of reach in video-based virtual care, until now.
Built on Gemini and Project Astra
The new version of AMIE is built on Gemini and Project Astra, using a multi-agent architecture rather than a single model handling the whole conversation. That architecture is designed to let the system interpret visual and auditory signals alongside spoken language, feeding all of it into its diagnostic reasoning rather than treating video as a mere backdrop to text.
Testing across the body's major systems
Google's evaluation used fifteen trained actors portraying conditions spanning cardiopulmonary, abdominal, HEENT, neurological or psychiatric, and musculoskeletal presentations. That range was meant to stress-test AMIE's ability to synthesize spoken symptoms with observable physical cues — the kind of case mix a primary care physician might see across a single week of appointments.
"Clinical evaluators rated AMIE's video consultation performance on par with primary care physicians across several core measures — a first for AI in live, audiovisual clinical settings."
A research system, not a rollout
Google is careful to frame AMIE as a research system, not a product ready for clinics. The company has said studies involving real patients and their own health information are needed before anything resembling real-world deployment, and it hasn't laid out a timeline for when — or whether — that testing will begin.
The jump from text to video is a meaningful technical leap, but the harder test is still ahead: real patients, real stakes, and none of the scripted predictability that trained actors provide. Whether AMIE's expert-level ratings hold up outside a controlled study will determine if this becomes a genuine tool for virtual care or another impressive research demo.
Editorial Note
The research sources strongly corroborate the core factual claims in the article: AMIE's video consultation capabilities, its multi-agent architecture based on Gemini and Project Astra, the use of 15 trained actors across multiple clinical domains, and its performance parity with primary care physicians. The sources confirm this represents a first demonstration of expert-level AI performance in real-time audiovisual clinical settings. The article's framing of AMIE as a research system requiring further validation with real patients also aligns with Google's cautious positioning in the sources.
Claim Tracker
AI-assessed
Source 3 (Google Research blog) and Source 5 (LinkedIn post) both confirm AMIE demonstrated 'expert-level AI capabilities' in real-time video consultations with patient actors, with performance comparable to primary care clinicians.
Source 3 references the evaluation methodology using trained actors portraying diverse clinical presentations, consistent with the article's claim about case mix scope.
Source 3 explicitly states: 'Built on Gemini and Project Astra using a multi-agent architecture, AMIE' interprets visual and auditory signals.
Sources 2, 4, and 6 confirm AMIE was initially evaluated in text-based simulated consultations with performance comparable to primary care clinicians.
Source 5 states: 'expert-level performance has remained elusive for AI in this setting' (video consultations) until the current advance demonstrated by AMIE.
Ask AI about this story
// discussion
sign in to join the discussion