Blog

<h3>What Two New Studies Tell Us About Where Clinical AI Fails — and How to Fix It</h3>Yesterday the Stanford AI Index came out and what its medicine chapter actually tells us: the tools winning in clinical settings aren’t the most sophisticated ones. They’re the ones designed around how clinicians actually work. Construct determines efficacy. Not capability.Two studies published this week, both peer-reviewed, both from leading academic medical centers, explain exactly why that’s true. One shows the failure at the reasoning level. The other shows the architectural fix. Together, they make the most compelling case I’ve seen for why human-centered design isn’t a nice-to-have in clinical AI. It’s the variable that determines whether these tools help or harm.The Failure FirstA team from Harvard Medical School and Mass General Brigham just published a landmark evaluation of 21 frontier LLMs — including GPT-5, Claude 4.5 Opus, Gemini 3.0, and Grok 4 — across the full arc of clinical reasoning. Differential diagnosis, diagnostic testing, final diagnosis, management, and clinical reasoning questions. Every model. Every stage.The finding that should stop every healthcare leader cold: Failure rates exceeded 80% on differential diagnosis across every single model tested — meaning fewer than 1 in 5 differentials were fully correct. GPT-5. Grok 4. Claude 4.5 Opus. All of them. The most capable AI systems ever built fail more than 8 times out of 10 at the task that sits at the beginning of every clinical encounter.At the same time, final diagnosis accuracy was relatively strong — often above 85%. The same models that can’t generate a differential diagnosis reliably can identify the correct final answer when given enough information.This looks paradoxical until you understand what’s actually happening. The authors explain it precisely: clinicians preserve uncertainty and iteratively refine differential diagnoses, whereas LLMs collapse prematurely onto single answers. As corresponding author Marc Succi, MD, stated:“Differential diagnoses are central to clinical reasoning and underlie the art of medicine that AI cannot currently replicate.”Clinicians know what patients feel here: uncertainty is the hardest part of waiting through test results for the diagnosis. The brain loops. Stress stays high. People want an answer — any answer — to make it stop. LLMs are great at giving answers. That's the problem. They give them too early, collapsing possibilities when a human would hold them open. That’s not a data problem. It’s not a benchmark problem. It’s not something that will be solved by the next model release. It’s a reasoning architecture problem that has persisted across every generation of models tested.One important nuance: reasoning-optimized models — GPT-o1, DeepSeek R1, Claude 4.5 Opus — significantly outperformed standard models overall. But even they failed differential diagnosis more than 70% of the time. Better, not safe. The gap narrowed; the problem didn’t go away.LLMs are extraordinarily good at pattern recognition against known information. They are structurally weak at holding multiple competing hypotheses open simultaneously while iterating toward a conclusion under uncertainty. That gap — between what they can do and what early clinical reasoning actually requires — is where the danger lives.The Architecture That Fixes It A second study, from the Windreich Department of AI and Human Health at Mount Sinai, tested a lightweight orchestrator that routes each task to a dedicated worker agent handling one specific job.The results were striking. As workload scaled, as it always does in real clinical environments, single-agent accuracy collapsed from 73% to 16.6%. Multi-agent accuracy held at 65.3% even at the highest load tested.The same models. The same technology. A 48-percentage-point accuracy gap at scale, determined entirely by how the work was structured.The mechanism matters. Each worker agent receives only the tokens relevant to a single decision. Attention isn’t diluted across irrelevant material. The effective context stays within the range the model was designed to handle.Co-author Mahmud Omar put it plainly: “When a single agent handles everything, you can’t trace where it went wrong. With the orchestrator, every step is logged — which tool was called, what it returned, and how the answer was assembled. At 80 simultaneous tasks, the single agent dropped to 16 percent accuracy while burning 65 times more compute — and you’d have no way to figure out why. That kind of transparency isn’t optional in medicine.”What These Two Studies Say Together The Harvard/MGH paper tells you where LLMs break. They break at the beginning — at the open-ended, uncertainty-laden, iterative reasoning that clinical diagnosis actually requires. They perform well when the task is bounded and the answer is deterministic. They fail when the task requires holding ambiguity and reasoning forward.The Mount Sinai paper tells you what to do about it. You don’t ask the AI to do the things it can’t do. You constrain the task. You partition the work. You route each decision to the right agent with the right scope. And you keep the human exactly where clinical judgment is required — not as a checkpoint at the end, but as the authority at every stage where uncertainty is real.To be precise: the Mount Sinai team didn’t test differential diagnosis — they tested bounded tasks like retrieval, extraction, and dosing calculations. Their orchestrator architecture hasn’t been proven to fix the 80% differential diagnosis failure rate. But their architectural logic — constrain scope, isolate context, make every step auditable — is exactly what the Harvard paper suggests as the only responsible way to deploy LLMs clinically today.This is why the ambient AI scribe works. It doesn’t diagnose. It listens and transcribes. Bounded task. Reviewable output. Clinician authority intact. This is why the sepsis prediction tool works. One input, one output, one threshold. The AI surfaces the alert. The clinician decides.And this is why general-purpose clinical reasoning deployments fail. They ask the AI to do the one thing the Harvard paper proves it cannot do — hold diagnostic uncertainty, generate meaningful differentials, reason iteratively toward a conclusion.There is now direct empirical evidence for what happens when you put the human back in. A separate 2025 study of hybrid human-AI collectives — where physician diagnoses and LLM outputs are combined — found that these hybrid groups outperform both human-only and AI-only groups on differential diagnosis across multiple medical specialties. The reason is precise: when LLMs fail, physicians often provide the correct diagnosis. The human isn’t decorative oversight. The human is an active accuracy contributor — not a supervisor, but a participant — at exactly the point where AI is structurally weakest.The Design Principle Underneath Both Papers Neither research team frames their findings as a design argument. But that’s what they are.The tools that work started with the human — with how clinicians actually move through a patient encounter, where their cognitive load peaks, what decision authority they need to retain, what burden they’d give anything to put down. The technology was shaped around that reality.The tools that fail started with the capability — with what the model can theoretically do, and the assumption that clinical value would follow from technical performance. It doesn’t.The Mount Sinai paper’s conclusion is a design specification: constrain the task, partition the work, keep the context bounded, make every step auditable. That’s not an AI architecture document. That’s a human-centered design document written in the language of LLM infrastructure.The Harvard paper’s conclusion is a warning and a prescription in the same sentence: their most responsible role today is targeted, clinician-supervised use in low-uncertainty tasks. The researchers who just tested every major frontier model are telling us these systems should only be deployed on tasks where uncertainty is low and a clinician is supervising.That’s a narrow runway. Most of what vendors are currently marketing doesn’t fit inside it.What This Means for Healthcare Leaders Right Now The question isn’t whether AI belongs in clinical settings. It does, and the evidence for specific, well-designed tools is now robust. The question is whether the tools you’re evaluating were designed with these constraints in mind or in spite of them. What is the specific task this tool is performing? The more bounded and deterministic, the stronger the evidence base. The more open-ended and uncertainty-laden, the more caution is warranted, regardless of benchmark performance.Where does the human retain decision authority? Not as a final reviewer, but as an active participant at every stage where clinical uncertainty is real. Tools designed for this produce 112% ROI. Tools that aren’t produce fewer than 1 in 5 correct differentials.Can every intermediate step be audited? The Mount Sinai architecture produces a logged, replayable audit trail by design. Most single-agent deployments don’t. In a regulatory environment where AI incidents rose 55% in a single year, that difference is not academic.The technology is extraordinary. The failures are predictable. The gap between them is design. Sources: <a data-mce-href="https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2847679?utm_campaign=articlePDF&amp;utm_medium=articlePDFlink&amp;utm_source=articlePDF&amp;utm_content=jamanetworkopen.2026.4003" href="https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2847679?utm_campaign=articlePDF&amp;utm_medium=articlePDFlink&amp;utm_source=articlePDF&amp;utm_content=jamanetworkopen.2026.4003" title="Large Language Model Performance and Clinical Reasoning Tasks" target="_blank" rel="noopener">Rao AS et al.</a> Large Language Model Performance and Clinical Reasoning Tasks. JAMA Network Open. 2026;9(4):e264003. April 13, 2026.<a data-mce-href="https://www.nature.com/articles/s44401-026-00077-0" href="https://www.nature.com/articles/s44401-026-00077-0" title="Orchestrated multi agents sustain accuracy under clinical-scale workloads compared to a single agent" target="_blank" rel="noopener">Klang E, Omar M et al</a>. Orchestrated Multi Agents Sustain Accuracy Under Clinical-Scale Workloads Compared to a Single Agent. npj Health Systems. 2026;3:23.<a data-mce-href="https://arxiv.org/abs/2406.14981" href="https://arxiv.org/abs/2406.14981" title="Human-AI collectives produce the most accurate differential diagnoses" target="_blank" rel="noopener">Zöller N et al.</a> Human-AI Collectives Produce the Most Accurate Differential Diagnoses. arXiv. 2406.14981. 2025.<a data-mce-href="https://hai.stanford.edu/ai-index/2026-ai-index-report" href="https://hai.stanford.edu/ai-index/2026-ai-index-report" title="Stanford HAI 2026 AI Report" target="_blank" rel="noopener">2026 AI Index Report</a>, Stanford Institute for Human-Centered Artificial Intelligence (HAI). April 13, 2026.

The Art of Medicine is Still Human

Stanford’s 2026 AI Index was published this morning. All 423 pages of it. And while much of the coverage over the next few days will focus on the U.S. - China model gap closing and the $285 billion in U.S. private investment, I want to talk about what the medicine chapter actually tells us — because it validates something I’ve been arguing for years, with data that most healthcare leaders haven’t seen yet.The tools winning in clinical settings are not the most sophisticated ones. They’re the ones built around how clinicians actually work.That distinction sounds simple. It isn’t. &nbsp; Three Tools. Documented Results at Scale. Stanford identifies three categories of clinical AI that moved from pilot to enterprise-scale in 2025 with measurable outcomes: ambient AI documentation, sepsis prediction, and generative AI embedded directly in EHR workflows. Ambient AI scribes saw the broadest adoption of any clinical AI category. Abridge expanded from approximately 100 to over 150 health systems in a single year, reaching 63% adoption among hospitals running on Epic. The outcomes were consistent across institutions in ways that are rare in clinical AI research:• &nbsp; &nbsp; &nbsp; Sharp HealthCare: 83% reduction in note-writing effort • &nbsp; &nbsp; &nbsp; University of Chicago Medicine: 47% reduction in cognitive load, &nbsp; &nbsp; 58% increase in undivided patient attention • &nbsp; &nbsp; &nbsp; MaineHealth: 23% reduction in note time, used in 70.3% of encounters • &nbsp; &nbsp; &nbsp; Northwestern Medicine: 11.3 additional patients seen per month, 24% reduction in documentation time, 112% return on investment • &nbsp; &nbsp; &nbsp; Stanford Health Care: Statistically significant reductions in burnout, median 20-minute savings per half day of clinic That kind of consistency across different institutions, different patient populations, and different market segments is not an accident. It’s a signal about construct. Sepsis prediction produced the most clinically striking outcomes. The TREWS system, deployed across 13 Cleveland Clinic hospitals, reported an 18.7% relative reduction in sepsis mortality and an 89% clinician adoption rate. UC San Diego’s COMPOSER system reported a 17% reduction in sepsis mortality across more than 6,000 admissions, with an estimated 50 lives saved annually. These aren’t small studies. These are enterprise deployments with documented mortality impact. &nbsp; The Same Technology. Radically Different Outcomes. Here’s where it gets important. The Stanford-Harvard ARISE Network reviewed over 500 clinical AI studies and found that nearly half relied on exam-style questions rather than real patient data. Only 5% used real clinical data. Separately, the NOHARM benchmark found that leading LLMs — the same underlying models powering many of these tools — produced between 11.8 and 14.6 severely harmful recommendations per 100 clinical cases, with 76.6% of those errors being errors of omission: failing to recommend a critical test or intervention. Same models. Catastrophically different results. Stanford is explicit about why: those harmful recommendation rates apply to general-purpose LLMs deployed on open-ended clinical reasoning tasks. The tools driving real adoption — ambient scribes and sepsis alerts — operate within constrained workflows with clinician oversight. The technology doesn’t change. The construct does. This is not a subtle point. It is the entire argument for how healthcare organizations should be evaluating AI investments right now. &nbsp; Construct Determines Efficacy. Not Capability. The tools that are winning share three design characteristics that have nothing to do with model sophistication: 1. They operate inside existing workflows, not alongside them. The ambient scribe doesn’t ask a clinician to change anything about how they conduct a patient visit. It listens. It drafts. The human reviews and signs. The workflow is preserved; the burden is removed. 2. They constrain the AI’s decision surface. Sepsis prediction doesn’t ask the AI to reason across all of medicine. It monitors a specific set of variables and surfaces an alert at a defined threshold. The AI does one thing. The clinician decides what to do about it. 3. The human retains decision authority at every step. This isn’t “human in the loop” in the checkbox sense. It’s the human as the through-line — initiating, conferring, implementing, refining. The AI doesn’t replace clinical judgment; it protects the time and cognitive space required to exercise it. When these three conditions are met, the results are consistent. When they aren’t — when AI is deployed broadly on complex reasoning tasks without constrained scope or human oversight — the results are dangerous. &nbsp; The Governance Gap Is Wider Than Most Leaders Realize The FDA authorized 258 AI-enabled medical devices in 2025. But a peer-reviewed analysis of all authorizations through December 2024 found that only 2.4% of devices with clinical studies were supported by randomized controlled trial data. The vast majority entered the market through pathways that rely on existing evidence rather than new clinical trials. Healthcare leaders are being asked to make AI deployment decisions with an almost entirely absent real-world evidence base. Most clinical AI research still doesn’t use real patient data. Most cleared devices haven’t been validated with rigorous trial methodology. Multi-agent diagnostic systems are beginning to show extraordinary benchmark results — Microsoft’s AI Diagnostic Orchestrator scored 85.5% on complex NEJM cases versus 20% for unaided physicians. But benchmark results are not clinical outcomes, and the gap between authorization and deployment readiness is still wide. &nbsp; What This Means for Your Organization The question I hear most often from health system leaders right now is some version of: “Where do we start?” The Stanford data gives a clear answer: start with tools that reduce burden on clinicians within workflows they already own. The evidence base is there. The ROI is documented. The adoption rates are high because clinicians don’t experience these tools as disruption — they experience them as relief. The tools that are failing — or more precisely, the tools producing dangerous results — are the ones deployed with misplaced ambition. Asking a general-purpose AI to reason across an entire patient case is a different design challenge than asking it to transcribe a conversation or alert on a specific physiological threshold. There’s one more finding worth sitting with. Stanford documents that patient trust in AI is not driven by the technology itself — it’s clinician-mediated. Provider endorsement is the key determinant of patient acceptance. This means that reducing burden on clinicians isn’t just an operational win. It directly produces patient trust in AI tools across the care experience. The human isn’t a checkpoint in this system. The human is the through-line that makes the whole thing work.Read Stanford's full report <a data-mce-href="https://hai.stanford.edu/ai-index/2026-ai-index-report" href="https://hai.stanford.edu/ai-index/2026-ai-index-report" title="Stanford AI Index Report" target="_blank" rel="noopener">here</a>.

The Stanford AI Index:  The Winning Tools in Healthcare

Discernment and Gratitude exists simultaneouslyThe Complaint and the Gratitude Are the Same Voice.We hear the complaint first — but that's not because the gratitude comes later. It's because we stopped listening.Both emotions are present at the same moment. The person flagging the inconsistent handoff and the person saying you saved my life are the same person, in the same sentence, feeling both things fully and at once. Complaint and gratitude are not a sequence. They are a duality. What we do with that determines everything.What's actually happening is sensemaking.Patients in a hospital are already in a vulnerable, often frightened state. They are watching — carefully. When the third physician enters and washes their hands differently, addresses the family differently, or doesn't return when promised, it registers. Not as a personal failing of that physician. As a pattern break. And pattern breaks in uncertain environments feel like signals of something worse.That heightened state doesn't produce complaint or gratitude. It produces both — simultaneously, at full intensity. The fear and the trust. The frustration and the relief. Holding two things at once is not contradiction. It is the very human experience of being cared for when something is at stake.The staff receiving that feedback often hears only one half of it.They gave years to this work. The complaint lands as not good enough — when the patient is rarely saying that at all. When we only receive the complaint, we miss the gratitude that was always attached to it. And when staff carry only that half, they carry a wound that was never the patient's intention to leave.Learning to hear both at once — to hold the duality rather than collapse it into a single signal — is one of the most underestimated skills in care delivery.Most of the time, it's not about the person. It's about the process.When a patient starts to complain, something predictable happens. Staff pull back. Rounding slows. The instinct is to create distance from someone seen as difficult — to wait it out, document it, route it to patient relations.But that patient is not difficult. They are available.A patient willing to voice a concern in the moment is offering something most patients never will — real-time, unfiltered experience before it hardens into a survey score or a formal grievance. They are not afraid to talk. They are actually inviting you in. The instinct should be to move toward them, not away. That conversation — the one most staff are quietly avoiding — is where the gratitude is also sitting, waiting to come out alongside the concern.Instead what patients report is that they see fewer staff. Except for someone appearing to "check on the complaint." Which they notice. And which confirms every fear they already had about whether they are being cared for or managed.Behavior and process are not the same thing. When we look closely, the source of friction is usually the handoff, the protocol gap, the inconsistency baked into how care is delivered — not the human executing it. Addressing the person without addressing the process doesn't hold. The duality in what patients express is often their way of pointing directly at that gap — praising the people, flagging the system.AI is amplifying this problem in real time.As automation enters clinical workflows, the pattern breaks multiply. Processes that were never clearly defined — never documented from the lived experience of staff and patients — are now being handed to tools that will execute them at scale. The inconsistency gets faster. The blame starts landing on the AI, when the root issue was always upstream.The fix isn't better AI. It's better design before the AI arrives.Understanding what is actually happening — from the staff doing the work, from the patients living the experience — before a single workflow is automated is what determines whether the tool helps or amplifies the harm. Co-creation isn't a values statement. It's a design requirement.Build the process right first. Then build the tool on top of it.Because the patients watching your new AI implementation are doing exactly what they've always done — holding gratitude and concern at the same time. The question is whether the system you built was designed to hear both.

The Duality of Patient Feedback

The April Effect: Why Your Satisfaction Scores Are About to Dip (And Why It's Not Your Fault)It's April 1st, and if you lead patient experience, employee engagement, or any function measured by human satisfaction — your scores are likely sliding. They'll bottom out this month, recover through summer, and dip again (less severely) in fall.This isn't about your service recovery. It's not about staffing. It's biology.Years ago, a reporter called me at NYU Langone asking if I was worried about summer satisfaction dips from new residents. I couldn't answer — and that question launched a research journey I'm still on. What I found: the dip isn't summer. It's spring. And it's not residents. It's neurochemistry.Press Ganey confirmed the pattern across thousands of hospitals. I've since validated it in systems across the US and Middle East. Then I found it in the Google Misery Index — consumer search volumes for sadness, depression, anxiety, and stress peak in the same window. Then employee engagement data. Then financial markets. Then legal decisions. Then movie sentiment.The pattern is everywhere humans rate their experience of anything.The mechanism? Serotonin transporter binding follows a seasonal cycle — it peaks in spring, creating a window of neurochemical vulnerability that affects how we perceive and report satisfaction. April isn't when care gets worse. It's when our ability to feel satisfied gets harder.What this means for you:Your patients need more warmth right now — not because something's broken, but because their biology is working against contentment. Your staff shouldn't carry the burden of "failing" a score that's fighting physics.It matters if you're running a hospital, physician practice, building an app or measuring anything humans rate.How does your data look? Is your dip over or just beginning?

The April Effect

<h3>What Healthcare Gained. What It Lost. Where We Are.</h3>I've spent thirty years inside healthcare — as a Chief Experience Officer at Yale New Haven Health, NYU Langone, and Humana, and now as a founder building AI workflow tools for the industry I know best.But healthcare found me long before I found it. My father's illness made me a patient family member first — a child watching a system do its best and still fall short.In all that time I've watched healthcare reinvent itself repeatedly. Each reinvention solved a real problem. Each one created a new one.What I’ve come to understand is that healthcare hasn’t evolved randomly. It has moved through distinct eras — each with a clear promise, a hidden cost, and a defining human experience.I call this The Evolution of Care Framework.<figure id="figure-1rspe1e5e8mmwuxdta" class=""><img id="1rspe1e5e8mmwuxdta" src="https://cdn.durable.co/blocks/20qgIDOW3qtrQSBgbyDnkpZDtQ6rldIImHWZM9pNsdHioVQE99vMRVk8kzDY8Wwi.png" alt="A framework for understanding how healthcare has evolved through six eras — and why the age of AI may be the most consequential shift yet." title="A framework for understanding how healthcare has evolved through six eras — and why the age of AI may be the most consequential shift yet." class="object-cover bg-gray-100 rounded-2xl md:rounded-3xl lg:rounded-4xl" style="aspect-ratio: 16/9; object-position: 10% 5%;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="10%" data-positiony="5%" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/20qgIDOW3qtrQSBgbyDnkpZDtQ6rldIImHWZM9pNsdHioVQE99vMRVk8kzDY8Wwi.png" data-mce-style="aspect-ratio: 16/9; object-position: 10% 5%;"></figure>Relational Care The era of the known patient &nbsp;— &nbsp;Pre-1900sOnce, healthcare was personal because it had to be. The doctor knew your name, your family, your story — not because of a philosophy, but because there was no other way. House calls. Bedside manner. A relationship built over years.But its virtue was also its limit. Relational care could not scale. It reached those within walking distance and left everyone else beyond the hill. The villain: No scale. &nbsp; Institutional Care The era of the treated body &nbsp;— &nbsp;1900s – 1960s The hospital became the cathedral of medicine. Expertise concentrated. Specialization multiplied — the heart here, the lungs there, the brain in its own wing. This era solved for scale. But the cost was fragmentation. The patient became a collection of organs, each assigned to a different expert. No one held the whole story. The body was treated. The person was lost. The villain: Fragmentation. &nbsp; Systematized Care The era of the process unit &nbsp;— &nbsp;1970s – 1990s If fragmentation was the problem, process was the answer. Protocols. Pathways. Managed care. The system learned to standardize, to measure, to optimize. Efficiency soared. Variation dropped. But somewhere in the machinery, the human moment got scheduled out. The patient became a process unit — a set of tasks to complete, boxes to check. The villain: Bureaucracy. &nbsp; Consumer Care The era of the empowered shopper &nbsp;— &nbsp;1990s – 2010s The system had become efficient but impersonal. So the pendulum swung. The patient would now be a consumer. Choice. Satisfaction scores. Hospital ratings. You could shop for your care like you shopped for a car. This era solved for access — for those with resources. But it turned health into a transaction. The relationship became exchange. And those who couldn’t shop were left at the curb. The villain: Transaction. &nbsp; Quantified Care The era of the data self &nbsp;— &nbsp;2010s – 2020s Before the next era could arrive, something had to be built: the data infrastructure. Wearables. Portals. Genomics. The patient began generating data constantly — steps, sleep, labs, clicks. This wasn’t yet a new model of care. It was the bridge. It solved for measurement. For the first time, we could see ourselves in numbers. But the numbers multiplied faster than sense could make of them. The villain: Data overload. &nbsp; Orchestrated Care The era of the passenger &nbsp;— &nbsp;2025 – present And so we arrive here. The Orchestrated Era solves for intelligence. Agentic AI coordinates across every surface — scheduling, triage, documentation, diagnosis. Ambient intelligence listens in the exam room. The system acts on your behalf, seamlessly, continuously, invisibly. It knows you better than you know yourself. It predicts what you need before you ask. It orchestrates everything. And that is precisely the danger. The villain of the Orchestrated Era is Displacement — the slow, seamless experience of being moved out of your own health story. The system doesn’t replace you. It just doesn’t need you. Your judgment, your preferences, your presence become optional. You are still in the vehicle. You are still present. You are still, technically, the patient. But you are no longer the driver. You are the Passenger. &nbsp; The Question That Remains Every era solved the problem before it. And every era created a new one. In the Relational Era, we had no scale — but we had a soul. In the Orchestrated Era, we have infinite intelligence — and no one at the wheel. The challenge of the next decade isn’t building a better AI. It’s building a seat for the human that the AI can’t displace. &nbsp;

The Evolution of Care Framework

Artificial intelligence is advancing at extraordinary speed. In diagnostics, drug discovery, knowledge synthesis, and customer support, AI systems are already outperforming human cognition in defined domains. But here is the uncomfortable truth: Cognitive superiority does not automatically translate into better outcomes. Healthcare has already taught us that. Intelligence ≠ Impact Physicians are, by training, more clinically knowledgeable than their patients. They diagnose faster, understand physiology at a systems level, and remain current on evolving guidelines. Yet decades of research reveal a paradox: superior knowledge alone does not guarantee superior results. What patients need is not just information. They need connection, trust, and emotional alignment. A landmark review in the British Journal of General Practice found that physician empathy is significantly associated with improved clinical outcomes across conditions ranging from diabetes to the common cold. Another study demonstrated that patients treated by more empathic physicians had measurably better control of chronic disease and higher medication adherence. Clinical accuracy matters. But relational resonance changes behavior. Being right does not change lives. Being understood does. If this is true in medicine—one of the most knowledge-intensive fields in the world—it should give us pause in AI development. If AI Learns Only to Be Smart, It Will Miss the Point Most AI systems today are optimized for precision, speed, and breadth of knowledge. These are necessary capabilities, particularly in high-stakes industries. But they are not sufficient. If we build AI that outperforms humans cognitively yet underperforms emotionally, we risk repeating one of healthcare’s deepest systemic failures: mistaking information for transformation. A system can generate flawless discharge instructions in seconds. But if the tone feels clinical and detached, if the language ignores literacy barriers, cultural context, or emotional overwhelm, the information may never convert into action. We have already seen this dynamic in human clinicians. Why would AI be immune to it? When intelligence scales without relational calibration, trust erodes at scale. And without trust, adoption stalls. Empathy Is Not Soft. It Is Structural. Empathy in AI is often dismissed as cosmetic—something that makes systems more pleasant but not more powerful. That framing is outdated. Empathy is not a personality trait. It is a design variable. It can be intentionally engineered through: · &nbsp; &nbsp; &nbsp; Context-aware tone modeling · &nbsp; &nbsp; &nbsp; Transparent acknowledgment of uncertainty · &nbsp; &nbsp; &nbsp; Adaptive pacing based on cognitive load · &nbsp; &nbsp; &nbsp; Culturally attuned language framing · &nbsp; &nbsp; &nbsp; Follow-up prompts that reinforce understanding These are not superficial enhancements. They are performance multipliers in any domain that depends on human behavior change—healthcare, finance, education, employee experience, consumer engagement. In these environments, emotional miscalibration does not merely feel awkward. It suppresses trust, adherence, and long-term engagement. The competitive edge will not belong to the system that answers fastest. It will belong to the system that understands best. Designing for Collaborative Intelligence The future of AI is not human versus machine. It is human and machine—intentionally designed to amplify one another. This requires moving beyond raw computational strength toward what can be called Collaborative Intelligence: the deliberate integration of cognitive performance and relational intelligence. Collaborative Intelligence operates across three dimensions: 1. &nbsp; &nbsp; Cognitive Strength — accuracy, reasoning, synthesis, predictive capability 2. &nbsp; &nbsp; Emotional Calibration — tone, empathy, contextual awareness, transparency 3. &nbsp; &nbsp; Behavioral Design — nudging, reinforcement, adaptive engagement, sustained follow-through When these three dimensions align, AI does more than inform. It influences. It supports. It builds trust. And trust is the currency of every human system. Lessons Healthcare Already Learned Patient experience research offers critical guidance for AI architects: Trust is built on transparency, not perfection. Clinicians who openly discuss uncertainty and invite questions often build stronger trust than those who project absolute certainty. AI systems that obscure limitations or overstate confidence risk alienating users. Context matters more than content. A discharge plan delivered with presence and reassurance often outperforms a perfectly formatted document. Similarly, AI that senses when to pause, clarify, or check understanding creates a fundamentally different experience than one that simply outputs an answer. Efficiency is not always the primary need. In moments of vulnerability, people do not necessarily want speed. They want acknowledgment. AI systems optimized exclusively for throughput may unintentionally undermine engagement. These insights extend far beyond medicine. They apply wherever technology interfaces with human uncertainty. Beyond Healthcare: The Broader Implication Whether designing AI for finance, education, retail, biotech, or enterprise operations, the principle holds: The most effective systems will not simply be intelligent. They will be relational. Experience design in the AI era is no longer just about usability or interface clarity. It is about anticipating emotional state, adapting to cognitive load, and responding to unspoken needs. As AI becomes more capable, its success will depend less on whether it can perform—and more on whether people feel safe engaging with it. The Choice Ahead We have an opportunity to define the next phase of AI development intentionally. Rather than focusing exclusively on replicating human expertise, we can design systems that amplify human understanding and strengthen trust. If intelligence alone were sufficient, healthcare would already deliver consistently optimal outcomes. The evidence suggests otherwise. For AI to move beyond performance metrics and genuinely transform results, empathy must be embedded as a structural element of design. In the coming decade, the systems that succeed will not simply be the most intelligent. They will be the ones that people trust enough to use. 1. &nbsp; &nbsp; Derksen, F., Bensing, J., &amp; Lagro-Janssen, A. (2013). Effectiveness of empathy in general practice: a systematic review. British Journal of General Practice, 63(606), e76–e84. 2. &nbsp; &nbsp; Hojat, M., Louis, D.Z., Markham, F.W., et al. (2011). Physicians’ empathy and clinical outcomes for diabetic patients. Academic Medicine, 86(3), 359–364. 3. &nbsp; &nbsp; Mazor, K.M., et al. (2004). Disclosure of medical errors: what factors influence how patients respond? Journal of General Internal Medicine, 19(8), 716–721.

Intelligence Isn’t Enough: What AI Must Learn from the Human Side of Healthcare

RoarHX

RoarHX helps organizations design the future of human-AI experience.
Founded by former healthcare Chief Experience Officer Dr. Joan Kelly, RoarHX provides frameworks, tools, and advisory services that help leaders identify where AI can transform workflows, improve patient and consumer experience, and unlock new operational value.
Through persona modeling, workflow mapping, and AI readiness assessments, RoarHX helps organizations move from AI curiosity to real-world implementation.

Insights

The Art of Medicine is Still Human

The Stanford AI Index: The Winning Tools in Healthcare

The Duality of Patient Feedback

The April Effect

The Evolution of Care Framework

Intelligence Isn’t Enough: What AI Must Learn from the Hu...