Artificial intelligence (AI) is changing health and health care at an unprecedented pace. The potential benefits are enormous—but so are the risks. The JAMA Summit on AI brought together experts across medicine, technology, policy, and ethics to discuss one urgent question: How should health care AI be developed, evaluated, regulated, and monitored to improve outcomes for all?
The Expanding Role of AI in Health and Health Care
AI now spans nearly every corner of the health ecosystem. It powers clinical tools like sepsis alerts or diabetic-retinopathy screening, consumer apps for wellness and diagnostics, operational systems that optimize scheduling and billing, and hybrid tools that combine both clinical and business functions.
Many of these tools already affect real patients, even when they fall outside formal regulatory oversight. Evaluating their impact is complex, because performance depends not only on the model itself but also on the human-computer interface, user training, and clinical context.
Although some standards for responsible AI use exist, most focus on safety monitoring or institutional compliance—not on whether AI actually improves health outcomes. The Summit concluded that to ensure equitable, effective deployment, progress must occur in four key areas:
- Multistakeholder engagement across the AI life cycle
- Development of evaluation and monitoring tools
- Creation of national data infrastructure and learning environments
- Alignment of incentives through policy and market forces
The Many Faces of Medical AI
The report organizes AI applications into four categories:
1. Clinical Tools
These are used directly by clinicians for diagnosis and treatment. Examples include autonomous screening for diabetic retinopathy, AI-assisted echocardiography, and EHR-based sepsis alerts.
More than 1,200 such tools have FDA clearance—most in medical imaging, where AI adoption exceeds 90 % among U.S. systems. Yet outside imaging, uptake is slower due to training costs, infrastructure demands, and concerns about bias, trust, and uncertain reimbursement.
2. Direct-to-Consumer Tools
Apps and wearables put AI directly in patients’ hands, from mental-health chatbots to arrhythmia detection on smartwatches. With over 350,000 health apps worldwide, usage is high, but regulatory oversight is light. Many are marketed as “general wellness” products and rarely share data with clinicians.
3. Business-Operations Tools
Hospitals deploy AI to improve efficiency—for example, bed management, staffing, supply chains, and revenue cycle management. Yet these systems can indirectly shape patient care. If an algorithm optimizes surgical schedules, it could improve access—or unintentionally delay critical operations. Such tools often escape rigorous evaluation.
4. Hybrid Tools
Some AI systems bridge the clinical and operational realms. AI scribes, for instance, record patient-clinician conversations, generate documentation, and suggest diagnoses or treatments. Adoption is accelerating, and early studies show high clinician and patient satisfaction—though evidence of direct health benefits remains limited.
Why Evaluating AI Is So Hard
The Summit highlighted several barriers to robust evaluation:
- Defining the intervention and context. Effectiveness depends on how an AI tool is deployed, trained, and integrated into workflow.
- Identifying mechanisms of action. AI’s inner workings are often opaque (“black boxes”), making interpretability and explainability essential yet difficult.
- Capturing relevant outcomes. Measuring real-world impact—on health, safety, equity, and efficiency—is time-consuming and expensive.
- Inferring causality. Randomized trials are slow and costly, while observational studies may lack precision. Health care rarely adopts rapid A/B testing, unlike other industries.
The Summit stressed that rigorous, transferable knowledge requires a mix of randomized, quasi-experimental, and observational approaches—supported by adequate data infrastructure.
Regulation: A Patchwork Framework
No single comprehensive regulatory structure exists for health care AI in the U.S.
- The FDA oversees AI tools defined as medical devices, using a risk-based approach. Yet many AI systems—such as scheduling or wellness software—are explicitly excluded from this definition.
- The Office of the National Coordinator for Health Information Technology (ONC) offers voluntary certification for EHR-related tools.
- The Centers for Medicare & Medicaid Services (CMS) regulates certain laboratory and billing applications.
- The Federal Trade Commission (FTC) governs DTC products, focusing on privacy and advertising.
This fragmented landscape leaves major gaps, especially for tools that straddle categories. The Summit noted that emerging generative and agentic AI—capable of autonomous decision-making—may soon challenge traditional device definitions entirely.
Implementation and Monitoring Challenges
Even after deployment, monitoring AI tools remains difficult. Effectiveness varies with training quality, context, and user adoption. Health systems often lack the expertise and infrastructure to validate ongoing performance.
Without robust evaluation, organizations cannot know when a tool remains safe—or when to de-adopt it. Meanwhile, DTC products raise separate risks around trust, fairness, and privacy.
Former FDA Commissioner Robert Califf summed it up bluntly: “I have looked far and wide, and I do not believe there’s a single health system in the United States capable of validating an AI algorithm in clinical care.”
Four Strategic Solutions
The JAMA Summit proposed an action plan built around four complementary strategies:
1. Engage Stakeholders Throughout the Life Cycle
AI requires holistic, continuous collaboration among developers, clinicians, regulators, and patients—from design to monitoring. This approach fosters transparency, safety, and adaptability.
2. Develop and Implement Better Evaluation Tools
Existing guidelines—like CONSORT-AI, SPIRIT-AI, TRIPOD-AI, and APPRAISE-AI—can standardize reporting and oversight. The goal is “algorithmovigilance”: continuous surveillance of AI safety and effectiveness, similar to pharmacovigilance for drugs.
3. Build Data Infrastructure and a National Learning Environment
The U.S. needs representative, interoperable, and continuously updated data systems linking AI use to patient outcomes. Federated learning models, modeled after the FDA Sentinel Program, could enable real-time monitoring across health systems while protecting privacy.
4. Create Aligned Incentives
Adoption, evaluation, and responsible use require financial and policy support. Market forces alone may not guarantee fairness or safety. Federal initiatives—similar to the HITECH Act that spurred EHR adoption—could accelerate responsible AI integration and close resource gaps in underserved settings.
Implications for the Health Care Workforce
AI’s integration will reshape clinical roles and expectations.
- Task redistribution: AI-assisted diagnostics may shift responsibilities among clinicians.
- New skills: Providers need AI literacy, not to code algorithms, but to understand their strengths, biases, and limits.
- Equity concerns: Early AI access often favors wealthier institutions; deliberate design is needed to prevent widening disparities.
- Workload and burnout: While AI aims to reduce administrative burden, it may also introduce new demands or expectations for productivity.
- Continuous learning: Clinicians will participate in an ongoing feedback loop, helping AI evolve safely in real time.
Ethics, Privacy, and Liability
The Summit also examined emerging ethical and legal dilemmas:
- Data rights and ownership: Health data power AI systems, but who truly owns or controls that data remains unclear.
- Ethical oversight: Should AI deployments be treated as quality improvement or human-subjects research? The answer determines what protections apply.
- Liability: If an AI-assisted decision causes harm, who is responsible—the clinician, the health system, or the developer?
As AI becomes standard, failing to use proven tools might itself become a liability risk, marking a profound shift in medical accountability.
The Takeaway
Artificial intelligence will disrupt every aspect of health care—access, cost, quality, and workforce. That disruption represents both an incredible opportunity and a major challenge.
The JAMA Summit makes one thing clear: success depends on building an ecosystem capable of rapid, efficient, and robust learning about how AI truly affects health. Only then can we ensure the technology fulfills its promise—to improve outcomes safely, fairly, and for everyone.
Check out the cool NewsWade YouTube video about this article!
Citation:
Angus, D. C., Khera, R., Lieu, T., et al. (2025). AI, Health, and Health Care Today and Tomorrow: The JAMA Summit Report on Artificial Intelligence. JAMA. Published online October 13, 2025. doi:10.1001/jama.2025.18490
