A lot of health AI isn’t where you think it is, and it’s not overseen the way you might expect

June 13, 2026 - 04:06

A growing number of artificial intelligence tools are quietly reshaping American healthcare, but not in the ways most patients assume. Many of these systems operate outside the traditional oversight of the Food and Drug Administration, and the same algorithm can fall under different regulatory bodies depending on where it is deployed.

Maya Sandalow, a policy expert tracking AI governance, explained the fragmented landscape. "The same AI may be regulated by different entities depending on where it is used, the data that it's trained on," she said. This means a diagnostic tool used in a hospital might face FDA scrutiny, while a similar model used by a health insurer to process claims could be governed by state insurance commissioners or the Department of Health and Human Services.

The result is a patchwork of rules that leaves many AI applications effectively unchecked. For example, algorithms that help doctors read medical scans are often classified as medical devices and must pass FDA review. But AI that schedules patient appointments, predicts hospital readmission rates, or flags billing errors typically falls outside that framework. These tools are considered administrative or operational, even though their decisions can directly affect patient care and costs.

Critics argue this creates dangerous gaps. An AI that recommends treatment plans might be regulated as a device in one clinic but treated as a simple software tool in another. Without consistent standards, there is little transparency about how these models are tested, what data they were trained on, or how often they make mistakes.

As hospitals and insurers race to adopt AI, experts like Sandalow urge a closer look at the rules. "Just because a tool is called 'health AI' doesn't mean a doctor or a federal agency has vetted it," she warned. The technology is already here, but the oversight is still catching up.