Shadow AI Is Now a Verification Problem

What Happened at Weill Cornell Medicine

On November 5, Weill Cornell Medicine posted a patient notice about a privacy incident. The organization learned on September 6 that patient information from a radiology research study had been entered into AI tools and processed using machine learning, without authorization and in violation of its policies. The entry occurred between June 10 and June 20, 2026, using data from Weill Cornell Imaging at NewYork-Presbyterian or NewYork-Presbyterian Hospital.

The information involved medical record numbers, radiology accession numbers, dates and times of service, facility and provider names, and clinical details including diagnoses, treatment histories, procedural records, and medications. It did not include names, addresses, Social Security numbers, or financial information.

Weill Cornell reports no evidence that an unauthorized individual accessed the information or that it was misused. The organization suspended the research activity, requested deletion of the data from the AI tools involved, and provided staff with additional education on AI use policies.

This Is a Pattern, Not an Outlier

The Weill Cornell incident fits a larger documented pattern. In July 2026, Paubox surveyed more than 150 healthcare leaders at organizations that handle PHI (protected health information) electronically and use AI coding assistants. The published findings describe an environment where controls frequently lag behind actual tool use.

Seventy percent of respondents found an AI-built tool that had handled patient data without IT or security knowing. Ninety-three percent say real PHI reaches AI coding assistants at least some of the time.

Paubox notes that the 87% who say a security or compliance professional reviews AI code before go-live are only capturing tools that someone submitted for review. Tools that bypass submission bypass the review entirely.

The Policy Gap Behind the Data

Only half of surveyed organizations have a fully implemented written policy on AI coding assistants. Fifty-one percent say AI-generated or AI-assisted code has already contributed to PHI exposure. Fifty-four percent say it has contributed to hardcoded credentials or application programming interface (API) keys embedded in production code.

The gap between policy intent and actual tool inventory is the core problem. An organization can maintain a strong written policy while still running AI-built tools in production that no one registered, reviewed, or authorized. Paubox recommends starting governance with an inventory of every AI-built tool, whether it touches PHI, and who owns it.

The Rules Are Moving to the States

The regulatory backdrop is shifting quickly. Manatt Health's Q3 Health AI Tracker, published October 27, counts more than 280 state AI bills introduced this year, with 40 enacted across 24 states. Healthcare organizations operating across state lines now face a patchwork of requirements that vary by jurisdiction and change on separate timelines.

Federal guidance from agencies including the Department of Health and Human Services (HHS) continues to evolve alongside state action. Organizations without a systematic way to track state-specific AI rules are building compliance obligations that will need to be addressed retroactively.

Why This Is a Verification Problem

A policy describes the expectation. Evidence shows whether it happened.

When an unapproved tool touches patient data, compliance officers, legal teams, and regulators need to answer the same questions: what tool was used, who authorized it, under what documented scope it was operating, and what records survive. If those answers are not available, the policy did not translate into verifiable control.

CareLumi is The Verification Layer for Healthcare AI. The platform is designed to give healthcare organizations a continuous, documented record of which AI tools are operating in their environment and whether each is running under verified, current authorization. Third-party review timelines, approvals, and regulatory decisions remain outside the platform's control.

What to Put in Place Now

The steps below apply whether or not an incident has occurred. Acting before an incident reduces exposure. Acting after one creates the documentation needed to demonstrate remediation.

1. Build an AI inventory. Include tools staff built or deployed independently, and assign an accountable owner to each one.
2. Enforce data handling at the tool level. Use synthetic data for development and testing. Do not rely on training or policy acknowledgment alone.
3. Give each AI agent a defined identity and a narrow, documented scope of operation, along with a kill switch and a complete activity log.
4. Keep a human in the review workflow. Track what automated review misses, including false negatives and cases outside the tool's training distribution.
5. Map your state-level exposure. Track state AI legislation on a rolling basis, jurisdiction by jurisdiction, against your operational footprint.
6. Prepare an incident playbook before you need one. Assign a single owner, define what proof you can produce quickly, and test the response.