California Bans AI Pretending to Be Your Doctor Now

California’s AB 489 draws a hard line between human clinicians and AI-generated medical advice. Signed into law in late 2024, it’s the most significant state-level move yet on this issue. I’ve watched this space for a decade, so that’s not a statement I make lightly.

California isn’t acting alone, though. Texas, New York, and federal agencies are all racing to regulate AI in healthcare at the same time. So AI vendors, hospital systems, and telehealth platforms are staring down a patchwork of rules that gets messier every month. This guide breaks down what AB 489 actually changed, how other states compare, and what compliance looks like in practice, not just in theory.

How AB 489 Bans AI From Pretending to Be a Doctor

AB 489 targets a specific, very human problem. Patients often don’t know whether they’re talking to a person or a machine. That uncertainty has real consequences when the topic is their health.

The problem AB 489 was built to solve

The bill requires that any AI system communicating with patients in a clinical setting must clearly disclose its non-human nature upfront. It covers chatbots, virtual assistants, and AI-driven diagnostic tools used in healthcare. I’ve tested dozens of these tools, and the disclosure problem is more widespread than most people realize.

Picture a common scenario. A patient logs into a telehealth portal after hours, types in some symptoms, and gets a detailed, reassuring reply that reads exactly like something a physician would write. The tone is warm. The phrasing sounds clinical. Nowhere on the screen does it say “AI.” That patient might follow that guidance anyway — adjusting a medication dose, delaying an ER visit, or skipping a follow-up — based on something a licensed human never actually wrote. AB 489 exists precisely to prevent that moment of misplaced trust.

What the law actually requires

• AI systems have to identify themselves as artificial intelligence before any patient interaction begins.
• Disclosures need to be “clear, conspicuous, and understandable” to an average consumer.
• Healthcare providers can’t use AI to impersonate licensed professionals.
• Violations carry civil penalties and potential license review for healthcare entities.
• And patients keep the right to request a human provider at any point.

The law doesn’t ban AI from healthcare, and that distinction matters. It bans deception, not the technology itself. AI tools can still triage patients, suggest diagnoses, and support clinical decisions. They just can’t do it while pretending to be Dr. Smith from internal medicine.

AB 489 also covers both real-time and delayed communications. Chatbot conversations, automated email follow-ups, and AI-generated voice calls all fall under the disclosure requirement. That scope is deliberately broad, and honestly, it needs to be — an AI-generated voicemail reminding a patient to adjust an insulin dose carries the same obligation as a live chat. The medium doesn’t change the risk.

Enforcement matters here too. California Attorney General’s holds primary authority, but individual patients can also file complaints through existing consumer protection channels. Penalties scale with severity and frequency, so this isn’t just symbolic legislation.

How Other States Compare to AB 489 on Healthcare AI

California moved first, but other states are close behind. Each one is taking a slightly different approach to the same core problem, which is either encouraging or exhausting depending on your perspective.

Texas, New York, and the softer-touch states

Texas has folded its AI healthcare rules into existing medical practice acts. The Texas Medical Board now requires that AI-assisted diagnoses carry explicit labeling, but Texas doesn’t impose the same real-time disclosure requirement that AB 489 demands during patient-facing interactions. It’s a softer touch, with more paperwork and less friction at the point of care. A Texas patient might receive an AI-generated clinical summary in their portal without any real-time heads-up, as long as the record itself is labeled correctly. That’s a meaningful gap compared to California’s approach.

New York introduced its own healthcare AI transparency bills during the 2024-2025 session. Like California, New York emphasizes patient consent, but it goes further by requiring third-party audits for bias and accuracy in clinical AI systems. That audit requirement surprised me when I first read the proposal — it’s a real added burden for vendors. A startup deploying an AI triage tool in a New York hospital would need to budget for external auditors before going live, which is a very different cost structure than adding a disclosure banner.

Colorado’s SB 24-205 addresses AI discrimination broadly across sectors, including healthcare. It isn’t healthcare-specific, but its rules around “high-risk AI systems” capture most medical AI applications anyway.

The scale of the patchwork

The National Conference of State Legislatures tracks these developments across all fifty states, and at least seventeen states introduced healthcare-specific AI bills in 2024 alone. That number keeps climbing. California requires real-time disclosure with civil fines and license review. Texas requires labeling on records, enforced through Board sanctions, without a bias audit requirement. New York’s proposed rules add consent plus a bias audit requirement, with civil fines pending. Colorado requires disclosure for high-risk AI, backed by civil liability and a bias audit requirement starting in 2026. Illinois has limited disclosure requirements through its expanded AI Video Interview Act. Washington’s proposed HB 1951 would add disclosure and a bias audit requirement, still under review.

This is exactly why “fifty states, fifty AI laws” isn’t an exaggeration. Vendors building healthcare AI products need state-by-state compliance strategies, because a chatbot that’s perfectly legal in Texas might violate AB 489 without significant changes — and that’s a painful thing to discover after launch.

Where Federal FDA Rules Meet AB 489

While states pass their own rules, the federal government isn’t sitting idle either. The FDA has been expanding its oversight of AI-enabled medical devices for years. But the FDA’s framework and state laws like AB 489 address genuinely different concerns, and understanding that distinction is the real key here.

Two different questions, one compliance burden

The FDA asks whether an AI tool actually works safely and effectively. AB 489 asks whether the patient knows they’re talking to AI in the first place. These frameworks don’t conflict. They stack.

An AI diagnostic tool might need FDA clearance as a Software as a Medical Device and also comply with AB 489’s disclosure requirements. It might need to satisfy HIPAA’s data-handling rules on top of that. The compliance burden adds up fast. A mid-sized telehealth company deploying an AI symptom checker could be navigating FDA classification, AB 489 disclosure obligations, HIPAA’s minimum-necessary standard, and CMS billing rules all at once, if any AI-assisted service triggers a reimbursement claim. Each layer has its own paperwork, timeline, and enforcement body.

Federal touchpoints worth knowing include

• FDA premarket review for AI and machine-learning medical devices, with over 950 authorized as of early 2025;
• transparency requirements from the Office of the National Coordinator for certified health IT;
• CMS billing rules for AI-assisted services;
• and FTC enforcement against deceptive AI marketing in healthcare.

Federal preemption doesn’t apply here in most cases. The FDA hasn’t signaled any intent to override state transparency laws, so compliance with AB 489 stays necessary even for FDA-cleared devices. This dual-layer system adds real complexity, but it also creates stronger patient protections, which is ultimately the point. The Biden administration’s October 2023 Executive Order on AI Safety directed HHS to develop healthcare AI safety guidelines, and those guidelines reinforce many of the same transparency principles behind AB 489. For now, at least, the federal and state signals point in the same direction.

A Compliance Checklist for AB 489 and Beyond

Whether you’re building healthcare AI tools or deploying them in a clinical setting, compliance with AB 489 isn’t optional. I’ve talked to enough legal teams at health tech companies to know that “we’ll figure it out later” isn’t a strategy.

What vendors and developers need to do

• Set up clear, upfront AI disclosure in every patient-facing interface
• Add a persistent visual indicator, like a badge or banner, showing AI involvement
• Build a “request human” escalation path into every patient interaction flow
• Document your disclosure mechanism for regulatory review
• Test your disclosure language for readability at a sixth-grade reading level or below
• Maintain audit logs of all AI-patient interactions with timestamps
• Review your product against each state’s specific requirements before launch
• Check the American Medical Association’s AI policy guidance for clinical best practices along the way

On readability specifically: run your disclosure language through a free Flesch-Kincaid calculator before finalizing it. “This interaction is facilitated by an artificial intelligence system” clears the legal bar but fails the plain-language test. “You’re chatting with an AI, not a doctor” does both. AB 489 requires the former standard; your patients deserve the latter.

What healthcare providers need to do

• Audit every current AI tool for AB 489 compliance
• Update patient intake forms to include disclosure language
• Train staff on when and how AI tools interact with patients
• Set up a patient complaint process specifically for AI-related concerns
• Review vendor contracts for indemnification clauses covering transparency violations
• Monitor state legislative updates every quarter.

That vendor contract review deserves particular attention. Many health systems are running AI tools under contracts written before AB 489 existed, which means indemnification language almost certainly doesn’t address transparency violations at all. If a vendor’s chatbot generates a non-compliant interaction, you want clarity in writing about who bears the liability before a regulator asks the same question.

Vendors operating across multiple states should also build a compliance matrix, mapping each product feature against every applicable state law. This prevents the common mistake of assuming California compliance covers everywhere else — it doesn’t, and that assumption gets expensive fast. AB 489 defines “impersonation” one way; other states define it differently. Texas focuses more on documentation than real-time disclosure, while New York’s proposed rules would require pre-interaction written consent, going further than AB 489 in that specific respect. Colorado’s bias audit requirement adds yet another dimension entirely.

Penalties and Enforcement Under AB 489

Laws without teeth don’t change behavior. So does AB 489 have teeth? Mostly, yes.

What the fines actually look like

• A first violation carries civil penalties up to $2,500 per incident.
• Repeat violations climb to $7,500 per incident.
• Healthcare entities also face additional license review from the relevant medical board, plus class action exposure for systematic non-compliance.

Those numbers might look modest for a large health system, but the “per incident” language changes the math fast. A chatbot serving 10,000 patients without proper disclosure could generate millions in potential liability. A regional hospital system with 50,000 annual patient portal interactions could theoretically face $375 million in maximum exposure from a single misconfigured disclosure screen. No enforcement action will ever reach that ceiling in practice, but the number explains why general counsel at large health systems started paying attention the moment AB 489 passed.

Where enforcement stands right now

California’s Attorney General handles primary enforcement, with a maximum per-incident fine of $7,500 and a private right of action available to patients. Texas relies on Medical Board discretion instead, with limited private right of action. New York’s proposed framework would add the Attorney General plus the Health Department, a proposed $10,000 maximum fine, and its own private right of action. Federal enforcement through the FDA and FTC varies by device class and generally doesn’t include a private right of action, though criminal penalties are possible in fraud cases.

Enforcement under AB 489 is still in its early stages. No major cases have been publicly reported yet, but regulators are watching closely. California’s AG has signaled that AI transparency in healthcare is a priority area, which means the first high-profile case is probably a matter of when, not if.

One detail worth flagging: AB 489 applies regardless of intent. Even accidental non-disclosure, like a missing disclaimer caused by a software bug, can still trigger penalties. That strict liability approach means vendors can’t claim ignorance as a defense, which is a higher bar than most companies are used to.

How AB 489 Fits the Bigger 50-State AI Law Picture

AB 489 is one piece of a much larger regulatory puzzle. States are addressing AI across healthcare, employment, housing, and criminal justice, and healthcare is moving fastest because the stakes are highest.

Three challenges this creates for vendors

• Compliance fragmentation means no single product configuration satisfies every state at once.
• Update velocity means new bills pass monthly, requiring constant monitoring rather than a one-time review.
• And definitional inconsistency means states define “AI,” “healthcare,” and “disclosure” differently from each other.

That third challenge is subtler than it sounds. AB 489 uses a fairly broad definition of AI that covers machine learning models, rule-based chatbots, and AI-generated voice systems. Colorado’s SB 24-205, by contrast, focuses on “algorithmic decision-making” in ways that might exclude certain narrow automation tools. A vendor who assumes their product falls outside a state’s AI definition should get a second legal opinion before betting on that conclusion.

Building to the strictest standard

The EU’s AI Act classifies medical AI as “high-risk,” requiring conformity assessments before market entry, so companies selling globally face even more complexity on top of the US patchwork. For AI vendors, the practical strategy is building to the strictest standard available. If your product complies with AB 489, New York’s proposed rules, and the EU AI Act, you’ll likely satisfy less restrictive states automatically. This “comply to the ceiling” approach costs more upfront, in real engineering and legal investment, but it saves significant exposure down the line. I’ve seen companies try the cheaper path. It rarely stays cheap.

Some vendors instead geo-fence their products, deploying different configurations based on the user’s state. That works technically, but it creates maintenance headaches and audit complexity that compound over time. It also raises a question nobody’s fully answered yet: if a patient travels across state lines and accesses a telehealth platform configured for their home state, whose rules apply? Regulators haven’t weighed in definitively.

AB 489 has set a template other states are actively following. Treating it as the baseline, not the ceiling, is the smartest compliance strategy available right now.

Frequently Asked Questions About AB 489