HIPAA Compliant AI: 3 Controls Clinicians and Developers Must Verify

HIPAA Compliant AI: 3 Controls Clinicians and Developers Must Verify

Isometric illustration of AI compliance controls

HIPAA compliant AI is achievable, but only inside a program built on three levers working together. You need a signed Business Associate Agreement with every vendor touching protected health information, technical safeguards like encryption and access controls that HHS expects under the Security Rule, and a data strategy that strips or minimizes PHI before it ever reaches a model. Skip any one lever and the other two don’t matter. MedScrub and similar tools built around on-device processing illustrate practical approaches to compliance.

TL;DR:

  • HIPAA compliance for AI requires a signed Business Associate Agreement, technical safeguards like encryption, and a data strategy to minimize PHI exposure.
  • On-device processing and de-identification significantly reduce legal and technical risks, but all three compliance levers must be used together.
  • Auditors verify configurations such as encryption, access control, and audit logs, with continuous monitoring essential to catch model drift and maintain compliance.
  • De-identification methods like Safe Harbor or Expert Determination have trade-offs between speed, data utility, and re-identification risk, requiring ongoing validation.
  • Prioritize governance, risk assessments, and testing with de-identified data before deploying real patient PHI into AI systems, and verify vendor compliance documentation upfront.

Table of Contents

What HIPAA Actually Requires From AI Technology

The HIPAA Security Rule doesn’t name AI, chatbots, or large language models anywhere in its text. It requires “reasonable and appropriate” administrative, physical, and technical safeguards for electronic PHI, and it’s deliberately technology neutral so it scales from a two-provider clinic to a hospital network running dozens of models. That flexibility is a gift and a trap: it means you’re responsible for translating a decades-old regulation into controls that make sense for a transformer model, and nobody at HHS is going to hand you a compliance checklist tailored to GPT-style inference.

Electronic PHI (ePHI) covers far more than diagnosis codes. HIPAA lists 18 identifiers that, combined with health information, make data protected: names, dates tied to an individual, phone numbers, medical record numbers, biometric identifiers, even IP addresses in some contexts. For an AI pipeline, that means:

  • A transcribed clinical note fed to a summarization model carries ePHI the moment it includes a patient’s name or date of birth.
  • A de-identified dataset that still contains a rare zip code combined with an age over 89 can be re-identifiable and therefore still regulated.
  • Chat logs, embeddings, and cached prompts can all count as ePHI if they retain any of the 18 identifiers.

Any vendor that creates, receives, maintains, or transmits PHI on your behalf, including an AI company processing patient notes, is a business associate under federal law. That single distinction, covered entity versus business associate, decides whether you can legally send that vendor a single patient record.

A signed BAA isn’t paperwork you get around to eventually. It’s the legal precondition for sending PHI to any AI vendor, and HHS treats a missing BAA as a reportable violation regardless of how good your encryption is. HHS telehealth guidance makes the same point for remote care technology: the vendor has to enter a BAA and meet HIPAA’s rules before it touches PHI, full stop.

Before you sign, verify these clauses actually protect you:

  1. Data flow mapping. The BAA should specify exactly where PHI goes, including any subprocessors or cloud infrastructure the vendor uses.
  2. Subcontractor flow-down. If your AI vendor uses a third-party model provider, that provider needs its own BAA obligations flowed down, not left as a gap.
  3. Incident response timelines. The agreement should commit the vendor to notify you within a specific window, ideally under 72 hours, when something goes wrong.
  4. Retention and deletion terms. Standard consumer AI retention (often 30 days or indefinite for abuse monitoring) usually violates your minimum-necessary obligations unless it’s explicitly modified.
  5. No resale or secondary use clause. PHI used to improve a vendor’s general-purpose model without your authorization is a contract violation waiting to surface.

Governance doesn’t end at signature. Review the vendor’s security posture annually, confirm they carry cyber liability insurance, ask for independent audit or penetration test summaries, and require advance notice before they change subprocessors or model providers.

Pro Tip: Ask AI vendors for their BAA template before you ask about pricing. A vendor that can’t produce a real BAA on request, or tries to substitute a generic terms-of-service page, has already told you everything you need to know.

Technical Safeguards That Auditors Actually Check

Legal agreements set expectations. Technical controls prove you met them. When an auditor or a breach investigator looks at your AI deployment, they’re checking specific, verifiable configurations, not intentions.

Encryption comes first. Data at rest needs AES-256 or equivalent, and data in transit needs TLS 1.2 or, preferably, TLS 1.3. Key management matters as much as the algorithm: keys belong in a hardware security module or a managed key vault, never hardcoded in application code or checked into a repository.

Access control follows the principle of least privilege. Every user needs a unique ID, multi-factor authentication should be mandatory for anyone touching PHI-adjacent systems, and role-based access control should limit what a given login can see. Session timeouts and automatic log-off aren’t optional extras; they’re baseline expectations under the Security Rule.

Audit logging needs to be tamper-evident, feeding into a SIEM system where possible, with retention policies long enough to support an investigation months after an incident. If your AI tool can’t tell you who queried which patient record and when, you have a logging gap that will surface at the worst possible moment.

  • Confidential computing and isolated VPCs or private endpoints keep model inference away from shared, multi-tenant infrastructure.
  • Third-party penetration testing and vulnerability scans should happen on a recurring schedule, not once at launch.

Trust follows directly from these controls. Surveys on clinician trust in healthcare AI show adoption tracks closely with governance and documented privacy assurance, not just model accuracy. Clinicians who can see the audit trail are the ones who actually use the tool.

De-Identification, Redaction, and Synthetic Data Strategies

The safest PHI is PHI that was never sent to the model in the first place. Two HIPAA-recognized de-identification paths exist, and they suit different situations.

Safe Harbor removes all 18 specific identifiers, a mechanical, checklist-driven process that’s fast to implement but can strip clinically useful detail (exact dates, granular geography) that a model might need for accuracy. Expert Determination uses statistical methods to certify that re-identification risk is very small, preserving more data utility but requiring a qualified statistician and ongoing revalidation as data changes.

Automated detection tools do the heavy lifting at scale. Named entity recognition (NER) models trained to spot names, dates, and medical record numbers, paired with regex patterns for structured identifiers like phone numbers, catch most PHI automatically. Neither approach is complete on its own. Human review remains necessary because NER models miss context-dependent identifiers, particularly rare conditions or unusual demographic combinations that indirectly identify someone.

  • Synthetic data trains models on statistically representative but fabricated records, useful for early development, though poorly generated synthetic data can still leak patterns from the source data.
  • Differential privacy adds calibrated noise to protect individuals in aggregate datasets.
  • Tokenization replaces identifiers with reversible tokens, useful when you need to re-link results back to a real patient later, but only if the token vault itself is secured as tightly as the PHI it stands for.

Every one of these carries residual re-identification risk. Treat de-identification as risk reduction, not risk elimination, and design your default: send the model the minimum necessary or a de-identified proxy, and reserve raw PHI for cases where nothing else will do.

Pro Tip: Run a re-identification test on your own de-identified dataset before a regulator or researcher does it for you. Combining a handful of “harmless” fields is often enough to single out a patient in a small population.

Choosing Where the AI Actually Runs

Architecture decides how much contractual and technical risk you’re carrying, and the four common patterns trade off differently.

  1. On-device or local inference. Processing happens on the clinician’s own machine or a local server, so PHI never crosses the network to a third party. This shrinks your BAA surface dramatically because there’s often no external vendor receiving raw PHI at all, only reversible tokens or de-identified proxies.
  2. Private cloud with a signed BAA. Runs inference in a vendor’s HIPAA-eligible environment behind private endpoints, inside your own VPC. You still need to confirm the specific product tier is covered, not just the vendor’s brand name generally.
  3. Federated learning and confidential computing. Multiple institutions train a shared model without any single party centralizing raw PHI, useful for research consortiums that need broader data diversity without a central data lake.
  4. HIPAA-eligible public cloud AI services. Requires verifying the exact service, correct retention configuration, and administrative settings, since not every feature a cloud AI vendor offers is automatically covered by the BAA you signed.

That last point trips up more teams than anything else. A vendor’s marketing page says “HIPAA compliant.” Their actual BAA coverage might apply to three of their twelve product tiers, and the wrong toggle in an admin console can quietly put PHI into a non-covered feature.

The Vendor Evaluation Checklist Before You Sign Anything

Run through these in order, because sequence matters. Skipping ahead to a pilot before the paperwork is signed is the single most common compliance failure teams make with AI vendors.

  1. Procurement first. Get the signed BAA before a single record moves. Verify the vendor’s cyber liability insurance and request their most recent SOC 2 report or, for cloud infrastructure, FedRAMP authorization status.
  2. Technical validation. Run integration tests in a staging environment first. Confirm retention settings are actually modified from vendor defaults, and commission a penetration test or vulnerability scan if the vendor hasn’t recently completed one.
  3. Data handling verification. Test the de-identification pipeline against known edge cases, quantify re-identification risk where you can, and confirm every access to PHI is logged and attributable to a specific user.
  4. Operational readiness. Train the workforce on the new tool’s specific risks, run an incident response tabletop exercise before go-live, and document your standard operating procedures so an auditor sees a program, not improvisation.

The Security Risk Assessment Tool from HealthIT.gov is a solid starting point for the risk analysis HIPAA requires before any of this, and it’s free.

Pro Tip: Schedule your first post-launch audit for 90 days out, not a year. Early problems in an AI rollout tend to surface in workflow gaps and access-control misconfigurations long before they show up in a formal annual review.

MedScrub in Practice: Keeping PHI Local While Automating Documentation

MedScrub’s architecture maps directly onto the controls above rather than treating them as an afterthought. It syncs with EMR systems including Epic and Oracle Health, then runs anonymization on the clinician’s own device before any data leaves that machine. Developers get a reversibly de-identified FHIR proxy instead of raw PHI, so building and testing a custom assistant never requires touching a real patient record.

That design choice does real work against the BAA and technical requirements covered earlier:

  • On-device processing shrinks the contractual surface, since PHI often never reaches an external server that would trigger business-associate obligations in the first place.
  • Reversible tokenization lets a clinician re-link a summary back to the real chart without storing raw identifiers in transit.
  • Nightly chart preparation and automated care-gap tracking, the kind of workflow highlighted in MedScrub’s case studies, show that compliance and clinical usefulness aren’t a trade-off. A tool that flags an unresolved treatment plan at 6 AM is doing compliance and clinical work simultaneously.

The eSpiral integration is the clearest example: AI embedded directly in the chart, with PHI never leaving the device it’s viewed on. That’s the architecture pattern discussed above, on-device inference, applied to a real clinical setting rather than a whiteboard diagram.

What the Privacy Rule Adds on Top of Security Controls

The Security Rule governs how ePHI is protected technically. The Privacy Rule governs what you’re allowed to do with it in the first place, and AI applications get tangled in both simultaneously.

Under the Privacy Rule, PHI can be used for treatment, payment, and healthcare operations without new patient authorization. An AI tool summarizing a chart for the treating physician generally falls inside that scope. But feeding patient records into a model to build a general-purpose commercial product, or using de-identified data derived from PHI to train a model you plan to sell, drifts toward secondary use that can require separate authorization or a data use agreement.

The line gets blurrier with generative AI specifically. A model that generates a discharge summary is operating within treatment. A model that retains prompts to improve a vendor’s product across all customers is arguably repurposing PHI for something the patient never agreed to. Ask every AI vendor directly: does our data ever train your general model, or does it stay isolated to our environment? The answer should be in writing, not verbal reassurance from a sales call.

Minimum necessary applies here too. If an AI summarization tool only needs the last three visit notes to generate a follow-up reminder, feeding it the patient’s entire fifteen-year chart history violates the Privacy Rule’s minimum necessary standard even if every technical safeguard is airtight.

Breach Notification When an AI System Is the Point of Failure

An AI incident is still a breach if it exposes unsecured PHI, and the notification clock doesn’t care whether a human or a model caused the exposure. HIPAA’s breach notification rule requires covered entities to notify affected individuals without unreasonable delay once an incident is identified.

AI introduces failure modes that didn’t exist in older systems. A model that hallucinates a patient detail into a document isn’t a breach by itself, but a misconfigured logging pipeline that stores raw PHI in a support ticket, or a vector database that inadvertently indexes patient names into an unsecured search index, absolutely is. Prompt injection attacks that trick a chatbot into revealing another patient’s information out of context are a newer risk category entirely, one most legacy breach playbooks were never written to cover.

Your incident response plan needs an AI-specific branch: who checks model logs, who verifies whether a vendor’s subprocessor was involved, and how quickly you can determine the scope of exposure when the “system” in question is a black-box model rather than a database with clear table boundaries. Vendors are contractually obligated to notify you promptly under the BAA’s incident response clause, but you’re still the one who has to notify affected patients and, depending on the scale, the HHS Office for Civil Rights and potentially the media.

Test this plan before you need it. A tabletop exercise that specifically simulates an AI vendor incident, not just a stolen laptop scenario, surfaces gaps that a written policy alone won’t catch.

Breach Notification When an AI System Is the Point of Failure — overview diagram

Building Continuous Monitoring Into an AI Program

A one-time risk assessment satisfies an auditor for a moment. It doesn’t protect you six months later when the vendor updates their model, changes a subprocessor, or quietly expands what data their product retains by default.

AI-specific risk management needs to be ongoing, not episodic. Model updates from a vendor can silently change behavior, a model that previously refused to output PHI in a response might behave differently after a version bump nobody told you about. Build a review cadence: reassess vendor configurations quarterly, not annually, and require vendors to notify you before any material change to their model, retention policy, or subprocessor list.

Monitoring itself should be continuous rather than a checkbox. That means real-time anomaly detection on access logs (unusual query volume from one account is a common early breach signal), regular re-testing of de-identification pipelines as your patient population and data fields evolve, and periodic re-validation that BAA-covered configurations haven’t drifted, especially after a cloud vendor’s product update.

Treat model drift as a compliance issue, not just a performance one. A summarization model that starts including more granular dates or geographic detail than it used to, because of a retraining update on the vendor’s side, can silently push previously de-identified outputs back toward identifiability. Nobody flags this in a standard uptime dashboard. It requires someone on your team actually looking at sample outputs on a schedule.

Building Continuous Monitoring Into an AI Program — overview diagram

Most AI-assisted documentation and chart preparation doesn’t require new patient consent because it falls under treatment operations already covered by existing HIPAA authorizations and standard notice-of-privacy-practices disclosures. The complication arrives when the AI’s use case moves beyond direct care.

Research use, model training for commercial products, and any secondary analysis that goes beyond the original purpose of collection generally need either de-identification robust enough to fall outside HIPAA entirely, or a specific authorization from the patient. Many patients don’t realize their chart data might be touched by an AI tool at all, which is a transparency gap more than a legal one, but it’s becoming a real institutional risk as patients grow more aware of how their health data gets used.

Practical steps matter more than legal theory here. Update your notice of privacy practices to describe AI-assisted documentation in plain language. Give patients a real answer when they ask whether an AI tool “read” their chart. And draw a bright internal line between AI used for direct patient care, which needs no new consent, and AI used for research, model improvement, or any purpose beyond the original visit, which usually does.

Where the Regulatory Ground Is Shifting

HIPAA hasn’t been substantially rewritten for the AI era, and that gap is exactly why more scrutiny is coming. State-level AI and health data laws are expanding faster than federal rulemaking, and some states now impose stricter consent or disclosure requirements for AI-assisted clinical tools than HIPAA itself does, meaning multi-state operators increasingly need to comply with the strictest applicable state rule, not just the federal floor.

Federal agencies are also paying closer attention to algorithmic transparency and bias, separate from privacy but increasingly bundled into the same compliance conversations vendors have with health systems. Expect procurement questionnaires to keep expanding: it’s no longer enough to prove a model doesn’t leak PHI, buyers increasingly want documentation on how a model was trained, what data went into it, and how its outputs are validated for clinical safety.

The practical takeaway for teams building or buying AI right now: build your program assuming the bar rises, not stays flat. A vendor with clean documentation, granular consent controls, and an architecture that keeps PHI local rather than centralized will need less retrofitting than one relying purely on today’s minimum BAA language.

What I’d Prioritize First If I Were Building This Program

Start with governance before a single model runs on real data. The teams that get burned aren’t the ones with weak encryption. They’re the ones who piloted a promising tool on real patient charts before anyone checked whether a BAA existed. Run your first pilot on synthetic or heavily de-identified data, expand from there, and treat measurable outcomes, real clinician time saved, fewer missed follow-ups, as your actual acceptance criteria instead of a demo that looked impressive in a conference room.

— Clint

Put These Controls to Work Without Building Them Yourself

Everything covered above, on-device de-identification, EMR-synced workflows, audit-ready logging, is exactly what MedScrub builds around rather than bolts on afterward. Instead of assembling a NER redaction pipeline, a separate encryption layer, and a homegrown chart-summary tool, you get a system where PHI stays on your machine by default and the EMR sync (Epic, Oracle Health, athenahealth, eClinicalWorks) does the integration work most in-house builds spend months on.

Medscrub

Clinicians typically reclaim a significant amount of time previously lost to documentation, freeing that time for actual patient care instead of chart cleanup. Developers get a reversibly de-identified FHIR proxy to build and test custom assistants without ever touching raw PHI during development.

If you’re evaluating this for your practice, start with the clinician overview and request a security summary or BAA directly. If you’re building integrations, the developer documentation walks through the API and de-identified proxy in detail.

Where to Verify Everything in This Article

This list is a starting point, not a substitute for legal counsel reviewing your specific contracts and configurations.

Sources

Written using BabyLoveGrowth

Related articles

Developers: 3 Rules for Epic FHIR Integration, TEFCA/IAS and MedScrubReliable FHIR Bulk Export: NDJSON, Polling, Token Tips for Developers8–12 Week Clinic Pilot: EMR Integrated Prior Authorization Automation