HIPAA for AI and health apps: building with health data and AI raises questions the rules never named directly. Here is how HIPAA actually applies to modern AI products.
TL;DR
If your AI product processes protected health information, HIPAA applies to it exactly as it would to any other system. The model, the API behind it, and any pipeline that sees PHI can each be a business associate needing a BAA.
The new risks are around training data, prompts, logs, and third-party model providers, all places PHI can leak. The controls are familiar: encryption, access control, logging, and a BAA with every party that touches the data.
On this page
When HIPAA applies to an AI or health app
The principle is simple and unchanged: HIPAA follows the data, not the technology. If your application creates, receives, stores, or transmits protected health information on behalf of a healthcare customer, it is covered, whether the processing happens in a traditional database or inside a large language model. AI does not create an exception. If PHI goes into your product, the rules come with it.
Follow the data through the AI
The clearest way to reason about HIPAA in an AI product is to trace where PHI actually goes.
PHI can enter through prompts or uploads, pass through a model or third-party API, and end up in outputs, logs, or caches. Each of those stages is a place the data must be protected, and each external party that handles it may need a business associate agreement. The AI pipeline does not shrink your HIPAA scope, it can widen it.
AI providers become business associates
This is the point founders most often miss. If you send PHI to a third-party AI model or API, that provider is handling protected health information on your behalf, which generally makes them a business associate. You need a BAA with them, and not every AI provider will sign one. Choosing model providers that offer a BAA, and configuring them not to retain or train on your data, is a core HIPAA decision for an AI health product.
The training-data trap
Using real patient data to train or fine-tune models is one of the biggest risks in AI health products. PHI absorbed into a model can be difficult to remove and may resurface in outputs. If you train on health data, that data and the resulting model are in scope, and you must be able to control and account for it. Wherever possible, de-identification before training dramatically reduces this risk, because properly de-identified data is no longer PHI.
The controls that matter for AI health apps
Despite the novelty of AI, the safeguards are the familiar HIPAA ones, applied to new data paths.
| Control | What it means for an AI product |
|---|---|
| Encryption | PHI protected in transit to models and at rest in logs and stores |
| Access control | Strict limits on who and what can reach PHI and model data |
| Audit logging | A record of how PHI moves through prompts, models, and outputs |
| BAAs | Signed with every provider and subcontractor that handles PHI |
| De-identification | Removing identifiers before training or analysis where possible |
The lean-team path for AI health products
An AI health startup carries all the usual HIPAA obligations plus new data paths through models and providers. The safeguards, encryption, access control, and logging over everywhere PHI flows, are the same, but the surface is larger and moves faster. Assembling that across tools while shipping an AI product is exactly where gaps open.
Build with AI without losing control of PHI.
Osto is the one-stop cybersecurity and compliance platform built for fast-moving startups. Protect and prove control over health data across your AI pipeline on one platform mapped to HIPAA. No security team required.
Frequently asked questions
Does HIPAA apply to AI health apps?
Yes, whenever the app processes protected health information for a healthcare customer. HIPAA follows the data, not the technology, so PHI handled inside an AI model or pipeline is covered exactly as it would be in any other system.
Is an AI provider a business associate under HIPAA?
Generally yes, if you send it PHI. A third-party model or API that processes protected health information on your behalf is handling PHI, which makes it a business associate requiring a BAA. Not all AI providers will sign one.
Can I train an AI model on patient data?
Only with great care. Real PHI used in training is in scope and can be hard to remove or may resurface in outputs. De-identifying data before training dramatically reduces risk, because properly de-identified data is no longer PHI.
What should I check before sending PHI to an AI API?
Confirm the provider will sign a BAA and that your data will not be retained or used to train their models. If they will not offer a BAA, sending PHI to them is a compliance problem regardless of the model’s quality.
What controls does an AI health app need for HIPAA?
The familiar HIPAA safeguards applied to new data paths: encryption of PHI in transit and at rest, strict access control, audit logging of how PHI moves through the AI, BAAs with every provider, and de-identification before training where possible.
Does de-identified data fall under HIPAA?
Properly de-identified data, meeting HIPAA’s de-identification standards, is no longer considered PHI and falls outside those requirements. This is why de-identification is a powerful risk-reducer for AI and analytics on health data.