Hi friend,

Most mental health AI businesses have chosen to avoid regulation.

They see the process as costly and the rewards as uncertain. A historical analysis of mental health businesses that have pursued regulatory authorisation would support their decision (see Pear, Akili and more).

Perhaps they are correct. However, like all strategy, this choice has tradeoffs.

The majority of mental healthcare spending is on clinical activities that involve diagnosing, treating or managing mental health conditions — activities that are typically subject to medical regulation. Companies that restrain their AI products to non-regulated activities (like coaching, note-taking, or mood tracking) limit their addressable market. Many hope payers will eventually pay for non‑regulated tools if they can show value. But as of today, such payment remains scarce. Avoiding regulation entirely risks ending up in a smaller, increasingly crowded market — while the larger, higher‑value clinical space remains tied to authorised, clinician‑integrated tools.

Most founders (and their investors) may be allergic to regulatory pursuits. But when everyone is zigging, should some zag? 

The answer to this question depends on the ROI leaders can expect. The numerator of this equation is determined by the market size for authorised products and whether the organisation believes they can capture that market — this is where previous businesses have failed. 

The denominator is determined by the process of regulatory authorisation — the likelihood of getting authorised as well as the cost and time associated with such a process. For many, this process is unclear. Regulations can be difficult to understand and vary across jurisdictions. As AI prevalence has increased, the regulations themselves are changing too. Where they will go next and the burden that will place on organisations, is an important question for mental health leaders to understand.

In this three-part Hemingway Guide, we attempt to answer those questions, providing clarity on how regulators are approaching mental health AI products in 2026 and what we might expect in the future. To create this series, we’ve teamed up with May Lee, a regulatory professional of 12 years, with expertise in software and AI as medical devices. May has an engineering and legal background, as well as a deep understanding of the complex landscape of AI in healthcare and how it applies to mental health AI products specifically. 

In Part I of this series, we will provide a 101 on medical device regulation and how it applies to mental health AI products. We will also provide an overview of the current state of US regulation of mental health AI, how it is evolving and what it means for people building in this space. Parts II and III will focus on Europe, the UK and a deep dive into Predetermined Change Control Plans and other hot topics of debate in this space.

Let’s get into it. 

How to read this Guide: If you are already familiar with medical device regulation frameworks, feel free to skip to Chapter 3 (Where Gen-AI Challenges the Current Framework). If you would like a fundamental understanding of how regulators approach medical device regulation and how the US FDA works, start with Chapter 1.

Join the Hemingway Community

If you’re interested in this article, it’s probably because you want tactical information on building a successful mental health organisation. If you’d like more of that, and to meet peers building similar businesses, consider joining the Hemingway Community.

As a member of the community, you’ll join over 500 other mental health leaders and get access to exclusive content, events and resources.

1. Understanding Medical Device Regulation

To understand regulators, you must understand that they are all trying to answer the same question: how do we make sure products that affect patients are safe and effective? They want to allow useful innovation to reach the market, but are constantly balancing that with minimising patient harm.

Regulators first developed frameworks for physical medical devices. As software began diagnosing, monitoring and treating disease, those frameworks expanded to include Software as a Medical Device (SaMD). Today, AI — particularly generative AI — is the latest chapter in that story. It is challenging regulators to again adjust how they regulate a new technology. They do not regulate AI simply because it is AI. They regulate products according to the medical purpose they serve.

Because of that, most are responding to AI by updating their existing systems. We therefore need to understand those existing systems and then look at how they are being adapted.

Medical Devices

Medical device regulation was originally designed for physical products such as pacemakers, MRI scanners and insulin pumps. Because these products can directly affect patient safety, regulators developed frameworks to ensure they were safe and effective before reaching patients. These frameworks are generally risk-based: the greater the potential harm if a product fails, the greater the regulatory scrutiny it receives.

Software as a Medical Device (SaMD)

As software began diagnosing, monitoring and treating disease, regulators faced a new challenge: the software itself had become the medical device. Rather than creating an entirely new regulatory system, they expanded the existing framework through Software as a Medical Device (SaMD). The key question determining if a product was a medical device remained the same: what medical purpose does the product serve? An app that lets users journal their mood is unlikely to be regulated. The same app that claims to monitor, diagnose or treat depression most likely will be.

AI and generative AI

AI is the latest chapter in this evolution. So far, regulators have been clear that their approach to regulating AI in mental health is to adapt existing medical device frameworks to accommodate this new technology. The challenge is that those frameworks were largely designed for software with predictable behaviour. This is particularly acute in mental health, where there is rarely a single correct “answer”. Evaluating whether an AI has correctly identified a tumour on an X-ray is very different from evaluating the quality and safety of a therapeutic conversation. It is with these challenges — not AI itself — that regulators in the US, UK and Europe are now grappling.

2. The US Regulatory Framework and Pathways

In the United States, medical devices are regulated by the Food and Drug Administration (FDA). Like all medical device regulators, the FDA takes a risk-based approach: products that could cause greater harm if they fail are subject to greater regulatory scrutiny.

For a company building a mental health AI product, the first question to answer is: “Is it a medical device?" If the product is intended to diagnose, treat, prevent or monitor a mental health condition, it is likely to fall within the FDA's medical device framework. If it is positioned as a documentation tool, coaching app or general wellbeing product, it may not.

In practice, mental health products fall into one of three broad categories. 

• Wellness products. These products are designed to support general well-being — such as meditation, journaling or stress management — and are not intended to diagnose or treat disease. They generally fall outside FDA medical device regulation. 

• Enforcement discretion. These are products that technically meet the definition of a medical device, but which the FDA has decided pose sufficiently low risk that it generally chooses not to actively enforce certain regulatory requirements. This policy can allow lower-risk digital health products to reach the market without full medical device review, although the FDA retains the authority to intervene if safety concerns arise.

• Regulated medical devices. These are products that claim to diagnose, treat, prevent or monitor a medical condition and therefore require FDA oversight before they can be marketed.

The Regulatory pathways

The FDA has several different regulatory pathways. These are different approval processes that apply depending on how novel a product is and the level of risk it presents. Each pathway requires manufacturers to provide evidence that their product is safe and effective, but the amount and type of evidence differs depending on the circumstances.

• 510(k). The most common is the 510(k) pathway. Rather than proving a product from first principles, a manufacturer demonstrates that it is substantially equivalent to another device already legally on the market (known as a predicate device). If FDA agrees that the new device is no less safe or effective than the existing one, it can be marketed. This pathway works well for products that build on established technologies.

• De Novo. Many mental health AI products are genuinely novel. There is often no existing device they can point to as a suitable predicate. In these cases, companies are more likely to use the De Novo pathway. Here, rather than comparing the product to an existing device, FDA evaluates it on its own merits. If approved, FDA creates a new device classification that future products may be able to use as a reference.

• Premarket Approval (PMA). The final pathway is Premarket Approval (PMA). This is reserved for the highest-risk medical devices and requires the strongest evidence that a product is safe and effective. Most mental health AI products are unlikely to follow this route, although it remains relevant for higher-risk devices.

For developers building novel AI products in mental health today, De Novo is generally expected to be the most relevant pathway. There are currently no approved generative AI mental health devices that could act as predicate devices for a 510(k) submission. On the verge of securing de novo classification, Kintsugi, the voice biomarker AI, shut down its operations after spending four years and $30M attempting to gain clearance. Woebot Health is another body that lies in the mental health regulatory graveyard. The generative CBT product received breakthrough designation, but shut down operations in 2025.

3. Where Gen-AI Challenges the Current Framework

In November 2025, the FDA's Digital Health Advisory Committee began examining the unique challenges posed by generative AI in healthcare. These include hallucinations, performance drift, biased responses, sycophancy and other traits of AI systems.

FDA pathways were originally designed for medical devices whose behaviour is largely predictable. Whether the device is a pacemaker or a piece of clinical software, regulators can evaluate its performance before approval with the expectation that it will behave in much the same way once it reaches patients. Generative AI clearly challenges that assumption.

Unlike traditional software, which produces the same output for the same input, large language models are probabilistic. They generate responses in real time and may produce different outputs to the same prompt, even when nothing about the model has changed. This makes them inherently more difficult to test, validate and regulate, particularly in mental health where there is often no single "correct" response.

A second challenge is that AI products evolve rapidly after they are deployed. Like other Software as a Medical Device, developers continually improve their products through software updates. But AI models are often updated more frequently, with changes to model weights, prompts, safety guardrails and other components. Requiring a completely new regulatory submission for every planned update would make it incredibly difficult to improve products at the pace expected of modern software.

4. How FDA is responding: Updates over Rewrites

Last year we wrote an article on AI regulation which noted that we expected FDA to update and clarify its processes, as opposed to conducting a total overhaul to deal with AI. This continues to be true. FDA's response so far leans on a few structural fixes rather than a wholesale rewrite of its evidence standards.

For specific features, FDA has signalled that it wants a human somewhere in the loop for most use cases, along with red-teaming to bound what the model can say and clear escalation paths when a conversation turns into a safety concern. The agency has admitted it isn't yet sure how to treat a built-in crisis detection feature that might, on its own, turn an otherwise unregulated wellness product into a device.

In terms of pathways, because no generative AI mental health device has ever been cleared, there's no predicate to point to, which is why FDA has said most of these products will need to go through De Novo rather than the faster 510(k) pathway.  

On the monitoring side, the agency's January 2025 draft guidance tries to extend oversight across the full life of the product instead of just at launch. FDA has openly asked the public whether that framework goes far enough for genuinely generative systems. Some outside researchers have pushed further, arguing that postmarket checks aren't enough for a model that updates constantly, and proposing that regulators get live API access so they can test model behaviour and simulate crisis scenarios in real time.

FDA's response to the evolution problem of AI has been the Predetermined Change Control Plan (PCCP). Rather than approving a single, fixed version of a product, a PCCP allows manufacturers to agree in advance which types of changes they expect to make after approval and how those changes will be validated. If the FDA authorises the plan, those pre-specified updates can be implemented without requiring an entirely new regulatory submission, provided they remain within the agreed scope. Congress gave the FDA explicit authority to use PCCPs in 2022. The agency finalised guidance for AI-enabled medical devices in December 2024 and updated it in August 2025. That guidance, alongside joint principles published with Health Canada and the UK's MHRA, now forms the foundation of the FDA's approach to continuously learning AI systems.

5. What this means for Mental Health organisations

Be intentional about your intended use

FDA’s “intended use” doctrine doesn’t stop at what a company says in its marketing copy. Under the Agency’s current intended use regulations (21 CFR § 801.4), intended use can be established by “any relevant source of evidence,” including a product’s design, its functionality, and the circumstances of how it’s distributed and used. In other words, a carefully worded landing page doesn’t settle the question if the product itself behaves like a clinical tool. Many mental health businesses are dancing on this fine line.

FDA has been willing to act on this basis even without explicit disease claims. In July 2025, the agency issued a warning letter to Whoop over a blood pressure feature on its wearable, despite the fact that none of the company’s promotional language made an explicit diagnostic claim. The letter focused instead on how the feature functioned and how users were likely to interpret it. Around the same time, FDA flagged SeniorLife Technologies for marketing claims like detecting “early signs of Alzheimer’s” without the required premarket clearance. FDA is looking at what a product does and how it’s positioned in context, not just the specific words chosen to describe it. 

At FDA’s November 2025 Digital Health Advisory Committee meeting on generative AI-enabled mental health devices, agency staff specifically raised concerns about chatbots that function in individualised, “therapist-like” ways, with or without a clinician in the loop, regardless of how they’re marketed. At the same time, some companies building these tools have been explicit that they intend to stay on the wellness side of the line. Slingshot AI told FDA’s committee directly that its product is meant to provide “general wellness”, not treatment or diagnosis. The American Psychological Association’s 2026 health advisory on AI chatbots and wellness apps used for mental health support flagged that many of these tools aren’t designed or validated for clinical use, even though that’s increasingly how people are using them. 

The practical takeaway for mental health leaders in this space is to start with intended use. The FDA will look at the totality of a product’s design, claims, and behaviour rather than just its marketing copy. The real risk is in not making a deliberate choice on your regulatory status (wellness or device). Without a clear strategic choice, product, sales, and clinical teams can drift between these two states. Leaders must make sure this choice has been made and that their teams are aligned with that direction.

In addition, human oversight, crisis escalation, and honest disclosure are already showing up as baseline expectations across the regulators’ discussions and should be considered by all teams.

Engage Early

There are still no FDA-authorised generative AI medical devices for mental health, meaning the first successful De Novo approvals are likely to shape the regulatory expectations that follow. In many respects, FDA is building the playbook for this category in real time, and the companies entering it today will help define what good evidence, good validation and good clinical safety look like for the next generation of mental health AI.

Companies that engage the FDA early through the Q-submission process, fund that engagement on a realistic multi-year timeline, and build the cross-functional governance to manage a model that keeps changing after launch will spend less time retrofitting later. Mental health leaders need to have that foresight to ensure the longevity of their mental health product.

6. Where to next?

The clearest lesson from the DHAC process is that the FDA does not have this solved and it is quite open about that. The agency also recognises that the demand for these AI mental health products is being driven by a care access gap. It recognises that the digital therapeutics currently authorised (across healthcare) are mostly adjunctive tools layered onto human-delivered care, which keeps risk low but does little to close this access gap. The products that could close this care gap are exactly the ones that would carry the most novel risk.

The direction of travel is towards a risk taxonomy that treats “adjunctive with clinician oversight” and “standalone, unsupervised” as separate evidence thresholds. For companies, deciding which side of the line a product sits on is now an important design decision.

There are a few dates and developments worth tracking over the next year. FDA’s January 2025 guidance on AI-enabled device software functions may be finalised and will be the moment to see whether the agency actually built in enough flexibility for generative systems, or whether it defaults to the same static-model assumptions that shaped the guidance’s first draft. This draft guidance is on FDA’s priority list to finalise within Fiscal Year 2026.

For leaders of mental health organisations, perhaps the most important point is that the category remains largely open. The companies engaging today will help define what good evidence, good validation, and good clinical safety look like for the next generation of mental health AI.

Many will choose to continue to steer clear of regulation entirely. Others will adopt a wait-and-see approach. A small few will engage in the process of gaining authorisation for generative AI medical devices in mental health. If they can succeed (and survive long enough), a huge market will be waiting for them. Whether they can then go and capture that market will be a bigger challenge still.

That’s all for this edition of The Hemingway Report. Many thanks to May Lee for her expert contributions to this piece. Make sure to keep an eye out for Parts II and III in the coming weeks.

And if you found this valuable and want more practical guides to building a successful mental health organisation, consider becoming a Hemingway Pro Member. As a member, you will get access to exclusive content and be invited to join our community of mental health innovators.

Keep fighting the good fight!

Steve

Founder of Hemingway