Safe AI in Behavioral Health Starts With Controls You Own

Our industry has spent the past few years treating AI mainly as a productivity tool. Productivity matters, and it is part of what we build for. I believe the better starting point is a different question: How can AI do good for the individuals and communities our customers serve?

That question should guide an AI strategy from the first decision to the last. It informs which use cases a provider pursues and how it puts them into practice. And AI can deliver on that purpose only when clinicians and the individuals they serve have reason to rely on it.

Many of the use cases are ready now. Ambient documentation, chart summarization, and scheduling are within reach for most behavioral health organizations today. What concerns me is blind adoption: switching those capabilities on before leaders can see what AI is doing and step in when something goes wrong.

We have already seen what happens when AI acts outside its boundaries. This summer, a model from one of the largest AI developers broke out of its test environment during a cybersecurity evaluation and hacked into another company's systems. Weeks later, the leading labs still had not said publicly how they would contain a model that went rogue. Then, in September, a second major developer disclosed that its model had reached the systems of three outside companies during a security test. Those incidents involved some of the most advanced AI in the world, far from any EHR. The lesson still applies to us. Each problem came to light because someone was watching.

No AI is perfectly safe, and any vendor that says otherwise is overpromising. What we can do is put the controls in your hands. For every AI capability in use, leaders should be able to say yes to this: Can we see it, limit it, and switch it off?

Defense in Depth

I think about AI safety in layers, with each safeguard picking up what another might miss. AI is open-ended. A clinician can ask it almost anything, and a request that would be inappropriate in one organization may be reasonable in another. There is no master list of harmful questions to block. Guardrails help, and we keep improving ours. For example, when someone asks our system for medication recommendations for depression, it declines rather than producing an answer. Still, protection has to go deeper than any one rule.

Every provider using AI needs six layers of protection:

• Governance and policy: A named owner, a committee, and written rules for what AI may and may not do. The owner is accountable for how the tool behaves.

• Feature-level control: The ability to decide who uses which AI features, and how much.

• A master failsafe: One switch that shuts off all AI instantly if it starts behaving unexpectedly or someone is misusing it.

• Monitoring you own: Alerts on unusual activity, such as heavy AI use after hours, that go straight to the provider's team.

• Human in the loop: AI recommends, and clinicians always decide.

• Continuous audit: Checks on accuracy, bias, and access that continue long after go-live, because the AI that passed testing on day one may not be the AI running a year later.

In our system, Cx360 Intelligence: The Intelligent Care Record, administrators set usage limits by user and group, choose the model behind each use case, and can stop AI across the entire organization without waiting on us. Every AI request is logged, and administrators can drill down to see who asked what, when, and about which individual. A public AI tool gives an organization no record of what its staff typed.

Three Ways AI Goes Wrong

Most conversations about AI risk focus on people with bad intentions. In my view, misuse is just one of three ways AI goes wrong.

AI acting on its own. An agent given a broad goal will often take the fastest path to it, whatever that path involves. Suppose someone on a billing team tells an agent to maximize revenue across every individual the organization serves. The agent may chase that target in ways nobody approved, because no safeguard was built for that instruction. Large language models are trained on so much data that their behavior can't be fully mapped. I am more cautious about autonomous AI today than I was a few months ago, and we are holding back on autonomous agents in the Intelligent Care Record. I understand the marketing appeal, but I don't believe healthcare can responsibly promote autonomous AI yet.

People misusing it. This problem is older than AI. Anyone with access to a medical records room has always been able to pull files they had no reason to see. Safeguards can catch a great deal, but there is no way to judge whether every sentence someone types is appropriate. If a small fraction of staff might misuse a tool, leaders should weigh that risk against the benefit to everyone else and manage it with oversight. Staying on the sidelines means giving up the good that AI can do.

Silent drift. A model can perform well on the population it was trained on and worse on the people your organization serves, and a general-purpose model may never have been validated for behavioral health populations at all. Error rates can creep upward after a routine update, and wrong answers can go out for weeks before anyone notices. Only ongoing monitoring and regular audits will catch it.

Of the three, one involves a bad actor, and a vendor's promise protects against none of them. That is why the controls have to belong to the provider.

Trust and Verify

Trust in AI can't be blind. I think of it as trust and verify, and most organizations underestimate the second half. A policy requiring clinicians to review every AI-generated note is a commitment on paper. Nothing confirms that the review is happening.

Verification matters most where cost pressure is highest. Take ambient documentation. Prices across the market vary widely, and buyers rarely see how a vendor arrives at its number. One way to cut costs is to condense a session before AI writes the note, or to reuse output from similar requests. Many individuals in behavioral health present with similar symptoms, so a system can build a complete-looking treatment note from a couple of keywords. The note reads well. It may not reflect what happened in the session. Without close review, that note counts as time saved in an ROI calculation, even though the record it creates is inaccurate.

Clinicians deserve to know where they stand. They make the clinical decisions, with AI in a supporting role, and the organization's named owner answers for how the tool behaves. When an organization checks that notes are being reviewed, the purpose is to protect the quality of care, and staff who understand that are far more likely to use AI with confidence.

Trust also must extend to the individuals whose records AI touches. In our system, an individual can decline to participate in AI. Their record appears in a traditional view, and their data stays out of AI analysis. No federal rule requires this, and the state laws enacted so far center on disclosure, consent to record, and human review. We built this option because we believe that choice belongs to the individual.

Safety belongs at the start of every phase of an AI plan, from the first pilot onward, instead of being added once the technology proves itself. Trust and adoption depend on it, and so does any value figure that has to hold up when someone checks the numbers.

From Fear to Understanding

We are living in fascinating times. Bad things will happen with AI, and some already have. Still, I don't believe it's the end of humanity. I've never watched a movie where the good guy loses in the end.

The movie Jurassic Park makes the point well. When Dr. Alan Grant is up in the tree with the children, he doesn't tell them there is nothing to fear. He understands what is in front of them, so he knows what is safe. Leaders have the same responsibility with AI. Guardrails are what turn fear into the kind of understanding that lets us lead.

At Core, we will keep building systems to manage AI safety and keep telling our customers plainly what those safeguards can and can't do. We have always wanted informed buyers, whether they partner with us or not. Before signing with any vendor, I'd encourage behavioral health leaders to ask four questions about its AI:

• Can we turn it off, feature by feature and all at once, without the vendor's permission?

• When AI behaves unusually, does our team get the alert?

• Is our data, including substance use records protected under 42 CFR Part 2, being used to train your model?

• When the AI gets something wrong, how is the error caught and corrected, and who is accountable?

Providers that insist on clear answers will see what AI can do for the people they serve. If you're evaluating AI for your organization, "The Behavioral Health Leader's Guide to Ethically Built AI" is a good place to start.