AI in therapy with Genevieve Bartuski
In this episode (quick summary):
AI in therapy is accelerating, but Genevieve Bartuski argues the therapist-client relationship remains irreplaceable. Dan King and Genevieve unpack where AI can safely support group practices (billing, documentation, audits) and where ethical risk spikes (session recording, hallucinations, accountability, and client overreliance).
TL;DR:
Use AI to reduce admin load, not to replace clinical judgment or human connection.
What you’ll learn:
- Why the therapeutic alliance matters more than any tool
- How AI hallucination and model drift create real governance risk
- Where AI can streamline operations without undermining client trust
- How to think about crisis gaps and client dependence with human oversight
Genevieve Bartuski’s quote: “I don’t think it will ever replace a therapist or replace that human connection.”
Best for: Group practice owners, clinical directors, and ops leads exploring AI tools for compliance, billing, and documentation — without compromising client safety.
Key terms: AI in therapy, HIPAA compliant tools, therapeutic alliance, clinical judgement/judgment, organisation/organization policy, counselling/counseling ethics, documentation load, model drift, AI hallucination, client care
AI in therapy: why it won’t replace clinicians and where it can safely help
“People still need that humanity. They need that human touch.”– Genevieve Bartuski
Genevieve Bartuski, co-founder of Unicorn Intelligence Tech Partners, joins Dan King to unpack what AI in therapy really means for group practice owners. With a background in forensic and cyberpsychology and a focus on AI governance, Genevieve offers a balanced view: AI can be a powerful support infrastructure, but it cannot substitute the therapeutic alliance or the clinical accountability therapists are held to.
The conversation starts with the fear sitting under many practice owners’ decisions: “Is AI going to take our jobs?” Genevieve’s answer is direct. The strongest predictor of positive outcomes is still the relationship between therapist and client. Tools can assist, but the work of therapy relies on empathy, trust, context, and judgment — especially in moments involving risk to self or others, where standards of care and professional board obligations apply.
Where Genevieve becomes most cautious is in governance. Large language models can “hallucinate” (or, in her words, confabulate), fill gaps with false certainty, and change behaviour over time through drift. In a clinical environment, those failure modes aren’t minor glitches — they raise real questions about liability, duty of care, and who is responsible when technology causes harm. Even with guardrails and red-teaming, she argues the messy reality of human behaviour will always generate edge cases that systems aren’t ready for.
For practice owners looking for practical leverage, Genevieve draws a clear boundary: prioritise backend efficiency first. Billing, insurance workflows, audits, compliance documentation, and admin processes are “time vampires” for clinicians — and areas where AI can reduce the load without sitting inside the clinical relationship. She also urges caution with AI note-takers that record sessions: even if the intent is efficiency, the trust impact can be significant, and unclear data handling can undermine psychological safety for clients.
The episode also explores a more hopeful, system-level application: access gaps. With long waits, overwhelmed crisis centres, and emergency departments holding people for days, Genevieve can imagine AI acting as a bridge between crisis evaluation and the first therapy appointment — but only with explicit human oversight, appropriate triage boundaries, and governance that prioritises safety. The takeaway for operators is simple: adopt AI intentionally, treat outputs as drafts, and keep clinical judgment in the driver’s seat.
Takeaways
- AI won’t replace the therapeutic alliance; human connection remains the core predictor of outcomes.
- Assume AI can be confidently wrong — build policies for verification, review, and escalation.
- Use AI first in operations: billing, audits, documentation workflows, and compliance support.
- Be cautious with AI session recording and note-taking; client trust and data governance must come first.
- Define accountability upfront: who reviews outputs, how errors are caught, and what happens when risk is detected.
- AI in therapy should be treated as an assistant, not an authority — clinical judgment remains non-negotiable.
Transcript: AI in therapy with Genevieve Bartuski
“I don't think it will ever replace a therapist or replace that human connection.” — Genevieve Bartuski
Read the full transcript
Dan King:
[0:00] Hello, everyone. Daniel King here with another episode of Your Group Practice. I'm very much looking forward today to talking to Zendia Bartuski. She is a psychologist with an MBA. She's a double threat, and she's charted in forensic and cyberpsychology and working at sort of the cutting edge of AI governance. So we're going to talk about some things that we don't often talk about here, and I'm looking forward to it. Welcome, Zendia.
Genevieve Bartuski:
[0:26] Thank you. Merci.
Dan King:
[0:28] Good to have you. So what are you most excited about right now? You know, you're working across a few different domains. What's most exciting about you in your work?
Genevieve Bartuski:
[0:36] I think the most exciting thing is every day I learn something new. I never thought I would end up in cyber psychology or technology because I started out in forensics and I fell into this realm. And over, I just think back over the past year, how much I've learned and grown and changed, how much technology is changing. And it's kind of interesting to be in this spot right now.
Dan King:
[0:57] So in our world, things are fast changing in the group practice world. Is AI going to take our jobs?
Genevieve Bartuski:
[1:02] I don't think so. I don't think so. People still need that humanity. They need that human touch. Anybody who works in therapy should know the research behind this. Best predictor for a good therapeutic outcome is always going to be that relationship that you develop with your therapist. That's hands down over everything consistently. While I think that there is a spot for AI to augment therapy and to be, you know, and to be used in the background, I mean, it can be used in a variety of different ways. I don't think it will ever replace a therapist or replace that human connection.
Dan King:
[1:36] I think you're right. It's just to play devil's advocate for a bit and have some fun.
Genevieve Bartuski:
[1:39] Okay, go for it.
Dan King:
[1:41] Look, so many clients can so easily access AI, and so many tech companies are probably chomping at the bit to reduce their costs and see ways that they can diminish humanity and expand automations, right? So even if it's, let's say that it's true, and I would certainly agree with you, that that therapeutic patient-therapist alliance is so vitally important and so human, there are a bunch of incentives at work that will diminish the lull of good old-fashioned human connection.
Genevieve Bartuski:
[2:15] I don't think so.
Dan King:
[2:16] Okay.
Genevieve Bartuski:
[2:16] Yeah, I don't think so, because it's well, first of all, we can't we can't really trust AI because AI, they say they call it hallucinating. To me, it sounds more like confabulating, like AI just if it doesn't have the information to fill stuff in and AI evolves over time, you have which is called drift. So there's too as a tech company, I think there's too much of a risk to make it a replacement for therapy because it's it could potentially go off the rails. And also, you know, like as a therapist, I'm held to my board standards. I'm held to all these standards of practice, the standards of care, these things I have to do. I have to, you know, warn, you know, if there's a danger to self or others, I have to, you know, take steps to protect people. AI doesn't have that. Like, AI is not held to those same standards of practice, standards of care, standards of everything. So it's, you know, coming from forensic psychology, I'd be a little bit kind of worried, like, where is this going to go? If it causes harm? Who's responsible? How are these lawsuits going to play out? Yeah.
Dan King:
[3:15] I mean, I guess there's the outcomes that in a worst case scenario we might see, and then there's what individual actors are going to do, and there's what their incentives are, right? But your view, it sounds like, look, the ultimate outcomes are so problematic and so obviously problematic that even actors with incentives to... Ultimately look to replace humans, even those actors are going to be forced to confront the obvious truth.
Genevieve Bartuski:
[3:44] Yeah. And even if you put the guardrails in, you know, have people red teaming in the background trying to break it, anybody who's worked in mental health or medical, as soon as you say like, all right, we've covered all our bases, we've seen it all, somebody is going to come around and do something completely bizarre and mess up the system. And so as a developer, I think, you know, the developers need to be aware of
Genevieve Bartuski:
[4:05] that and need to be aware of where are their boundaries should be.
Dan King:
[4:08] So if we know that it's highly unlikely that therapists will be truly replaced, but as you say, there's a role for AI in a therapy practice. What does that role look like? If I'm running a group practice and I know AI is coming in some shape or form, how should I seek to leverage AI?
Genevieve Bartuski:
[4:28] Start with what, because therapists, especially any therapist in private practice knows that the backend stuff is time consuming. It's a time suck. It's like a time vampire. Doing billing and all of that stuff. So if there's AI that can help with billing, again, make sure everything is HIPAA compliant. As somebody tells you they're HIPAA certified, kind of, that there's no such thing. Make sure it is HIPAA compliant. Find out how the AI is being trained, what data is used, how are they removing any personal information, stuff like that. But you can use it, you know, I would love to see somebody be able to streamline insurance billing with AI because it is a nightmare when you're doing it yourself. It is an absolute nightmare.
Genevieve Bartuski:
[5:05] You can use it to make sure your note search are good, you know, or to help write your notes. Make sure everything's going to hit all the boxes that you need, especially if you're billing insurance companies. I'm still weary about recording therapy sessions and having AI write the notes. I'm a little weary about that because in therapy, I want my clients to be open and feel safe. And I don't know that if I were sitting on the other end of the couch, I'd be like, that I would feel comfortable with AI recording me and like these, you know, in some of my darkest moments and putting it up on the screen and then what happens to that information. But also I see we have so many gaps within the mental health system itself, not just here, but in the UK as well, where people can wait for weeks and weeks and weeks to get an appointment to get in to see a therapist.
Genevieve Bartuski:
[5:49] A crisis centers across the country are overwhelmed. People sit in hospitals for days waiting for beds. With human oversight, I stress this completely, I could see AI stepping in there.
Genevieve Bartuski:
[6:00] Because a lot of people who go to crisis centers, they don't necessarily need to be in the hospital, but then they might have to wait six to 12 weeks for an appointment. So what do you do in the meantime?
Genevieve Bartuski:
[6:09] Everybody has a smartphone. It's not like just rich people have smartphones anymore. You have to have a smartphone to exist in this world. So is there something that we could do to fill in that gap between like a crisis eval and the first appointment that would just kind of get them through just as a bridge with human oversight, with making sure they're appropriate, stuff like that. There's also people who will sit in emergency rooms three, four days before there's a bed that opens up in a hospital. And so it's very real. You know, people very, very, you know, they suffer with this. And so this would be a really good place where AI or technology could step in and just help with because the system's broken so where are the gaps in the system right there that would be fantastic it would also be fantastic again with coming back to the billing being able to streamline that and like organize the billing and all the back end stuff um that's that's huge for people who are in private practice billing.
Dan King:
[7:02] And audits right which are so complicated and so
Genevieve Bartuski:
[7:06] Onerous compliance.
Dan King:
[7:08] Yeah, yeah. And I think it's often, my experience has been, it's pretty rare to find clinician owners and clinicians who are phenomenally good at holding space and the skills that we expect of therapists, as well as being sufficiently detail-oriented.
Genevieve Bartuski:
[7:27] Oh, yeah, exactly. Oh, my gosh, my head just starts spinning when I look on the back end. I'm like, I don't want to know.
Dan King:
[7:34] So one of the areas where you express some reluctance is AI note takers. So there's a concern you had that will a patient trust a device? With their innermost secrets
Genevieve Bartuski:
[7:48] Right yeah if it's recording it it's i think it's a little different like people will use like you know chat gpt and gemini and other ones they'll they'll type stuff in i think that's a little different than sitting in a therapy session and having it recorded that's that that just to me and it could be just me personally i have issues with that um are you watching the season of the pit no no you need to yes it's such a good show i worked in emergency rooms that I watch that and I'm like, oh my goodness. Although they do get the Pennsylvania mental health law wrong. It's not three days. It's 120 hours for an involuntary commitment. But nitpicky here. But there is a doctor who's all like about the AI and it consistently is making mistakes like where she's recording it. The notes are coming through. It's not accurate. The doctors are so busy that they're that one of them, you know, one of the records ends up going to like surgery or something. And it completely hallucinated a disorder that the person didn't have. So, and they're doing it in such a way that's like very, I love it because it's not preachy. They're not saying AI is good, AI is bad. It's just there. And it's, you know, I mean, that's what we're dealing with now on the front lines.
Dan King:
[8:54] Yes. And it's all happening so quickly. It's very difficult to find the space to reflect and to come up with a worldview that you can be confident in here. The ground is changing beneath our feet so quickly. It's hard for our nervous
Dan King:
[9:07] systems to regulate and it's hard. It's hard for us to be confident that we have a worldview that we can really defend and move forward with.
Genevieve Bartuski:
[9:15] Mm-hmm. Absolutely. And the other concern I have is that AI is supposed to make everything easier, but I worry it's the same thing. A colleague of mine made this point, so I can't credit this as my own idea, but I thought she was brilliant for this. She was talking about when she worked in marketing years ago and BlackBerrys came out and it was like, oh, you're going to be more efficient because you can get your email on your phone. And it turned out that the position she was in, she ended up being busier because the expectations changed because she could do more. She was more accessible. And that's one of my fears with AI. And actually, Noah Wiley's character on The Pit made that same argument that if you can do more, you're expected to do more. It doesn't, there's like never, if you think about it, it's never an end. It's like you can do more, keep doing more instead of like, hey, you know, you did more work, you know, nine to 12 and call the rest of it a day.
Dan King:
[10:08] At the core, it's such an interesting point. At the core of how we work is the distinction I always feel between expectations and agreements. And unconscious expectations are hovering over all work. Any work involving teams and multiple people, which is virtually all the work there is.
Genevieve Bartuski:
[10:28] Right.
Dan King:
[10:28] Unconscious expectations are a hugely powerful force. I would argue to move unconscious expectations into the realm of conscious agreements, but that takes intention and time. It takes slowing down. It does. And it's harder for people to slow down now, right? You have to be more deliberate because there are so many opportunities to speed up. I find it as someone that, as we were talking about peace, when we first began this conversation off air, it's very hard to find those moments of peace, which I think are necessary to move unconscious expectations into the realm of consciousness.
Genevieve Bartuski:
[11:03] I absolutely agree.
Dan King:
[11:05] And you're right. The speed with which new unconscious expectations are emerging is so fast.
Genevieve Bartuski:
[11:12] It is. It's incredibly fast. And it's hard to keep up with.
Dan King:
[11:17] Yeah. I have an ADHD mind that loads very quickly. I find it very hard to keep up with.
Genevieve Bartuski:
[11:22] Same.
Dan King:
[11:23] And I will give AI some credit. It really helps because new situations that I haven't thought about that are complicated emerge, and I have to manage a lot of complexity. And to have someone that I trust to not solve my problems, but play with me in the mess of complexity, that I don't have to call, I don't have to wait for
Dan King:
[11:44] a human to get back to me, Right. That I can bring in someone and hopefully I can ensure that I don't trust it wholeheartedly, that I don't rely on it the way I would a human emotionally. But I know that it can at least give me things to think about that I won't have thought about. And it can give me some even some groundedness.
Genevieve Bartuski:
[12:03] I agree. One of my side passions is language and speaking French and learning Italian. And AI has been helpful in some of those questions that people don't tell you when you're learning a second language. How do you learn how to think? How do you learn how to use it? How do you, when you're learning a third language, how do you not mix it up with the, with the second one. Although I do. I mean, I still do. I do all that.
Dan King:
[12:27] It's so confusing.
Genevieve Bartuski:
[12:29] It will. Or sometimes, or the fact that sometimes when you're learning another language, your brain just does not remember a word in any of them. Like, I'll just look at something and be like, what is that word? And I'm like, am I going, like, is something wrong with me? No, it's, you know, an AI helps with that stuff. But I was talking to my Italian tutor a couple weeks ago about AI, and I was asking it, so I'm learning the pronouns. And they are nothing like anything in English and they say it's related to French, totally different. AI does not understand Italian pronouns because it gets them wrong. It does. And I was talking to him about it. He said he can tell when something in Italian is written by AI because it just doesn't understand the nuance of the pronouns because you have pronouns that you use in speech, which are different than pronouns that you use writing. And AI doesn't understand the difference between those and mixes them all up. And then they have compound pronouns. Yeah.
Dan King:
[13:19] Some things like, I'm surprised to hear that because things like that you would think should be relatively easily solvable for AI. So that surprises me. But yeah, probably there are going to be these bizarre knowledge gaps. We don't know unless you're super technically minded and you really understand the architecture of LLMs. I don't know why that's the case. I guess I'm not super shocked on reflection, but you might find another AI that doesn't have that issue.
Genevieve Bartuski:
[13:44] Yeah. And I was using ChatGPT, not to call them out, because it's a good one. I mean, And, you know, I have nothing against it, but it's the one that's probably got the most information. And it was having trouble giving me the charts and picking out the right pronouns to help me. I was like, this is not helping me at all, like grammatically. And I talked to my tutor about that. And he was like, he's like, I've not seen an AI that can get Italian.
Dan King:
[14:08] And it comes at us with such authority, such authority. And it even it takes a little bit of courage, exactly, but a little bit of presence to be able to question it and say, wait a second, I'm not sure about that.
Genevieve Bartuski:
[14:21] Yes.
Dan King:
[14:22] I had a, it's interesting, I was working on a transaction and our, you know, we were, I was debating a point with our legal counsel. And they said, you know, we're pretty convinced this is true because AI told us it wasn't. And then we pushed back and then we won the argument. And I sort of think to myself, well, that doesn't necessarily prove your point because AI is strange. On the one hand, it presents with a lot of authority. And on the other hand, chat GPT with its default setting, you can totally argue with it and it went back. Right. And both of those truths are kind of frightening to me.
Genevieve Bartuski:
[14:55] Yeah. And, you know, I think people need to understand that as well when they're using it. Like, it's not a human. It's not, it doesn't have actual intelligence. It's a large language model. It basically, it looks for patterns and spits them back out to you. That's what it does. But I do like those that ChatGPT did get rid of their 4.0 because that was the one that was most, they called it the love model or something like that. Because people were actually getting romantically attached to it. And there is very interesting. I found a subreddit thread of about 20,000 people, 20,000 talk turns on it of people who are in love with their AI and actually having relationships with it and going to fertility clinics. And I'm like, okay, I get this is like something therapists aren't necessarily aware of, like that there is this other thing you have to watch out for with
Genevieve Bartuski:
[15:46] your clients is that people are developing these unhealthy attachments to a machine.
Dan King:
[15:50] Well, you brought up the last topic I wanted to bring in to this conversation, which is when are clients using AI too much? So you just came up with a perfect example.
Genevieve Bartuski:
[16:00] Yes like when some of it's not quite so you know insidious as that some of it's you know when they're outsourcing their thinking to ai or they stop trusting themselves because ai said something different those are those are concerns also if people can't resolve conflicts that's another thing because ai doesn't i mean you can some some of the other ai systems not necessarily like the OpenAI or Gemini, but there are like, you know, character building ones. I don't want to mention any specific names. Like you can, you can tell it to argue with you and create characters and personas. And, but for the most part, it's not going to, it wants you to like it. It wants you to engage with it. It wants you to continue using it. And I say want very loosely. It doesn't actually think it's programmed. And so you're getting an echo chamber. You're getting a reflection of what you want to hear.
Dan King:
[16:49] You are. Sometimes that can be healing. with. Sometimes that can be exactly what you need in the moment.
Genevieve Bartuski:
[16:56] Yes. Rogerian psychology. Yep. Exactly. That unconditional positive regard.
Dan King:
[17:01] Yeah. Yeah. And so it's not an accident. I would imagine that Chachi BT's default setting, that's our program that way. And you pointed out before we hit record that that unconditional positive regard can have many unintended consequences amongst them. The kids not necessarily develop male conflict resolution skills. Someone that you rely on is now ultimately going to agree with you all the time.
Genevieve Bartuski:
[17:27] Absolutely. And just a random, fun, interesting fact that is one of the first AI systems, ELISA, which was built in MIT in the late 60s, they deliberately used Ruggierian psychology. And immediately they started noticing and it was very very rudimentary like nothing like what we have now there are pre-programmed conversations but very quickly educated people at harvard or at mit not harvard were developing feelings for the ai not necessarily romantic feelings but like an emotional attachment to it and the i can't think of his name off the top of my head but the researcher the computer programmer was like this is not good like this is kind of concerning how fast this happened and that was back in the late 60s it's.
Dan King:
[18:10] Like oh wow
Genevieve Bartuski:
[18:11] Okay that's.
Dan King:
[18:12] Not super widely
Genevieve Bartuski:
[18:13] Known no and they called the eliza effect um because they they called it eliza uh based off of eliza in pygmalion because you could dress it dress it up to be something different wow super.
Dan King:
[18:24] Interesting nuggets for for our listeners so if i were to kind of sum up as we've been talking about the role that ai can play and other fast-moving technologies in the group practice world it sounds like you're your kind of mental model is be very careful when it comes to touching patients. But everything in the back end that's unrelated to patients, if it comes to relating to insurance companies or governments, that kind of thing, because there tends to be so much detail, so much need for detail, that automation can just speed up and simplify and improve processes. So where it comes to touching patients, be cautious.
Dan King:
[19:02] But every other element of the back end, AI can play a hugely helpful role. Is that your summary of your mental model?
Genevieve Bartuski:
[19:09] Pretty much. Yeah. And I mean, it's well, we've always seen, you know, if you do any kind of psychological testing for years, we've had computer programs that can give you, you know, that can spit out a report for you. The same thing with that, like what we've been doing as psychologists for years is like when I get an MMPI report, I don't take it at face value. I look at it. I see what it's given me and I'm like, okay, this kind of makes sense, but this doesn't fit. Or no, the person does, you know, is not showing hyperchondriasis. It's because they have an actual stuff. Like same thing. You take it within relationship to the person and use your clinical judgment. Yeah.
Dan King:
[19:43] I think there's something I have found in my own just sort of creative processes. Getting started can be so, so hard sometimes. And so the speed with which you can get some output is so fast. And yeah, you have to be super skeptical about that output and scrutinize it. But it can sort of get the juices flowing really quickly.
Genevieve Bartuski:
[20:03] Yes. Yes. It's really good if you write stuff. You know, I wouldn't use it creatively, but that's just me personally because I'm like, If I'm going to write something creative, it's coming straight from here. But if I have to write a report or something like that, you know, I would never use patient information as I say that. Give me an outline. What point should I make? You know, I want to include this. Is there anything I'm missing? Can you find me the research? My goodness, if I had this when I was doing my dissertation, like being able to pull the research, that would have been fantastic. It's so fast now. That's the good stuff for it because it's like, okay, at least now I have an outline. At least now I can think
Genevieve Bartuski:
[20:38] about this and let this percolate in my head and get back to writing.
Dan King:
[20:41] Makes perfect sense. So since we've been talking about AI, I would love to end on a human note. And we will usually end the interviews on a human note by asking. When you're not doing this super cool work at the set of nexus as AI, governance and psychology, what do you do for fun?
Genevieve Bartuski:
[20:57] I read a lot. I'm an avid reader. I have three dogs. I live in the mountains, study Italian, pretty chill.
Dan King:
[21:04] I like it. I like it. Pleasure to meet you and talk to you today. And I'm sure listeners will benefit from thinking a bit more intentionally about AI and what they can do with it and what they shouldn't do with it.
Genevieve Bartuski:
[21:17] Yes. And thank you so much for having me. I enjoyed this.
Transcript: AI in therapy with Genevieve Bartuski
“I don't think it will ever replace a therapist or replace that human connection.” — Genevieve Bartuski
Read the full transcript
Dan King:
[0:00] Hello, everyone. Daniel King here with another episode of Your Group Practice. I'm very much looking forward today to talking to Zendia Bartuski. She is a psychologist with an MBA. She's a double threat, and she's charted in forensic and cyberpsychology and working at sort of the cutting edge of AI governance. So we're going to talk about some things that we don't often talk about here, and I'm looking forward to it. Welcome, Zendia.
Genevieve Bartuski:
[0:26] Thank you. Merci.
Dan King:
[0:28] Good to have you. So what are you most excited about right now? You know, you're working across a few different domains. What's most exciting about you in your work?
Genevieve Bartuski:
[0:36] I think the most exciting thing is every day I learn something new. I never thought I would end up in cyber psychology or technology because I started out in forensics and I fell into this realm. And over, I just think back over the past year, how much I've learned and grown and changed, how much technology is changing. And it's kind of interesting to be in this spot right now.
Dan King:
[0:57] So in our world, things are fast changing in the group practice world. Is AI going to take our jobs?
Genevieve Bartuski:
[1:02] I don't think so. I don't think so. People still need that humanity. They need that human touch. Anybody who works in therapy should know the research behind this. Best predictor for a good therapeutic outcome is always going to be that relationship that you develop with your therapist. That's hands down over everything consistently. While I think that there is a spot for AI to augment therapy and to be, you know, and to be used in the background, I mean, it can be used in a variety of different ways. I don't think it will ever replace a therapist or replace that human connection.
Dan King:
[1:36] I think you're right. It's just to play devil's advocate for a bit and have some fun.
Genevieve Bartuski:
[1:39] Okay, go for it.
Dan King:
[1:41] Look, so many clients can so easily access AI, and so many tech companies are probably chomping at the bit to reduce their costs and see ways that they can diminish humanity and expand automations, right? So even if it's, let's say that it's true, and I would certainly agree with you, that that therapeutic patient-therapist alliance is so vitally important and so human, there are a bunch of incentives at work that will diminish the lull of good old-fashioned human connection.
Genevieve Bartuski:
[2:15] I don't think so.
Dan King:
[2:16] Okay.
Genevieve Bartuski:
[2:16] Yeah, I don't think so, because it's well, first of all, we can't we can't really trust AI because AI, they say they call it hallucinating. To me, it sounds more like confabulating, like AI just if it doesn't have the information to fill stuff in and AI evolves over time, you have which is called drift. So there's too as a tech company, I think there's too much of a risk to make it a replacement for therapy because it's it could potentially go off the rails. And also, you know, like as a therapist, I'm held to my board standards. I'm held to all these standards of practice, the standards of care, these things I have to do. I have to, you know, warn, you know, if there's a danger to self or others, I have to, you know, take steps to protect people. AI doesn't have that. Like, AI is not held to those same standards of practice, standards of care, standards of everything. So it's, you know, coming from forensic psychology, I'd be a little bit kind of worried, like, where is this going to go? If it causes harm? Who's responsible? How are these lawsuits going to play out? Yeah.
Dan King:
[3:15] I mean, I guess there's the outcomes that in a worst case scenario we might see, and then there's what individual actors are going to do, and there's what their incentives are, right? But your view, it sounds like, look, the ultimate outcomes are so problematic and so obviously problematic that even actors with incentives to... Ultimately look to replace humans, even those actors are going to be forced to confront the obvious truth.
Genevieve Bartuski:
[3:44] Yeah. And even if you put the guardrails in, you know, have people red teaming in the background trying to break it, anybody who's worked in mental health or medical, as soon as you say like, all right, we've covered all our bases, we've seen it all, somebody is going to come around and do something completely bizarre and mess up the system. And so as a developer, I think, you know, the developers need to be aware of
Genevieve Bartuski:
[4:05] that and need to be aware of where are their boundaries should be.
Dan King:
[4:08] So if we know that it's highly unlikely that therapists will be truly replaced, but as you say, there's a role for AI in a therapy practice. What does that role look like? If I'm running a group practice and I know AI is coming in some shape or form, how should I seek to leverage AI?
Genevieve Bartuski:
[4:28] Start with what, because therapists, especially any therapist in private practice knows that the backend stuff is time consuming. It's a time suck. It's like a time vampire. Doing billing and all of that stuff. So if there's AI that can help with billing, again, make sure everything is HIPAA compliant. As somebody tells you they're HIPAA certified, kind of, that there's no such thing. Make sure it is HIPAA compliant. Find out how the AI is being trained, what data is used, how are they removing any personal information, stuff like that. But you can use it, you know, I would love to see somebody be able to streamline insurance billing with AI because it is a nightmare when you're doing it yourself. It is an absolute nightmare.
Genevieve Bartuski:
[5:05] You can use it to make sure your note search are good, you know, or to help write your notes. Make sure everything's going to hit all the boxes that you need, especially if you're billing insurance companies. I'm still weary about recording therapy sessions and having AI write the notes. I'm a little weary about that because in therapy, I want my clients to be open and feel safe. And I don't know that if I were sitting on the other end of the couch, I'd be like, that I would feel comfortable with AI recording me and like these, you know, in some of my darkest moments and putting it up on the screen and then what happens to that information. But also I see we have so many gaps within the mental health system itself, not just here, but in the UK as well, where people can wait for weeks and weeks and weeks to get an appointment to get in to see a therapist.
Genevieve Bartuski:
[5:49] A crisis centers across the country are overwhelmed. People sit in hospitals for days waiting for beds. With human oversight, I stress this completely, I could see AI stepping in there.
Genevieve Bartuski:
[6:00] Because a lot of people who go to crisis centers, they don't necessarily need to be in the hospital, but then they might have to wait six to 12 weeks for an appointment. So what do you do in the meantime?
Genevieve Bartuski:
[6:09] Everybody has a smartphone. It's not like just rich people have smartphones anymore. You have to have a smartphone to exist in this world. So is there something that we could do to fill in that gap between like a crisis eval and the first appointment that would just kind of get them through just as a bridge with human oversight, with making sure they're appropriate, stuff like that. There's also people who will sit in emergency rooms three, four days before there's a bed that opens up in a hospital. And so it's very real. You know, people very, very, you know, they suffer with this. And so this would be a really good place where AI or technology could step in and just help with because the system's broken so where are the gaps in the system right there that would be fantastic it would also be fantastic again with coming back to the billing being able to streamline that and like organize the billing and all the back end stuff um that's that's huge for people who are in private practice billing.
Dan King:
[7:02] And audits right which are so complicated and so
Genevieve Bartuski:
[7:06] Onerous compliance.
Dan King:
[7:08] Yeah, yeah. And I think it's often, my experience has been, it's pretty rare to find clinician owners and clinicians who are phenomenally good at holding space and the skills that we expect of therapists, as well as being sufficiently detail-oriented.
Genevieve Bartuski:
[7:27] Oh, yeah, exactly. Oh, my gosh, my head just starts spinning when I look on the back end. I'm like, I don't want to know.
Dan King:
[7:34] So one of the areas where you express some reluctance is AI note takers. So there's a concern you had that will a patient trust a device? With their innermost secrets
Genevieve Bartuski:
[7:48] Right yeah if it's recording it it's i think it's a little different like people will use like you know chat gpt and gemini and other ones they'll they'll type stuff in i think that's a little different than sitting in a therapy session and having it recorded that's that that just to me and it could be just me personally i have issues with that um are you watching the season of the pit no no you need to yes it's such a good show i worked in emergency rooms that I watch that and I'm like, oh my goodness. Although they do get the Pennsylvania mental health law wrong. It's not three days. It's 120 hours for an involuntary commitment. But nitpicky here. But there is a doctor who's all like about the AI and it consistently is making mistakes like where she's recording it. The notes are coming through. It's not accurate. The doctors are so busy that they're that one of them, you know, one of the records ends up going to like surgery or something. And it completely hallucinated a disorder that the person didn't have. So, and they're doing it in such a way that's like very, I love it because it's not preachy. They're not saying AI is good, AI is bad. It's just there. And it's, you know, I mean, that's what we're dealing with now on the front lines.
Dan King:
[8:54] Yes. And it's all happening so quickly. It's very difficult to find the space to reflect and to come up with a worldview that you can be confident in here. The ground is changing beneath our feet so quickly. It's hard for our nervous
Dan King:
[9:07] systems to regulate and it's hard. It's hard for us to be confident that we have a worldview that we can really defend and move forward with.
Genevieve Bartuski:
[9:15] Mm-hmm. Absolutely. And the other concern I have is that AI is supposed to make everything easier, but I worry it's the same thing. A colleague of mine made this point, so I can't credit this as my own idea, but I thought she was brilliant for this. She was talking about when she worked in marketing years ago and BlackBerrys came out and it was like, oh, you're going to be more efficient because you can get your email on your phone. And it turned out that the position she was in, she ended up being busier because the expectations changed because she could do more. She was more accessible. And that's one of my fears with AI. And actually, Noah Wiley's character on The Pit made that same argument that if you can do more, you're expected to do more. It doesn't, there's like never, if you think about it, it's never an end. It's like you can do more, keep doing more instead of like, hey, you know, you did more work, you know, nine to 12 and call the rest of it a day.
Dan King:
[10:08] At the core, it's such an interesting point. At the core of how we work is the distinction I always feel between expectations and agreements. And unconscious expectations are hovering over all work. Any work involving teams and multiple people, which is virtually all the work there is.
Genevieve Bartuski:
[10:28] Right.
Dan King:
[10:28] Unconscious expectations are a hugely powerful force. I would argue to move unconscious expectations into the realm of conscious agreements, but that takes intention and time. It takes slowing down. It does. And it's harder for people to slow down now, right? You have to be more deliberate because there are so many opportunities to speed up. I find it as someone that, as we were talking about peace, when we first began this conversation off air, it's very hard to find those moments of peace, which I think are necessary to move unconscious expectations into the realm of consciousness.
Genevieve Bartuski:
[11:03] I absolutely agree.
Dan King:
[11:05] And you're right. The speed with which new unconscious expectations are emerging is so fast.
Genevieve Bartuski:
[11:12] It is. It's incredibly fast. And it's hard to keep up with.
Dan King:
[11:17] Yeah. I have an ADHD mind that loads very quickly. I find it very hard to keep up with.
Genevieve Bartuski:
[11:22] Same.
Dan King:
[11:23] And I will give AI some credit. It really helps because new situations that I haven't thought about that are complicated emerge, and I have to manage a lot of complexity. And to have someone that I trust to not solve my problems, but play with me in the mess of complexity, that I don't have to call, I don't have to wait for
Dan King:
[11:44] a human to get back to me, Right. That I can bring in someone and hopefully I can ensure that I don't trust it wholeheartedly, that I don't rely on it the way I would a human emotionally. But I know that it can at least give me things to think about that I won't have thought about. And it can give me some even some groundedness.
Genevieve Bartuski:
[12:03] I agree. One of my side passions is language and speaking French and learning Italian. And AI has been helpful in some of those questions that people don't tell you when you're learning a second language. How do you learn how to think? How do you learn how to use it? How do you, when you're learning a third language, how do you not mix it up with the, with the second one. Although I do. I mean, I still do. I do all that.
Dan King:
[12:27] It's so confusing.
Genevieve Bartuski:
[12:29] It will. Or sometimes, or the fact that sometimes when you're learning another language, your brain just does not remember a word in any of them. Like, I'll just look at something and be like, what is that word? And I'm like, am I going, like, is something wrong with me? No, it's, you know, an AI helps with that stuff. But I was talking to my Italian tutor a couple weeks ago about AI, and I was asking it, so I'm learning the pronouns. And they are nothing like anything in English and they say it's related to French, totally different. AI does not understand Italian pronouns because it gets them wrong. It does. And I was talking to him about it. He said he can tell when something in Italian is written by AI because it just doesn't understand the nuance of the pronouns because you have pronouns that you use in speech, which are different than pronouns that you use writing. And AI doesn't understand the difference between those and mixes them all up. And then they have compound pronouns. Yeah.
Dan King:
[13:19] Some things like, I'm surprised to hear that because things like that you would think should be relatively easily solvable for AI. So that surprises me. But yeah, probably there are going to be these bizarre knowledge gaps. We don't know unless you're super technically minded and you really understand the architecture of LLMs. I don't know why that's the case. I guess I'm not super shocked on reflection, but you might find another AI that doesn't have that issue.
Genevieve Bartuski:
[13:44] Yeah. And I was using ChatGPT, not to call them out, because it's a good one. I mean, And, you know, I have nothing against it, but it's the one that's probably got the most information. And it was having trouble giving me the charts and picking out the right pronouns to help me. I was like, this is not helping me at all, like grammatically. And I talked to my tutor about that. And he was like, he's like, I've not seen an AI that can get Italian.
Dan King:
[14:08] And it comes at us with such authority, such authority. And it even it takes a little bit of courage, exactly, but a little bit of presence to be able to question it and say, wait a second, I'm not sure about that.
Genevieve Bartuski:
[14:21] Yes.
Dan King:
[14:22] I had a, it's interesting, I was working on a transaction and our, you know, we were, I was debating a point with our legal counsel. And they said, you know, we're pretty convinced this is true because AI told us it wasn't. And then we pushed back and then we won the argument. And I sort of think to myself, well, that doesn't necessarily prove your point because AI is strange. On the one hand, it presents with a lot of authority. And on the other hand, chat GPT with its default setting, you can totally argue with it and it went back. Right. And both of those truths are kind of frightening to me.
Genevieve Bartuski:
[14:55] Yeah. And, you know, I think people need to understand that as well when they're using it. Like, it's not a human. It's not, it doesn't have actual intelligence. It's a large language model. It basically, it looks for patterns and spits them back out to you. That's what it does. But I do like those that ChatGPT did get rid of their 4.0 because that was the one that was most, they called it the love model or something like that. Because people were actually getting romantically attached to it. And there is very interesting. I found a subreddit thread of about 20,000 people, 20,000 talk turns on it of people who are in love with their AI and actually having relationships with it and going to fertility clinics. And I'm like, okay, I get this is like something therapists aren't necessarily aware of, like that there is this other thing you have to watch out for with
Genevieve Bartuski:
[15:46] your clients is that people are developing these unhealthy attachments to a machine.
Dan King:
[15:50] Well, you brought up the last topic I wanted to bring in to this conversation, which is when are clients using AI too much? So you just came up with a perfect example.
Genevieve Bartuski:
[16:00] Yes like when some of it's not quite so you know insidious as that some of it's you know when they're outsourcing their thinking to ai or they stop trusting themselves because ai said something different those are those are concerns also if people can't resolve conflicts that's another thing because ai doesn't i mean you can some some of the other ai systems not necessarily like the OpenAI or Gemini, but there are like, you know, character building ones. I don't want to mention any specific names. Like you can, you can tell it to argue with you and create characters and personas. And, but for the most part, it's not going to, it wants you to like it. It wants you to engage with it. It wants you to continue using it. And I say want very loosely. It doesn't actually think it's programmed. And so you're getting an echo chamber. You're getting a reflection of what you want to hear.
Dan King:
[16:49] You are. Sometimes that can be healing. with. Sometimes that can be exactly what you need in the moment.
Genevieve Bartuski:
[16:56] Yes. Rogerian psychology. Yep. Exactly. That unconditional positive regard.
Dan King:
[17:01] Yeah. Yeah. And so it's not an accident. I would imagine that Chachi BT's default setting, that's our program that way. And you pointed out before we hit record that that unconditional positive regard can have many unintended consequences amongst them. The kids not necessarily develop male conflict resolution skills. Someone that you rely on is now ultimately going to agree with you all the time.
Genevieve Bartuski:
[17:27] Absolutely. And just a random, fun, interesting fact that is one of the first AI systems, ELISA, which was built in MIT in the late 60s, they deliberately used Ruggierian psychology. And immediately they started noticing and it was very very rudimentary like nothing like what we have now there are pre-programmed conversations but very quickly educated people at harvard or at mit not harvard were developing feelings for the ai not necessarily romantic feelings but like an emotional attachment to it and the i can't think of his name off the top of my head but the researcher the computer programmer was like this is not good like this is kind of concerning how fast this happened and that was back in the late 60s it's.
Dan King:
[18:10] Like oh wow
Genevieve Bartuski:
[18:11] Okay that's.
Dan King:
[18:12] Not super widely
Genevieve Bartuski:
[18:13] Known no and they called the eliza effect um because they they called it eliza uh based off of eliza in pygmalion because you could dress it dress it up to be something different wow super.
Dan King:
[18:24] Interesting nuggets for for our listeners so if i were to kind of sum up as we've been talking about the role that ai can play and other fast-moving technologies in the group practice world it sounds like you're your kind of mental model is be very careful when it comes to touching patients. But everything in the back end that's unrelated to patients, if it comes to relating to insurance companies or governments, that kind of thing, because there tends to be so much detail, so much need for detail, that automation can just speed up and simplify and improve processes. So where it comes to touching patients, be cautious.
Dan King:
[19:02] But every other element of the back end, AI can play a hugely helpful role. Is that your summary of your mental model?
Genevieve Bartuski:
[19:09] Pretty much. Yeah. And I mean, it's well, we've always seen, you know, if you do any kind of psychological testing for years, we've had computer programs that can give you, you know, that can spit out a report for you. The same thing with that, like what we've been doing as psychologists for years is like when I get an MMPI report, I don't take it at face value. I look at it. I see what it's given me and I'm like, okay, this kind of makes sense, but this doesn't fit. Or no, the person does, you know, is not showing hyperchondriasis. It's because they have an actual stuff. Like same thing. You take it within relationship to the person and use your clinical judgment. Yeah.
Dan King:
[19:43] I think there's something I have found in my own just sort of creative processes. Getting started can be so, so hard sometimes. And so the speed with which you can get some output is so fast. And yeah, you have to be super skeptical about that output and scrutinize it. But it can sort of get the juices flowing really quickly.
Genevieve Bartuski:
[20:03] Yes. Yes. It's really good if you write stuff. You know, I wouldn't use it creatively, but that's just me personally because I'm like, If I'm going to write something creative, it's coming straight from here. But if I have to write a report or something like that, you know, I would never use patient information as I say that. Give me an outline. What point should I make? You know, I want to include this. Is there anything I'm missing? Can you find me the research? My goodness, if I had this when I was doing my dissertation, like being able to pull the research, that would have been fantastic. It's so fast now. That's the good stuff for it because it's like, okay, at least now I have an outline. At least now I can think
Genevieve Bartuski:
[20:38] about this and let this percolate in my head and get back to writing.
Dan King:
[20:41] Makes perfect sense. So since we've been talking about AI, I would love to end on a human note. And we will usually end the interviews on a human note by asking. When you're not doing this super cool work at the set of nexus as AI, governance and psychology, what do you do for fun?
Genevieve Bartuski:
[20:57] I read a lot. I'm an avid reader. I have three dogs. I live in the mountains, study Italian, pretty chill.
Dan King:
[21:04] I like it. I like it. Pleasure to meet you and talk to you today. And I'm sure listeners will benefit from thinking a bit more intentionally about AI and what they can do with it and what they shouldn't do with it.
Genevieve Bartuski:
[21:17] Yes. And thank you so much for having me. I enjoyed this.