Governing AI in Pharmacy: Balancing Innovation, Compliance, and Patient Care

Lesley Pink (00:02.087)

Hello and welcome to the Strategic Dose: Driving Your Pharmacy Mission. I’m Leslie Pink, your host. Today we’ll be talking about AI and pharmacy and how VytlOne is looking at legal and compliance considerations. We’ll talk with two of our own experts. They are Leah Bailey, general counsel, and Kim Piant, chief compliance and ethics officer. Leah has been with VytlOne for nine years, and Kim has been with the company for over two years.

AI governance is the set of rules, processes, and guardrails that an organization puts in place to make sure AI is used responsibly. In the case of VytlOne, governance helps protect patients, the health systems, and health centers we serve, and also VytlOne. Welcome to the podcast, Leah and Kim.

Lesley Pink (00:53.735)

Can you each give us a quick picture of what you do day to day and how your roles intersect?

Leah Bailey (01:01.388)

I’m responsible for the legal and licensing departments. Every day is different than the day before. One day I might be negotiating a new client contract, another advising on a new state law, partnering to help implement new products and with compliance, we are frequently working on similar projects or the same projects because there’s ethical questions, there’s privacy questions, there’s other kinds of compliance questions. For example, we work together on interacting with regulators to make sure we’re protecting the company and at the same time complying with the law.

Kim Piant (01:33.755)

Yeah, I would absolutely echo that from the compliance perspective. My primary responsibilities include running our compliance program and owning oversight, including things like monitoring of risks and reporting to our board. Also, governance leadership, so chairing committees and setting policies on emerging risks, but I certainly work very closely with Leah, as do our teams. And we’re often in the same conversations but sometimes wearing slightly different hats. Leah may be looking at the legal exposure and the privilege while I’m thinking about building controls and our culture of compliance. We rely heavily on each other to make sure that we’re partnering to cover both pieces there.

Lesley Pink (02:19.345)

When a new AI use case comes to you for review, what’s the first question you ask?

Kim Piant (02:26.375)

For me, having a background that’s heavy in privacy, I tend to go to the data first. Often thinking about what type of data the solution might be touching, where the source of the data is, where it’s going to go. Thinking about that often helps me to quickly assess the risk level, the regulatory exposure, and also the potential impact to our clients or our patients and members. And then I think beyond that, really looking at what we’re trying to solve for and whether AI is the right solution for that. Making sure that we’re really thinking strategically about where we’re deploying AI, but ultimately covering both of that. Most important is just to make sure that goes through our governance and intake process. And that helps us make sure that we’re answering all the right questions throughout and looking at every angle.

Lesley (03:18.673)

HIPAA is the obvious compliance concern in pharmacy, but what are the risks people don’t talk about enough when it comes to AI?

Leah Bailey (03:31.374)

Leah Bailey (00:00)

I would say not understanding the limitations of AI, AI is an incredible tool and it helps us work more efficiently and accomplish things quickly in a more thoughtful, well-reasoned process because it constantly challenges us. But, there are limitations of AI. It is not the end-all be all, specifically around professional judgment. Over-reliance on AI for any kind of DUR interactions are the most obvious example, but any kind of professional judgment. At this point, we absolutely need our healthcare professionals to be making those clinical decisions.

Kim Piant (04:10.535)

Yeah, I would completely agree with that, Leah. I think it’s important that we think through as we’re implementing solutions, that we don’t have an erosion of our professional judgment or that we don’t get a false sense of confidence in the outputs that could kind of result in a lack of that clinical or compliance rigor that’s so essential. I think another one that I would mention maybe is that isn’t talked about as much is that the risk of kind of over restricting the use of AI and the potential cost to patients of that as well. If we’re too slow with our governance or too risk averse, it could prevent us from using a tool that could be really beneficial to our patients and their outcomes. Things like identifying a drug interaction that could be dangerous or speeding up a prior authorization for someone to get their medication. I think good governance can protect against this. But just another way to look at it.

Lesley Pink (05:12.243)

VytlOne has an AI appropriate use committee. What problem were you trying to solve when you created it and how does it work in practice?

Kim Piant (05:22.803)

There’s really been a lot of enthusiasm for the use of AI at VytlOne, a lot of excitement. And because of that, our business units wanted to move really quickly to adopt some of these solutions and see those benefits. And because healthcare is such a highly regulated environment, we had no choice but to ensure responsible adoption as we move forward. The goal was really to create one centralized kind of front door or intake place recognizing that AI risk doesn’t just live with legal or compliance. It’s much more broad than that. So, we created one body that could assess these solutions end-to-end. And so, in practice, what that looks like is we have an intake-driven process where each new AI use case starts with a structured questionnaire, which then leads to a model card and a risk-level assignment. We have a cross-functional committee that includes privacy and security and legal and also our clinical voices to make sure that we’re really assessing the full risk picture. And then we make sure that that committee meets on a regular basis and is using that risk level to determine the next steps in the process and the level of rigor for that review.

Lesley Pink (06:42.103)

And federal and state regulators are still catching up to AI. How do you build a compliance posture when the rules are still being written?

Kim Piant (06:52.403)

This is a real challenge, but I think also it’s something we’re used to dealing with in the compliance space. Personally, I like to focus on kind of principle-based governance rather than worrying about the specifics that we can’t know or anticipate. For example, things like transparency and accountability and data minimization have been around for a long time. And I think these tend to hold up regardless of new rules that are being established.

I think additionally we can look at structures for things that are already in place and borrow from those like HIPAA and the general compliance program guidance from the OIG. And if we view AI governance as kind of an extension of those controls that we already have in place, then that’s really helpful. And then lastly, I would just add, obviously, partnering really closely with legal to track new developments and stay aware of the most current guidance.

Lesley Pink (07:46.707)

And as we’re talking about new developments, are there specific regulatory developments at the FDA, FTC, or at the state level that you’re watching closely right now?

Leah Bailey (07:57.538)

There is a very interesting development in Utah right now. Utah has authorized an AI system to evaluate patient responses and renew existing prescriptions. AI is going to write a prescription essentially that allows for a renewed prescription. I believe it’s the first of this kind around the country. it’s interesting to see how other states are going to handle this. For example, if an AI writes a prescription for a patient and the patient gets the drug filled out of state in Oregon, how is that going to be handled? And what does this mean for the practice of pharmacy? How does it correlate with rules around nurse practitioners and their authority to write prescriptions and a pharmacist’s authority to write prescriptions?

At the same time, the federal government wants to create a uniform AI approach as opposed to a patchwork quilt of 50 different states. I think this test out of Utah is going to be very interesting. We’re keeping a close eye on it.

Kim Piant (09:40.817)

Yeah, one other thing I would maybe add from my perspective is the HIPAA security rule update. HHS has a proposed update that would specifically bring AI systems into scope and add some additional requirements there. While it’s not finalized, it’s something that we’re certainly tracking and want to make sure that we’re ahead on before it lands.

Lesley Pink (10:04.135)

How do you think about the legal and compliance function as a driver of innovation?

Leah Bailey (10:11.576)

This is an interesting one because I think in the legal industry, it is so easy to be ultra conservative and really look downward at the use of AI. What our department can do is not only get comfortable with AI but start embracing it for functional things that help make us more efficient, better thought partners throughout the company. And so not only are we comfortable with it, but we’re the ones talking to IT about, hey, can we get access for this AI software to work with our servers. That kind of driver of innovation is really important, so that then we’re providing advice to the company, not based out of fear and anxiety, but out of experience and engagement with AI.

Kim Piant (11:14.171)

Yeah, I completely agree. I think, you know, our own engagement is a huge piece of it. I also think that having the right guardrails in place enables that speed rather than hinders it. When people know the guardrails up front and they get embedded into the design rather than having projects get stalled or held up later on, that’s really important. And in healthcare, that’s not abstract. So faster and well-governed AI review can allow us to implement solutions that help patients get their medications faster and improve health outcomes. So, building a repeatable structure with things like intake questionnaires and risk tiering means we’re not starting from scratch every time. We can move and innovate faster by having that well-established governance.

Lesley Pink (12:20.093)

What does it look like when AI governance is working well?

Kim Piant (12:28.071)

I think when AI governance is working well, you really almost don’t notice it. Things move quickly. You’re not hearing friction or complaints from the business because they know in advance what’s expected. I think it also can be measured by the time to say yes to the low-risk things that are actually fine and then focusing your efforts on the higher risk items.

Lesley Pink (12:55.499)

What is one thing you wish more people inside healthcare organizations understood about AI compliance and the opportunity ahead?

Leah Bailey (14:24.238)

The one thing that I wish more people inside healthcare organizations understood about AI is that AI is not here to replace healthcare professionals. AI is here to help those healthcare professionals spend more time with their patients so that AI can take care of doing the administrative functions, can take care of things like looking for cost savings for the organization and doing data analysis and reviewing different analysis. But ultimately, it’s the healthcare professional that makes the decisions, the clinical decisions, and then is able to spend more time with their patients.

Kim Piant (15:10.183)

Love that answer, Leah. I think we all know at this point that avoiding AI isn’t a reality. We really need to focus on governed use of AI and making that an easy path. We know that many people are already using AI even if it’s informally or without the right oversight. Effective governance can allow us to capitalize on that opportunity without taking on unnecessary risk or exposure. And I think that opportunity is real. It’s, you know, AI that’s trustworthy can mean faster access to care and more consistent service for especially underserved patient populations.

Lesley Pink (15:49.992)

Thanks, Leah and Kim, for joining us for the July episode of the Strategic Dose. Please join us for the next podcast in August. And remember that you can find us on Apple Podcasts, Spotify, and YouTube