On this bonus episode of Ropes & Gray’s Insights Lab’s Multidimensional Data Reversion podcast series, Shannon Capone Kirk and David Yanofsky are joined by Michelle DiMartino, a cultural psychologist, to discuss the challenges and human factors involved in AI adoption. Michelle provides insights into the psychological responses to AI, explaining how high-stakes, time-pressured environments can lead to mental shortcuts and differing reactions to ambiguity. The discussion covers three critical frames for understanding AI implementation: the technology itself, human interpretation and interaction with AI outputs, and the organizational culture and governance surrounding AI tools. Tune in to gain valuable perspectives on how to navigate the evolving landscape of AI, including strategies for effective AI rollout, risk management, and fostering a culture that supports responsible AI use.
Transcript:
Shannon Capone Kirk: And I’m Shannon Capone Kirk, managing principal and global head of Ropes & Gray’s advanced e-discovery and AI strategy group.
David Yanofsky: We have a special guest on the podcast today. But before we get to them, Shannon, you called me the other day—you had another challenging moment around AI. Tell me about it.
Shannon Capone Kirk: Yes, obviously we’ve had a number of conversations about AI and especially corporate clients wanting to implement generative AI, and we, at Ropes, are working very hard on helping our clients to do that. What I called you about was this observation that I have now made numerous times when being asked for advice on the implementation of generative AI in especially a global or corporate environment. Here’s the challenge of advising on that right now, and it’s somewhat, for me, in two extremes. You get a massive organization that will call and say, “Please advise us on the risks of implementing generative AI.” Let’s call that on the spectrum a very overly broad request—how that is such an exceedingly large ask. On the other end of the spectrum, you have folks that essentially ask that same question, but when you get on the phone with them they will fixate—and this happens a lot—on one very, very specific use case of GenAI, and that is whether or not to allow GenAI to basically listen in on recorded meetings and generate the top-line summary or bullets of the things that happened in that meeting. So, that specific corporate use case. They want advice on that larger issue, on, “Can I implement generative AI generally in my environment?” But then, the conversation will bleed to this one very specific issue. As a person with only a law degree, and no psychology degree, I think we present it to the actual person with the right degree, which is Michelle.
David Yanofsky: Yes, Michelle DiMartino recently joined the Insights Lab as a cultural psychologist. Michelle, when you hear Shannon talk about these two groups of folks and how they’re responding to AI, what does that sound like to you?
Michelle DiMartino: Thank you both for having me on. In a legal context, decisions are often high stakes—time pressured, ambiguous—and that means that for some people, their brains are taking mental shortcuts that are either defaulting to automation, hoping that either that speed and technology, they’re seeing high investment, or output that looks really good at prose that looks quite human, like somebody drafted it, that spent a lot of time on it, or really beautiful data visualizations that used to just require hours and hours of work that is a product of GenAI now. They might see that and say, “Okay, this is the safe decision, this is the silver bullet.” Or under those high-stakes, time-pressured environments, they navigate ambiguity by sticking with what they know and overcorrecting to familiarity when something feels risky. These are just very human mental shortcuts that are known, that have been studied for years, and they’re just human responses to ambiguity. So, Shannon, that observation that you’re seeing in these clients, that you’re observing time and time again, they’re just different reactions to that same ambiguity bearing out in different ways.
David Yanofsky: How do you help people get beyond those responses? Are there known or proven methods that expand people’s view?
Michelle DiMartino: I think it makes sense to take a step back and look at this issue in three different frames—how GenAI and where the human factor is important at three different levels. So, first there’s the tech itself: how these tools are built to recognize patterns based on historical data. That means that they can reproduce the blind spots baked into the systems they were trained on. For example, consider a company implementing a GenAI system meant to surface risk alerts based on keyword analysis from emails or reports. Over time, they realize that the system flagged certain departments constantly—not because those teams were necessarily riskier, but because they were documenting everything thoroughly compared to another team who flies under the radar by avoiding documentation altogether. AI here isn’t wrong in this scenario, it just was faithfully reflecting what the organization’s documentation culture was—it was just using skewed data from the teams that probably needed risk monitoring less. For organizations that were like, “Yes, AI is the answer here,” you might ask them, “How is this tool going to interact with the behaviors that already exist in our organization?”
Then, there’s the human side—how humans actually interpret and act on outputs. This is where you start to see the cognitive biases that come into play. So, over-trusting results that feel confident, or just believing the opposite of whatever the AI spits out because you didn’t like its answer—that’s something that happens a lot. It gives you an ambiguous response. You actually might just believe the opposite. That might not be true either—you might just be reacting to it and might make an equally bad decision. The way we perceive the outputs from these tools, and our state of mind when doing so, is just as important to consider when designing effective AI programs, just as important as solid tech itself.
Then, finally, there’s the organizational level—how these tools are introduced, socialized, and governed. Whether the tools get used well or not often has less to do with the tech and more to do with the cultures, workflows, and the way change is managed. You were talking about strategies for rollout—modeling the behavior and showcasing use cases that you want to see, doing hands-on trainings and modeling use cases that you want to see, and then role modeling those. So, having partners who are trusted leaders at your firm, using these tools in ways that you want to see emulated, using them responsibly.
Shannon Capone Kirk: What’s interesting about what Michelle just did there is she just rolled out a form of a framework in how to approach these questions. Those are the aspects that folks actually should be thinking about and articulating when they ask the question, “We want to roll out generative AI. What are the risks?”
David Yanofsky: When you’re talking to clients, are they focused on a single one of those three things: tech, human, organization? Where are people at in thinking in terms of this framework?
Shannon Capone Kirk: This is just my observation, and this is not every global corporation. First of all, you have a wide spectrum of approaches, but the majority are not at the high-level stage that Michelle spelled out of. First, let’s figure out in a coordinated, cross-functional way what AI tools we want to evaluate, how they’re built, how our humans in our environment are going to interpret output and what do we want them to do, and how does the organization socialize it and build role modeling. To me, that’s the top-line framework. What I’ve been doing is, just on the nature of the things I’ve been asked to do, jumping in at the next level down of framework, because our clients have jumped straight into using specific tools, or wanting to implement specific GenAI tools. The most common is Microsoft’s Copilot, and within Copilot, folks fix on essentially one use of it so far, and that is whether or not to use GenAI with Teams meetings. Because the majority of asks come by way of a specific rollout of Microsoft Copilot, and a couple of other what I call “general-purpose corporate AI”—so, “Draft me an email, read my emails and summarize things,” that sort of general corporate use—we built out what are the specific features and settings of specific tools that actually transcend to most all tools to be careful about and have it in writing in a chart so folks walk through that and have a structure within which to evaluate the tools. Some examples include, “When my folks use this tool and they prompt it, where are those prompts retained and preserved? How can I extract them and review them if I need to for compliance monitoring, enforcement, or litigation? How do I preserve the input, the output, etc.?”
David Yanofsky: The way that you would preserve any other corporate communication.
Shannon Capone Kirk: Right. And guess what? It is not so easy with some of these tools. “Who has access to what?” And sorting all of the access controls out. That’s a series of line items in our framework. Some of them are very tool specific, based on lessons learned from other people’s rollouts.
David Yanofsky: This is all risk-based, right? Michelle’s framework on the tech factors, the human factors, and the organizational factors, in every one of those there are risks and opportunities to be had. And what you’re talking about is when you are setting up the most popular tools that exist, in the nuance of those, in the real details of those, here are the risks at every settings level that you need to understand as you roll that out. Now, I imagine some people hear you describe the risk to them, and they hear someone saying “no.”
Shannon Capone Kirk: Exactly. Some organizations, when you say, “Here’s the risk if you do or don’t do this particular feature in this GenAI tool,” we’ve had some people say, “Nope, shut it down.” Zero tolerance. I would say that’s an outlier. Most organizations see the value of GenAI, they know that we’re, in the law at least, at the beginning stages of implementation and ensuring defensibility, etc., so they want to be in on it now, and they’re willing to hear those risks and accommodate them the best they can.
David Yanofsky: How do you guide someone to understanding whether or not the risk of a certain feature, the risk of a certain tool, is worth taking?
Shannon Capone Kirk: For us, as lawyers, especially those of us in the litigation realm, when we’re asked, “Should I or should I not do this thing?” our answers are always going to be somewhere on a spectrum of risks. And so, everything I do every single day is, “I would advise you that your risk of turning on this feature in AI is the following ramifications: it could lead to sanctions and XYZ litigation because you did not know where your input and outputs were, and you couldn’t preserve them. But what mitigates that risk is your litigation profile is very small.” So, that’s one way to evaluate these risks. It just literally comes down to every organization and every single use case. Let me make it harder, because then also, there’s a multitude of risks to consider: there’s data privacy laws, there’s copyright. But when we say these things, to get back to your point, it is not us saying “no.” That’s impractical, especially when GenAI is the new wave of how human beings are going to exist. It’s not saying “no”—it’s just saying, “Look, you asked me for what the risks are. I’m going to tell you what the risks are. I’m going to help you evaluate whether to do or not do this thing, or add in modifications that reduce your risk.” That’s all GenAI is, and that’s what it’s going to be.
David Yanofsky: Michelle, bringing it back to these two cohorts of people that we’ve been talking about, you described one of them as looking to take a shortcut. What Shannon was just describing is really complex decision making. When I hear that, I can imagine a shortcut person throwing up their hands and saying, “Oh, here’s complex risk. Let’s just not do it. I cannot take the time to deal with all of this. The easiest path forward for me right now is to just stop doing it, or to not do it.” What are ways to manage and push back against that type of response?
Michelle DiMartino: That is another way of making a shortcut. That is another way of saying, “It is the path of least resistance for me to do the thing that I’ve already been doing.” It’s path dependency. I’m navigating ambiguity by doing the thing that I’ve always been doing. I have more prompts for them than I do have solutions.
David Yanofsky: What are the prompts?
Michelle DiMartino: “Is this use case driving our thinking because it’s the most strategic or it’s the most familiar?” “Are we avoiding because it feels uncertain or undefined?” “If we weren’t worried about getting in trouble, what GenAI use cases would we actually explore?” These are maybe a little bit, like, blue-sky thinking, but it could be worth just getting conversations going. “Are we writing a policy for the tool we use or for the outcomes that we care about?” These are the kinds of things that I might ask a more skeptical adopter or a cautious adopter. I also might want to have that kind of three-levels conversation with them of the tech factor, the human factor, and the organizational factor. If they better understand more information about what the tech is good at and not good at, they might understand different use cases for it, beyond the tunnel vision use cases they see utility in or that they’re comfortable with—the kinds of patterns and risks that these tools are good at detecting. If they understand a little bit more about the tech and in ways that are accessible to them, they might see expanded use cases. If they understand a little bit more about how members of their team are more or less likely to engage with these tools in ways that are exposing their company to risk, or making better decisions, then they might be more open to trying things out. If they learn more about training programs, or organizational rollouts that work, role-modeling exercises that work, then they might be able to do it. Or if they see a peer try it, then they might actually catch on. Maybe using that higher-level framework for this more skeptical group, instead of diving into the tool-specific, or encouraging a tool that they’re not familiar with, or diving into that level-two framework, Shannon, maybe that could be worth trying. Bringing them back up a level before shopping around more specific tools that you actually think would be a good fit could be worth trying out.
Shannon Capone Kirk: I may sound like I’m being inconsistent with what I said in the beginning, and maybe that’s a nature of how we adapt to each different client, but I do think that with certain organizations, everybody is overwhelmed. Forget about AI for a second. Everyone is overwhelmed in the corporate world with work, with the world, with news. And along comes GenAI, and, for me, it’s not even really a decision anymore. The decision has been made. There will be GenAI. So, we’re past that, and we are straight into our frameworks. I do think that for the folks who are overwhelmed in general, and overwhelmed with GenAI—how to build policy around it, how to build training, how to know what is allowed, what’s not allowed, what features and settings to even do—we’ve had success with getting organizations to just start with a tool to help them understand, “This is how this would roll out.” And then, to your point, then we can expand, because it is overwhelming. And so, if you try to do the larger picture and all of the use cases at once, forget it—it’s like telling a kindergartner to go get a PhD.
David Yanofsky: As both of you were talking, Michelle, you brought up people in their jobs who have blinders on and only being able to think about the use cases that could apply specifically to them. Shannon, you just talked about the opposite. Michelle, you brought it up as a challenge. Shannon, you brought it up as an opportunity. If we can only get some of these people to fixate more on a single thing and just get it done for that one thing, then we can move on to the next. My question for you though, Shannon, is—and you brought this up in terms of video recording—it seems like the folks that you’re talking to are very focused on a single thing already, and it does seem to be the thing that they are most familiar with, because they are the people who are on video calls all day.
Shannon Capone Kirk: Exactly. And this is where I do think I might sound inconsistent, but let me fix, again, on the Teams recording. My challenge in my job is getting folks from multiple different departments in major organizations to align on where to start, because otherwise they spin in committee meetings for months and months and months talking about the vague generalities, or that fixed one thing everybody understands, which is whether or not to use AI in recorded meetings, and they spin in this, and they will talk about all of the impossibilities. My job is to then say, “Okay, you all understand the Teams meeting thing,” or whatever recorded meeting. “Great, that’s one use case. You’ve talked about it for weeks. Let’s have this framework, because we should now be elevating ourselves to think about broader use cases within a framework that has actual elements that all of you, from all these different groups—legal, privacy, cybersecurity, name all the groups, because they all have a say in it—can meet and have constructive and productive meetings so that at the end of a year, say, you can answer Michelle’s questions: ‘Do you understand how this GenAI was built? Do you understand the risks associated in how it was built? Do you understand how your humans in your organization interpret those outputs, and how your organization is behaviorally changing with it?’ You’re never going to be able to answer Michelle’s questions unless we start somewhere.” But you have to know Michelle’s questions when you start your committee meetings. “We want to answer Michelle’s questions in one year, that’s our goal, and we have a series of frameworks within which to work.”
Michelle DiMartino: And then, for the people who are maybe a little bit too overly enthusiastic, I have more questions: “What assumptions are we making about how this tool will save time or reduce costs? Have we tested those assumptions? If AI gives us the wrong answer, will we notice, and who is checking?” These are probably things that are built into a lot of the more risk-based frameworks that you’re used to implementing, Shannon, in the risk space, but you think about these from a more intuitive human factor, that technology factor. Somebody who doesn’t understand the terms and conditions side, from my perspective, maybe another more gut-sense one is: “Would I trust a junior associate to handle this without training or supervision? If not, why do I trust the AI?”
David Yanofsky: I’m going to need to ask a framing question here, because we keep on talking about AI this/AI that. With these frameworks, are we talking about all AI, or are we just talking about GenAI?
Shannon Capone Kirk: You did that because you know this is a hot-button issue with me, and I thank you. When I talk about this, I only mean generative AI. I specifically do not mean machine learning or analytics, which has been around for many years now in the law.
David Yanofsky: And so, GenAI—anything using a large language model? Any information coming through a chat interface, or only generative AI chat interfaces?
Shannon Capone Kirk: What I mean, and where the risks really are, are where a system is generating an answer, and it is not pre-programmed, automated, or trained directly by a human in what its results will be. When a system generates on its own the answer, the image, or the result—that’s what I’m talking about.
David Yanofsky: That type of definitive, singular result from interacting with an AI is different than, say, a computer vision AI, where you just say, “Dear computer vision AI, what do you see in this image?” And they say, “Well, we’re 98% sure it’s a cat. But there’s a 2% chance it’s a prairie dog.” That’s a less definitive use case that has less risk, because of some of the human ways of interpreting that, right, Michelle?
Michelle DiMartino: Here’s something that really caught my attention. Researchers ran an experiment where they showed people AI predictions—some right, some wrong—but they varied how confident the AI said it was. When people saw the AI express lower confidence, they actually slowed down and double-checked more—they caught way more of the wrong predictions. And it wasn’t just the general disclaimer anybody that might have used an AI chatbot might be familiar with, like the one that says “AI chatbots can sometimes be wrong” that is in fine print at the bottom—it was a timely confidence estimate that was specific to an output, a prediction, or a decision. Here’s why that really matters for legal tech: instead of just slapping that kind of disclaimer on everything when somebody opens a tool, let’s say, or in that fine print at the bottom of a tool, there are ways to design certain applications of these tools to engage critical thinking. When you’re dealing with contract reviews or compliance decisions, for example, that’s the difference between rubber-stamping something and real oversight.
Shannon Capone Kirk: I love that. I’m going to raise my own resistance question, which is: “How can we trust the GenAI to evaluate its own confidence?” I think probably there’s a higher-level use case from the computer vision, other than, “Tell me, is this a panda bear or not?”
David Yanofsky: The risk of computer vision is, “Tell me, is...”
Shannon Capone Kirk: “Is this cancer?”
David Yanofsky: “Is this cancer” is a big one. “Is this a crosswalk” is another. “Is this a human in front of my self-driving car, or is it a plastic bag?”
Michelle DiMartino: There’s some types of data where it’s so much more plausible to say that kind of thing than it is another. Data that fits really neatly into zeros and ones maps more neatly onto confidence intervals than, say, how culture shows up in an organization. How do you map confidence in a prediction against, “Yes, I think the culture in this part of your organization is high conflict or low conflict.” I think that that’s a really great question, Shannon—one to explore on a future episode.
David Yanofsky: We will have to. Michelle, thank you so much for joining us today. That’s going to be it for Multidimensional Data Reversion. On our next episode, we will still be talking about AI, so be sure to subscribe wherever you get your podcasts. I’m David Yanofsky.
Shannon Capone Kirk: I’m Shannon Capone Kirk.
David Yanofsky: Thank you for listening.
Show Notes:
