New research examines why AI-related lawsuits struggle to access critical evidence

As artificial intelligence becomes increasingly embedded in everyday life, a new research article warns that people who believe they have been harmed by AI systems may face significant obstacles when trying to prove their claims in court.

Researchers from Carnegie Mellon University’s CyLab Security and Privacy Institute and Stanford University have published a new paper, “Barriers to Evidence in AI-Related Cases and the Privatization of Proof,” which examines how unequal access to information, resources, and expertise can block litigation against AI developers and deployers. 

The paper will be presented at the 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT).

The researchers note that while much of the AI policy conversation has focused on government action, litigation remains a critical mechanism for holding AI companies accountable. But proving AI-related claims requires evidence, and obtaining evidence in AI-related cases can be especially difficult. For the most part, private companies control key information, including model architectures, training data, system logs, and technical documentation, which is especially closely guarded in AI due to trade secrecy protection.

“Litigation matters,” said Sarah Cen, assistant professor of Electrical and Computer Engineering and Engineering and Public Policy and lead author of the paper. “It's a really important part of our legal system, but it hasn’t been emphasized in a lot of policy conversations. The ability of individuals and groups to identify harm or wrongdoing and try to take it on themselves to litigate is an important part of AI accountability.”

Drawing on past and ongoing litigation involving AI systems, the researchers identified seven recurring sources of asymmetry that can disadvantage plaintiffs: access to models, data, documentation, logs, expertise, computing resources, and infrastructure. Together, these barriers contribute to what the authors call the “privatization of proof,” a situation in which private organizations control the information necessary to establish whether harm occurred.

“The case that first drew me in was the New York Times v. OpenAI case, which became almost an exemplar for how discovery battles played out in court,” said Hannah Ismael, a program associate at Mozilla and co-author of the paper. “We kept seeing the same patterns in discovery battles between developers and plaintiffs repeat across cases involving automated decision-making systems used in employment, immigration, health care, and other high-stakes contexts.The developer both controlled the court’s understanding of the system and held asymmetric power over investigating the validity of these claims.”

 

The ability of individuals and groups to identify harm or wrongdoing and try to take it on themselves to litigate is an important part of AI accountability.

Sarah Cen, assistant professor, Electrical & Computer Engineering and Engineering & Public Policy at Carnegie Mellon University

Across different evidentiary requests, OpenAI resisted access through several recurring arguments: the technical details were proprietary and commercially sensitive, user logs raised privacy concerns, and identifying relevant training data was technically burdensome. Each argument narrowed the plaintiffs’ ability to investigate what was copied and to the extent that it gets regurgitated and how, crucial information to building a successful copyright case.

“It was very clear the New York Times was at a structural disadvantage,” said Ismael.

This dynamic becomes even more complicated when companies are able to minimize the role AI played in the disputed decision. In the class action case Lokken v. UnitedHealth Group, UnitedHealth claimed that its nH Predict tool was merely guiding physicians for Medicare Advantage patients and that physicians ultimately remained responsible for patient care. Plaintiffs, however, argued that the tool overruled physicians and often resulted in the denial of life-saving treatment, despite company policies promising that health care decisions would be made by physicians.

“So now, we are starting to see cases in which private actors not only control the evidence, but also can cast the AI system as secondary or peripheral,” Ismael said. “And with the way discovery is structured, plaintiffs can be forced to prove that is not the case before they are allowed to access the evidence that would show how the system actually operated in practice.”

According to Cen, discovery disputes are not entirely new. Evidence has always been difficult to obtain in litigation. However, AI systems introduce additional challenges because the information, expertise, and resources needed to evaluate their behavior is often highly technical and concentrated within the organizations that develop or deploy them.

“What we're seeing is that decision-making power about evidence can implicitly shift away from judges and courts and toward private actors,” said Cen. “This happens when the only people who may have the ability to see the evidence, understand it, or analyze it are the organizations that control the systems.”

Without access to evidence, proving AI-related claims may be doomed. Yet it is impossible to overhaul our legal system at large, begging the question: what can be done?

The researchers propose a three-part test that courts can use when evaluating requests for access to AI-related evidence. The framework would assess the degree of asymmetry between the parties, determine the minimum level of access needed to evaluate the legal claim using the cause of action as a baseline, and consider whether reasonable alternative forms of access could provide equivalent information while addressing concerns about cost, confidentiality, and proprietary technology. This three-part test provides a way to systematically handle AI evidence requests while also balancing the concerns of both parties. 

The researchers emphasize that different forms of access can sometimes serve similar purposes. For example, limited access to data combined with additional computing resources may provide insights comparable to those gained through broader access to a model's internal workings. This fungibility is important because it creates degrees of freedom that parties can use to negotiate access. 

For Ismael, the stakes extend far beyond individual lawsuits. As AI systems become more deeply integrated into public and private institutions, she believes it is essential that people retain meaningful avenues for challenging harmful outcomes.

“As this technology gets integrated into agencies and institutions, we want to know that private individuals and groups can still go to the courts and be afforded the process they would receive if AI weren't involved,” said Ismael. “Otherwise, there can be a perception that anytime AI is involved, individuals are disempowered to contest decisions that affect their lives.”

The authors hope their work will encourage further discussion among legal scholars, policymakers, judges, and practitioners about how courts should handle evidence disputes involving AI systems. Cen said one of the team's long-term goals is to see the framework tested and refined through engagement with attorneys and courts handling real-world cases.

“When it comes to the paper, the next question is whether something like this could see the light of day in court,” said Cen. “Our hope is to make it a reality, to instantiate it, and to continue improving it as we learn from real-world experiences.”