Frequently Asked Questions About AI Oversight and Liability
As companies across sectors continue to add AI tools to their workflows both internally and externally — granting AI agents access to critical internal systems and allowing them to interact with customers — data security and privacy risks are growing. Below are some answers to frequently asked questions that arise in managing these risks.
Q: What are the most important types of AI oversight companies should consider?
A: Organizations should create protocols around 1) documenting data that has been input into an AI tool, 2) keeping a “human in the loop” with authority to override AI decisions, 3) testing tools against known outcomes before relying on it, 4) performing strong vendor diligence on security and data use, and 5) maintaining an audit trail showing what the model recommended and what the human decided. Each of these protocols requires careful consideration and planning, but the goal is to always be able to reconstruct the human reasoning behind a decision.
Q: Why does documentation matter so much if an AI-related dispute or investigation arises?
A: Thorough documentation functions as an organization’s primary line of defense if a dispute or regulatory inquiry occurs. If AI-related litigation or a regulatory investigation arises, having detailed records of what data was collected, why particular decisions were made, and how AI outputs were used becomes critical, alongside having an incident response process in place for AI-specific events such as a data leak or an unexpected model output. This record-keeping is what allows an organization to later reconstruct and defend the reasoning behind a given decision.
Q: When an AI system makes a mistake, who is liable?
A: Responsibility does not shift to the technology simply because it generated the output. Supervising professionals remain liable for AI-generated errors, which can quickly become consequential. Unlike a single human error, an AI agent’s mistake can repeat itself thousands of times in seconds, potentially causing widespread harm. For now, legal questions about liability in these situations are being evaluated through existing, established areas of law rather than an entirely new legal framework, such as product liability laws, consumer protection laws, data privacy and security laws, and negligence. That could change as new regulations are introduced at the federal and state levels in coming years.
Q: What should organizations look for when contracting with AI vendors?
A: Strong vendor contracts are a key defense against liability exposure. Agreements should specify that vendors will not use the organization’s data to train their own models, include data isolation provisions so information is not commingled with other customers on the same platform, require deletion of data if the relationship ends, and spell out liability and indemnification terms in case something goes wrong. Negotiating these protections up front is typically more effective than trying to allocate blame after an incident has already occurred.
Q: What is the role of employee training in AI risk management?
A: Because AI tools and the risks they introduce keep evolving, comprehensive, ongoing employee training is critical. This preparation may cover the use of the tools themselves, the ethical issues involved, best practices for protecting confidential information, secure data handling, and other topics. Training must constantly evolve as organizations reassess new AI tools for accuracy, privacy safeguards, and ethical risks.
Facing questions about safe implementation of AI tools in your organization? Reach out to Katie Nadro or another member of LP’s AI and Technology Team.