The AI You Didn’t Approve: The Auditor’s Role in Managing Hidden Vendor Risk

Key Takeaways

• The most immediate AI exposure is often the AI an organization never chose: shadow AI adopted by employees and AI features that approved vendors add to products already under contract
• Outsourcing a process does not outsource accountability. If a vendor’s AI produces a discriminatory, false, or non-compliant outcome, the organization may still answer for it, and many legacy contracts say nothing about AI use, training, or human review
• “Fourth-party” risk is a growing blind spot. AI features arriving through routine software releases can introduce a model provider the organization has no direct contract with, and limited visibility into
• Sound AI vendor risk management treats AI as a material vendor risk factor, not a product feature. This requires new inventories, contract requirements, and monitoring practices, along with coordination across legal, IT and risk to manage exposure the organization did not intentionally take on

How Unapproved AI Enters the Organization

Most conversations about artificial intelligence (AI) governance begin with what an organization intentionally built, purchased or approved. The more immediate exposure may be the AI it never consciously chose.

AI is arriving through software updates, vendor platforms, and outsourced services. It may summarize ERP transactions, rank candidates, flag compliance activity, or draft customer communications. The vendor stack has become an AI stack, often without a distinct procurement decision, risk assessment or clear owner.

Risk becomes most consequential when normalized. Shadow AI and embedded vendor AI are converging into one governance gap: employees adopt unapproved tools while approved vendors add AI to products already under contract. The auditor’s role is to make that exposure visible before it becomes a liability.

Two Categories of Compliance Exposure Organizations Are Missing

The first category is AI acting on the organization’s behalf. A vendor may use AI to screen applicants, review transactions, detect fraud, or prepare regulatory information. The organization may never see the model, its training data, or decision logic. Yet outsourcing a process generally does not outsource accountability. If the vendor’s AI produces a discriminatory outcome, false result or regulatory violation, the organization may still have to answer for it. Many legacy contracts, especially those negotiated before generative AI became mainstream, say little about AI use, training, human review or harmful outcomes.

The second category is AI embedded in existing infrastructure. ERP systems, financial platforms, HR applications, and compliance tools increasingly receive AI functionality through routine releases. These features may process sensitive information, influence decisions or introduce a model provider with which the organization has no direct contract. This “fourth-party” exposure leaves limited visibility into the model and data practices beneath a vendor’s product.

Agentic AI: When the Risk Moves Faster Than the Policy

Agentic AI is not simply a more capable chatbot. An agent can plan, make decisions, interact with systems, and take action with limited human input. FINRA’s 2026 oversight report highlights autonomy, scope and authority, auditability, transparency and data sensitivity, and points firms toward human-in-the-loop controls, action tracking and guardrails.

This breaks a basic assumption in many control frameworks: that a person approved the final decision. When an agent initiates a payment, changes a record or responds to a customer, accountability may be difficult to trace. Without logs showing what it accessed, did and was authorized to do, an organization may be unable to verify compliance or investigate an incident.

Key risk and compliance questions for agentic systems:

• Does this agent produce a traceable log of its reasoning, not just its outputs?
• Are there meaningful human-in-the-loop checkpoints, and are they actually being used?
• What happens when the agent encounters a scenario outside its training parameters?
• Is agent behavior subject to change management, version control and documented review?

The Five Questions Risk Advisors Should Be Asking, and Aren’t

1. What AI is the vendor deploying inside the services it performs for us?

Procurement and risk teams need to know where models are used, what decisions they influence, and whether that use is disclosed in the contract.

2. How was the AI trained, and on whose data?

Training practices are where privacy, bias, and intellectual property exposure often begins. Organizations should know whether their information may be retained or used to improve a model and what restrictions apply to regulated data.

3. What human oversight governs the vendor’s AI outputs?

A SOC 2 report or ISO/IEC 27001 certification may provide useful evidence about controls and information security, but neither, by itself, shows how a model is governed, tested or subjected to human review. SOC 2 addresses security, availability, processing integrity, confidentiality and privacy; ISO/IEC 27001 addresses information security management systems. Standards such as ISO/IEC 42001 are designed to address AI management specifically.

4. What compliance and audit rights does the contract provide?

Can the organization obtain information about model changes, testing, incidents, sub-processors, and control failures? A general audit clause may not provide meaningful access to AI-specific evidence.

5. If the AI causes harm, who is responsible?

Indemnities matter, but they do not eliminate regulatory or operational consequences. The organization should know who owns the decision, preserves evidence, notifies affected parties and funds remediation.

What Sound Risk Management Looks Like in Practice

A mature posture treats AI as a material vendor risk factor, not a product feature. AI-specific questions should be built into onboarding and reassessment. Contract templates should require disclosure of material AI use, training and data practices, human oversight, incidents, and significant model changes.

Organizations also need an AI vendor inventory identifying which vendors use AI, in what capacity, within which processes and with access to which data. FINRA’s 2026 report calls for ongoing due diligence, assessment of vendors’ GenAI use, inventories of vendor technology and firm data, and consideration of fourth-party risk.

Point-in-time reviews cannot keep pace with changing models and features. Continuous monitoring should include change notifications, release-note reviews, incident monitoring, periodic attestations and reassessment when a vendor expands an AI capability.

This work must cross functions. Legal understands contractual rights and liability. IT understands architecture, access, and data flows. Risk and audit assess governance, evidence, and control effectiveness. Effective programs coordinate all three while preserving the auditor’s independence.

The Regulatory Tailwind, and Why Waiting Is a Risk Decision

Regulatory expectations are becoming more concrete. Under the EU AI Act, specified transparency obligations apply beginning August 2, 2026, and the law can reach providers outside the EU when their systems’ outputs are used there. A May 2026 provisional Digital Omnibus agreement delays the Article 50(2) marking obligation for certain pre-existing generative-AI systems to December 2, 2026, and the EU has adopted later application dates of December 2, 2027, and August 2, 2028, for categories of high-risk systems, confirming that implementation is phased.

In the United States, FINRA’s 2026 report signals the direction for securities firms: existing obligations continue to apply when GenAI is used, and firms should maintain documented governance, testing, monitoring and vendor due diligence.

The organizations most exposed are not always those with the worst conduct. Often, they are the ones that cannot demonstrate what they knew, who approved it, how it was monitored, or what happened when a control failed.

The Risk Advisor’s Role in an AI-Driven World

Independence, professional skepticism, and the ability to identify what others have normalized have always defined effective auditors and risk advisors. AI environments demand those same capabilities, applied not only to processes, but to systems that act within them.

“Who is accountable, and how would we know?” is a familiar audit question. AI cannot answer it for itself. The advisors who build this capability now will help define responsible AI risk management for the next decade.