How to Assess a Vendor's AI and ML Usage for Risk: A Practical Framework

Why AI Adds a New Risk Dimension

When your vendor deploys AI or machine learning systems that process your data or affect your customers, you inherit a set of risks that traditional TPRM questionnaires do not adequately cover. AI introduces risks around bias and discrimination, opaque decision-making, training data privacy, model drift, and regulatory non-compliance.

The Cisco 2026 Data Privacy Benchmark Study found that 23% of organizations lack an AI governance committee, and Gartner predicts that by 2030, fragmented AI regulation will extend to 75% of the world's economies, driving over $1 billion in compliance spend. If your vendor uses AI without adequate governance, you bear the regulatory and reputational risk.

AI-Specific Risk Categories

Training Data Risks

• Was personal data used to train the model? If so, was consent obtained?
• Is training data representative, or could it introduce bias?
• Can the vendor demonstrate data provenance for training datasets?
• How is training data retained, and can it be deleted upon request?

Model Transparency

• Can the vendor explain how the model makes decisions (explainability)?
• Is the model a "black box" or does it support interpretable outputs?
• Can the vendor provide documentation on model architecture, inputs, and outputs?

Bias and Fairness

• Has the vendor tested the model for bias across protected characteristics (age, gender, race, etc.)?
• Does the vendor have a fairness testing and remediation process?
• Are bias testing results available for review?

Accuracy and Drift

• How does the vendor monitor model performance over time?
• Is there a process for detecting and correcting model drift?
• What is the model's error rate, and how is it measured?

Assessment Framework

Add these AI-specific questions to your vendor assessment for any vendor using AI/ML on your data:

1. AI inventory: What AI/ML systems are used in the services provided to us?
2. Data usage: Is our data used to train, fine-tune, or improve AI models? Can we opt out?
3. Human oversight: Are AI decisions subject to human review, especially for consequential decisions?
4. Incident history: Has the AI system produced harmful, biased, or incorrect outputs? What was done?
5. Regulatory compliance: Is the vendor compliant with applicable AI regulations (EU AI Act, state AI laws)?
6. Third-party audits: Has the AI system been audited by an independent party for bias, accuracy, or security?

Contractual Protections

Supplement your assessment with contractual clauses specific to AI:

• Prohibition on using your data for model training without explicit approval.
• Right to audit AI models and their outputs.
• Notification requirements for material changes to AI systems.
• Liability allocation for AI-caused errors or harms.

Jerisaliant's TPRM module includes AI-specific assessment templates, automated risk scoring for AI vendors, and clause libraries for AI-related contractual protections.