AI vendor risk assessment: what to check

Why vendor checks matter for AI

Most of the AI in your business comes from vendors, not code you wrote. That makes vendor checks the front line of AI governance. Before you adopt a tool that touches your information, a short assessment tells you whether it is safe to bring in, and on what terms.

AI tools raise questions ordinary software doesn’t. The big one is training: some vendors use what you send to improve their models, which can expose client and company information. Many tools also sit on a chain of sub-processors and underlying models, and the data terms are often buried.

What to check

• Data use: does the vendor train on your inputs, can you turn that off, where is information stored, and for how long?
• Security: is there an independent security report or certification you can review, and have there been recent incidents?
• Sub-processors and models: which third parties and underlying models sit behind the tool?
• Contract terms: is a data processing agreement available, and what do liability and indemnity look like?
• Privacy commitments: what does the vendor commit to about personal information, and does that fit your client contracts?
• Reliability and exit: what uptime is promised, and can you export your information and leave cleanly?
• Model changes: does the vendor document model behavior, updates and safeguards?

Score it, then decide

Rate each area high, medium or low risk with a short note, then make an overall call: adopt, adopt with conditions, or don’t adopt. Conditions might be turning off training, a signed data processing agreement, or limiting the tool to non-confidential information. Record the decision and revisit it when the tool or your use of it changes.

A worked example: scoring a meeting summary assistant

Meridian Creative, a fictional 42-person agency, wanted a meeting summary assistant for internal planning meetings. Before adopting it, the operations lead read the vendor’s terms, privacy page and security page, and scored each area. The findings below come from fictional vendor documents.

The overall call was adopt with conditions: training turned off, internal planning meetings only (no client meetings, HR matters or personal records), 30-day retention, and a witnessed deletion test by October 15. Each condition is something a person can check, and the open item has a date.

That decision went into the agency’s tool list with the operations lead as owner. When a client later asked whether Meridian’s AI tools train on client information, the answer was ready: this tool doesn’t, the setting is off, client meetings are excluded, and here is the date of the review.

Had training been impossible to turn off, the answer would have changed. Meridian would either have limited the tool to information it was comfortable sharing with the vendor, or declined it and looked for another.

Red flags that should pause an adoption

• The vendor can’t say in writing whether it trains on your inputs.
• There is no way to delete your information, or deletion takes an unspecified time.
• The tool asks for broad access to email, files or calendars when the use needs much less.
• Terms let the vendor change data use without telling you.
• Nobody on your side can say who will own the tool once it’s adopted.

A red flag doesn’t always mean no. It means the decision needs a condition you can check, or a more senior reviewer, before the tool goes in.

Keep it light for low-risk tools

A tool that only ever sees public information, such as a grammar checker used on published copy, doesn’t need a six-area review. Record the tool, its owner and the information it is allowed to see, and move on. Save the full check for tools that touch client, personal or confidential information.

Make it part of buying

The check only works if it happens before adoption, not after. Put it in the path every new AI tool has to pass through, alongside a simple approval, so every tool you adopt has an owner and a recorded decision from day one. That single step prevents most shadow AI problems.

Using your AI assistant

An AI assistant can pull the answers out of long vendor terms quickly, if it quotes the source for each answer and says when the documents are silent. Check the quotes yourself before you rely on them.

Using only the vendor documents I paste below, answer these questions and quote the sentence that supports each answer: Does the vendor train on our inputs, and can we turn that off? Where is data stored, and for how long? Which sub-processors or underlying models are named? Is a data processing agreement offered? How do we export our data and leave? If the documents don't answer a question, write NOT STATED. Do not use general knowledge about this vendor.

Vendor documents:
[paste the terms, privacy policy and security page]

Using the answers above, draft a short decision for our records: adopt, adopt with conditions, or do not adopt. List each condition (for example, training turned off, or no client information) and the questions still open. Mark it DRAFT FOR REVIEW. Do not state that anything has been approved.

Common questions

Related guides

• How to write an AI acceptable use policy
• AI risk assessment template: what to assess
• How to answer the AI section of a security questionnaire

Extrasphere Media LLC. Content for general informational purposes, not legal advice and not a substitute for qualified counsel. Independent and not affiliated with ISO, the IEC, NIST, or the European Union.