Buying an AI Tool
AI tool data privacy checklist for a small business: the trust criterion
Quick answer: trust has two halves. Accuracy on your job: on ten real outputs, how many contained an error you would be embarrassed by, and are the cited sources real when checked. And data handling: read the data processing terms, not the marketing page, and confirm whether training on your inputs is off by default, whether deletion actually deletes, where the data is stored, and what happened the last time the vendor changed terms. Twelve questions, weight four.
4 out of five.
4 out of five.
What trust means here
Trust is the criterion that replaces exit in this method. On most software decisions, the cost of leaving is high enough that the exit terms deserve their own weight. AI tools are different. Switching is cheap, most plans are monthly, and the data you put in is usually a copy. So exit becomes a gate, checked once, and the weight of four goes to the thing that actually varies between AI tools: whether you can stand behind the output and whether you can explain to a client what the tool did with their file.
Both halves are scored. Accuracy from the trial. Data handling from the terms. Neither from the vendor's home page.
Ten outputs, checked by hand
Question one is the accuracy test. Take ten real outputs from the trial and check them against the source, by hand, the way a client would. Count the errors you would have been embarrassed by. Not typos. Wrong figures, invented facts, a clause that is not in the contract. Two in ten is a tool that needs a checker on every run. Zero in ten, on your inputs, is rare and worth the weight.
Question two is the citation test. If the tool shows where it got something, check that the source exists and says what the tool says it says. Invented sources that look real are a specific failure of this category of software, and a tool that does it once will do it in front of a client.
Questions nine and ten round out accuracy. When it does not know, does it say so or does it fill the gap? And does it behave the same on Monday as it did in the demo? Models change under the product. The vendor may not tell you.
Read the terms, not the landing page
The landing page will say your data is safe. The data processing terms say what happens to it. Read the terms. All of them. Then rate the four data questions.
The AI Tool Buyer's Playbook
Six gates. Four weighted criteria. Five tools. $49, one download.
Get the playbook · $49- Is training on your inputs off by default, or is it a setting under an admin menu the trial user never saw? Off by default is a five. Off only if the vendor turns it off for you is a one.
- Delete a conversation, a file and a user during the trial. Does it actually delete, and does the vendor say how long a deleted item persists?
- Where is the data stored, and does that matter to your clients or your regulator? For some firms it does not. For a firm handling health, legal or financial records it may be the whole decision.
- Does the vendor state which model the product runs on, and what happens to your outputs when that changes?
Then question eight: search the vendor's name with "price increase" and "terms change." What happened the last time they moved, and how much notice did existing customers get? A vendor's history with its customers is the best evidence of its future with you.
Anonymise before you test
The trial happens before the trust score exists, which means you are putting inputs into a tool you have not yet decided to trust. Handle that in order. Run the data gate first, from the terms. If it passes, use real files. If it is unclear, replace names, figures and identifiers in a copy of the real file and run that. The tool will do the job on the copy nearly as well as on the original, and the trust questions can be scored without a conversation with a client you would rather not have.
The two questions to ask out loud
Question eleven: who is the vendor's actual customer, you or your data? If the free tier is generous and the terms are quiet, the business model may be the second answer. Question twelve: could you explain to a client exactly what this tool did to their file? If the honest answer is no, the score is low regardless of how good the output was.
Scoring it
Average the twelve ratings, multiply by four, carry the score to the scorecard. A tool that scores well on the job and poorly on trust is a tool you will spend the checking time on that you bought it to save. Weight accordingly, and remember that the data gate already removed the worst of them before scoring began.
Signs to walk away
- Training on your inputs is on by default and the admin toggle is buried.
- The data processing terms cannot be found or produced on request.
- Cited sources do not exist or do not say what the tool says they say.
What to check yourself
- Take ten real outputs from the trial and check them against the source, by hand.
- Read the data processing terms, not the marketing page. All of them.
- Delete a conversation, a file and a user during the trial. Does it actually delete?
The series
- How to choose an AI tool for your business without buying five
- How to test an AI tool before buying: the job comes first
- Why employees don't use AI tools, and how to score for it before buying
- AI tool data privacy checklist for a small business: the trust criterion
- AI tool pricing, per seat vs usage: how to get to one annual number
Frequently asked questions
Is it safe to put client data into an AI tool?
It depends on the terms, not the tool. If training on inputs is off, retention is stated, storage location is stated and deletion works, many firms are comfortable. If any of those is unclear, use anonymised data or do not use the tool for that job.
Where do I find the data processing terms?
Usually a link in the footer labelled data processing addendum, DPA, privacy or trust. If you cannot find it, ask the vendor for it in writing. A vendor that cannot produce one has answered the question.
What does "training on your data" mean?
The vendor uses what you type or upload to improve the underlying model, which may then produce something derived from it for another customer. Most business plans allow this to be turned off. Check whether it is off by default or off on request.
How many errors are acceptable?
That depends on the job. A first draft that a person will edit tolerates more than a figure that goes to a client. Score against the job in the shape, not against an abstract standard.
What if the tool cites sources?
Check them. Open the source and confirm it says what the tool says. Cited sources that do not exist or do not support the claim are a low score on question two and a reason to check every output.
Does storage region matter for a Canadian small business?
For some clients and some regulators, yes. If your client contracts or your professional body require Canadian residency, it is a gate, not a criterion. If not, note it and move on.
Why does vendor pricing history belong under trust?
Because a vendor that raised prices sharply on existing customers with little notice has shown you how it treats customers. That is trust. It also affects the deal, but the behaviour is scored here.
Can I trust a vendor's security certifications?
They are evidence, not proof. A certification means an auditor checked a process at a point in time. It does not tell you whether training on your inputs is on. Read the terms as well.
The list is not the decision. It is how you find the one or two things that are.
