Quick Guide · Buying well
How to Prepare for an AI Supplier Demonstration
Bring awkward business cases, ask for evidence and record what the demonstration did not prove.
The idea to keep
A smooth demo proves very little. Test the conditions that decide whether the tool is usable.
Best for
- Shortlisting AI suppliers
- Fair product comparisons
- Testing support and export claims
What you need
- Awkward scenarios
- Evidence questions
- A scorecard and decision owner
Test the awkward case first
1
Define outcome and failure
Write what success looks like and what would make the tool unsuitable.
2
Bring realistic edge cases
Use contradictory policy, missing sources, unsupported citations and sensitive-data examples.
3
Ask evidence questions
Ask about sources, privacy, export, limits, support and incident handling.
4
Record proved and unproved
Capture the result, follow-up evidence and decision owner.
Example
Supplier demo scorecard
Scenario
Summarise a policy with one exception and cite the source.
Observed
Clean case worked; exception was omitted.
Evidence requested
Source trace, export, support and data terms.
Decision note
Do not shortlist until gaps are answered.
Check before you use it
- Scenarios resemble real work.
- Failure conditions recorded.
- Claims have evidence owners.
- Unproved points visible.
Common mistake
Letting the supplier choose only the easiest scenario.
Nova9 view
Buy evidence, not theatre
The best demonstration makes limitations visible early enough to change the decision.
See the NCSC secure AI guidance.
