AI meeting notes test kit: one meeting, 11 traps

This kit lets you run the same test I ran on ChatGPT's Meetings plugin: one constructed project meeting, eleven traps, and an answer key written before the meeting was recorded. Play the recording into whatever note taker you use and see how many of the eleven it gets. Everything below is free, with no sign-up.

What is inside

The recording (MP3, 8 minutes, 11.5 MB). A weekly cutover checkpoint at Ashworth Logistics, a made-up company. Four speakers, one absent sponsor.
The meeting script (PDF). Every line, with speaker names and times from the MP3, so you can check your notes against what was said.
The answer key (PDF). The eleven traps with a pass, partial and fail rule for each, a score sheet and the four checks. Open it after your notes are written.

Who is this for

Anyone who has been told to let an AI take the minutes. Project and programme managers, PMO teams, team leads, anyone who chairs a weekly checkpoint and chases the actions afterwards. You don't need to know how the tools work. You need to know what was agreed.

What problem does it solve

AI meeting notes read well, and that's the risk. The summary sounds right, so the action list goes into the tracker unchecked. This meeting is built the way real ones go. Someone offers to do a job and gets overruled. A date is corrected in the same breath. A cost is queried: per weekend, or total? Two people talk over each other. Someone says "don't minute this". The chair's recap leaves an action out. Each of those has a right answer, so you can see exactly where your tool slips.

How to use it

Don't give the tool the script. The test is the audio.

Start your note taker as you would for a real call.
Play the recording all the way through, on your speakers or shared into a test call with yourself.
Let the tool write its notes. Don't edit them and don't re-run.
Open the answer key and score what it wrote. One mark for a pass, half for a partial.
If the tool lets you ask questions about the meeting, ask for every action with its owner and due date. Score that answer separately, so you know what you get without it.

The four checks

Rebuild the action list yourself. Every action needs an owner and a date. If one hasn't got both, that's a question for the meeting, not the tool.
Check every number and date against a recording or notes you keep. Some tools delete the audio once the notes are written. Get permission before you record anyone.
Check every decision. Was it decided, conditional or parked?
Check what shouldn't be there: anything said off the record, anything about a person, anything that would embarrass someone if it got forwarded.

What it will not do

It won't tell you which note taker to buy. The voices are AI-generated, so the audio is cleaner than a real call: one clear voice each, nobody on a bad microphone. A tool that scores well here may do worse on a noisy real meeting. What it will show you is how a tool handles owners, dates and decisions when you know the right answers.

Ashworth Logistics, Finity and everyone in the meeting are fictional. The voices were generated with ElevenLabs.