We count every word the AI note taker gets wrong.
Short answer: on a 17-minute meeting, Otter got the fewest words wrong (12.9%), with Granola (13.5%) and tl;dv (13.8%) almost the same. Granola and Fireflies caught the most decisions and action items (12 of 13).
What was saidRasa will tell the support team about the new date by Thursday.
-
Otter
wrote Rosainstead of Rasa will tell the support team about the new date by Thursday. 1 of 12 wrong -
Fireflies
wrote Rezainstead of Rasa will tell the support team about the new date by Thursday. 1 of 12 wrong -
Granola
wrote Russoinstead of Rasa will tellwrote thatinstead of the support team about the new date by Thursday. 2 of 12 wrong -
tl;dv
wrote Risainstead of Rasa will tellwrote thatinstead of the support team about the new date by Thursday. 2 of 12 wrong -
Fathom
wrote Rosainstead of Rasa will tell left out: the support team about the new date by Thursday. 10 of 12 wrong
That is the whole method. On all 272 words of that recording, Otter and Granola got 6.6% wrong and Fathom 12.9%. Every tool wrote the colleague's name Rasa as something else.
- Every tool hears the same recordings
- A script counts the errors
- Every number shows its test date and plan
- Commissions never change a result
All five, side by side
| Tool | Word error rate, meeting | Word error rate, non-native speaker | Summary items, meeting | Summary items, non-native | Cheapest paid plan, a month |
|---|---|---|---|---|---|
| Otter | 12.9% | 6.6% | 9 of 13 | 2 of 8 | $8.33 a user, billed yearly |
| Granola | 13.5% | 6.6% | 12 of 13 | 5 of 8 | $14 a user |
| tl;dv | 13.8% | 9.2% | 10 of 13 | 5 of 8 | €18 a seat, billed yearly |
| Fathom | 26.8% | 12.9% | 10 of 13 | 5 of 8 | $16 |
| Fireflies | not scored | 10.3% | 12 of 13 | 5 of 8 | $10 a seat, billed yearly |
Round 1, , all on free plans. Lower word error rate is better; the highlight marks the best result in a column. Fireflies' free plan hid most of one transcript, so its meeting transcript is not scored. Prices from each vendor's pricing page on the test date.
Why each number is what it is: the full test.
Which one is best for you
-
Fewest words wrong
Otter
12.9%of words wrong on the meeting
6.6% on the non-native recording. Granola was almost the same (13.5% and 6.6%). Otter's summaries caught the fewest items (9 of 13).
-
Most complete summaries
Granola and Fireflies
12 of 13decisions and action items caught
Granola also records on your Mac, so nobody sees a bot join the call.
-
Most generous free plan
Fathom
Unlimitedrecordings and transcription, free
But its transcript was the least complete in our test: 26.8% of words wrong, mostly words it left out.
Not sure? The full test says who should pick which, with prices.
How we test
Play the same recordings
A 17-minute public meeting with a human-made transcript, and a 2.5-minute script read by a non-native English speaker. Same audio, same call, same day.
Export everything unedited
Each tool's transcript and summary is copied exactly as the tool produced it.
Let a script count the errors
Word error rate is wrong, missed and added words divided by the words in the human transcript.
Check summaries against a checklist
Before the test we list each recording's decisions and action items (13 and 8). A summary scores one for each it contains.
Questions people ask
Which AI note taker is the most accurate?
In Round 1, Otter got the fewest words wrong on a four-person meeting (12.9%), with Granola (13.5%) and tl;dv (13.8%) almost the same. Fathom got 26.8% wrong.
What is word error rate?
The share of words a transcript gets wrong, compared with a human transcript of the same recording: wrong, missed and added words divided by the words in the human transcript. Lower is better. 10% means about one word in ten.
Do you get paid by the tools you test?
Some links will become affiliate links once a program approves us (none has yet): if you sign up through one, the vendor may pay a commission at no extra cost to you. Results come from the measurements, and a tool that pays nothing still wins when the numbers say so.
How often are the tests repeated?
Every 3 to 6 months, because these tools change fast. Every number on the site shows the date of the test it comes from.
Coming in a later round
The same tools on a noisy version of the same meeting. Every result carries its own test date, so you can see what changed.