Meeting Tools Lab

We tested 5 AI note takers on the same meeting

Tested · Last updated · By Miglė

On a 17-minute four-person meeting, Otter made the fewest word errors (12.9%), with Granola (13.5%) and tl;dv (13.8%) almost the same and Fathom far behind at 26.8%. On a short recording by a Lithuanian speaker of English, Otter and Granola tied at 6.6%. Granola and Fireflies caught the most decisions and action items in their summaries (12 of 13).

Results at a glance

Tested on 15 November 2026, all five on free plans.

Tool Word error rate, 4-person meeting Word error rate, non-native speaker Summary items caught, meeting (of 13) Summary items caught, non-native recording (of 8)
Otter 12.9% 6.6% 9 2
Granola 13.5% 6.6% 12 5
tl;dv 13.8% 9.2% 10 5
Fathom 26.8% 12.9% 10 5
Fireflies not scored (see below) 10.3% 12 5

Lower word error rate is better. A word error rate of 10% means roughly one word in ten was wrong, missing or invented. Fireflies’ free plan hid most of the transcript of the second recording, so it is scored on its summary only.

Transcript accuracy

The four-person meeting is the first scenario of the AMI Meeting Corpus (ES2008a): about 17 minutes, 2,414 words in the human reference transcript. Three tools finished within one point of each other: Otter 12.9%, Granola 13.5%, tl;dv 13.8%. Those differences are small enough that a different recording could change their order.

Fathom is the clear outlier. Its errors were mostly missed words (473 of them, against 159 to 177 for the other three): it left out whole exchanges, for example Ian’s question “are we designing a remote control for television only?”, and most short replies. Everything it did write was readable; there was just less of it.

Two caveats about the numbers. Part of every tool’s score is the people talking over each other in a real meeting, which is why even the best result is not close to zero. And at one place the human reference says the profit aim was “fifteen million” euro, while all five tools wrote “50 million”. We have not settled which is right, so we left the reference unedited (each tool loses the same one word) and counted either a “15 million” or a “50 million” summary as correct for that checklist item.

Non-native accents

The second recording is 2.5 minutes of a scripted weekly meeting read by a Lithuanian speaker of English, quietly and with an accent. Otter and Granola both scored 6.6%, tl;dv 9.2%, Fireflies 10.3% and Fathom 12.9%.

Numbers and everyday words were mostly fine. Names were not. The speaker’s colleagues are called Rasa, Ieva and Tomas; the tools wrote “Rosa”, “Russo”, “Reza”, “Risa” and “Java” for them, and Fireflies turned “the bugs” into “the box”. Fathom’s transcript began 1 minute 24 seconds in and skipped the end of the first topic. One speaker and one recording tell you about this voice, not about Lithuanian accents in general.

Summaries: what they caught and missed

Before the test we wrote a checklist of the decisions and action items in each recording, then counted how many each summary contained (reworded items count; we checked the borderline ones by hand).

  • Meeting (13 items): Granola and Fireflies caught 12, tl;dv and Fathom 10, Otter 9. Everyone missed “next meeting in 30 minutes”. Otter’s action items were generic (“Clarify remote control scope”) with no names and no mention that Rose would write the agenda. Fathom’s summary listed only two action items, both attributed to “Speaker 2”, and named the wrong person as the interface designer. tl;dv gave action items with owners and timestamps.
  • Non-native recording (8 items): four tools caught 5; Otter caught 2, because its action items list was empty and its summary left out who does what.
  • Invented or wrong details: Fireflies’ summary of the accented recording had “Russell” and “Reza” for the support contact and “Java” for the designer; Fireflies also wrote “Initial release move: March 21” for a decision about 15 February to 1 March. Check names and dates in any AI summary before you send it on.

What the plans cost

Prices from each vendor’s own pricing page on 15 November 2026, in the currency shown there. Meeting length limits and free-plan limits change often; check the page before you decide.

Tool Free plan, from the pricing page Cheapest paid plan
Fireflies Limited AI summaries and storage; in our test the transcript of the second recording was hidden after about 2.5 minutes with “0 free meetings left” Pro, $10 per seat per month billed annually ($18 monthly)
Otter 300 minutes a month, 30 minutes per conversation, 3 lifetime file imports Pro, $8.33 per user per month billed annually
tl;dv Unlimited recordings; AI notes for up to 10 meetings Pro, €18 per seat per month billed annually
Fathom Unlimited recordings and transcription, AI summaries; works without a bot on Mac (beta) Premium, $16 per month
Granola AI notes with limited meeting history; Mac app Business, $14 per user per month

Fireflies ran out of free meetings after our first recording, so if you rely on it for more than a couple of meetings, plan on paying. tl;dv’s free plan counts AI notes for 10 meetings; two of ours used up four. Fathom offered a Premium trial at sign-up. No trial confirmation email arrived, so we believe the test ran on the free plan, but we did not check the plan screen on the test day.

Which one should you pick?

  • Fewest word errors: Otter (12.9% on the meeting, 6.6% on the accented recording), with Granola almost the same. Visit Otter
  • Most complete summary: Granola and Fireflies (12 of 13). Visit Granola · Visit Fireflies
  • Best free plan: Fathom, going by its plan limits: unlimited recordings and transcription. Its transcript was the least complete in our test. Visit Fathom
  • Best without a bot in the call: Granola. It records on your Mac and nobody sees a notetaker join.
  • Best for non-native speakers: Otter and Granola tied in our test; check names in the output either way.
  • Action items with owners and timestamps: tl;dv gave both in our test. Visit tl;dv

How we tested

Same recordings, same call, exports unedited, scored by script. Fathom and Granola wrote lines from our own microphone (“Thank you very much”, “No, Mom, we’re not finished”) that were not in the audio under test; we removed those before scoring and kept them in the notes. Full details and limits: methodology.

Think a number here is wrong, or a tool has changed since the test? Tell me and I will rerun it and publish what changes.

Sources