Meeting Tools Lab

Some links are affiliate links. They never change a result. How we make money

AI note takers with background noise: how much accuracy they lose

Tested · Last updated · By Miglė

With steady background noise added to a 17-minute meeting, every AI note taker we tested got 2 to 3 times more words wrong. tl;dv held up best (35.6% word errors, up from 13.8% on the clean recording), followed by Otter (38.4%), Granola (39.2%) and Fireflies (42.5%); Fathom lost the most words, at 52.4%. The extra errors were mostly whole passages left out, not misheard words.

Results at a glance

Tested on 16 November 2026, all five on free plans, in one Google Meet call. The clean numbers are from our round 1 test on the same meeting.

Tool Word error rate, clean Word error rate, with noise Summary items caught, clean (of 13) Summary items caught, with noise (of 13)
tl;dv 13.8% 35.6% 9 8
Otter 12.9% 38.4% 8 7
Granola 13.5% 39.2% 11 8
Fireflies 14.7% 42.5% 11 7
Fathom 26.8% 52.4% 9 9

Lower word error rate is better: 35% means roughly one word in three was wrong, missing or invented.

Which AI note taker handles background noise best?

tl;dv, by a small margin. It had the fewest word errors with noise (35.6%) and lost the fewest words: its transcript shrank from 2,282 words to 1,866, against a human transcript of 2,414. Otter, which made the fewest errors on the clean recording, came second at 38.4%, and Granola was close behind at 39.2%. The gap between the top three (3.6 points) is small enough that a different recording could change their order.

Fireflies dropped to 42.5% and Fathom to 52.4%. Fathom was already the least complete on the clean recording; with noise its transcript was 1,445 words, about 60% of what was said.

What the noise did to the transcripts

The tools did not mostly mishear words; they dropped them. Missed words rose from 159–473 per tool on the clean recording to 590–987 with noise, while wrong words rose much less (from 117–142 to 124–259). Whole stretches went missing: Ian’s introduction, where he draws a whale, appears in four of the five clean transcripts and in none of the noisy ones.

The numbers were not safe either. With noise, Fathom wrote a profit aim of “60 million” euro and a production cost of “12, 15 euro” (it is 12.50). As on the clean recording, no tool wrote the profit aim correctly: the speaker says fifteen million, and the others wrote 50 million.

Did the summaries suffer as much?

Less than the transcripts. The summaries lost between none and four of the 13 decisions and action items on our checklist. Fathom’s still caught 9, as on the clean recording, despite having the worst transcript. Granola and Fireflies, the two most complete summaries on the clean recording (11 each), fell to 8 and 7.

Wrong details crept in. Fathom’s summary gave a “€60M profit” target and called Alima the “Cultural Designer”. Fireflies’ summary introduced a “Sam” who is not in the meeting. tl;dv named the industrial designer “Elena”. Granola listed Jessie as the user interface designer; it was Ian. If your meetings are noisy, check names and numbers in any summary before you send it on.

How we tested

We took the same recording as in round 1, meeting ES2008a of the AMI Meeting Corpus (CC BY 4.0, AMI Consortium / University of Edinburgh), and mixed in steady synthetic pink noise, a broadband hiss like a fan or a busy room, at about 8 dB below the speech. The length and the words did not change, so the same human transcript and checklist score both versions. We played the noisy file once into one Google Meet call from a Chrome tab, with Meet’s noise cancellation turned off so the tools heard the noise as recorded; Otter, tl;dv and Fireflies joined as bots, and Fathom and Granola recorded on the same Mac. Exports were scored unedited, except for lines that Fathom and Granola picked up from our own microphone, which we removed. Full details: methodology.

Limits: one meeting, one kind of noise at one level. Real noise (voices, keyboards, traffic) changes over time and may affect the tools differently. Fireflies ran on a new free account, because the free plan on our first account had run out.

Think a number here is wrong, or a tool has changed since the test? Tell me and I will rerun it and publish what changes.

Sources