Words wrong per 100 words spoken
Lower is better. Same 22,921 recordings for every model.
- STORM ASR 2.0
- SBPN Large
- Meta MMS-1B
Our most accurate speech recognition for Nigerian languages yet, tested against the leading Nigerian speech recognition models on the same 22,921 recordings in Yoruba, Hausa, Igbo, Nigerian Pidgin, Nigerian English and code-mixed speech.
Lower is better. Same 22,921 recordings for every model.
Words wrong per 100 words spoken. A wrong, missing or extra word each count as one error. Best result in each row highlighted.
| Language | Recordings | STORM ASR 2.0 | SBPN Large | Meta MMS-1B | NCAIR |
|---|---|---|---|---|---|
| Nigerian English | 1,188 | 4.5 | 9.0 | 19.3 | 12.1 |
| Hausa | 3,615 | 11.9 | 18.1 | 28.2 | n/a |
| Igbo | 4,662 | 22.2 | 28.7 | 52.3 | n/a |
| Nigerian Pidgin | 1,369 | 14.2 | 15.5 | 50.4 | n/a |
| Yoruba | 2,499 | 19.4 | 24.5 | 51.6 | 44.3 |
| Code-mixed (Yoruba + English) | 9,588 | 14.8 | 25.1 | 69.0 | 58.0 |
| All languages | 22,921 | 15.6 | 22.6 | 51.1 | n/a |
Letters wrong per 100 letters. A missing or wrong tone mark counts as a wrong letter, so this shows how well each model writes Yoruba, Igbo and Hausa as they are really written.
| Language | Recordings | STORM ASR 2.0 | SBPN Large | Meta MMS-1B | NCAIR |
|---|---|---|---|---|---|
| Nigerian English | 1,188 | 1.9 | 3.5 | 6.8 | 6.1 |
| Hausa | 3,615 | 3.2 | 4.7 | 7.4 | n/a |
| Igbo | 4,662 | 8.0 | 10.1 | 17.6 | n/a |
| Nigerian Pidgin | 1,369 | 7.4 | 7.6 | 21.4 | n/a |
| Yoruba | 2,499 | 7.7 | 9.8 | 18.8 | 20.1 |
| Code-mixed (Yoruba + English) | 9,588 | 5.3 | 11.1 | 27.4 | 36.4 |
| All languages | 22,921 | 5.6 | 8.7 | 18.7 | n/a |
Recordings from the test set, as each model wrote them.
Babangida ló hand over power fún interim government
Babangida ló hand over power fún interim government
babangida ló wan overpower fún interim government
Àlùfáà ìjọ aṣọ ara kan ṣe ìṣirò ìgbà tí Jésù yóò dé.
Àlùfáà ìjọ aṣọ ara kan ṣe ìṣirò ìgbà tí Jésù yóò dé.
àlùfáà ìjọ aṣọ ara kan ṣe ìṣe èrò tí jésù yóò dé
a naghị ekwe ka nwaanyị di ya nwụrụ rie nri e siiri ndị ezinụlọ ya
A naghị ekwe ka nwaanyị di ya nwụrụ rie nri e siiri ndị ezinụlọ ya
anaghị ekwe ka nwaanyị di ya nwụrụ rie nri e siri ndị ezinụlọ ya
Chicken wey run way from Borno go Ibadan go still end up inside pot of soup
Chicken wey run way from Borno go Ibadan go still end up inside pot of soup
chicken wey run away from borno go ibadan go still end up inside port of soup
First, Hurricane Rosa's path leads to landfall across the Baja California peninsula of Mexico.
First, Hurricane Rosa's path leads to landfall across the Baja California peninsula of Mexico.
first hurricane rose's path leads to landfall across the baha california peninsula of mexico
Each was run from its published weights with its default settings.
Files and real-time streams, no language setting needed, one API key.