Kaldi models testing

Many models and datasets become available recently, testing models against datasets becomes more complicated and in the same time more fun.

Recenly Kaldi Active Grammar Project released some new models and did some testing of our Vosk models, so I had to verify everything since the numbers we see and the numbers reported mismatch as usual.

So I tested the following models:

There is also lookahead model from kaldi-active-grammar which can quickly rebuild the graph:

Kaldi-active-grammar model graph was recompiled to use the same language model as en-us-aspire, our big en-us language model one so that more direct comparision is possible.

For testing I used the following datasets:

Vosk library was used for testing, see the testing script here in our repo. Here is the results I got:

Model Librispeech Tedlium Commands Fisher
en-us-aspire 13.49 12.53 55.62 17.39
en-us-daanzu 8.36 8.68 9.30 31.37
en-us-small 15.34 12.09 45.52 N/A
en-us-librispeech 4.37 N/A N/A N/A
deepspeech 6.12 18.03 N/A N/A

Some thoughts on the results: