Models

We have two types of models - big and small, small models are ideal for some limited task on mobile applications. They can run on smartphones, Raspberry Pi’s. They are also recommended for desktop applications. Small model typically is around 50Mb in size and requires about 300Mb of memory in runtime. Big models are for the high-accuracy transcription on the server. Big models require up to 16Gb in memory since they apply advanced AI algorithms. Ideally you run them on some high-end servers like i7 or latest AMD Ryzen. On AWS you can take a look on c5a machines and similar machines in other clouds.

Most small model allow dynamic vocabulary reconfiguration. Big models are static the vocabulary can not be modified in runtime.

Model list

This is the list of models compatible with Vosk-API.

To add a new model here create an issue on Github.

Punctuation models

For punctuation and case restoration we recommend the models trained with https://github.com/benob/recasepunc

Other models

Other places where you can check for models which might be compatible:

• https://kaldi-asr.org/models.html - variety of models from Kaldi - librispeech, aspire, chinese models
• https://github.com/daanzu/kaldi-active-grammar/blob/master/docs/models.md - Big dictation models for English
• https://github.com/uhh-lt/vosk-model-tuda-de - German models
• https://github.com/german-asr/kaldi-german - Another German project
• https://zamia-speech.org/asr/ - German and English model from Zamia
• https://github.com/pguyot/zamia-speech/releases - French models for Zamia
• https://github.com/opensource-spraakherkenning-nl/Kaldi_NL - Dutch model
• https://montreal-forced-aligner.readthedocs.io/en/latest/pretrained_models.html (GMM models, not compatible but might be still useful)
• https://github.com/goodatlas/zeroth - Korean Kaldi (just a recipe and data to train)
• https://github.com/undertheseanlp/automatic_speech_recognition - Vietnamese Kaldi project
• https://doc.linto.ai/#/services/linstt - LINTO project with French and English models
• https://community.rhasspy.org/ - Rhasspy (some Kaldi models for Czech, probably even more)
• https://github.com/feddybear/flipside_ph - Filipino model project by Federico Ang
• https://github.com/alumae/kiirkirjutaja - Estonian Speech Recognition project with Vosk models
• https://github.com/falabrasil/kaldi-br - Portuguese models from FalaBrasil project
• https://github.com/egorsmkv/speech-recognition-uk - Ukrainian ASR project with Vosk models
• https://github.com/Appen/UHV-OTS-Speech - repository from Appen for Scalable Data Annotation Pipeline for High-Quality Large Speech Datasets Development
• https://github.com/vistec-AI/commonvoice-th - Thai models trained on CommonVoice

Training your own model

You can train your model with Kaldi toolkit. The training is pretty standard - you need tdnn nnet3 model with i-vectors. You can check mini_librispeech recipe for details. Some notes on training:

• For smaller mobile models watch number of parameters
• Train the model without pitch. It might be helpful for small amount of data, but for large database it doesn’t give the advantage but complicates the processing and increases response time.
• Train ivector of dim 30 instead of standard 100 to save memory of mobile models.
• Latest mini_librispeech uses online cmvn which we do not support yet. Use this script to train nnet3 model.

PLEASE NOTE THAT THE SIMPLE GMM MODEL YOU TRAIN WITH “KALDI FOR DUMMIES” TUTORIAL DOES NOT WORK WITH VOSK. YOU NEED TO RUN MINI-LIBRISPEECH FROM START TO END, INCLUDING CHAIN MODEL TRAINING. You also need CUDA GPU to train. If you do not have a GPU, try to run Kaldi on Collab.

Model structure

Once you trained the model arrange the files according to the following layout (see en-us-aspire for details):

• am/final.mdl - acoustic model
• am/global_cmvn.stats - required for online-cmvn models, if present enables online cmvn on features.
• conf/mfcc.conf - mfcc config file. Make sure you take mfcc_hires.conf version if you are using hires model (most external ones)
• conf/model.conf - provide default decoding beams and silence phones. you have to create this file yourself, it is not present in kaldi model
• conf/pitch.conf - optional file to create feature pipeline with pitch features. Might be missing if model doesn’t use pitch
• ivector/final.dubm - take ivector files from ivector extractor (optional folder if the model is trained with ivectors)
• ivector/final.ie
• ivector/final.mat
• ivector/splice.conf
• ivector/global_cmvn.stats
• ivector/online_cmvn.conf
• graph/phones/word_boundary.int - from the graph
• graph/HCLG.fst - this is the decoding graph, if you are not using lookahead
• graph/HCLr.fst - use Gr.fst and HCLr.fst instead of one big HCLG.fst if you want to run rescoring
• graph/Gr.fst
• graph/phones.txt - from the graph
• graph/words.txt - from the graph
• rescore/G.carpa - carpa rescoring is optional but helpful in big models. Usually located inside data/lang_test_rescore
• rescore/G.fst - also optional if you want to use rescoring, also used for interpolation with RNNLM
• rnnlm/feat_embedding.final.mat - RNNLM embedding for rescoring. Optional if you have it.
• rnnlm/special_symbol_opts.conf - RNNLM model options
• rnnlm/final.raw - RNNLM model
• rnnlm/word_feats.txt - RNNLM model word feats