African company Vambo AI proposed the Morena language model with a 1.5 billion parameter scale. It supports 12 African languages, English, French, and can write code. Developers claim the model outperforms systems from Google, Meta on tasks related to African languages✴ and Alibaba, but is still eight times smaller than its peers.
Image credit: Sergey Pesterev / unsplash.com
In the key test, Vambo AI Morena achieved a score of 1,408 bpb (bits per tuple), the best of the 26 models tested. The closest model to it, 1.423 bpb, shows a factor of five – the lower the indicator, the better. This model supports the following local languages: Nigerian Pidgin, Igbo, Yoruba, Hausa, Swahili, Shona, Zulu, Xhosa, Kinyarwanda, Tswana, Afrikaans, and Ndebele Southern. Most projects using these languages use pre-trained open meta models✴ camel, but they kept a dictionary originally designed for English and programming.
Vambo AI did something different: before starting training, they first compared dictionary sizes in terms of cost and efficiency, and then optimized the tokenizer, the composition of the training material, and the language inventory. When encoding African text, the Morena dictionary uses 1.39 times fewer tokens than Google Gemma 3 and 1.53 times fewer than Llama 3.2 for the same fragment. The encoding cost for African text is 0.249 tokens per tuple, compared to 0.234 tokens for English – a 6% difference that developers cannot yet account for. The closest competitor is the Meta model version for Africa✴ Called Lugha-Llama-8B – it displays 1,423 bpb and supports only 8 languages instead of 12. In comparison, the 12 billion parameter Gemma model consumes approximately 11 times more computing resources than Morena.
Morena, optimized for conversational interaction (the command version), scored 1,441 bpb, lower than Lugha-Llama-8B overall but better than it in five common languages and using only 20% of its parameters. When English text was translated into five African languages, the guided version scored 45.8 on the chrF++ metric, which is statistically comparable to professional translation systems—they scored about 1.4 points higher, while models of comparable size to Morena typically showed 9 to 14 points. Morena processed 251.7 billion tokens in the pre-training phase and 63 billion tokens in the intermediate phase.
The development required more than 22,000 hours of work on an Nvidia A100 accelerator – approximately $40,000 had to be paid for computing resources. In addition to the main model with a parameter scale of 1.5 billion, there are also versions with 0.5 billion and 200 million parameters. The version with a parameter size of 500 million still outperformed models tested by other developers in 11 of 12 languages. The minimal version with 200 million parameters is intended to re-evaluate variants in speech recognition systems, keyboard applications and text normalization tasks. The latter costs 58 times less per tuple than Lugha-Llama-8B, and when running on a single Nvidia A100 accelerator, it generates 104 tokens per second. There is also a version that runs on a CPU or locally on a laptop. Vambo AI Morena supports chatbot applications, translation capabilities, and integration with other artificial intelligence tools.
If you find an error, select it with your mouse and press CTRL+ENTER.
