- Published on May 27, 2024
- In AI Features
Mixture of Experts Will Power the Next Generation of Indic LLMs
With Mixture of experts (MoE), India can blend existing multilingual experts into one multilingual LLM while keeping training and resource costs low.
Image by Nikhil Kumar
- By Sagar Sharma
The potential of MoE in making Indic LLMs is immense. In a recent podcast with AIM, CognitiveLab founder Aditya Kolavi said that the company has been using the MoE (Mixture of Experts) architecture to fuse Indian languages and build multilingual LLMs.
"We have used the MoE architecture to fuse Hindi, Tamil, and Kannada, and it worked out pretty well," he said.
Similarly, Reliance-backed TWO has released its AI model SUTRA, which uses MoE and supports 50+ languages, including Gujarati, Hindi, Tamil, and more, surpassing ChatGPT-3.5.
Ola Krutrim is also leveraging Databricks' Lakehouse Platform to enhance its data analytics and AI capabilities while hinting at using MoE to power its Indic LLM platform.
Apart from Indic LLMs, GPT-4, Mixtral-8x7B, Grok-1 and DBRX are
📣 Want to advertise in AIM? Book here
Sagar Sharma
A software engineer who loves to experiment with new-gen AI. He also happens to love testing hardware and sometimes they crash. While reviving his crashed system, you can find him reading literature, manga, or watering plants.
Related Posts
How This Coimbatore SaaS Firm Cracked Hidden Enterprise Problem Costing Millions
Founded in 2015, now based in Portland, Responsive supports more than 20% of Fortune 100 companies
Enterprises Beware: Agent-Washing Clouds the Future of AI
Vendors mislabel copilots as agents, raising regulatory and operational risks for firms chasing the promise of agentic AI.
How Neysa Stands Out in the IndiaAI GPU Race
Unlike other providers focused on GPU allocation, Neysa claims to deliver an end-to-end AI cloud platform.
Two Indian Engineers on a Mission to Automate Home Cooking for the World
In a live demonstration for AIM, Posha prepared paneer tikka masala in approximately 25 minutes
BharatGen and the Pursuit of Sovereign, Scalable AI for India
“Knowledge-driven components are important because we don’t want everything to be just algorithmic innovation.”
How Pradhi AI Embeds Emotional Intelligence in Voice AI
As businesses recognise the potential of voice-driven tech, Pradhi AI is laying the foundation for an empathetic, responsive AI ecosystem.
Mangaluru Looks to Build Its Own Tech Identity, Not Replicate Bangalore
“The coastal city could showcase tangible results by applying deep tech to areas it already dominates”
Google’s Gemini Nano Banana and the Cost of Convenience
The company’s new AI image and photo editor deepens concerns over data use and consent gaps, experts warn.
Download the easiest way to
stay informed
stay informed
