Multilingual AI Language Processing

Latest research from KAIST and Imperial College London introduces Zero-AVSR, an innovative framework that enables audio-visual speech recognition across languages without requiring training data in target languages. By learning language-agnostic speech representations through romanisation and leveraging LLMs, it can recognise speech even in languages never seen during training. What makes this approach interesting is the scale of language support. The team created MARC, a dataset spanning 2,916 hours of audio-visual speech across 82 languages—far beyond the 9 languages typical systems support. Their results show comparable performance to traditional multilingual systems while supporting this vastly larger language inventory. Zero-AVSR represents a significant advancement for speech tech in low-resource languages, potentially democratising access across thousands of languages without requiring extensive labelled datasets for each. The approach particularly excels when recognising languages from families similar to those in the training data, suggesting promising pathways for further expansion. Paper: https://lnkd.in/dnw_V7XK Authors: Jeong Hun Yeo, Minsu Kim, Chae Won Kim, Stavros Petridis, Yong Man Ro #SpeechRecognition #MultilingualAI #SpeechAI

🌟 Excited to share our latest research on enhancing multilingual capabilities in large language models! 🌟 Introducing SPHINX, a novel multilingual synthetic instruction tuning dataset created to address the performance gap in non-English languages. By translating instruction-response pairs from English into 50 languages, we achieved impressive results. In our study, fine-tuning models PHI-3-SMALL and MISTRAL-7B using SPHINX led to significant performance improvements, surpassing other multilingual datasets in benchmarks. Incorporating N-shot examples further boosted performance, showcasing the effectiveness and efficiency of SPHINX. This advancement marks a significant step forward in making large language models more inclusive and effective across diverse languages. Our research highlights the importance of sample efficiency and diversity while minimizing dataset creation costs. Excited for further discussions and collaborations in the realm of NLP, Multilingual AI, Machine Learning, and Artificial Intelligence! 🚀 Link to the paper : https://lnkd.in/g5CP9EZc Sanchit Ahuja Kumar Tanmay Hardik Chauhan Barun Patra Vishrav Chaudhary Monojit Choudhury Arindam Mitra Luciano Del Corro Tejas Indulal Dhamecha Ahmed Awadallah Sunayana Sitaram #NLP #MultilingualAI #MachineLearning #ArtificialIntelligence #Research #Innovation

SUTRA Scalable Multilingual Language Model Architecture In this paper, we introduce SUTRA, multilingual Large Language Model architecture capable of understanding, reasoning, and generating text in over 50 languages. SUTRA's design uniquely decouples core conceptual understanding from language-specific processing, which facilitates scalable and efficient multilingual alignment and learning. Employing a Mixture of Experts framework both in language and concept processing, SUTRA demonstrates both computational efficiency and responsiveness. Through extensive evaluations, SUTRA is demonstrated to surpass existing models like GPT-3.5, Llama2 by 20-30% on leading Massive Multitask Language Understanding (MMLU) benchmarks for multilingual tasks. SUTRA models are also online LLMs that can use knowledge from the internet to provide hallucination-free, factual and up-to-date responses while retaining their multilingual capabilities. Furthermore, we explore the broader implications of its architecture for the future of multilingual AI, highlighting its potential to democratize access to AI technology globally and to improve the equity and utility of AI in regions with predominantly non-English languages. Our findings suggest that SUTRA not only fills pivotal gaps in multilingual model capabilities but also establishes a new benchmark for operational efficiency and scalability in AI applications.

Byte Latent Transformer (BLT) Meta AI’s new Byte Latent Transformer (BLT) is making waves by rethinking how large language models (LLMs) process text-completely removing the need for tokenization. 🔍 Why Move Beyond Tokenization? Traditional LLMs (like GPT-4 or Llama 3) split text into fixed “tokens” using methods such as Byte-Pair Encoding (BPE). While effective, this approach has some real drawbacks: - Bias: Tokenizers often favor common languages and scripts, making it harder to support underrepresented languages. - Fragility: They can break down with typos, mixed-language input, or unusual text patterns. - Inefficiency: Every token gets the same compute, even when some are much more predictable than others. 🛠️ How Does BLT Work? BLT processes raw bytes instead of tokens and groups them into variable-length patches based on entropy (how predictable the text is): Complex or unpredictable text → Smaller patches → More compute. Simple or repetitive text → Larger patches → Less compute. This is similar to how humans read-skimming through easy parts and slowing down for challenging sections. 💡 Key Benefits Over Traditional LLMs - Efficiency: BLT matches Llama 3’s performance on key benchmarks while using up to 50% less compute at inference. - Robustness: Handles noisy or messy data (typos, uppercase, repeated characters) much better than token-based models. - Multilingual Support: No fixed vocabulary means BLT works equally well across 100+ languages and even code. -Byte-Level Precision: Excels at tasks that require detailed character-level understanding, like spelling correction or OCR. -Scalability: Enables new ways to scale models by adjusting both model size and patch size. Why This Matters BLT’s byte-level approach is a breakthrough for: Global applications: Consistent results across languages and scripts. Enterprise workflows: Better handling of real-world, messy data. AI safety: More precise control can help reduce hallucinations. BLT shows that tokenization isn’t a necessity anymore. Byte-level modeling could lead to more adaptable, efficient, and fair AI systems-especially for multilingual and diverse data environments. Read the paper: https://lnkd.in/e8m2CNZn Code: https://lnkd.in/eJgv8UsN Will dynamic byte patching become the new standard for LLMs? #AI #MachineLearning #LLM #GenerativeAI #NLP #Innovation #TechTrends

Meta went bonkers with this new open-source ASR that works for 1,600+ languages! 🤯 Now, businesses can reach customers in their native tongue, even in low-resource regions, without building ASR from scratch. → Fully open-source, supporting 500+ languages never covered by any ASR before → Trained on 4.3M hours of multilingual speech (1,600+ languages) → Best part: Works zero-shot on languages never seen during training How? Two breakthroughs: Dual-decoder architecture:  • CTC decoder for low-latency, real-time use  • LLM-ASR decoder (Transformer-based) for high-accuracy, context-aware transcription In-context learning: Just 5–10 speech-text examples at inference time, let it transcribe any new language even if the model was never trained on it. Even more surprising: → On FLEURS-81, Omnilingual ASR beats Whisper on 65/81 languages—including 24 of the world’s top 34 most spoken languages → Robust to noise: CER stays <10 even in the noisiest 5% of field recordings → Scales from edge to cloud: 300M (mobile) → 7B (max accuracy) But the real shift isn’t scale, it’s agency. Communities can now extend ASR to their own language with minimal data, compute, or expertise. Check out the carousel to know how it works in simple terms and what the challenges are in detail. Question for you: When building voice tech for underserved languages, do you prioritise zero-shot generalisation or lightweight fine-tuning and why? Follow me, Bhavishya Pandit, for honest takes on AI tools that actually work 🔥 P.S. Model card, inference code, and datasets in the first comment.

AI doesn’t speak just one language. It never should. It should speak to, and for, all of us! From the steppes of Mongolia to the villages of India and the ministries of Chile, local AI experts are proving that sovereign, locally useful AI models can flourish even with limited resources. These efforts show that the barriers to multilingual AI can be overcome with creativity, determination, and modest funding. The question now is: how can we support and scale these efforts globally? #Mongolia – Egune AI Very happy to see Bloomberg News highlight Egune AI today, a small startup that built the first Mongolian-language foundation model from scratch. This team made the country 1 of just 8 to develop its own national model. With only $3.5M in local seed funding, they now power over 70% of the nation’s AI market. Their work protects Mongolian language and culture through homegrown AI - a powerful example of what’s possible when communities build for themselves. #India – Bhashini India’s BHASHINI - (Digital India BHASHINI Division) is a government-backed, public–private mission to make AI inclusive for all Indian languages. Launched under the National Language Translation Mission, Bhashini supports over 35 languages through an open-source model which provides real-time translation tools in text -to-text, speech-to-text, and video translation services. Through the “Bhasha Daan” crowdsourcing initiative, thousands of people are contributing text, voice and video data and translations to help the AI learn. Bhashini bridges digital gaps across the country and creates datasets for underrepresented languages. It has  already hit 1 billion+ inferences.     #Chile (Latin America) – #LatamGPT Chile is leading a regional push for AI sovereignty through a Spanish-language foundation model called Latam GPT. Under the leadership of my dear friend Minister Aisen Etcheverry, the Ministry of Science, Technology, Knowledge and Innovation is building a model that reflects Latin America’s own histories, dialects, and values. With support from CENIA and a university-backed supercomputer, the project is advancing on just a few million dollars in funding. The model is designed to be open, adaptable, and shared across countries — “AI by Latin America, for Latin America.”    The call to action: Multilingual AI capacity is often described as a roadblock to universal access. But these efforts prove it doesn’t have to be. 🔹 How do we support and scale grassroots AI infrastructure? 🔹 Can we pool funding, talent, and knowledge to help more countries build their own models? 🔹 What does a global ecosystem look like when every language has a voice in shaping it? #AIforAll #LocalAI #MultilingualAI #Innovation #aipolicy Nick Martin Hugging Face Satwik Mishra Bloomberg News Nick Cain Mary Rodriguez, MBA Mathilde Barge Nagi Otgonshar  Ashwini Vaishnaw S Krishnan Abhishek Singh Tara Chklovski Room to Read Vivian Schiller Aspen Digital

India just got its own multilingual AI stack Not a demo. A real platform. Most AI still speaks English first. India does not. We keep talking about AI scale. But ignore language reality. Sarvam AI just shipped something important. An open-source foundational model suite built for 10 Indian languages and designed voice-first. That changes who AI is for. Here’s what stands out to me: India’s first open-source 2B Indic LLM trained on ~4 trillion tokens Voice agents deployable via phone WhatsApp and in-app workflows Speech → text → translation → synthesis in a single Indic stack Legal AI workbench for drafting redaction and regulatory Q&A Pricing that starts around ₹1 per minute for multilingual agents This is not chasing Silicon Valley scale. It’s solving Indian constraints. Smaller efficient models that run where India actually is Voice interfaces for users who skip keyboards Agentic workflows not just chat responses And the quiet but big idea: Sovereign AI infrastructure. Data stays local. Models align with Indian regulation. Control stays domestic. That matters for BFSI, legal, telecom and any sector touching sensitive data. The real unlock is inclusion. AI that works in Hindi, Tamil, Telugu Malayalam, Punjabi, Odia Gujarati, Marathi, Kannada, Bengali AI that listens before it types We keep saying India will be an AI market. This is India building AI rails. Open-source, voice-first, enterprise-ready That combination is rare. If this ecosystem compounds India does not just consume AI It exports it. Watching this space closely. Local language AI is the next growth curve. What sectors do you think adopt first?

Conversational #AI just hit a triple milestone 1️⃣ #RAG (Retrieval-Augmented Generation) • Grounds every answer in live, verifiable documents, cutting hallucinations and letting teams update knowledge in minutes, not months. 2️⃣ True text-and-voice #multimodality (#ElevenLabs Conversational AI 2.0) • One agent, any channel. Talk on the phone, type in chat, swap mid-conversation, and it never loses context. 3️⃣ Next-gen turn-taking models (#TurnGPT, VAP) • Predict millisecond hand-offs, so bots stop talking over you and feel as smooth as a real colleague. Why this is a very big deal • Trust climbs, risk falls. Regulated fields like healthcare, finance, and aviation can now adopt AI assistants that cite their sources and understand when to stay quiet. • Single build, global reach. Define a bot once and deploy it across web, mobile, telephony, and smart devices without separate codebases. • Always on, always current. Drop fresh PDFs, policies, or product docs into a vector store and your agent “knows” them instantly. • Human-grade flow. Micro-pause prediction means no awkward gaps, no interruptions, and real empathy cues such as quick back-channels (“mm-hmm… go on”). • Multilingual by default. Automatic language detection flips from English to Spanish (or 29+ other languages) inside the same call, opening whole new markets overnight. • Precision where it matters. Users can speak naturally, then type exact account numbers or medication names without starting over. • Cost and speed gains. Shorter call times, higher self-service rates, and fewer agent hand-offs translate into real bottom-line impact. What tomorrow looks like 🔹 Voice-first knowledge bases that quote chapter-and-verse references while you drive. 🔹 On-the-fly compliance coaches that listen to sales calls and whisper policy reminders before a rep misspeaks. 🔹 Hospital kiosks that greet patients in their native language, switch to text when the lobby is noisy, and sync notes straight into the EHR with full citations. 🔹 Zero-latency product experts embedded in every device, from wearables to smart tractors, updating themselves whenever the manual changes. The line between “chatbot” and “colleague” is getting thinner by the week. This trio of breakthroughs makes conversational AI more reliable, versatile, and human than ever. 💡 Question for you: Which industry will leapfrog first now that bots can know, listen, and speak like this? Drop your thoughts below. Harvey Castro MD #DrGPT #ConversationalAI #RAG #VoiceTech #AIInnovation #FutureOfWork

We just released a new study at Repello that fundamentally changes how we secure global AI. Your AI guardrails work in English. But your adversaries don't just speak English. If you are running enterprise AI globally, you have a "Multilingual Jailbreak" problem. Current safety alignments are biased toward high-resource languages (English, Spanish, Chinese). But in low-resource languages, jailbreak success rates spike drastically. The industry standard has been to "just add more data." But you can't label adversarial datasets for 100+ languages. It is statistically impossible. 𝗪𝗲 𝗷𝘂𝘀𝘁 𝗳𝗶𝘅𝗲𝗱 𝘁𝗵𝗶𝘀. Today, we are introducing CREST (𝗖𝗥oss-lingual 𝗘fficient 𝗦afety 𝗧ransfer). Instead of brute-forcing data, our research team used Cluster-Guided Transfer - training on just 13 "anchor" languages to structurally transfer safety signals to 100 targets. The results from the paper: 🚀 𝟭𝟬𝘅 𝗙𝗮𝘀𝘁𝗲𝗿 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲: Because we ditched the massive LLM-based guardrail architecture for a streamlined encoder approach, we are seeing 10x speedups compared to LlamaGuard-style models. 🛡️ 𝟭𝟬𝟬 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲𝘀 𝗦𝗲𝗰𝘂𝗿𝗲𝗱: We closed the gap between high-resource and low-resource safety. 📉 𝟱𝟲𝟬𝗠 𝗣𝗮𝗿𝗮𝗺𝗲𝘁𝗲𝗿𝘀: We proved you don't need a 7B model to be safe. You need a smarter one. This is the difference between "translated safety" and "native safety." Read the full technical deep dive and try the model below. 📄 Read the Paper: https://lnkd.in/gCD843VR 🤗 Try the Model: https://lnkd.in/gBFEjWtv Blog for more details : https://lnkd.in/gtR8R3ZQ (We are open sourcing the base model, CREST-Base. CREST-Large, our more capable model is available for deployment to Repello AI customers from today, contact us if you want to try it out - https://lnkd.in/g8qUfan4) #AISafety #MachineLearning #RepelloAI #MultilingualAI #LLMSafety #AIResearch #AIGovernance

𝗧𝗵𝗲 𝗻𝗲𝘅𝘁 𝗳𝗿𝗼𝗻𝘁𝗶𝗲𝗿 𝗶𝗻 𝗔𝗜 𝗶𝘀 𝗻𝗼𝘁 𝘁𝗲𝗮𝗰𝗵𝗶𝗻𝗴 𝗺𝗮𝗰𝗵𝗶𝗻𝗲𝘀 𝘁𝗼 𝘀𝗽𝗲𝗮𝗸 𝗺𝗼𝗿𝗲 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲𝘀. It is teaching them to stop flattening people into language names. “Arabic.” “English.” “Spanish.” ... These labels are convenient for datasets, but humans do not speak labels. We speak from villages, histories, memories, migrations, jokes, wounds, rituals, and relationships. A grandmother does not speak like a government form and a patient does not describe pain like a Wikipedia article. A farmer, a nurse, a student, a shopkeeper, a traveler, ... And yet, much of modern AI still behaves as if language was separable from the people who use it. This is one of the great unsolved problems in multilingual AI. And it is the global problem behind our new project: 𝗔𝗹𝗲𝘅𝗮𝗻𝗱𝗿𝗶𝗮. The name is intentional. Alexandria was never just a city. It was one of humanity’s boldest symbols of knowledge moving across languages, cultures, and worlds. A place where translation was not clerical work. It was civilization-building. Today, we borrow that name for a new question at the heart of AI: 𝗖𝗮𝗻 𝗺𝗮𝗰𝗵𝗶𝗻𝗲𝘀 𝘂𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱 𝗔𝗿𝗮𝗯𝗶𝗰 𝗻𝗼𝘁 𝗮𝘀 𝗼𝗻𝗲 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲, 𝗯𝘂𝘁 𝗮𝘀 𝗮 𝗹𝗶𝘃𝗶𝗻𝗴 𝘄𝗼𝗿𝗹𝗱 𝗼𝗳 𝘃𝗼𝗶𝗰𝗲𝘀? For too long, Arabic has been flattened in technology. But Arabic is not one voice. Arabic is Cairo and Casablanca. Beirut and Riyadh. Sana’a, Tripoli, Nouakchott, Gaza, Damascus, Algiers, Doha, Manama, and beyond. It is place, intimacy, politeness, humor, identity, and everyday life. So I am thrilled to share that 𝗔𝗹𝗲𝘅𝗮𝗻𝗱𝗿𝗶𝗮 will appear in 𝗔𝗖𝗟 𝟮𝟬𝟮𝟲 𝗠𝗮𝗶𝗻. 𝗔𝗹𝗲𝘅𝗮𝗻𝗱𝗿𝗶𝗮: 𝗔 𝗠𝘂𝗹𝘁𝗶-𝗗𝗼𝗺𝗮𝗶𝗻 𝗗𝗶𝗮𝗹𝗲𝗰𝘁𝗮𝗹 𝗔𝗿𝗮𝗯𝗶𝗰 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗧𝗿𝗮𝗻𝘀𝗹𝗮𝘁𝗶𝗼𝗻 𝗗𝗮𝘁𝗮𝘀𝗲𝘁 𝗳𝗼𝗿 𝗖𝘂𝗹𝘁𝘂𝗿𝗮𝗹𝗹𝘆 𝗜𝗻𝗰𝗹𝘂𝘀𝗶𝘃𝗲 𝗮𝗻𝗱 𝗟𝗶𝗻𝗴𝘂𝗶𝘀𝘁𝗶𝗰𝗮𝗹𝗹𝘆 𝗗𝗶𝘃𝗲𝗿𝘀𝗲 𝗟𝗟𝗠𝘀 Alexandria is our largest community MT project so far. It is not only a dataset. It is refusal to treat dialects as noise. Dialects are knowledge systems. Alexandria is our attempt to say something plainly: 𝗧𝗵𝗲 𝗳𝘂𝘁𝘂𝗿𝗲 𝗼𝗳 𝗺𝘂𝗹𝘁𝗶𝗹𝗶𝗻𝗴𝘂𝗮𝗹 𝗔𝗜 𝗰𝗮𝗻𝗻𝗼𝘁 𝗯𝗲 𝗯𝘂𝗶𝗹𝘁 𝗯𝘆 𝗮𝘃𝗲𝗿𝗮𝗴𝗶𝗻𝗴 𝗵𝘂𝗺𝗮𝗻𝗶𝘁𝘆 𝗶𝗻𝘁𝗼 𝘀𝘁𝗮𝗻𝗱𝗮𝗿𝗱 𝗳𝗼𝗿𝗺𝘀. We need systems that can handle variation, not erase it. That can model culture, not smooth it away. This is true for Arabic. It is also true globally. In the old Alexandria, knowledge moved across languages. With this new Alexandria, we hope to help AI move across communities. This project was only possible because of a remarkable community of collaborators across the world. Paper: https://lnkd.in/gGw2nJsG Project website: https://lnkd.in/gSxQ8Dhp Dataset: https://lnkd.in/gg9Spbvb Code and guidelines: https://lnkd.in/gPURFP8e 𝗔𝗜 𝘀𝗵𝗼𝘂𝗹𝗱 𝗻𝗼𝘁 𝗼𝗻𝗹𝘆 𝗹𝗲𝗮𝗿𝗻 𝗺𝗼𝗿𝗲 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲𝘀. 𝗜𝘁 𝘀𝗵𝗼𝘂𝗹𝗱 𝗹𝗲𝗮𝗿𝗻 𝘁𝗼 𝗵𝗲𝗮𝗿 𝗽𝗲𝗼𝗽𝗹𝗲.