Kyle O'Brien’s Post

My path took me from computer science to linguistics (where I found Mańczak) and back to computer science, where I worked on language models. Years later, when linguists began calling LLMs "stochastic parrots" and questioning whether they even model language, I recognized what was happening. They were using the same frameworks Mańczak had spent his career challenging. The critiques said more about the linguists' assumptions than about the models. So I wrote a paper. My argument: "stochastic parrots" don't merely mimic language—they reveal what language has been all along. LLMs are imperfect not because they fail to model language, but because they only model language. This isn't the first paper I've tried to write about Mańczak's ideas—just the first I've managed to finish. The dots keep appearing, and there's more to say. I hope this is the start, not the end. Link: https://lnkd.in/dab8eDGs

Do you want to learn a new language? Are you struggling with grammar rules? Or perhaps you’ve been watching series with subtitles in the target language. There is no universal formula for learning a new language because the process depends on a wide range of factors such as motivation, personality, cognitive style, and emotional state. Each learner’s experience is unique, which makes the study of second language acquisition (SLA) particularly complex. Let me explain the concept of falsifiability, introduced by Karl Popper, as the principle that a scientific theory must be testable and capable of being proven wrong. In SLA research, this is difficult to achieve because language learning involves interrelated biological, psychological, and social components that are hard to isolate or measure precisely. Several major theories—such as Krashen’s Input Hypothesis, Chomsky’s Universal Grammar, and interaction-based approaches—illustrate this problem. While each highlights important aspects of learning, none can be fully tested or entirely falsified. Key constructs like comprehensible input, i + 1, or innate grammar are too abstract for direct measurement. In conclusion, there is no single theory that can fully explain how humans acquire a second language. Each one provides a partial perspective—useful, but incomplete. Instead of seeking one perfect model, researchers and teachers should adopt a humble, pluralistic approach, treating theories as heuristic tools to guide practice and encourage further inquiry. Language learning, arguably, is a dynamic and deeply human process that resists simplification and requires ongoing exploration across multiple dimensions—cognitive, social, emotional, and cultural.

Excited to share our latest research: CCL-XCOT: An Efficient Cross-Lingual Knowledge Transfer Method for Mitigating Hallucination Generation, has been accepted at EMNLP 2025! ❓ Motivation: Multilingual Large Language Models (MLLMs) are powerful, but their factual accuracy often drops in low-resource languages, leading to harmful hallucinations. This happens because knowledge learned in high-resource languages like English doesn't transfer effectively. 💡 Our Method: We propose CCL-XCOT, a two-stage framework to bridge this gap without needing external tools. 1⃣ Curriculum-based Contrastive Learning (CCL): During pre-training, we explicitly align the semantic spaces of different languages. This teaches the model that the same concept, regardless of language, should be understood similarly. 2⃣ Cross-lingual Chain-of-Thought (XCoT): During fine-tuning, we prompt the model to "think" and reason in English (its strongest language) before generating the final answer in the target low-resource language. 📈 Results: The impact is significant. Our method reduces hallucination rates by up to 62% and substantially improves factual knowledge transfer across diverse languages. We achieve this efficiently, without relying on retrieval-augmented generation (RAG) or complex model ensembles. This work is a step towards building more reliable and equitable AI that works for everyone, everywhere. 🔗 Check out the full paper here: https://lnkd.in/gyrd8feT 🔗Code: https://lnkd.in/g4fXGuGN Excited to hear your thoughts and discuss further! #AI #NLP #LLM #MultilingualAI #ResponsibleAI #Hallucination #LowResourceLanguages #MachineLearning #Research

Stop Learning Individual Words: Why 'Chunking' Will Transform Your Language Skills Are you still memorizing vocabulary lists one word at a time? Science says there's a better way. Enter 'chunking' – a powerful cognitive technique that could revolutionize your language learning. What is chunking? Instead of learning isolated words, you group language into meaningful phrases or 'chunks' that your brain processes as single units. Here's why research shows it works: • **Reduced cognitive load** – Your brain processes chunks more efficiently than individual words, allowing you to store more information within your limited short-term memory • **Enhanced fluency** – Studies show learners who master chunks speak more smoothly with fewer hesitations than those who learn isolated vocabulary • **Faster processing** – When you retrieve pre-assembled chunks from memory, you eliminate the need to construct sentences word-by-word in real-time • **Better retention** – Meaningful chunks create stronger neural connections than isolated words How to implement chunking today: 1. Identify common phrases in your target language (greetings, transitions, expressions) 2. Practice whole sentences instead of individual vocabulary items 3. Listen for and record recurring patterns in native speech 4. Learn grammar through chunk patterns rather than abstract rules The beauty of chunking is that it mirrors how children naturally acquire language – not through grammar rules, but through meaningful patterns. Which language are you learning, and how might you start implementing chunking in your practice today? 💬 Chat your way to fluency: https://lnkd.in/guV3GnPD #languagechunking #fluencytechniques #cognitivelearning

🚨New paper on what large language models tell us about human language learning! 🤖💬📈 [Link in comments] I've previously argued that LLMs demonstrate that core aspects of grammar can be learned through stastical learning from input alone. However, we know that humans don’t simply acquire language passively: we learn through interaction with others, and these interactions shape the language we use. Modern LLMs also learn from a kind of feedback, what's known as Reinforcement Learning from Human Feedack (RLHF). This step is a crucial factor in influencing how models solve downstream tasks. But does RLHF change the linguistic behaviour of LLMs? Along with Pablo Contreras Kallens and Morten C., we ran some simple experiments to test LLM linguistic behaviour before and after RLHF. We find that RLHF acts as an form of idealized language feedback, nudging LLMs toward more human-like linguistic behavior - including reproducing human-like errors! We argue that this is further evidence that feedback and interaction are foundational to language development, both for humans and machines, and consider the implications of this for linguistic theory.

And that's why editors, writers, and translators will never be replaced. LLMs are like dumb printers, without context, they will churn out nonsense... "Over the past few years, the abilities of large language models (LLMs) have received extensive attention, which have performed exceptionally well in complicated scenarios such as logical reasoning and symbolic inference. A significant factor contributing to this progress is the benefit of in-context learning and few-shot prompting. However, the reasons behind the success of such models using contextual reasoning have not been fully explored. Do LLMs have understand logical rules to draw inferences, or do they ``guess'' the answers by learning a type of probabilistic mapping through context? Based on our analysis, it is found that LLMs do not truly understand logical rules; rather, in-context learning has simply enhanced the likelihood of these models arriving at the correct answers. If one alters certain words in the context text or changes the concepts of logical terms, the outputs of LLMs can be significantly disrupted, leading to counter-intuitive responses. This work provides critical insights into the limitations of LLMs, underscoring the need for more robust mechanisms to ensure reliable logical reasoning in LLMs." https://lnkd.in/eMEkYSux

🚀 Excited to share new work with Barend Mons: “The Grammar of FAIR: A Granular Architecture of Semantic Units for FAIR Semantics, Inspired by Biology and Linguistics” 👉 https://lnkd.in/ec-A4u2z The preprint explores how granularity shapes FAIRness and develops the idea of a Grammar of FAIR, i.e., a FAIR Semantics framework inspired by biology (levels of organization) and linguistics (grammar, concept formation). We argue that a classification of different types of Semantic Units, organized into granular hierarchies, can serve as the building blocks for FAIR Digital Objects (FDOs) and the Internet of FAIR Data and Services (IFDS). This perspective connects human cognition with machine-actionability, paving the way for modular, AI-ready, and citation-granular scholarly communication. The abstract, (for those interested in more details): The FAIR Principles aim to make data and knowledge Findable, Accessible, Interoperable, and Reusable, yet current digital infrastructures often lack a unifying semantic framework that bridges human cognition and machine-actionability. In this paper, we introduce the Grammar of FAIR: a granular and modular architecture for FAIR semantics built on the concept of semantic units. Semantic units, comprising atomic statement units and composite compound units, implement the principle of semantic modularisation, decomposing data and knowledge into independently identifiable, semantically meaningful, and machine-actionable units. A central metaphor guiding our approach is the analogy between the hierarchy of level of organisation in biological systems and the hierarchy of levels of organisation in information systems: both are structured by granular building blocks that mediate across multiple perspectives while preserving functional unity. Drawing further inspiration from concept formation and natural language grammar, we show how these building blocks map to FAIR Digitial Objects (FDOs), enabling format-agnostic semantic transitivity from natural language token models to schema-based representations. This dual biological-linguistic analogy provides a semantics-first foundation for evolving cross-ecosystem infrastructures, paving the way for the Internet of FAIR Data and Services (IFDS) and a future of modular, AI-ready, and citation-granular scholarly communication.

The two-sided character of the FAIR principles (machine and human readable) will become extremely important in ‘explainable’ ‘AI’ driven applications. We hope this article will help to (a) build scalable FAIR compliant data infrastructures and (b) ease the emerging danger that various compositions of FDOs with different creators and pet names will be presented as ‘THE way’ to implement FDOs. There will be many implementations of composite FDOs, like functional molecular structures in biology. The biological inspiration of this work is not a coincidence. Both Lars and I are biologists by training and we should realise that biological systems deal with complex information for a bit longer than computer scientists (who are also biological sytems of course) and that we better learn from such a resilient and evolving approach.

Reviving Lost Tongues: AI-Powered Grammar for Language Preservation Reviving Lost Tongues: AI-Powered Grammar for Language Preservation Imagine a world where languages fade into silence, their stories and wisdom lost forever. For many indigenous communities, this is a stark reality. The lack of digital resources for languages with fewer speakers poses a significant challenge to their preservation and integration into the modern digital landscape. At the heart of this solution lies a powerful tool: context-free grammars (CFGs). Think of a CFG as a detailed blueprint for a language, specifying the rules for constructing grammatically correct sentences. This allows us to intelligently generate new sentences based on a limited set of examples, effectively expanding the available training data for language models. This approach breathes new life into low-resource languages, opening doors for machine translation, speech recognition, and a wealth of other NLP applications. By synthesizing more data, we can train better AI models that understand and support https://lnkd.in/gfKZEarJ

🌟This week at C-LaBL🌟 📢Phonology Reading Group 📅21.10 (today!), 14:15 CEST 🎙️Nick Kalivoda 📄Properties of the Italo-Romance coda typology 🔗Teams: https://lnkd.in/dzHksVQn 📢Brain domain seminar with Lancaster University 📅22.10 (tomorrow!), 15:00 UK time 🎙️Morten H. Christiansen (Cornell University & Aarhus University) 📄Squeezing through the Bottleneck: The Importance of Chunking in Language Learning 🔗Zoom: https://lnkd.in/d6JQFrfb Abstract: Language happens in the here-and-now. During normal linguistic interaction, we are faced with an immense challenge by the combined effects of rapid input, short-lived sensory memory, and severely limited sequence memory. How then can the brain successfully handle the continual deluge of linguistic input? I argue that, to deal with this “Now-or-Never” bottleneck, the brain must incrementally compress and recode the linguistic input as rapidly as possible before it is gone. As a consequence, incoming language incrementally gets recoded into chunks at increasingly more abstract levels of linguistic representation, from phonemes and syllables to multiword sequences and beyond. To illustrate, I present results from statistical learning, psycholinguistic experiments, and Large Language Models, highlighting the key role of chunking in first- and second-language learning.