André Lindenberg’s Post

𝗪𝗵𝘆 𝗙𝗮𝗺𝗶𝗹𝗶𝗮𝗿 𝗧𝗮𝘀𝗸𝘀 𝗖𝗮𝗻 𝗦𝘁𝗶𝗹𝗹 𝗣𝗿𝗼𝗱𝘂𝗰𝗲 𝗨𝗻𝗲𝘅𝗽𝗲𝗰𝘁𝗲𝗱 𝗠𝗶𝘀𝘁𝗮𝗸𝗲𝘀 Repeating tasks over time leads to cognitive processing shortcuts. Subtle variations in those tasks can accordingly lead to unexpected errors. This article highlights some common examples of these adaptations, and how small changes can reshape task execution. https://lnkd.in/etfTzYNn #CognitivePerformance #Task #Errors

We might be optimizing the wrong signal in LLM post-training. A common pipeline assumption: 👉 Strong SFT (Supervised Fine-Tuning) performance → better starting point for RL. But recent evidence suggests the opposite can happen. Models that look "perfect" after SFT often: • Memorize patterns instead of generalizing • Collapse output diversity • Leave little headroom for RL to improve reasoning In other words; they peak too early. 👉 What actually correlates with stronger RL gains? → Higher generalization gap after SFT → Better performance at Pass@k (k ≫ 1) rather than Pass@1 → Latent diversity in reasoning trajectories, not just correctness This reframes how we should think about post-training: SFT is not the finish line; it’s a conditioning phase Over-optimization at this stage can hurt downstream alignment Diversity > determinism when preparing for RL 👉 Practical implication: If your model looks “too good” after SFT, it might already be limiting its own future improvement. The real question isn’t: "Did SFT converge?" It’s: "Did we preserve enough uncertainty for RL to explore?" #LLM #AIResearch #ReinforcementLearning #DeepLearning #GenerativeAI

Daily DL Review #33 — Representation Collapse A model can learn nothing — and still minimize loss. --- What is collapse? All inputs map to the same embedding: z1 = z2 = z3 = ... --- Why it happens - no negative samples - weak objective - poor training setup --- Effect - no structure - no separability - useless embeddings --- Intuition Model finds trivial solution: “everything is the same” --- Where it appears - contrastive learning - self-supervised models - multimodal alignment --- How to fix - contrastive loss (positive vs negative) - regularization - architectural constraints --- Deeper insight Good learning = preventing trivial solutions --- Common pitfall Thinking loss ↓ means representation ↑ #DeepLearning #Representation #Collapse #SelfSupervised

Today I learned about ELIZA, a 1960s chatbot created by MIT professor Joseph Weizenbaum that simulated a psychotherapist. It was so convincing that some users, including Weizenbaum’s secretary, became emotionally attached to it. In 2023, ELIZA even outperformed GOT3.5 in a Turing test study.

The latest edition of our newsletter Distributed Cognition has our Communications Guy Ian Belknap examining questions of effort and replaced-by-robots-ness. https://lnkd.in/eiMty_X7

What if we could understand how people think — without labeling them? We’ve just launched an early prototype at SynapticFour: 👉 https://lnkd.in/e8yA_q_A Most cognitive and psychological tools are built on very narrow assumptions about how people think — often based on Western populations. We’re exploring a different approach. Instead of labeling people, we map cognitive patterns in a way that is culturally adaptable, behavior-based, and usable across very different contexts. The goal is to understand how people think, adapt, and experience their environment — without forcing them into diagnostic categories. This is especially important in places where traditional assessment is inaccessible or stigmatized. Still early, still learning — feedback very welcome. #Neurodiversity #CognitiveScience #EdTech #HumanCenteredDesign #CrossCultural #Research #MentalHealth

Friction as Insight: Harnessing Cognitive Delay to Strengthen Debugging Intuition Explore how intentional friction in AI-assisted coding enhances problem-solving skills, reduces mental fatigue, and accelerates error detection through strategic cognitive delay techniques.

Writing under the influence...? Too LLMed to....[fill in the blank...] This is just out from Paul Kedrosky, a research fellow at MIT's Institute for the Digital Economy. "There is sharply reduced brain activity when using LLMs, and that has consequences beyond the activity itself. Chat model usage can be like driving your brain drunk." This raises serious questions for all these areas of the #University that are pushing, endorsing, celebrating the use of #LLMs in their courses... We are literally enforcing cognative impairment amongst upon students (who to be honest, probably suffer enough cognitive impairment via other means outside the classroom...) But what does this mean for the workplace, for society, for democracy...? If you drink to drunkeness all day, everyday, at work and home and at university...what is the impact...and yet we think a similar cognitive suppressent should be celebrated and in many cases in many couses, made mandatory...? Hamish Rennie David Dempsey

Your ASR (Automatic Speech Recognition) vendor's pricing page is missing a line item 👀 Deepgram's base rate looks reasonable. Until you add diarization. Then redaction. Then realize you still need emotion detection and accent classification - and those aren't on the menu at any price. Here's what the real math looks like: 💰Batch all-in: Modulate $0.03/hr → Deepgram $0.38/hr 💰Streaming all-in: Modulate $0.06/hr → Deepgram $0.58/hr That's a 10-12x gap - before you account for the features Deepgram simply doesn't offer 🙅 Modulate includes emotion detection (20+ emotions), accent detection (20+ accents), deepfake detection, diarization, and redaction. All in the base rate. 🚨 For a team running millions of minutes a month, "cheaper ASR" is actually an expensive decision. What feature is missing from your current ASR stack that you've had to build in-house?

The explanation breaks first. The same patterns stop fitting the same language. That’s when something deeper begins to move. By the time it’s named, it’s already been visible for a while. #CognitionFirst #ShunyaMind #CognitiveInfrastructure #LearningScience #LearningSystems #LearningDesign