Using Text-Based AI for Autonomous Learning

AI agents are widely misunderstood due to their broad scope. To clarify, let's derive their capabilities step-by-step from LLM first principles... [Level 0] Standard LLM: An LLM takes text as input (prompt) and generates text as output, relying solely on its internal knowledge base (without external information or tools) to solve problems. We may also use reasoning-style LLMs (or CoT prompting) to elicit a reasoning trajectory, allowing more complex reasoning problems to be solved. [Level 1] Tool use: Relying upon an LLM’s internal knowledge base is risky—LLMs have a fixed knowledge cutoff date and a tendency to hallucinate. Instead, we can teach an LLM how to use tools (by generating structured API calls), allowing the model to retrieve useful info and even solve sub-tasks with more specialized / reliable tools. Tool calls are just structured sequences of text that the model learns to insert directly into its token stream! [Level 2] Orchestration: Complex problems are hard for an LLM to solve in a single step. Instead, we can use an agentic framework like ReAct that allows an LLM to plan how a problem should be solved and sequentially solve it. In ReAct, the LLM solves a problem as follows: 1. Observe the current state. 2. Think (with a chain of thought) about what to do next. 3. Take some action (e.g., output an answer, call an API, lookup info, etc.). 4. Repeat. Decomposing and solving problems is intricately related to tool usage and reasoning; e.g., the LLM may rely upon tools or use reasoning models to create a plan for solving a problem. [Level 3] Autonomy: The above framework outlines key functionalities of AI agents. We can make such a system more capable by providing a greater level of autonomy. For example, we can allow the agent to take concrete actions on our behalf (e.g., buying something, sending an email, etc.) or run in the background (i.e., instead of being directly triggered by a user’s prompt). AI agent spectrum: Combining these concepts, we can create an agent system that: - Runs asynchronously without any human input. - Uses reasoning LLMs to formulate plans. - Uses a standard LLM to synthesize info or think. - Takes actions in the external world on our behalf. - Retrieves info via the Google search API (or any other tool). Different tools and styles of LLMs provide agent systems with many capabilities-the crux of agent systems is seamlessly orchestrating these components. But, an agent system may or may not use all of these functionalities; e.g., both a basic tool-use LLM and the above system can be considered “agentic”.

AI EDITING TIP Here’s how you can turn AI into your own personal writing and editing teacher (not that I want to put myself out of business!). One of my favourite ways to work with AI tools is to ask them to list what needs editing and WHY they made certain edits (not just spit out an edited version of my text). This approach adds real value. So, instead of just tidying up your writing, you're learning along the way. If there's no compelling reason for an edit, maybe the edit doesn’t need to be made! Keeping your voice authentic is key. It's important you know what is being changed and why. Instead of simply asking ChatGPT or Gemini to edit, enhance or improve your writing, ask it to explain the reasoning behind its suggestions. Here's a sample prompt you can use: "Edit the following passage for clarity and readability. Then, summarise the changes and explain why each one was necessary so I can understand and improve my own editing skills." If you don't want an edited version, you can also try this prompt: "Review the text and list what you would edit with an explanation for each change so I can learn along the way." Using AI like this turns it into a learning partner that helps you build your editing skills. Give it a try and let me know how you go!

Autonomous Learning: Paradigm Shift in LLM The general trend with LLMs so far has involved fine-tuning models for various applications. But what if LLMs could engage in Autonomous Learning, where they independently extract conceptual understanding from large text corpora, develop hypotheses based on these concepts, and rigorously test them? Only the hypotheses that pass these tests would then be used to fine-tune the models. The concept of using Autonomous Learning opens up a plethora of opportunities in the industry where LLMs can autonomously learn the semantics of stock market, finance, and law. This changes the dynamics from generic LLMs to highly specialized domain-specific LLMs. The beauty is that one is not constrained by human-annotated data; the idea is to let the LLM understand the concepts at play in any field, generate data, and test that hypothesis via test cases. Only the responses that pass the tests are used for the actual fine-tuning. This idea can also be extended to CodeLLMs. The most effective way for a developer to learn new coding concepts is to study them, write code, and then test their hypotheses. Similarly, a code-based LLM could adopt the same approach. Introducing SelfCodeAlign SelfCodeAlign is an approach where LLMs train independently, without relying on human annotations or knowledge transfer from large models. SelfCodeAlign involves: -Generating instructions autonomously by extracting diverse coding concepts from seed data. -Using these concepts to create unique tasks and generating multiple responses. -Pairing responses with automated test cases and validating them in a controlled sandbox environment. -Sending only the responses that pass the validation for final instruction tuning. SelfCodeAlign begins by extracting code snippets from a large corpus with an emphasis on diversity and quality. Specifically, the initial dataset, "The Stack V1," is filtered to select 250,000 high-quality Python functions from an original pool of 5 million, using stringent quality checks. The model then breaks down each function into fundamental coding concepts, such as data type conversion or pattern matching. Tasks and responses are generated based on these concepts, with difficulty levels and categories assigned to ensure variety. Results The effectiveness of SelfCodeAlign was rigorously tested with the CodeQwen1.5-7B model. Benchmarked against models like CodeLlama-70B, SelfCodeAlign significantly outperformed many state-of-the-art solutions, achieving a HumanEval+ pass@1 score of 67.1% — a 16.5-point improvement over its baseline model, CodeQwen1.5-7B-OctoPack. Conclusion SelfCodeAlign provides an innovative solution to the challenges of training instruction-following models in code generation. https://lnkd.in/dKY-yxVg #ai #llm #deeplearning

Stanford researchers just introduced a new way to optimize AI models using text-based feedback instead of traditional backpropagation! Deep learning has long relied on numerical gradients to fine-tune neural networks. But, optimizing generative AI systems has been much harder because they interact using natural language, not numbers. 𝗧𝗲𝘅𝘁𝗚𝗿𝗮𝗱 𝗶𝘀 𝘁𝗵𝗲 𝗳𝗶𝗿𝘀𝘁 𝗳𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 𝘁𝗼 𝗯𝗮𝗰𝗸𝗽𝗿𝗼𝗽𝗮𝗴𝗮𝘁𝗲 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗺𝗼𝗱𝗲𝗹 𝗳𝗲𝗲𝗱𝗯𝗮𝗰𝗸, 𝗲𝗻𝗮𝗯𝗹𝗶𝗻𝗴 𝗔𝗜 𝘁𝗼 𝗶𝘁𝗲𝗿𝗮𝘁𝗶𝘃𝗲𝗹𝘆 𝗿𝗲𝗳𝗶𝗻𝗲 𝗶𝘁𝘀 𝗼𝘂𝘁𝗽𝘂𝘁𝘀 𝗮𝗰𝗿𝗼𝘀𝘀 𝗱𝗶𝘃𝗲𝗿𝘀𝗲 𝘁𝗮𝘀𝗸𝘀. 1. Improved AI performance in PhD-level science Q&A, raising accuracy from 51.0% to 55.0% on GPQA and from 91.2% to 95.1% on MMLU physics. 2. Optimized medical treatment plans, outperforming human-designed radiotherapy plans by better balancing tumor targeting and organ protection. 3. Enhanced AI-driven drug discovery by iteratively refining molecular structures, generating high-affinity compounds faster than traditional methods. 4. Boosted complex AI agents like Chameleon, increasing multimodal reasoning accuracy by 7.7% through iterative feedback refinement. 𝗧𝗵𝗲 𝘂𝘀𝗲 𝗼𝗳 "𝘁𝗲𝘅𝘁𝘂𝗮𝗹 𝗴𝗿𝗮𝗱𝗶𝗲𝗻𝘁𝘀" 𝗶𝗻𝘀𝘁𝗲𝗮𝗱 𝗼𝗳 𝗻𝘂𝗺𝗲𝗿𝗶𝗰𝗮𝗹 𝗴𝗿𝗮𝗱𝗶𝗲𝗻𝘁𝘀 𝗶𝘀 𝗽𝗿𝗲𝘁𝘁𝘆 𝗱𝗮𝗿𝗻 𝗰𝗼𝗼𝗹. It treats LLM feedback as “textual gradients” which are collected from every use of a variable in the system. By aggregating critiques from different contexts and iteratively updating variables (using a process analogous to numerical gradient descent), the method smooths out individual inconsistencies. 𝗜'𝗺 𝗰𝘂𝗿𝗶𝗼𝘂𝘀 𝗮𝗯𝗼𝘂𝘁 𝗵𝗼𝘄 𝗳𝗼𝗿𝗺𝗮𝗹𝗶𝘇𝗶𝗻𝗴 𝗺𝗲𝘁𝗵𝗼𝗱𝘀 𝘁𝗼 𝘃𝗮𝗹𝗶𝗱𝗮𝘁𝗲 𝗮𝗻𝗱 𝗰𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻 𝘁𝗲𝘅𝘁𝘂𝗮𝗹 𝗴𝗿𝗮𝗱𝗶𝗲𝗻𝘁𝘀 beyond formalization of the propagation and update process via the equations could be developed to enhance robustness. Perhaps training secondary models to evaluate the quality and consistency of textual gradients or an ensemble approach of generating multiple textual gradients using different LLMs or multiple prompts? Just throwing some ideas out there; this stuff is pretty cool. Here's the awesome work: https://lnkd.in/gX8ABsdM Congrats to Mert Yuksekgonul, Federico Bianchi, Joseph Boen, James Zou, and co! I post my takes on the latest developments in health AI – 𝗰𝗼𝗻𝗻𝗲𝗰𝘁 𝘄𝗶𝘁𝗵 𝗺𝗲 𝘁𝗼 𝘀𝘁𝗮𝘆 𝘂𝗽𝗱𝗮𝘁𝗲𝗱! Also, check out my health AI blog here: https://lnkd.in/g3nrQFxW

Two recent studies, one from OpenAI's analysis of 2.5 billion daily ChatGPT messages and the other from Google's controlled trial of AI-augmented textbooks, provide converging evidence of a fundamental shift in how people learn. ChatGPT, with 700 million weekly users, sees 10% of all messages dedicated to tutoring, predominantly from users aged 18-25. Surprisingly, students primarily use AI to deepen understanding rather than complete tasks: 49% of interactions seek explanations and comprehension, not ready-made answers. This organic adoption shows students creating personalized learning experiences that traditional one-size-fits-all textbooks cannot provide. Google's Learn Your Way validates this approach experimentally. By personalizing textbook content to student interests and reading levels, explaining physics through basketball or economics through music, the system improved test scores by 13 percentage points. Both studies show AI transforms passive reading into active engagement through questions, multiple content representations, and immediate feedback. The gender gap in usage has closed, and adoption is accelerating in lower-income countries, though educated professionals still dominate work-related usage. The convergence is becoming more clear: millions of students aren't waiting for institutions to provide AI learning tools, they're already using GenAI as a personalized tutor. The data suggests GenAI works best as a learning companion that enhances understanding rather than replacing formal education. As we move forward, the question isn't whether AI will transform education, that transformation is already underway, driven by millions of students who have discovered that AI can provide something traditional educational materials cannot: personalized, patient, always-available support for learning. The question is how educational institutions, policymakers, and technology developers will respond to and shape this transformation to ensure it enhances rather than undermines human learning and development. https://lnkd.in/gpAxJrfF