How Data can Improve Student Learning

Today, I would like to share an AI SoTL article entitled, “Experimentally testing AI-powered content transformations on student learning” by Heldreth et al. (2025) (https://lnkd.in/eanRDerM ). This study provides evidence that AI can measurably improve student learning outcomes when used to transform academic content. In a between-subjects experimental design with 60 U.S. high-school students, researchers compared learning a neuroscience textbook chapter using either a traditional digital PDF reader or an AI-powered platform called Learn Your Way, which transformed the same content into multiple interactive formats (immersive text, quizzes, slides, audio lessons, videos, and mind maps). The results were consistent and statistically significant. Students using the AI-powered system demonstrated higher immediate recall and superior long-term retention (3–7 days later) compared to those using the digital reader. Importantly, performance gains were not attributable to differences in prior knowledge, reading ability, interest, or assessment difficulty all were carefully controlled. Beyond test scores, students using Learn Your Way reported more positive learning experiences, including greater perceived understanding, higher enjoyment, stronger confidence, and a greater desire to reuse the tool. Qualitative data revealed why: students valued multimodal representations, chunked content, embedded quizzes, and timely feedback, all of which supported metacognitive monitoring and reduced cognitive overload. Grounded in multimedia learning theory, dual-coding theory, and self-directed learning principles, this study reinforces that AI is most effective when it re-represents content in cognitively supportive ways, rather than simply generating answers. Notably, learning gains were driven less by the number of AI features used and more by student agency in choosing representations that matched their learning needs. For teaching and learning, the implication is that AI can be used as a learning architecture, one that supports retrieval practice, feedback, personalization, and learner control at scale. Reference Heldreth, C., Vardoulakis, L. M., Miller, N. E., Haramaty, Y., Akrong, D., Hackmon, L., & Belinsky, L. (2025). Experimentally testing AI-powered content transformations on student learning. arXiv.

🚀 Can teaching students “how to learn” actually change how they engage with their coursework? In this study published in the British Journal of Educational Technology, we used over 257,000 online learning “clicks” from biology students to track how their study habits evolved. Researchers moved beyond simply counting clicks—they mapped patterns of engagement, like how regularly students moved between different resources (quizzes, notes, calendars). Key findings: Students who received a short “science of learning to learn” training showed more organized, regular study patterns—and kept them up all semester. This regularity (think: consistent, purposeful learning routines) was a strong predictor of final grades—above and beyond just how much students clicked. Complexity-based network analysis offers powerful, AI-ready ways to monitor and support student self-regulated learning in real time. 💡 The big idea: Success isn’t just about what you study—it’s about building adaptive, organized habits you can sustain. https://lnkd.in/er9mmBfa

You ran the data meeting on Friday. Everyone nodded. Nothing changed on Monday. Here's what really happened. Data was collected. The team discussed the data. But nobody decided 𝙝𝙤𝙬 𝙩𝙤 𝙩𝙚𝙖𝙘𝙝 𝙙𝙞𝙛𝙛𝙚𝙧𝙚𝙣𝙩𝙡𝙮. Here's the problem: we've confused 𝘤𝘰𝘭𝘭𝘦𝘤𝘵𝘪𝘯𝘨 data with 𝘶𝘴𝘪𝘯𝘨 it. Data without a clear instructional response isn't a system. It's a filing cabinet. So what does acting on data actually look like? After your next assessment, before your data meeting, ask your team one question: "𝗕𝗮𝘀𝗲𝗱 𝗼𝗻 𝘁𝗵𝗶𝘀 𝗱𝗮𝘁𝗮, 𝘄𝗵𝗮𝘁 𝗮𝗿𝗲 𝘄𝗲 𝗳𝗼𝗰𝘂𝘀𝗶𝗻𝗴 𝗼𝗻 𝗮𝗻𝗱 𝗵𝗼𝘄 𝗮𝗿𝗲 𝘄𝗲 𝘁𝗲𝗮𝗰𝗵𝗶𝗻𝗴 𝗶𝘁 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁𝗹𝘆 𝗻𝗲𝘅𝘁 𝘁𝗶𝗺𝗲?" Not re-teaching the same lesson. Not moving on and hoping it clicks. 𝗛𝗼𝘄 𝗮𝗿𝗲 𝘄𝗲 𝗮𝗽𝗽𝗿𝗼𝗮𝗰𝗵𝗶𝗻𝗴 𝗶𝘁 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁𝗹𝘆? Here's a simple three-step protocol to make that question actionable: 𝗦𝘁𝗲𝗽 𝟭: ��𝗮𝗺𝗲 𝘁𝗵𝗲 𝗺𝗶𝘀𝗰𝗼𝗻𝗰𝗲𝗽𝘁𝗶𝗼𝗻, 𝗻𝗼𝘁 𝗷𝘂𝘀𝘁 𝘁𝗵𝗲 𝗺𝗶𝘀𝘁𝗮𝗸𝗲. Don't stop at "students got question 4 wrong." Ask why. Was it a procedural error? A conceptual gap? A language barrier? The misconception tells you how to respond. The mistake only tells you something went wrong. 𝗦𝘁𝗲𝗽 𝟮: 𝗠𝗮𝘁𝗰𝗵 𝘁𝗵𝗲 𝗶𝗻𝘀𝘁𝗿𝘂𝗰𝘁𝗶𝗼𝗻𝗮𝗹 𝗺𝗼𝘃𝗲 𝘁𝗼 𝘁𝗵𝗲 𝗺𝗶𝘀𝗰𝗼𝗻𝗰𝗲𝗽𝘁𝗶𝗼𝗻. If students have a conceptual gap, teachers should use the CRA model (Concrete, Representational, Abstract) as a guide. Start with manipulatives or real-world context, move to visuals, then rebuild the abstract. If it's procedural, slow down the steps and make student thinking as visible as possible. The response has to match the root cause, not just re-cover the content. 𝗦𝘁𝗲𝗽 𝟯: 𝗣𝗿𝗮𝗰𝘁𝗶𝗰𝗲 𝗮𝗻𝗱 𝗮𝘀𝘀𝗶𝗴𝗻 𝗼𝘄𝗻𝗲𝗿𝘀𝗵𝗶𝗽 𝗯𝗲𝗳𝗼𝗿𝗲 𝗹𝗲𝗮𝘃𝗶𝗻𝗴 𝘁𝗵𝗲 𝗿𝗼𝗼𝗺. Every instructional response needs a name attached to it. Who is trying what, in which class, by when and what does that instruction actually look like? Without ownership, the plan dies in the meeting. 𝗗𝗮𝘁𝗮 𝗺𝗲𝗲𝘁𝗶𝗻𝗴𝘀 𝘀𝗵𝗼𝘂𝗹𝗱 𝗲𝗻𝗱 𝘄𝗶𝘁𝗵 𝗮 𝘁𝗲𝗮𝗰𝗵𝗶𝗻𝗴 𝗽𝗹𝗮𝗻, 𝗻𝗼𝘁 𝗷𝘂𝘀𝘁 𝗮 𝘁𝗮𝗹𝗸𝗶𝗻𝗴 𝗽𝗼𝗶𝗻𝘁. ♻️ If this idea resonates, repost to help school leaders and math teams turn data into action, not just conversation. 📧 If you're interested in more practical strategies like this, I'm launching a new newsletter called The 3-1-4, where I share practical strategies for improving math instruction and leadership. The first issue goes out on Pi Day (March 14). Link in the comments. _______________________________ Hi, I'm Dwight Williams. A proud first-gen everything, and I help schools and districts strengthen math instruction through coaching, curriculum support, and data-informed systems that drive student confidence and achievement. 👍🏿 Like | 🔔 Follow | 💬 Comment | 🔁 Repost

Everywhere you look, there’s a survey asking students if they’re using AI for learning. The problem is, we don't have real user data on how much are they using it and more importantly, how are they using it? Last month, Anthropic released a paper on CLIO (Claude Insights and Observations) — a tool that analyzes usage patterns of Claude while protecting user privacy. Think of it as Google Trends, but for LLMs. It didn’t get nearly the attention it deserved. (link in the comments) Imagine applying this idea to education: CLIO for learning. Instead of relying on surveys, real anonymized data would help education leaders understand how high school and college students are engaging with AI tools like LLMs in their coursework. 1️⃣ How often are students using AI tools? 2️⃣ Are they using them as “answer engines” or for deeper exploration of topics? 3️⃣ What drives brief, one-and-done interactions versus extended, curiosity-driven engagement in a topic? Right now, we have no real data points for teachers or school leaders to understand how students are interacting with these tools. Banning AI doesn’t work. AI detection tools are ineffective at best. School and district leaders empowered with data on volume of use, types of use, and what contributes to use that furthers learning, sets up the millions of gifted educators across the country with the information they need to evolve learning environments that keep rigor, improve engagement, and help young people thrive in the future. 

In schools today, we’re surrounded by a plethora of data - from assessments and observations to a variety of dashboards and feedback loops. But data only matters if it informs what we do next. That’s why here at American International School of Guangzhou we’ve developed the 𝐅𝐀𝐂𝐓𝐒 𝐃𝐚𝐭𝐚 𝐏𝐫𝐨𝐭𝐨𝐜𝐨𝐥 – a structured process designed to help teams move from data collection to meaningful action. FACTS guides us to: 🔎 𝐅𝐨𝐜𝐮𝐬 on the data that matters most 📊 𝐀𝐧𝐚𝐥𝐲𝐳𝐞 insights and gaps 🎉 𝐂𝐞𝐥𝐞𝐛𝐫𝐚𝐭𝐞 successes and positive trends 🎯 𝐓𝐚𝐫𝐠𝐞𝐭 strategies and interventions 🚀 Define clear 𝐒𝐭𝐞𝐩𝐬 for action and accountability We’ve recently rolled this out with faculty, middle leaders, senior leadership - as well as with our Operations Team. All with the goal of shifting the way we talk about and act on data across the whole school. A strong data protocol matters because it: * 𝐄𝐧𝐬𝐮𝐫𝐞𝐬 𝐮𝐧𝐢𝐟𝐨𝐫𝐦𝐢𝐭𝐲 – establishing consistent guidelines and vocabulary, keeping coherence across departments and educators. * 𝐂𝐮𝐥𝐭𝐢𝐯𝐚𝐭𝐞𝐬 𝐭𝐞𝐚𝐦𝐰𝐨𝐫𝐤 – giving staff a shared approach that elevates teaching and learning collaboratively. * 𝐅𝐚𝐜𝐢𝐥𝐢𝐭𝐚𝐭𝐞𝐬 𝐢𝐧𝐟𝐨𝐫𝐦𝐞𝐝 𝐜𝐡𝐨𝐢𝐜𝐞𝐬 – empowering decision-makers to rely on dependable data and implement strategies that truly improve student learning. Just as importantly, a protocol helps us 𝐝𝐞𝐟𝐢𝐧𝐞 𝐭𝐡𝐞 𝐯𝐚𝐥𝐮𝐞 𝐨𝐟 𝐝𝐚𝐭𝐚 itself: Does it suit our needs? Are there important data points missing? Can we find a way to access them? Having vast amounts of data is one thing - having useful data is another. A protocol like FACTS ensures we make that distinction quickly and clearly. We’ve also dedicated significant time to our school improvement plan: our 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐯𝐞 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠 𝐅𝐫𝐚𝐦𝐞𝐰𝐨𝐫𝐤. FACTS helps us implement, monitor, and analyse its impact with greater clarity. By running the full protocol, we ensure every data dive is structured, organised, and results in actionable steps - not just endless exploration. And beyond the walls of our classrooms and offices, data also helps us 𝐞𝐧𝐠𝐚𝐠𝐞 𝐨𝐮𝐫 𝐰𝐢𝐝𝐞𝐫 𝐜𝐨𝐦𝐦𝐮𝐧𝐢𝐭𝐲 - celebrating successes, building trust, and showing the impact of our collective efforts. Ultimately, regardless of the protocol you use, the true value lies in the cycle itself - structured, collaborative, and action-driven. It’s this cycle that turns information into impact, ensuring data is never for its own sake, but always driving improvement, strengthening our community, and helping every student thrive.

A few years ago, I worked with an online education platform facing challenges with student engagement. While they had a significant number of users enrolling in courses, they struggled with low participation rates in course discussions and activities, leading to a decline in course completion rates. The platform needed to identify the causes behind low engagement and implement strategies to encourage more active participation. Improving Student Engagement Using Data Analytics 1️⃣ Analyzing Engagement Data We began by analyzing user interaction data, focusing on metrics such as time spent on the platform, participation in discussions, video completion rates, and quiz scores. Using SQL, we aggregated the data to identify patterns and pinpoint where students were losing interest. SELECT student_id, course_id, AVG(time_spent) AS avg_time_spent, COUNT(discussion_post_id) AS posts_made, AVG(quiz_score) AS avg_quiz_score FROM student_activity GROUP BY student_id, course_id; 🔹 Insight: We identified that students who interacted with course discussions and quizzes had higher completion rates, while others dropped off quickly. 2️⃣ Building a Predictive Model We then created a predictive model to determine which students were at risk of disengaging based on their activity patterns. The model incorporated features such as time spent on the platform, participation in discussions, and progress through the course material. # Pseudocode for Predictive Model def predict_student_engagement(student_data): model = train_engagement_model(student_data) predictions = model.predict(student_data) return predictions 🔹 Insight: This model helped us flag students who were likely to disengage early, allowing for timely interventions. 3️⃣ Implementing Engagement Strategies Based on insights from the model, we implemented strategies such as sending personalized emails with reminders, offering incentives for completing activities, and increasing interaction opportunities through live Q&A sessions. # Pseudocode for Engagement Follow-Up def send_engagement_reminder(student_data): if model.predict(student_data) == 'at_risk': send_email_reminder(student_data) 🔹 Insight: Personalized engagement and incentives led to an increase in student participation. Challenges Faced Identifying meaningful engagement metrics that were predictive of success. Finding the right balance between engaging students without overwhelming them. Business Impact ✔ Student engagement improved, leading to higher completion rates. ✔ Retention rates increased, as more students continued with courses. ✔ Revenue grew, driven by more active and satisfied students. Key Takeaway: By analyzing user activity and leveraging predictive analytics, businesses can identify disengaged customers early and implement strategies to improve engagement and retention.

📊 KCSE 2025 Analysis: When Data Speaks, Education Transforms Behind every KCSE result is more than a grade — there’s a story, a system, and a signal. After analyzing the KCSE 2025 results, one thing is clear: data has the power to reshape education if we choose to listen to it. KCSE analysis is not just about who passed or failed. It reveals: Hidden inequalities between regions and schools Performance gaps tied to resources, teacher deployment, and learning environments Subject-level trends that signal future workforce strengths and weaknesses Early warnings for students and counties at risk of being left behind This is where data science meets policy. 🔍 With proper data analysis, governments can: Allocate teachers and resources equitably Design targeted interventions instead of blanket policies Track the real impact of curriculum reforms Predict outcomes early and act before failure happens 🎓 With the right insights, educators can: Identify struggling learners early Improve subject-specific teaching strategies Learn from high-performing schools and scale what works 📈 And for students? Data-driven education means fairer opportunities, informed decisions, and a system that supports—not surprises—you. The KCSE 2025 analysis is proof that education systems should not be run on intuition alone. They must be guided by evidence, analytics, and foresight. 💡 Data does not replace educators or policymakers — it empowers them. The future of education in Kenya and across Africa will belong to those who: Ask the right questions Trust the data Turn insights into action Let’s move from results announcement to results intelligence. #KCSE2025 #EducationData #DataScienceInEducation #EvidenceBasedPolicy #EdTech #LearningAnalytics #DataForImpact #FutureOfEducation #PolicyInnovation

In almost every school I've ever visited, the issue isn't the teachers. It's not the leaders. And it's definitely not the kids. But here's the reality in too many schools: inconsistent instruction, stagnant student achievement, frustrated teachers, and overwhelmed leaders. That story was no different in a network of 7 schools we've been working with this year. But it's not the story now. Let me share what we did, not because I think it's magic, but because I think anyone can do it. Here's what we did: 1. Defined the vision for every block of the day: We mapped out what excellence looked like in every key instructional block: - What should an effective reading lesson look like? - What are non-negotiables in math instruction? - How do we leverage history to build background knowledge? - How does science become high rigor and high engagement? - What does student engagement actually look like, sound like, and feel like when we walk into any space in the school? That level of clarity removed guesswork for teachers and gave leaders a shared framework for observations. 2. Every teacher was coached, every week. - Short, focused observations (15-20 minutes, not full-period evaluations) - Immediate, actionable feedback on one key lever, not a laundry list of suggestions - Weekly one-on-one coaching meetings held sacred 3. Set weekly goals to measure progress: Instead of waiting for benchmark assessments, we built simple, weekly indicators of progress: - Are students engaged in learning in every block of the day? - Are students getting plenty of time to independently practice? - Are math exit tickets showing mastery of the lesson objective? - Are teachers implementing feedback from the last coaching session? Small wins led to big momentum. A narrow focus helped teachers and leaders stop feeling like they were doing the most and not seeing any progress. 4. Action planning based on data: No more “data meetings” that were just numbers on a slide. - We reviewed student work together, identified breakdowns, and built immediate next steps. - Teachers left each meeting with a plan they could apply the next day, not vague goals for next quarter. The results: Student proficiency increased by double digits in both reading and math benchmarks within one year. Teachers felt more supported and reported higher confidence in their instruction. Leaders shifted from putting out fires to proactively coaching and driving instructional improvement. If your school or network is struggling with initiative overload, the answer isn’t more programs. It’s more clarity. And the discipline to do some simple things really, really well.

The Gap No One Talks About Schools love to analyze data. They sort, group, and re-group students as soon as test results roll in. But here’s the truth no one says out loud: grouping students does not close the gap. The next critical step is explicit teaching — and too often, it never happens. Teachers talk about the strategy but never teach through it. Students hear “find the main idea” or “make an inference,” but they don’t know the what, why, when, or how. The result? Rambling lessons, strategy confusion, surface compliance, and widening gaps. 👉 Data without explicit teacher talk is just paperwork. The key is progress monitoring. It tells us whether the teaching has been effective — whether it’s sticking with students. When I found that instruction wasn’t sticking, I had teachers record their lessons. What I discovered was eye-opening. One teacher’s explicit teaching matched the progress of her students. Eighty-eight percent of her students passed. The few who didn’t were already receiving SPED support, and we made sure our SPED teachers were trained in the same explicit methods. This is what works: progress monitoring + explicit teacher talk. Without it, schools will keep repeating the same cycle: data → groups → worksheets → failure. Explicit Teacher Talk is the missing step between data and success.

Hey #highered leaders - if you're still using static pivot tables to inform strategy, this post is for you ⤵ Take a peak at the below screenshot. This example, which shows two "paired predictors", is just one way you can turn data into action: 📈 ▶ The top right quadrant are “high achievers”. They have a high GPA + high credit earn ratio. These students might simply receive a message of encouragement. ▶ The top left quadrant are “strivers”. They have lower GPAs, but higher credits earned. These students might receive a nudge related to maximizing their use of available academic resources. ▶ The bottom right quadrant are “setbacks”. They have higher overall GPA, likely from good grades in their early coursework, but are earning fewer credits towards graduation requirements in key courses in their major. These students should probably receive messaging about the need for high-touch interaction with their advisors to stay on track and not lose their early momentum. ▶ The students in the bottom left quadrant are in "survival mode”. They are below average in both areas. These students are probably due for some real human-to-human conversation to better understand their needs. They may need in-depth intervention, with accompanied supports for finding the most successful path towards goals that match the students’ strengths and interests. You may consider nudging and re-nudging them throughout a term. ⤵ There's so many more examples of how Civitas Learning partners are disaggregating data to close equity gaps. If you're curious to learn more, let's connect 💌 #studentsuccessanalytics