Training Needs Assessment Methods

Remember that bad survey you wrote? The one that resulted in responses filled with blatant bias and caused you to doubt whether your respondents even understood the questions? Creating a survey may seem like a simple task, but even minor errors can result in biased results and unreliable data. If this has happened to you before, it's likely due to one or more of these common mistakes in your survey design: 1. Ambiguous Questions: Vague wording like “often” or “regularly” leads to varied interpretations among respondents. Be specific—use clear options like “daily,” “weekly,” or “monthly” to ensure consistent and accurate responses. 2. Double-Barreled Questions: Combining two questions into one, such as “Do you find our website attractive and easy to navigate?” can confuse respondents and lead to unclear answers. Break these into separate questions to get precise, actionable feedback. 3. Leading/Loaded Questions: Questions that push respondents toward a specific answer, like “Do you agree that responsible citizens should support local businesses?” can introduce bias. Keep your questions neutral to gather unbiased, genuine opinions. 4. Assumptions: Assuming respondents have certain knowledge or opinions can skew results. For example, “Are you in favor of a balanced budget?” assumes understanding of its implications. Provide necessary context to ensure respondents fully grasp the question. 5. Burdensome Questions: Asking complex or detail-heavy questions, such as “How many times have you dined out in the last six months?” can overwhelm respondents and lead to inaccurate answers. Simplify these questions or offer multiple-choice options to make them easier to answer. 6. Handling Sensitive Topics: Sensitive questions, like those about personal habits or finances, need to be phrased carefully to avoid discomfort. Use neutral language, provide options to skip or anonymize answers, or employ tactics like Randomized Response Survey (RRS) to encourage honest, accurate responses. By being aware of and avoiding these potential mistakes, you can create surveys that produce precise, dependable, and useful information. Art+Science Analytics Institute | University of Notre Dame | University of Notre Dame - Mendoza College of Business | University of Illinois Urbana-Champaign | University of Chicago | D'Amore-McKim School of Business at Northeastern University | ELVTR | Grow with Google - Data Analytics #Analytics #DataStorytelling

𝐖𝐞 𝐚𝐥𝐦𝐨𝐬𝐭 𝐥𝐨𝐬𝐭 𝐚 ₹6,00,000 𝐜𝐨𝐧𝐭𝐫𝐚𝐜𝐭 𝐛𝐞𝐜𝐚𝐮𝐬𝐞 𝐈 𝐫𝐞𝐟𝐮𝐬𝐞𝐝 𝐭𝐨 𝐫𝐞𝐦𝐨𝐯𝐞 𝐨𝐧𝐞 𝐬𝐥𝐢𝐝𝐞. The CEO loved the program. Her leadership team loved the design. Budget was approved. Then the CHRO said: “Can you just skip the pre and post assessments? The team finds evaluations stressful.” I said no. The room went quiet. Here’s what I knew that she didn’t want to hear, without assessment data, this program is just an expensive event. There’s no before, No after, No proof that anything changed. The Reality: 70% of training programs fail to show measurable behavior change, not because the content was bad, but because nobody measured anything to begin with. You can’t manage what you don’t measure. And you can’t defend your training budget in the next board meeting without numbers. They pushed back twice. I held the line twice. We got the contract. With the assessments intact. Here’s what that moment taught me: 1. Your non-negotiables are your credibility: The moment you dilute your methodology to close a deal, you’ve told the client your standards are negotiable. 2. Clients don’t always know what they need: Your job is to protect the outcome, not just the relationship. 3. The right clients respect the pushback: If they don’t, they were never the right fit. Protect your process. It’s the only thing that guarantees your results. 𝐖𝐨𝐮𝐥𝐝 𝐲𝐨𝐮 𝐡𝐚𝐯𝐞 𝐫𝐞𝐦𝐨𝐯𝐞𝐝 𝐭𝐡𝐞 𝐚𝐬𝐬𝐞𝐬𝐬𝐦𝐞𝐧𝐭𝐬 𝐨𝐞 𝐡𝐞𝐥𝐝 𝐭𝐡𝐞 𝐥𝐢𝐧𝐞? #training #leadership

Measuring Success: How Competency-Based Assessments Can Accelerate Your Leadership If it’s you who feels stuck in your career despite putting in the effort. To help you gain measurable progress, one can use competency-based assessments to track skills development over time. ����Why Competency-Based Assessments Matter: They provide measurable insights into where you stand, which areas you need improvement, and how to create a focused growth plan. This clarity can break through #career stagnation and ensure continuous development. 💡 Key Action Points: ⚜️Take Competency-Based Assessments: Track your skills and performance against defined standards. ⚜️Review Metrics Regularly: Ensure you’re making continuous progress in key areas. ⚜️Act on Feedback: Focus on areas that need development and take actionable steps for growth. 💢Recommended Assessments for Leadership Growth: For leaders looking to transition from Team Leader (TL) to Assistant Manager (AM) roles, here are some assessments that can help: 💥Hogan Leadership Assessment – Measures leadership potential, strengths, and areas for development. 💥Emotional Intelligence (EQ-i 2.0) – Evaluates emotional intelligence, crucial for leadership and collaboration. 💥DISC Personality Assessment – Focuses on behavior and communication styles, helping leaders understand team dynamics and improve collaboration. 💥Gallup CliftonStrengths – Identifies your top strengths and how to leverage them for leadership growth. 💥360-Degree Feedback Assessment – A holistic approach that gathers feedback from peers, managers, and subordinates to give you a well-rounded view of your leadership abilities. By using these tools, leaders can see where they excel and where they need development, providing a clear path toward promotion and career growth. Start tracking your progress with these competency-based assessments and unlock your full potential. #CompetencyAssessment #LeadershipGrowth #CareerDevelopment #LeadershipSkills

Most Fabric semantic models get diagnosed backwards. Teams blame data volume first. The actual problem is usually one of five things, none of which is the data. Here's the optimisation checklist I run on every underperforming model: 1. MODE SELECTION — Check this first Is the model in Direct Lake, Import, or DirectQuery? Direct Lake falls back to DirectQuery silently when model size exceeds capacity or unsupported DAX patterns are present. If you haven't checked fallback events in Fabric Capacity Metrics this week, your model may already be in DirectQuery without you knowing. Fix: monitor fallback events. Switch to Import if model exceeds 10GB or query patterns are unpredictable. 2. DAX PATTERNS — The hidden performance killer Three patterns that destroy query performance: RANKX on large tables, EARLIER inside iterators, nested CALCULATE with multiple filter arguments. Fix: replace RANKX with pre-aggregated rank columns in the Gold layer. Restructure EARLIER patterns using variables. Limit CALCULATE nesting to two levels. 3. RELATIONSHIP ARCHITECTURE — Where the debt hides Bidirectional relationships on large tables create filter propagation in both directions. Many-to-many without bridge tables produces Cartesian join behaviour under the hood. Fix: enforce single-direction relationships. Use bridge tables. Remove inactive relationships that serve no current report. 4. REFRESH STACKING — The silent capacity killer Multiple large models refreshing simultaneously compete for the same CU allocation. A model that refreshes fine in isolation fails under concurrent load. Fix: stagger refresh schedules by 20–30 minutes. Set incremental refresh on models over 5GB. Remove models with 0 active consumers. 5. COLUMN AND MEASURE HYGIENE — The accumulation problem Every unused column loaded into the model consumes memory. Every unused measure adds evaluation overhead. Most models I review carry 20–40% redundant columns from deprecated reports never removed. Fix: run Model Analyser in Power BI Desktop. Remove unused columns at the source query level. Archive deprecated measures. Save this for the next slow report complaint. Which of the five does your environment hit most? #MicrosoftFabric #PowerBI #DataEngineering

Designing effective surveys is not just about asking questions. It is about understanding how people think, remember, decide, and respond. Cognitive science offers powerful models that help researchers structure surveys in ways that align with mental processes. The foundational work by Tourangeau and colleagues provides a four-stage model of the survey response process: comprehension, retrieval, judgment, and response selection. Each step introduces potential for cognitive error, especially when questions are ambiguous or memory is taxed. The CASM model -Cognitive Aspects of Survey Methodology- builds on this by treating survey responses as cognitive tasks. It incorporates working memory limits, motivational factors, and heuristics, emphasizing that poorly designed surveys increase error due to cognitive overload. Designers must recognize that the brain is a limited system and build accordingly Dual-process theory adds another important layer. People shift between fast, automatic responses (System 1) and slower, more effortful reasoning (System 2). Whether a user relies on one or the other depends heavily on question complexity, scale design, and contextual framing. Higher cognitive load often pushes users into heuristic-driven responses, undermining validity. The Elaboration Likelihood Model explains how people process survey content: either centrally (focused on argument quality) or peripherally (relying on surface cues). Users may answer based on the wording of the question, the branding of the survey, or even the visual aesthetics rather than the actual content unless design intentionally promotes central processing. Cognitive Load Theory offers tools for managing effort during survey completion. It distinguishes intrinsic load (task difficulty), extraneous load (poor design), and germane load (productive effort). Reducing the unnecessary load enhances both data quality and engagement. Attention models and eye-tracking reveal how layout and visual hierarchy shape where users focus or disengage. Surveys must guide attention without overwhelming it. Similarly, the models of satisficing vs. optimizing explain when people give thoughtful responses and when they default to good-enough answers because of fatigue, time pressure, or poor UX. Satisficing increases sharply in long, cognitively demanding surveys. The heuristics and biases framework from cognitive psychology rounds out this picture. Respondents fall prey to anchoring effects, recency bias, confirmation bias, and more. These are not user errors, but expected outcomes of how cognition operates. Addressing them through randomized response order and balanced framing reduces systematic error. Finally, modeling approaches like like cognitive interviewing, drift diffusion models, and item response theory allow researchers to identify hesitation points, weak items, and response biases. These tools refine and validate surveys far beyond surface-level fixes.

A good survey works like a therapy session. You don’t begin by asking for deep truths, you guide the person gently through context, emotion, and interpretation. When done in the right sequence, your questions help people articulate thoughts they didn’t even realize they had. Most UX surveys fall short not because users hold back, but because the design doesn’t help them get there. They capture behavior and preferences but often miss the emotional drivers, unmet expectations, and mental models behind them. In cognitive psychology, we understand that thoughts and feelings exist at different levels. Some answers come automatically, while others require reflection and reconstruction. If a survey jumps straight to asking why someone was frustrated, without first helping them recall the situation or how it felt, it skips essential cognitive steps. This often leads to vague or inconsistent data. When I design surveys, I use a layered approach grounded in models like Levels of Processing, schema activation, and emotional salience. It starts with simple, context-setting questions like “Which feature did you use most recently?” or “How often do you use this tool in a typical week?” These may seem basic, but they activate memory networks and help situate the participant in the experience. Visual prompts or brief scenarios can support this further. Once context is active, I move into emotional or evaluative questions (still gently) asking things like “How confident did you feel?” or “Was anything more difficult than expected?” These help surface emotional traces tied to memory. Using sliders or response ranges allows participants to express subtle variations in emotional intensity, which matters because emotion often turns small usability issues into lasting negative impressions. After emotional recall, we move into the interpretive layer, where users start making sense of what happened and why. I ask questions like “What did you expect to happen next?” or “Did the interface behave the way you assumed it would?” to uncover the mental models guiding their decisions. At this stage, responses become more thoughtful and reflective. While we sometimes use AI-powered sentiment analysis to identify patterns in open-ended responses, the real value comes from the survey’s structure, not the tool. Only after guiding users through context, emotion, and interpretation do we include satisfaction ratings, prioritization tasks, or broader reflections. When asked too early, these tend to produce vague answers. But after a structured cognitive journey, feedback becomes far more specific, grounded, and actionable. Adaptive paths or click-to-highlight elements often help deepen this final stage. So, if your survey results feel vague, the issue may lie in the pacing and flow of your questions. A great survey doesn’t just ask, it leads. And when done right, it can uncover insights as rich as any interview. *I’ve shared an example structure in the comment section.

Often when we ask caregivers what they need, they say information. Curious health, social, and community providers will ask: Information about what? Curiosity is excellent. But what about having a structured way to ask? What about a needs assessment tool designed to elicit caregiver-defined support needs — not just burden or stress? We wondered: What caregiver needs’ assessment tools already exist? Do they truly capture what caregivers say they need? And why aren’t these tools routinely used in practice? Our rapid scoping review — just published — begins to answer those questions: “Mapping Caregiver Needs’ Assessment Tools for Family and Friend Caregivers: A Rapid Scoping Review” 📍 International Journal of Environmental Research and Public Health (Vol. 23, Issue 3) We identified 19 caregiver needs’ assessment instruments (17 instrument families) across 43 studies. What we found was both encouraging and concerning. Across tools, we identified seven domains of caregiver-defined support needs: 1️⃣ Caregiver health and self-care 2️⃣ Emotional and psychological support 3️⃣ Information, communication, and navigation 4️⃣ Practical and instrumental support 5️⃣ Social and relational support 6️⃣ Autonomy and life participation 7️⃣ Spiritual, cultural, and existential support Information and navigation were most frequently assessed. Autonomy and spiritual domains were least represented. Importantly, many instruments demonstrated what we call “construct drift.” Instead of explicitly eliciting caregiver-defined support needs, they often measured burden, strain, or preparedness. These are important constructs — but they are not the same as asking caregivers what support they need. We also found: Few tools are designed for longitudinal reassessment Limited attention to workflow integration Minimal integration into electronic medical records Limited support for interdisciplinary care pathways If we are serious about embedding caregiver-centered care into routine practice, we need tools that: ✔ Explicitly elicit caregiver-defined support needs ✔ Support ongoing reassessment ✔ Integrate into clinical workflows ✔ Enable documentation and shared care planning Caregivers are already integral to health and social systems. Our assessment tools should reflect that reality. I’d love to hear from clinicians, leaders, and researchers: Are you using a caregiver needs’ assessment tool in practice? If so, how is it working? #CaregiverCenteredCare #FamilyCaregivers #HealthSystemTransformation #IntegratedCare #CarePartners

Data collection is the foundation of credible research and evidence-based decision-making. The Guide to Effective Data Collection provides a complete roadmap for designing and executing high-quality surveys that deliver accurate, actionable insights. It walks through every stage—from defining research questions and indicators to selecting data collection methods, writing strong survey questions, and managing field implementation. Key highlights include: ↳ The importance of data quality and how poor survey design leads to unreliable results ↳ Developing clear research questions, outcomes, and measurable indicators ↳ Comparing data collection methods such as observation, interviews, questionnaires, and focus group discussions ↳ Applying qualitative and quantitative approaches effectively ↳ Using the MECE (Mutually Exclusive, Collectively Exhaustive) framework for clear and consistent survey questions ↳ Sampling strategies, including probability and non-probability techniques, and reducing sampling errors ↳ Piloting surveys, training field teams, and ensuring ethical and accurate implementation Good data starts with good design. Reliable evidence depends on rigorous planning, sound methodology, and strong execution. #DataCollection #Research #SurveyDesign #MonitoringAndEvaluation #DataQuality #ImpactMeasurement #QuantitativeResearch #QualitativeResearch #EvidenceBasedDecisionMaking #LearningAndAccountability

I'm excited to share our latest paper led by Orestis Efthimiou, "Measuring the Performance of Prediction Models to Personalize Treatment Choice," now published in Statistics in Medicine! In this study, we introduce a framework for evaluating prediction models designed to estimate individualized treatment effects, and propose three key dimensions: (1) Discrimination for Benefit – Can the model correctly distinguish who benefits from treatment? (2) Calibration for Benefit – How well does the model quantify treatment effects at the individual level? (3) Decision Accuracy – Does the model correctly identify patients for whom treatment benefit exceeds a given threshold? We illustrate these approaches using simulated data and a real-world depression trial dataset. All methods are implemented in the R package predieval , available from CRAN. These methods will also be discussed in our upcoming book. A big thank you to my fantastic co-authors Orestis Efthimiou, Jeroen Hoogland, Michael Seo, Toshiaki A. Furukawa, Matthias Egger, and Ian White! 🎉 https://lnkd.in/dPg7Jy2q #CausalInference #PersonalizedMedicine #MachineLearning #PredictionModels #Biostatistics #ClinicalTrials #Rstats #Statistics

I'm pleased to make available my upcoming DATE 2025 paper, the result of a project led by my PhD student Nicholas Wendt together with Mahesh Ketkar from Intel and myself. Nicholas prepared a crystal-clear video presentation, which makes the paper's complex concepts easy to understand. The paper is titled: "SPIRE: Inferring Hardware Bottlenecks from Performance Counter Data". The paper introduces SPIRE (Statistical Piecewise Linear Roofline Ensemble), a novel performance modeling approach that combines the accessibility of roofline models with the detailed insights of hardware performance counters. Unlike existing performance analysis tools like VTune or Perfmon, SPIRE generates an ensemble of piecewise linear roofline models trained on performance counter data to estimate a processor’s maximum throughput and identify bottlenecks. It uses the models to automate the interpretation of the performance counter measurements, quickly zeroing in on microarchitectural bottlenecks such as front-end stalls, memory latency, and core execution inefficiencies. Unlike traditional analysis tools that require architecture-specific tuning, SPIRE automatically learns processor characteristics, making it applicable across different architectures with minimal deployment effort. This automated and generalized approach provides accurate performance insights, aiding both software optimizations and hardware design improvements. You can see Nicholas' short presentation here: https://lnkd.in/gcEBcub7 And you can read the full paper here: https://lnkd.in/gW5xhpi4 #computerarchitecture #performanceanalysis #research