A/B Testing in Marketing

Everyone’s excited to launch AI agents. Almost no one knows how to measure if they’re actually working. Over the last year, we’ve seen brands launch everything from GenAI assistants to support bots to creative copilots but the post-launch metrics often look like this: • Number of chats • Average latency • Session duration • Daily active users Useful? Yes. But sufficient? Not even close. At ALTRD, we’ve worked on AI agents for enterprises and if there’s one lesson it’s this: Speed and usage mean nothing if the agent isn’t solving the actual problem. The real performance indicators are far more nuanced. Here’s what we’ve learned to track instead: 🔹 Task Completion Rate — Can the AI go beyond answering a question and actually complete a workflow? 🔹 User Trust — Do people come back? Do they feel confident relying on the agent again? 🔹 Conversation Depth — Is the agent handling complex, multi-turn exchanges with consistency? 🔹 Context Retention — Can it remember prior interactions and respond accordingly? 🔹 Cost per Successful Interaction — Not just cost per query, but cost per outcome. Massive difference. One of our clients initially celebrated their bot’s 1 million+ sessions - until we uncovered that less than 8% of users actually got what they came for. That 8% wasn’t a usage issue. It was a design and evaluation issue. They had optimized for traffic. Not trust. Not success. Not satisfaction. So we rebuilt the evaluation framework - adding feedback loops, success markers, and goal-completion metrics. The results? CSAT up by 34% Drop-off down by 40% Same infra cost, 3x more value delivered The takeaway: Don’t just measure what’s easy. Measure what matters. AI agents aren’t just tools - they’re touchpoints. They represent your brand, shape user experience, and influence business outcomes. P.S. What’s one underrated metric you’ve used to evaluate AI performance? Curious to learn what others are tracking.

5 Powerful Metrics to Showcase Your Impact as a UX Researcher (Perfect for Your CV & Reviews) As UX researchers, proving our value is just as important as delivering it. From performance reviews to CVs, stakeholder updates, project wrap ups, and even budget discussions we’re constantly finding ways to illustrate the influence of our work. But what should you actually measure? Here are 5 impactful metric categories you can use to demonstrate your contribution and influence with examples you can plug straight into your CV or next review: 1. Activity Metrics The scope and scale of research activities you (or your team) lead interviews conducted, usability tests run, insights delivered. 2. Internal Metrics How your work drives internal alignment influencing decisions, sparking cross-team collaboration, shaping strategies. 3. Product Metrics The tangible improvements in the product post research reduced friction, increased task success, higher engagement. 4. Business Metrics The measurable business outcomes your research supports revenue uplift, reduced churn, cost savings, higher conversions. 5. Context Metrics The breadth, reach, and visibility of your work global vs. local impact, diverse audiences, strategic initiatives. By tracking and sharing these metrics, you can clearly communicate how your research fuels better products, smarter decisions, and stronger business outcomes.

If you’re still testing creatives the same way you did in 2022… you’re already losing. 2025 is a different animal. More platforms. More formats. More distractions. And algorithms that reward speed of iteration over “perfect” production. Here’s the truth no one likes to admit: Your creative testing process is probably the single biggest choke point in your growth. 1. Volume > Perfection I get it. You want that one polished, cinematic ad that “crushes it” for six months. But the reality? That’s a lottery ticket. The brands winning in 2025 aren’t looking for a unicorn. They’re running dozens of creative variations every week, different hooks, angles, and formats, because they know frequency of testing beats guessing right. The question is no longer: “Will this ad work?” It’s: “How fast can we find the winners and kill the losers?” 2. Hooks Are the New Battlefield: Scroll speed is at an all-time high. If your hook doesn’t land in 1.5 seconds, you’re invisible. I break hooks into three buckets when testing: Problem-first: Agitate the exact pain point your audience feels. Pattern interrupt: Visual or copy that makes them stop mid-scroll. Social proof: Start with customer words, not brand words. In testing, the hook is the variable I isolate first, because it’s the highest-leverage change you can make for a faster read on performance. 3. Test for the Funnel Stage, Not Just the Platform: Too many marketers test “TikTok creatives” or “Meta creatives” without thinking about where in the customer journey they’re being shown. Your mid-funnel retargeting ad doesn’t need to be a 30-second cinematic story. It might just need a quick, benefit-driven product demo with a direct CTA. Conversely, cold traffic needs more narrative and education. Test creative by awareness stage, not just ad placement. 4. Creative → Landing Page Consistency: One of the biggest creative testing fails I see? Finding a winning ad… and sending traffic to a generic, unaligned landing page. In 2025, ad-to-landing-page congruence is non-negotiable. Your hero image, headline, and CTA on the LP should feel like the next chapter of the ad’s story, not a reset. We test landing page variants in parallel with ads, because the best creative in the world can’t fix a conversion drop-off. 5. Data Loops, Not Guess Loops: If your creative team isn’t meeting weekly with your media buyers, you’re leaving money on the table. We run tight data loops: Media buyers bring performance data. Creative team uses that data to inform the next round of variations. No egos. No “pet” creatives. If it’s not performing, it dies. In 2025, creative strategy is just as much a numbers game as it is an art.

Meta ads have changed AGAIN. And the advice most marketers are still following? Completely outdated. Here’s the so-called “expert wisdom” I still hear every day: ❌ “Audience testing doesn’t matter” ❌ “Launch 50 creatives every week” ❌ “Just keep tweaking the same ad” ❌ “Horizontal scaling is the way to grow” ❌ “Your account structure is everything” ❌ “Track CPM obsessively” ❌ “Always sell benefits” Sounds familiar? The truth: this playbook is backdated. But then what does work in 2025? After working across accounts with ₹50Cr+ in ad spend, here’s what’s actually driving results right now: ✅ Mobile-first video creatives win. Reels + Stories with thumb-stopping hooks in the first 3 seconds crush static ads. Meta’s algorithm thrives when you feed it creative variety. ✅ First-party data is gold. Syncing customer lists (buyers, subscribers, high-LTV users) → building value-based lookalikes = 25–40% lower CAC. ✅ Advantage+ & AI-led optimization. Advantage+ Shopping + dynamic creative testing consistently outperform manual setups, with 20–30% stronger ROAS. ✅ UGC > Studio-polished ads. Raw, authentic content-unboxings, testimonials, everyday use-gets up to 3–4× higher engagement than polished creative. If you’re still stuck on old “best practices,” you’ll burn budgets fast. But if you lean into these new levers, Meta ads still scale like crazy in 2025. 👉 What’s one Meta ads “rule” you think is completely dead in 2025? Drop it in the comments - I’ll share my take on it. #PerformanceMarketing #MetaAds #PaidSocial #EcommerceGrowth #DigitalMarketing #MarketingStrategy

This is not a new marketing idea… In fact, it’s one of the oldest tricks in the beauty industry and it’s making a comeback. Decades ago, Avon and Oriflame made scratch-and-sniff perfume samples an everyday experience. They tucked fragrance strips into catalogues and magazines, allowing customers to smell new launches while sitting at home, no stores, no queues, no sales pressure. Luxury brands followed suit. Calvin Klein, Dior, Chanel, Lancôme, Gucci, YSL - all used scented pages in Vogue, Elle, and Harper’s Bazaar. You’d flip the page, rub your wrist, and instantly decide if that perfume was “you.” The magic? You experienced the product at your own pace, in your own environment, without the pressure of a salesperson hovering nearby. Now in 2025… it’s trending again. Rare Beauty just took the same concept and reimagined it for Gen Z and Millennials with their “Snatch & Sniff” billboards in New York. You’re walking down the street in SoHo, you spot the billboard, you peel a strip, smell it… and maybe, you film it for TikTok. It’s the same strategy Avon and Chanel used in print, but recontextualised for the urban, content-driven, on-the-go world we live in now. Why this works brilliantly today: Self-directed experience → No sales pitch. You control the moment. Frictionless product trial → No counter, no tester bottle, no store visit. Sensory engagement → People don’t just see your brand; they smell and interact with it. Built for virality → Every peel-off could be a TikTok or Instagram story. Memorable → You remember where you were when you first smelled it. The real marketing lesson: - You don’t always need to invent something brand new. - You need to find timeless strategies that worked… and adapt them to a new context, platform, and audience. From coffee tables in the ‘80s to NYC streets in 2025, the principle is the same: Let the product sell itself. Would you stop for a snatch & sniff billboard? Or would you walk right past it? 👇 ----- Repost ♻️ to help your network ----- Follow Priya Kodali for more like this #MarketingStrategy #LuxuryMarketing #ExperientialMarketing #BrandActivation #RareBeauty #FragranceMarketing #OutdoorAdvertising #BrandEvolution

🔍 Design Metrics in the Era of AI The shift towards AI-powered products impacted not only how we design products but also how we measure design success. Traditional design metrics such as task success rate, time on task, error rate, and satisfaction (SUS/NPS) work well for deterministic, human-controlled systems, but AI-powered systems, however, are probabilistic and adaptive. The focus shifts from “did the user complete the task?” to “did the system collaborate effectively with the user to reach intent?” Here are 4 core dimensions of metrics that will help you measure AI power systems 1️⃣ Collaboration Quality It measures how efficiently human and AI co-create, not just how fast the task finishes. Metric examples:  ✓ Correction rate ✓ Number of re-prompts ✓ “Undo” frequency ✓ Time to acceptable output 2️⃣ Model Transparency This helps understand whether users grasp why AI made a certain choice. It is a key predictor of trust and long-term adoption. Metric examples:  ✓ Perceived explainability ✓ Satisfaction with rationale visibility 3️⃣ Personalization Efficacy Track whether adaptive systems genuinely learn user preferences. Metric examples:  ✓ Relevance score ✓ Personalization satisfaction ✓ % of successful reuse of generated assets 4️⃣ Emotional Trust & Safety Ensure that AI interactions feel supportive, not invasive or manipulative. Metric examples:  ✓ Trust index ✓ Perceived safety ✓ Emotional comfort (via surveys or sentiment analysis) ❗ Does it mean that we should abandon our traditional product metrics when building an AI-powered product? Absolutely not. In fact, we should use a hybrid measurement framework that will have a balanced set of metrics that combine quantitative, qualitative, and behavioral signals: ✅ System performance: measure model accuracy, latency, and hallucination rate. Use telemetry and LLM evaluation sets for that.  ✅ Human experience: measure trust, satisfaction, correction rate, and transparency. Use surveys, in-app feedback for that.  ✅ Business impact: retention, repeat usage, outcome efficiency. Use analytics, A/B testing for that.  ✅ Ethical dimension: bias incidents, fairness perception. Use audits, user interviews. #UX #design #measure #productdesign #uxdesign

Most “creative testing” isn’t testing at all. ❌ What most brands call testing looks like this: - Swap a background color - Change the CTA button - Recut a 15s video into 12s That’s not testing concepts. That’s tweaking. And tweaks won’t scale. Here’s what true creative testing actually looks like: 1️⃣ Test Big Levers, Not Pixels ↳ Messaging angles, hooks, story structures, and offers drive performance, design polish doesn’t. ↳ Each new concept should explore a unique customer insight, not a color palette. 2️⃣ Volume is the Multiplier ↳ One new ad a month isn’t testing, it’s gambling. ↳ High-velocity testing means dozens of new concepts, fast iteration cycles, and letting data pick the winners. 3️⃣ Kill Fast, Scale Winners ↳ Read signals early: thumbstop rates, CTR, cost per add-to-cart. ↳ Don’t wait weeks,ads tell you in 48 hours if they’re worth scaling. 4️⃣ Iteration > Reinvention ↳ Once you have a winning concept, spin 10-20 variants: new hooks, new visuals, different headlines. ↳ Keep what works, evolve what doesn’t. The best brands treat creative testing as R&D for growth. Not a side project. Not a design exercise. A system that fuels scale. Found this useful? Like, follow, and repost ♻️ so others can too! ps. struggling with creative bottlenecks? We can help.

The top ad hooks of 2025 aren’t what you think. I analyzed 5000+ ads to prove it. Generally creative strategists + media buyers will tell you to: ❌ State the problem ❌ Show your product in the first 3 seconds But the top performers I’m seeing look a lot more top of funnel: think more “content-first” vs typical problem-solution ads. Here’s what actually is working: 1. Investment Hooks: Explain the “investment” (time or money) someone wasted before finding your product. Example: “I spent over $100 on SOCKS…” 👆 You fell for this one already. That was the hook of this post. 2. Scam Hooks: Even just using the word “scam” can spike performance. Why? It triggers loss aversion + creates a curiosity gap. Example: “I honestly thought Ryze was a scam…” 3. POV You hate X: 15% of top-performing hooks we tested began with “POV.” Pair it with the word “hate” → even better. Because sometimes people bond faster over what they dislike than what they like. In this week’s episode, I break down 7 more hooks that are driving performance in the second half of 2025. These are easy wins you can test in your next round of creative. 👉 Curious which one you’d test first? https://lnkd.in/en6qfTf3

AG1 is pulling back on spend as their performance slows. When AG1 Makes move, DTC founders should watch. Look at their creative launch cadence: May '24: ~650 creatives Jan '25: ~1,950 creatives (3x increase!) May '25: ~850 creatives (way down again) They've pulled back in prep for Q3/Q4, health is a tough focus during and after summer, people are busy, and then going back to school. AG1 is keeping their ad budgets and total creatives balanced. They've kept their videos (more top of funnel) and trimmed statics (lower funnel) as they build up awareness for Q4. Smart brands align creative production with spend forecasts. After testing 10,000+ ads on Facebook and analyzing how category leaders like AG1 operate, here's the framework for deciding your monthly creative output: Budget First, Volume Second 1. Your ad production should scale with spend, not vanity metrics. 2. Target: ~1 CONCEPT per $10k in monthly spend 3. Spending $500k/mo? That's 50 concepts. 4. If your ads fatigue fast or you're in rapid growth mode, increase production. Split by Format Based on Your Product 5. Start with a 70% video spend target for ecom. 6. Does your product need explanation? More videos. 7. For our $500k example: 35 Video concepts / 15 Static concepts Variations Per Concept 8. Videos: 2-3 hooks per concept & Statics: 5-8 variations per concept 9. Most video variation testing should be hooks 10. That's 70 videos and 90 statics for a $500k/mo brand. New vs. Iterations 11. For established brands: aim for roughly 50/50 split 12. Track everything. This ratio will shift based on performance, but make sure you're not just iterating on winners OR only launching untested concepts. ONLY NOW Choose Your Styles New concepts: Talk to customers, analyze competitors, study organic content in your niche Iterations: Take top performers and test new creators, re-edit with new hooks/music/captions, ask winning creators for their own variations Example breakdown for 35 video concepts: - 5 Manufacturing/installation - 5 Founder stories - 5 Promotion concepts - 2 Street interviews - 3 Customer interviews - 5 UGC concepts - 10 Other based on performance data Then cut the losers, AG1 is cutting 70%+ of their creatives before they make it to two weeks, thats thousands of creatives only lasting days before getting culled. Why this works: - Your creative team gets clarity (creativity thrives in constraints) - Your media buyers get options (options breed winners, winners create scale) - You won't overspend testing low-quality AI slop "because more is better" - This exact strategy has taken brands from $20k → $85M and $50M → $150M/year in revenue with better margins. Having problems with creative?  Email me @ Curtis@misfitmarketing.co I operate on inbox 0, Ill get back to you asap.