Interactive Design with Machine Learning

A few months ago, this wasn’t even part of my hiring process. Now it’s one of the first things I look at. Recently, I interviewed two designers for the same role. Both had strong portfolios. Both understood modern UI. Both could use Figma well. But one question changed the entire conversation: “How do you use AI in your design workflow?” One designer said: “I use ChatGPT sometimes for content ideas.” The other designer showed me how they use AI to: turn rough client briefs into structured UX flows generate multiple user journey ideas in minutes speed up UX writing organize research findings improve accessibility checks explore layout directions faster before moving into UI And honestly… The gap was impossible to ignore. Not because AI made them more creative. ↳ But because it made them more efficient. That’s the shift happening right now in design. AI is no longer just a tool designers casually experiment with. It’s becoming part of the workflow. Especially after tools like Claude started changing how designers think about execution, ideation, and speed. After 18 years in UX and leading a design agency, here’s what I’m noticing: The designers growing the fastest right now are not necessarily the ones with the flashiest visuals. They’re the ones who know: what to automate what to simplify and where human thinking still matters most So if you’re a designer trying to stay ahead, start here: Step 1: Use AI before opening Figma Most designers still jump straight into UI. Instead, ask AI: “Act as a UX strategist. Help me plan the structure for a [project type].” Ask for: user pain points user flows feature suggestions onboarding ideas information architecture You’ll start designing with more clarity from the beginning. Step 2: Use AI to speed up UX thinking AI shouldn’t replace your process. ↳ It should remove friction from it. Ask: “Review this landing page structure and identify: possible UX issues confusing sections weak hierarchy drop-off risks” You’ll save hours of manual review. Step 3: Use AI as a design reviewer This part is underrated. Upload your screen and ask: “Act as a senior UX reviewer. Give me honest feedback on: usability accessibility hierarchy CTA clarity cognitive load” Sometimes AI catches things your own eyes miss after staring at a screen too long. That’s where the industry is heading. Not toward “AI replacing designers.” But toward designers who know how to combine: ✓ design thinking ✓ human empathy ✓ and AI efficiency Because clients are starting to expect faster thinking, faster iteration, and smarter workflows. And AI is now part of that expectation. Are designers adapting fast enough? (If this resonated, repost it ♻️)

🪂 How To Make Your Design System AI-Ready (https://lnkd.in/dtnpy7CM), a practical guide on how to reduce drifts, minimize mistakes, maintain context and improve the quality of AI-generated prototypes — with structured spec files, automated auditing and token layers. Put together by Hardik Pandya from Atlassian. --- 🔹 1. Design Decisions Are Infrastructure AI-generated prototypes often don't deliver consistently decent results because of tiny inconsistencies scattered all across a design system. Often it's decisions made but not documented, hard-coded values never cleaned up, or relying too much on AI making sense of mock-ups or design flows on its own. Unsurprisingly, better AI prototypes come from better data — but also from better human guidance. We shouldn’t assume that AI knows how to choose the right component, and how to design with accessibility in mind. It needs priorities, a clear path on how we make decisions, design principles, examples, do's and don'ts. In fact, we should treat design decisions as infrastructure. That means that every time we make a decision — not just a design decision, but even decision on how actually prioritize our work and how we make decisions around here — it must find a path into the spec file that is then consumed by AI. --- 🔶 2. Three Layers: Spec Files + Token Layer + Audit To ensure quality, we establish design principles, guidelines, rules in a form of “spec files”). It's structured Markdown files that include spacing rules, color choices, component usage guidelines, priorities etc. AI is going to read and reuse that spec file every time it's going to generate a prototype. Because the spec files are text files, it's much more cost-effective, but also much more accurate just because we don't rely on AI recognizing or decoding patterns from mock-ups, but gets specific guidelines instead. In fact, extending code is often a more effective way than generating code from mock-ups. Token layer lists and keeps updated all tokens used throughout the design system. AI always chooses from a closed set of named variables instead of inventing plausible values ad-hoc. An audit script catches what AI gets wrong. It scans the prototype and flags every hard-coded value and flags it if necessary. It can be a regular software doing that, with AI waiting for its feedback to come back. Finally, when a design system ships updates, a sync routine flags which spec files need updating. The goal is to make sure that AI always reads up-to-date, current specs, not the ones written against an outdated version. --- 🔺 3. Examples of AI-Ready Design Systems ⌾ Atlassian: https://lnkd.in/dVsGc3Cp ⌾ Carbon: https://lnkd.in/d4zq4WWb ⌾ CMS Design System: https://lnkd.in/dHHzV3en ⌾ Nordhealth: https://lnkd.in/d8C4j2ZA Yet again, AI can’t magically resolve technical debt or design debt — it needs guidance, decisions, priorities and principles.

Product managers & designers working with AI face a unique challenge: designing a delightful product experience that cannot fully be predicted. Traditionally, product development followed a linear path. A PM defines the problem, a designer draws the solution, and the software teams code the product. The outcome was largely predictable, and the user experience was consistent. However, with AI, the rules have changed. Non-deterministic ML models introduce uncertainty & chaotic behavior. The same question asked four times produces different outputs. Asking the same question in different ways - even just an extra space in the question - elicits different results. How does one design a product experience in the fog of AI? The answer lies in embracing the unpredictable nature of AI and adapting your design approach. Here are a few strategies to consider: 1. Fast feedback loops : Great machine learning products elicit user feedback passively. Just click on the first result of a Google search and come back to the second one. That’s a great signal for Google to know that the first result is not optimal - without tying a word. 2. Evaluation : before products launch, it’s critical to run the machine learning systems through a battery of tests to understand in the most likely use cases, how the LLM will respond. 3. Over-measurement : It’s unclear what will matter in product experiences today, so measuring as much as possible in the user experience, whether it’s session times, conversation topic analysis, sentiment scores, or other numbers. 4. Couple with deterministic systems : Some startups are using large language models to suggest ideas that are evaluated with deterministic or classic machine learning systems. This design pattern can quash some of the chaotic and non-deterministic nature of LLMs. 5. Smaller models : smaller models that are tuned or optimized for use cases will produce narrower output, controlling the experience. The goal is not to eliminate unpredictability altogether but to design a product that can adapt and learn alongside its users. Just as much as the technology has changed products, our design processes must evolve as well.

Most of the AI-meets-design conversation right now is about converting. Designs to code. Code to designs. Back and forth. It's great. But what about creating? I started with 4 things: → A blank Figma canvas → Claude Code → Figma Console MCP → Material 3's component library One prompt: "build a mobile fintech login screen using the existing components and tokens." Claude analyzed the full design system, picked the right components, set the right properties, and composed the layout directly on the canvas. Real components, real variables, fully bound to tokens. But I didn't stop there! THEN, I asked it to invent a Brutalist theme. → It spun up one of our custom UI designer sub-agents → Created a new variable mode from scratch (acid yellow, zero radii, Space Mono) → cloned the original layout, and restyled everything Same components, completely different look/feel. Switch modes and it all holds together. 15 minutes. Start to finish. The magic is how to stack tooling, not a single tool. MCP for the canvas, Claude Code for orchestration, sub-agents for specialized design thinking, and a solid design system underneath it all (very important). This is a creative tool, not just a conversion tool. Style exploration, mood boards, rapid variable mode testing, pushing your token architecture to see what it can handle... I did this in 15 minutes. I want to see what you can do in an hour. Grab the Figma Console MCP, plug in your design system, and show me! If you need help getting set up or want to talk about making your design system AI-ready, reach out. Check out the new easy-to-follow community setup guides - https://lnkd.in/eNmzhh5S

Love this analogy for the emerging chapter of UX: "We’ve moved from designing 'waterslides,' where we focused on minimizing friction and ensuring fluid flow — to 'wave pools,' where there is no clear path and every user engages in a unique way." That's Alex Klein in this article: https://lnkd.in/eRpmzUEd Over the past several years, the more that I’ve worked with AI and machine learning—with robot-generated content and robot-generated interaction—the more I’ve realized I’m not in control of that experience as a designer. And that’s new. Interaction designers have traditionally designed a fixed path through information and interactions that we control and define. Now, when we allow the humans and machines to interact directly, they create their own experience outside of tightly constrained paths. This has some implications that are worth exploring in both personal practice and as an industry. We’ve been working in all of these areas in our product work at Big Medium over the past few years SENTIENT DESIGN. This is the term I’ve been using for AI-mediated interfaces. When the robots take on the responsibility for responding to humans, what becomes possible? What AI-facilitated experiences lie beyond the current fascination with chatbots? How might the systems themselves morph and adapt to present interfaces and interaction based on the user’s immediate need and interest? This doesn’t mean that every interface becomes a fever dream of information and interaction, but it does mean moving away from fixed templates and set UI patterns. DEFENSIVE DESIGN. We’re used to designing for success and the happy path. When we let humans and robots interact directly, we have to shift to designing for failure and uncertainty. We have to consider what could go wrong, how to prevent those issues where we can, and provide a gentle landing when we fail. PERSONA-LESS DESIGN. As we get the very real ability to respond to users in a hyper-personalized way, do personas still matter? Is it relevant or useful to define broad categories of people or mindsets, when our systems are capable of addressing the individual and their mindset in the moment? UX tools like personas and journey maps may need a rethink. At the very least, we have to reconsider how we use them and in which contexts of our product design and strategy. These are exciting times, and we’re learning a ton. At Big Medium, we’ve been working for years with machine learning and AI, but we’re still discovering new interaction models every day—and fresh opportunities to collaborate with the robots. It’s definitely a moment to explore, think big, and splash in puddles—or as Klein might put it, leave the waterslide to take a swim in the wave pool.

UX designers are being left behind. These 11 skills will keep you relevant. I've been running a UX design firm for 20 years. I've never seen things change so quickly. The role is transforming faster than most designers are willing to adapt. 𝗛𝗲𝗿𝗲'𝘀 𝘁𝗵𝗲 𝗽𝗿𝗼𝗯𝗹𝗲𝗺: • Designers are still thinking about interfaces when they should be thinking about influencing design beneath the surface (i.e., the substrate) • They're treating AI like a nice-to-have tool instead of a fundamental shift in how design work gets done at a system level. The reality is, UX designers need to reskill. Fast. Because the designers who master AI-augmented design will leave everyone else behind. 𝗛𝗲𝗿𝗲 𝗮𝗿𝗲 𝘁𝗵𝗲 𝟭𝟭 𝗔𝗜 𝗨𝗫 𝗱𝗲𝘀𝗶𝗴𝗻 𝘀𝗸𝗶𝗹𝗹𝘀 𝘆𝗼𝘂 𝗻𝗲𝗲𝗱 𝘁𝗼 𝗺𝗮𝘀𝘁𝗲𝗿: 1. 𝗣𝗿𝗼𝗺𝗽𝘁 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 • Craft structured prompts that guide AI to generate wireframes, flows, and copy aligned with design intent 2. 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗙𝗿𝗮𝗺𝗶𝗻𝗴 • Build project context for AI tools (personas, journeys, brand tone) to improve design relevance 3. 𝗔𝗜 𝗪𝗶𝗿𝗲𝗳𝗿𝗮𝗺𝗶𝗻𝗴 • Use AI to auto-generate design layouts, screen flows, and interaction patterns 4. 𝗘𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲 𝗦𝗶𝗺𝘂𝗹𝗮𝘁𝗶𝗼𝗻 • Leverage AI to test designs through simulated user behavior and eye-tracking prediction 5. 𝗔𝗜 𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗦𝘆𝗻𝘁𝗵𝗲𝘀𝗶𝘀 • Automate analysis of interviews, surveys, and session data to uncover key insights 6. 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗦𝘁𝗼𝗿𝘆𝗯𝗼𝗮𝗿𝗱𝗶𝗻𝗴 • Use AI to visualize user journeys or product scenarios 7. 𝗩𝗶𝗯𝗲 𝗗𝗲𝘀𝗶𝗴𝗻 • Rapidly prototype and validate ideas through natural language conversation with AI 8. 𝗔𝗰𝗰𝗲𝘀𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝘆 𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗼𝗻 • Leverage AI to detect accessibility gaps and generate alternative formats 9. 𝗘𝘁𝗵𝗶𝗰𝗮𝗹 & 𝗥𝗲𝘀𝗽𝗼𝗻𝘀𝗶𝗯𝗹𝗲 𝗔𝗜 𝗗𝗲𝘀𝗶𝗴𝗻 • Design experiences that are transparent, fair, and aligned with user trust principles 10. 𝗗𝗲𝘀𝗶𝗴𝗻 𝗪𝗼𝗿𝗸𝗳𝗹𝗼𝘄 𝗢𝗿𝗰𝗵𝗲𝘀𝘁𝗿𝗮𝘁𝗶𝗼𝗻 • Integrate multiple AI tools (research, prototyping, copy) into a single creative pipeline 11. 𝗛𝘂𝗺𝗮𝗻-𝗔𝗜 𝗖𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝗼𝗻 • Master when to lead, when to delegate, and how to iterate with AI as a design partner Here's what separates designers who grow from designers who struggle: The ones who advance aren't just using AI for faster output. They're thinking about design at a system level. How can I influence how AI responds to context? How can I alter the experience at the substrate level? They've moved from designing interfaces to designing conversations and context. The good news? These skills are learnable. The bad news? Things are moving fast, so you need to start thinking two steps ahead. Complacency will get you in trouble in this market. --- ♻️ Share if this resonates ➕ Follow Jason Moccia for more insights on growth and leadership

Most #AI interfaces still assume users are prompt whisperers. But let’s be honest: In the real world, people don’t want to “#tune” prompts. They want to get things done — precisely, quickly, reliably. That’s why I found Microsoft Research’s latest work on Promptions worth highlighting. It’s a lightweight framework that wraps dynamic UI controls around prompts, so users can steer generative AI without starting from scratch every time. What stood out 👇 1️⃣ Prompting is becoming interactive Users get buttons, sliders, toggles — not just a blinking cursor. 2️⃣ Controls evolve with context As the conversation flows, so do the options — making interaction feel intelligent, not rigid. 3️⃣ No more prompt fatigue Users in early studies got more accurate outputs with less mental strain. 4️⃣ Works across models Promptions is model-agnostic. Use it with GPT, local LLMs, or internal enterprise agents. 5️⃣ Developer-friendly It’s open-source (MIT license) and easily pluggable into existing agent UIs. 6️⃣ Built for productivity Whether you're generating copy, answering support queries, or analyzing data — it guides users to outcomes faster. 7️⃣ Reimagines prompting as design This moves the interface from words-as-code to controls-as-clarity. Bottom line: Promptions shifts the paradigm — from prompt engineering to prompt experience design. And that opens doors for scalable, low-friction AI use across the enterprise. 🔗 Read the full research blog here: https://lnkd.in/g4KMRTWu #UXForAI #PromptDesign #ProductivityAI #AgentUX #AIInteraction #MicrosoftResearch

AI is finding its interface. The chatbox was always temporary. We kind of knew it. That text input field was good for conversation, slow for real work. The chat promised infinite possibility but delivered linear constraint. But there have been gradual shifts in how AI interacts with us. - Perplexity Labs started building complete dashboards. Ask it to analyze financial trends, and it generates interactive charts and structured data tables/charts - Claude Artifacts evolved from showing code snippets to becoming a full workspace. Now it's an app builder where you describe what you need, and minutes later you're using a functional tool. - And recently, ChatGPT apps. Ask it to find apartments, and Zillow renders an interactive map inside your chat. Request a playlist, Spotify appears with controls. Instacart Smart Shop doesn't just recommend groceries. It reshapes your entire store interface based on your dietary patterns in real-time. Here's the pattern: Adaptive interfaces over static ones. AI builds the right tool for your specific task. Writing needs an editor workspace. Code needs inline review. Data analysis needs interactive dashboards. Shopping needs personalized aisles. One interface can't serve every intent. For product and design teams, this changes everything. You're not designing for "the user" anymore. You're architecting systems that generate the right interface for this user, this task, this moment. The UI becomes the output, not just the input mechanism. We're moving from conversational AI to generative/adaptive UI. The interface doesn't just listen but shapes itself to match what you're actually trying to do. At Figr, we're exploring what this means for design itself. When your design agent understands not just what to build, but how the interface should adapt to different contexts. That's a real unlock.

We’re no longer just designing digital tools—we’re designing AI-native experiences. That requires experimentation and a deep understanding of how AI communicates, reasons, and behaves. To explore best practices in conversational design, I used three AI platforms—Manus AI, Gemini 2.5 Pro, and Chat GPT o3-Pro—to run a focused research sprint. In under 20 prompts, I didn’t just get answers—I built a structured, interactive site that synthesizes standards, behaviors, and design guidelines into a usable resource. The era of isolated text answers, images, or one-off videos is behind us. We now need to design AI outputs as full experiences, tailored to users’ workflows—experiences that inform, guide, and enable action in real time. This is what next-gen AI makes possible: not just learning through AI, but learning with AI—where the output is already an experience, ready to use. Here’s a look at what that output can become: [ https://lnkd.in/e32AqUDu ] #AIExperience #AI #ConversationalDesign #Standards #BestPractices #ManusAI #o3Pro #GeminiPro

💡Design Process in the AI Era: Stingray Model The traditional Double Diamond framework has long guided innovation teams in problem-solving and solution development. However, in today's fast-paced, AI-driven landscape, this framework should be updated to address the needs of the product design landscape. Geoff Gibbins proposed Stingray model (https://lnkd.in/d9Fzaz5D). Stingray model embraces AI and helps enhance innovation by integrating Generative AI throughout the process. Here's how this model transforms the product design journey: 1️⃣ Train: Designer sets deliberate goals and trains AI to accommodate consumer needs. This enables teams to focus on developing concepts that are desirable, feasible, and viable. 2️⃣ Develop: Simultaneously explore problems and solutions with AI-assisted ideation. This allows exponential and exhaustive analysis of problem spaces. 3️⃣ Iterate: Validate tangible solutions through iterative processes, incorporating synthetic testing and AI-enabled tools to assess concepts against various factors (including sustainability and market trends). This speeds up iteration cycles, which is especially critical in the era of AI. By leveraging AI, the Stingray Model helps product creators accelerate concept validation and reduce biases. Note that despite a significant shift from the conventional approach, this model still puts the human designer front and center—design starts with the human (designer trains model) and ends with the human (designer validates the output generated by AI). 📺 Video tutorials ✔ UI design with ChatGPT https://lnkd.in/dqgXuu-z ✔ Build iOS app in minutes using Cursor AI https://lnkd.in/dTMm6SZt #UI #uidesign #productdesign #uxdesign #ux