Intuitive Interaction Methods

Human conversation is interactive. As others speak you are thinking about what they are saying and identifying the best thread to continue the dialogue. Current LLMs wait for their interlocutor. Getting AI to think during interaction instead of only when prompted can generate more intuitive and engaging Humans + AI interaction and collaboration. Here are some of the key ideas in the paper "Interacting with Thoughtful AI" from a team at UCLA, including some interesting prototypes. 🧠 AI that continuously thinks enhances interaction. Unlike traditional AI, which waits for user input before responding, Thoughtful AI autonomously generates, refines, and shares its thought process during interactions. This enables real-time cognitive alignment, making AI feel more proactive and collaborative rather than just reactive. 🔄 Moving from turn-based to full-duplex AI. Traditional AI follows a rigid turn-taking model: users ask a question, AI responds, then it idles. Thoughtful AI introduces a full-duplex process where AI continuously thinks alongside the user, anticipating needs and evolving its responses dynamically. This shift allows AI to be more adaptive and context-aware. 🚀 AI can initiate actions, not just react. Instead of waiting for prompts, Thoughtful AI has an intrinsic drive to take initiative. It can anticipate user needs, generate ideas independently, and contribute proactively—similar to a human brainstorming partner. This makes AI more useful in tasks requiring ongoing creativity and planning. 🎨 A shared cognitive space between AI and users. Rather than isolated question-answer cycles, Thoughtful AI fosters a collaborative environment where AI and users iteratively build on each other’s ideas. This can manifest as interactive thought previews, real-time updates, or AI-generated annotations in digital workspaces. 💬 Example: Conversational AI with "inner thoughts." A prototype called Inner Thoughts lets AI internally generate and evaluate potential contributions before speaking. Instead of blindly responding, it decides when to engage based on conversational relevance, making AI interactions feel more natural and meaningful. 📝 Example: Interactive AI-generated thoughts. Another project, Interactive Thoughts, allows users to see and refine AI’s reasoning in real-time before a final response is given. This approach reduces miscommunication, enhances trust, and makes AI outputs more useful by aligning them with user intent earlier in the process. 🔮 A shift in human-AI collaboration. If AI continuously thinks and shares thoughts, it may reshape how humans approach problem-solving, creativity, and decision-making. Thoughtful AI could become a cognitive partner, rather than just an information provider, changing the way people work and interact with machines. More from the edge of Humans + AI collaboration and potential coming.

Intelligent interfaces make real-time design choices. For designers, sharing design decisions with robots can be… uncomfortable. But delegation ≠ abdication. The new work for designers is to give context and guidance to help the system make good choices. I made a guide, demo, and video for designers (link in the comments) about how to do this and keep the results on the rails. Done right, the result is a radically adaptive experience that responds to user context and intent. Layouts that rearrange themselves. Forms that choose smart defaults. Chat that “speaks” with well chosen GUI elements instead of text. It’s easier and more reliable than you might expect. The guide includes a simple, directional pattern library for giving the LLM its marching orders. For designers, sketching in simple plain-language system prompts becomes part of the design process, at least as important as drawing interfaces in your design tool. Instead of designing every interaction, you’re designing the *physics* of your application’s tiny universe. You define the behavior and constraints for making design decisions in the interface. It’s design system work for real-time decision-making. The basic recipe for wiring interface to intent: 1. Provide a constrained set of UI outputs. 2. Map those outputs to intent (“use this pattern to address that intent”). 3. Ask the LLM to understand intent and choose the right UI or action. It used to be really, really hard for systems to determine user intent from natural language or other cues. Now LLMs just get it. They grasp underlying semantics, they get slang, they can infer from context. LLMs may hallucinate facts, but they’re brilliant at interpreting intent and the shape of the expected response. This makes them a powerful and reliable partner for interpreting user meaning and delivering an appropriate interface. Check out the demo and give it a try yourself. Start writing; the interface is listening. Link in the comments (because you know, LinkedIn).

Are the interactions in your e-learning course about clicking, not learning? Try this 3-step method to fix it. You spend hours trying to design interactive e-learning—adding clicks, drag-and-drops, and hotspots. But learners rush through, and leadership barely notices. 𝘚𝘰𝘶𝘯𝘥 𝘧𝘢𝘮𝘪𝘭𝘪𝘢𝘳? Many instructional designers feel stuck; they don’t know how to create meaningful interactions instead of interactions that let people click. The key? 𝘚𝘵𝘳𝘢𝘵𝘦𝘨𝘺 𝘰𝘷𝘦𝘳 𝘥𝘦𝘤𝘰𝘳𝘢𝘵𝘪𝘰𝘯. Here’s a simple 3-step method to design interactions that truly enhance your e-learning courses: 1️⃣ 𝗦𝘁𝗮𝗿𝘁 𝘄𝗶𝘁𝗵 𝘁𝗵𝗲 𝗘𝗻𝗱 𝗶𝗻 𝗠𝗶𝗻𝗱 ✅ 𝗗𝗢: Before designing an interaction, ask yourself: *What should learners be able to do after this?* ❌ 𝗗𝗢𝗡’𝗧: Add interactions to make a course "look engaging." 📌 𝗤𝘂𝗶𝗰𝗸 𝗘𝘅𝗮𝗺𝗽𝗹𝗲: If you aim to teach customer service skills, don’t just add a drag-and-drop activity where employees match cybersecurity terms to definitions. Create a simulated phishing attack in which learners must identify suspicious emails, decide whether to open links, and take appropriate action to protect company data. 2️⃣ 𝗙𝗼𝗰𝘂𝘀 𝗼𝗻 𝗗𝗲𝗰𝗶𝘀𝗶𝗼𝗻-𝗠𝗮𝗸𝗶𝗻𝗴, 𝗡𝗼𝘁 𝗝𝘂𝘀𝘁 𝗖𝗹𝗶𝗰𝗸𝗶𝗻𝗴 ✅ 𝗗𝗼: Use interactions that make learners think, not just click. ❌ 𝗗𝗢𝗡’𝗧: Overuse simple interactions (like clicking hotspots) without real engagement. 📌 𝗤𝘂𝗶𝗰𝗸 𝗘𝘅𝗮𝗺𝗽𝗹𝗲: Instead of a basic hotspot where learners click on different parts of a customer service desk to "learn more," create a decision-based hotspot interaction. For example, learners see a busy retail counter with different customer scenarios. Based on urgency and priority, they must click on the right customer to assist first. 3️⃣ 𝗧𝗲𝘀𝘁, 𝗔𝗻𝗮𝗹𝘆𝘇𝗲, 𝗮𝗻𝗱 𝗜𝗺𝗽𝗿𝗼𝘃𝗲 ✅ 𝗗𝗢: Gather feedback and track learner performance. ❌ 𝗗𝗢𝗡’𝗧: Assume that an interaction is effective because it "looks fun." 📌 𝗤𝘂𝗶𝗰𝗸 𝗘𝘅𝗮𝗺𝗽𝗹𝗲: Check if learners are engaged or just rushing through. If they struggle with assessments, go back and refine the interaction—maybe it needs more explicit instructions, better feedback, or a stronger real-world connection. By following these steps, you’ll move beyond generic interactions and create learning experiences that help learners retain knowledge—while making your work stand out. Which of these 3 steps do you already use? Follow me - Mark Spermon - to learn more about creating e-learning courses that engage and deliver results with Articulate Storyline #InstructionalDesign #eLearning #CareerGrowth #L&D #ArticulateStoryline

Remember when talking to your computer felt like something out of a sci-fi movie? Well, welcome to the future, where voice and gesture-based interfaces are not just real—they're everywhere! From asking your phone for the weather to swiping through your favorite playlist with a wave, the way we interact with technology is evolving fast. And guess what? Web and graphic design need to keep up. Creating designs for voice and gesture interfaces is like learning a new dance. It’s all about anticipating your users' next move—literally! Here are a few tips to make your designs as smooth as a voice command: 1️⃣ 𝗞𝗲𝗲𝗽 𝗜𝘁 𝗖𝗼𝗻𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗼𝗻𝗮𝗹: When designing for voice, think about how people naturally speak. Short, clear commands work best. Your interface should feel like a friendly chat, not a pop quiz. 2️⃣ 𝗚𝗲𝘀𝘁𝘂𝗿𝗲𝘀 𝗧𝗵𝗮𝘁 𝗙𝗲𝗲𝗹 𝗡𝗮𝘁𝘂𝗿𝗮𝗹: For gesture-based design, imagine what feels intuitive. A swipe should feel like… well, a swipe! Avoid complicated gestures that require users to learn a secret handshake just to turn the page. 3️⃣ 𝗙𝗲𝗲𝗱𝗯𝗮𝗰𝗸 𝗶𝘀 𝗬𝗼𝘂𝗿 𝗙𝗿𝗶𝗲𝗻𝗱: Users need to know that their voice command or gesture was recognized. Whether it’s a subtle sound or a quick visual cue, feedback reassures users that they’re in control. 4️⃣ 𝗔𝗰𝗰𝗲𝘀𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝘆 𝗠𝗮𝘁𝘁𝗲𝗿𝘀: Not everyone interacts with technology in the same way. Consider users with different abilities, and ensure your designs are inclusive. For voice interfaces, think about accent recognition. For gestures, think about ease of use for all ages and abilities. 5️⃣ 𝗧𝗲𝘀𝘁, 𝗧𝗲𝘀𝘁, 𝗧𝗲𝘀𝘁: Finally, don’t assume you’ve nailed it until you’ve tested it with real people. What makes sense to you might feel like a bad game of charades to someone else. As voice and gesture-based interfaces continue to rise, the way we design for them will define how intuitive and accessible they become. #VoiceUI #GestureDesign #UXDesign #WebDesign #Accessibility #Innovation #DigitalTransformation

Testing the Commerce Kit on the Spectacles using adjusted 3D models from a previous AR campaign we built for Audemars Piguet. We’ve been experimenting with gestures to refine how users interact with 3D products in immersive spaces. A few takeaways: - Using open/close gestures to spawn products feels seamless. - Moving your hand to spin the product is intuitive and feels natural. - The "swipe to confirm" purchase trigger is surprisingly satisfying.

The future of interaction is here and it’s controlled with a wave📲 The latest from Huawei introduces AI-powered air-gesture control that lets you interact with your device and even objects without touching the screen. Think beyond taps and swipes: simple hand movements now trigger actions, from sharing files to controlling functions intuitively in mid-air. ✨ Why this matters: • 🖐️ Touch-less interaction: Wave to take actions no contact needed between your hand and the phone. • 🤖 AI + sensors: Advanced cameras and AI vision track your gesture intent and translate it into commands. • 🔄 Control real functions: From file transfer to device control, this moves us closer to post-touch computing. Features like this signal a broader shift: human motion becomes part of the user interface blending natural behavior with digital control. It’s not just a gimmick; it’s a preview of how AI and gesture tech will reshape everyday interactions. 💡 Discussion Prompt: How will gesture-based control redefine productivity and user experience in phones, wearables, or even physical devices we live with?