AI Chatbot Usage Insights

The most dangerous thing about hallucinations in AI isn't that they're wrong. It's that they don't look wrong. You ask for a source, it gives you a figment. You ask for facts, it makes them up. It doesn’t just lie - it lies eloquently, with citations, formatting, and a tone that screams “trust me.” Just enough jargon to fool the average reader- and sometimes, the expert. In consumer settings, a hallucination is annoying. In a courtroom, hospital, or trading desk, it's catastrophic. That’s why hallucinations are the biggest blocker to AI adoption: they turn an otherwise brilliant assistant into that unreliable coworker whose numbers you always have to double-check. At best, they waste time. At worst, they create liability. Researchers have thrown the kitchen sink at hallucinations: ▪️ Retrieval-Augmented Generation (RAG) - Give the model a search engine sidekick. Instead of free-styling from memory, it fetches real documents, so it answers with receipts. ▪️Self-Critique Loops - Tools like SelfCheckGPT or Chain of Verification reread outputs like a paranoid editor. ▪️Fine-Tuning with Human Feedback - Pavlov method: humans reward outputs that look good. ▪️Conservative Decoding - Language models have a 'creativity dial'. High temperature makes them improvise like jazz musicians; low temperature makes them stick to the teleprompter. These techniques work, but trade-offs loom: accuracy costs latency and compute; grounding kills creativity. Which is why many teams now run two modes - “idea jam” (high temp, hallucinations tolerated) and “serious business” (low temp + retrieval + guardrails). Last week, OpenAI released a new paper titled “Why language models hallucinate”. Their core point: hallucinations aren’t just an artifact of messy training data or exotic transformer math - they’re the rational outcome of a badly designed reward system. Current benchmarks reward certainty and correctness but don’t penalize confident errors or give credit for saying “I don’t know.” This can implicitly push models to guess. RLHF today trains models to be helpful, harmless, polite. Human raters tend to upvote answers that are fluent and well-structured even if they're factually shaky. This optimizes for charm, not epistemic hygiene. OpenAI argues for a new system: reward calibrated uncertainty and punish confident wrongs. In other words, give points for “I don’t know” and dock points for swaggering mistakes. So while both approaches use reinforcement, the values baked in are different. - RLHF gave us ambitious interns - always have an answer, always sound polished. - OpenAI is pushing for seasoned experts - confident when right, silent when not. It’s corporate culture 101. Promote people for speaking up regardless of accuracy, and you’ll soon have a room full of confident nonsense.

*Let's talk about how we use AI tools in our work and personal life without increasing the risk for accidental data leakage, breaches, or extortion* First and foremost, feeding national intelligence documents (or any sensitive docs) into an AI tool to determine which parts should remain classified is not the move (see photo below). Why? Many AI-based systems lack strong contextual decision-making, which can lead to accidental disclosure of private or classified materials from the AI tool. *When it comes to work related AI-usage we have to consider the following*: - Does your org have a policy against or for AI tool usage? What about local AI tool instances that are recommended for your team? - Does your organization use prompt protection tools to avoid accidental data leakage from your user's questions (aka prompts)? - Does the doc have secrets, proprietary data, employee data, customer data, passwords, etc embedded within it? Be careful and avoid entering this data to avoid data leakage -- redact first. - Do you secure your LLM tools with long random unique passwords + MFA? If you reuse passwords (and that password shows up in a data breach) it could lead to a hack and subsequent leak of your sensitive AI queries, work, details of an M&A that hasn't been announced etc leading to an even larger breach or data leak. *What does this mean for everyday folks who use AI for their everyday life?* You still can use AI! Just redact sensitive info before entering it into AI tools. For example: if you want to use AI tools to understand, say, a report from your doctor, I recommend removing personal details like your name, address, birthdate, etc before feeding it into AI tools to avoid accidental data leakage from the AI tool. *I predict within the next 6 months bad actors will start heavily targeting credentials for AI chat bots used by organizations to leak and extort prompt history and other sensitive questions and data* - They may attempt to leverage a reused password against your organization to gain access to your team AI tools and leak/extort your history that has sensitive data within it - They may attempt to phish individual's passwords/codes to leak or extort high net worth individuals AI chat bot history outside of the office - Bad actors may increase their targeting of AI chat bot infrastructure to encourage the tool to inadvertently leak sensitive details or proprietary info that users have entered into the tool *Actions to protect yourself and your team from AI tool risk* 1. Redact sensitive or proprietary info before entering into AI tools 2. Secure your AI tools with long random unique passwords + MFA to avoid extortion for hacked embarrassing or sensitive AI query history 3. Notice and report phishing against those AI credentials -- attackers use urgency and fear (such as "your account has been compromised, click here to secure") to get you to click and enter a password/code for that AI tool

This Stanford study examined how six major AI companies (Anthropic, OpenAI, Google, Meta, Microsoft, and Amazon) handle user data from chatbot conversations.  Here are the main privacy concerns. 👀 All six companies use chat data for training by default, though some allow opt-out 👀 Data retention is often indefinite, with personal information stored long-term 👀 Cross-platform data merging occurs at multi-product companies (Google, Meta, Microsoft, Amazon) 👀 Children's data is handled inconsistently, with most companies not adequately protecting minors 👀 Limited transparency in privacy policies, which are complex and hard to understand and often lack crucial details about actual practices Practical Takeaways for Acceptable Use Policy and Training for nonprofits in using generative AI: ✅ Assume anything you share will be used for training - sensitive information, uploaded files, health details, biometric data, etc. ✅ Opt out when possible - proactively disable data collection for training (Meta is the one where you cannot) ✅ Information cascades through ecosystems - your inputs can lead to inferences that affect ads, recommendations, and potentially insurance or other third parties ✅ Special concern for children's data - age verification and consent protections are inconsistent Some questions to consider in acceptable use policies and to incorporate in any training. ❓ What types of sensitive information might your nonprofit staff  share with generative AI?  ❓ Does your nonprofit currently specifically identify what is considered “sensitive information” (beyond PID) and should not be shared with GenerativeAI ? Is this incorporated into training? ❓ Are you working with children, people with health conditions, or others whose data could be particularly harmful if leaked or misused? ❓ What would be the consequences if sensitive information or strategic organizational data ended up being used to train AI models? How might this affect trust, compliance, or your mission? How is this communicated in training and policy? Across the board, the Stanford research points that developers’ privacy policies lack essential information about their practices. They recommend policymakers and developers address data privacy challenges posed by LLM-powered chatbots through comprehensive federal privacy regulation, affirmative opt-in for model training, and filtering personal information from chat inputs by default. “We need to promote innovation in privacy-preserving AI, so that user privacy isn’t an afterthought." How are you advocating for privacy-preserving AI? How are you educating your staff to navigate this challenge? https://lnkd.in/g3RmbEwD

"We examine the labor market effects of AI chatbots using two large-scale adoption surveys (late 2023 and 2024) covering 11 exposed occupations (25,000 workers, 7,000 workplaces), linked to matched employer-employee data in Denmark. AI chatbots are now widespread—most employers encourage their use, many deploy in-house models, and training initiatives are common. These firm-led investments boost adoption, narrow demographic gaps in take-up, enhance workplace utility, and create new job tasks. Yet, despite substantial investments, economic impacts remain minimal. Using difference-in-differences and employer policies as quasi-experimental variation, we estimate precise zeros: AI chatbots have had no significant impact on earnings or recorded hours in any occupation, with confidence intervals ruling out effects larger than 1%. Modest productivity gains (average time savings of 3%), combined with weak wage pass-through, help explain these limited labor market effects. Our findings challenge narratives of imminent labor market transformation due to Generative AI. " Anders Humlum & Emilie Vestergaard

Prompting is not about typing better sentences. It’s about transferring intent clearly. When AI outputs feel off, incomplete, or confusing, the issue is rarely intelligence. It’s almost always a gap in instruction - missing context, unclear goals, or poorly defined boundaries. This guide lays out 20 practical rules of prompt engineering that address exactly those gaps. It shows how small changes in how you ask can completely change what you get back. The framework covers how to: - Clearly define what you want and why you’re asking - Assign the right role so the model responds from the correct perspective - Provide context that removes assumptions and guesswork - Control structure, tone, and level of detail in advance - Break complex requests into smaller, sequential steps - Use examples to anchor expectations instead of hoping the model guesses - Apply constraints to reduce fluff, repetition, and irrelevant output - Iterate deliberately instead of rewriting prompts from scratch - Validate responses and catch logical gaps early These rules don’t make prompts longer. They make them more intentional. Once you apply this approach, AI stops feeling unpredictable. Responses become more consistent, more usable, and closer to what you actually had in mind. Prompting then shifts from trial-and-error to a repeatable workflow - one you can rely on for writing, analysis, coding, planning, and decision support. If AI is part of how you think and work, this kind of structure quietly improves everything that comes after. Would love to know which of these rules you already use and which ones surprised you.

A few months ago, a colleague screamed at Microsoft Copilot like he was auditioning for Bring Me The Horizon. He typed, “Make this into a presentation.” Copilot spat out something. He yelled, “NO, I SAID PROFESSIONAL!” It revised it. Still wrong. “WHY ARE YOU SO STUPID?” And that, dear reader, is when it hit me. It’s not the AI. It’s you. Or rather, your prompts. So, if you've ever felt like ChatGPT, Copilot, Gemini, or any of those AI Agents are more "artificial" than "intelligent"? Then rethink how you’re talking to them. Here are 10 prompt engineering fundamentals that’ll stop you from sounding like you're yelling into the void. 1. Lead with Intent. Start with a clear command: “You are an expert…,” “Generate a monthly report…,” “Translate this to French…" This orients the model instantly. 2. Scope & Constraints First. Define boundaries up front. Length limits, style guides, data sources, even forbidden terms. 3. Format Your Output. Specify JSON schema, markdown headers, or table columns. Models love explicit structure over free form prose. 4. Provide Minimal, High Quality Examples. Two or three exemplar Q→A pairs beat a paragraph of explanation every time. 5. Isolate Subtasks. Break complex workflows into discrete prompts (chain of thought). One prompt per action: analyze, summarize, critique, then assemble. 6. Anchor with Delimiters. Use triple backticks or XML tags to fence inputs. Cuts hallucinations in half. 7. Inject Domain Signals. Name specific frameworks (“Use SWOT analysis,” “Apply the Eisenhower Matrix,” “Leverage Porter’s Five Forces”) to nudge depth. 8. Iterate Rapidly. Version your prompts like code. A/B test variations, track which phrasing yields the cleanest output. 9. Tune the “Why.” Always ask for reasoning steps. Always. 10. Template & Automate. Build parameterized prompt templates in your repo. Still with me? Good. Bonus tips. 1. Token Economy Awareness. Place critical context in the first 200 tokens. Anything beyond 1,500 risks context drift. 2. Temperature vs. Prompt Depth. Higher temperature amplifies creativity. Only if your prompt is concise. Otherwise you get noise. 3. Use “Chain of Questions.” Instead of one long prompt, fire sequential, linked questions. You’ll maintain context and sharpen focus. 4. Mirror the LLM’s Own Language. Scan model outputs for phrasing patterns and reflect those idioms back in your prompts. 5. Treat Prompts as Living Docs. Embed metrics in comments: note output quality, error rates, hallucination frequency. Keep iterating until ROI justifies the effort. And finally, the bit no one wants to hear. You get better at using AI by using AI. Practice like you’re training a dragon. Eventually, it listens. And when it does, it’s magic. You now know more about prompt engineering than 98% of LinkedIn. Which means you should probably repost this. Just saying. ♻️

Unlock the potential of Generative AI to enhance your writing, creativity, and coding skills through prompt engineering. Prompt engineering is a key skill that involves crafting detailed, structured inputs to guide AI towards generating precise, useful outputs. Here are the core strategies to master: - Guide Precisely: Provide detailed instructions for clear, targeted outcomes. - Rich Context: Supply comprehensive background information for more accurate and relevant responses. - Experiment: Start with the basics, then explore more complex requests as you become more comfortable. Improve your AI interactions with these tips: 1. Specificity and Iterations: Craft detailed prompts and refine based on the AI's feedback. 2. Contextual Depth: The more context you provide, the better the AI understands your request, leading to more tailored outputs. 3. Multi-Modal Inputs: Beyond text, incorporate images, code, or data for varied and rich outputs. 4. Example Use: Include examples of what you're aiming for and what you want to avoid to guide the AI more effectively. 5. Advanced Features: Tweak settings like creativity level and response length to get the results you need. 6. Unique Capabilities: Utilize the AI's broad knowledge and support for specific tasks, such as coding assistance. ✍️ Suppose you want to learn a new skill. Here's a prompt template incorporating the above principles: 'I'm eager to learn [Skill Name], aiming to use it for [specific purpose or project]. My background is in [Your Background], and my experience with similar skills is [Your Experience Level]. I aim to build a foundational understanding and complete my first project within [Timeframe]. Could you provide a structured learning path that includes: The key concepts and fundamentals of [Skill Name] I should focus on. Recommendations for online courses, tutorials, and books suitable for beginners. Practical exercises or projects for applying what I learn. Tips for staying motivated and overcoming challenges. Strategies for applying [Skill Name] in real-world situations or job opportunities.' This approach ensures a personalized, goal-oriented learning strategy, leveraging AI's capabilities to support your journey in mastering a new skill. #generativeai #ai #promptengineering #upskill #learning

You paid your lawyer $500 for a one-hour legal strategy session. Then you pasted it into ChatGPT to "understand it better." CONGRATS: opposing counsel can now subpoena that chat. This isn't hypothetical. In United States v. Heppner, a CEO pasted his lawyer's defense strategy into Claude AI. The FBI seized his devices and found all the chats. All 31 of them. And when he tried to claim privilege — the court shut it down. Reason: Attorney-client privilege only works when the conversation stays confidential. When you share it with a third party, that protection is gone. And AI platforms are third parties. ChatGPT, Claude, Gemini. These are companies with servers, data policies, terms of service. None of them owe you confidentiality. That's not a private conversation anymore. That's a record. And the other side can ask for it. I get it. AI feels private. Like a notes app. Like thinking out loud. But legally, it's not. And don't get me wrong. I'm not anti-AI. I run a law firm. We use it too. But instead of public AI, we use enterprise tools with safeguards that don't train on client data. If it's legal, strategic, or sensitive: DO NOT paste it into a chatbot. And if you're still in doubt, ask yourself: Would you hand this to opposing counsel? If the answer is no, don't hand it to ChatGPT either.

AI Agents Are the New Attack Surface! Are We Ready for That? AI agents powered by large language models (LLMs) introduce entirely new vulnerabilities across confidentiality, integrity, and availability. Here’s what’s new and why it matters: AI Agents execute actions: Unlike typical LLMs, agents interact with tools, systems, and APIs, meaning a hallucinated or adversarial output can change files, leak data, or flood networks. Session management is a blind spot: Most agents don’t isolate user sessions robustly. Result: chat histories bleed across users, leading to data leaks and misassigned actions. Model pollution is real: Malicious inputs can subtly "poison" fine-tuned models, degrading performance and trust without being obviously adversarial. Sandboxing isn’t optional: Experiments showed that 90 out of 95 malicious prompts were accepted by a state-of-the-art agent, with 80% successfully executed, unless sandboxed. Promising defense directions: Session-aware memory and formal monads for state tracking, Encryption-preserving inference (like FPETS and FHE) to process sensitive data safely or toolchain access controls that isolate file systems and limit network requests. 📣 Bottom line: The same autonomy that makes AI agents exciting also makes them dangerous. Without secure-by-design architectures, they could become powerful attack vectors. What security practices are you considering for deploying AI agents in your org?

AI hallucinations aren’t about bad tech. They’re about bad incentives. Kids prove it every day. Want to see how? Step into a classroom full of kids 👧👦 Ask why the sun sets → “because it’s sleepy.” Ask why we have two eyes → “so one can sleep while the other watches TV.” Ask why objects fall → “because they are tired of standing.” All said with total conviction. All completely wrong ❌. PhD confidence. Kindergarten accuracy 🤷♀️. That is hallucination in AI. Confident. Convincing. But wrong. ***** How does AI get trained in the first place? 🤔 It reads huge amounts of text like books, articles, and websites and learns to guess the next word. If I say ‘salt and …’ it guesses the next word is ‘pepper.’ Every time it guesses right, it gets a reward 🎁. Every time it guesses wrong… nothing. No punishment. No red mark. Not even “see me after class". And here’s the catch. “I don’t know” never gets a reward, and wrong answers never get penalized either. So what does AI learn? Be that kid in class who always shouts an answer… usually wrong, but never punished 😅. ***** So what happens when AI is trained this way? It keeps answering, even when the answer does not exist. Ask it to narrate the story of a movie 🎬 and it flows smoothly… until it throws in a scene that even the director would be shocked to see. Ask it to summarise a book 📖 and it sounds convincing… until a random character wanders in from another novel. 𝑻𝒉𝒂𝒕 𝒊𝒔 𝒉𝒂𝒍𝒍𝒖𝒄𝒊𝒏𝒂𝒕𝒊𝒐𝒏. 𝑪𝒐𝒏𝒇𝒊𝒅𝒆𝒏𝒕. 𝑫𝒆𝒕𝒂𝒊𝒍𝒆𝒅. 𝑪𝒐𝒎𝒑𝒍𝒆𝒕𝒆𝒍𝒚 𝒘𝒓𝒐𝒏𝒈. ***** Sounds familiar, right? Kids do the same thing. Silence gets them nothing. But a confident answer, even a wrong one, at least gets them a smile, attention, or sometimes even praise. Or at minimum, a shiny “nice try” sticker 😅. AI is no different. It was never taught that “I don’t know” can be the right answer. So just like kids who blurt out something to avoid silence, AI learns to always respond. ***** So, is there a way to stop this? 🤔 Researchers at OpenAI say yes. The problem is how we test AI. Right guesses get full marks ✅. ‘I don’t know’ gets zero. Wrong answers aren’t punished. So it learns to always answer. The fix is simple. Change the tests so that “I don’t know” also counts, and wrong confident answers lose points. In other words, stop grading AI like the kid who aces a multiple-choice exam by pure luck. ***** People do this too. In interviews and meetings, confidence often gets rewarded more than honesty. But the smarter answer is “I don’t know, but I will find out.” That is who you can actually trust. 🌟 ***** Whether it is a child in class, an AI model, or a candidate in an interview, one rule holds true. 𝗪𝗵𝗮𝘁 𝘄𝗲 𝗿𝗲𝘄𝗮𝗿𝗱 𝗶𝘀 𝘄𝗵𝗮𝘁 𝘄𝗲 𝗴𝗲𝘁 Let’s start rewarding the courage to say “I don’t know, but I will find out.” That is how we build trust in people, in organizations, and in technology. And that’s how we stop hallucinations in AI, and in ourselves 🎤