Podcast Data Collection Methods

I recently heard someone say that working with MCP makes them feel like they switched from drinking coffee to straight rocket fuel. With that as inspiration, I bring you my most recent MCP experiment... I wanted to analyze the Vibecasts podcast that Sam McKay and I do regularly. However, the only data I had were the YouTube videos themselves, which didn't even include transcripts. I had flashbacks to my first data job as a 21-year-old intern doing phone interviews for Gallup and hand-coding qualitative interview transcripts.  This was painstakingly difficult and boring work. And if you are concerned only about AI's hallucinations, let me humbly suggest you have never worked the night shift hand-coding data with a bunch of college students. So, here are the steps I took, all within the Claude MCP client and all guided by entirely by natural language voice commands. I didn't do a single word of typing nor a single line of coding. 1. Had Claude 4 write a Python script to extract the transcripts from all ten of our podcasts using a specialized library made for this task called YT-DLP. 2. Created a new Supabase (MCP-enabled Postgres database back end) project with a schema built to hold a retrieval-augmented generation (RAG) database - which allows you to train an AI model on local info outside of its original training dataset. 3. I directed Claude to write Python scripts to perform chunking and embedding of the transcript data. Chunking is breaking the transcripts into much smaller pieces since it's more efficient for the AI to process small pieces than entire documents at once. Embedding then takes those chunks and converts them into numbers that represent their meaning. So, basically chunks about similar topics get similar number patterns. The system can then find chunks with the most similar "fingerprints" to your question. 4. I then directed Claude to write a script to load all of this into the Supabase schema. All in all, Claude wrote over 700 lines of Python code with one error (which ended up being caused by my giving it an incorrect path - so much for the "human in the loop"...) This is the point where the video below picks up. Each of the analyses it shows are unedited except to speed the video up in time with the music. Oh, and I did leave MCP once to create the virtual Sam McKay, CFA voice at the end... #ai #mcp #voice #claude Abu Bakar N. Alvi Darragh Murray Olivier Travers

Here's something valuable I discovered recently while researching user needs: Podcasts can be an incredible source of user insights. While traditional user research methods—interviews, observations, and usability testing—remain essential, they can be slow and challenging when it comes to reaching precisely the right people. Podcasts turned out to be a surprisingly efficient complement for me, especially when trying to understand the real-world challenges clinicians face. Here's how I used podcasts to enhance my research: 1. Identify Relevant Podcasts: I found episodes where doctors with their own practices (members of my ICP—Ideal Customer Profile) shared candidly about their daily challenges, frustrations, and solutions. This provided direct insights into their authentic experiences. 2. Listen Actively (During Commuting or Workouts): Listening while commuting or exercising (people who know me know how much I hate running, but I have to do it for my Hyrox challenges) made research enjoyable and highly productive (and running less of a pain). 3. Easy Transcription: Using YouTube transcripts or free services like converter.app to turn MP3s into text, I quickly created transcripts of key episodes. 4. AI-Assisted Analysis: I pasted the transcripts into a tool like GPT-o3 with this prompt: "From the transcript provided, extract clearly, (a) the key problems mentioned, (b) Jobs-to-be-done discussed, (c) Current workarounds clinicians are using, and (d) Challenges clinicians encounter with these workarounds." 5. Identify "Earlyvangelists": On top of the insights, this approach helped me pinpoint clinicians who are already deeply engaged in solving the problems I care about—perfect candidates for deeper interviews, beta testing, or co-development. I’ve found 'podcast mining' to be an insightful cost-effective and super fast addition to traditional UX methods. It allows deeper understanding and better-prepared interactions, complementing rather than replacing more targeted research methods. Curious—have you ever used podcasts for your UX research? I'd love to hear about your experiences!

When I ask most creators how they're using AI, they usually tell me they use ChatGPT or another chatbot to help with idea generation or writing scripts for videos. But not much else. Chris Williamson, the host of the popular podcast Modern Wisdom, is taking a different approach. He’s hired a software engineer to develop what’s called a RAG, or retrieval augmented generation, system. This involves gathering data on his YouTube transcripts, blog posts and other content. Then his staff plans to build what’s known as a vector database to store the information for searching. Finally, he’ll give a large language model, such as OpenAI’s GPT-4o, access to the information for answers. It’s a lot of work, but Williamson says it will be worth it. “This is like having your own ultimate research assistant for your own body of work,” he told me at the tech and arts festival South by Southwest earlier this month. Williamson said he’s pursued this approach because ready-made alternatives like asking OpenAI chatbot ChatGPT for a rundown of a podcast episode have sometimes come up with inaccurate results. Plus, he wanted something more personalized. Read our full interview here: https://lnkd.in/eqDfdJnx

There's way too many podcasts to keep up with these days... so what can you do? 🤯 LLMs and AI can help us prioritize our information acquisition like never before. Here's how... We can use Python, an LLM like OpenAI's GPT models, and application middleware/microservices to create a Podcast Information Summarizer 🎙️💻 Had such a blast building this project over the past week, and it's programmed to run on podcasts from Alex Hormozi, Gary Vaynerchuk, and TWIML, but you can pick and choose from any podcast you want. Here's how it works: 💾 I used OpenAI's GPT-trained LLM for data fetching about the podcaster. If it doesn't know much about the podcaster, I could augment the LLM with RAG using data fetched from wikipedia. If there wasn't any Wikipedia data available, I added a script to scrape the podcaster's website's "about" section. 🎙️ Using Python, I configured a way to convert the .mp4 audio file (the podcast), into a transcript as text, so that I could then tokenize this information when I made a prompt up to GPT. Turning audio into text is surprisingly easy with Python, and a great way to extend what LLMs can normally use as data inputs, since they can't "listen" to podcasts like we can. 💬 By using the LLM directly or using LLM + RAG, I was able to generate prompts that would include information about the podcaster, certain parts of the transcript, and information about each episode based on listening to the RSS feed. 🤖 What is generated is a .csv file for the podcaster's latest episode by feeding in a RSS Feed of the podcast. It generates, almost as if by magic, a comprehensive overview of each episode, including: (1.) the title of the episode, (2.) an LLM-generated summary of the episode with info about the podcaster, (3.) a LLM-generated list of the podcast guests and information about them, and (4.) an LLM-generated list of "Key Moments" from the podcast. The Tech Stack I used: - Front-End: Python - Back-End: Python + Project Jupyter - AI Model: OpenAI's LLM + Wikipedia, the Free Encyclopedia - Cloud Functions: Modal Labs - CI/CD: GitHub - Hosting: Streamlit Thank you Ted Sanders, Sidharth Ramachandran, Shreya Vora, Mike Ion, and Uplimit (Formerly CoRise) for such a great course and program to dive into building products with Python, AI, and LLMs! 📜 View my new Uplimit credential via Accredible for "Building AI Products with OpenAI": https://scq.io/VGsarPN #TechStackPlaybook #ai #machinelearning #llm #fullstackengineer #softwareengineering #softwaredevelopment #openai #gpt #tech #ml