Why Redis is the #1 choice for AI agents in 2025

Big news from the Redis crew! We’re welcoming the Featureform team to the family. This is a huge step in bringing real-time structure and orchestration to AI and ML workloads. 🔧 Featureform + Redis = next-level AI orchestration: Featureform is all about defining, managing, and reusing features across training and production. Now, with Redis, you can pair structured data and vector embeddings in one powerful, lightning-fast platform. 🚀 Why this matters • No more patchwork pipelines or extra glue code • Define your features once and use them everywhere • Keep your models accurate and accountable with better observability and governance • Blend structured data with embeddings to build smarter, faster AI systems 📦 What this means for builders If you’re creating AI agents, ML systems, or real-time applications, this combo hits all the right notes. You get simplicity, speed, and scale all in one place. Let’s elevate what’s possible together. https://lnkd.in/ewJJzpna

Exciting update for Azure Cosmos DB users! Microsoft introduces Binary Encoding, delivering up to 70% storage savings and faster query performance—especially for large datasets. Automatically enabled for new containers and rolling out to existing ones in 2025, no action needed! Bonus: SDK updates will unlock even more speed. A big step toward cost-efficient, high-performance data solutions. #AzureCosmosDB #CloudComputing #AI #NoSQL #MicrosoftAzure

Redis acquired Featureform to make it even easier for developers to deliver real-time, structured data into AI agents and applications. This brings together Redis’ speed and scalability with Featureform’s feature store technology. A huge step forward for anyone building production-grade AI systems that rely on fast, reliable data. Read more here: https://lnkd.in/eYzQc5KC

Where Should You Store Embeddings? 🧠 Vector embeddings are not just data — they're search indexes. Storing them right can be the difference between sub-second relevance and costly compute overhead. Here’s my go-to storage pattern for GenAI systems on AWS: 🔹 DynamoDB → For metadata (IDs, timestamps, labels, S3 URIs) Ultra-low latency key-value access. Ideal for pairing vector search results with display data or controlling access logic. 🔹 OpenSearch → For semantic vector search Supports ANN using HNSW graphs — a highly efficient algorithm for fast, high-dimensional similarity search. You can also use KNN mode for brute-force accuracy when precision > speed. Perfect for: Retrieval-Augmented Generation (RAG) Personalization systems Anomaly or similarity detection 🔹 S3 → For raw unstructured data (PDFs, docs, images, audio) Immutable and durable, ideal for storing original content used to generate embeddings or for downstream tasks like reprocessing or retraining. 🧠 Combine all three for a modular, scalable, and efficient vector architecture: 🔍 Search embeddings in OpenSearch 📄 Fetch metadata from DynamoDB 📂 Load source data from S3 (if needed) Takeaway: 👉 Embeddings ≠ metadata ≠ raw data Match each to the right storage engine — and your GenAI pipeline will scale with performance, clarity, and cost-efficiency. #AmazonOpenSearch #DynamoDB #AmazonS3 #GenerativeAI #VectorSearch #AWS #Embeddings #SemanticSearch

📣 New White Paper Alert 📣 🔓Unlock 1000x performance boost querying parquet files with this latest whitepaper, coauthored by engineers at Salesforce and Alluxio 🚀Discover how Alluxio's distributed caching can supercharge your queries on data lake with low latency and just a fraction of the cost compared to AWS S3 Express One Zone. Read Now: https://lnkd.in/gMfbzMS4 #AI #MachineLearning #DeepLearning #S3 #S3Express #RAG #LLM #Query #DataLake #Storage #Compute

Vector search is now a first-class feature in Azure Cosmos DB for NoSQL—you can store, index, and query embeddings alongside your data. This integrated approach removes the need for a separate vector database, simplifies architecture, and reduces data duplication. Azure Cosmos DB supports multiple index types—including DiskANN, flat, and quantized flat—to optimize accuracy, latency, and cost. Use vector search for recommendations, semantic search, anomaly detection, agent memory, and more. Learn how to enable it, configure vector policies, and run similarity queries: https://msft.it/6049sLY49 #AzureCosmosDB #VectorSearch #AI #SemanticSearch

AI is everywhere but structured data still rules. Every wave of tech promises to “replace” relational databases. Yet here we are, in the AI era, and PostgreSQL still quietly powers the world’s most critical systems. From data integrity to AI-driven workloads, PostgreSQL is now the open-source default across every major cloud bridging structured data, vector search, and intelligent insights. The truth? AI doesn’t replace relational databases it makes them essential. Without structure, data is just noise. PostgreSQL gives AI meaning. Follow Haider Zah for weekly insights on PostgreSQL reliability, performance, and AI-era database engineering. #PostgreSQL #AI #DataEngineering #RelationalDatabases #StructuredData #VectorSearch #OpenSource #DatabasePerformance #AIInsights #CloudData #TechLeadership #DataReliability #DataEngineer #DatabaseEngineer

Installing the brain (with backups) for AETHLEN — pre-boot status Today I set up the local AI engine that will power AETHLEN (A-E-T-H-L-E-N). Not “an AI,” but a stack with fallbacks so it’s reliable from day one. What I installed (local, on my machine): Ollama runtime → hosts on-device LLMs over HTTP. Model trio (with failover): Qwen-2.5 7B (primary) – general reasoning & instruction following. Llama-3.1 8B (backup, q4_K_M) – a second strong voice, quantized for speed/VRAM. Qwen-2.5 3B (fast fallback) – light, quick responses when we need latency wins. Postgres 16 + pgvector → structured data + vector columns for search/RAG storage. Why local (not cloud): Privacy – data stays with us. Cost control – no per-token API bills. Latency – tight loop, faster experimentation. Resilience – if one model chokes, fallbacks pick up the slack. How this fits together (at a glance): Apps (E-commerce • Doc Copilot • Reviews PWA • Membership • Vault • Chatbot) → AI Core (Rust/Axum) → Ollama (LLMs) + Postgres/pgvector (storage & retrieval) What this means in practice: We can run chat, rewrite, summarize, classify locally. We’re set up for retrieval-augmented answers (vector search in Postgres) once embeddings/reranking are in place. The same AI core serves multiple apps, each with its own endpoint/contract, but one shared backbone. This isn’t “launch” copy—the system isn’t awake yet. It’s the foundation: models installed, runtime wired, database ready, and the architecture designed for reliability (primary + backup + fallback). #AI #LLM #EdgeAI #OpenSource #Rust #Axum #Ollama #Postgres #pgvector #AETHLEN

🚀 𝗙𝗮𝗿𝗴𝗮𝘁𝗲 𝘃𝘀. 𝗟𝗮𝗺𝗯𝗱𝗮: 𝗖𝗵𝗼𝗼𝘀𝗶𝗻𝗴 𝘁𝗵𝗲 𝗥𝗶𝗴𝗵𝘁 𝗧𝗼𝗼𝗹 𝗳𝗼𝗿 𝗥𝗲𝗮𝗹-𝗧𝗶𝗺𝗲 𝗦𝗰𝗮𝗹𝗲 Following up on the Quiz Showdown architecture challenge, the choice of compute - 𝗔𝗪𝗦 𝗙𝗮𝗿𝗴𝗮𝘁𝗲 vs 𝗔𝗪𝗦 𝗟𝗮𝗺𝗯𝗱𝗮 was critical. We didn't just pick one; we chose the right one for each service's job. It all comes down to 𝗦𝘁𝗮𝘁𝗲𝗳𝘂𝗹𝗻𝗲𝘀𝘀 𝘃𝘀. 𝗘𝘃𝗲𝗻𝘁𝗳𝘂𝗹𝗻𝗲𝘀𝘀. 𝟭. 𝗔𝗪𝗦 𝗙𝗮𝗿𝗴𝗮𝘁𝗲: 𝗧𝗵𝗲 𝗟𝗼𝗻𝗴-𝗗𝗶𝘀𝘁𝗮𝗻𝗰𝗲 𝗥𝘂𝗻𝗻𝗲𝗿𝘀 🏃♂️ We chose 𝗙𝗮𝗿𝗴𝗮𝘁𝗲 (our serverless container engine) for services that need to run continuously and 𝗺𝗮𝗶𝗻𝘁𝗮𝗶𝗻 𝘀𝘁𝗮𝘁𝗲 in memory for a period of time.  • 𝗤𝘂𝗶𝘇 𝗦𝗲𝗿𝘃𝗶𝗰𝗲: This service is ���𝘁𝗮𝘁𝗲𝗳𝘂𝗹. It must hold the live 𝗶𝗻-𝗺𝗲𝗺𝗼𝗿𝘆 𝗴𝗮𝗺𝗲 𝘀𝘁𝗮𝘁𝗲 (scores, connection IDs) for the entire 60-second quiz. Fargate provides the necessary 𝗣𝗲𝗿𝘀𝗶𝘀𝘁𝗲𝗻𝗰𝗲, guaranteeing the runtime needed to manage the entire session. If we used Lambda here, the game would crash between requests!  • 𝗠𝗮𝘁𝗰𝗵𝗺𝗮𝗸𝗶𝗻𝗴 𝗦𝗲𝗿𝘃𝗶𝗰𝗲: This requires a dedicated, 𝗮𝗹𝘄𝗮𝘆𝘀-𝗼𝗻 𝘄𝗼𝗿𝗸𝗲𝗿 𝗽𝗿𝗼𝗰𝗲𝘀𝘀 that continuously monitors the Redis queues. Fargate provides the 𝗖𝗼𝗻𝘁𝗶𝗻𝘂𝗼𝘂𝘀 𝗢𝗽𝗲𝗿𝗮𝘁𝗶𝗼𝗻 needed to run this background task efficiently. 𝟮. 𝗔𝗪𝗦 𝗟𝗮𝗺𝗯𝗱𝗮: 𝗧𝗵𝗲 𝗦𝗽𝗿𝗶𝗻𝘁 𝗦𝗽𝗲𝗰𝗶𝗮𝗹𝗶𝘀𝘁𝘀 💨 We chose 𝗟𝗮𝗺𝗯𝗱𝗮 (our event-driven serverless functions) for services that are short-lived, triggered by a specific event, and entirely 𝘀𝘁𝗮𝘁𝗲𝗹𝗲𝘀𝘀.  • 𝗦𝗰𝗼𝗿𝗶𝗻𝗴 𝗦𝗲𝗿𝘃𝗶𝗰𝗲: This is 𝗕𝘂𝗿𝘀𝘁𝘆 & 𝗦𝘁𝗮𝘁𝗲𝗹𝗲𝘀𝘀. It's only needed for a single task: calculate the score and push it to Kinesis. Lambda provides 𝗜𝗻𝘀𝘁𝗮𝗻𝘁 𝗦𝗰𝗮𝗹𝗲 & 𝗖𝗼𝘀𝘁 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆, scaling instantly to meet the massive burst of answers without pre-provisioning capacity, helping us meet that sub-200ms SLA.  • 𝗟𝗲𝗮𝗱𝗲𝗿𝗯𝗼𝗮𝗿𝗱 𝗦𝗲𝗿𝘃𝗶𝗰𝗲: This is 𝗔𝘀𝘆𝗻𝗰𝗵𝗿𝗼𝗻𝗼𝘂𝘀 & 𝗘𝘃𝗲𝗻𝘁-𝗗𝗿𝗶𝘃𝗲𝗻. It's triggered only when a GAME_ENDED event arrives from Kinesis. Lambda wakes up, updates the database, and shuts down. This maximizes 𝗖𝗼𝘀𝘁 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆 while ensuring reliable database writes. 𝗧𝗵𝗲 𝗧𝗮𝗸𝗲𝗮𝘄𝗮𝘆 By combining Fargate and Lambda, we achieved the best of both worlds: 𝗰𝗼𝘀𝘁 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 and 𝗿𝗲𝗹𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆. This layered approach is the essence of building a robust, modern cloud-native system. What trade-offs do you prioritize when choosing between containers and serverless functions for a real-time app? 🤔 #AWSServerless #Fargate #AWSLambda #SystemDesign #CloudNative #TechSeries

🚀 𝗔𝗜_𝗘𝘅𝘁𝗿𝗮𝗰𝘁 𝗶𝘀 𝗻𝗼𝘄 𝗚𝗲𝗻𝗲𝗿𝗮𝗹𝗹𝘆 𝗔𝘃𝗮𝗶𝗹𝗮𝗯𝗹𝗲! Big moment for unstructured data in the Snowflake AI Data Cloud. AI_Extract, powered by our in-house Arctic-Extract model, is now GA across all deployments and platforms. 🧊 What it is An AISQL function that lets you extract structure from documents, text blobs, and images — directly in Snowflake. No pipelines. No orchestration. Just SQL. ⚙️ Why I love it ✅ One API for text, images, and documents ✅ Works in-place on your cloud storage (S3, Azure, or GCS) — no data movement ✅ Supports 20+ common file types — PDFs, DOCX, PNGs, JPEGs, TXTs, and more ✅ Flexible JSON outputs — define exactly what you need ✅ 29 languages supported 💡 What’s new in GA • Support for full-table JSON extractions • 4K token context window • Prompt context injection for guided extractions • Brand-new upgraded model — higher quality on DOCVQA + table extraction tasks AI_Extract turns unstructured data into queryable insights — instantly, with the same simplicity as SELECT *. #Snowflake #AIExtract #Arctic #AISQL #DataCloud #Developers #UnstructuredData #GenAI