Peng Li

In-memory databases are the architectural backbone of real-time AI. They’re a necessary advantage for memory-intensive workloads that rely on instant access to context. Modern systems like Redis blend durability, multi-model flexibility (JSON, vectors, time-series), and tiered storage to deliver the speed of memory with the reliability of disk. The result? Real-time analytics, instant personalization, and AI systems that actually feel intelligent. Here’s how we deliver the speed of memory with the reliability of disk: https://lnkd.in/gSm65kdB

49

Kikoff is constantly innovating to build smarter, faster financial tools. In an interview with Built In SF, Engineering Manager Patrick Wang shares how the team is tackling technical debt and optimizing performance to solve problems for customers, faster. Read more on what’s ahead: https://lnkd.in/gBvk7cQ9

10

When Steve Huynh was a principal engineer at Amazon, meetings began with a "study hall." Amazon had a "reading culture" even among engineers, Huynh recently told the Pragmatic Engineer podcast, speaking of his time at the tech giant. Employees frequently drafted six-page memos, which they shared with the company to update progress and demonstrate new projects. Huynh, who said the company's embrace of writing and reading the memos was part of its "secret sauce," said employees' writing was often constrained to the format, whether it was a business strategy or press release. Huynh started at Amazon in 2006, only a few years after the company turned its first profit and while Jeff Bezos was at the helm. Read more about how Bezos instilled this culture of memo-writing from the top down on Business Insider: https://lnkd.in/gBqbEf_j (Credit: Daniel Berman) #amazon #reading #jeffbezos

216

2 Comments

An agent has no consistent definition. You can think of an agent as an autonomous or semi-autonomous system that can take actions on behalf of the user in a given environment, state, and make decisions or take actions that can accomplish certain tasks along a given time-frame, either by calling tools, dynamically selecting the next action to take or use a deterministic "workflow" (systems where LLMs and tools are orchestrated through predefined code paths). Using an Agent framework that gives you the ability to built systems in both, a reliable and a dynamic way can accelerate your agent development journey using agent abstractions. This means building Agentic systems that can reliably call tools, store memory (episodic/semantic/procedural), have comprehensive logging and observability, human in the loop workflows and the ability to build various multi-agent patterns flexibly based on your use case. A successful Agentic system in production is usually a combination of both, dynamic and predictable/reliable multi-agent systems. Strands Agents SDK gives you exactly these capabilities by treating each “agent” as a combination of a foundation model plus a suite of tools. You define a prompt and register your tools (decorated functions) in code, then Strands handles reasoning→planning→tool-execution cycles, local testing, and cloud deployment (ECS, Fargate, Lambda, EC2), along with support for all other agent abstractions provided above. Excited to share a hedge-fund analyst multi agent system: This uses the newest Anthropic's Claude 4 Sonnet/Opus that powers the Lead Analyst Agent, routing incoming queries to specialized sub-agents for fundamental, technical, and market analyses. Each specialist is wrapped as a callable tool (using the “agents-as-tools” multi-agent pattern), so the orchestrator never has to implement domain logic itself and can handoff the task to an agent as a tool. For sensitive operations (insider lookups), we utilize a HITL approval step that halts execution until a human grants consent. We also use meta-tooling that enables the Lead Analyst to generate, load, and invoke new custom tools at runtime—whether it’s a portfolio beta calculator or a pricer—without redeployment. Strands also embeds observability (Langfuse) and OpenTelemetry tracing so you can trace reasoning events, tool invocations, errors, and end-to-end workflows in real time. View more information on the code implementation here: https://lnkd.in/gJmwVyGi Code implementation: https://lnkd.in/gzTtJvJq Thanks to 🏄‍♂️ Cagatay Cali for being a reviewer/collaborator on this! Feel free to try it out and reach out with any questions/ideas. #aws #agenticAI #strands #agents #generativeAI

60

6 Comments

You are in a system design interview at Google for the L5 Senior Engineer role, and the interviewer leans in and asks: “Why does Spotify keep playing when I drive into a tunnel with no signal, but YouTube Music often stops or buffers? If you were designing a music streaming system, what different design choices would lead to these two behaviors?” Here is how you break it down. Btw, if you’re preparing for system design/coding interviews, check out our mock interview tool. You can use it for free here: https://lnkd.in/gpCn7t2T We’ve added new features as well:  -Company-specific interviews -In-built interview scheduler  -performance insights & trends — Both apps look like simple music players. Under the hood, they are optimized for very different priorities. [1] Spotify style – Cache first, stream second Idea: The client behaves like a smart offline player that happens to stream. The backend is built to support aggressive prefetching. What happens when you hit play - Client requests the track from a CDN or edge node. - Instead of tiny chunks, it downloads a big buffer ahead of the playhead. - In parallel, it starts pre downloading the next few tracks in the queue. - Data is written to an encrypted cache on disk, not just memory. - Playback reads from that local cache, not directly from the network socket. What this means for tunnels and bad networks - When the network drops, the player already has tens of seconds or entire tracks cached. - Because the next one or two songs are already downloaded, you can be offline for a while and never notice. - Cold start cost is a bit higher. First play might take slightly longer, but then everything feels smooth. - It burns more local storage and possibly more data, because not every prefetched song will be listened to fully. In a design answer, you can mention - Local disk cache with eviction policies (LRU per user, per device). - Background prefetch of N upcoming tracks based on queue. - Download manager that adapts how aggressively it prefetches based on network quality and user settings. - CDN tuned for larger object delivery and range requests. - Explicit offline mode that pins playlists into cache. [2] YouTube Music style – Stream first, cache is minimal Idea: Treat audio like video streams. Cost and bandwidth are optimized first. When you hit play - Player requests audio (and sometimes video) via HLS or DASH-style chunks. - Each chunk is only a few seconds long. - Client keeps a small rolling buffer in memory, not a large queue on disk. - Prefetch of future songs is limited, because video tracks are large and expensive to fetch speculatively. What this means for tunnels and bad networks - If your connection dies, the player only has a few seconds of buffered data. - As soon as those chunks are consumed, playback stalls. - Startup can feel snappy and data usage is controlled, especially for casual listeners. - Works very well on stable networks, feels fragile in spotty coverage.

1,221

38 Comments

Google continues with support for production-ready agents - MCP protocol now supports gRPC natively as part of pluggable MCP transports. The benefits are many, my favorites are : ✅ Method-based authorization (per MCP tool) e.g., an agent is authorized to ReadFile but not DeleteFile ✅ Unified OTel observability (distributed tracing) Developers can trace a single user prompt through every subsequent multi-agent interactions ✅ Built-in flow control (back pressure) - agents do not need to take a coffee break and we expect MCP endpoints to be easily overwhelmed. Take a read - https://lnkd.in/gUCspabg

428

4 Comments

Thrilled to share that we launched the new AWS EC2 C8gn instances with Graviton4 processors: 30% better compute performance vs. C7gn (Graviton3-based). 🚀 Offers up to 600 Gbps network bandwidth - highest among network-optimized EC2 instances. Supports up to 48xlarge instances, 384 GiB memory, and 60 Gbps EBS bandwidth. See full details: https://lnkd.in/dpqczT7E #AWS #CloudComputing

34

Anyscale 🤝GKE Through our continued collaboration with the Google Kubernetes Engine (GKE) team, we’re advancing the experience to scale Ray and Kubernetes in the cloud. This work includes: Label Selectors for improved scheduling: https://lnkd.in/gUZ8iVUW New Features for AI Scaling: https://lnkd.in/gAGSMiir Enhanced TPU Support: https://lnkd.in/gsKDvwpv

65

3 Comments

Introducing the M5 The M5 chip brings significant advancements in performance and efficiency across various applications. 1. Massive GPU / AI performance boost The M5 features a next-generation 10-core GPU architecture, with each core equipped with an integrated Neural Accelerator. This results in over 4× the peak GPU compute performance for AI tasks compared to the M4. 2. Stronger CPU and enhanced Neural Engine With up to a 10-core CPU (6 efficiency cores + 4 performance cores), the M5 offers approximately 15% faster multithreaded performance over the M4. Additionally, the Neural Engine has been upgraded to 16 cores, enhancing performance for on-device AI and intelligent features. 3. Higher memory bandwidth / unified memory improvements The M5 provides around 153 GB/s of unified memory bandwidth, which is approximately 30% more than the M4. This unified memory architecture allows all components (CPU, GPU, Neural Engine) to access a larger memory pool, enabling larger AI models to run entirely on-device. 4. Graphics & rendering enhancements The improvements in the GPU also boost graphics performance, including ray tracing capabilities. Apple claims up to a ~45% uplift compared to the M4 in ray-traced workloads, enhancing visual fidelity, rendering speed, and smoothness in demanding 3D and graphics applications. 5. Deployment in new hardware + energy efficiency / environmental goals The M5 will power new devices, including the 14-inch MacBook Pro, iPad Pro, and upgraded Vision Pro, all of which are available for pre-order.

7

“In a very binary sense, Metronome made self-service at Anyscale possible.” That’s Matt Weber, Software Engineer, reflecting on Anyscale's expansion into new revenue channels. As the team added PLG and cloud marketplaces (AWS, GCP, and Azure), their manual billing setup couldn’t support the volume or complexity. Reporting, reconciliation, and manual workflows were slowing them down. With Metronome as their billing foundation, Anyscale unified every revenue motion on one platform and accelerated growth across the business. They launched self-serve billing end-to-end, brought AWS, GCP, and Azure marketplaces live in under three weeks each, and freed up three engineers and a PM from dedicated billing work. 🎉🤝 Big congratulations to the Anyscale team on the expansion—this is what it looks like when innovative product and modern monetization infrastructure come together. Read the full story at the link in the comments.

37

1 Comment