Aayush Ahuja

My research paper just got published 🎉, and we were able to improve the cache-hit ratio for nginx workloads by 146%. Last week, Aayush and I were brainstorming an idea to improve the cache hit ratio, and that's when we applied Deep Q-Networks to see if we could get some major benefits. The core idea is - instead of evicting the least recently used key, we trained a small model that, given various features of an NGINX request, chooses the best key to evict among the coldest k. Give it a read to understand our thesis and nuances. The link is in the comment. The paper is titled "Cold-RL: Learning Cache Eviction with Offline Reinforcement Learning for NGINX".

4,967

75 Comments

At Zomato, the Home page and Search are powered by in-house, self-managed Solr clusters. Over time, the team acquired the expertise needed to handle the nuances of self-management. Despite that, we still experienced some unexpected downtimes — a few of which were relatively unknown to the broader community, probably because we were using Solr at a significantly high scale and had modelled our use cases quite differently. I’ve discussed one such downtime in detail in this blog: https://lnkd.in/gDJuBU_P While we kept improving the availability of the systems, we also built fallback disaster recovery mechanisms to handle downtimes. I had written about one of those in a previous post — it talked about serving the Zomato Home page using DynamoDB when the main flow was down. But that was only a partial recovery mechanism. Needless to say, customers rely heavily on Search to find their favourite dishes and restaurants. So we had to build a fallback mechanism for Search as well — since DynamoDB doesn’t support full-text search. Customers typically search either for a Dish/Cuisine or a Restaurant. While restaurant search is hyperlocal, Dish/Cuisine search is more universal. Interestingly, the majority of searches are for dishes — and even more interestingly, ~99% of those dish searches are confined to fewer than 1000 dishes globally. We looked at data for the top ~1000 dishes and discovered popular keywords customers use to search for them — e.g., sezwan for Schezwan, panir for Paneer, etc. We created an in-memory Trie of these strings, populated at container boot time. So, if the Solr index is unavailable, we can use this Trie to return matching dishes for the input keyword. This automatically handles phonetic matches and spelling mistakes for most user inputs. We could’ve gone further and implemented 1–2 edit distance matching, but the impact would’ve been negligible — the keyword data was already sufficient to serve the majority of real customer queries. All in all, this was a sub-10MB, almost no-cost in-memory data structure, capable of serving Dish/Cuisine searches with sub-millisecond latency on the server side. The result ranking was based on the universal popularity of dishes. While the results aren’t personalised and don’t include restaurant data, they still allow customers to search for their favourite dishes during downtimes — and in such cases, availability takes precedence over consistency. #datastructure #solr #trie #search

430

20 Comments

Netflix reduced connection churn in its API gateway by 10x. Connection churn is a massive problem with proxies like API gateways, and it effectively means the consistent opening and closing of TCP connections. Under heavy traffic, this leads to increased CPU utilization and latency. To reduce connection churn, Netflix used two things 1. H2 multiplexing - allowing multiple requests over a single TCP connection. 2. Subsetting - spread traffic across shards, each with its own connection pool. This not only reduced the number of TCP connections needed to open and close but also improved scalability i.e., with fewer connections to manage, Zuul (their API Gateway) was able to handle more traffic without sacrificing performance. They have written an amazing blog going deeper into the actual optimization and algorithm. Definitely one of the most fascinating articles I've read recently - link in the comments.

965

12 Comments

Address matching is a super interesting problem and something I highly recommend trying to solve from scratch. The core idea is to determine whether two addresses refer to the same location, for example  - 221B Baker St., London NW1 6XE, UK  - 221-B Baker Street, NW1 6XE London, United Kingdom The core problem statement is to identify if two slightly different-looking address strings actually refer to the same physical location. Things become interesting as you would need to accommodate typos, abbreviations, formatting differences, missing components, or even language variations. If you give this a shot, you will learn about  - fuzzy string matching  - normalizing and parsing text data  - rule-based vs machine learning-based entity resolution  - improve match accuracy with geospatial data Again, use the LLM of your choice, but make sure you dig deeper and understand all the nuances. Hope it helps. Have fun.

1,120

34 Comments

Why is Redis so fast? There are many design decisions that make Redis blisteringly quick. We’ll focus on two that carry most of the weight. - First is Redis’s in-memory design + single-threaded event loop. Keeping data in RAM and processing commands in one core loop avoids locks and random I/O. - Second is its purpose-built data structures and lean wire protocol. Most commands hit O(1) or O(log N) paths with tiny per-op overhead. What happens when you read from Redis👇 Step 1: A client sends a command using RESP (a simple, compact text/binary protocol). Minimal parsing and small payloads reduce CPU and network overhead. No big JSON/XML blobs. Step 2: Redis’s event loop accepts the socket and queues work. Single-threaded command execution means no lock contention or context-switch thrash. If you want to use more CPUs you need to run more Redis instances in a cluster. Step 3: The command is parsed and routed via a hash table lookup to the target keyspace entry.  Lookups are O(1) on average thanks to efficient dictionaries and cache-friendly memory layouts. Step 4: The operation runs entirely in memory using specialized structures:   Strings, lists (quicklist), hashes (compact encodings), sets/intsets, sorted sets (skiplist + hash), streams, etc. They're are engineered for predictable, fast operations with tight CPU caches. Step 5: The response is written back through the same event loop.   Pipelining and batching can amortize syscalls and round trips, pushing throughput even higher. Step 6: Persistence and replication are off the critical path.   AOF uses append-only, sequential writes with configurable fsync; RDB snapshots happen in a child process.  Basically: we can lose data! This is a tradeoff Redis makes. Replication is async by default. The slow stuff is handled in the background so the hot path stays hot. Because Redis keeps data in RAM, executes commands in a single, lock-free event loop, and uses highly optimized data structures and a lean protocol, it avoids the latency traps of disk I/O, heavy parsing, and lock contention. That’s why it’s fast. Reed more about Redis here! https://lnkd.in/g2TEVzvx

163

Spot nodes are cheap until volatility bites. At Statsig, we automated GKE spot node selection and now pay 50% less for the same spot capacity. • Daily price scraping ranks every node family by region • IaC regenerates node pools and ComputeClasses whenever prices shift • Continuous rollout applies updated priorities across clusters automatically Result: half the spend on spot nodes, higher reliability, and a smoother developer workflow with zero manual tuning. Full article in the comments walks through the details and the hurdles we tackled along the way.

39

1 Comment

Looks like 100s of engineers joined Cloudflare yesterday and all of them were fired for causing the outage... Twitter was down Canva was down ChatGPT was down Claude was down Even the down detector, the site used to detect the down status of websites, was itself down. But in such a situation, one quality comes in that makes or breaks your caree,r and that is Ownership... Outages don’t ruin careers. Lack of ownership does. When things go down, good engineers don’t panic. They don’t hide. They don’t throw blame. They do three things that decide the next 10 years of their career. 1) They step forward, not backward When a system is burning, most people subconsciously wait for someone else to act. A good engineer says “I’m taking this part, you take the next” and moves fast. Clear roles. Zero ego. You become the person teams trust. 2) They communicate like adults Outages get worse because people stay silent. A good engineer gives short, crisp updates. What we know.  What we don’t. What changed last. Which component is suspicious. Leaders aren’t looking for genius during downtime.  They are looking for clarity. 3) They fix it first, learn later Every outage has two phases: Mitigation and investigation. Good engineers separate them. Bring the system back. Stabilize. Only then open the postmortem and dig. And trust me, when you’ve been around long enough, you realise one thing: Your career doesn’t grow from all the good days. It grows from the three or four bad days where everything collapses at once. Yesterday, the internet broke because of maybe one bad change. It happens. It will happen again. Every company, every stack, every team faces this at some point. But the people who rise after days like this are the ones who say: “Yeah, this was on me. Here’s what happened. Here’s what I’m fixing next.” Ownership is a habit. And nothing will shape your career more than how you behave on your worst technical day.

1,485

23 Comments

Don’t use a database for items that are mostly static, hardly ever change are in the hundreds. E.g. 50 states in the US. 🇺🇸 If you don't want incremental ints, then consider and in this order. 1. ULIDs 2. CUIDs 3. Hash Ids And yes, there is UUID v7, but still I would consider lower than the ones above.

134

138 Comments

Sheetal Singh & I have been paring for over a month on some Rust OSS projects: our first submission to the community is Parseltongue - came out of the need for code-noobs like me to understand code at the abstract level of LLDs - as Interface Signature Graphs - which expresses the overall structure of code in just 5% of total code LOC / tokens - or even lesser - so you can reason with the codebase in far lesser context window every time you update the code, within a few milli-seconds you will be able to see the updated ISG as a visualization and can also find the blast radius of the change you intend so you can run it as - a live code daemon - a gitingest txt codebase analyzer (we did it for tokio library) our work is primary inspired by Shreyas Doshi's Product Sense course - which both of us attended and which nudged us to think for ourselves on problems seeking a differentiated experience for our users via the tools / libraries this is an initial draft & we will be applying to delta residency with this idea seek feedback from folks on what we could improve further link to github repo in first comment our vision with Parseltongue: use Interface Signature Graphs to reason large codebases with LLMs in a faster & more accurate manner onbv 🙏

20

1 Comment

ACTORS + AI: Director/Actor/Producer/Writer AI Chef 🧑‍🍳 Hans Obma visits the Cozy AI Kitchen 🎂 https://lnkd.in/gjiAvEfP to share his insights on how AI has transformed his creative processes. You may have seen Chef Hans in such 📺 hits as Better Call Saul, Wandavision, or Narcos — and yes, he's probably the most handsome AI 🧑‍🍳 chef we've had in the kitchen to date :-). Hans needed to learn a new language ... how did he do it? He got multimodal AI models to help him speak Welsh. Hans needed a way to scale how he communicated with his stakeholders ... how did he do it? He created an authentic-to-him way of scaling by creating an instant entourage of supporting agents. HT 🧑‍🍳 Ross Heise 🧑‍🍳 Matt Scholz and the power of the Wisconsin network for bringing such a storied talent to the Cozy AI Kitchen! --- This 🍰 episode: https://lnkd.in/gjiAvEfP All 50 episodes: https://lnkd.in/g6upvbGX

89

11 Comments

WhatsApp’s Architecture: Simplicity at Scale Most companies scale by throwing more complexity at the problem. WhatsApp went the other way initially: 1) Erlang actors instead of threads 2) Hotloading instead of redeploying 3) Semi-manual operations over heavy automation And yet, a tiny team reliably ran the world’s largest messaging system. In my latest post, I break down how WhatsApp pulled it off, https://lnkd.in/g5YnvW9p

70

All this talk about the PRD being dead but the funny thing is getting AI to write a detailed PRD with requirements, tech stack, design guidelines, and clear milestones upfront dramatically improves its subsequent coding output.

84

24 Comments