Apoorv Upreti

One of the most common realisations for teams starting to self-host LLMs: Inference engine ≠ inference stack. It usually starts simple - spin up vLLM or NIMs, run a basic test, maybe get a demo or pilot app working. But then as you look to productionise, things get messy: - You need to meet certain performance / reliability SLAs  - Figuring out what / how to monitor isn’t obvious - Configuring Kubernetes requires a different skillset / team to be involved We’ve seen this pattern again and again - across teams with different goals, stacks, and levels of experience. And the problem is: there’s no standard playbook. Most of what’s working is coming from Slack threads, GitHub issues, and internal experiments — stitched together by teams figuring it out as they go. At Doubleword, we’re working with a bunch of teams trying to get inference to production-grade — across different hardware, environments, and latency/cost constraints. We’re learning a lot. Not just about what works — but where things fall apart, and what tends to be overengineered too early. Over the coming weeks & months, I’ll be sharing our real-world learnings and best practices about all things self-hosting: ☑️ Common friction points to prepare for ☑️ How teams are solving for scale, latency, and observability ☑️ Where to spend time early, and what can wait To start, I’ve attached a snapshot of our current view on what a best-in-class self-hosted inference stack looks like at the enterprise level. It’s not a checklist - you definitely don’t need every component on day one. But hopefully it gives a clearer picture of where things tend to head as usage scales and use cases become more mission-critical. I’m also putting together a video/blog series to go deeper into some recurring questions: - How many users can your GPU serve - What metrics should you be tracking in an LLM stack - How to make your load balancer LLM-aware - How to serve 100+ models on a single GPU with no cold starts If there are areas you’d want to see covered or if something in the diagram seems we’d love your feedback - we’re very much still learning too! Just drop a comment or DM.

60

1 Comment

💡 A perspective for founders on angel money Many founders ask for very small cheques from angels — ₹5–10L. Nothing wrong with that, but here’s the reality: 👉 The smaller the cheque, the lesser the involvement. Why? Because time is also an investment. If the stake is tiny, the incentive to spend hours guiding, mentoring, and opening networks naturally reduces. As angels, we love to support founders. But meaningful guidance, regular reviews, and network push usually come when there’s meaningful skin in the game. So if you’re raising, think beyond “let me collect 20 small cheques.” Instead, ask yourself: “Which 2–3 angels can back me meaningfully with both money and time?” ⚡ Funding isn’t just about filling the cap table. It’s about building the right circle of believers who walk with you. #AngelInvestor #FounderTips #Fundraising #StartupInsights #DeepakBhagnaniFamilyOffice

106

60 Comments

I did a very candid podcast with Sanskar Goyal and we went deep into something every founder eventually struggles with: how to raise funds. We talked about everything founders struggle with: how to pitch, how to run a fundraise, and how the process really works behind closed doors. The real, inside look at how investors think, decide, and commit. If you’re founder, or someone who wants to raise money, this episode will save you months of guessing. We broke down:  • How to structure your pitch  • What investors care about (and what they don’t)  • The 6-step process to close a round  • Best practices from both sides of the table If you’re planning to raise money, now or later, this one’s worth your time. Go check it out - https://lnkd.in/g5-RJmp6

142

6 Comments

The hidden co-founder efficiency hack is workout integration. Shreyas Kumar and I have twice-a-week training sessions that have completely transformed our FERMÀT journey. We've discovered a multi-dimensional approach that simultaneously addresses our physical wellbeing, relationship dynamics, and company vision. What makes this particularly powerful is the intermittent work discussion format. Unlike scheduled meetings with their rigid structure, workout sessions create natural conversation flows. We discuss FERMÀT vision during rest periods, problem-solve between sets, and align our thinking organically. Having this casual social time programmed twice weekly has become invaluable for our founder relationship—which fundamentally determines company health. It's rare to have regular, non-work-specific interactions built into our schedules, yet these moments create essential connection points. The conversations during physical exertion have a different quality—more honest, more creative, more aligned than traditional boardroom discussions. The implementation mechanics are straightforward but critical: mid-day scheduling creates natural breaks, our trainer handles all programming (eliminating workout decision fatigue), and we execute quick transitions back to business. This founder optimization unlocks triple benefits: physical health, stronger friendship, and enhanced company outcomes. Most founders separate wellness from company building—we've discovered their integration creates exponential returns rather than just additive ones.

73

11 Comments

While running SRE at Zomato, one thing stood out very quickly: Delivering a great user experience at scale requires an extraordinary amount of discipline. Zomato puts an insane amount of effort into quality. In practice, almost 30–40% of the time before a release goes into carefully verifying every part of the app, making sure no bugs reach users. That investment is deliberate. At that scale, even small issues show up immediately and matter to real customers. From an engineering point of view, it’s natural to ask: Why not automate more of this work? Tools like Appium and Selenium existed long before AI, and teams explored them seriously. But what became clear over time was that the ROI of end-to-end mobile automation using traditional frameworks was close to zero. Not because teams weren’t capable. But because the problem itself was hard. Mobile interfaces change frequently. End-to-end tests were brittle, expensive to maintain, and often failed to provide the confidence teams actually needed. What this experience highlighted wasn’t a gap in effort or intent. It highlighted how difficult reliable verification really is. Later, working with fast-scaling teams, I kept seeing the same pattern: strong engineering teams, fast release cycles, and a large amount of human attention devoted to making sure things felt right. That accumulation of experience eventually led us to start Zenact AI The idea is simple and modest: Verification should behave like a careful human reviewer - but be easier to sustain, earlier in the cycle, and continuous. Not more tests. Better assurance. We’re still learning what that looks like in practice, and we’ll share what we learn as we go. #EngineeringThoughts #SRE #ReliabilityEngineering #FounderPerspective #B2BSoftware #ZenactAI

56

Ankit Shukla and Aakash Gupta did an awesome study on PM salaries in India They found that average PM CTC grows from 17 Lakhs per annum as an APM to 130 Lakhs as a CPO. Of course, there's some bands around all of this. Here's the total comp 25th to 75th percentiles: Associate PM: 14 to 20 lakhs PM: 18 to 38 lakhs SPM: 37 to 62 lakhs PPM/GPM: 62 to 97 lakhs Director: 61 to 133 lakhs VP: 77 to 135 lakhs CPO: 52 to 194 lakhs But it's interesting to look at the average. For instance, here's the average split between Fixed/Variable/ESOPs Associate PM: 82%/ 11%/ 6% PM: 81%/ 6%/ 13% SPM: 76%/ 9%/ 14% PPM/GPM: 74%/ 13%/ 14% Director: 65%/ 7%/ 28% VP: 65%/ 7%/ 28% CPO: 55%/ 7%/ 38% The ramp up in variable and ESOPs is very dramatic at senior levels, from 17% to 45% of total compensation. And that leads to huge increases in Total CTC. For more details - mediums, locations, company, and other cuts - check out the full report: https://lnkd.in/djbctbAV

292

18 Comments