Benefits of Asynchronous Processing

𝗛𝗼𝘄 𝘁𝗼 𝗽𝗿𝗼𝗰𝗲𝘀𝘀 𝟭𝟬𝟬𝗸 𝗺𝗲𝘀𝘀𝗮𝗴𝗲𝘀/𝘀𝗲𝗰 𝘄𝗶𝘁𝗵 𝗽𝗿𝗼𝗽𝗲𝗿 𝗮𝘀𝘆𝗻𝗰 Yesterday's async mistakes are fixable with surprisingly simple patterns. The same infrastructure that struggled with 1000 requests can handle 100,000 when async is done right. The key principles: → Never block on async code → Always return Task (never async void) → Control concurrency explicitly → Use ValueTask for hot paths Async all the way means no .Result, no .Wait(), no GetAwaiter().GetResult(). These blocking calls defeat the entire purpose of async operations. Parallel operations with Task.WhenAll reduce latency dramatically. Two 100ms calls take 100ms in parallel, not 200ms sequentially. ConfigureAwait(false) in library code prevents SynchronizationContext captures, avoiding deadlocks and improving performance. SemaphoreSlim provides async-friendly concurrency control. Unlike lock statements, it doesn't block threads while waiting. ValueTask eliminates allocations for synchronously-completing operations. Critical for high-frequency methods where every allocation matters. Typical improvements after fixing async patterns: ● Response time: 10-100x faster ● Throughput: 50-100x increase ● Thread count: 80% reduction ● Memory allocation: 60% less ● Cloud costs: 50-70% savings The investment in proper async patterns pays for itself within the first month through reduced infrastructure costs alone. ���� 𝗞𝗲𝘆 𝗧𝗮𝗸𝗲𝗮𝘄𝗮𝘆: Async isn't just syntax - it's a fundamental shift in how applications handle I/O operations. 𝗪𝗵𝗮𝘁'𝘀 𝘁𝗵𝗲 𝗺𝗼𝘀𝘁 𝘀𝘂𝗿𝗽𝗿𝗶𝘀𝗶𝗻𝗴 𝗮𝘀𝘆𝗻𝗰 𝗯𝘂𝗴 𝘆𝗼𝘂'𝘃𝗲 𝗳𝗼𝘂𝗻𝗱 𝗶𝗻 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻? #AsyncAwait #DotNet #CSharp #PerformanceOptimization #SoftwareEngineering #ValueTask #Concurrency

Op-Ed: Why Not Async? This question has been on my mind ever since Platform Events were introduced: Why not make async via Platform Events the default solution for transactions within the Salesforce platform? Consider this simple example: Every time an Opportunity is set to Closed-Won, a set of Tasks needs to be created for the invoicing team. Synchronous Solution: Record Trigger Flow on Opportunity that creates the Tasks. Async/PE Solution: Record Trigger Flow fires an “Opportunity Closed” Platform Event. A second Flow on PE listens to that event and creates the Tasks in its own transaction. Key Differences: Wait Time: Sync operations have a considerably longer wait time for the user. Async, on the other hand, is super fast. Errors: Sync operations show all errors directly to the user, while async solutions hide the errors from the user. Coupling: Sync implementations tightly couple closing an Opportunity with the creation of Tasks. Async solutions can be triggered from many different places, offering greater flexibility. Limits: Async implementations get their own set of limits for each “transaction/functionality.” Say goodbye to optimizing for CPU time, heap size, SOQL… But… Limits: There’s a lot of confusion around PE limits. Simply put, as long as you stay in Triggers or Flows, the limits are pretty high. The limit is 250k events per hour, which should be enough for most orgs. Error Handling: “Users are not supposed to replace an error log.” Yes, errors in downstream processes will not stop the upstream process anymore, and the user will not be informed about any errors. This means errors have to be logged and handled by a support team. Debugging can be more challenging. Immediate Records: True, the Task records from our example will not show up immediately to the user. But honestly, users usually don’t care; they just want to close their Opportunity. From my experience, they trust the system that the tasks have been created. This is a task for BAs to understand: What does the user really need? Storytime: This concept is not theoretical. I know two large orgs that implemented an “async by default” pattern many years ago. It works perfectly fine. Wait time for users was reduced by up to 80%. Conclusion: For most requirements, an async implementation is the way to go. Trigger Frameworks: An async architecture (almost) eliminates the need for complex trigger frameworks because it removes much of the complexity that trigger frameworks introduce.

Not All Apex Should Run in the Same Transaction A user clicks Save. Behind the scenes: Triggers execute Flows run Validation rules fire Integrations may trigger All inside one synchronous transaction. Now imagine that transaction also tries to: Process 50k records Call multiple external APIs Perform heavy calculations The result? CPU limits exceeded. Timeouts. Failed transactions. This is why Asynchronous Apex exists. 🔄 Why Async Processing Matters Asynchronous processing allows Salesforce to move heavy work outside the main transaction. Benefits include: Reduced CPU pressure Better scalability Higher processing limits Improved user experience The user transaction finishes quickly while heavy processing continues in the background. 🧩 The Four Main Async Apex Tools 🔹 Future Methods Used for simple background operations. Best for: Callouts Lightweight asynchronous logic Limitations: Limited monitoring Cannot chain jobs 🔹 Queueable Apex More flexible than Future methods. Best for: Chaining jobs Passing complex objects Monitoring job execution Queueables are now the preferred async pattern. 🔹 Batch Apex Designed for large data processing. Best for: Millions of records Scheduled large operations Data cleanup tasks Each batch executes in smaller chunks. 🔹 Scheduled Apex Used to run jobs at a specific time. Best for: Nightly processing Regular maintenance tasks Periodic integrations 🧠 Architectural Insight A common mistake developers make is trying to do everything in one transaction. But scalable Salesforce systems split work between: Synchronous logic (fast) Asynchronous processing (heavy) 💬 When building automation, how do you decide what should run asynchronously? #Salesforce #Apex #AsyncApex #QueueableApex #BatchApex #SalesforceDeveloper

Kafka Use Case 5: Event-Driven Microservices REST APIs are great, but they couple services tightly. Kafka enables asynchronous microservice communication. Services emit events like user.created, order.placed. Other services consume and react (e.g., send emails, update inventory). Events are durable and replayable. Benefits: - Loose coupling - Horizontal scalability - Observability into service interactions - Event sourcing and CQRS patterns But remember, event-driven systems introduce their own tradeoffs: - Debugging is harder - Event ordering can be complex - Testing and tracing become more distributed - Overhead if the use case doesn’t need asynchronicity 👉 Not every system needs to be event-driven. Is your system reactive or request-bound? #Kafka #Microservices #EventDriven #CQRS #Architecture

⚡ Asynchronous Microservices Communication — Event-Driven vs Message-Driven As systems scale to hundreds of services, synchronous request/response patterns alone become brittle — latency grows, failures cascade, and teams struggle to evolve independently. Asynchronous communication solves this by decoupling services and letting them react to events or messages without waiting for immediate responses. ⸻ 🔹 Event-Driven (Publish/Subscribe) • Core Idea: Services emit domain events (e.g., OrderPlaced, PaymentCompleted). Consumers subscribe to topics and react asynchronously. Architecture: • Producers publish events to a broker (Kafka, Pulsar, SNS). • Topics can be partitioned for horizontal scalability. • Consumers form consumer groups — load is balanced automatically. Strengths: • High scalability & decoupling — producers don’t know who consumes. • Real-time pipelines for analytics, personalization, monitoring. • Replay & event sourcing for rebuilding state. Challenges: • Eventual consistency — no guaranteed transaction across services. • Schema evolution & backward compatibility required (Avro/Protobuf). • Harder end-to-end tracing. ⸻ 🔹 Message-Driven (Point-to-Point) • Core Idea: A producer sends a message to a specific queue; one consumer instance processes it. Architecture: • Queues (RabbitMQ, ActiveMQ, SQS) act as buffers. • Each message is delivered to exactly one consumer. Strengths: • Simpler mental model — one message, one consumer. • Reliable delivery with acknowledgment & retry semantics. • Great for task distribution and guaranteed processing. Challenges: • Less flexible — single consumer pattern, no fan-out. • Harder to scale consumers dynamically across unrelated use cases. • No built-in replay beyond queue retention. ⸻ ✅ Practical Guidance • Use event-driven for broad distribution, reactive workflows, analytics, and audit trails. • Use message-driven when strict 1:1 processing and delivery guarantees are critical (payments, job execution). • Combine with dead-letter queues, retry policies, and observability (OpenTelemetry, Jaeger, Prometheus) to build robust async systems. The key insight: asynchronous communication isn’t one-size-fits-all — you pick the pattern to balance decoupling, reliability, and complexity. ⸻ #Microservices #EventDrivenArchitecture #MessageDriven #SystemDesign #Kafka #RabbitMQ #CloudNative #DistributedSystems #Observability #EngineeringExcellence #TechLeadership

When building low-latency, high-scale systems, a key strategy of mine is simple: “Push as much processing as possible to later.” Why It Matters? 🤔 In many systems—checkout, login, trade execution—latency matters because someone (or something) is waiting: - A customer at a point of sale - A user at a login screen - A system waiting on a transaction confirmation Platforms that support these scenarios must respond in milliseconds. If not, requests will fail, and user experiences will suffer. My Approach 🧠 I typically divide these platforms into two sub-platforms to optimize for speed and scale. 🏎️ Real-Time Platform: Optimized for scale and speed, only performing what is essential before responding to the request. 📥 Event-Driven Platform (sometimes Batch): Handles processing deferred from the real-time platform. It is still built for scale, but seconds, not milliseconds. Deciding What Belongs Where 🗃 I try to break down processing into steps, and for each step I ask: “Does this step need to happen before we respond or after?” ✅ If it MUST be performed before the response, use a real-time path. ⏭ If it can wait until after, event-driven path. Things that tend to follow the event-driven path are: - Audit logging - Downstream asynchronous notifications - Enrichment and Transformations - Checks that trigger out-of-band tasks These are not slow but don’t need to be “blocking.” Final Thoughts ✍️ The key message is that the more you do on the real-time path, the slower it is. This pattern is a good way to reduce the real-time workload. But the trick is to find a reliable and fast way to move work from a real-time to an event-driven system. Pub/Sub and gRPC streams are two of my go-to options. What is your favorite way to connect real-time and event-driven platforms? #Bengineering 🧐