Defend the architecture when a dependency fails.

Requirements, load, failure and cost. Make the trade-offs explicit.

Try a design decision

Peak load. One missing zone.

12,000 jobs/second · authored exercise

Zone ARunning
Zone BRunning
Zone CUnavailable
Test your fleet

Five decisions worth defending

02 A lost reply is not a failed write

“A timeout or failure doesn't necessarily mean that side effects haven't happened.” (opens the source)

Marc Brooker, Author, Amazon Builders’ Library · AWS Builder Center, Dec 5, 2019

Choose one operation identity across repeated attempts.

Brooker’s retry rule is to “retry at a single point in the stack.” (Marc Brooker, AWS Builder Center, opens the source) A timeout policy also needs a total deadline, an attempt limit and evidence that retrying can help.

  • Marc Brooker AWS

03 One customer must not consume every worker

“reduces the blast radius by limiting infra-level incidents to a shard-level impact.” (opens the source)

Florencia Bonansea, Senior Principal Engineer, Atlassian · Inside Atlassian, Dec 15, 2025

Name both the isolated failure and the dependencies still shared.

In his EC2 account, van der Merwe protects the existing data plane: “VMs that are already running need to keep working.” (Zak van der Merwe, All Things Distributed, opens the source) A new configuration request must not become a dependency for every live request.

  • Zak van der Merwe AWS

04 Fresh enough is a business rule

“preventing race conditions or out-of-order events that could expose unauthorized data.” (opens the source)

Florencia Bonansea, Senior Principal Engineer, Atlassian · Inside Atlassian, Dec 15, 2025

Authorize the object version you will actually serve.

Meyer says an agent needs more than a ticket: “It needs the requirements, the relevant architecture, the decision history, and the constraints the team already knows.” (Dave Meyer, Inside Atlassian, opens the source) Include the consistency requirement before accepting its proposed cache.

  • Dave Meyer Atlassian

05 Keep the delivery path independent

“When code is cheap, the bottleneck moves to judgment” (opens the source)

Zak van der Merwe, EC2 and DSQL control-plane engineer, AWS · All Things Distributed, Aug 4, 2026

Own what happens after the diagram becomes a running service.

  • Peter Salanki CoreWeave

Sources

  1. Amazon, SDE III candidate guidance, at the time. Amazon Jobs, “SDE III/Sr. SDE Interview Prep”, Undated. Open the source. Checked against the published text.
  2. Atlassian, Engineering candidate guidance, at the time. Atlassian, “Atlassian engineering interview handbook”, Undated. Open the source. Checked against the published text.
  3. Mike Furr, Author, Amazon Builders’ Library, at the time. AWS Builder Center, “Static stability using Availability Zones”, Dec 5, 2019. Open the source. Checked against the published text.
  4. Marc Brooker, Author, Amazon Builders’ Library, at the time. AWS Builder Center, “Timeouts, retries, and backoff with jitter”, Dec 5, 2019. Open the source. Checked against the published text.
  5. Florencia Bonansea, Senior Principal Engineer, Atlassian, at the time. Inside Atlassian, “How We Unlocked Performance at Scale with Jira Platform”, Dec 15, 2025. Open the source. Checked against the published text.
  6. Zak van der Merwe, EC2 and DSQL control-plane engineer, AWS, at the time. All Things Distributed, “On building scalable control planes”, Aug 4, 2026. Open the source. Checked against the published text.
  7. Dave Meyer, Author, Inside Atlassian, at the time. Inside Atlassian, “How we’re evolving Jira for AI-native software development”, Jul 15, 2026. Open the source. Checked against the published text.
  8. Peter Salanki, CTO and co-founder, CoreWeave, at the time. The Stack Overflow Podcast, “Do you have what it takes to run AI in production?”, May 26, 2026. Open the source. Checked against the published text.

Last reviewed Nov 10, 2026