ARCHITECTURE / SYSTEM DESIGN

DESIGN FOR FAILURE.
ASSUME EVERYTHING BREAKS.

Architecture is not about drawing boxes. It is about understanding dependencies, failure domains, traffic, state, recovery and operational reality.

01 / PRINCIPLES

Start with the failure model.

A production architecture should answer one question before technology is selected:

What happens when this component fails?

Not whether it can fail. It will.

  • What is the failure domain?
  • What happens to traffic?
  • Where does state live?
  • Can the system recover automatically?
  • What requires human intervention?
  • How much data can be lost?
  • How long can the service be unavailable?
  • How do we know it has failed?
FAILURE DOMAINS

Don't build one giant blast radius.

Separate systems across hosts, racks, availability zones, networks and administrative boundaries where appropriate.

STATE

Know where your data lives.

Stateless applications are easy to replace. Stateful systems require deliberate storage, replication and recovery design.

DEPENDENCIES

Every dependency is part of your system.

DNS, databases, identity, certificates, storage, networks and external APIs all belong in the failure model.

RECOVERY

HA is not DR.

High availability reduces downtime. Disaster recovery restores service after larger failures. They solve different problems.

02 / REFERENCE ARCHITECTURE

A production service.

A simplified reference architecture for a modern internet-facing application.

TRAFFIC FLOW
                    INTERNET
                        |
                        v
                +---------------+
                |      DNS      |
                +---------------+
                        |
                        v
                +---------------+
                |  CDN / WAF    |
                +---------------+
                        |
                        v
                +---------------+
                | LOAD BALANCER |
                +---------------+
                        |
              +---------+---------+
              |                   |
              v                   v
        +-----------+       +-----------+
        |  INGRESS  |       |  INGRESS  |
        |    AZ-1   |       |    AZ-2   |
        +-----------+       +-----------+
              |                   |
              +---------+---------+
                        |
                        v
                +---------------+
                |  APPLICATION  |
                |   KUBERNETES  |
                +---------------+
                        |
             +----------+----------+
             |          |          |
             v          v          v
         +-------+  +-------+  +-------+
         | REDIS |  |  API  |  | QUEUE |
         +-------+  +-------+  +-------+
                        |
                        v
                 +-------------+
                 |  DATABASE   |
                 |  PRIMARY /  |
                 |  REPLICA    |
                 +-------------+
                        |
                        v
                 +-------------+
                 |   BACKUPS   |
                 +-------------+
FIELD NOTE

The diagram is not the architecture. The relationships between the components are.

For every arrow ask: What protocol? What authentication? What happens on timeout? What happens on retry?

03 / FAILURE DOMAINS

Separate the blast radius.

                         REGION
                            |
          +-----------------+-----------------+
          |                                   |
          v                                   v
     AVAILABILITY ZONE A                AVAILABILITY ZONE B
          |                                   |
     +----+----+                         +----+----+
     |         |                         |         |
     v         v                         v         v
  NODE-01   NODE-02                   NODE-03   NODE-04
     |         |                         |         |
     v         v                         v         v
  PODS      PODS                      PODS      PODS
HOST FAILURE

A single server disappears. Workloads should be rescheduled elsewhere.

ZONE FAILURE

An entire availability zone becomes unavailable. Replicas must exist outside that zone.

REGION FAILURE

The entire region is unavailable. Multi-region architecture and DR become relevant.

HUMAN FAILURE

A deployment introduces a defect. Rollback and progressive delivery become critical.

Replication is not automatically resilience.

Three replicas on three nodes in the same failure domain can still disappear together.

04 / HIGH AVAILABILITY

Remove single points of failure.

Component Single Instance HA Approach
Load Balancer Single endpoint Managed / redundant pair
Ingress One replica Multiple replicas across nodes
Application One pod Multiple replicas
Node Single worker Multiple workers
Database Single primary Replication / managed HA
Storage Single disk Redundancy + backup
DNS Single resolver Redundant infrastructure
IMPORTANT

Removing one single point of failure can expose another.

Example: three Kubernetes replicas do not provide application availability if all three depend on one unavailable database.

05 / KUBERNETES

Treat the cluster as a distributed system.

                    KUBERNETES CLUSTER

                +-----------------------+
                |     CONTROL PLANE     |
                |                       |
                | API Server            |
                | Scheduler             |
                | Controllers           |
                | etcd                  |
                +-----------+-----------+
                            |
             +--------------+--------------+
             |              |              |
             v              v              v
        +---------+    +---------+    +---------+
        | NODE 01 |    | NODE 02 |    | NODE 03 |
        |         |    |         |    |         |
        | kubelet |    | kubelet |    | kubelet |
        | runtime |    | runtime |    | runtime |
        |         |    |         |    |         |
        | PODS    |    | PODS    |    | PODS    |
        +---------+    +---------+    +---------+

Design questions

  • What happens if a node disappears?
  • What happens if the control plane is unavailable?
  • Where are replicas scheduled?
  • Can workloads accidentally concentrate on one node?
  • Where does persistent state live?
  • What happens when the cluster network fails?
  • How are certificates rotated?
  • How does a deployment get rolled back?
ANTI-AFFINITY

Spread workloads.

Prevent replicas from becoming dependent on the same node or failure domain.

PDB

Protect availability.

PodDisruptionBudgets can limit voluntary disruption during maintenance operations.

PROBES

Health must be meaningful.

Liveness, readiness and startup probes have different purposes. Don't use them interchangeably.

RESOURCES

Make scheduling predictable.

Requests and limits influence scheduling, capacity planning and workload behaviour.

06 / NETWORKING

Follow the packet.

Application architecture is inseparable from network architecture.

CLIENT
  |
  | HTTPS / 443
  v
DNS
  |
  v
CDN / WAF
  |
  v
LOAD BALANCER
  |
  | HTTP / HTTPS
  v
INGRESS
  |
  | Service
  v
APPLICATION
  |
  | TCP
  v
DATABASE
NORTH / SOUTH

Traffic entering or leaving the environment.

EAST / WEST

Traffic between internal services and workloads.

SEGMENTATION

Separate systems according to trust boundaries and operational requirements.

DNS

Name resolution is infrastructure. When DNS fails, everything appears broken.

TROUBLESHOOTING RULE

Don't start with the application. Start with the packet path.

DNS
 →
Route
 →
Firewall
 →
Load Balancer
 →
Listener
 →
Ingress
 →
Service
 →
Pod
 →
Application
 →
Dependency
07 / CLOUD

Use the cloud as infrastructure. Not as architecture.

Cloud services provide primitives. Architecture determines how those primitives interact.

                    CLOUD ACCOUNT
                         |
          +--------------+--------------+
          |                             |
          v                             v
       NETWORK                       IDENTITY
          |                             |
     +----+----+                   +----+----+
     |         |                   |         |
     v         v                   v         v
   PUBLIC   PRIVATE               IAM      SECRETS
   SUBNET   SUBNET
     |         |
     v         v
    LB       WORKLOADS
              |
              v
           DATABASE
              |
              v
           BACKUPS

Architecture layers

Layer Questions
Identity Who can access what?
Network Who can communicate with whom?
Compute Where does the workload run?
Storage Where does persistent state live?
Security What happens when trust is breached?
Observability How do we know the system is healthy?
Recovery How do we restore service?
08 / GITOPS

Make the desired state visible.

                 GIT REPOSITORY
                       |
                       | commit
                       v
                 +-----------+
                 |   CI/CD   |
                 +-----------+
                       |
                       v
                 +-----------+
                 |   IMAGE   |
                 | REGISTRY  |
                 +-----------+
                       |
                       v
                 +-----------+
                 |   GITOPS  |
                 | CONTROLLER|
                 +-----------+
                       |
                       v
                KUBERNETES API
                       |
                       v
                    CLUSTER
                       |
                       v
                   WORKLOADS
SOURCE

Git is the desired state.

Configuration should be reviewable, versioned and auditable.

DRIFT

Detect reality diverging.

Manual changes should become visible rather than quietly becoming the new production configuration.

ROLLBACK

Previous state should be recoverable.

A deployment is not complete if there is no reliable way to reverse it.

AUDIT

Know who changed what.

Git history provides an operational record of configuration changes.

09 / OBSERVABILITY

You cannot operate what you cannot see.

METRICS

What is happening?

CPU, memory, latency, throughput, errors, saturation and application-specific signals.

LOGS

What happened?

Events, errors, requests, state transitions and diagnostic information.

TRACES

Where did the request go?

Follow distributed requests across services and dependencies.

ALERTS

What requires action?

Alert on symptoms and service impact rather than every metric that changes.

THE GOLDEN SIGNALS
LATENCY
ERRORS
TRAFFIC
SATURATION

Observability should help an engineer move from symptom → evidence → cause → action.

10 / SECURITY

Assume compromise.

Security architecture is not a firewall diagram. It is the collection of controls limiting what an attacker can do after something goes wrong.

                    INTERNET
                        |
                     [ WAF ]
                        |
                  [ LOAD BALANCER ]
                        |
                  [ INGRESS ]
                        |
              +---------+---------+
              |                   |
          [ SERVICE A ]       [ SERVICE B ]
              |                   |
              +---------+---------+
                        |
                  [ DATABASE ]
                        |
                    [ BACKUP ]
LEAST PRIVILEGE

Every identity gets only the permissions required to perform its function.

SECRETS

Credentials should not be embedded in images, repositories or configuration files.

SEGMENTATION

Limit lateral movement between systems and trust zones.

AUDIT

Authentication, authorization and administrative actions should leave evidence.

11 / STATE

Stateless is easy. State is where architecture gets serious.

STATELESS
REQUEST
   |
   v
+--------+
| POD A  |
+--------+

POD A dies

   |

   v

+--------+
| POD B  |
+--------+

The workload can simply be replaced.

STATEFUL
REQUEST
   |
   v
+--------+
| APP    |
+--------+
   |
   v
+----------+
| DATABASE |
+----------+
   |
   +---- REPLICATION
   |
   +---- BACKUP
   |
   +---- RECOVERY

The workload and its data have different recovery requirements.

ASK THESE QUESTIONS
  • What is the source of truth?
  • How is data replicated?
  • How is corruption detected?
  • How often are backups taken?
  • Have restores actually been tested?
  • What is the RPO?
  • What is the RTO?
12 / RECOVERY

RPO and RTO are architecture decisions.

Term Meaning Example
RPO Maximum acceptable data loss 15 minutes
RTO Maximum acceptable recovery time 30 minutes
HA Reduce normal service interruption Multi-AZ application
DR Recover from major failure Secondary region
A BACKUP THAT HAS NEVER BEEN RESTORED IS AN ASSUMPTION.

Recovery procedures need to be tested.

13 / CAPACITY

Capacity is part of reliability.

CPU

Understand utilization, saturation and CPU throttling.

MEMORY

Memory pressure can produce eviction, OOM kills and cascading failure.

STORAGE

Disk exhaustion can break databases, logging, schedulers and operating systems.

NETWORK

Bandwidth, latency, packet loss and connection limits can become application bottlenecks.

AVAILABLE CAPACITY
        |
        v
CURRENT LOAD
        |
        v
GROWTH RATE
        |
        v
FAILURE RESERVE
        |
        v
SCALING DECISION
14 / ARCHITECTURE REVIEW

The architecture review checklist.

  1. Traffic
    How does a request enter and leave the system?
  2. Dependencies
    What external and internal systems does it require?
  3. Failure
    What happens when each component disappears?
  4. State
    Where does persistent data live?
  5. Scale
    What happens when traffic increases by 10×?
  6. Security
    What happens if an identity or workload is compromised?
  7. Observability
    How will the operations team know something is wrong?
  8. Deployment
    How does new software reach production?
  9. Rollback
    How quickly can a bad deployment be reversed?
  10. Recovery
    How is the system restored after catastrophic failure?
  11. Cost
    What does the architecture cost at current and peak scale?
  12. Human factors
    Can an engineer actually operate this system at 03:00?
15 / THE 03:00 TEST

Would you want to operate this system at 03:00?

Production architecture should be evaluated by the person who has to troubleshoot it when everything is on fire.

If diagnosing a failure requires tribal knowledge, ten dashboards and three people who happen to be asleep, the architecture has an operational problem.

RED FLAGS
  • Unknown dependencies
  • Manual production changes
  • No tested rollback
  • No restore testing
  • Single points of failure
  • Missing monitoring
  • Unbounded retries
  • Unclear ownership
16 / PHILOSOPHY

Good architecture makes failure boring.

A good system does not prevent every failure.

A good system limits the blast radius.

A good system detects failure quickly.

A good system recovers predictably.

A good system tells the engineer what happened.

A good system can be changed without fear.