The art of designing systems that scale

Imagine you're the architect of a restaurant. You don't just decide where tables go, but how the kitchen flows, how many cooks you need, where you store ingredients, and what happens when 500 customers arrive instead of 50.

System design is exactly that: planning how to build software that works well today and can grow tomorrow.

Good system design isn't the most complex one, but the one that solves the current problem with room to grow.

Monolith vs Microservices

The first architectural decision you'll face.

Monolith: All in one

┌─────────────────────────────────────┐
│           APPLICATION               │
│  ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐   │
│  │Auth │ │Users│ │Orders│ │Pay  │   │
│  └─────┘ └─────┘ └─────┘ └─────┘   │
│         Single database             │
│              ┌───┐                  │
│              │ DB│                  │
│              └───┘                  │
└─────────────────────────────────────┘

Advantages:

Simple to develop and deploy
Easy to debug (everything in one place)
Single database = consistency
Ideal for small teams (<10 devs)

Disadvantages:

Scale all or nothing
One bug can bring everything down
Risky deployments
Hard to maintain as it grows

Microservices: Divide and conquer

┌─────────┐  ┌─────────┐  ┌─────────┐
│  Auth   │  │  Users  │  │ Orders  │
│ Service │  │ Service │  │ Service │
└────┬────┘  └────┬────┘  └────┬────┘
     │            │            │
┌────┴────┐  ┌────┴────┐  ┌────┴────┐
│Auth DB  │  │Users DB │  │Orders DB│
└─────────┘  └─────────┘  └─────────┘

Advantages:

Scale only what you need
Independent teams
One service fails, not all
Different technologies per service

Disadvantages:

High operational complexity
Distributed debugging is hard
Eventual consistency (not immediate)
Requires mature DevOps

When to use each

Scenario	Recommendation
Startup, MVP, < 5 devs	Monolith
Proven product, > 20 devs	Microservices
Parts with very different loads	Hybrid
Don't know which to choose	Monolith

Golden rule: Start with monolith. Extract microservices when the pain is real, not imagined.

The CAP Theorem

In distributed systems, you can only have 2 of 3:

        Consistency
           /\
          /  \
         /    \
        /      \
       /   ??   \
      /          \
     /____________\
Availability    Partition
                Tolerance

Consistency (C): Everyone sees the same data at the same time
Availability (A): The system always responds
Partition Tolerance (P): Works even with network failures

In practice

Network partitions CAN ALWAYS happen. So you really choose between:

System	Chooses	Sacrifices	Example
CP	Consistency	Availability	Banks, inventory
AP	Availability	Consistency	Social networks, cache

Real example: In a bank, if there's a network failure, you prefer the ATM to say "Not available" (CP) rather than let you withdraw money you don't have (AP).

Scaling: Vertical vs Horizontal

Vertical: Bigger machine

Before:         After:
┌─────┐         ┌─────────┐
│ 4GB │   →     │  64GB   │
│ 2CPU│         │  32CPU  │
└─────┘         └─────────┘

Simple: just upgrade the server
Has physical limits
Single point of failure

Horizontal: More machines

Before:         After:
┌─────┐         ┌─────┐ ┌─────┐ ┌─────┐
│ 4GB │   →     │ 4GB │ │ 4GB │ │ 4GB │
└─────┘         └─────┘ └─────┘ └─────┘

Theoretically infinite
Requires Load Balancer
Your app must be stateless

Load Balancers

Distribute traffic among multiple servers.

                    ┌─────────┐
                    │  Load   │
        Users   →   │Balancer │
                    └────┬────┘
                         │
           ┌─────────────┼─────────────┐
           ▼             ▼             ▼
      ┌─────────┐   ┌─────────┐   ┌─────────┐
      │Server 1 │   │Server 2 │   │Server 3 │
      └─────────┘   └─────────┘   └─────────┘

Distribution algorithms

Algorithm	How it works	When to use
Round Robin	1, 2, 3, 1, 2, 3...	Equal servers
Least Connections	To the one with fewer	Long connections
IP Hash	Same client → same server	Sticky sessions
Weighted	More to the powerful one	Different servers

Scaling Databases

Replication: Read copies

     Writes
        │
        ▼
   ┌─────────┐
   │ Primary │──────────────┐
   │  (RW)   │              │ Replication
   └─────────┘              │
        │                   │
        ▼                   ▼
   ┌─────────┐         ┌─────────┐
   │ Replica │         │ Replica │
   │  (RO)   │         │  (RO)   │
   └─────────┘         └─────────┘
        ▲                   ▲
        │                   │
      Reads              Reads

Scales reads, not writes
Eventual consistency (replication lag)

Sharding: Split the data

user_id 1-1000      user_id 1001-2000    user_id 2001-3000
       │                   │                    │
       ▼                   ▼                    ▼
  ┌─────────┐         ┌─────────┐         ┌─────────┐
  │ Shard 1 │         │ Shard 2 │         │ Shard 3 │
  └─────────┘         └─────────┘         └─────────┘

Scales both reads and writes
Complexity: JOINs between shards are expensive
Choosing a good shard key is critical

Caching: The key to performance

Cache strategies

Cache-Aside (Lazy Loading)

1. App requests data
2. Cache miss? → Read from DB → Store in cache
3. Cache hit? → Return from cache

┌─────┐  miss   ┌─────┐        ┌────┐
│ App │ ──────→ │Cache│        │ DB │
│     │ ←────── │     │        │    │
└─────┘  hit    └─────┘        └────┘
    │                              ▲
    └──────────────────────────────┘
         miss: read and store

Write-Through

Write to cache AND DB at the same time
- Data always consistent
- Slower writes

Write-Behind (Write-Back)

Write to cache, then async to DB
- Fast writes
- Risk of data loss if cache fails

What to cache

Candidate	Priority
Data that doesn't change (config)	High
Frequently read data	High
Expensive calculation results	High
Active user data	Medium
Data that changes every second	Low

Message Queues

For asynchronous communication between services.

┌─────────┐     ┌─────────────┐     ┌─────────────┐
│Producer │ ──→ │    Queue    │ ──→ │  Consumer   │
│ (API)   │     │ (RabbitMQ)  │     │  (Worker)   │
└─────────┘     └─────────────┘     └─────────────┘

Use cases

Email sending: API enqueues, worker sends
Image processing: Upload enqueues, worker processes
Notifications: Event enqueues, multiple consumers notify

Popular tools

Tool	Best for
RabbitMQ	Traditional messaging, complex routing
Redis Streams	Simple, you already have Redis
Kafka	High volume, event sourcing
SQS	AWS native, simple

Practical case: Designing a URL Shortener

Requirements

Functional:

Shorten long URL → short code
Redirect code → original URL
URLs expire (optional)

Non-functional:

100M new URLs/month
10:1 read:write ratio
Latency < 100ms

Estimations

URLs/month: 100M
URLs/sec: 100M / (30 * 24 * 3600) ≈ 40 URLs/sec writes
Reads: 40 * 10 = 400 URLs/sec reads

Storage (5 years):
100M * 12 * 5 = 6B URLs
6B * 500 bytes = 3TB

Short code design

Base62: [a-zA-Z0-9] = 62 characters

7 characters = 62^7 = 3.5 trillion combinations
Enough for 100M/month for centuries

Final architecture

                    ┌───────────┐
     Users     ───→ │    LB     │
                    └─────┬─────┘
                          │
              ┌───────────┴───────────┐
              ▼                       ▼
         ┌─────────┐             ┌─────────┐
         │ API 1   │             │ API 2   │
         └────┬────┘             └────┬────┘
              │                       │
              └───────────┬───────────┘
                          │
                    ┌─────┴─────┐
                    │   Redis   │ (Cache hot URLs)
                    └─────┬─────┘
                          │
                    ┌─────┴─────┐
                    │ Postgres  │ (Sharded by hash)
                    └───────────┘

Recommended resources

Book: "Designing Data-Intensive Applications" - Martin Kleppmann
Book: "System Design Interview" - Alex Xu
Web: system-design-primer
Practice: Exercism System Design

Practice

-> Architecture Workshop - Design a real system step by step

System Design