Back to blog System Design

50 Scalability & System Design Interview Questions

29 min readSystem Design

Core Scalability Concepts

1. What is scalability? Explain horizontal vs. vertical scaling.

Scalability is the ability of a system to handle increased load (users, traffic, data) by adding resources without performance degradation. Horizontal scaling is preferred for large-scale, high-availability systems.

Vertical Scaling (Scale Up)

  • Add more power to a single machine (CPU, RAM, Disk)
  • Example: Upgrade server from 8GB RAM to 64GB

Pros

  • Simple to implement
  • No code changes

Cons

  • Hardware limit
  • Single point of failure
  • Expensive

Horizontal Scaling (Scale Out)

  • Add more machines and distribute load
  • Example: Add multiple servers behind a load balancer

Pros

  • High availability
  • No hard limit
  • Cost-effective at scale

Cons

  • More complex (distributed systems, data consistency)

2. How would you design a system that scales from 1 to 10 million users?

3. What are the pros/cons of vertical vs. horizontal scaling?

4. Explain the Scale Cube and how it applies to system scaling.

5. What is the CAP theorem and why is it important?

6. How do you estimate capacity and load before designing a scalable system?

7. When would you choose microservices over monoliths for scalability?

8. Explain the trade-offs between consistency and availability in distributed systems.

9. What are common scalability bottlenecks, and how would you identify them?

10. How does auto-scaling work in cloud environments (AWS/GCP/Azure)?

Load Handling & Traffic Management

1. Design a load balancer for a high-traffic web service.

High-level design:

  • Client → DNS → Load Balancer → App Servers
  • Load balancer distributes traffic across multiple stateless servers

Key components:

  • Algorithms: Round-robin, least connections, weighted
  • Health checks: Remove unhealthy instances
  • TLS termination: Offload SSL
  • Auto-scaling: Add/remove backend servers dynamically

2. How do proxies (forward/reverse) help scalability?

3. What are sticky sessions and when should they be used?

4. What’s the difference between an API gateway and a load balancer?

5. How would you handle sudden traffic spikes?

6. How would you design global load balancing across regions?

7. What metrics would you monitor for scalability (latency, throughput)?

8. How to design rate limiting for millions of concurrent users?

9. What strategies exist for graceful degradation under heavy load?

10. How do CDN and edge caching improve scalability?

Data Storage & Database Scaling

1. How would you scale a database for billions of records?

To scale a database for billions of records, multiple strategies are applied together to ensure performance, availability, and reliability:

  1. Horizontal Scaling (Sharding)
    Split data across multiple databases based on a shard key (e.g., user ID, region) so no single database becomes a bottleneck.
  2. Read Replicas
    Use read replicas to distribute read-heavy traffic and reduce load on the primary database.
  3. Index Optimization
    Create efficient indexes on frequently queried fields to speed up reads while balancing write performance.
  4. Caching Layer
    Introduce caching (Redis/Memcached) to serve frequently accessed data and minimize database hits.
  5. Data Partitioning & Archival
    Partition large tables and move old or infrequently accessed data to cold storage.
  6. Polyglot Persistence
    Use different databases for different needs (SQL for transactions, NoSQL for large-scale reads or logs).
  7. Asynchronous Writes & Batching
    Batch writes and use async processing to handle high write throughput efficiently.
  8. Monitoring & Capacity Planning
    Continuously monitor performance metrics and plan capacity ahead to prevent hotspots.

2. When and how do you implement sharding?

3. What are the scalability challenges with SQL vs NoSQL?

4. Design a distributed cache (e.g., memcached/Redis).

5. How would you handle cache invalidation in a distributed cache?

6. How do read replicas improve scalability?

7. What’s the role of asynchronous processing (queues) in scalability?

8. Design a data partitioning strategy for multi-tenant data.

9. How would you scale write-heavy workloads?

10. What is hot partition / hot key problem and how do you solve it?

High-Level / Real Systems

1. Design a scalable notification service.

A scalable notification service must reliably deliver messages (push, email, SMS) to millions of users with low latency and high availability, while handling traffic spikes.

High-level design:

  • Clients trigger notification requests via an API Gateway
  • Requests are validated and pushed to a message queue
  • Notification workers consume messages asynchronously
  • Channel-specific services handle Email / SMS / Push
  • External providers (FCM, APNs, SMS gateways) deliver messages
  • Delivery status is stored for tracking and retries

Scalability strategies:

  • Asynchronous processing with queues to absorb spikes
  • Horizontal scaling of worker consumers per channel
  • Rate limiting & batching to respect provider limits
  • Retry with exponential backoff and dead-letter queues
  • User preference & template caching to reduce DB reads

Reliability & optimization:

  • Idempotent message handling to avoid duplicates
  • Priority queues for critical notifications (OTP, alerts)
  • Monitoring delivery success, latency, and failures

This design ensures the notification system remains fast, resilient, and cost-efficient at massive scale.

2. How would you design a system to support millions of concurrent users?

3. Design a scalable push notification platform.

4. How would you scale a real-time chat service?

5. Design a leaderboard system that supports millions of players.

6. How would you design a distributed logging / analytics system?

7. Design a scalable CDN or content distribution layer.

8. How would you scale a search engine for rapid responses?

9. How would you handle multi-region deployment and failover?

10. Design a fault-tolerant system with zero single point of failure.

Advanced & Scenario-Based

1. How would you build a globally replicated database?

A globally replicated database is designed to provide low latency, high availability, and disaster recovery for users across different geographic regions, while carefully managing consistency.

Core design approach:

  • Deploy database nodes in multiple regions (active–active or active–passive)
  • Choose a replication strategy:
    • Synchronous replication for strong consistency (higher latency)
    • Asynchronous replication for better scalability and availability
  • Partition data geographically so reads and writes stay close to users

Consistency & reliability strategies:

  • Use conflict resolution mechanisms (timestamps, version vectors, or last-write-wins)
  • Apply leader election or quorum-based writes to maintain correctness
  • Support automatic failover and region health checks
  • Ensure backup, restore, and data validation across regions

This design balances latency, consistency, and fault tolerance, enabling the database to scale globally while remaining reliable.

2. How to optimize a system for latency (in addition to scalability)?

3. What patterns help scale message queues? (e.g., Kafka partitioning)

4. Design event-driven architecture for scalable workflows.

5. How would you scale ML model serving for thousands of requests/second?

6. How would you handle eventual consistency across distributed caches?

7. Design an API gateway pattern for efficient service orchestration.

8. What are key trade-offs between synchronous vs asynchronous systems?

9. How would you scale a financial transaction processing system?

10. Describe back-of-envelope estimation for system components during design.

Tips for Answering Scalability Questions

✔️ Clarify requirements first: Ask about traffic patterns, data size, SLAs, and failure expectations.
✔️ Draw architecture diagrams: Use boxes and arrows to show components and data flows.
✔️ Discuss trade-offs: E.g., consistency vs availability, simplicity vs performance.
✔️ Use real examples: Reference YouTube, Instagram, Twitter for scalable components.

InterviewExpert.org

Writing about interviews, stacks, and getting hired.

Leave a comment