October 19, 2023

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SUP203T7S4

Scaling your Startup to your first

10 million users

Zac Burns (He/Him)

Startup Solutions Architect,
Amazon Web Services

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Build Measure

Learn

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
So let’s start from

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Users: >1

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Day 1 the old way – Single instance

Previously you would see a single-instance-

based starting architecture that would host
the 3 main layers of an application: User
Amazon Route 53
1. Front-End VPC

2. Backend
3. Database Elastic IP
address

Instance

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s changed?

This old way of starting host-based has grown out of favor due
mostly to the popularity of frontend framework technology:
• Node.js React, jQuery, Angular, and Vue.js round out top web frameworks
and technologies in the Stack Overflow Developer Survey 2023

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s the impact?

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Users >1: With modern frontend

With a modern frontend, developers are

choosing to deploy them to specialized hosting
products. User Amazon Route 53
Why?

• Greatly reduced operations overhead

• Built in scale/performance Backend
• Integrations with the modern frontend frameworks
AWS Amplify Hosting

• Aligned Developer experience capabilities

The backend then becomes a different component(s)

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amplify Hosting
FEATURES FOR HOSTING MODERN WEB APPLICATIONS

Easy custom Simplified continuous

Globally available domain setup workflows

Feature
Atomic deployments Password protection
branch deployments

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What about the
backend?

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Evaluating compute options

The instance-based model is still one

possible model for hosting your backend
business logic and data tiers, but with clear User
disadvantages: Amazon Route 53

• No failover
• No redundancy VPC

Elastic IP
• Can’t scale individual components independently address

• Constrained on technology choices for AWS Amplify Hosting

individual components
AWS’s guidance: make use of managed compute for Instance

your backend and managed databases for your data

tier.

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Evaluating managed compute on AWS
More opinionated
AWS manages Customer manages

• Data source integrations • Application code

AWS Lambda • Physical hardware, software, networking,
Serverless functions and facilities
• Provisioning

• Container orchestration, provisioning • Application code

AWS Fargate • Cluster scaling • Data source integrations
Serverless containers • Physical hardware, host OS/kernel, • Security config and updates, network config,
networking, and facilities management tasks

• Container orchestration control plane • Application code

• Physical hardware software, • Data source integrations
ECS/EKS networking, and facilities • Work clusters
Container-management as a service
• Security config and updates, network config,
firewall, management tasks

• Physical hardware software, • Application code

EC2 networking, and facilities • Data source integrations
Infrastructure-as-a-Service • Scaling
• Security config and updates, network config,
management tasks
• Provisioning, managing scaling, and
patching of servers
Less opinionated

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Exposing business logic to the frontend
THREE OPTIONS FOR EXPOSING AN API

Amazon API Gateway Application Load Balancer AWS AppSync

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS App Runner

• Build, deploy, and run containerized web

applications and API services
• Reduced overall operational overhead
• Built on ECS with Fargate, Auto Scaling,
Elastic Load Balancing (ELB), and Amazon
Elastic Container Repository (Amazon ECR)
• Supports popular language runtimes such
as Node.js, Python, php, Go, Java, .Net,
AWS App Runner and Rails
• Both public and private applications

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Users >1: with modern frontend and backend

Developers are looking to leverage User Amazon Route 53

managed compute to rapidly start building
and deploying their backend applications.
Why?
• Greatly reduced operations overhead
• Built in scale/performance AWS Amplify Hosting App Runner

• Integrations with the modern backend

frameworks
• Aligned Developer experience capabilities
Database

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
To NoSQL, or not to NoSQL?

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start with SQL databases

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why start with SQL?
• Established and well-known technology
• Lots of existing code, communities, books, and tools
• You aren’t going to break SQL databases with your first millions of users.
No, really, you won’t.*
• Clear patterns to scalability

*Unless you are doing something super peculiar with the data or you have massive
amounts of it, but even then SQL will have a place in your stack

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Aurora
Relational databases built for the cloud – performance and availability of commercial
databases at 1/10th the cost

Performance Availability
Highly secure Fully managed
and scalability and durability
Several times faster than Fault-tolerant self-healing storage Network isolation Managed by Amazon RDS:
standard MySQL and no hardware provisioning,
6 copies of data across 3 AZs Encryption at rest/transit
PostgreSQL software patching, setup,
15 read replicas Single global database with configuration, or backups
cross-Region replication

The fastest growing service in the history of AWS

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Aurora Serverless v2

On-demand and auto scaling configuration

Automatically scales capacity based on application needs

Simple pay-per-use pricing per second

Next version scales instantly to support demanding applications

Worry-free database capacity management

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Users >1:

By leveraging a managed services for frontend, User Amazon Route 53

backend, and database we can start off day 1, user
1, with a great foundation and little overhead.

• No self-managed infrastructure
• Built in scalability
AWS Amplify Hosting App Runner
• Built in high availability
VPC
• Layers of security and access controls
• Aligned costs to value

From here we can go pretty far! Amazon Aurora

Serverless v2

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Users: >100

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Users: >1000

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Users: >10,000

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
 y b e ?
m a
Users: >10,000

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Users: >10,000. What starts to go wrong?

The current stack will scale incredibly far,

but the scaling of single tier/monolithic User Amazon Route 53

applications can sometimes only go so far.

You’ll eventually run into issues common in
most architectures:
• Varied needs of the product AWS Amplify Hosting App Runner
complicating others
• Poor performance in one part impacting VPC

other parts
• Slowing queries in the database due to
large table sizes/index growth Amazon Aurora
Serverless v2

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Let’s dive into the frontend tier

Amazon Route 53

Frontend Backend
AWS Amplify Hosting App Runner

VPC

Data storage
Amazon Aurora
Serverless v2

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scaling the frontend

Generally speaking, Amplify Hosting can scale to

meet customer needs. Amazon CloudFront

Built on top of the 410+ Amazon CloudFront PoPs

globally
Performance typically comes from
• Tuning frontend code
• Reducing the number of backend calls
• Caching images/JavaScript/CSS effectively

Amazon CloudFront Points of

Presence (PoP)

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Let’s dive into the data tier

Amazon Route 53

Frontend Backend
AWS Amplify Hosting App Runner

VPC

Database
Amazon Aurora
Serverless v2

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora Serverless v2: scaling
Compute fleet
Scales in place in under a second by adding more CPU
and memory resources
No impact due to scaling even when running hundreds
of thousands of transactions
Compute fleet continuously monitored and scaled
horizontally for heat management
Up to 15x faster scale downs compared to v1 Automatically grows

Storage fleet

Automatically grows

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora Serverless v2: scaling
Up to 15 read replicas act as failover targets Availability Availability Availability
All instances inherit capacity configuration from the Zone 1 Zone 2 Zone 3
cluster
Tier 0 and 1 read replicas match the size of the
primary instance
Tier 0 Tier 1
Tier 14 Tier 15
Deploy across separate AZs Writer Reader
Reader Reader

Multi-AZ Aurora clusters supported by 99.99%

uptime SLA

Cluster volume

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon RDS Proxy
A FULLY MANAGED, HIGHLY AVAILABLE DATABASE PROXY FOR AMAZON RDS AND AMAZON AURORA

Pool and share DB Increase app Manage app data Fully managed DB
connections for availability and reduce security with DB proxy, compatible with
improved app scaling DB failover times access controls your database

Amazon RDS Proxy supports Aurora Serverless v2, including mixed configurations
with Aurora provisioned and serverless instances within a cluster

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The best database queries are the
ones you never need to make (often).

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon ElastiCache

Managed Memcached or Redis

Scale from one to many nodes
Self-healing (replaces dead instance)
Single-digit millisecond speeds
(usually)
Amazon Multi-AZ deployments for availability
ElastiCache

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scaling the data tier

Three main methods for scaling our data

tier: VPC

• Increasing the size of the instance(s)

used App Runner
ElastiCache for
Memcached

• Adding read replicas and a proxy to help

scale read queries
Amazon Route 53
Amazon RDS Amazon Aurora
proxy Serverless v2

• Typically minor application changes

• Using caches to remove queries from
AWS Amplify Hosting
Amazon Aurora Amazon Aurora
read replica read replica

even needing to be made

• requires more significant application changes
and new logic to handle

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Let’s dive into the backend tier

Amazon Route 53

Frontend Backend
AWS Amplify Hosting App Runner

VPC

Data storage
Amazon Aurora
Serverless v2

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
App Runner: Under the hood

App Runner service accounts

ECS Fargate Tasks

HTTP
request L7 Request
Router
Clients Internet

Internet facing
NLB

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
App Runner: scaling instance sizes

Individual instances are configured for a mix of

CPU and Memory (see table)
ECS Fargate Tasks
Maximum number of concurrent requests per
instance: 200
Maximum number of instances per service: 25

Current limit of 5000 concurrent requests per CPU Memory

App Runner service (your deployed application) 0.25 vCPU 0.5GB, 1GB
0.5 vCPU 1GB
1 vCPU 2GB, 3GB, 4GB
2 vCPU 4GB, 6GB
4vCPU 8GB, 10GB, 12GB
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scaling the backend tier

With basic configuration you can hit 5000 concurrent

requests.
VPC

• For context, at 2 seconds per request you could

perform ~150K requests per minute ElastiCache for
Memcached
App Runner
Application performance tuning remains key
• Reducing slow database queries Amazon Route 53
Amazon RDS Amazon Aurora
proxy Serverless v2

• Profiling code with CodeGuru and similar tools for

costly/slow logic
AWS Amplify Hosting

Caching in the edge/application client where

Amazon Aurora Amazon Aurora
• read replica read replica

possible

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Users: >100,000

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Users: >1,000,000

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Users: >1,000,000

At some point you will outgrow the patterns

we’ve discussed so far: VPC

• Need more scale than App Runner can

support today in single service App Runner
ElastiCache for
Memcached

• Database writes begin to become a

bottleneck
Amazon Route 53
Amazon RDS Amazon Aurora
proxy Serverless v2

• Application complexity/feature growth

begins to necessitate new infrastructure AWS Amplify Hosting
Amazon Aurora
read replica
Amazon Aurora

needs
read replica

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Going the microservices route
Moving to a service-oriented or microservices based architecture
is a refactor that requires deep planning across all layers.
• Start with with the easiest to cut away features/capabilities
that don’t involve too many cross-function ties
• Data domain mapping
• Business function mapping
• Good time to evaluate other compute technologies for
specific needs
• Will need to think about how to “glue” everything together

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Database federation
VPC

• Split up databases by function/purpose Forums database Amazon RDS

proxy
Amazon Aurora
Serverless v2

• Repeat scaling patterns discussed previously for

each
Amazon Aurora Amazon Aurora
read replica read replica

• Can mix and match to align with specific business

needs
Users database
• Harder to do cross-function queries Amazon RDS
proxy
Amazon Aurora
Serverless v2

• Essentially delays sharding/NoSQL

• Won’t help with single huge queries or tables with Amazon Aurora

incredible amounts of data read replica

Products database
• Often the result of misaligned workloads to the
technology. E.g., Data warehouse workloads on Amazon Aurora

RDBMSes
Serverless v2

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Shifting functionality to NoSQL

Leverage managed services such as

DynamoDB
• Supports massive scale with
consistent low latency
Example use cases
• “Hot” tables
Amazon
• Metadata/lookup tables
DynamoDB
• Leaderboards/scoring
• Temporary data needs (cart data)

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Breaking up the backend tier

Breaking up the backend can mirror the data tier:

Amazon Route 53
• Split the application into new federated services aligned
to data patterns
• Revisit which managed compute best aligns
Explore what business logic can move to internal services
• Moving from sync to async AWS Amplify Hosting Amazon API Gateway

• Leveraging queues, topics, buses, and streams to build VPC

event-driven architectures
/ /users

App Runner Lambda function

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
10 million+ users

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Users: >10 million

• More distribution of features/functionality across internal

microservices

• Deeply analyze your entire stack’s performance and continue

to find areas to improve

• Possibly start to build on self-managed compute

• Evaluating how to improve caching at all tiers

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
To infinity . . .

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!
Zac Burns (He/Him)
Startup Solutions Architect,
Amazon Web Services

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
 Please complete
the session survey

© 2021,
2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.