Part 074 — Capacity Engineering, Quotas, Reservations, and Forecasting

Capacity failure is different from cost failure.

Cost failure says:

we paid too much

Capacity failure says:

we could not get enough of the right resource when the workload needed it

On AWS, capacity is elastic, not infinite.

You can still fail because:

• EC2 capacity unavailable for desired instance type/AZ
• Service Quotas too low
• Auto Scaling max capacity too low
• subnet IPs exhausted
• EBS volume modification limit hit
• FSx throughput/storage cannot be increased fast enough
• S3 request pattern hot spots
• backup copy jobs lag behind RPO
• Spot capacity disappears
• GPU capacity unavailable during campaign
• DR Region lacks quota/capacity
• recovery account cannot launch instances
• Capacity Reservations are unused or misconfigured

Top-tier engineers treat capacity as a reliability system.

They forecast, reserve, diversify, test, and automate capacity readiness.

1. Problem yang Diselesaikan

Part ini membahas:

• capacity engineering mental model
• headroom and time-to-exhaustion
• Service Quotas
• EC2 On-Demand Capacity Reservations
• Capacity Blocks for ML
• Reserved Instances vs capacity reservations
• Spot capacity diversification
• Auto Scaling capacity planning
• subnet/IP capacity
• EBS/EFS/FSx/S3 capacity surfaces
• backup/DR capacity
• capacity forecasting
• quota automation
• capacity game days
• runbooks for insufficient capacity

2. Mental Model

2.1 Capacity is a promise you must validate

AWS gives scalable services, but your account/workload still has constraints:

service quota
regional capacity
zonal capacity
account limit
subnet IP space
resource-specific maximum
scaling speed
control-plane availability
application warmup time
budget/cost guardrail

Capacity planning asks:

Can the workload obtain enough usable capacity before SLO breaks?

2.2 Capacity has four layers

You must satisfy all four.

2.3 Headroom is reliability

Headroom:

headroom = available_capacity - current_required_capacity

But useful headroom is time-based:

time_to_exhaustion = headroom / growth_rate

Example:

EFS has 5 TiB free equivalent but grows 1 TiB/day -> 5 days
subnet has 80 IPs free but scale event needs 120 -> already insufficient
ASG max 100, current 90, scale forecast 130 -> max capacity breach

2.4 Capacity can be wrong shape

You may have "enough vCPU" but wrong shape:

• wrong instance family
• wrong AZ
• wrong architecture
• wrong GPU type
• insufficient memory
• insufficient network
• insufficient EBS bandwidth
• incompatible AMI
• no matching Capacity Reservation

Capacity must match the workload.

2.5 Capacity and cost conflict

More headroom costs more.

Too little headroom causes outages.

The engineering problem:

minimum reliable headroom for the failure/demand scenarios we care about

3. Service Quotas

3.1 What Service Quotas solves

Service Quotas helps view and manage quotas for AWS services from a central location. Some quotas can be increased by request.

Examples:

• EC2 On-Demand vCPU quota
• Spot vCPU quota
• EBS volume/snapshot quotas
• Auto Scaling group quotas
• VPC/security group/subnet-related limits
• EFS/FSx quotas
• backup vault/recovery point quotas
• Lambda concurrency
• ECS/Fargate quotas

3.2 Quota is not capacity

Quota means:

you are allowed to request up to X

It does not guarantee the underlying EC2 capacity exists at the exact time/AZ/type.

For guaranteed EC2 capacity, evaluate Capacity Reservations/Capacity Blocks.

3.3 Quota inventory

Maintain:

quota:
  service: ec2
  name: Running On-Demand Standard vCPUs
  region: ap-southeast-1
  currentLimit:
  currentUsage:
  forecastPeak:
  drRequired:
  owner:
  increaseRequested:

3.4 Quota headroom

Track:

quota_utilization = current_usage / quota_limit

Alert when:

• 70% for growing workloads

• 85% for stable workloads

• projected peak exceeds quota
• DR test cannot launch
• new campaign planned

3.5 Quota request lead time

Quota increases can take time.

Plan before:

• launch events
• migrations
• Black Friday/holiday traffic
• ML campaigns
• DR game days
• regional expansion
• new tenant onboarding

3.6 Quota-as-code

Where possible:

• record desired quota
• check actual quota in CI/CD
• block rollout if quota insufficient
• create quota increase tickets/requests
• include quota in readiness review

3.7 Quota runbook

1. Identify quota code/service/Region.
2. Check current usage.
3. Check forecasted need.
4. Submit increase request.
5. Attach business justification.
6. Track approval.
7. Re-run capacity test.
8. Update dashboard.

4. EC2 Capacity

4.1 EC2 capacity surfaces

Capacity can fail at:

• On-Demand vCPU quota
• instance type availability
• AZ-level availability
• Spot pool availability
• subnet IP availability
• placement group capacity
• GPU scarcity
• launch template/AMI issue
• capacity reservation mismatch
• ASG max capacity
• service-linked role/IAM

4.2 On-Demand Capacity Reservations

EC2 On-Demand Capacity Reservations reserve compute capacity for EC2 instances in a specific Availability Zone for any duration. They are useful when workloads require capacity assurance.

Use when:

• business-critical workload needs guaranteed launch capacity
• DR failover requires capacity
• known event will need fixed capacity
• singleton/cluster cannot tolerate capacity failure
• exact instance type/AZ required
• warm standby must scale quickly

Caution:

• charged at equivalent On-Demand rate whether used or not
• reservation is AZ-specific
• matching attributes matter
• unused reservations are waste
• reservation does not replace Savings Plan economics automatically

4.3 Capacity Reservations vs Savings Plans

Savings Plans:

billing discount for committed spend

Capacity Reservations:

capacity assurance in specific AZ

They solve different problems.

You may use both:

Capacity Reservation guarantees capacity.
Savings Plan discounts eligible usage.

4.4 Capacity Blocks for ML

Capacity Blocks can reserve GPU instances for ML workloads for a future duration. Use when:

• GPU campaign needs assurance
• training schedule fixed
• capacity scarcity risk high
• workload can fit reservation window

Design:

• reserve only what campaign needs
• checkpoint/export before block ends
• align data staging
• test environment before block start
• cost per successful training run

4.5 Capacity Reservation targeting

Instances can target reservations by:

• open reservation matching attributes
• specific reservation
• capacity reservation group

Design for:

• critical fleet
• DR fleet
• batch campaign
• blue/green with reserved capacity
• migration cutover

4.6 Capacity Reservation runbook

1. Forecast required instance type/count/AZ.
2. Check quota.
3. Create reservation.
4. Validate instances match reservation.
5. Monitor utilization.
6. Release after event/campaign if no longer needed.
7. Record cost/utilization.

4.7 Unused reservation waste

Track:

• reservation utilization
• start/end expectation
• owner
• reason
• expiry review
• matching instances

Unused reservations cost like On-Demand capacity.

5. Spot Capacity Engineering

5.1 Spot pool mental model

Spot capacity depends on unused EC2 capacity in specific pools.

A pool is roughly:

instance type + AZ + OS/platform

More pools = better chance of capacity.

5.2 Diversification

Diversify by:

• instance families
• sizes
• AZs
• architectures if app supports
• capacity-optimized allocation
• mixed instance policy
• flexible job sizing

5.3 Interruption-aware design

Required:

• checkpoint
• idempotency
• job retry
• graceful termination
• queue visibility timeout
• partial work commit
• no single critical state on Spot
• On-Demand baseline for critical path

5.4 Mixed capacity

Pattern:

baseline = On-Demand / Reserved / Capacity Reservation
burst = Spot diversified
critical coordinator = On-Demand
workers = Spot

5.5 Spot capacity runbook

If Spot capacity insufficient:

1. Check affected pools.
2. Increase instance type flexibility.
3. Add AZs.
4. Increase On-Demand percentage temporarily.
5. Reduce job parallelism.
6. Use Capacity Reservations for critical demand.
7. Reschedule non-urgent jobs.
8. Review interruption/capacity metrics.

5.6 Spot game day

Terminate/interruption simulate:

• workers receive signal
• checkpoint
• queue message returns safely
• replacement launches from other pools
• job completes within SLO
• cost per successful job measured

6. Auto Scaling Capacity Planning

6.1 ASG capacity settings

Key fields:

• min
• desired
• max
• target tracking
• scaling policies
• cooldown/warmup
• health check
• instance refresh
• mixed instance policy
• capacity rebalance
• warm pool

6.2 Min capacity

Min capacity protects baseline availability.

Set from:

• steady traffic
• redundancy requirement
• AZ failure tolerance
• cold start time
• deployment surge
• maintenance
• DR standby

6.3 Max capacity

Max capacity is a reliability limit.

If max too low, scaling fails.

If max too high, runaway cost risk.

Design:

max = expected peak + failure scenario + buffer

Guard with:

• budgets
• anomaly detection
• rate limits
• queue backpressure
• WAF/rate limiting
• per-tenant quotas

6.4 Warmup time

Capacity is not usable immediately.

Include:

• instance launch time
• bootstrap
• image pull
• application startup
• cache warmup
• target group health check
• JIT/GC warmup
• data preload

Scale early enough.

6.5 Scheduled scaling

Use for predictable events:

• daily peak
• batch windows
• launches
• campaigns
• reporting periods

Scheduled scaling avoids waiting for reactive metrics.

6.6 AZ failure capacity

For N AZs, if one AZ fails, remaining AZs must absorb traffic.

Example with 3 AZs:

normal per-AZ load = 33%
after one AZ loss = 50% each remaining

Need enough capacity in remaining AZs.

6.7 ASG capacity game day

• reduce one AZ capacity
• simulate launch failures
• test max capacity
• test quota breach
• test warm pool
• test mixed instance fallback
• validate SLO

7. Subnet and Network Capacity

7.1 IP exhaustion

Subnets have finite IPs.

Capacity consumers:

• EC2 instances
• ENIs
• Lambda in VPC
• EKS pods/ENIs
• ECS tasks awsvpc
• RDS/ElastiCache/FSx/EFS mount targets
• load balancer nodes
• VPC endpoints
• NAT gateways

7.2 Subnet planning

Design:

• enough IPs per AZ
• separate workload classes
• growth buffer
• blue/green deployments
• failover capacity
• EKS pod density
• interface endpoint count
• DR capacity

7.3 IP exhaustion runbook

1. Identify subnet.
2. List ENIs by owner.
3. Check ASG/EKS/ECS scale event.
4. Remove stale ENIs if safe.
5. Expand to larger subnet/new subnets if architecture supports.
6. Add subnets to ASG/LB/EKS.
7. Update capacity forecast.
8. Add alarm on free IPs.

7.4 NAT and egress capacity/cost

NAT gateways can become bottleneck/cost center.

Capacity/cost mitigations:

• VPC endpoints for AWS services
• per-AZ NAT
• avoid cross-AZ NAT
• reduce unnecessary egress
• cache dependencies
• monitor bytes/packets/errors

7.5 Load balancer capacity

ALB/NLB scale automatically but still has operational factors:

• target health
• per-target capacity
• subnet IPs
• security groups
• TLS termination CPU internal to service
• request surge behavior
• health check accuracy

Monitor target saturation, not only LB health.

8. Storage Capacity Planning

8.1 EBS capacity

Plan:

• volume size
• IOPS/throughput
• filesystem free space
• snapshot retention
• growth rate
• modify-volume limits
• instance EBS bandwidth
• AZ placement

Alert on:

• filesystem full soon
• volume approaching performance limit
• burst balance low
• snapshot RPO breach
• unattached volume sprawl

8.2 EBS modify-volume

EBS volumes can be modified, but modifications are not instant. Filesystem/partition growth also needed.

Capacity plan:

alert before emergency
modify during safe window
validate filesystem growth

8.3 S3 capacity

S3 storage scales, but capacity engineering still matters:

• request rate per prefix/pattern
• small object count
• lifecycle transitions
• replication backlog
• inventory/report scale
• list/query scalability
• KMS request quotas
• cost of growth
• object lock retention

8.4 EFS capacity

EFS storage is elastic, but throughput/cost can be capacity-related.

Plan:

• throughput mode
• PercentIOLimit
• storage class distribution
• lifecycle policy
• client connections
• file count/directory shape
• backup/replication lag
• access point limits

8.5 FSx capacity

FSx capacity can be more explicit:

• storage capacity
• throughput capacity
• provisioned IOPS
• file system family limits
• volume size
• snapshots/clones
• backup storage
• capacity pool tier
• scaling operation timing
• deployment type

Plan changes before peak.

8.6 Backup capacity

Backups consume:

• vault storage
• copy job throughput/time
• restore target capacity
• recovery account quotas
• KMS usage
• network/data transfer

Capacity planning includes:

can we restore fast enough with enough target capacity?

8.7 Forecasting

Use:

days_to_full = free_capacity / daily_growth
days_to_quota = (quota - current_usage) / daily_growth
peak_required = baseline + seasonal_peak + failure_buffer

For storage:

• daily growth
• p95 growth
• seasonal events
• cleanup jobs
• retention changes
• business forecast
• tenant onboarding

9. S3 Request Capacity and Layout

9.1 S3 performance scales with prefixes

S3 supports high request rates per prefix and can scale as request rate increases. Workloads that exceed baseline per-prefix request patterns should distribute load and handle gradual scaling behavior.

Design:

• distribute hot keys/prefixes
• avoid single hot object/key where possible
• use retries with backoff
• use multipart upload
• use byte-range GET for large objects
• use CloudFront/cache where read-heavy
• monitor 503 Slow Down

9.2 Hot key problem

One key repeatedly read/written can bottleneck application semantics.

Mitigate:

• cache
• versioned immutable objects
• sharded manifests
• object per partition
• database/state service for mutable counters
• CloudFront for read fanout

9.3 LIST capacity

LIST is not a database query.

Avoid:

• listing huge prefixes on request path
• using S3 prefix as queue
• scanning entire bucket for state

Use:

• catalog/database
• inventory
• manifest
• events/queues
• partitioned prefixes

9.4 KMS as request capacity

SSE-KMS may introduce KMS quotas/throttling.

Monitor:

• KMS request rate
• throttles
• AccessDenied
• key policy changes
• bucket key usage where appropriate

10. DR Capacity

10.1 DR capacity is often forgotten

Backup exists, but DR cannot launch because:

• vCPU quota low
• no EC2 capacity
• subnets too small
• KMS missing
• AMI not copied
• EFS/FSx restore quota issue
• database class unavailable
• GPU unavailable
• Service Quotas not raised in DR Region

10.2 DR capacity inventory

drRegion: ap-southeast-3
compute:
  requiredVcpu:
  quota:
  capacityReservation:
  imagesCopied:
network:
  subnetFreeIps:
  endpoints:
storage:
  backupCopies:
  restoreTargets:
  kmsKeys:
  fsxQuota:
  efsQuota:
traffic:
  route53:
  arc:

10.3 DR capacity tests

Test:

• launch minimum stack
• launch full stack
• restore data
• scale standby
• handle one-AZ loss in DR
• run user journey
• failback

10.4 Capacity Reservations for DR

Use when:

• DR RTO low
• capacity scarce
• exact instance type needed
• regulatory/business requirement
• warm standby scale-up must be assured

Balance cost vs risk.

10.5 Backup restore capacity

Restoring creates resources:

• EBS volumes
• EC2 instances
• EFS file systems
• FSx file systems
• S3 objects/buckets
• databases
• KMS operations

Ensure quotas and target environment can host restore.

11. Capacity Forecasting System

11.1 Inputs

Collect:

• historical utilization
• business forecast
• traffic forecast
• tenant onboarding
• data growth
• marketing launches
• batch schedules
• failure scenarios
• deployment surge
• DR needs
• cleanup/lifecycle changes

11.2 Forecast by resource

Compute:

required_instances = peak_rps / per_instance_safe_rps

Storage:

projected_storage = current + growth_rate * days - cleanup

Backup:

backup_storage = protected_data * retention_multiplier * change_rate

Network:

required_bandwidth = peak_bytes_per_sec + replication + backup

11.3 Safe capacity per instance

Do not use theoretical max.

Measure:

safe_rps_per_instance = throughput at p95 latency within SLO with CPU/memory/network headroom

11.4 Forecast intervals

Use:

• daily for fast-growing storage
• weekly for compute rightsizing
• monthly for budgets/commitments
• quarterly for quotas/DR/capacity reservations
• pre-event for launches/campaigns

11.5 Forecast uncertainty

Use ranges:

p50 forecast
p90 forecast
p99 stress scenario

Plan headroom for p90/p99 according to criticality.

12. Capacity Runbooks

12.1 Insufficient EC2 capacity

1. Identify instance type/AZ.
2. Check quotas.
3. Try alternate AZs.
4. Use mixed instance policy.
5. Use smaller/larger compatible sizes.
6. Use On-Demand instead of Spot.
7. Use Capacity Reservation if planned.
8. Reduce non-critical demand.
9. Escalate to AWS support/account team for critical planned events.

12.2 Service quota breach

1. Identify quota.
2. Stop rollout if capacity insufficient.
3. Submit quota increase.
4. Reduce demand or use alternate Region/account.
5. Update capacity dashboard.
6. Add pre-deployment quota check.

12.3 ASG max capacity hit

1. Check user impact.
2. Check if max intentionally low.
3. Check quota/subnet capacity.
4. Increase max if safe.
5. Scale out.
6. Add cost guardrail.
7. Review autoscaling policy.

12.4 Subnet IP exhaustion

1. Identify subnet/AZ.
2. Identify ENI consumers.
3. Remove stale ENIs if safe.
4. Add larger/new subnet.
5. Update ASG/LB/EKS/ECS.
6. Rebalance.
7. Update IP forecast.

12.5 EFS/FSx throughput saturation

1. Confirm app latency.
2. Check throughput metrics.
3. Check client count/file pattern.
4. Increase provisioned/elastic throughput if appropriate.
5. Separate noisy workload.
6. Optimize file layout.
7. Review cost.

12.6 S3 503 Slow Down

1. Identify prefix/key pattern.
2. Check request rate.
3. Add exponential backoff/retries.
4. Distribute prefixes/keys.
5. Use CloudFront/cache for read hot spots.
6. Use multipart/range strategies.
7. Monitor recovery.

12.7 Backup copy lag

1. Check backup/copy jobs.
2. Check destination vault/KMS.
3. Check data size spike.
4. Check quotas/service health.
5. Alert RPO breach.
6. Adjust schedule/copy policy.
7. Test restore from latest copy.

13. Capacity Game Days

Scenario 1 — Launch failure due quota

Expected:

• deployment precheck detects insufficient vCPU
• rollout stops
• quota request submitted
• fallback instance types evaluated

Scenario 2 — AZ evacuation

Expected:

• remaining AZs absorb traffic
• subnet IPs sufficient
• ASG scales
• SLO remains acceptable

Scenario 3 — Spot pool loss

Expected:

• workloads move to other pools
• On-Demand fallback activates
• checkpoint/retry works
• cost impact measured

Scenario 4 — DR full-scale launch

Expected:

• DR Region quota sufficient
• capacity reservation used if needed
• stack scales
• data restored/promoted
• traffic test passes

Scenario 5 — S3 request hot prefix

Expected:

• 503/retry signal visible
• clients back off
• key distribution fix deployed
• app recovers

Scenario 6 — File system throughput saturation

Expected:

• dashboard identifies FSx/EFS bottleneck
• scale/tune action taken
• noisy neighbor isolated

14. Dashboards

14.1 Quota dashboard

Show:

• quota limit
• current usage
• projected peak
• utilization %
• days to breach
• requested increases
• DR requirement
• owner

14.2 Compute capacity dashboard

Show:

• ASG desired/min/max
• in-service capacity
• launch failures
• Spot interruption rate
• instance type distribution
• Capacity Reservation utilization
• Savings Plans coverage separately
• subnet IP capacity
• warm pool size
• AZ distribution

14.3 Storage capacity dashboard

Show:

• EBS volume free space
• EBS performance utilization
• S3 object/byte growth
• S3 request rate/error
• EFS throughput/storage
• FSx capacity/throughput
• backup storage/copy lag
• snapshot growth
• days to full/quota

14.4 DR capacity dashboard

Show:

• recovery Region quotas
• AMI/snapshot copies
• KMS readiness
• standby capacity
• subnet free IPs
• Capacity Reservations
• last full-scale test
• latest RPO/RTO actual

14.5 Campaign dashboard

For events/ML launches:

• reserved capacity
• current usage
• burn rate
• remaining capacity window
• job progress
• checkpoint status
• cost per successful job
• fallback capacity

15. Governance

15.1 Capacity review cadence

Weekly:

• fast-growing services
• near-quota resources
• launch failures
• capacity anomalies

Monthly:

• quota utilization
• reservation utilization
• rightsizing and capacity balance
• storage growth forecasts

Quarterly:

• DR capacity test
• major quota review
• launch/campaign readiness
• savings/commitment vs capacity assurance review

15.2 Pre-launch capacity review

Before major launch:

• traffic forecast
• compute capacity
• quotas
• subnet IPs
• database/storage capacity
• S3/KMS request capacity
• backup/replication capacity
• rollback capacity
• budgets
• alarms
• runbooks

15.3 Capacity exception

If service chooses low headroom:

exception:
  workload:
  risk:
  reason:
  owner:
  expiry:
  mitigation:
  acceptedBy:

15.4 Capacity owner

Every capacity surface needs owner:

• service team
• platform team
• storage team
• network team
• security/recovery team
• finance for reservation cost

16. Terraform/IaC Concepts

16.1 ASG max capacity as explicit design

resource "aws_autoscaling_group" "api" {
  min_size         = 6
  desired_capacity = 9
  max_size         = 30

  vpc_zone_identifier = var.private_subnet_ids

  mixed_instances_policy {
    instances_distribution {
      on_demand_base_capacity                  = 6
      on_demand_percentage_above_base_capacity = 50
      spot_allocation_strategy                 = "capacity-optimized"
    }

    launch_template {
      launch_template_specification {
        launch_template_id = aws_launch_template.api.id
        version            = "$Latest"
      }

      override {
        instance_type = "m7i.large"
      }

      override {
        instance_type = "m7a.large"
      }

      override {
        instance_type = "m6i.large"
      }
    }
  }
}

16.2 Capacity Reservation concept

aws ec2 create-capacity-reservation \
  --instance-type m7i.large \
  --instance-platform Linux/UNIX \
  --availability-zone ap-southeast-1a \
  --instance-count 20 \
  --instance-match-criteria open

Validate current CLI fields before use.

16.3 Service quota check concept

aws service-quotas get-service-quota \
  --service-code ec2 \
  --quota-code L-1216C47A

Quota codes vary; store them in runbook/config.

16.4 Pre-deploy capacity check pseudocode

required_vcpu = forecast_peak_instances * vcpu_per_instance
quota = get_service_quota("ec2", "running_on_demand_standard_vcpu")
current = get_current_usage("ec2_vcpu")

if current + required_vcpu > quota * 0.8:
    fail_deploy("Insufficient vCPU quota headroom")

16.5 Expiry tag for reservations

CapacityReservation:
  Owner: payments-platform
  Reason: black-friday-2026
  ExpiresAt: 2026-12-05
  ReviewAfter: 2026-11-30

17. Anti-Patterns

17.1 Quota discovered during incident

Fix:

• quota dashboard
• pre-deploy checks
• capacity game days

17.2 Single instance type dependency

Fix:

• mixed instance policy
• compatibility testing
• family diversification

17.3 Capacity reservation as cost optimization

Capacity Reservation is capacity assurance, not discount. Use Savings Plans/RI separately for discount.

17.4 Spot without checkpointing

Spot can become expensive if interruptions force full restart.

17.5 Subnet IPs ignored

Scaling fails even when EC2 quota exists.

17.6 DR Region untested

Backup copies exist but no launch quota/capacity.

17.7 Storage scale at emergency time

Some storage changes take time or have limits. Forecast earlier.

17.8 Headroom without owner

Unused capacity/reservations become waste if not owned/reviewed.

18. Mini Case Study — Black Friday API Fleet

18.1 Requirement

API normally runs 30 instances.

Forecast peak:

150 instances

Needs:

• 3-AZ availability
• one-AZ failure tolerance
• p95 latency under 250ms
• no Spot-only critical path

18.2 Capacity plan

• ASG max 240
• subnet IPs support 300+
• On-Demand quota raised
• mixed instance policy across 6 compatible types
• scheduled scaling before event
• partial Capacity Reservation for critical baseline
• Spot only for async workers
• pre-event load test
• cost budget with forecast alert

18.3 Game day

• simulate one AZ loss
• simulate launch failures for one instance family
• validate remaining pools
• check route/LB health
• measure p95

18.4 Invariant

Peak capacity plan includes failure capacity, not just forecast demand.

19. Mini Case Study — DR Restore Cannot Launch

19.1 Incident

Production Region impaired.

Backups exist in DR Region.

Restore fails because DR account has low EC2 vCPU quota and no subnet IP capacity.

19.2 Root cause

DR plan tested backup copy but not full capacity launch.

19.3 Fix

• DR quota dashboard
• quarterly full-scale restore
• minimum DR capacity reservation
• pre-created larger subnets
• AMI/KMS/secrets verified
• recovery runbook includes quota check

19.4 Invariant

A backup in another Region is not DR unless recovery capacity exists there.

20. Summary

Capacity engineering is the reliability discipline of ensuring enough usable resource is available at the right time, shape, place, and account boundary.

Key principles:

1. Quota is not capacity.
2. Capacity must match instance type, AZ, Region, and workload shape.
3. Headroom is time-to-exhaustion, not just percent free.
4. On-Demand Capacity Reservations provide capacity assurance but cost when unused.
5. Savings Plans/RI are billing discounts, not capacity guarantees.
6. Spot requires diversification and interruption-aware design.
7. Auto Scaling max capacity is a reliability limit.
8. Subnet IPs are capacity.
9. Storage capacity includes throughput, IOPS, file count, request pattern, and restore capacity.
10. DR requires capacity in the recovery site.
11. Forecasts and game days catch capacity failures before incidents.

The core rule:

Elastic cloud capacity still needs engineering. You must forecast, diversify, reserve, test, and govern it.

Next, Part 075 starts the final implementation synthesis section: building production-ready reference architectures that combine compute, storage, backup, observability, cost, security, and operations into end-to-end workload blueprints.

References

• AWS Service Quotas User Guide — What is Service Quotas?: https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html
• Amazon EC2 User Guide — On-Demand Capacity Reservations and Capacity Blocks for ML: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/capacity-reservation-overview.html
• Amazon EC2 User Guide — Reserve compute capacity with On-Demand Capacity Reservations: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-capacity-reservations.html
• Amazon EC2 User Guide — Capacity Reservation pricing and billing: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/capacity-reservations-pricing-billing.html
• Amazon EC2 User Guide — Instance purchasing options: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-purchasing-options.html
• Amazon EC2 Auto Scaling User Guide — Mixed instances groups: https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-mixed-instances-groups.html
• Amazon EC2 Auto Scaling User Guide — Spot allocation strategies: https://docs.aws.amazon.com/autoscaling/ec2/userguide/allocation-strategies.html
• Amazon EC2 Auto Scaling User Guide — Scheduled scaling: https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-scheduled-scaling.html
• Amazon S3 User Guide — Performance guidelines: https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance-guidelines.html
• AWS Well-Architected Reliability Pillar — Manage service quotas and constraints: https://docs.aws.amazon.com/wellarchitected/latest/reliability-pillar/rel_manage_service_quotas.html