Part 001: compute-storage mental model
Part 002: workload taxonomy and decision map
Part 003: placement, time, state, failure domains
Part 004: cloud capacity as control system
Part 005: storage as contract, not disk
Part 006: AWS compute-storage service selection
Part 007: production invariants

EC2 = server
S3 = object storage
EBS = disk
EFS = shared file system
Lambda = function
ECS/EKS = containers
Batch = batch jobs

Where does request enter?
Where does synchronous work stop?
Where does asynchronous work begin?
Where is durable state written?
Where is temporary state allowed?
Where are failure domains?
Where can retries duplicate work?
Where is capacity controlled?
Where is recovery proven?
Where does cost grow?

1. Data path
2. Control path
3. Recovery path

client -> compute -> storage -> response/event/output

deployment -> scaling -> config -> provisioning -> scheduling

detect -> isolate -> degrade -> replace -> restore -> reconcile

- Apakah request synchronous atau bisa asynchronous?
- Apakah upload langsung ke compute atau langsung ke S3?
- Apakah request harus menunggu durable write?
- Apakah retry client bisa menduplikasi side effect?
- Apakah ingress bisa mengurangi traffic ke AZ bermasalah?

- EC2 Auto Scaling Group,
- ECS service,
- EKS Deployment,
- Lambda function,
- application runtime behind ALB/API Gateway.

- bounded latency,
- safe retry,
- no hidden durable state,
- graceful shutdown,
- health check meaningful,
- scaling metric sesuai bottleneck.

- menyerap burst,
- melindungi downstream storage,
- memberi retry contract,
- memberi backpressure signal,
- memisahkan lifecycle compute API dan worker,
- membuat partial failure lebih terkendali.

- idempotency,
- deduplication,
- poison message handling,
- queue age monitoring,
- replay/reconciliation,
- ordering decision.

- ECS worker service,
- EKS deployment/job,
- EC2 ASG worker,
- Lambda event consumer,
- AWS Batch job.

- input idempotent,
- checkpoint jelas,
- interruption safe,
- concurrency controlled,
- output deterministic,
- side effect auditable.

- immutable blobs,
- upload/download,
- artifacts,
- data lake layout,
- backup/export,
- event-driven processing,
- large object handling,
- cross-service durable handoff.

- object key adalah API contract,
- metadata bukan hanya filename,
- versioning/lifecycle dirancang,
- object write idempotent,
- multipart cleanup ada,
- listing tidak menjadi critical transaction path,
- small-file problem dipahami untuk analytics.

- OS/root volume,
- database volume,
- low-latency mutable filesystem,
- single-node stateful workload,
- WAL/log/data separation,
- workloads requiring block device semantics.

- zonal coupling jelas,
- snapshot/restore diuji,
- filesystem consistency dipahami,
- volume performance sesuai workload,
- attachment lifecycle jelas,
- failover bukan asumsi otomatis.

- shared filesystem,
- legacy apps expecting POSIX/SMB/NFS-like access,
- shared model/data directory,
- media processing workspace,
- HPC shared path,
- multi-compute read/write file semantics.

- metadata operation cost dipahami,
- locking behavior diuji,
- throughput mode cocok,
- mount target/path tersedia di failure domain yang tepat,
- permission model tidak drift,
- aplikasi tidak menganggap local disk latency.

- S3: versioning, Object Lock, replication, lifecycle.
- EBS: snapshots, AMI, application-consistent backup if needed.
- EFS/FSx: backup policy and restore drill.
- Compute: AMI/container image/IaC versioning.

- restore target di mana?
- apakah key encryption tersedia?
- apakah dependency identity/network/config tersedia?
- apakah app bisa membaca restored data tanpa migration manual?
- apakah RTO/RPO terukur?

- file upload,
- document processing,
- media pipeline,
- user-generated content,
- artifact service.

- Lambda jika processing singkat dan event-driven.
- ECS/Fargate jika processing containerized dan durasi lebih fleksibel.
- Batch jika processing heavy, scheduled, parallel, atau resource-intensive.

- S3 sebagai source of truth object.
- Metadata store untuk status/index/reference.
- Optional EFS/FSx jika processing membutuhkan shared working directory.

- S3 event/queue delivery can retry.
- Worker must be idempotent.
- Object key/version should identify input.
- Metadata update must handle duplicate processing.

- existing app server,
- high-control runtime,
- custom OS tuning,
- predictable service behind load balancer.

- instance replacement safe,
- no durable local user data,
- AMI/Launch Template versioned,
- health check protects traffic,
- lifecycle hook handles draining,
- scale-in does not interrupt critical work.

- single-node database-like workload,
- vendor software,
- block-device requirement,
- controlled stateful appliance.

- zonal state,
- failover may be manual or semi-automated,
- snapshot may be crash-consistent only,
- restore time may exceed expectation,
- kernel/filesystem tuning matters.

- containerized stateful application,
- platform standardization,
- Kubernetes/ECS operational model,
- workload requiring mounted storage.

- Is the volume single-AZ or shared/multi-AZ?
- What happens when scheduler moves workload?
- Can the workload tolerate detach/attach delay?
- Is file locking required?
- Is the CSI/storage driver part of the operational dependency chain?

- object-created processing,
- lightweight transformations,
- notifications,
- metadata extraction,
- short event handlers.

- handler idempotent,
- timeout bounded,
- memory and ephemeral storage sized,
- concurrency protects downstream,
- DLQ/retry behavior understood,
- duplicate event safe.

- large-scale ETL,
- ML preprocessing,
- simulation,
- rendering,
- scientific/HPC jobs,
- heavy parallel workers.

- object store vs shared file vs scratch disk,
- checkpoint interval,
- Spot interruption tolerance,
- queue priority,
- data locality,
- output commit protocol,
- cost per job.

Do not make your only emergency path depend on just-in-time control-plane success.

Workload: document ingestion and extraction
Runtime: sync API + async workers
Duration: API < 200ms, worker 30s-10m
State: S3 raw object, S3 processed object, metadata DB
Mutability: object immutable, metadata mutable
Placement: API multi-AZ, S3 regional, workers multi-AZ
Scaling: API request count, worker queue age
Recovery: replay queue/object, restore metadata DB
Cost: object storage + worker compute per document
Operation: ECS/Fargate initially, Batch later for heavy extraction

# Shape only, not complete copy-paste code.

module "compute_service" {
  source = "./modules/compute-service"

  name              = "document-api"
  runtime           = "ecs-fargate"
  desired_capacity  = 3
  min_capacity      = 3
  max_capacity      = 30
  availability_zones = ["az-a", "az-b", "az-c"]

  health_check = {
    path = "/health/ready"
  }

  scaling = {
    metric = "request_count_per_target"
    target = 100
  }
}

module "object_store" {
  source = "./modules/object-store"

  bucket_name = "document-raw-prod"
  versioning  = true

  lifecycle_rules = [
    {
      prefix = "raw/"
      transition_after_days = 30
      expire_after_days     = 2555
    }
  ]
}

module "worker" {
  source = "./modules/worker-service"

  name          = "document-worker"
  runtime       = "ecs-fargate"
  queue_name    = "document-events"
  scale_metric  = "queue_age"
  max_capacity  = 100

  interruption_safe = true
}

module "recovery" {
  source = "./modules/recovery-contract"

  restore_test_schedule = "monthly"
  rpo_minutes           = 15
  rto_minutes           = 60
}

- runtime contract,
- state contract,
- capacity contract,
- recovery contract,
- observability contract.

Signals:
- rising 5xx,
- target unhealthy,
- queue age rising,
- pod/task restarts,
- CPU/memory saturation,
- throttling.

Actions:
- identify affected AZ/service/version,
- reduce traffic if needed,
- pause deployment,
- scale out if safe,
- rollback if correlated with release,
- drain bad nodes/tasks,
- protect downstream concurrency.

Signals:
- EBS latency/queue depth,
- EFS/FSx client errors,
- S3 4xx/5xx spike,
- object write failures,
- disk full,
- metadata operation latency,
- backup failure.

Actions:
- classify source of truth vs cache/scratch,
- stop unsafe writes if needed,
- enable read-only/degraded mode,
- retry with backoff,
- fail over if designed,
- restore/replay if necessary,
- reconcile duplicate/partial writes.

Signals:
- scaling max reached,
- quota errors,
- Spot interruption spike,
- no capacity in selected instance type/AZ,
- queue backlog growing faster than drain rate.

Actions:
- shed non-critical load,
- expand instance diversification,
- use On-Demand fallback,
- raise max capacity if quota exists,
- lower per-worker concurrency if downstream saturated,
- activate degraded business mode.

Signals:
- backup age exceeds RPO,
- restore test failed,
- snapshot lifecycle unexpected,
- KMS/key access failure,
- cross-account restore blocked.

Actions:
- restore to isolated environment,
- validate application can read restored data,
- compare record/object counts,
- run synthetic transaction,
- document actual RTO,
- fix dependency drift.

Move non-immediate work behind async boundary.

Move durable data to explicit storage contract.

Use file storage for file semantics, not coordination/transaction semantics.

Use metadata/index/manifest patterns; avoid hidden transactional assumptions.

Make StorageClass, PVC binding, topology, and failure domain explicit.

Use queues, concurrency limits, throttling, and admission control.

Restore drill with measured RTO/RPO.

- user-facing API must stay responsive,
- documents must be durable,
- evidence retention must be explicit,
- generated documents must be reproducible/auditable,
- processing can be asynchronous,
- workloads may spike near enforcement deadlines,
- recovery must be defensible.

- Officer upload should not depend on worker availability.
- Evidence object key must be deterministic or tied to immutable document id.
- Worker duplicate processing must not create duplicate authoritative documents.
- Audit event must be emitted around durable state transition, not best-effort afterthought.
- Restore test must validate both bytes and metadata relationships.
- Degraded mode should allow case read access even if document generation is delayed.

What is the execution contract?
What is the state contract?
Where is the failure domain?
Where is the async boundary?
Where can work duplicate?
Where does capacity saturate?
Where is recovery proven?
Where does cost grow?