Capstone Production Readiness Scorecard for Enterprise Data Models

1. Core Idea

Setelah membahas seluruh aspek enterprise data modelling, kita butuh cara objektif untuk menilai:

Apakah data model ini siap masuk production?

Scorecard ini menggabungkan semua bagian seri menjadi satu assessment framework.

Mental model:

Production readiness is not a feeling. It is evidence that the model is correct, owned, constrained, versioned, secure, observable, testable, recoverable, and operable under failure.

Scorecard ini dapat digunakan untuk:

• design review,
• PR review,
• architecture review,
• release readiness,
• incident prevention,
• migration planning,
• onboarding assessment,
• team maturity evaluation.

2. Scoring Scale

Gunakan skor 0–4.

Guideline:

• critical financial/security/lifecycle model should score 3+,
• production-critical cross-service flow should score 4 in observability/reconciliation,
• no release if any hard gate is 0 for critical area.

3. Assessment Categories

Categories:

1. Domain semantics.
2. Ownership/source-of-truth.
3. Identity/reference/lineage.
4. Lifecycle/state machine.
5. Invariants/constraints.
6. API/event/file contracts.
7. Physical persistence/performance.
8. Concurrency/idempotency.
9. Migration/schema evolution.
10. Security/privacy/retention.
11. Cache/search/projection.
12. Observability/supportability.
13. Data quality/reconciliation.
14. Testing/synthetic data.
15. Operations/repair/DR.
16. Documentation/governance.

Each category has scoring prompts.

4. Category 1 — Domain Semantics

Score questions:

• Is the business concept clearly defined?
• Are names precise?
• Are similar concepts distinguished?
• Are null semantics defined?
• Are units/currency/timezone defined?
• Is current vs snapshot vs history clear?
• Are derived fields identified?
• Is domain glossary updated?

Score 0 if:

No one can explain what the field/entity means consistently.

Score 4 if:

Definitions are documented, reviewed, discoverable, and reflected in code/contracts/tests.

Example risk:

order.status means fulfillment in one report and lifecycle in another.

5. Category 2 — Ownership and Source of Truth

Score questions:

• Which service/team owns the data?
• Who can mutate it?
• Is external authority identified?
• Are projections distinguished from source?
• Are duplicate sources avoided?
• Is conflict resolution defined?
• Is owner metadata documented?
• Is support escalation clear?

Score 0 if:

Multiple services write same truth with no owner.

Score 4 if:

Ownership is documented, enforced by API/repository permissions, and visible in metadata/runbooks.

6. Category 3 — Identity, Reference, and Lineage

Score questions:

• Are technical IDs, business numbers, external IDs separated?
• Are source entity/version references preserved?
• Are tenant/environment scopes included?
• Are external mappings governed?
• Is lineage from source to derived objects available?
• Can support trace end-to-end?
• Is identifier history preserved?

Score 0 if:

Support cannot trace invoice/order/product back to source quote or external system.

Score 4 if:

Lineage and external references are persisted, queryable, audited, and included in support timeline.

7. Category 4 — Lifecycle and State Machine

Score questions:

• Are states documented?
• Are transitions defined?
• Are terminal states clear?
• Are cancellation/amend/retry/fallout paths defined?
• Are status dimensions separated?
• Are status histories captured?
• Are transition commands explicit?
• Are stale/out-of-order transitions guarded?

Score 0 if:

Status can be patched freely with no transition rules.

Score 4 if:

State machine is command-driven, guarded, tested, audited, observable, and reconciled.

8. Category 5 — Invariants and Constraints

Score questions:

• What must always be true?
• Are local invariants enforced by domain and DB?
• Are cross-system invariants reconciled?
• Are unique constraints aligned with business rules?
• Are temporal/effective-dated rules enforced?
• Are validation errors structured?
• Are overrides governed?

Score 0 if:

Critical invariants exist only in developer memory.

Score 4 if:

Invariants are enforced in code/DB/tests and monitored through reconciliation where distributed.

9. Category 6 — API, Event, and File Contracts

Score questions:

• Are contracts versioned?
• Are consumers known?
• Are compatibility rules defined?
• Are samples provided?
• Are contract tests run?
• Are enum/date/money semantics documented?
• Are sensitive fields reviewed?
• Is deprecation process defined?
• Are file/feed contracts explicit?

Score 0 if:

Payload changes can happen without consumer impact review.

Score 4 if:

Contracts are registered, versioned, tested, usage-tracked, and governed through CI/review.

10. Category 7 — Physical Persistence and Performance

Score questions:

• Is table grain clear?
• Are types appropriate?
• Are indexes aligned with queries?
• Are constraints defined?
• Is tenant indexing considered?
• Is JSONB used intentionally?
• Is history/audit growth considered?
• Is partitioning/archival considered for large tables?
• Are query plans tested with realistic data?

Score 0 if:

Schema exists but no query/index/volume review.

Score 4 if:

Physical design is benchmarked/observed, with index strategy, retention, and growth plan.

11. Category 8 — Concurrency and Idempotency

Score questions:

• Is optimistic/pessimistic locking needed?
• Is expected version used?
• Are state transitions atomic?
• Are create/convert/charge operations idempotent?
• Are duplicate events/callbacks handled?
• Are worker queues claimed safely?
• Are stale events ignored?
• Are deadlocks considered?

Score 0 if:

Retry can create duplicate order/charge.

Score 4 if:

Concurrency hazards are tested with real persistence and observable through conflict/retry metrics.

12. Category 9 — Migration and Schema Evolution

Score questions:

• Is change additive or breaking?
• Is expand/contract plan defined?
• Is backfill idempotent?
• Is migration progress tracked?
• Are exceptions captured?
• Are old/new app versions compatible?
• Is rollback/forward-fix defined?
• Are validation/shadow compare queries ready?
• Is production volume tested?

Score 0 if:

Migration is a one-shot unbounded update with no validation.

Score 4 if:

Migration is staged, idempotent, observable, tested at scale, and has repair/rollback plan.

13. Category 10 — Security, Privacy, and Retention

Score questions:

• Is classification assigned?
• Is field-level access defined?
• Is masking required?
• Is encryption/tokenization needed?
• Are events/cache/search/reports safe?
• Is audit access captured?
• Is retention/purge/legal hold defined?
• Are derived copies included in privacy workflow?
• Are lower environments safe?

Score 0 if:

Sensitive fields are copied without classification or access rules.

Score 4 if:

Sensitivity controls are enforced, tested, audited, and propagated to derived stores.

14. Category 11 — Cache, Search, and Projection

Score questions:

• Is source-of-truth clear?
• Are cache keys tenant/scope/version-aware?
• Is TTL/invalidation defined?
• Is stale data acceptable?
• Does projection store source version?
• Can search/projection be rebuilt?
• Does purge/anonymization propagate?
• Is projection lag monitored?

Score 0 if:

Cache/search can serve wrong or cross-tenant data with no detection.

Score 4 if:

Derived stores are versioned, invalidated, rebuilt, secured, and monitored for lag/drift.

15. Category 12 — Observability and Supportability

Score questions:

• Is correlation ID propagated?
• Are business keys searchable?
• Is audit/status history available?
• Are outbox/inbox/integration states visible?
• Is support timeline available?
• Are structured error/retry fields stored?
• Are owner/runbook links available?
• Can incident timeline be reconstructed?

Score 0 if:

Debugging requires guessing from scattered logs.

Score 4 if:

Support can trace full flow from business number/correlation ID across services and derived stores.

16. Category 13 — Data Quality and Reconciliation

Score questions:

• Are DQ rules defined?
• Are reconciliation rules defined?
• Is severity/owner/SLA assigned?
• Are results persisted?
• Are alerts/dashboards available?
• Is repair workflow linked?
• Are false positives/suppressions governed?
• Are reconciliation tests available?

Score 0 if:

Cross-system drift is discovered only by customers.

Score 4 if:

Drift is detected, owned, alerted, repairable, verified, and trendable.

17. Category 14 — Testing and Synthetic Data

Score questions:

• Are fixtures/golden scenarios defined?
• Are edge cases tested?
• Are DB constraints tested with real DB?
• Are contract tests run?
• Are idempotency/concurrency tests included?
• Are migration tests realistic?
• Are reconciliation tests included?
• Is test data privacy-safe?
• Are synthetic/volume datasets available?

Score 0 if:

Only happy-path unit tests exist.

Score 4 if:

Tests cover domain, DB, event, migration, security, reconciliation, and production-like volume.

18. Category 15 — Operations, Repair, and DR

Score questions:

• Is manual repair workflow defined?
• Is break-glass controlled?
• Are before/after snapshots captured?
• Is verification required after repair?
• Are backup/restore implications known?
• Are projections/caches rebuilt after recovery?
• Is external reconciliation required after restore?
• Are runbooks available?
• Are operational commands idempotent?

Score 0 if:

Production repair is ad-hoc SQL with no audit.

Score 4 if:

Operational repair, recovery, and verification are controlled, audited, and rehearsed.

19. Category 16 — Documentation and Governance

Score questions:

• Is ADR needed and written?
• Is data dictionary updated?
• Is glossary updated?
• Is owner documented?
• Is contract registry updated?
• Is lineage/usage updated?
• Is review approval captured?
• Is internal verification checklist completed?
• Is post-release verification planned?

Score 0 if:

Design decision exists only in a PR comment or chat memory.

Score 4 if:

Decision is documented, discoverable, reviewed, and linked to metadata/contracts/tests/runbooks.

20. Hard Gates

For critical production data, block release if any hard gate fails.

Hard gates:

• no source-of-truth defined,
• no tenant/security boundary for tenant-owned sensitive data,
• no idempotency for retryable create/financial operation,
• no migration/backfill validation for large data change,
• no contract review for breaking API/event/file change,
• no privacy classification for PII/restricted field,
• no audit for financial/manual repair change,
• no rollback/forward-fix for risky migration,
• no reconciliation for known cross-system financial/customer-impacting drift,
• no test for known incident reproduction.

Hard gates are not optional.

21. Scorecard Table Template

Use:

| Category | Score | Evidence | Gaps | Owner | Due Date |
|---|---:|---|---|---|---|
| Domain semantics |  |  |  |  |  |
| Ownership/source-of-truth |  |  |  |  |  |
| Identity/reference/lineage |  |  |  |  |  |
| Lifecycle/state machine |  |  |  |  |  |
| Invariants/constraints |  |  |  |  |  |
| Contracts |  |  |  |  |  |
| Persistence/performance |  |  |  |  |  |
| Concurrency/idempotency |  |  |  |  |  |
| Migration/evolution |  |  |  |  |  |
| Security/privacy/retention |  |  |  |  |  |
| Cache/search/projection |  |  |  |  |  |
| Observability/support |  |  |  |  |  |
| DQ/reconciliation |  |  |  |  |  |
| Testing/data |  |  |  |  |  |
| Operations/DR |  |  |  |  |  |
| Governance/docs |  |  |  |  |  |

Evidence matters more than score opinion.

22. Readiness Levels

Interpret total readiness:

Suggested:

Any hard gate fail -> Red
Average score < 2.5 -> Red/Amber
Average score 2.5–3.2 -> Amber
Average score 3.2–3.7 -> Green
Average score > 3.7 -> Gold

Adjust for team policy and risk.

23. Example Assessment — Quote Acceptance

Categories requiring high score:

• domain semantics,
• lifecycle,
• invariants,
• idempotency,
• contracts,
• audit,
• events,
• migration if changing existing model,
• tests.

Hard gates:

• accepted quote immutability,
• approval applies to version,
• one order per accepted quote version,
• quote accepted event transactional with state change,
• conflict handling for stale version.

If quote acceptance is weak, downstream order/billing correctness is weak.

24. Example Assessment — Billing Charge Activation

High-risk categories:

• financial semantics,
• product/charge linkage,
• billing account source,
• effective date,
• idempotency,
• external billing mapping,
• reconciliation,
• audit,
• repair,
• security.

Hard gates:

• no duplicate active charge,
• charge source traceable,
• external billing status tracked,
• active product without charge reconciliation,
• issued invoice correction policy.

Billing defects carry customer and financial risk.

25. Example Assessment — Search Projection

Important categories:

• source-of-truth,
• projection version,
• cache/search security,
• tenant isolation,
• freshness,
• rebuild,
• purge propagation,
• observability.

Hard gates:

• search query must filter tenant/scope server-side,
• projection must not overwrite newer source with stale event,
• PII purge must remove/anonymize index document,
• search not used as command source-of-truth unless designed.

26. Evidence Examples

Good evidence:

• ADR link,
• schema migration,
• OpenAPI/AsyncAPI diff,
• event schema compatibility report,
• DB constraint DDL,
• query plan,
• test report,
• migration dry-run result,
• reconciliation dashboard,
• runbook link,
• support timeline screenshot/reference,
• security review approval,
• data dictionary entry,
• incident reproduction test.

Weak evidence:

• "We discussed it."
• "Should be fine."
• "Frontend prevents it."
• "No one will retry."
• "This is temporary."
• "We can fix manually."

27. Risk-Based Depth

Not all changes need full scorecard.

Low-risk:

• internal non-sensitive optional metadata,
• small index addition,
• documentation-only update.

Medium-risk:

• new read model,
• new API optional field,
• new reference code,
• new background job.

High-risk:

• lifecycle/status change,
• billing/financial field,
• customer-visible contract,
• migration/backfill,
• PII/security field,
• cross-service source-of-truth,
• product/order/charge creation logic.

Use full scorecard for high-risk changes.

28. Using Scorecard in PR Review

PR comment pattern:

I scored this as Amber because lifecycle and contract are clear, but migration validation and reconciliation are not yet addressed. Before merge, please add:
1. Backfill validation query.
2. Data quality rule for active product without charge.
3. Contract sample for new event field.

Scorecard should help focus review, not shame authors.

29. Using Scorecard in Architecture Review

Architecture review output:

Decision:
  Approved with conditions.

Required before release:
  - Add idempotency key.
  - Add unique constraint.
  - Add outbox event test.
  - Add support timeline entry.
  - Document external ID mapping.
  - Add reconciliation rule.

Post-release verification:
  - Check no duplicate orders.
  - Check outbox lag.
  - Check projection freshness.

This turns review into actionable readiness plan.

30. Using Scorecard After Incident

After incident, score the failed area.

Questions:

• Which category scored low before incident?
• Was gap known?
• Was hard gate missed?
• What evidence was absent?
• What preventive control should be added?
• How will score improve?

Example:

Duplicate order incident:
  concurrency/idempotency score was 1.
  Add idempotency table + unique constraint + duplicate retry test.

31. Executive-Quality Summary Template

For senior/staff communication:

## Summary
The proposed data model is [Green/Amber/Red].

## Strengths
- Clear source-of-truth for ...
- Strong idempotency for ...
- Migration plan includes ...

## Key risks
- Reconciliation missing for ...
- Contract consumers not fully identified ...
- Projection purge handling incomplete ...

## Required actions
1. ...
2. ...

## Release recommendation
Proceed / proceed with mitigation / block.

## Verification plan
- Monitor ...
- Run ...
- Validate ...

This helps communicate technical data risk to non-specialists.

32. Capstone Checklist

Before declaring an enterprise data model production-ready:

• Business meaning is clear.
• Owner/source-of-truth is clear.
• Identity and lineage are preserved.
• Lifecycle is command-driven and audited.
• Invariants are enforced.
• Contracts are versioned and tested.
• Persistence/indexing supports query and scale.
• Idempotency/concurrency are handled.
• Migration/backfill is safe.
• Security/privacy/retention are addressed.
• Cache/search/projections are controlled.
• Observability/support timeline exists.
• Reconciliation detects distributed drift.
• Tests cover edge/failure cases.
• Repair/DR/runbooks exist.
• Documentation/governance is discoverable.

33. Internal Verification Checklist

Customize scorecard with internal standards:

• CSG/team architecture review policy.
• Quote/order/billing domain owner list.
• Required DB migration checks.
• Required event/API contract checks.
• Required security/privacy classifications.
• Required TMF mapping review.
• Required data quality/reconciliation framework.
• Required support observability.
• Required production repair process.
• Required on-prem/cloud deployment constraints.
• Required test evidence.

Internal standards should be more specific than this generic scorecard.

34. Summary

This scorecard converts the whole series into an operational decision tool.

A production-ready enterprise data model must score well across:

• semantics,
• ownership,
• lineage,
• lifecycle,
• invariants,
• contracts,
• persistence,
• concurrency,
• migration,
• security,
• derived stores,
• observability,
• reconciliation,
• testing,
• operations,
• governance.

The key principle:

A model is ready for production when the team can prove not only that it works on the happy path, but that it remains correct, explainable, recoverable, and governed under retry, failure, scale, change, and human operation.