Enterprise Data Modelling Mastery Map for Senior Backend Engineers

1. Core Idea

Enterprise data modelling mastery bukan berarti hafal semua pattern table.

Mastery berarti mampu:

Translate ambiguous business reality
into explicit data truth
that remains correct under scale, failure, retry, integration, security, migration, reporting, and human operation.

Untuk konteks CPQ / Quote / Order / Billing / Telco BSS/OSS, seorang senior backend engineer harus bisa berpikir di beberapa layer sekaligus:

• domain semantics,
• data ownership,
• lifecycle,
• invariants,
• relational design,
• event-driven consistency,
• integration contracts,
• operational metadata,
• reporting and reconciliation,
• migration and evolution,
• security/privacy,
• production debugging,
• governance and trade-offs.

Mental model:

Data modelling mastery is the ability to protect business truth across time, systems, teams, and failure modes.

2. The Five Levels of Mastery

Level 1 — Table Builder

Can create tables and fields.

Characteristics:

• translates UI form to table,
• knows primary key and foreign key,
• can write simple SQL,
• thinks mostly in CRUD,
• limited view of lifecycle/failure.

Risk:

Creates data structure that works for happy path but fails under real production flow.

Level 2 — Relational Designer

Can model relationships and constraints.

Characteristics:

• understands normalization,
• uses foreign keys,
• knows indexes,
• understands one-to-many/many-to-many,
• can model history/basic audit,
• can reason about query performance.

Risk:

Good database structure, but may miss domain lifecycle, ownership, and distributed consistency.

Level 3 — Domain Data Modeller

Can model business truth.

Characteristics:

• distinguishes quote/order/product/invoice concepts,
• understands source-of-truth,
• models snapshots,
• defines state machines,
• knows invariants,
• separates DTO/domain/entity,
• understands domain events.

Risk:

Good domain model, but may still under-design migration, observability, reconciliation, and operations.

Level 4 — Production Data Architect

Can design for failure, scale, and operations.

Characteristics:

• designs idempotency,
• handles concurrency,
• uses outbox/inbox,
• models reconciliation,
• designs migrations/backfills,
• understands cache/search/projection consistency,
• handles privacy/retention,
• supports incident debugging,
• creates operational runbooks.

Risk:

May still need stronger enterprise governance, cross-team alignment, and strategic trade-off communication.

Level 5 — Staff/Principal Data Systems Thinker

Can align business, architecture, teams, and long-term evolution.

Characteristics:

• defines modelling standards,
• drives ADRs,
• coaches teams,
• designs platform-level patterns,
• balances TMF/canonical/internal models,
• manages trade-offs explicitly,
• prevents architectural drift,
• links incidents to systemic improvements,
• creates reusable scorecards and playbooks.

Goal:

Make data correctness a team capability, not an individual hero skill.

3. Mastery Dimension 1 — Domain Semantics

You must be able to answer:

• What business concept is this?
• Is this object a proposal, instruction, installed truth, financial artifact, or operational evidence?
• What is current vs historical?
• What is source vs snapshot?
• What is derived?
• What does null mean?
• What does status mean?
• Who uses this data?

Examples:

Quote = commercial proposal.
Order = executable customer/product instruction.
Product instance = installed commercial truth.
Service instance = technical realization.
Charge = billable financial obligation.
Invoice line = issued financial statement.

Mastery indicator:

You can explain why quote item, order item, product instance, charge, and invoice line must not be collapsed into one generic "item".

4. Mastery Dimension 2 — Identity and References

You must distinguish:

• technical ID,
• business number,
• external ID,
• legacy ID,
• canonical ID,
• idempotency key,
• correlation ID,
• natural key.

You should know when each is used.

Example:

order.id
  internal technical ID

order.order_number
  human/support reference

external_reference.external_id
  billing/OSS/CRM identifier

idempotency_key
  retry protection

correlation_id
  traceability across flow

Mastery indicator:

You can design a model where a customer gives only a billing system order number and support can trace it back to local order, quote, product, and invoice.

5. Mastery Dimension 3 — Lifecycle and State Machines

You must model state as controlled lifecycle, not free text.

Questions:

• What states exist?
• What transitions are legal?
• What transitions are terminal?
• What transitions require reason/approval?
• What transitions trigger events?
• What transitions affect billing/fulfillment?
• What histories must be kept?
• How are stale or duplicate transitions handled?

Mastery indicator:

You refuse to implement PATCH status for critical lifecycle and instead design command endpoints with guards, audit, and events.

6. Mastery Dimension 4 — Invariants and Constraints

You must identify what must always be true.

Examples:

One order per accepted quote version.
Accepted quote cannot be modified.
Approval applies to exact quote version.
Active billable product should have active charge.
Issued invoice should not be overwritten.
Current effective-dated records should not overlap.
Duplicate usage event should not be billed twice.

Mastery means mapping invariants to:

• domain code,
• database constraints,
• idempotency keys,
• reconciliation rules,
• tests,
• alerts.

Mastery indicator:

For every critical business rule, you can say where it is enforced and how it is monitored if distributed.

7. Mastery Dimension 5 — Temporal Modelling

Enterprise data changes over time.

You must understand:

• valid time,
• transaction time,
• effective dating,
• revision/version,
• snapshot,
• history,
• supersession,
• backdating,
• future-dated changes,
• timezone/cutoff.

Examples:

Catalog version valid when quote was created.
Quote revision accepted at a timestamp.
Product billing assignment effective from next cycle.
Invoice issued for billing period.
Usage event occurred at network event time but ingested later.

Mastery indicator:

You do not overwrite historical truth with current values when customer, price, billing account, or catalog changes.

8. Mastery Dimension 6 — Financial and Billing Correctness

For quote-to-cash systems, money data requires extra discipline.

You must model:

• amount,
• currency,
• precision/scale,
• tax included/excluded,
• discount/margin,
• charge type,
• billing period,
• proration,
• adjustment,
• credit/debit,
• invoice immutability,
• payment status,
• revenue stage.

Mastery indicator:

You can explain the difference between quoted amount, booked amount, activated MRR, billed amount, collected amount, and recognized revenue.

9. Mastery Dimension 7 — Product, Service, and Resource Boundary

In telco/BSS/OSS contexts, you must distinguish:

Product = what customer buys.
Service = how product is technically delivered.
Resource = technical/logical/physical asset used.

Mastery means you can model:

• product offering,
• product specification,
• product instance,
• service specification,
• service instance,
• resource specification,
• resource instance,
• product-service-resource realization.

Mastery indicator:

You can trace a customer-facing product to technical service/resource inventory without mixing commercial and network truth.

10. Mastery Dimension 8 — Integration and Cross-System Mapping

Enterprise systems are integrated systems.

You must model:

• external references,
• source systems,
• canonical references,
• callback correlation,
• external status mapping,
• external ID lifecycle,
• mapping conflicts,
• integration messages,
• retry/DLQ,
• reconciliation.

Mastery indicator:

You never use a naked external_id without source_system, entity_type, tenant/environment, lifecycle, and uniqueness strategy.

11. Mastery Dimension 9 — Event-Driven Consistency

You must understand:

• domain event,
• integration event,
• outbox,
• inbox,
• aggregate version,
• ordering,
• replay,
• duplicate delivery,
• stale events,
• DLQ,
• consumer idempotency,
• saga,
• compensation.

Mastery indicator:

You can design quote acceptance so state change, audit, and QuoteAccepted event are committed atomically through outbox, and order conversion is idempotent.

12. Mastery Dimension 10 — PostgreSQL Physical Design

You must connect conceptual design to physical database design.

Skills:

• primary key strategy,
• unique constraints,
• partial indexes,
• composite indexes,
• tenant-aware indexes,
• effective-dated uniqueness,
• JSONB trade-off,
• numeric/time data types,
• FK vs logical reference,
• partitioning,
• hot table design,
• queue table concurrency,
• online migration.

Mastery indicator:

You can translate a business invariant into a PostgreSQL constraint or explain why it must be reconciled instead.

13. Mastery Dimension 11 — Java Backend Implementation

You must translate model into code safely.

Skills:

• DTO/domain/entity separation,
• command-oriented API,
• aggregate behavior,
• application service transaction boundary,
• repository ownership,
• validation layers,
• stable error codes,
• optimistic locking,
• idempotency table,
• outbox/inbox,
• contract tests,
• integration tests with real DB.

Mastery indicator:

You avoid setter-based status mutation and implement domain commands with version guard, audit, and outbox event.

14. Mastery Dimension 12 — Migration and Evolution

Data models evolve.

You must handle:

• additive changes,
• breaking changes,
• expand/contract,
• dual-write,
• backfill,
• exception handling,
• shadow read,
• cutover,
• rollback/forward fix,
• projection rebuild,
• contract deprecation,
• on-prem upgrade path.

Mastery indicator:

You can design a migration for millions of rows without locking hot tables, losing compatibility, or hiding exceptions.

15. Mastery Dimension 13 — Security, Privacy, and Retention

You must know:

• PII classification,
• commercial-sensitive data,
• field-level authorization,
• masking,
• encryption/tokenization,
• audit access,
• export governance,
• retention,
• legal hold,
• purge/anonymization,
• derived copy propagation,
• lower environment data safety.

Mastery indicator:

Before adding a field to an event/search index/export, you ask whether it is sensitive and whether derived copies are governed.

16. Mastery Dimension 14 — Observability and Supportability

Enterprise data must be debugged.

You must model:

• correlation ID,
• causation ID,
• business key,
• status history,
• audit,
• operational event,
• integration message,
• retry metadata,
• support timeline,
• data repair case,
• external reference,
• data quality result.

Mastery indicator:

You can reconstruct why a customer was billed by tracing invoice line -> charge -> product instance -> order item -> quote item -> quote.

17. Mastery Dimension 15 — Reconciliation and Data Quality

You must assume distributed drift happens.

You must design:

• DQ rules,
• reconciliation rules,
• severity,
• owner,
• SLA,
• result persistence,
• suppression/false positive handling,
• repair workflow,
• dashboards,
• tests.

Mastery indicator:

For every eventual consistency boundary, you can name the reconciliation rule that catches drift.

18. Mastery Dimension 16 — Reporting and Analytics

You must know operational model is not automatically reporting model.

Skills:

• fact grain,
• dimension,
• slowly changing dimension,
• event time/business time/ingestion time,
• metric definition,
• KPI lineage,
• snapshot fact,
• funnel analysis,
• MRR/churn/backlog/revenue distinctions,
• dashboard freshness,
• access control.

Mastery indicator:

You can prevent sales, finance, and product teams from using one ambiguous revenue number for different business meanings.

19. Mastery Dimension 17 — Governance and Communication

You must communicate decisions.

Skills:

• ADR,
• data dictionary,
• business glossary,
• contract registry,
• owner metadata,
• review workflow,
• scorecard,
• PR checklist,
• internal verification checklist,
• trade-off framework.

Mastery indicator:

You can write a data model proposal that explains context, options, decision, trade-offs, migration, risks, and verification plan.

20. The Senior Engineer Mental Checklist

Before implementing data model change, ask:

What truth is represented?
Who owns it?
Is it current, historical, snapshot, or projection?
What lifecycle does it follow?
What invariants apply?
What IDs/references must be preserved?
What happens under retry?
What happens under concurrency?
What events/contracts change?
What migration is needed?
What sensitive data is involved?
What derived copies exist?
How will support debug this?
How will reconciliation catch drift?
What tests prove correctness?
What trade-off are we accepting?

This checklist should become instinct.

21. Common Transition from Mid-Level to Senior

Mid-level tends to ask:

How do I add this field?

Senior asks:

What does this field mean?
Who owns it?
Is it source or snapshot?
Who consumes it?
What happens when it changes?
Can it leak?
Can it drift?
Can it be migrated?
Can it be debugged?

That shift is the essence of senior data modelling.

22. Common Transition from Senior to Staff

Senior optimizes local design.

Staff optimizes system/team capability.

Staff asks:

How do we prevent this class of issue across teams?
What standard should exist?
What tooling/checklist should enforce this?
What incident pattern tells us the model is weak?
What governance is missing?
How do we teach this to others?

Staff-level mastery turns individual judgement into organization-level patterns.

23. Mastery Practice Plan

Daily

• Read one PR through data model lens.
• Ask one source-of-truth/invariant question.
• Trace one entity in logs/database/events.
• Note one ambiguous term.

Weekly

• Review one incident/postmortem.
• Map one flow end-to-end.
• Study one migration.
• Inspect one event schema.
• Improve one test or documentation gap.

Monthly

• Run scenario lab.
• Review scorecard for one critical model.
• Update glossary/data dictionary.
• Write or update ADR.
• Add/validate one reconciliation rule.

24. Mastery Evidence Portfolio

Keep examples of:

• data model proposal,
• ADR,
• migration plan,
• production incident analysis,
• reconciliation rule,
• event contract review,
• PostgreSQL index/constraint design,
• DTO/API contract review,
• privacy classification decision,
• support timeline improvement,
• golden test scenario,
• refactoring plan.

This portfolio demonstrates senior/staff capability.

25. Mastery Red Flags

If you frequently say these, pause:

• "It's just a field."
• "Frontend validates it."
• "We can fix it manually."
• "No one will retry."
• "Status is obvious."
• "External ID is unique."
• "This report can join tables directly."
• "Cache will update eventually."
• "We'll migrate later."
• "This is temporary."

These are common paths to production data debt.

26. Internal CSG/Team Verification Map

For the actual team/system, verify:

• internal definition of tenant/customer/account,
• actual quote versioning model,
• actual order lifecycle,
• actual product/service/resource ownership,
• actual billing integration ownership,
• actual TMF API usage,
• actual event schema conventions,
• actual PostgreSQL migration rules,
• actual outbox/inbox implementation,
• actual data quality/reconciliation framework,
• actual production repair process,
• actual support tooling,
• actual on-prem/cloud constraints.

Generic mastery must be grounded in internal reality.

27. Final Mastery Rubric

Rate each 1–5:

Target for senior engineer:

Most dimensions >= 3
Critical current work dimensions >= 4
No blind spot in billing/security/migration/consistency areas

Target for staff/principal:

Most dimensions >= 4
Can teach/review/standardize across teams
Can connect incidents to systemic patterns

28. Summary

Enterprise data modelling mastery is multi-dimensional.

You need competence in:

• domain,
• identity,
• lifecycle,
• invariants,
• temporal correctness,
• financial correctness,
• BSS/OSS boundaries,
• integration,
• events,
• PostgreSQL,
• Java implementation,
• migrations,
• security,
• observability,
• reconciliation,
• analytics,
• governance,
• trade-offs.

The key principle:

A senior backend engineer protects business truth in code. A staff-level engineer protects business truth across systems, teams, time, and failure modes.