Data Dictionary, Business Glossary, and Metadata Catalog Model

1. Core Idea

Enterprise data model harus bisa ditemukan, dipahami, dan dipercaya oleh engineer, analyst, support, QA, architect, and product/domain owner.

Data dictionary menjawab:

Field/table ini apa artinya?

Business glossary menjawab:

Istilah bisnis ini berarti apa dalam domain kita?

Metadata catalog menjawab:

Data asset ini ada di mana, siapa owner-nya, apa lineage-nya, apa sensitivity-nya, siapa menggunakannya, dan apakah aman dipakai?

Mental model:

A data model that only exists in code and database tables is not enough. Enterprise data needs a searchable knowledge layer.

2. Why Metadata Matters

Tanpa data dictionary/glossary/catalog:

• engineer salah pakai field,
• analyst membuat KPI dari field yang salah,
• support tidak tahu arti status,
• team berbeda memakai istilah sama dengan meaning berbeda,
• onboarding engineer lambat,
• ownership tidak jelas,
• sensitive field tidak diketahui,
• field dihapus padahal dipakai dashboard,
• duplicate table dibuat karena existing asset tidak ditemukan,
• quote/order/billing semantics dipelajari dari tribal knowledge,
• incident review sulit karena field meaning tidak documented.

Metadata catalog mengubah data model dari hidden implementation menjadi shared enterprise knowledge.

3. Data Dictionary vs Business Glossary vs Metadata Catalog

They overlap but serve different users.

4. Data Dictionary Entry

A data dictionary entry should define:

asset_name
entity/table
field/column
data_type
business_definition
technical_definition
allowed_values
nullable
default_value
source_of_truth
owner_group
data_classification
lineage
quality_rules
example_values
introduced_in_version
deprecated_at

Example:

product_order.source_quote_version

Definition:
  Quote revision/version from which this order was created.

Why important:
  Prevents order from being traced to wrong quote revision.

Source:
  Accepted quote revision during quote-to-order conversion.

Nullability:
  Nullable for orders not created from quote.

Owner:
  Order service / Quote-to-order domain.

5. Business Glossary Term

Glossary term should define business meaning.

Fields:

term
definition
domain_area
synonyms
related_terms
owner_group
status
examples
not_to_be_confused_with

Example:

Term: Billing Account

Definition:
  Account-level payer/billing responsibility entity used for invoicing, billing profile, billing cycle, payment terms, and billing contact.

Not to be confused with:
  Customer account, service account, product instance, invoice.

Glossary prevents semantic confusion.

6. Domain Vocabulary Problem

In quote-to-cash, the same word can mean different things.

Examples:

Glossary must clarify context.

7. Metadata Asset Model

Asset types:

• database schema,
• table,
• column,
• API endpoint,
• DTO field,
• event schema,
• event field,
• file contract,
• dataset,
• dashboard,
• metric,
• code set,
• reference value,
• job/pipeline,
• projection/read model,
• cache/search index,
• report/export.

Catalog should support multiple asset types, not only database tables.

8. Asset Ownership

Every asset should have owner.

Fields:

owner_group
technical_owner
business_owner
data_steward
support_group
escalation_group

Ownership answers:

• who approves change?
• who fixes data quality issue?
• who explains meaning?
• who handles incident?
• who approves access?
• who retires asset?

Ownerless data is unmanaged data.

9. Source-of-Truth Metadata

Catalog should identify source-of-truth.

Example:

product_order.status
source_of_truth = order-service.product_order

Projection field:

order_projection_for_billing.status
source_of_truth = order-service.product_order.status
derived = true
freshness_sla = 2 minutes

This prevents read model from being mistaken as authoritative.

10. Classification Metadata

Every sensitive asset/field should be classified.

Fields:

classification
sensitivity_level
pii_flag
commercial_sensitive_flag
security_sensitive_flag
masking_policy
export_policy
retention_policy

Example:

quote.margin_amount
classification = RESTRICTED
masking_policy = finance/deal-desk only
export_allowed = restricted

Classification must be discoverable before field is added to API/event/report.

11. Lineage Metadata

Catalog should link upstream/downstream.

Example:

invoice_line.amount
upstream:
  charge.amount
  tax_calculation.tax_amount
downstream:
  fact_invoice_line.amount
  revenue_dashboard.billed_revenue

Lineage helps:

• impact analysis,
• incident debugging,
• privacy purge,
• metric trust,
• migration planning.

12. Data Quality Metadata

Field/table should link to DQ rules.

Examples:

product_instance.status
DQ rules:
  ACTIVE_PRODUCT_REQUIRES_ACTIVE_CHARGE
  TERMINATED_PRODUCT_NO_ACTIVE_CHARGE

Metadata:

quality_score
dq_rule_count
open_issue_count
last_checked_at

Catalog can show whether data is trustworthy.

13. Usage Metadata

Catalog should know who uses data.

Usage examples:

• API clients,
• event consumers,
• dashboards,
• jobs,
• reports,
• support tools,
• exports,
• ML/analytics pipelines.

Usage metadata supports:

• deprecation,
• impact analysis,
• access review,
• cost optimization.

Fields:

asset_usage
- asset_id
- consumer_name
- usage_type
- criticality
- last_seen_at

14. Lifecycle Metadata

Asset lifecycle:

DRAFT
ACTIVE
DEPRECATED
RETIRED
ARCHIVED

Fields:

introduced_at
deprecated_at
replacement_asset_id
retired_at
retention_policy

Never delete metadata for retired assets used historically.

15. Glossary Synonyms and Conflicts

Glossary should track synonyms.

Example:

Billing Account
Synonyms:
  bill-to account
  payer account
  invoice account

Also track conflicts:

Account
Conflict:
  In IAM context, account may mean user login.
  In billing context, account may mean payer entity.

This helps onboarding and cross-team communication.

16. Examples and Anti-Examples

Good glossary includes examples.

Example:

Product Offering:
  "Business Internet 500 Mbps" as sellable catalog offer.

Product Specification:
  Technical/commercial blueprint defining characteristics.

Product Instance:
  Customer-specific installed product after order fulfillment.

Anti-example:

Do not use product offering ID to represent installed product.

Anti-examples prevent common modelling mistakes.

17. Metadata Freshness

Metadata can become stale.

Fields:

last_reviewed_at
review_frequency
review_status
verified_by
confidence_level

Critical domain fields should be reviewed periodically.

Stale metadata can be worse than no metadata if people trust it incorrectly.

18. Metadata from Code vs Manual Curation

Sources:

• database introspection,
• OpenAPI/AsyncAPI,
• schema registry,
• code annotations,
• migration files,
• lineage tools,
• manual business definitions,
• data quality system,
• access control system,
• dashboard/BI metadata.

Automated extraction gives structure. Manual curation gives meaning.

Both are needed.

19. Naming Standards

Metadata catalog should enforce naming standards.

Examples:

• *_id for identifiers,
• *_number for human business numbers,
• *_at for timestamp,
• *_date for date-only,
• *_status for lifecycle/status code,
• *_amount with currency field,
• source_* for lineage,
• external_* for external reference,
• effective_from/effective_to for validity period.

Naming conventions reduce semantic ambiguity.

20. Data Dictionary for Time Fields

Time fields require precise definition.

Example:

created_at:
  system transaction time when row was created.

accepted_at:
  business time when quote was accepted.

effective_from:
  business validity start, inclusive.

billing_period_start:
  local billing period start date.

ingested_at:
  analytics ingestion time.

Do not treat all timestamps as interchangeable.

21. Data Dictionary for Money Fields

Money fields should document:

• amount semantics,
• currency,
• tax included/excluded,
• discount included/excluded,
• precision/scale,
• rounding rule,
• effective period,
• source calculation,
• snapshot vs recomputed.

Example:

quote.total_amount:
  total proposed amount for quote version, excluding tax unless tax_included flag is true.

22. Data Dictionary for Status Fields

Status fields should document:

• status code set,
• lifecycle state machine,
• terminal status,
• transition owner,
• external mapping,
• reporting category,
• customer visibility,
• billing/fulfillment effect.

Example:

order.fulfillment_status != order.lifecycle_status

This must be explicit.

23. Metadata Catalog Physical Design

Asset table:

create table metadata_asset (
  id uuid primary key,
  asset_type text not null,
  asset_name text not null,
  domain_area text,
  system_name text,
  description text,
  lifecycle_status text not null,
  owner_group text,
  data_classification text,
  source_of_truth boolean not null default false,
  created_at timestamptz not null,
  updated_at timestamptz not null,
  unique (asset_type, system_name, asset_name)
);

Field metadata:

create table metadata_field (
  id uuid primary key,
  asset_id uuid not null references metadata_asset(id),
  field_name text not null,
  data_type text,
  business_definition text,
  technical_definition text,
  nullable boolean,
  default_value text,
  allowed_values_reference text,
  data_classification text,
  source_of_truth_field text,
  introduced_at timestamptz,
  deprecated_at timestamptz,
  unique (asset_id, field_name)
);

Glossary term:

create table business_glossary_term (
  id uuid primary key,
  term text not null unique,
  domain_area text,
  definition text not null,
  owner_group text,
  status text not null,
  examples text,
  not_to_be_confused_with text,
  created_at timestamptz not null,
  updated_at timestamptz not null
);

Asset usage:

create table metadata_asset_usage (
  id uuid primary key,
  asset_id uuid not null references metadata_asset(id),
  consumer_name text not null,
  usage_type text not null,
  criticality text,
  last_seen_at timestamptz,
  owner_group text
);

Indexes:

create index idx_metadata_asset_domain
on metadata_asset (domain_area, lifecycle_status);

create index idx_metadata_field_name
on metadata_field (field_name);

create index idx_metadata_field_classification
on metadata_field (data_classification);

create index idx_glossary_domain
on business_glossary_term (domain_area, status);

create index idx_asset_usage_consumer
on metadata_asset_usage (consumer_name, usage_type);

24. Java/JAX-RS Backend Implications

Internal catalog APIs:

GET /metadata/assets
GET /metadata/assets/{id}
GET /metadata/assets/{id}/fields
GET /metadata/glossary
GET /metadata/glossary/{term}
GET /metadata/search?q=billing account
GET /metadata/assets/{id}/lineage
GET /metadata/assets/{id}/quality

Potential integrations:

• generate metadata from OpenAPI,
• generate metadata from DB schema,
• publish contract metadata,
• link DQ rules,
• link lineage,
• expose owner/runbook.

25. Metadata Search

Search should support:

• term search,
• field name search,
• asset name search,
• owner search,
• domain search,
• classification search,
• deprecated asset search,
• source-of-truth search,
• "where is this field used?"

Examples:

Search: billingAccountId
Results:
  API field: CreateQuoteRequest.billingAccountId
  Table column: quote.billing_account_id
  Table column: product_order.billing_account_id
  Glossary: Billing Account
  Event field: ProductOrderCreated.billingAccountId

Discoverability is the goal.

26. Metadata and Onboarding

For new engineers, catalog should answer:

• What is the difference between quote item and order item?
• Which service owns product inventory?
• Which table stores current product state?
• Which event creates billing charge?
• What does fallout mean?
• What is the source of invoice data?
• Is billing account internal or external-owned?
• Which data is sensitive?
• Which dashboard/KPI is official?

Good metadata reduces onboarding time dramatically.

27. Metadata and PR Review

PR reviewers can use catalog to check:

• field definition exists,
• owner exists,
• classification exists,
• source-of-truth is clear,
• lineage updated,
• DQ rules linked,
• contract/consumer usage known,
• deprecated fields not reused.

Metadata catalog should be part of engineering workflow, not separate unused wiki.

28. Data Quality Checks

Examples:

-- Active fields without business definition
select ma.asset_name, mf.field_name
from metadata_field mf
join metadata_asset ma on ma.id = mf.asset_id
where ma.lifecycle_status = 'ACTIVE'
  and (mf.business_definition is null or mf.business_definition = '');

-- Sensitive fields without classification
select ma.asset_name, mf.field_name
from metadata_field mf
join metadata_asset ma on ma.id = mf.asset_id
where lower(mf.field_name) like '%email%'
  and mf.data_classification is null;

-- Active assets without owner
select asset_type, system_name, asset_name
from metadata_asset
where lifecycle_status = 'ACTIVE'
  and owner_group is null;

29. Failure Modes

30. PR Review Checklist

When reviewing metadata/catalog changes, ask:

• Is asset registered?
• Is field definition clear?
• Is business meaning distinct from technical type?
• Is owner assigned?
• Is source-of-truth marked?
• Is classification assigned?
• Is allowed value/reference data linked?
• Is lineage updated?
• Is DQ rule linked?
• Is usage/consumer known?
• Is lifecycle status correct?
• Are synonyms/confusing terms documented?
• Are examples included for important domain terms?
• Is metadata generated/curated consistently?
• Is it searchable/discoverable?

31. Internal Verification Checklist

Verify these in the internal CSG/team context:

• Existing data dictionary or metadata catalog.
• Existing business glossary for CPQ/order/billing/telco terms.
• Whether metadata is generated from DB/API/event schemas.
• Whether field definitions include business semantics.
• Whether owners are assigned per asset/field.
• Whether sensitive classification is stored in metadata.
• Whether lineage/usage data is discoverable.
• Whether official KPI/metric catalog exists.
• Whether deprecated fields are tracked.
• Whether onboarding docs link to metadata catalog.
• Whether incidents mention field confusion, ownerless data, stale docs, or unknown consumer.

32. Summary

Metadata makes enterprise data understandable and discoverable.

A strong model must define:

• data dictionary,
• business glossary,
• metadata asset,
• field metadata,
• owner,
• source-of-truth,
• classification,
• lineage,
• quality rules,
• usage,
• lifecycle,
• synonyms,
• examples,
• naming standards,
• metadata freshness,
• search/discoverability,
• PR review integration.

The key principle:

A data model is not truly enterprise-ready until its meaning, ownership, sensitivity, lineage, usage, and lifecycle are discoverable by people who did not design it.