Part 088 — Amazon Neptune: Graph Modeling, Traversal, Relationship-Heavy Workloads

Core idea: graph database bukan database untuk “data yang punya relasi”. Semua database punya relasi. Graph database berguna ketika relasi itu sendiri adalah data utama, traversal adalah hot path, dan jawaban bisnis muncul dari struktur koneksi: path, neighborhood, reachability, centrality, similarity, influence, fraud ring, ownership chain, dependency chain, atau knowledge graph.

1. Tujuan Pembelajaran

Setelah bagian ini, kamu harus bisa:

1. Menentukan kapan Amazon Neptune lebih tepat daripada Aurora/RDS, DynamoDB, DocumentDB, atau OpenSearch.
2. Memilih antara property graph dan RDF berdasarkan bentuk data, query, semantics, dan tim.
3. Mendesain vertex, edge, label, property, cardinality, dan identity secara production-grade.
4. Menulis traversal yang bounded, index-friendly, dan tidak meledakkan fanout.
5. Memahami Gremlin, openCypher, dan SPARQL sebagai query surface dengan trade-off berbeda.
6. Mendesain ingestion, bulk load, change propagation, graph projection, dan rebuild strategy.
7. Mengoperasikan Neptune dari sisi query latency, memory pressure, replica, failover, slow traversal, dan incident runbook.

2. Mental Model

Relational database bertanya:

How do tables join?

Document database bertanya:

What aggregate document should I load?

Key-value database bertanya:

What key/index answers this access pattern?

Graph database bertanya:

What path through relationships answers the question?

Contoh pertanyaan graph:

Which entities are indirectly controlled by this organization?
Which accounts share devices, addresses, officers, and transactions?
Which cases are connected through the same evidence chain?
Which service dependencies are impacted by this database migration?
Which subjects are within 3 hops of a sanctioned entity?

Jika pertanyaan utama mengandung:

connected to
path to
within N hops
owns indirectly
similar to
shares with
depends on
influences
reachable from

maka graph model layak dipertimbangkan.

3. AWS Positioning

Amazon Neptune adalah fully managed graph database service untuk highly connected datasets. Neptune mendukung property graph query languages seperti Apache TinkerPop Gremlin dan openCypher, serta RDF/SPARQL untuk semantic graph/knowledge graph workloads.

Neptune cocok untuk:

1. fraud detection,
2. identity graph,
3. entity resolution,
4. knowledge graph,
5. recommendation graph,
6. dependency graph,
7. network/security graph,
8. ownership/control graph,
9. relationship-heavy investigation.

Tidak cocok untuk:

1. simple CRUD aggregate,
2. key-value lookup,
3. general relational transactions,
4. high-volume append-only time-series,
5. full-text relevance search,
6. arbitrary analytics warehouse.

4. Graph Is Not “Join But Fancy”

A relational query can answer relationship questions, but cost grows when relationship depth and variability grow.

Relational pattern:

SELECT ...
FROM company c
JOIN ownership o1 ON ...
JOIN company c2 ON ...
JOIN ownership o2 ON ...
JOIN company c3 ON ...

Graph pattern:

start from Company A
follow OWNS edges 1..N hops
filter effective control
return reachable companies and paths

Graph becomes valuable when:

1. path length is dynamic,
2. relationship type matters,
3. relationship property matters,
4. traversal direction matters,
5. neighborhood expansion matters,
6. graph topology changes the answer,
7. the query is naturally expressed as traversal.

5. Decision Framework

Rule:

Do not choose graph because data has relationships.
Choose graph when relationship traversal is the product.

6. Property Graph vs RDF

Neptune supports two major modeling worlds.

6.1 Property Graph

Property graph has:

1. vertices,
2. edges,
3. labels,
4. properties on vertices and edges.

Example:

(:Company {id, name, country})
(:Person {id, name})
(:Case {id, status})

(:Person)-[:DIRECTOR_OF {from, to}]->(:Company)
(:Company)-[:OWNS {percent, from, to}]->(:Company)
(:Case)-[:INVOLVES]->(:Company)

Query surfaces:

1. Gremlin,
2. openCypher.

Best for:

1. application graph,
2. fraud graph,
3. recommendation graph,
4. operational dependency graph,
5. entity relationship graph.

6.2 RDF

RDF represents facts as triples/quads.

subject predicate object

Example:

:CompanyA :owns :CompanyB .
:CompanyA :jurisdiction :Indonesia .
:PersonX :directorOf :CompanyA .

Query surface:

SPARQL

Best for:

1. knowledge graph,
2. ontology-driven model,
3. linked data,
4. semantic interoperability,
5. reasoning-adjacent workloads.

6.3 Selection Table

The choice is hard to reverse. Treat it as architecture decision, not implementation detail.

7. Graph Modeling Principles

7.1 Start From Questions

Do not start with entities. Start with graph questions.

Q1: Find all companies directly or indirectly controlled by subject X.
Q2: Find cases connected by shared bank account, address, director, or device.
Q3: Find shortest path between subject and sanctioned entity.
Q4: Find investigators impacted by a policy change through case assignments.
Q5: Find suspicious clusters where many entities share weak signals.

Then identify:

1. starting vertices,
2. edge labels,
3. max traversal depth,
4. filters,
5. path constraints,
6. result shape,
7. freshness requirement,
8. source of truth.

7.2 Vertex Design

Vertex should represent identity with stable lifecycle.

Good vertices:

Person
Company
Case
Evidence
Address
BankAccount
Device
License
Regulation
ControlRequirement
Service
Database

Bad vertices:

CaseStatusChangedEvent as vertex for every event when traversal does not need it
RawLogLine as vertex
UIFilter as vertex
JSONField as vertex without semantics

7.3 Edge Design

Edges are first-class facts.

(Person)-[:DIRECTOR_OF {from, to, source, confidence}]->(Company)
(Company)-[:OWNS {percent, from, to, source}]->(Company)
(Case)-[:INVOLVES {role}]->(Company)
(Case)-[:USES_EVIDENCE]->(Evidence)
(Person)-[:SHARES_ADDRESS {from, to, confidence}]->(Address)

Edge design questions:

1. Is direction meaningful?
2. Is edge type stable?
3. Does edge have properties?
4. Does edge expire?
5. Does edge have confidence/source?
6. Can duplicate edge exist?
7. Is edge authoritative or derived?
8. Is edge part of audit/history?

7.4 Direction Matters

Choose direction by traversal frequency.

If query is:

from person -> companies they direct

Edge:

(Person)-[:DIRECTOR_OF]->(Company)

If reverse query is also hot:

from company -> directors

Graph databases can usually traverse incoming edges, but you must verify query cost and index/start-point strategy. Do not duplicate reverse edges unless necessary and governed.

8. Identity Strategy

Identity bugs destroy graph quality.

Every vertex needs stable ID.

Company: COMPANY#jurisdiction#registrationNumber
Person: PERSON#country#nationalId or internal mastered person id
Address: ADDRESS#normalizedHash
BankAccount: BANK_ACCOUNT#iban/hash
Case: CASE#caseNumber

Avoid:

Company name as ID
free-text address as ID
email as immutable person ID
phone number as primary identity

8.1 Entity Resolution Boundary

Do not hide entity resolution inside graph writes.

Pipeline:

Graph query quality depends on mastered identity quality.

9. Source of Truth vs Graph Projection

Most enterprise Neptune deployments should treat graph as projection, not primary source of truth.

Aurora/DynamoDB/DocumentDB = authoritative transactional state
Neptune = relationship query projection

Why?

1. Source systems own business transactions.
2. Graph materializes relationships optimized for traversal.
3. Projection can be rebuilt.
4. Data lineage remains defensible.
5. Graph schema can evolve separately.

Pattern:

Graph projection invariants:

1. Projector is idempotent.
2. Edge writes include source event/version.
3. Deletes/tombstones are modeled.
4. Rebuild path exists.
5. Projection freshness is observable.
6. Source record can explain every graph fact.

10. Property Graph Modeling Example

Regulatory enforcement graph:

10.1 Vertex Labels

10.2 Edge Labels

11. Query Surface: Gremlin, openCypher, SPARQL

11.1 Gremlin

Gremlin is traversal-oriented.

g.V('COMPANY#ID#123')
 .repeat(outE('OWNS').has('percent', gte(25)).inV().simplePath())
 .times(3)
 .path()

Good when:

1. traversal logic is procedural,
2. step-by-step path control matters,
3. application team is comfortable with traversal APIs,
4. you need explicit control over expansion.

11.2 openCypher

openCypher is pattern-oriented.

MATCH path = (c:Company {id: 'COMPANY#ID#123'})-[:OWNS*1..3]->(target:Company)
RETURN path, target

Good when:

1. graph pattern matching reads naturally,
2. team knows Cypher/Neo4j style,
3. query is declarative,
4. analysts/developers prefer pattern syntax.

11.3 SPARQL

SPARQL is RDF query language.

SELECT ?company ?owned
WHERE {
  ?company :owns ?owned .
  ?company :jurisdiction "ID" .
}

Good when:

1. RDF/ontology matters,
2. standard semantic graph ecosystem matters,
3. linked data/interoperability matters,
4. facts are naturally triples.

11.4 Rule

Do not choose query language only because it looks nicer. Choose based on:

1. graph model,
2. team capability,
3. query complexity,
4. tooling,
5. compatibility needs,
6. long-term maintainability.

12. Traversal Design

Graph incident often comes from unbounded traversal.

Bad:

start from broad set -> expand all edges -> filter late -> unlimited depth

Good:

start from selective vertex -> filter edges early -> cap depth -> cap result -> return bounded shape

12.1 Traversal Checklist

1. Is start vertex selective?
2. Is max depth bounded?
3. Are edge labels explicit?
4. Are edge properties filtered early?
5. Is path cycle controlled?
6. Is result count limited?
7. Is timeout set?
8. Is query profiled?
9. Is result size safe for API response?
10. Is query safe under high-degree vertex?

12.2 High-Degree Vertex Problem

High-degree vertices are graph hot keys.

Examples:

Address shared by thousands of entities
Generic regulation linked to millions of companies
Country vertex connected to every company
Common device/browser fingerprint

Mitigation:

1. avoid traversing through generic vertices in hot path,
2. model relationship with more selective edge properties,
3. cap traversal depth/result,
4. precompute summary/projection,
5. use query-specific graph projection,
6. filter early,
7. split high-degree concepts.

13. Path Query Semantics

Path query must define semantics explicitly.

Example: indirect ownership.

Questions:

1. What edge labels count as ownership?
2. What percentage threshold applies?
3. How deep can ownership chain go?
4. Are inactive edges ignored?
5. How are cycles handled?
6. Is effective control cumulative or per-edge?
7. What timestamp version of graph is used?
8. Does jurisdiction affect ownership interpretation?

Pseudo query semantics:

A controls B if there exists a path from A to B where:
  - every edge label is OWNS
  - every edge is active at evaluation time
  - every edge percent >= 25
  - path length <= 5
  - path has no repeated vertex

This is domain logic. Put it in a named graph query service, not copied across UI code.

14. Write Patterns

14.1 Authoritative Graph Writes

If Neptune is source of truth, writes must enforce graph invariants.

Example:

create vertex if absent
create edge if absent
update edge properties if source version newer
write audit metadata

But Neptune is not usually the best place for complex multi-aggregate business transactions. Keep domain commands in authoritative systems when possible.

14.2 Projection Writes

Projection event:

{
  "eventId": "01J...",
  "eventType": "CompanyOwnershipChanged",
  "source": "company-registry",
  "aggregateId": "COMPANY#ID#123",
  "version": 44,
  "occurredAt": "2026-07-07T10:00:00Z",
  "payload": {
    "ownerCompanyId": "COMPANY#SG#991",
    "ownedCompanyId": "COMPANY#ID#123",
    "percent": 45,
    "validFrom": "2026-01-01"
  }
}

Projector logic:

if event version <= lastAppliedVersion(edgeIdentity): skip
upsert owner vertex
upsert owned vertex
upsert OWNS edge with properties
record lastAppliedVersion

14.3 Idempotent Edge Upsert

Edge identity:

EDGE#OWNS#OWNER#COMPANY#SG#991#OWNED#COMPANY#ID#123#SOURCE#registry

Properties:

{
  "percent": 45,
  "validFrom": "2026-01-01",
  "validTo": null,
  "source": "company-registry",
  "sourceVersion": 44,
  "updatedAt": "2026-07-07T10:00:00Z"
}

Idempotency rule:

same event twice -> same graph state
older event after newer event -> ignored or creates historical edge if model supports bitemporal data

15. Delete and Temporal Modeling

Delete is dangerous in graph because paths disappear.

Options:

For regulatory systems, temporal edges are often better.

(Person)-[:DIRECTOR_OF {from: 2020-01-01, to: 2024-02-01}]->(Company)

Query must filter active interval.

16. Bulk Loading and Rebuild

Graph projection must be rebuildable.

Rebuild phases:

1. freeze graph schema contract,
2. export authoritative sources,
3. transform to vertex/edge load files,
4. bulk load into new graph/cluster or clean target,
5. validate counts and sample paths,
6. apply delta events since export point,
7. switch read traffic,
8. keep old graph for rollback window.

Validation:

17. Query Tuning

Neptune query performance is highly dependent on start point, expansion, filters, and high-degree nodes.

17.1 Tuning Loop

1. capture slow query
2. identify start condition
3. inspect fanout at each step
4. move filters earlier
5. reduce traversed edge labels
6. cap depth/result
7. test on production-like graph
8. compare p50/p95/p99 and result correctness

17.2 Explain/Profile Discipline

For any critical query, keep:

1. query text,
2. intended semantics,
3. expected cardinality,
4. sample graph size,
5. profile/explain result,
6. tuning notes,
7. owner.

Repository layout:

graph/
  queries/
    indirect-ownership.gremlin
    shared-signal-ring.opencypher
    regulation-impact.sparql
  profiles/
    indirect-ownership-profile-2026-07-07.md
  tests/
    indirect-ownership-fixture.json

18. API Layer Pattern

Do not expose raw Gremlin/openCypher/SPARQL to frontend.

Graph API should expose domain questions.

GET /cases/{caseId}/relationship-map?depth=2
GET /entities/{entityId}/ownership-chain?maxDepth=5
GET /entities/{entityId}/shared-signals?types=ADDRESS,DEVICE,BANK_ACCOUNT
GET /regulations/{regulationId}/impact-map

Service boundary:

Why:

1. prevent arbitrary expensive queries,
2. enforce authorization,
3. enforce max depth/result,
4. keep domain semantics centralized,
5. enable query profiling and versioning,
6. protect sensitive relationships.

19. Authorization and Privacy

Graph data is privacy-sensitive because it reveals relationships.

A user may be allowed to see a case but not all connected entities.

Authorization dimensions:

1. tenant,
2. jurisdiction,
3. case assignment,
4. data classification,
5. relationship type,
6. relationship confidence,
7. source system restriction,
8. investigation purpose,
9. time-bound access.

Do not assume graph traversal can simply filter after query. Sometimes filtering after traversal leaks existence through count, path shape, latency, or UI hints.

Safer model:

permission-aware start set
allowed edge labels
allowed max depth
redacted vertex properties
result count cap
purpose-of-access audit

20. Graph and Search Together

Graph answers path/relationship questions. Search answers text/relevance questions.

Combined pattern:

Example:

1. Search company by fuzzy name in OpenSearch.
2. User selects candidate entity.
3. Graph query finds ownership/control relationships in Neptune.
4. Case API loads authoritative case/entity details from source systems.

Do not force Neptune to do fuzzy text search. Do not force OpenSearch to do graph traversal.

21. Graph and Relational Together

Relational remains useful for authoritative transactions.

Rule:

Use relational/document/key-value stores for authoritative state.
Use Neptune for relationship-optimized query projection unless graph is truly the source domain.

22. Operability Metrics

Monitor:

Application metrics should use domain names:

graph.query.indirectOwnership.latency.p95
graph.query.sharedSignalRing.timeout.count
graph.projector.companyOwnership.lag.seconds
graph.projector.event.failure.count
graph.vertex.company.count
graph.edge.owns.count

23. Failure Modes

24. Reconciliation

Graph projection needs reconciliation because events can be missed, duplicated, delayed, or applied incorrectly.

Reconciliation job:

for each authoritative relationship batch:
  compute expected edge identity
  check graph has current edge
  check sourceVersion matches
  check inactive/deleted state
  emit mismatch report
  optionally repair idempotently

Mismatch types:

25. Testing Strategy

25.1 Unit Test Named Query Semantics

Fixture graph:

A owns 30% B
B owns 40% C
C owns 10% D
A owns 5% E
B owns 50% A  // cycle

Expected:

control(A, threshold=25, maxDepth=3) = B, C
D excluded because C->D below threshold
E excluded because A->E below threshold
cycle A->B->A ignored

25.2 Load Test Traversals

Test with:

1. high-degree vertices,
2. deep paths,
3. skewed relationship distribution,
4. concurrent query mix,
5. ingestion while querying,
6. replica reads,
7. timeout behavior,
8. result size boundaries.

25.3 Projection Test

Events:

create vertex
create edge
update edge property
older event arrives
delete/tombstone edge
duplicate event arrives

Expected:

idempotent final graph state
sourceVersion monotonic
no duplicate edge
inactive relationship excluded by active query

26. Deployment and Schema Evolution

Graph schema evolves through labels/properties/queries/projectors.

26.1 Safe Evolution

1. add new property to projector
2. support missing property in query
3. backfill graph
4. validate query results
5. switch named query version
6. deprecate old property

26.2 Query Versioning

indirectOwnership.v1 = OWNS edges only
indirectOwnership.v2 = OWNS + CONTROLS edges with confidence threshold

API can expose:

GET /entities/{id}/ownership-chain?modelVersion=v2

For regulatory systems, record which model version produced a decision.

27. Incident Runbooks

27.1 Slow Graph Query

1. identify named query and parameters
2. check start vertex cardinality
3. check depth/result limit
4. inspect high-degree vertices in path
5. profile/explain query
6. disable risky query variant if needed
7. add temporary lower result cap
8. deploy tuned query
9. add regression test

27.2 Projection Lag

1. check event source lag
2. check SQS queue age/depth
3. check projector errors
4. isolate poison event
5. replay from checkpoint if safe
6. mark graph freshness degraded
7. run reconciliation after recovery

27.3 Wrong Relationship Result

1. capture query parameters and result
2. inspect source facts for involved vertices/edges
3. check graph edge sourceVersion
4. check temporal filters
5. check authorization redaction
6. repair projection if stale
7. patch query if semantics wrong
8. write fixture regression test

27.4 High-Degree Vertex Incident

1. find vertex causing fanout
2. identify queries traversing through it
3. cap result/depth or block query temporarily
4. add selective edge filters
5. consider model refactor or precomputed projection

28. Cost and Capacity Thinking

Graph cost is driven by:

1. instance size,
2. read replicas,
3. storage,
4. I/O/query workload,
5. bulk load/rebuild frequency,
6. inefficient traversals,
7. overconnected model,
8. projection duplicate/bloat,
9. cross-AZ/network patterns,
10. operational environments.

Cost trap:

A single unbounded traversal can consume more capacity than thousands of targeted lookups.

Capacity worksheet:

29. Anti-Patterns

29.1 Graph for Everything

Not every relationship deserves graph database.

User has orders
Order has items
Item has product

Relational/document model may be simpler.

29.2 Unbounded Public Query

Letting users submit arbitrary Gremlin/openCypher/SPARQL is dangerous.

29.3 No Source Lineage

A graph edge without source/version/confidence is hard to defend.

29.4 Modeling Generic Hubs

A vertex like Indonesia connected to every company may destroy traversal selectivity if used in hot paths.

29.5 Graph as Search Engine

Use OpenSearch for text/relevance.

29.6 Missing Temporal Semantics

If past relationships matter, active-only graph is insufficient.

29.7 No Rebuild Strategy

If you cannot rebuild the graph, projection corruption becomes permanent.

30. Production Readiness Checklist

Modeling

• Graph questions documented.
• Property graph vs RDF decision recorded.
• Vertex labels and identities defined.
• Edge labels, direction, properties, and lifecycle defined.
• High-degree vertex risks identified.
• Temporal relationship strategy defined.
• Source lineage properties included.

Query

• Named query catalog exists.
• Max depth/result limit defined per query.
• Authorization policy defined per query.
• Query fixtures and expected paths tested.
• Slow query/profile artifacts stored.
• Frontend cannot submit arbitrary graph queries.

Projection

• Source of truth documented.
• Projector idempotent.
• Edge identity deterministic.
• Source version guard implemented.
• Rebuild path tested.
• Reconciliation job exists.
• Projection lag monitored.

Operability

• Query latency dashboard by named query.
• Error/timeout alarms.
• Projector lag and failure alarms.
• Storage/vertex/edge growth tracked.
• High-degree vertex report.
• Backup/restore/rebuild drill.
• Incident runbooks tested.

Governance

• Sensitive relationship classification.
• Purpose-of-access logging.
• Query result redaction.
• Tenant/jurisdiction isolation.
• Model version recorded for decisions.

31. Regulatory Case Study

Use case:

Find whether a company under investigation is indirectly controlled by sanctioned or high-risk parties through ownership, director, address, and bank account relationships.

Architecture:

Named queries:

Defensibility requirements:

1. Every edge has sourceSystem, sourceRecordId, sourceVersion.
2. Every query result logs query model version.
3. Every investigation access logs purpose and actor.
4. Relationship confidence is visible.
5. Temporal edge filter is explicit.
6. Result can be reproduced for audit.

32. Heuristics

1. A graph is valuable when relationship traversal is the workload, not merely when data is related.
2. Start with graph questions, not entity nouns.
3. Bound every traversal.
4. Filter early, especially before high-fanout expansion.
5. Treat high-degree vertices as hot partitions.
6. Put graph queries behind named API operations.
7. Keep source lineage on every important edge.
8. Prefer graph projection when authoritative state lives elsewhere.
9. Make projectors idempotent and rebuildable.
10. If a graph result affects enforcement decisions, record query version and input time.

33. References

• Amazon Neptune documentation: https://docs.aws.amazon.com/neptune/
• What is Amazon Neptune: https://docs.aws.amazon.com/neptune/latest/userguide/intro.html
• Neptune graph data model: https://docs.aws.amazon.com/neptune/latest/userguide/feature-overview-data-model.html
• Querying a Neptune graph: https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-queries.html
• Accessing Neptune graph with openCypher: https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-opencypher.html
• General best practices for Gremlin with Neptune: https://docs.aws.amazon.com/neptune/latest/userguide/best-practices-gremlin.html
• Neptune best practices using SPARQL: https://docs.aws.amazon.com/neptune/latest/userguide/best-practices-sparql.html
• Amazon Neptune ML: https://docs.aws.amazon.com/neptune/latest/userguide/machine-learning.html

34. Penutup

Amazon Neptune adalah pilihan kuat untuk workload yang relationship-heavy: fraud, investigation, ownership chain, knowledge graph, identity graph, dependency graph, dan recommendation. Tapi graph database menuntut discipline berbeda.

Jangan berpikir:

"Kita punya banyak relasi, jadi pakai graph."

Berpikirlah:

"Pertanyaan utama kita adalah traversal dan path. Relasi adalah data utama. Kita bisa membatasi query, menjaga lineage, dan mengoperasikan projection dengan aman."

Graph yang baik bukan hanya model yang ekspresif. Graph yang baik adalah graph yang:

semantically clear
query-bounded
authorization-aware
lineage-rich
idempotently projected
rebuildable
observable
defensible

Itulah perbedaan antara graph demo yang terlihat menarik dan graph system yang bisa hidup di production.