Part 043 — Domain Events, Integration Events, Event Contracts, Ordering, Replay, and Event-Driven Operations

Positioning

Enterprise Quote-to-Order menghasilkan banyak business facts:

• Catalog publication activated;
• configuration validated;
• price calculated;
• Quote submitted;
• approval granted;
• Offer presented;
• Offer accepted;
• Agreement created;
• Product Order submitted;
• fulfillment node completed;
• Product activated;
• Billing charge created.

Event-driven architecture dapat mengurangi temporal coupling dan memungkinkan:

• independent consumers;
• long-running processes;
• audit;
• analytics;
• reconciliation;
• dan scalable fan-out.

Namun event-driven architecture juga dapat memperburuk sistem jika event:

• tidak memiliki semantic owner;
• hanya mereplikasi database rows;
• berubah tanpa versioning;
• mengandalkan global ordering;
• tidak idempotent;
• atau dipakai untuk menghindari boundary decisions.

Core thesis: event adalah published business fact dengan owner, identity, schema, ordering key, delivery contract, retention, dan evolution policy. Event bukan remote command terselubung, bukan database row dump, dan bukan pengganti explicit domain authority.

1. Event-Driven Architecture

Event-Driven Architecture adalah architectural style di mana components berkomunikasi melalui immutable facts mengenai perubahan atau kejadian.

2. Event

Event menyatakan sesuatu yang telah terjadi.

Contoh:

OfferAccepted
ProductOrderSubmitted
ProductActivated
BillingChargeCreated

3. Event Tense

Gunakan past tense untuk facts.

4. Command versus Event

Command

Meminta sesuatu dilakukan.

CreateProductOrder
ActivateBillingCharge

Event

Menyatakan sesuatu telah terjadi.

ProductOrderCreated
BillingChargeActivated

5. Query versus Event

Query meminta informasi.

Event menyebarkan fact.

6. Notification versus Event

Notification ditujukan untuk channel/recipient.

Event adalah domain/integration fact yang dapat memiliki banyak consumers.

7. Signal versus Event

Signal dapat menunjukkan technical condition.

Example:

PricingWorkerHeartbeatMissed

Tidak semua signal adalah domain event.

8. Domain Event

Domain Event adalah fact bermakna di dalam bounded context.

9. Integration Event

Integration Event adalah published contract yang ditujukan untuk external contexts/consumers.

10. Domain Event versus Integration Event

11. Internal Event

Bisa digunakan untuk:

• aggregate side effect;
• local projection;
• workflow continuation;
• or audit.

12. External Event

Published to:

• another bounded context;
• partner;
• analytics;
• or external customer.

13. Event Ownership

Publisher bounded context owns:

• event meaning;
• schema;
• lifecycle;
• and deprecation.

14. Fact Ownership

Event publisher should be authoritative for the fact.

15. Event Producer

Technical component that publishes event.

Producer may be adapter/worker, but semantic owner remains context.

16. Event Consumer

Consumer subscribes and reacts.

17. Consumer Autonomy

Consumer should not require synchronous acknowledgement for publisher transaction.

18. Event Contract

Defines:

• event identity;
• event type;
• source;
• aggregate/resource identity;
• version;
• timestamp;
• payload;
• ordering;
• delivery;
• security;
• and evolution.

19. Event Envelope

Representative envelope:

{
  "eventId": "evt-...",
  "eventType": "OfferAccepted",
  "eventVersion": 2,
  "occurredAt": "2026-07-10T09:30:00+07:00",
  "recordedAt": "2026-07-10T09:30:01+07:00",
  "tenantId": "tenant-1",
  "aggregateType": "Offer",
  "aggregateId": "OFF-123",
  "aggregateVersion": 17,
  "correlationId": "corr-...",
  "causationId": "cmd-...",
  "payload": {}
}

20. Event ID

Globally unique event occurrence identity.

21. Event Type

Stable semantic name.

22. Event Version

Schema/semantic contract version.

23. Aggregate Version

Ordering/optimistic state sequence for one aggregate.

24. Occurred At

When business fact became true.

25. Recorded At

When event was persisted/recorded.

26. Published At

When broker received/published event.

These times may differ.

27. Tenant ID

Mandatory where multi-tenancy applies.

28. Correlation ID

Groups related operations in one business flow.

29. Causation ID

Identifies command/event that caused this event.

30. Trace Context

Can propagate distributed tracing identifiers.

31. Source

Identify authoritative context/service.

32. Subject

Optional resource/aggregate reference.

33. Partition Key

Controls broker partitioning and relative ordering.

34. Schema Reference

May identify schema registry subject/version.

35. Payload

Contains minimum useful business data.

36. Payload Reference

For large/sensitive payloads, publish reference instead of full snapshot.

37. Event Granularity

Event can be:

• coarse resource lifecycle;
• item-level;
• component-level;
• or process-level.

38. Too Coarse

Example:

QuoteChanged

Consumer cannot know relevant business fact.

39. Too Fine

Example:

QuoteDescriptionCharacterAdded

No stable business value.

40. Business-Significant Granularity

Prefer:

• QuoteSubmitted;
• QuoteRevisionFinalized;
• OfferPresented;
• OfferAccepted;
• ProductOrderItemCompleted.

41. Event Naming

Use domain language, not database language.

42. Database Event Smell

Examples:

• QuoteRowInserted;
• StatusColumnUpdated;
• OrderTableChanged.

43. CRUD Event Smell

Generic Created/Updated/Deleted can be insufficient for complex lifecycle.

44. Lifecycle Event

Examples:

• QuoteApproved;
• ProductOrderCancelled;
• ProductTerminated.

45. Decision Event

Examples:

• PromotionQualified;
• ApprovalRejected;
• OrderChangeApproved.

46. Evidence Event

Examples:

• AcceptanceEvidenceRecorded;
• CompletionEvidenceVerified.

47. Process Event

Examples:

• QuoteToOrderTransformationCompleted;
• FulfillmentPlanPublished.

48. Failure Event

Examples:

• BillingChargeActivationFailed;
• ProductOrderFalloutDetected.

49. Failure Event Security

Avoid leaking stack traces or sensitive internal details.

50. Event Fact versus Request

Event should not say:

PleaseCreateOrder

That is a command.

51. Event as Trigger

A consumer may treat fact as trigger for local command.

52. Choreography

Contexts react to events and produce new events.

53. Orchestration

A coordinator consumes events and issues commands.

54. Hybrid Event-Driven Process

Common pattern:

Domain Event
-> Process Manager
-> Command
-> Domain Event

55. Event Storming

Useful to discover events, commands, actors, policies, and aggregates.

56. Event Taxonomy for Quote-to-Order

Candidate event groups:

• Catalog;
• Configuration;
• Pricing;
• Quote;
• Approval;
• Proposal;
• Acceptance;
• Agreement;
• Product Order;
• Fulfillment;
• Inventory;
• Billing;
• and Recovery.

57. Catalog Events

Examples:

• CatalogPublicationActivated;
• ProductOfferingPublished;
• ProductOfferingRetired;
• PriceDefinitionChanged.

58. Catalog Event Consumer Risk

Existing accepted Quote should not reinterpret using latest catalog event.

59. Configuration Events

Examples:

• ConfigurationSessionStarted;
• ConfigurationValidated;
• ConfigurationCompleted;
• ConfigurationExpired.

60. Pricing Events

Examples:

• PriceEvaluationRequested;
• PriceSnapshotCreated;
• PricingFailed;
• RepricingRequired.

61. Quote Events

Examples:

• QuoteCreated;
• QuoteRevisionCreated;
• QuoteSubmitted;
• QuoteApproved;
• QuoteFinalized;
• QuoteExpired.

62. Proposal Events

Examples:

• ProposalGenerationRequested;
• ProposalGenerated;
• ProposalPublished;
• ProposalWithdrawn.

63. Acceptance Events

Examples:

• OfferPresented;
• OfferAccepted;
• OfferDeclined;
• OfferExpired;
• OfferWithdrawn;
• CounterofferSubmitted.

64. Agreement Events

Examples:

• AgreementCreationRequested;
• AgreementCreated;
• AgreementAmended;
• AgreementActivated;
• AgreementTerminated.

65. Product Order Events

Examples:

• ProductOrderCreated;
• ProductOrderSubmitted;
• ProductOrderAcknowledged;
• ProductOrderItemStarted;
• ProductOrderItemCompleted;
• ProductOrderPartiallyCompleted;
• ProductOrderCancelled.

66. Fulfillment Events

Examples:

• FulfillmentPlanCreated;
• FulfillmentNodeDispatched;
• FulfillmentNodeCompleted;
• FulfillmentNodeFailed;
• FulfillmentPlanReplanned.

67. Inventory Events

Examples:

• PlannedProductCreated;
• ProductActivated;
• ProductModified;
• ProductSuspended;
• ProductTerminated;
• ProductInventoryCorrectionApplied.

68. Billing Events

Examples:

• BillingHandoffRequested;
• BillingChargeActivated;
• BillingChargeStopped;
• BillingChargeAdjusted;
• InvoicePosted.

69. Recovery Events

Examples:

• OrderFalloutDetected;
• RecoveryPlanned;
• RecoveryAttemptFailed;
• OrderFalloutResolved.

70. Event Source of Truth

Event does not automatically become source of truth unless architecture intentionally uses event sourcing.

71. Event Notification Architecture

State stored in database; events notify changes.

72. Event-Carried State Transfer

Event includes enough state for consumer projection.

73. Event Sourcing

Aggregate state reconstructed from event history.

74. Event Sourcing Is Not EDA

A system can use EDA without event sourcing.

75. Event Sourcing Suitability

Potential benefits:

• complete history;
• temporal queries;
• replay;
• and audit.

Costs:

• schema evolution;
• replay complexity;
• event-store operations;
• and debugging.

76. Event-Sourced Aggregate

State derived from ordered events for one aggregate.

77. Snapshot

Speeds event-sourced aggregate rehydration.

78. Snapshot Version

Must match event sequence.

79. Snapshot Rebuild

Can be regenerated from canonical event history.

80. Event Store

Append-only source for event-sourced aggregates.

81. Broker Is Not Necessarily Event Store

Broker retention/compaction semantics may not meet canonical history needs.

82. Event Log

Broker topic can be durable integration log, but ownership/retention must be explicit.

83. Kafka-Like Log

Supports partitions, offsets, retention, and consumer groups.

84. Queue versus Log

Queue

Work distribution, often message removed/acknowledged.

Log

Ordered retained stream consumed independently.

85. Topic

Logical stream of related event records.

86. Topic Design

Possible strategies:

• one topic per context;
• one topic per aggregate/event family;
• one shared enterprise topic;
• or hybrid.

87. One Enterprise Topic Smell

Creates:

• weak ownership;
• ACL complexity;
• schema chaos;
• and consumer filtering burden.

88. Too Many Topics

Creates operational overhead.

89. Context-Owned Topics

Often a strong default.

90. Event Family Topic

Example:

quote.events
product-order.events
product-inventory.events

91. Command Topic

Separate commands from facts.

92. Dead-Letter Topic

Technical parking, not business resolution.

93. Retry Topic

Can support delayed retry but requires clear ownership and ordering semantics.

94. Compacted Topic

Keeps latest value per key.

Useful for state projections, not complete audit history.

95. Retention

Define based on:

• replay needs;
• audit;
• reconciliation;
• consumer downtime;
• and cost.

96. Infinite Retention Myth

Operational and legal costs matter.

97. Data Classification

Retention and access depend on sensitivity.

98. Payload Size

Large events hurt:

• broker throughput;
• memory;
• replication;
• and consumer latency.

99. Claim Check Pattern

Store large payload externally and publish secure reference.

100. Claim Check Risks

• reference expiry;
• authorization;
• availability;
• and immutability.

101. Ordering

Distributed systems rarely provide global ordering.

102. Partition Ordering

Ordering usually guaranteed only within partition.

103. Aggregate Ordering Key

Use aggregate ID when consumers need lifecycle order.

104. Order Item Ordering Key

Use item ID if item progression is independent.

105. Tenant Ordering

Partitioning only by tenant can create hot partitions.

106. Global Ordering Smell

Requiring all events globally ordered harms scalability and availability.

107. Aggregate Version

Consumer can detect stale/out-of-order event.

108. Sequence Number

Monotonic within aggregate/stream.

109. Gap Detection

Consumer may detect missing sequence.

110. Gap Handling

Options:

• wait/buffer;
• query authority;
• replay;
• or mark projection incomplete.

111. Out-of-Order Event

Possible due to:

• multiple partitions;
• retries;
• producer concurrency;
• or cross-topic flow.

112. Late Event

Old event arrives after newer state.

113. Stale Event Guard

Do not overwrite current state if event version is older.

114. Terminal State Guard

Older event cannot reopen completed/cancelled state.

115. Timestamp Ordering Risk

Clock skew makes timestamps unsafe for strict ordering.

116. Event Delivery Semantics

Common delivery models:

• at-most-once;
• at-least-once;
• broker-level exactly-once;
• and effectively-once effects.

117. At-Most-Once

May lose messages, avoids redelivery.

118. At-Least-Once

Messages may repeat.

Consumers must be idempotent.

119. Exactly-Once Scope

Usually limited to one platform/transaction boundary.

120. Effectively-Once

Business effect applied once through:

• idempotency;
• uniqueness;
• sequence guards;
• and reconciliation.

121. Event ID Deduplication

Store processed event ID.

122. Logical Deduplication

Better for repeated events with different transport IDs.

Example:

acceptedChargeId + activationGeneration

123. Inbox Pattern

Consumer stores deduplication marker with local effect.

124. Inbox Transaction

Persist:

• processed event marker;
• local state;
• and outgoing outbox

atomically where possible.

125. Outbox Pattern

Producer stores event intent with domain transaction.

126. Outbox Publisher

Publishes asynchronously.

127. Duplicate Outbox Publish

Expected and handled by consumers.

128. Outbox Ordering

Preserve aggregate ordering where needed.

129. Outbox Polling

Simple but may add latency.

130. CDC Outbox

Change data capture publishes outbox rows.

131. CDC Risk

Operational complexity and schema coupling.

132. Dual Write Anti-Pattern

Database commit and broker publish separately.

133. Lost Event

DB commits but publish fails permanently.

134. Phantom Event

Publish succeeds but DB transaction rolls back.

135. Transactional Outbox Invariant

Domain state and event intent commit together.

136. Consumer Failure

Consumer may crash:

• before local commit;
• after local commit;
• before broker acknowledgement.

137. Ack after Commit

Supports at-least-once with deduplication.

138. Ack before Commit Risk

Message lost while effect not applied.

139. Poison Event

Deterministic processing failure.

140. Retry Policy

Classify:

• transient;
• permanent;
• schema;
• authorization;
• and business conflict.

141. Retry Storm

A bad event can repeatedly hit all instances.

142. Dead-Letter Handling

DLQ record should include:

• original event;
• error;
• attempts;
• consumer;
• and correlation.

143. DLQ Is Not Resolution

Business fact may remain unprocessed.

144. DLQ Ownership

Consumer domain/team owns remediation.

145. Replay

Reprocessing historical events for:

• projection rebuild;
• bug fix;
• new consumer;
• and recovery.

146. Replay versus Redelivery

Replay intentionally reprocesses history.

Redelivery repeats unacknowledged current processing.

147. Replay Safety

Consumer side effects must be:

• idempotent;
• disabled;
• or replay-aware.

148. Replay Mode

Could distinguish:

• projection rebuild;
• simulation;
• validation;
• and production effect replay.

149. External Side Effects during Replay

Never resend customer email or create external order unintentionally.

150. Replay Checkpoint

Track replay range and progress.

151. Replay Ordering

Respect partition/aggregate ordering.

152. Replay Version Compatibility

Old events may use old schema.

153. Upcaster

Transforms old event representation to current internal model.

154. Downcaster

Rarely used to serve older consumers; often better to publish versioned contract.

155. Event Versioning

Strategies:

• version in event type;
• schema version field;
• compatible evolution;
• new topic;
• or adapter.

156. Additive Field

Usually safe if consumer ignores unknown fields.

157. Required Field Addition

Breaking for old producers/replay.

158. Field Removal

Breaking unless optional/deprecated period.

159. Field Meaning Change

Always semantic breaking change.

160. Enum Evolution

Strict consumers may fail on new values.

161. Numeric Type Change

Potentially breaking.

162. Default Value Risk

Can hide semantic absence.

163. Schema Registry

Stores schema versions and compatibility rules.

164. Compatibility Modes

Conceptually:

• backward;
• forward;
• full;
• and none.

165. Backward Compatibility

New consumer reads old data.

166. Forward Compatibility

Old consumer reads new data.

167. Full Compatibility

Both directions.

168. Semantic Compatibility

Schema compatibility does not ensure meaning compatibility.

169. Event Contract Review

Review:

• domain owner;
• data classification;
• key;
• ordering;
• schema;
• compatibility;
• consumer impact;
• and retention.

170. Consumer Registry

Track:

• team;
• use case;
• event version;
• criticality;
• and contact.

171. Unknown Consumer Risk

Publishing public topic without registry complicates change.

172. Event Deprecation

Publish:

• deprecation date;
• replacement;
• migration guide;
• and sunset.

173. Dual Publish

During migration, producer may publish old and new events.

174. Dual Publish Risk

Consumers may process both and duplicate effects.

175. Bridge/Translator

Translate old topic/event to new contract.

176. Event Contract Adapter

Keeps internal event independent from public version.

177. Event-Carried State Transfer

Consumer builds local projection without synchronous query.

178. Thin Event

Carries only ID; consumer calls producer.

179. Thin Event Trade-Off

Creates temporal coupling and thundering herd.

180. Fat Event Trade-Off

Data duplication, privacy, and schema coupling.

181. Right-Sized Event

Carry stable data needed by known consumers, not entire aggregate.

182. Snapshot Event

Periodic full state event for projections.

183. Delta Event

Contains changed fields.

184. Delta Event Risk

Consumer must have all prior events.

185. Hybrid Snapshot + Delta

Useful for recovery.

186. Event Security

Events may contain sensitive:

• PII;
• pricing;
• contracts;
• topology;
• and financial details.

187. Topic ACL

Restrict producer and consumer identities.

188. Producer Authorization

Only owning context publishes authoritative event type.

189. Consumer Authorization

Consumers access only needed topics/data.

190. Encryption

Use in transit and at rest according to platform policy.

191. Field Encryption

May be used for highly sensitive values, with key-management complexity.

192. Data Minimization

Publish minimal required data.

193. Tenant Isolation

Tenant should be part of authorization and event validation.

194. Cross-Tenant Event Risk

A bad partition key or missing tenant filter can leak data.

195. PII Retention

Long broker retention may conflict with privacy obligations.

196. Right to Erasure

Immutable event history creates legal/architectural challenge.

Use:

• tokenization;
• reference indirection;
• cryptographic erasure;
• or minimized personal data

according to legal policy.

197. Event Integrity

Protect against tampering using platform controls/signatures where needed.

198. External Webhook

Event delivered over HTTP to partner.

199. Webhook Security

Use:

• signature;
• timestamp;
• nonce/event ID;
• retry;
• and replay protection.

200. Webhook Subscription

Store:

• subscriber;
• event filters;
• endpoint;
• secret/key;
• status;
• and delivery policy.

201. Webhook Delivery Attempt

First-class attempt identity.

202. Webhook Retry

Bounded with backoff.

203. Webhook Dead Letter

Requires operator/customer visibility.

204. Event Observability

Track:

• publish latency;
• consume latency;
• lag;
• retries;
• and failures.

205. Producer Metrics

• events produced;
• outbox backlog;
• publish failures;
• and duplicate publishes.

206. Consumer Metrics

• processed;
• failed;
• retry;
• DLQ;
• and processing latency.

207. Consumer Lag

Difference between latest broker offset and consumer offset.

208. Business Lag

Time between business fact and downstream applied effect.

209. End-to-End Event Latency

OccurredAt to consumer completion.

210. Projection Freshness

Last applied aggregate version/time.

211. Event Loss Detection

Compare authoritative state with expected emitted/consumed events.

212. Event Duplication Detection

Track repeated logical effects.

213. Ordering Violation Metric

Count stale/gap/out-of-order events.

214. Schema Failure Metric

Consumer deserialization/compatibility failures.

215. Event SLI

Examples:

• all committed domain facts reach outbox;
• critical consumer lag below target;
• zero unowned DLQ records;
• and all event-driven side effects idempotent.

Internal targets must be verified.

216. Event Trace

Distributed trace links:

command
-> domain transaction
-> outbox
-> broker
-> consumer
-> local effect

217. Trace Sampling

High-volume events may need adaptive sampling.

218. Correlation Search

Support should search by:

• Quote ID;
• Acceptance ID;
• Product Order ID;
• Product ID;
• Billing Charge ID;
• correlation ID;
• and event ID.

219. Event Catalog

Store:

• event type;
• owner;
• schema;
• topic;
• key;
• retention;
• classification;
• consumers;
• and runbook.

220. Event Documentation

Include examples and semantic notes.

221. AsyncAPI Documentation

Useful for topic/message contracts.

222. Event Contract Testing

Test:

• schema compatibility;
• semantic examples;
• producer;
• consumer;
• and replay.

223. Producer Contract Test

Verify event generated with required fields and semantics.

224. Consumer Contract Test

Verify consumer handles current and compatible future data.

225. Golden Event

Versioned representative event payload.

226. Property-Based Event Test

Properties:

• event ID unique;
• aggregate version monotonic;
• duplicate event creates one effect;
• and old event cannot regress state.

227. Replay Test

Rebuild projection from history.

228. Chaos Test

Inject:

• duplicate;
• delay;
• reorder;
• broker outage;
• and consumer crash.

229. Event-Driven Workflow Testing

Verify long-running sequence under redelivery and partial failure.

230. Eventual Consistency Test

Assert convergence, not immediate state.

231. Event Anti-Patterns

Event as Remote Procedure Call

Publisher expects immediate consumer action.

Database Row Event

Semantic meaning absent.

Generic EntityUpdated

Consumers inspect diff and infer business meaning.

Shared Topic without Ownership

Contracts become chaotic.

Consumer Calls Back for Every Event

Temporal coupling returns.

Infinite Retry

Poison events block progress.

Replay with Side Effects

Duplicate external actions.

Event Schema Equals Internal Class

Internal refactor becomes public break.

232. Event Smells

• no event ID;
• no aggregate version;
• no owner;
• mutable payload;
• and undocumented key.

233. Ordering Smells

• global order assumption;
• timestamp-only ordering;
• and no gap handling.

234. Delivery Smells

• exactly-once assumed;
• consumer not idempotent;
• and ack before commit.

235. Evolution Smells

• field meaning changed in place;
• no schema registry;
• and new enum crashes consumers.

236. Security Smells

• full proposal PDF in event;
• PII retained forever;
• and broad topic ACL.

237. Operations Smells

• DLQ no owner;
• lag monitored only technically;
• and no correlation from business ID.

238. Domain Event Template

## Event Name and Version

## Owning Context

## Business Meaning

## Trigger / Source Aggregate

## Event Identity

## Aggregate Identity / Version

## Occurred / Recorded Time

## Payload

## Ordering Key

## Security Classification

## Retention

## Consumers

## Compatibility / Deprecation

## Reconciliation

239. Event Envelope Template

eventId:
eventType:
eventVersion:
source:
tenantId:
aggregateType:
aggregateId:
aggregateVersion:
occurredAt:
recordedAt:
correlationId:
causationId:
traceContext:
payload:

240. Topic Template

Topic:
Owner:
Purpose:
Key:
Partitions:
Retention:
Compaction:
Schema subject:
Producer ACL:
Consumer ACL:
DLQ:
Runbook:

241. Consumer Template

Consumer:
Team owner:
Event types/versions:
Business purpose:
Idempotency key:
Ordering assumptions:
Retry policy:
DLQ policy:
Replay behavior:
Local effect:

242. Event Evolution Template

Current event/version:
Proposed change:
Schema compatibility:
Semantic compatibility:
Affected consumers:
Dual-publish/adapter:
Migration:
Deprecation:
Sunset:

243. Replay Plan Template

Replay purpose:
Topic/event range:
Consumer mode:
Side effects disabled/idempotent:
Ordering:
Checkpoint:
Expected projection:
Validation:
Rollback:

244. Event Reconciliation Template

Source fact:
Expected event:
Expected consumer effect:
Observed event/effect:
Window:
Classification:
Repair/replay:
Evidence:

245. Event Invariants

Representative invariants:

• event represents a fact owned by publisher context;
• event identity is globally unique;
• aggregate event version is monotonic;
• domain transaction and outbox intent commit atomically;
• consumer effects are idempotent;
• stale events cannot regress authoritative state;
• event evolution preserves declared compatibility;
• and replay cannot create unintended external side effects.

246. Worked Example: Offer Accepted

Quote/Offer context commits:

• Acceptance record;
• Offer state;
• outbox event.

Event:

OfferAccepted

Agreement and Quote-to-Order process managers consume idempotently.

247. Worked Example: Duplicate Acceptance Event

Broker redelivers event.

Agreement consumer uses Acceptance ID.

Product Order consumer uses Acceptance + conversion group.

No duplicates are created.

248. Worked Example: Out-of-Order Product Events

ProductModified v9 arrives before ProductActivated v8.

Projection buffers/queries authority and prevents regression.

249. Worked Example: Quote Event Schema Evolution

A new marketId field is added optional.

Older consumers ignore it.

New consumers can use it.

250. Worked Example: Breaking Semantic Change

validUntil changes from inclusive to exclusive.

Schema unchanged, but semantics break.

Publish new version and migration guide.

251. Worked Example: Consumer Replay

A new analytics projection replays Product Order events.

External side effects are absent; processing is deterministic and checkpointed.

252. Worked Example: Billing Projection Rebuild

Billing read projection rebuilds from charge lifecycle events.

It does not recreate Billing charges because side-effecting commands are not part of replay.

253. Worked Example: Poison Event

One event contains unsupported enum.

Consumer sends to owned DLQ after bounded retries and opens contract incident.

254. Worked Example: Event Loss

Outbox backlog reveals unpublished ProductActivated.

Publisher resumes; event is published.

Consumer applies idempotently.

255. Worked Example: Late Callback Event

Supplier completion event arrives for superseded attempt.

Attempt/generation guard records but does not complete current node.

256. Worked Example: Event-Carried State

Product Inventory publishes customer-visible Product snapshot.

Portal projection updates without synchronous Inventory call.

Sensitive technical references are omitted.

257. Worked Example: Webhook Delivery

Partner receives ProductOrderStateChanged.

Webhook is signed, retried, and deduplicated by event ID.

258. Worked Example: Hot Partition

All enterprise Orders keyed by tenant overload one partition.

Key changes to Order ID while tenant remains payload/security scope.

259. Worked Example: Shared Topic Migration

A giant enterprise topic is split by bounded context.

Bridge translates legacy consumers during migration.

260. Worked Example: Systemic Consumer Bug

Consumer applies duplicate charge activation.

Incident identifies event version and consumer deployment.

Containment pauses consumer, reconciles duplicates, and rolls out idempotency fix.

261. Senior Engineer Operating Model

Publish facts, not requests

Commands and events have different ownership.

Separate domain and integration events

Protect internal evolution.

Choose ordering scope deliberately

Usually aggregate or item, not global.

Expect duplicates and lag

Use outbox, inbox, idempotency, and reconciliation.

Version semantics, not only schema

Meaning changes are breaking.

Make replay safe

No uncontrolled external side effects.

Keep event payload right-sized

Avoid thin-event callback storms and fat-event leakage.

Operate streams

Lag, DLQ, outbox backlog, schema failures, and business latency.

Maintain event catalog and consumer registry

Ownership is part of correctness.

262. Internal Verification Checklist

Event taxonomy and ownership

• Apa domain events utama?
• Which events are internal versus integration contracts?
• Who owns each event type and topic?
• Are commands incorrectly represented as events?

Envelope and identity

• Are event ID, tenant, aggregate ID/version, correlation, and causation present?
• Are occurred, recorded, and published times distinguished?
• What is the partition/ordering key?
• Are large payloads referenced safely?

Delivery and idempotency

• Is delivery at-least-once?
• How do consumers deduplicate?
• Are inbox and outbox used?
• What business key protects each side effect?

Ordering and consistency

• What ordering guarantee exists?
• How are gaps, stale events, and late events handled?
• Can old events regress terminal state?
• Are timestamps wrongly used as total order?

Schema evolution

• Is there a schema registry?
• Which compatibility mode applies?
• How are semantic changes reviewed?
• Can consumers tolerate new enum values?

Replay and retention

• Which topics/events can be replayed?
• Are external side effects disabled/idempotent?
• What retention supports recovery/audit?
• How are historical schemas upcast?

Security and privacy

• Are topic ACLs and tenant checks enforced?
• Is PII minimized?
• How are external webhooks signed and replay-protected?
• What data-retention obligations apply?

Operations

• Are producer/consumer lag, DLQ, outbox backlog, and business latency visible?
• Does every DLQ have an owner/runbook?
• Can support search by business IDs and event IDs?
• What incidents reveal event-contract or ordering failures?

263. Practical Exercises

Exercise 1 — Event inventory

List 60 business events and classify domain versus integration.

Exercise 2 — Topic/key design

Choose topic, partition key, retention, and ACL for each context.

Exercise 3 — Idempotent consumer

Design inbox and local transaction for Product activation.

Exercise 4 — Evolution

Classify 25 event changes as compatible or breaking.

Exercise 5 — Replay

Rebuild a projection without resending external side effects.

Exercise 6 — Failure injection

Test duplicate, reorder, gap, poison event, broker outage, and consumer crash.

264. Part Completion Checklist

You are done if you can:

• distinguish command, domain event, integration event, notification, and signal;
• define event ownership and contracts;
• design event envelopes and partition keys;
• choose topic and retention strategy;
• handle at-least-once delivery idempotently;
• implement outbox and inbox patterns;
• detect gaps, stale, and out-of-order events;
• evolve schemas and semantics compatibly;
• replay safely;
• secure and operate event streams;
• and create an internal event-architecture verification backlog.

265. Key Takeaways

1. Events are immutable published facts.
2. Domain events and integration events should be separated.
3. Publisher context owns event meaning.
4. Ordering is scoped, usually not global.
5. At-least-once delivery requires idempotent consumers.
6. Outbox and inbox reduce dual-write and duplicate-effect risk.
7. Semantic changes can break even when schema does not.
8. Replay must not recreate external side effects.
9. DLQ is an operational queue, not business resolution.
10. Internal CSG events, topics, and delivery contracts must be verified.

266. References

Conceptual baseline:

• Event-Driven Architecture, domain events, integration events, event notification, event-carried state transfer, and event sourcing.
• Apache Kafka-style distributed logs, partitions, offsets, retention, compaction, consumer groups, and schema registries.
• Transactional outbox, inbox/deduplication, at-least-once delivery, effectively-once effects, and replay.
• Domain-Driven Design bounded contexts, aggregates, process managers, published languages, and anti-corruption layers.
• AsyncAPI, event-contract testing, webhook security, and operational stream observability.

These references do not define internal CSG event names, Kafka topology, schemas, retention, or ownership.