Part 049 — Domain Tests, Contract Tests, Migration, Backward Compatibility, Cloud/On-Prem, and Upgrade

Positioning

Perubahan pada enterprise CPQ dan Quote-to-Order jarang bersifat lokal.

Perubahan kecil pada:

• Catalog characteristic;
• rule precedence;
• rounding;
• Quote state;
• Product Order action;
• event schema;
• Inventory mapping;
• atau Billing effective-date policy

dapat memengaruhi lifecycle customer yang sudah berjalan selama hari, bulan, atau tahun.

Testing yang hanya memverifikasi HTTP 200 tidak cukup.

Deployment yang hanya memastikan pod baru hidup juga tidak cukup.

Migration yang hanya menghitung jumlah rows juga tidak cukup.

Core thesis: evolution strategy harus melindungi domain meaning dan lifecycle continuity. Verification harus mencakup invariants, decisions, state machines, contracts, replay, migrations, mixed versions, recovery, and operational evidence. Setiap perubahan harus memiliki compatibility horizon, rollout plan, rollback/forward-fix strategy, dan reconciliation.

1. Evolution Is a Domain Problem

Platform evolution mengubah:

• semantics;
• data;
• contracts;
• behavior;
• operational assumptions;
• and customer outcomes.

Karena itu, evolution tidak boleh diperlakukan hanya sebagai CI/CD concern.

2. Verification Pyramid

A useful verification pyramid contains:

1. static checks;
2. unit/value-object tests;
3. aggregate/state-machine tests;
4. rule/decision tests;
5. property/model-based tests;
6. component/integration tests;
7. contract tests;
8. replay/migration tests;
9. end-to-end tests;
10. production verification.

3. Test Portfolio

Tidak semua risiko membutuhkan jenis test yang sama.

Map risk to test.

4. Static Verification

Includes:

• compiler;
• type system;
• nullability;
• linters;
• architecture rules;
• schema validation;
• and dependency checks.

5. Architecture Tests

Verify boundaries such as:

• Quote module cannot write Inventory tables;
• domain package cannot depend on transport DTO;
• Billing adapter cannot depend on Quote aggregate internals;
• and API layer cannot bypass application commands.

6. Dependency Rules

Examples:

domain -> no framework dependency
application -> domain
adapter -> application/domain contracts
api -> application

Exact layering can vary.

7. Schema Linting

Validate:

• OpenAPI;
• AsyncAPI;
• JSON Schema;
• Avro/Protobuf;
• configuration schemas;
• and custom-field definitions.

8. Unit Test

Verifies one small behavior in isolation.

9. Value Object Test

Examples:

• Money;
• Quantity;
• Currency;
• Percentage;
• EffectivePeriod;
• CharacteristicValue;
• and ProductReference.

10. Money Tests

Verify:

• currency mismatch;
• scale;
• rounding;
• addition;
• allocation;
• negative values;
• and overflow.

11. Effective Period Tests

Verify:

• inclusive/exclusive boundaries;
• open-ended periods;
• overlap;
• gaps;
• and timezone conversions.

12. Aggregate Test

Executes commands against aggregate state and verifies:

• transition;
• invariant;
• version;
• emitted domain events;
• and rejection reason.

13. Aggregate Test Style

Prefer:

Given state/history
When command
Then state/events/error

14. Quote Aggregate Tests

Examples:

• Draft Quote can add Item;
• Finalized revision cannot mutate;
• Accepted Offer cannot accept twice;
• expired Offer rejects acceptance;
• material change invalidates approval;
• and superseded revision cannot be presented.

15. Product Order Aggregate Tests

Examples:

• invalid action rejected;
• DELETE requires existing Product reference;
• dependency relationship cannot point to unknown item;
• terminal state cannot regress;
• and cancellation preserves completed scope.

16. Product Inventory Aggregate Tests

Examples:

• stale Product version rejected;
• duplicate activation idempotent;
• terminated Product not reactivated by stale event;
• replacement preserves lineage;
• and correction differs from Product modification.

17. State-Machine Tests

State-machine verification should cover:

• every state;
• every allowed transition;
• every forbidden transition;
• terminal states;
• re-entry;
• timeout;
• supersession;
• and concurrency races.

18. Transition Matrix

Create explicit matrix:

19. Exhaustive Transition Test

For finite state machines, generate tests from matrix.

20. Forbidden Transition Test

A forbidden transition should fail with stable reason code.

21. Terminal-State Property

Once terminal, no ordinary event may regress state.

22. State-Age Timer Test

Verify timeout or escalation triggers at exact boundaries.

23. Clock Control

Inject business clock.

Do not depend on wall-clock sleep.

24. Temporal Tests

Use fixed timestamps and timezone.

25. DST and Timezone Tests

Where relevant, verify daylight-saving and local calendar behavior.

26. Effective-Dating Tests

Verify past/current/future versions and no invalid overlaps.

27. Decision-Table Tests

Best for:

• approval policy;
• eligibility;
• discount precedence;
• cancellation capability;
• and change materiality.

28. Decision Table Structure

Inputs × expected outcome × reason × rule version.

29. Boundary-Value Tests

Examples:

• discount exactly at threshold;
• margin exactly zero;
• quantity at tier edge;
• Offer expiration exact second;
• and approval authority exact limit.

30. Equivalence Classes

Reduce cases while preserving meaningful partitions.

31. Pairwise Testing

Useful for many configuration dimensions, but not sufficient for critical interactions.

32. Combinatorial Testing

Use t-way coverage for:

• market;
• channel;
• product;
• currency;
• tax;
• edition;
• and tenant config.

33. Representative Golden Scenarios

Golden scenarios are reviewed domain examples with exact expected results.

34. Golden Configuration Cases

Include:

• valid simple product;
• nested bundle;
• optional child;
• mutually exclusive options;
• dependency;
• cardinality edge;
• and invalid characteristic.

35. Golden Pricing Cases

Include:

• one-time;
• recurring;
• usage;
• tiered;
• discount;
• cross-item adjustment;
• tax estimate;
• proration;
• multi-currency;
• and rounding.

36. Golden Quote Cases

Include:

• revision;
• approval;
• proposal;
• acceptance;
• expiry;
• and withdrawal.

37. Golden Order Cases

Include:

• ADD;
• MODIFY;
• DELETE;
• REPLACE;
• partial completion;
• cancellation;
• and fallout recovery.

38. Golden Evidence

Store exact:

• input;
• semantic versions;
• expected components;
• reasons;
• and provenance.

39. Golden Test Review

Domain experts review expected outcomes, not only engineers.

40. Golden Drift

A changed golden output may be:

• intended product change;
• bug;
• data-version change;
• or nondeterminism.

Every drift needs classification.

41. Snapshot Test

Compares serialized output to baseline.

42. Snapshot Test Risk

Large opaque snapshots produce careless approvals.

43. Semantic Snapshot

Compare meaningful fields/components rather than full volatile payload.

44. Property-Based Testing

Generates many inputs and verifies invariants.

45. Pricing Properties

Examples:

• total equals sum of components under defined rounding;
• discount never exceeds allowed base unless explicitly credit;
• currency never changes silently;
• allocation sum equals source amount;
• and repeated evaluation is deterministic.

46. Configuration Properties

Examples:

• completed configuration satisfies all hard constraints;
• invalid cardinality never validates;
• adding unrelated item does not change independent item result;
• and same pinned inputs produce same output.

47. Quote Properties

Examples:

• accepted revision is immutable;
• one Offer has at most one effective Acceptance;
• superseded revision cannot become current;
• and state version increases monotonically.

48. Order Properties

Examples:

• duplicate conversion creates no duplicate Order;
• completed Item never returns to pending;
• compensation preserves residual effects;
• and every non-terminal Item has next-action semantics.

49. Model-Based Testing

Define a reference model and generate command sequences.

50. State Model

Tracks simplified legal states and invariants.

51. Command Sequence Generation

Generate:

• create;
• edit;
• submit;
• approve;
• expire;
• accept;
• cancel;
• amend;
• retry.

52. Concurrency Model Test

Generate competing commands.

53. Linearizability-Relevant Test

For local atomic operations such as Acceptance uniqueness.

54. Metamorphic Testing

Change input in known way and verify expected relation.

55. Pricing Metamorphic Example

Doubling quantity under simple linear price doubles base amount, unless tier/discount rule explicitly changes relation.

56. Reordering Property

Reordering independent items should not change total.

57. Unit Conversion Property

Equivalent units yield equivalent normalized result.

58. Differential Testing

Compare old and new implementations.

59. Shadow Evaluation

Run new engine against production-like inputs without affecting result.

60. Difference Classification

• identical;
• expected change;
• improved explanation only;
• defect;
• unsupported case;
• and unknown.

61. Approval Differential Test

New approval policy compared to current decisions over historical scenarios.

62. Pricing Differential Test

New pricing engine compared component by component.

63. Decomposition Differential Test

New planner compared by target outcomes, not necessarily identical internal nodes.

64. Mutation Testing

Mutates code to evaluate whether tests detect incorrect behavior.

65. Mutation Targets

Useful for:

• comparison operators;
• threshold boundaries;
• boolean rule branches;
• and state guards.

66. Component Test

Tests one bounded context with real database/message adapter where valuable.

67. Repository Test

Verify:

• mapping;
• constraints;
• optimistic locking;
• effective dating;
• and transaction behavior.

68. Database Constraint Test

Prove uniqueness and overlap constraints under concurrency.

69. Serialization Test

Verify stable API/event representations.

70. Adapter Test

Verify anti-corruption mapping to external domain.

71. External Dependency Stub

Must model:

• latency;
• retryable failure;
• rejection;
• timeout;
• duplicate;
• out-of-order callback;
• and unknown outcome.

72. Happy Stub Smell

Always-fast successful stub hides distributed risk.

73. Integration Test

Tests multiple actual components across boundary.

74. Integration Test Scope

Keep focused on contract/transaction behavior.

75. Testcontainers/Disposable Infrastructure

Can provide realistic DB/broker dependencies where supported.

76. Contract Test

Verifies provider/consumer compatibility.

77. API Provider Test

Provider conforms to declared API.

78. Consumer-Driven Contract

Consumer publishes expectations.

79. Event Contract Test

Verifies:

• schema;
• type;
• required fields;
• compatibility;
• ordering metadata;
• and semantic examples.

80. Error Contract Test

Verify status, error code, retryability, and redaction.

81. Idempotency Contract Test

Same key/payload gives same outcome.

Same key/different payload conflicts.

82. Concurrency Contract Test

Stale ETag/version rejected.

83. Pagination Contract Test

Stable ordering and continuation behavior.

84. Extension Contract Test

Unknown optional fields tolerated according to governance.

85. Backward Compatibility Test

New provider accepts old valid request.

86. Forward Compatibility Test

Old consumer handles new compatible response/event.

87. Mixed-Version Test

Old and new services coexist during rollout.

88. Compatibility Matrix

Track supported combinations of:

• API;
• event;
• DB schema;
• workflow;
• config;
• plugin;
• and application version.

89. End-to-End Test

Validates critical complete business flow.

90. E2E Test Scope

Use a small curated suite.

91. Core E2E Scenarios

• Configure–Price–Quote–Accept–Order;
• Product ADD to activation/Billing;
• Product MODIFY from Inventory baseline;
• cancellation;
• and fallout recovery.

92. E2E Test Evidence

Assert:

• state;
• monetary components;
• events;
• lineage;
• and audit.

93. E2E Fragility

Avoid coupling to volatile UI layout or timing.

94. UI Test

Focus on high-value user journeys and accessibility.

95. API-Level E2E

Often more stable for domain verification.

96. Synthetic Production Test

Safe test tenant/data verifies deployed path continuously.

97. Synthetic Safety

No real customer, supplier, or Billing effect.

98. Replay Test

Reprocess historical events or decisions.

99. Event Replay Test

Verify projection rebuild and old schema handling.

100. Workflow Replay Test

Reconstruct process state from durable history/checkpoints.

101. Decision Replay Test

Re-evaluate historical pricing/qualification/approval in shadow mode.

102. Replay Side-Effect Guard

No customer email, external Order, or Billing charge during replay.

103. Deterministic Replay

Same event/input/version yields same projection/decision where required.

104. Upcaster Test

Every historical event version transforms correctly.

105. Replay Range Test

Test from oldest retained history, not only recent events.

106. Data Migration Test

Verifies data transformation and lifecycle continuity.

107. Migration Inventory

List:

• tables;
• documents;
• events;
• files;
• caches;
• indexes;
• and external references.

108. Migration Scope

May include:

• schema;
• data;
• identity;
• ownership;
• state;
• effective time;
• and lineage.

109. Migration Invariant

Define what must remain true before and after.

110. Migration Precondition

Verify source state/data quality.

111. Migration Transformation

Versioned and deterministic.

112. Migration Output

Track per-record/item result.

113. Migration Evidence

Store:

• source checksum/count;
• target checksum/count;
• errors;
• version;
• and execution time.

114. Dry Run

Runs full transformation without authoritative cutover.

115. Rehearsal

Uses production-like volume and topology.

116. Migration Wave

Move bounded tenant/cohort/data set.

117. Canary Migration

Small low-risk scope first.

118. Migration Manifest

Lists exact data partitions and status.

119. Checkpoint

Enables resume without reprocessing all data.

120. Migration Idempotency

Rerun produces same target or safe no-op.

121. Migration Deduplication

Stable source identity prevents duplicates.

122. Source Freeze

Temporarily stop writes to simplify cutover.

123. Online Migration

Supports ongoing writes during migration.

124. Dual Read

Read old/new and compare.

125. Dual Write

Write both systems.

High risk; needs idempotency and reconciliation.

126. Change Capture

Capture source changes after baseline copy.

127. Backfill

Copy historical data.

128. Catch-Up

Apply changes accumulated during backfill.

129. Cutover

Switch authority/read/write path.

130. Post-Cutover Verification

Compare source/target and business invariants.

131. Source Retirement

Only after confidence and retention policy.

132. Expand–Migrate–Contract

A common zero-downtime data evolution pattern.

133. Expand

Add backward-compatible schema/capability.

134. Migrate

Populate/use new representation while old remains supported.

135. Contract

Remove old representation after all consumers migrate.

136. Column Rename

Use new column + dual compatible read/write + backfill + remove old.

137. Type Change

Add new typed field, transform, validate, switch, retire old.

138. Required Field Addition

Add nullable/default, backfill, validate, then enforce.

139. Table Split

Introduce new tables and compatibility view/adapter.

140. Table Merge

Preserve source identities and lineage.

141. Effective-Dated Migration

Avoid invalid overlaps/gaps.

142. State Migration

Map old lifecycle states to new state machine.

143. State Mapping Table

For every old state specify:

• new state;
• loss;
• follow-up;
• and invalid cases.

144. Ambiguous State

Route to reconciliation/manual classification.

145. History Migration

Do not collapse meaningful lifecycle history into current state only.

146. Identity Migration

Preserve stable IDs where possible.

147. ID Translation

Maintain mapping registry when IDs change.

148. External Reference Migration

Validate uniqueness and source system.

149. Tenant Migration

Preserve tenant scope, keys, residency, and routing.

150. Customer-Specific Migration

Account for custom:

• fields;
• rules;
• workflow;
• connectors;
• and historical anomalies.

151. Legacy Data Quality

Classify:

• valid;
• repairable;
• incomplete;
• duplicate;
• and unknown.

152. Migration Repair Policy

Do not silently invent commercial truth.

153. Legacy Provenance

Mark migrated/reconstructed/unknown source.

154. Data Reconciliation

Compare:

• counts;
• checksums;
• totals;
• states;
• relationships;
• effective periods;
• and lineage.

155. Business Reconciliation

Examples:

• accepted Quotes remain accepted;
• active Products remain active;
• active charges remain linked;
• and pending Orders retain next action.

156. Financial Reconciliation

Compare monetary totals and charge schedules.

157. Relationship Reconciliation

No orphan/missing parent-child links.

158. Temporal Reconciliation

No overlapping invalid effective periods.

159. Migration Rollback

Return authority to old path.

160. Rollback Feasibility

Difficult after new writes/features occur.

161. Forward Fix

Often safer than rollback after irreversible migration.

162. Point of No Return

Explicitly identify before cutover.

163. Backup

Necessary but not sufficient migration rollback.

164. Restore Time

May exceed acceptable outage.

165. Migration Abort Criteria

Examples:

• mismatch above threshold;
• critical invariant failure;
• unacceptable latency;
• and unknown data loss.

166. Migration Observability

Track:

• throughput;
• failure;
• lag;
• reconciliation;
• and ETA confidence.

167. Migration Runbook

Defines preflight, execution, abort, recovery, and validation.

168. Migration Ownership

Named domain/data/operations owners.

169. Deployment

Delivery of executable/config/schema changes to environment.

170. Deployment Unit

Could be:

• application;
• service;
• worker;
• schema;
• config;
• rule;
• workflow;
• plugin;
• and connector.

171. Release

Business-visible set of changes.

172. Deploy versus Release

Code can be deployed but feature not released.

173. Feature Flag Release

Separates deployment from activation.

174. Configuration Release

Config/rule/workflow publication is a production change.

175. Database Release

Schema/data changes need compatibility plan.

176. Event Contract Release

Schema compatibility and consumer readiness.

177. Deployment Strategies

• rolling;
• blue-green;
• canary;
• recreate;
• and shadow/dark launch.

178. Rolling Deployment

Instances replaced gradually.

Requires mixed-version compatibility.

179. Blue-Green

Two environments, switch traffic.

Data/schema remains shared challenge.

180. Canary

Small cohort/traffic gets new version.

181. Recreate

Stop old then start new.

May be acceptable for non-critical/on-prem maintenance.

182. Dark Launch

New capability executes without authoritative result.

183. Shadow Traffic

Duplicate safe request to new version.

184. Deployment Compatibility Window

During rolling/canary, old and new versions coexist.

185. Backward-Compatible Database

Old and new application versions work with schema.

186. Backward-Compatible Event

Old consumers handle new producer output.

187. Workflow Version Compatibility

Running workflow instances continue safely.

188. Long-Running Process Upgrade

Options:

• pin old definition;
• migrate instance;
• route by version;
• or restart with compensation.

189. Workflow Pinning

Default for safety.

190. Workflow Migration

Requires state mapping and deterministic migration.

191. Timer Migration

Preserve deadlines and duplicate prevention.

192. Active Attempt Migration

Avoid losing/duplicating external calls.

193. Rule Version Pinning

Existing Quote/Order retains version used.

194. Config Version Pinning

Long-running process behavior stays reproducible.

195. Connector Version Pinning

In-flight operation may need original mapping.

196. API Version Compatibility

Clients may upgrade independently.

197. Event Consumer Compatibility

Multiple consumer versions may exist.

198. Deployment Ordering

Sometimes required:

1. expand schema;
2. deploy tolerant readers;
3. deploy new writers;
4. backfill;
5. switch;
6. contract.

199. Consumer-First Change

When producer will add new behavior not tolerated by old consumers.

200. Producer-First Change

When consumers can tolerate additive event/API field.

201. Coordinated Release

Sometimes unavoidable, but minimize scope and rehearse.

202. Release Manifest

Records exact:

• application image;
• schema migration;
• config;
• rule;
• workflow;
• plugin;
• and contract versions.

203. Artifact Provenance

Build, sign, scan, and attest artifacts.

204. Immutable Artifact

Same artifact promoted across environments.

205. Environment Configuration

Separate from built artifact, but versioned.

206. Infrastructure as Code

Version runtime/network/storage configuration.

207. GitOps

Desired state through version-controlled reconciliation where used.

208. Drift Detection

Detect manual environment changes.

209. Secret Deployment

Use managed secrets, not repository plaintext.

210. Deployment Preflight

Check:

• compatibility;
• capacity;
• migration state;
• consumer versions;
• and rollback readiness.

211. Deployment Health

Technical and business health.

212. Business Release Gate

Examples:

• pricing golden tests pass;
• acceptance-to-order synthetic succeeds;
• no reconciliation regression;
• and canary business SLI healthy.

213. Smoke Test

Small critical checks immediately after deployment.

214. Canary Analysis

Compare control and canary by:

• error;
• latency;
• business decisions;
• mismatches;
• and resource use.

215. Automated Rollback

Safe only when rollback itself is compatible.

216. Rollback Risk

Old binary may not understand new data/event state.

217. Roll-Forward

Deploy corrective version.

218. Kill Switch

Disable risky feature/integration while keeping platform available.

219. Tenant-Scoped Rollout

Activate per tenant/cohort.

220. Market-Scoped Rollout

Useful for regional variation.

221. Large-Deal Safeguard

Exclude critical large Orders from early canary where appropriate, then test explicitly.

222. Release Freeze

May apply near business-critical dates.

223. Emergency Change

Still requires evidence, peer review, and follow-up.

224. Cloud Deployment

Often supports:

• frequent releases;
• centralized operations;
• automated rollout;
• and provider-controlled infrastructure.

225. On-Prem Deployment

Often includes:

• customer-controlled schedule;
• environment variation;
• limited telemetry;
• restricted access;
• and longer support windows.

226. Dedicated Customer Deployment

Between SaaS and on-prem characteristics.

227. Cloud Release Cadence

Can be continuous or scheduled.

228. On-Prem Upgrade Cadence

May be quarterly, annual, or customer-specific.

229. Version Skew

On-prem customers may run several old versions.

230. Support Matrix

Define supported:

• product versions;
• Java/runtime;
• database;
• Kubernetes/platform;
• plugins;
• connectors;
• and API/event versions.

231. Long-Term Support Version

A stabilized version with longer maintenance.

232. Backport Policy

Which fixes are backported and to which branches.

233. Security Patch Policy

Critical fixes need defined response across cloud/on-prem.

234. Upgrade Package

Contains:

• binaries/images;
• schema migrations;
• config changes;
• compatibility report;
• and runbooks.

235. Upgrade Precheck

Validate:

• source version;
• database;
• disk;
• configuration;
• custom extensions;
• and data quality.

236. Upgrade Rehearsal

Customer-like environment and data volume.

237. Upgrade Dry Run

Reports issues without authoritative mutation where possible.

238. Upgrade Backup

Verified and restorable.

239. Upgrade Checkpoint

Resume or diagnose step.

240. Upgrade Postcheck

Business invariants, not only service health.

241. On-Prem Telemetry

May require opt-in, offline bundles, or local diagnostics.

242. Diagnostic Bundle

Contains redacted:

• versions;
• health;
• config metadata;
• logs;
• and operation timelines.

243. Air-Gapped Environment

Needs offline:

• artifacts;
• signatures;
• dependencies;
• and documentation.

244. Customer Extension Compatibility

Upgrade must validate plugins/rules/connectors.

245. Customer Fork Compatibility

Expensive and risky; maintain explicit inventory.

246. Upgrade Blocker

Unsupported custom code or data anomaly.

247. Upgrade Waiver

Explicit risk acceptance with owner and expiry.

248. Upgrade Rollback

May require database restore; test realistic duration.

249. Data Downgrade

Often unsupported.

Document clearly.

250. Compatibility Horizon

How long old clients/events/data remain supported.

251. Deprecation Policy

Define:

• announcement;
• replacement;
• migration guide;
• metrics;
• and sunset.

252. Usage Telemetry

Identify remaining old clients/features.

253. Deprecation Enforcement

Warnings, headers, dashboards, and eventually rejection.

254. Sunset Exception

Time-bound customer waiver.

255. Evolutionary Architecture

Architecture can change incrementally while preserving fitness functions.

256. Fitness Function

Automated check for desired architecture quality.

257. Domain Fitness Functions

Examples:

• one effective Acceptance per Offer;
• every active Product has lineage;
• no Billing Charge without accepted source;
• and all state transitions use explicit commands.

258. Contract Fitness Functions

Examples:

• no breaking API change;
• event compatibility maintained;
• and deprecated usage below threshold.

259. Operational Fitness Functions

Examples:

• all alerts have runbook;
• all services have owner;
• and recovery command available for known ambiguity.

260. Security Fitness Functions

Examples:

• cross-tenant negative tests;
• no secret in image;
• and privileged actions audited.

261. Performance Fitness Functions

Examples:

• large Quote pricing p95;
• memory envelope;
• and incremental/full equivalence.

262. Strangler Pattern

Replace legacy capability gradually behind boundary.

263. Strangler Steps

1. define target boundary;
2. observe current behavior;
3. create adapter/facade;
4. route small scope;
5. compare;
6. increase;
7. retire old path.

264. Branch by Abstraction

Introduce abstraction before switching implementation.

265. Parallel Run

Old and new execute for comparison.

266. Shadow Read

Compare new read model.

267. Shadow Write

High risk; isolate side effects.

268. Dual Authority Risk

During migration, exactly one source must be authoritative.

269. Authority Switch

Explicit cutover record and time.

270. Reconciliation during Transition

Continuous old/new comparison.

271. Modernization Slice

Choose end-to-end bounded capability, not random technical layer.

272. Thin Vertical Slice

Example:

One Product family
-> configuration
-> pricing
-> Quote
-> Order
-> Inventory

273. Modernization Sequencing

Prefer high-learning, bounded-risk slices.

274. Legacy Encapsulation

Place ACL around unstable legacy model.

275. Data Ownership Extraction

Stop new writes first, then migrate history.

276. Event Introduction

Publish domain facts from authoritative transaction via outbox.

277. API Introduction

Facade with stable contract.

278. Workflow Extraction

Move process state without moving all domain authority at once.

279. Rollout by Tenant/Product

Useful for contained modernization.

280. Modernization Exit Criteria

Old path no longer receives writes/traffic and evidence reconciles.

281. Test Data Management

High-quality test data is strategic.

282. Synthetic Data

Generated to cover domain combinations.

283. Anonymized Production Data

Only with privacy controls and residual-identification review.

284. Golden Dataset

Small curated exact scenarios.

285. Scale Dataset

Large realistic distributions.

286. Failure Dataset

Contains malformed, duplicate, incomplete, and stale data.

287. Historical Dataset

Supports replay and migration.

288. Data Factory

Creates repeatable domain entities and graphs.

289. Test Clock

Controls time.

290. Test ID Generator

Deterministic IDs where useful.

291. Test Isolation

No shared mutable tenant/state between tests.

292. Parallel Test Safety

Unique tenant/schema/resources.

293. Flaky Test

Non-deterministic pass/fail.

294. Flaky Test Causes

• real time;
• async timing;
• shared state;
• random without seed;
• and external dependency.

295. Flaky Test Policy

Quarantine is temporary; assign owner and deadline.

296. Eventual Assertion

Poll with bounded timeout and diagnostic output.

297. Sleep-Based Test Smell

Arbitrary sleep is slow and flaky.

298. Test Observability

On failure capture:

• state;
• events;
• logs;
• correlation;
• versions;
• and pending operations.

299. Test Execution Layers

• pull request;
• merge;
• nightly;
• pre-release;
• migration rehearsal;
• and production synthetic.

300. Fast Feedback

PR suite should be reliable and focused.

301. Nightly Deep Verification

Property/model/replay/large datasets.

302. Release Verification

Compatibility, migration, performance, and security.

303. Change Risk Classification

Classify change by affected:

• domain invariant;
• data;
• API/event;
• security;
• performance;
• and deployment.

304. High-Risk Change

Examples:

• monetary calculation;
• acceptance;
• state migration;
• multi-tenant keying;
• and Billing activation.

305. Change Verification Plan

Every high-risk change lists specific tests/evidence.

306. Traceability Matrix

Requirement/invariant -> code -> test -> deployment evidence.

307. Test Coverage

Line coverage is insufficient.

308. Domain Coverage

Track:

• states;
• transitions;
• rules;
• invariants;
• failure modes;
• and compatibility versions.

309. Contract Coverage

Operations, errors, auth, idempotency, and versions.

310. Migration Coverage

Every source classification and transformation branch.

311. Failure-Mode Coverage

Timeout, duplicate, reorder, partial success, and unknown outcome.

312. Test Evidence Retention

Critical release/migration results may need retention.

313. Release Qualification

A release candidate passes defined quality gates.

314. Quality Gate

Possible:

• no critical test failure;
• compatibility pass;
• migration reconciliation pass;
• security scan pass;
• performance within budget;
• and runbooks ready.

315. Manual Exploratory Testing

Useful for complex UX and emergent behavior.

316. Domain Expert Acceptance

Review exact business outputs.

317. Operational Acceptance

Operations verifies monitoring, alerts, repair, and rollback.

318. Security Acceptance

Security verifies controls and risk.

319. Customer Upgrade Acceptance

For dedicated/on-prem, customer may validate in UAT.

320. UAT Risk

Customer UAT cannot replace provider verification.

321. Production Verification

After release:

• smoke;
• synthetic;
• canary metrics;
• reconciliation;
• and support readiness.

322. Release Observation Window

Do not immediately declare success.

323. Latent Failure

Some issues appear at:

• Offer expiry;
• month-end Billing;
• renewal;
• or long-running workflow.

324. Delayed Verification

Schedule checks for temporal behavior.

325. Rollback Decision

Based on:

• customer impact;
• correctness;
• migration reversibility;
• and time to forward fix.

326. Rollback Verification

Prove old version functions with current data/contracts.

327. Roll-Forward Verification

Corrective version plus reconciliation.

328. Incident-to-Test Feedback

Every escaped defect should produce appropriate regression/fitness test where feasible.

329. Test Smells

• mock everything;
• assert only status code;
• and no domain reasons.

330. Golden Test Smells

• giant opaque snapshots;
• no owner;
• and baseline updated automatically.

331. Contract Test Smells

• schema only;
• no semantics/errors/idempotency;
• and provider mocks itself.

332. Migration Smells

• row count only;
• no dry run;
• one giant transaction;
• and no resume.

333. Deployment Smells

• schema break before app compatibility;
• no mixed-version test;
• and rollback assumed.

334. On-Prem Smells

• “works in our cloud”;
• undocumented prerequisites;
• and no extension compatibility report.

335. Evolution Smells

• permanent dual write;
• two authorities;
• legacy never retired;
• and no deprecation telemetry.

336. Anti-Patterns

Test Only Happy Path

Distributed failures escape.

Mock Every Boundary

Real transaction/serialization behavior is unknown.

Snapshot Everything

Semantic changes are hidden in large diffs.

Big-Bang Migration

No bounded learning or rollback.

Breaking Schema First

Mixed-version deployment fails.

Rollback by Hope

New data cannot be read by old code.

Customer UAT as Quality Strategy

Provider defects reach customer.

Permanent Compatibility Layer

Legacy complexity never retires.

Fork Every On-Prem Customer

Upgrade and security support collapse.

337. Verification Strategy Template

## Change / Risk

## Affected Invariants

## Unit / Aggregate / State Tests

## Decision / Golden Scenarios

## Property / Model Tests

## Component / Integration Tests

## API / Event Contracts

## Replay / Migration

## Performance / Security

## E2E / Synthetic

## Production Verification

338. Golden Scenario Template

Scenario ID:
Business narrative:
Inputs:
Catalog/config/rule versions:
Expected decisions:
Expected prices/components:
Expected state/events:
Expected evidence:
Owner/approval:

339. State-Machine Test Template

Initial state/version:
Command/event:
Actor/context:
Clock:
Expected state:
Expected event:
Expected reason/evidence:
Forbidden alternatives:

340. Migration Plan Template

## Source / Target / Authority

## Invariants

## Data Classification

## Mapping / Transformation Version

## Dry Run / Rehearsal

## Backfill / Change Capture

## Checkpoints / Idempotency

## Cutover

## Reconciliation

## Abort / Rollback / Forward Fix

## Retention / Retirement

## Owners / Runbook

341. Deployment Compatibility Template

342. Release Manifest Template

Release:
Application artifacts:
Database migrations:
API/event schemas:
Config/rules:
Workflow definitions:
Plugins/connectors:
Feature flags:
Compatibility window:
Rollback/forward-fix:
Evidence links:

343. Upgrade Runbook Template

## Supported Source / Target Versions

## Preconditions

## Backup / Restore Test

## Custom Extension Compatibility

## Dry Run / Precheck

## Migration Steps

## Checkpoints

## Health and Business Validation

## Abort / Rollback

## Post-Upgrade Reconciliation

## Diagnostic Bundle

## Owner / Escalation

344. Fitness Function Template

Quality attribute:
Invariant/target:
Automated check:
Execution cadence:
Owner:
Failure action:
Evidence:

345. Evolution Invariants

Representative invariants:

• old and new versions coexist safely during rollout where required;
• data migration preserves identity, state, effective time, and lineage;
• one authority exists at every migration phase;
• historical decisions remain reproducible;
• contract changes follow declared compatibility policy;
• running workflows retain valid definitions/state;
• rollback/forward-fix behavior is tested;
• and post-change reconciliation validates business outcomes.

346. Worked Example: Pricing Rule Change

Change:

• discount precedence.

Verification:

• decision-table boundaries;
• golden pricing cases;
• property tests;
• historical shadow evaluation;
• component performance;
• canary cohort;
• and Quote/Billing reconciliation.

347. Worked Example: Quote State Addition

New state PENDING_CUSTOMER_INFO.

Plan:

• additive enum handling;
• old consumer fallback;
• state-machine migration;
• dashboard/runbook update;
• and mixed-version test.

348. Worked Example: Event Field Addition

Add optional agreementId.

Verify:

• schema compatibility;
• old consumer tolerance;
• new consumer behavior;
• replay of historical events;
• and consumer registry.

349. Worked Example: Event Meaning Change

ProductActivated previously meant technical activation; new proposal means customer-ready.

This is semantic breaking change.

Publish new event/version instead of changing in place.

350. Worked Example: Column Split

price_total becomes structured charge table.

Use expand–migrate–contract:

• add table;
• dual compatible write/read;
• backfill;
• reconcile totals;
• switch;
• remove old column later.

351. Worked Example: State Migration

Legacy FAILED maps to:

• retryable;
• permanent;
• unknown;
• or fallout-open.

Ambiguous rows require classification rather than arbitrary mapping.

352. Worked Example: Product Inventory Migration

Preserve:

• Product ID;
• lifecycle;
• effective history;
• Order lineage;
• Billing reference;
• and tenant scope.

353. Worked Example: Cloud Rolling Deployment

v1 and v2 coexist.

DB and events remain backward-compatible.

Feature activates only after all instances and consumers are ready.

354. Worked Example: Workflow Upgrade

Running Product Orders stay on workflow v5.

New Orders use v6.

Selected safe instances migrate via explicit state map.

355. Worked Example: On-Prem Upgrade

Customer moves from v8 to v10.

Precheck finds custom connector incompatible with v10.

Upgrade blocks until certified connector version is installed.

356. Worked Example: Rollback Failure

New writer persists enum unknown to old app.

Automated rollback would crash old version.

Release uses forward fix and compatibility expansion instead.

357. Worked Example: Shadow Pricing Engine

New engine evaluates copied production inputs.

Differences grouped by expected policy changes versus defects.

No customer-facing result changes until approval.

358. Worked Example: Large-Deal Migration

10,000-site Quote is migrated in partitions with manifest/checkpoints.

Global totals and revision checksum reconcile before authority switch.

359. Worked Example: Tenant Data Move

Tenant moves database shard.

Baseline copy, change capture, routing switch, event offset handoff, and reconciliation prevent split authority.

360. Worked Example: Legacy API Deprecation

Telemetry identifies three remaining consumers.

Migration guide and adapter are provided.

Sunset occurs only after validated replacement.

361. Worked Example: Escaped Defect

Duplicate Billing Charge due to missing logical idempotency.

Regression suite adds:

• duplicate event;
• timeout;
• replay;
• and concurrency property tests.

362. Senior Engineer Operating Model

Test invariants, not implementation trivia

States, money, identity, and evidence.

Build representative golden scenarios

Reviewed by domain experts.

Use properties and models for combinatorial risk

Not only example cases.

Test real contracts and failure modes

Timeout, duplicate, reorder, and ambiguity.

Treat migration as a product lifecycle

Manifest, checkpoints, idempotency, and reconciliation.

Design deployment for mixed versions

Schema, events, workflows, and config.

Distinguish rollback from forward fix

Irreversible data changes matter.

Include cloud and on-prem compatibility

Version skew and custom extensions.

Use production fitness functions

Correctness, security, operations, and performance.

Feed incidents back into verification

Prevent recurrence.

363. Internal Verification Checklist

Domain verification

• Apa representative golden scenarios untuk configuration/pricing?
• Are state machines and forbidden transitions exhaustively tested?
• Which invariants use property/model-based tests?
• Are monetary results compared component-by-component?

Contracts

• Bagaimana API/event compatibility diuji?
• Are error, idempotency, concurrency, pagination, and enum evolution covered?
• Is there a consumer registry and compatibility matrix?
• Can historical events be replayed through current consumers/upcasters?

Failure behavior

• Are timeout, duplicate, out-of-order, partial success, and unknown outcome tested?
• Are retries and compensations idempotent under fault injection?
• Can workflows resume from durable checkpoints?
• Do chaos/game-day results become regression tests?

Migration

• Bagaimana customer-specific data migration direhearsal?
• Are identities, states, effective dates, relationships, and lineage preserved?
• Are dry run, canary waves, manifests, checkpoints, and reconciliation implemented?
• Is authority singular during dual-read/write transition?

Deployment

• Are database/API/event/workflow changes backward-compatible during rollout?
• Are mixed old/new application versions tested?
• Are business SLIs and correctness checks part of canary analysis?
• Is rollback actually compatible with new data?

Cloud/on-prem

• Apa perbedaan release/upgrade cloud dan on-prem?
• What versions, runtimes, databases, extensions, and connectors are supported?
• Are upgrade prechecks, diagnostic bundles, and offline/air-gapped processes available?
• How are critical security fixes delivered to older supported versions?

Evolution

• Which legacy capabilities are being strangled?
• Are fitness functions guarding architecture boundaries?
• Are deprecations measured and sunset deliberately?
• What technical compatibility layers have no retirement plan?

364. Practical Exercises

Exercise 1 — Verification pyramid

Map 100 critical behaviors to the cheapest reliable test level.

Exercise 2 — Golden pricing suite

Create exact cases for recurring, usage, discount, tax, and rounding.

Exercise 3 — State model

Generate command sequences for Quote and Product Order lifecycles.

Exercise 4 — Migration rehearsal

Design expand–migrate–contract for one high-volume table/event.

Exercise 5 — Mixed-version release

Prove app, DB, event, workflow, and config compatibility.

Exercise 6 — On-prem upgrade

Create precheck, rollback, diagnostics, and extension certification plan.

365. Part Completion Checklist

You are done if you can:

• build a domain-focused verification pyramid;
• test aggregates and state machines exhaustively;
• use golden, property, model, differential, and replay tests;
• verify API/event semantics and compatibility;
• model realistic distributed failures;
• plan idempotent, checkpointed, reconciled data migrations;
• use expand–migrate–contract safely;
• deploy under mixed-version conditions;
• distinguish cloud from dedicated/on-prem upgrades;
• define rollback and forward-fix strategies;
• create architecture fitness functions;
• and create an internal testing/migration/evolution verification backlog.

366. Key Takeaways

1. Testing must protect domain meaning, not only code paths.
2. Golden cases need exact semantic versions and expert ownership.
3. Property and model tests expose combinatorial lifecycle defects.
4. Contract compatibility includes behavior, errors, and semantics.
5. Replay tests are essential for event-driven evolution.
6. Migration needs identity, checkpoints, idempotency, and reconciliation.
7. Expand–migrate–contract enables safer zero-downtime evolution.
8. Rollback is not always possible after data/semantic changes.
9. Cloud and on-prem require different upgrade strategies.
10. Internal CSG verification, migration, and release mechanisms must be measured and verified.

367. References

Conceptual baseline:

• Test pyramid, state-machine testing, decision tables, golden-master/scenario testing, property-based testing, model-based testing, mutation testing, differential testing, and metamorphic testing.
• API/event consumer-driven contract testing, schema compatibility, replay testing, and mixed-version verification.
• Database and event migration patterns including expand–migrate–contract, backfill, change capture, dual-read/write, canary migration, checkpoints, and reconciliation.
• Rolling, blue-green, canary, dark launch, shadow traffic, feature flags, GitOps, artifact provenance, and release qualification.
• Evolutionary architecture, fitness functions, strangler, branch by abstraction, and modernization sequencing.

These references do not define internal CSG test suites, migration tooling, release cadence, cloud topology, on-prem support matrix, or upgrade process.