API Contract Testing: Provider Verification, Consumer Expectations, and Drift Detection
Learn Java API Contract Engineering, Event Contract Engineering & Schema Governance - Part 012
API contract testing for Java systems: OpenAPI validation, provider verification, consumer-driven contracts, Pact, Spring Cloud Contract, stubs, CI gates, and drift detection.
Part 012 — API Contract Testing: Provider Verification, Consumer Expectations, and Drift Detection
Tujuan Pembelajaran
Contract tanpa test adalah dokumen dengan harapan. Dalam sistem enterprise, kita membutuhkan bukti otomatis bahwa:
- provider benar-benar memenuhi contract;
- consumer expectations tidak rusak;
- runtime behavior tidak drift dari OpenAPI;
- generated clients tetap compile dan dapat dipakai;
- perubahan contract tidak breaking tanpa review;
- error, header, status, dan edge case ikut diuji.
Part ini membahas API contract testing di Java secara mendalam.
Setelah part ini, kamu harus mampu:
- membedakan unit, integration, end-to-end, schema validation, provider contract test, dan consumer-driven contract test;
- memilih OpenAPI validation, Pact, Spring Cloud Contract, atau kombinasi;
- membuat test strategy yang menghindari false confidence;
- menguji success response, error response, headers, content type, pagination, idempotency, dan concurrency;
- memasukkan contract diff dan generated client compile ke CI gate;
- mendesain stub publishing untuk consumer;
- membangun drift detection antara spec, server, docs, gateway, dan SDK.
1. Why Contract Testing Exists
Masalah distributed systems:
Provider deploys change.
Consumer compiles fine.
Integration environment looks okay.
Production consumer breaks because one field changed semantics.
Contract testing mencoba memindahkan failure dari production ke build pipeline.
Tujuan bukan mengganti semua integration tests. Tujuan adalah mempercepat feedback untuk boundary compatibility.
2. Testing Taxonomy
| Test type | Scope | Strength | Weakness |
|---|---|---|---|
| Unit test | class/function | fast, precise | no boundary confidence |
| Controller slice test | HTTP adapter | validates mapping/status/errors | may mock domain too much |
| OpenAPI schema validation | payload vs spec | catches shape drift | may miss semantics |
| Provider contract test | provider vs contract | proves provider implements spec | may not reflect real consumer usage |
| Consumer-driven contract test | consumer expectations vs provider | tests used interactions | may not cover unused but documented API |
| Integration test | real services or dependencies | good confidence | slower, brittle |
| End-to-end test | full workflow | highest scenario confidence | slow, expensive, hard to debug |
| Generated client compile test | SDK vs spec | catches generated breakage | not runtime behavior |
| Semantic regression test | documented behavior | catches non-schema changes | must be manually designed |
Good strategy combines several.
3. Contract Testing Mental Model
There are three truths:
Failures:
| Drift | Example |
|---|---|
| Contract vs provider | OpenAPI says 201, server returns 200 |
| Provider vs consumer | Consumer expects status, provider removed it |
| Consumer vs contract | Consumer relies on undocumented field |
| Contract vs docs | portal shows stale spec |
| Contract vs SDK | generated client cannot parse nullable field |
| Contract vs gateway | gateway requires header not documented |
Contract testing should target these drift lines.
4. OpenAPI-Based Provider Verification
OpenAPI-based provider verification checks runtime request/response against OpenAPI.
4.1 What to Verify
For each operation:
- method/path;
- request content type;
- request body schema;
- request headers;
- response status;
- response content type;
- response body schema;
- response headers;
- examples;
- error responses.
4.2 Example with MockMvc Concept
Pseudo-code:
@Test
void createCustomerShouldMatchOpenApiContract() {
mockMvc.perform(post("/customers")
.contentType("application/json")
.header("Idempotency-Key", "idem_123")
.content("""
{
"externalReference": "CRM-928812",
"fullName": "Ayu Lestari",
"birthDate": "1994-05-18"
}
"""))
.andExpect(status().isCreated())
.andExpect(header().exists("Location"))
.andExpect(content().contentType("application/json"))
.andExpect(openApi().isValid("createCustomer"));
}
The exact matcher depends on chosen library. The important practice: response must be validated against the OpenAPI operation.
4.3 Provider Verification Is Not Enough
If you only test happy path, provider can still break:
- validation errors;
- business errors;
- pagination;
- unknown input;
- idempotency conflict;
- concurrency conflict;
- rate limit;
- auth failures;
- nullable fields;
- edge enum values.
Contract tests must include negative and edge cases.
5. OpenAPI Example Validation
Examples in OpenAPI should be executable.
Example file:
src/test/resources/openapi/examples/customer-created.json
src/test/resources/openapi/examples/validation-problem.json
src/test/resources/openapi/examples/customer-not-eligible-problem.json
Test strategy:
@Test
void allExamplesShouldMatchReferencedSchemas() {
OpenApiDocument document = OpenApiDocument.load("customer-api.yaml");
document.examples().forEach(example -> {
Schema schema = document.schemaFor(example);
assertThat(schemaValidator.validate(example.value(), schema)).isEmpty();
});
}
This prevents docs drift.
Bad example:
example:
customerId: string
lifecycleStatus: true
Schema would catch lifecycleStatus boolean if it should be string.
6. Contract Diff Testing
Before testing runtime, detect dangerous changes statically.
Contract diff categories:
| Change | Gate |
|---|---|
| removed path | fail |
| removed operation | fail |
| removed response status | fail |
| removed response field | fail or review |
| added required request field | fail |
| changed type | fail |
| tightened min/max/pattern | fail/review |
| added enum value to closed enum | review/fail |
| changed error response content type | fail |
| added optional response field | pass |
| added operation | pass |
| added optional request field | pass/review |
CI flow:
Static diff cannot catch semantic changes. It catches structural changes early.
7. Consumer-Driven Contract Testing
Consumer-driven contract testing starts from consumer expectation.
The consumer says:
“When I send this request, I require this response shape/status/header.”
Provider later verifies it can satisfy that expectation.
Consumer-driven contracts are useful because they test what consumers actually use, not every documented possibility.
8. Pact Mental Model
Pact is a consumer-driven contract testing tool. The consumer test generates a contract from interactions used by the consumer. The provider then verifies those interactions.
Important characteristics:
- consumer writes tests against mock provider;
- contract artifact records interactions;
- provider verifies contract against real provider;
- Pact Broker can coordinate publication and verification;
- “can I deploy?” workflow checks compatibility.
8.1 Consumer Test Concept
Pseudo Java/JUnit style:
@Pact(consumer = "case-management-service", provider = "customer-service")
RequestResponsePact getCustomerPact(PactDslWithProvider builder) {
return builder
.given("customer cus_123 exists")
.uponReceiving("a request for customer cus_123")
.path("/customers/cus_123")
.method("GET")
.willRespondWith()
.status(200)
.headers(Map.of("Content-Type", "application/json"))
.body(newJsonBody(body -> {
body.stringValue("customerId", "cus_123");
body.stringType("lifecycleStatus", "ACTIVE");
}).build())
.toPact();
}
Consumer test:
@Test
void shouldLoadCustomerFromProvider(MockServer mockServer) {
CustomerClient client = CustomerClient.builder()
.baseUrl(mockServer.getUrl())
.build();
Customer customer = client.getCustomer(CustomerId.of("cus_123"));
assertThat(customer.lifecycleStatus())
.isEqualTo(CustomerLifecycleStatus.ACTIVE);
}
8.2 Provider Verification Concept
@TestTemplate
@ExtendWith(PactVerificationInvocationContextProvider.class)
void verifyPact(PactVerificationContext context) {
context.verifyInteraction();
}
Provider states are used to set up data:
@State("customer cus_123 exists")
void customerExists() {
testData.createCustomer("cus_123", "ACTIVE");
}
Exact annotations and setup depend on Pact JVM version and test framework. The model matters more than copy-paste.
8.3 Pact Strengths
- tests real consumer needs;
- reduces over-testing unused provider behavior;
- supports independent deployability;
- good for many consumers;
- provides verification workflow.
8.4 Pact Weaknesses
- consumer tests can encode poor expectations;
- provider state setup can be hard;
- contract sprawl if poorly governed;
- does not replace provider API spec;
- may miss undocumented but important platform guarantees;
- requires discipline around broker and versioning.
9. Spring Cloud Contract Mental Model
Spring Cloud Contract supports consumer-driven and producer-driven contract testing in the Spring ecosystem. It lets teams define contracts and generate tests/stubs.
Basic idea:
9.1 Contract Example Concept
YAML-style contract concept:
description: should create customer
request:
method: POST
url: /customers
headers:
Content-Type: application/json
body:
externalReference: CRM-928812
fullName: Ayu Lestari
birthDate: "1994-05-18"
response:
status: 201
headers:
Content-Type: application/json
Location: /customers/cus_123
body:
customerId: cus_123
lifecycleStatus: ACTIVE
Provider verification generates a test that calls provider and checks response.
Consumer side can use generated stubs.
9.2 Strengths
- strong Spring integration;
- generated provider tests;
- generated stubs;
- good for producer-owned contracts;
- works well in JVM/Spring organizations.
9.3 Weaknesses
- can duplicate OpenAPI if not integrated;
- DSL contract may become another source of truth;
- requires governance around contract ownership;
- may overfit to Spring ecosystem;
- generated tests can give false confidence if base setup is weak.
10. OpenAPI vs Pact vs Spring Cloud Contract
They solve overlapping but different problems.
| Tool/Approach | Best at | Not enough for |
|---|---|---|
| OpenAPI | public API description, schema, docs, generation | actual consumer expectations |
| OpenAPI validation | provider shape compliance | consumer usage and semantic expectations |
| Pact | consumer-driven interactions | full API documentation/catalog |
| Spring Cloud Contract | JVM/Spring contract tests/stubs | external API standard by itself |
| Integration tests | real service interaction | fast compatibility feedback |
| Generated client compile | SDK build compatibility | runtime behavior |
Recommended combinations:
| Context | Strategy |
|---|---|
| Public API | OpenAPI + provider validation + generated client tests |
| Internal multi-consumer API | OpenAPI + Pact or SCC + diff gates |
| Spring-only org | OpenAPI + Spring Cloud Contract |
| Many heterogeneous consumers | OpenAPI + Pact Broker + SDK tests |
| Regulated API | OpenAPI + provider verification + examples + audit trail |
| Prototype | code-first spec + smoke contract tests |
11. Provider Contract Test Matrix
For each operation, define matrix.
Example: POST /customers
| Scenario | Expected |
|---|---|
| valid create | 201 + CustomerResponse + Location |
| invalid JSON | 400 + Problem |
| missing required field | 400 + Problem violations |
| invalid birthDate format | 400 + Problem violations |
| birthDate future | 400/422 depending policy |
| duplicate externalReference | 409 or idempotent result |
| KYC/business rejection | 422 + reasonCode |
| unauthorized | 401 + Problem |
| forbidden | 403 + Problem |
| idempotency key conflict | 409 + Problem |
| dependency unavailable | 503 + retryable true |
Each scenario should be traceable to contract docs.
12. Testing Headers as Contract
Headers are often forgotten.
Test:
@Test
void createCustomerShouldReturnLocationAndCorrelationId() {
mockMvc.perform(post("/customers")
.contentType("application/json")
.header("X-Correlation-Id", "corr_test")
.content(validCreateCustomerJson()))
.andExpect(status().isCreated())
.andExpect(header().exists("Location"))
.andExpect(header().string("X-Correlation-Id", "corr_test"));
}
Headers to test:
Location;ETag;Retry-After;- correlation/request ID;
- deprecation/sunset;
- rate limit headers;
- content type;
- cache control;
Varywhen header versioning is used.
13. Testing Content Type
Do not only test status and body.
.andExpect(content().contentType("application/problem+json"))
For success:
.andExpect(content().contentType("application/json"))
If provider returns HTML error page under some failure path, generated clients may fail badly.
Contract test should catch this.
14. Testing Error Contracts
14.1 Validation Error
@Test
void missingFullNameShouldReturnValidationProblem() {
mockMvc.perform(post("/customers")
.contentType("application/json")
.content("""
{
"externalReference": "CRM-928812",
"birthDate": "1994-05-18"
}
"""))
.andExpect(status().isBadRequest())
.andExpect(content().contentType("application/problem+json"))
.andExpect(jsonPath("$.code").value("VALIDATION_FAILED"))
.andExpect(jsonPath("$.retryable").value(false))
.andExpect(jsonPath("$.violations[?(@.field == '/fullName')]").exists());
}
14.2 Business Error
@Test
void kycNotVerifiedShouldReturnCustomerNotEligible() {
givenCustomerKycStatus("cus_123", "PENDING");
mockMvc.perform(post("/customers/cus_123/accounts")
.contentType("application/json")
.content(validOpenAccountJson()))
.andExpect(status().isUnprocessableEntity())
.andExpect(content().contentType("application/problem+json"))
.andExpect(jsonPath("$.code").value("CUSTOMER_NOT_ELIGIBLE"))
.andExpect(jsonPath("$.reasonCode").value("KYC_NOT_VERIFIED"))
.andExpect(jsonPath("$.retryable").value(false));
}
14.3 Unexpected Error Does Not Leak
@Test
void unexpectedErrorShouldNotExposeStackTrace() {
forceUnexpectedError();
mockMvc.perform(post("/customers")
.contentType("application/json")
.content(validCreateCustomerJson()))
.andExpect(status().isInternalServerError())
.andExpect(content().contentType("application/problem+json"))
.andExpect(jsonPath("$.code").value("INTERNAL_ERROR"))
.andExpect(jsonPath("$.stackTrace").doesNotExist());
}
15. Testing Idempotency Contract
Scenarios:
| Scenario | Expected |
|---|---|
| same key + same payload | same result or documented replay response |
| same key + different payload | 409 idempotency conflict |
| missing required key | 400 |
| retry after timeout | no duplicate side effect |
| key expired | documented behavior |
Pseudo-test:
@Test
void sameIdempotencyKeyWithDifferentPayloadShouldFail() {
String key = "idem_123";
postCreateCustomer(key, createCustomerJson("CRM-1", "Ayu"))
.andExpect(status().isCreated());
postCreateCustomer(key, createCustomerJson("CRM-2", "Budi"))
.andExpect(status().isConflict())
.andExpect(jsonPath("$.code")
.value("IDEMPOTENCY_KEY_REUSED_WITH_DIFFERENT_PAYLOAD"));
}
This is more than schema test. It verifies behavior contract.
16. Testing Optimistic Concurrency
Scenarios:
- GET returns ETag;
- update with matching If-Match succeeds;
- update with stale If-Match returns 412;
- update without If-Match returns 428 or allowed depending policy;
- ETag changes after update.
Pseudo:
@Test
void staleIfMatchShouldReturnPreconditionFailed() {
Versioned<Customer> current = getCustomerVersioned("cus_123");
updateCustomerElsewhere("cus_123");
mockMvc.perform(patch("/customers/cus_123")
.header("If-Match", current.etag())
.contentType("application/json")
.content(updateProfileJson()))
.andExpect(status().isPreconditionFailed())
.andExpect(jsonPath("$.code").value("VERSION_MISMATCH"));
}
17. Testing Pagination Contract
Scenarios:
- first page returns
itemsandpage; - cursor is opaque;
hasMorecorrect;- nextCursor null when no more;
- limit max enforced;
- unsupported sort rejected;
- stable default sort;
- filter semantics.
Pseudo:
@Test
void searchCustomersShouldReturnPageWrapper() {
mockMvc.perform(get("/customers")
.param("limit", "50"))
.andExpect(status().isOk())
.andExpect(jsonPath("$.items").isArray())
.andExpect(jsonPath("$.page.hasMore").exists())
.andExpect(jsonPath("$.page.nextCursor").exists());
}
Semantic test for default sort:
@Test
void defaultSortShouldRemainCreatedAtDescending() {
List<CustomerResponse> items = api.searchCustomers(defaultSearch()).items();
assertThat(items)
.extracting(CustomerResponse::createdAt)
.isSortedAccordingTo(Comparator.reverseOrder());
}
18. Consumer Contract Tests for SDK
Consumer SDK should have tests against stubs/mock server.
18.1 Mock Server from OpenAPI Examples
Test SDK mapping without real provider:
@Test
void sdkShouldParseCustomerResponseFromStub() {
stubServer.stubGet(
"/customers/cus_123",
200,
"application/json",
"""
{
"customerId": "cus_123",
"lifecycleStatus": "ACTIVE",
"createdAt": "2026-06-29T02:15:00Z"
}
"""
);
Customer customer = client.getCustomer(CustomerId.of("cus_123"));
assertThat(customer.lifecycleStatus())
.isEqualTo(CustomerLifecycleStatus.ACTIVE);
}
18.2 Unknown Error Test
@Test
void sdkShouldHandleUnknownProblemCode() {
stubServer.stubGet(
"/customers/cus_123",
499,
"application/problem+json",
"""
{
"type": "https://api.acme.com/problems/new-problem",
"title": "New problem",
"status": 499,
"code": "NEW_PROBLEM",
"retryable": false,
"correlationId": "corr_123"
}
"""
);
assertThatThrownBy(() -> client.getCustomer(CustomerId.of("cus_123")))
.isInstanceOf(UnknownCustomerApiException.class);
}
This prevents SDK from crashing on future-compatible errors.
19. Stub Publishing
Stubs let consumers test without provider running.
Artifacts:
customer-service-stubs-1.18.0.jar
customer-api-openapi-1.18.0.yaml
customer-sdk-java-3.8.0.jar
Consumer build:
Benefits:
- faster consumer tests;
- fewer shared test environment dependencies;
- reproducible interactions;
- versioned contract artifacts.
Risks:
- stale stubs;
- stubs not matching provider;
- overly simplistic behavior;
- false confidence if stubs do not include errors;
- stub version mismatch.
Governance:
- stubs generated from approved contract;
- provider verification passes before publishing;
- stub version tied to contract version;
- consumers pin stub version;
- changelog explains changes.
20. Contract Broker / Registry Workflow
For Pact-like workflows:
Key concepts:
- consumer version;
- provider version;
- environment;
- tags/branches;
- verification result;
- deployability matrix.
This is useful for independent deployability, but requires operational ownership.
21. Avoiding False Confidence
Contract tests can lie.
21.1 Schema-Only False Confidence
Schema says:
riskScore:
type: integer
Provider changes range from 0-1000 to 0-100. Schema still passes.
Need semantic test.
21.2 Mocked Domain False Confidence
Controller test mocks application service with expected DTO. It proves controller mapping, not domain behavior.
Need at least some provider verification with real application path or carefully configured test slices.
21.3 Consumer Pact Too Weak
Consumer only asserts customerId, but production uses lifecycleStatus.
Contract will not protect that field.
Consumer tests must assert what consumer actually uses.
21.4 Stubs Too Happy-Path
Consumer never tests 409/422/429. Production breaks on errors.
21.5 Generated Client Compile Only
Compile proves types exist, not that runtime parsing and error handling work.
21.6 Environment Drift
Contract test uses local config, production gateway adds required header.
Test gateway-level contract or include gateway rules in contract governance.
22. Contract Test Data Management
Provider verification often needs states.
Examples:
customer cus_123 exists
customer cus_missing does not exist
customer cus_kyc_pending exists with KYC=PENDING
case case_123 exists in DRAFT
account acc_123 has version 19
Rules:
- provider states must be deterministic;
- test data setup should be isolated;
- cleanup must be reliable;
- avoid shared mutable test data;
- avoid time-dependent flaky states;
- seed IDs should be stable but not production-like sensitive data;
- state setup should use domain/application APIs where possible.
Bad provider state:
@State("customer exists")
void setup() {
// depends on whatever data is in shared database
}
Good:
@State("customer cus_123 exists with lifecycle ACTIVE")
void setupCustomer() {
testData.reset();
testData.createCustomer("cus_123", "ACTIVE");
}
23. Testing Security Contract
Security is part of contract.
Scenarios:
- missing token -> 401;
- invalid token -> 401;
- expired token -> 401;
- valid token missing scope -> 403;
- valid scope but wrong tenant -> 403/404 based on policy;
- hidden resource not leaked;
- security headers present if public;
- OpenAPI security scheme matches runtime.
Pseudo:
@Test
void missingTokenShouldReturn401Problem() {
mockMvc.perform(get("/customers/cus_123"))
.andExpect(status().isUnauthorized())
.andExpect(content().contentType("application/problem+json"))
.andExpect(jsonPath("$.code").value("AUTHENTICATION_REQUIRED"));
}
24. Contract Testing and Gateways
Provider app may pass tests, but gateway behavior can break contract.
Gateway may add:
- authentication;
- rate limiting;
- header requirements;
- path rewriting;
- response transformation;
- timeout;
- CORS;
- request size limits;
- compression;
- custom error format.
If gateway is part of public API, test it.
Strategies:
- run contract tests against deployed gateway in staging;
- include gateway config diff in contract review;
- validate gateway error responses use same Problem Details;
- test rate limit headers and 429;
- test path and header versioning.
25. CI/CD Contract Gate Blueprint
Gates:
| Gate | Fails when |
|---|---|
| validate | invalid OpenAPI |
| lint | style/governance violation |
| diff | breaking without approval |
| generate | generator cannot generate |
| compile | generated/server/client compile fail |
| provider test | runtime mismatch |
| consumer test | consumer expectation broken |
| SDK compatibility | public SDK break |
| publish | artifact metadata missing |
26. Contract Test Ownership
Ownership matters.
| Artifact | Owner |
|---|---|
| OpenAPI spec | provider team, reviewed by platform/domain |
| Pact consumer contracts | consumer team |
| Pact provider verification | provider team |
| Spring Cloud Contract producer contracts | provider or shared ownership |
| SDK tests | SDK/platform team |
| diff policy | platform/API governance team |
| stubs | provider/platform |
| examples | provider, validated by CI |
| semantic tests | provider + domain experts |
Do not centralize all test writing in platform team. Platform sets guardrails; domain/provider/consumer own semantics.
27. Contract Test Granularity
Avoid one giant test per endpoint. Prefer scenario-based tests.
Bad:
@Test
void customerApiWorks() {}
Good:
createCustomer_validRequest_returns201
createCustomer_missingFullName_returnsValidationProblem
createCustomer_duplicateExternalReference_returnsConflict
createCustomer_reusedIdempotencyKeyWithDifferentPayload_returnsConflict
getCustomer_missingToken_returns401
getCustomer_unknownCustomer_returns404
searchCustomers_defaultSort_isCreatedAtDescending
Names should reveal contract behavior.
28. Semantic Contract Tests
Some contract must be expressed as tests, not schema.
Examples:
28.1 Idempotency
Same key and same payload should not create duplicate.
28.2 Default Sort
Search default remains createdAt desc.
28.3 State Machine
Cannot approve draft case.
28.4 Retryability
503 response includes retryable=true and Retry-After.
28.5 Authorization Hiding
Unauthorized tenant sees 404, not 403, if policy says hide existence.
28.6 Monetary Semantics
Amount value is decimal string with currency and no floating rounding.
28.7 Time Semantics
createdAt is UTC instant, not server-local time.
These tests require domain understanding.
29. Contract Coverage
Do not chase 100% endpoint count only. Track meaningful coverage.
Metrics:
- operations with provider verification;
- operations with error contract tests;
- operations with examples;
- operations with generated client compile;
- operations with consumer-driven contracts;
- deprecated fields with usage telemetry;
- dangerous changes reviewed;
- gateway-level scenarios tested;
- semantic invariants tested;
- stubs published.
Example dashboard:
Customer API Contract Coverage
- Operations: 42
- Provider-verified operations: 39
- Error scenarios covered: 81%
- OpenAPI examples validated: 100%
- Generated Java client compile: passing
- Active Pact consumers: 17
- Deprecated fields with telemetry: 6/6
- Breaking diff gate: enabled
30. Contract Testing Anti-Patterns
30.1 Treating Contract Test as E2E Test
Contract tests should be fast and focused. Do not require entire enterprise landscape.
30.2 Only Testing 200
Most integration pain happens outside 200.
30.3 Testing Implementation Details
Do not assert database table side effects in API contract test unless contract promises them.
30.4 Consumer Tests Not Publishing Contracts
Consumer tests pass locally but provider never sees expectations.
30.5 Provider Ignores Consumer Contracts
Pact generated but verification not blocking release.
30.6 Contract Stubs Not Versioned
Consumers unknowingly test against stale stubs.
30.7 Contract Diff Warnings Ignored
If dangerous changes are always overridden, gate is theater.
30.8 Snapshot Testing Entire JSON Blindly
Snapshot tests can be too brittle and block safe additive changes.
Prefer targeted assertions plus schema validation.
30.9 Mocking Away Error Mapper
If test bypasses exception handling, error contract is untested.
30.10 No Negative Tests
Validation, conflict, auth, rate limit, and dependency failures untested.
31. Practical Contract Testing Stack
A practical Java enterprise stack might include:
- OpenAPI spec validation in build;
- OpenAPI linting with organization rules;
- OpenAPI breaking diff;
- MockMvc/WebTestClient/JAX-RS tests validating responses;
- Pact for consumer-driven interactions;
- Spring Cloud Contract for Spring-heavy producer/consumer stubs;
- generated Java client compile test;
- SDK mapping/error tests;
- Testcontainers for realistic dependencies when needed;
- gateway staging contract smoke tests.
Do not add all tools blindly. Start from risk:
| Risk | Tooling response |
|---|---|
| OpenAPI drift | provider validation |
| consumer-specific breakage | Pact |
| Spring stub ecosystem | Spring Cloud Contract |
| generated client breakage | compile SDK/client |
| semantic behavior change | semantic regression tests |
| gateway mismatch | gateway contract smoke |
| docs examples stale | example validation |
32. Example End-to-End Build Policy
contractTestingPolicy:
openapi:
validate: true
lint: true
breakingDiff: true
examplesMustValidate: true
provider:
requireOperationTests: true
requireErrorTests: true
requireHeaderTests: true
consumerDriven:
enabledForTier1Consumers: true
verificationBlocksProviderDeploy: true
sdk:
generatedClientCompile: true
publicApiDiff: true
unknownEnumTests: true
errorMappingTests: true
gateway:
stagingSmokeForPublicApis: true
approvals:
dangerousChangeRequiresApiReview: true
breakingChangeRequiresMigrationPlan: true
This policy can be implemented gradually.
33. Practice Lab
Lab 1 — Build Test Matrix
For endpoint:
POST /cases/{caseId}:approve
Requirements:
- case must be
SUBMITTED; - stale
If-Matchreturns 412; - missing approval reason returns validation error;
- unauthorized user returns 403;
- successful approval returns 200 with new state;
- duplicate idempotency key returns same result;
- same idempotency key with different payload returns 409.
Create provider contract test matrix.
Lab 2 — Choose Testing Approach
Choose tools:
- public API with OpenAPI and 100 external consumers;
- internal Spring-to-Spring APIs;
- multi-language consumers with critical workflows;
- simple admin API used by one UI;
- regulated case transition API.
Lab 3 — Write Consumer Pact Scenario
Consumer needs:
GET /customers/{id}
and uses:
customerId;lifecycleStatus;kycStatus.
Design Pact interaction and explain provider state.
Lab 4 — Detect False Confidence
Find weakness:
- provider test only checks 200;
- consumer Pact only asserts customerId;
- OpenAPI diff passes but default sort changed;
- generated client compiles but error parsing fails;
- stub returns success only;
- gateway returns HTML error page;
- validation error pointer format changed.
Lab 5 — CI Gate Design
Design CI pipeline for contract PR:
- validate spec;
- lint;
- breaking diff;
- generate server;
- compile;
- provider tests;
- consumer contract verification;
- SDK tests;
- publish preview;
- require review for dangerous changes.
34. Senior Engineer Heuristics
- Contract tests prevent drift, not all bugs.
- Schema validation is necessary but not sufficient.
- Consumer-driven contracts protect actual usage, not imagined usage.
- Provider verification must include errors, headers, and edge cases.
- Generated client compile is a contract test for SDK feasibility.
- Semantic invariants need explicit tests.
- Mocks are useful only when they represent approved contracts.
- Stubs must be versioned and verified.
- Gateway behavior is part of public contract.
- Do not test only happy path.
- A weak consumer Pact gives weak protection.
- Contract diff gates must distinguish safe, dangerous, and breaking changes.
- Provider states must be deterministic.
- Examples are test assets, not decoration.
- If contract tests are not in CI, they are documentation.
35. Summary
API contract testing is the discipline that keeps provider behavior, declared contract, generated clients, and consumer expectations aligned. It is not a single tool. OpenAPI validation, Pact, Spring Cloud Contract, generated client compilation, semantic regression tests, and gateway smoke tests each cover different risk surfaces.
Main takeaways:
- contract tests are about compatibility feedback;
- provider verification proves implementation matches declared contract;
- consumer-driven tests prove provider satisfies actual consumer expectations;
- OpenAPI, Pact, and Spring Cloud Contract are complementary, not mutually exclusive;
- error responses, headers, idempotency, pagination, security, and concurrency must be tested;
- static diff catches structural breaking changes early;
- semantic changes require explicitly designed tests;
- generated clients and SDKs need their own compatibility tests;
- stubs must be versioned and verified;
- CI gates turn contract governance into enforceable engineering practice.
Part berikutnya mulai masuk ke event contract engineering: event sebagai fact, command, notification, atau state transfer; dan bagaimana mental model event contract berbeda dari HTTP API contract.
You just completed lesson 12 in build core. Use the series map if you want to review the broader track, or continue directly into the next lesson while the context is still warm.
Keep the momentum while the lesson is still fresh. Move backward for review or continue forward into the next concept.