Cache, Redis, Search Index, and Projection Consistency Model

1. Core Idea

Cache, Redis, search index, and projection are derived data stores.

They exist to improve:

• latency,
• throughput,
• user experience,
• search capability,
• dashboard performance,
• cross-context query,
• operational lookup.

But they introduce consistency risk.

Mental model:

Cache and projections are not source of truth. They are accelerated copies with freshness, invalidation, ownership, and reconciliation requirements.

In CPQ / Quote / Order / Billing, stale derived data can cause real business bugs:

• quote shown with old price,
• product shown active after termination,
• billing account shown active after credit hold,
• permission cache grants revoked access,
• search result returns deleted customer,
• read model misses latest order fallout,
• Redis lock expires too early,
• catalog cache serves old product rule,
• serviceability cache says location is serviceable after coverage changed.

2. Why Cache Consistency Matters

Cache is often treated as performance detail, but in enterprise systems it can affect correctness.

Risks:

• stale read used for command decision,
• cache key missing tenant/account scope,
• Redis data not purged after privacy deletion,
• search index leaks sensitive field,
• projection lag hidden from UI,
• cache invalidation event missed,
• value cached without version,
• TTL too long for dynamic data,
• stampede overloads source DB,
• partial update leaves projection inconsistent,
• cache stores old permission after role revoke.

Therefore cache/read model design must be explicit.

3. Cache vs Projection vs Search Index

Derived stores must know:

• source,
• version,
• freshness,
• invalidation,
• rebuild strategy,
• access/security rules.

4. Cacheable Data Categories

Not all data deserves cache.

5. Cache-aside Pattern

Flow:

Read:
  check cache
  if hit: return
  if miss: read DB/source
  put cache
  return

Write:
  update DB/source
  invalidate cache

Pros:

• simple,
• widely used,
• app controls cache.

Risks:

• stale after write if invalidation fails,
• cache stampede on miss,
• inconsistent invalidation across services,
• race between update and cache fill.

Use for read optimization, not critical command invariants.

6. Read-through and Write-through

Read-through

Cache loader fetches source automatically.

Write-through

Write goes to cache and source together.

Risks in distributed systems:

• cache write succeeds but DB write fails,
• DB write succeeds but cache write fails,
• cache becomes false source of truth.

For enterprise correctness, prefer DB/source-of-truth first, then invalidate/update cache with event/outbox if needed.

7. Write-behind

Write-behind writes to cache first and persists later.

Usually risky for quote/order/billing correctness.

Potentially acceptable for:

• non-critical telemetry,
• counters with reconciliation,
• ephemeral user preferences,
• rate limiting,
• temporary sessions.

Do not use write-behind for:

• quote acceptance,
• order creation,
• billing charge,
• invoice/payment,
• approval decision,
• product inventory state.

8. Cache Key Design

Cache key must include all dimensions that affect value.

Examples:

catalog:offering:{offeringId}:version:{version}
quote-summary:{tenantId}:{quoteId}:v:{quoteVersion}
account-permissions:{tenantId}:{userId}:permVersion:{version}
billing-account-status:{tenantId}:{billingAccountId}
serviceability:{locationId}:{offeringId}:{ruleVersion}

Common missing dimensions:

• tenant ID,
• account scope,
• user permission version,
• catalog version,
• currency,
• locale,
• effective date,
• product offering version,
• price list,
• customer segment,
• site/location.

Bad cache key can cause data leak or wrong price.

9. Cache Value Metadata

Cache value should include metadata where correctness matters.

{
  "data": {...},
  "sourceVersion": 42,
  "sourceUpdatedAt": "2026-07-12T10:00:00Z",
  "cachedAt": "2026-07-12T10:00:05Z",
  "expiresAt": "2026-07-12T10:05:05Z",
  "classification": "CONFIDENTIAL"
}

Metadata supports:

• stale detection,
• debugging,
• cache freshness display,
• safe invalidation,
• comparison with source.

10. TTL Strategy

TTL depends on data volatility and correctness risk.

TTL is not a substitute for invalidation when correctness matters.

11. Event-Driven Invalidation

When source changes, publish event and invalidate derived stores.

Example:

BillingAccountStatusChanged
  -> invalidate billing-account-status cache
  -> update projections
  -> update search index

Invalidation event should include:

• entity type,
• entity ID,
• new version,
• changed fields if useful,
• correlation ID.

Consumers should be idempotent.

If invalidation event is lost, reconciliation/rebuild must catch drift.

12. Versioned Cache

Versioned cache avoids invalidation complexity for immutable/versioned data.

Example:

catalog offering version 12 cache key:
catalog:offering:OFF-123:v12

When version 13 published, clients request v13.

Old v12 can expire naturally.

Good for:

• catalog published versions,
• price list versions,
• quote revisions,
• immutable snapshots,
• approval evidence.

Less good for current-state data that changes often.

13. Cache Stampede

Cache stampede occurs when many requests miss same key and hit source DB.

Mitigation:

• request coalescing,
• mutex/lock for cache fill,
• stale-while-revalidate,
• jittered TTL,
• prewarming,
• background refresh,
• rate limiting.

Be careful with Redis lock correctness. Use bounded TTL and safe fallback.

14. Stale-While-Revalidate

Serve stale value while refreshing in background.

Good for:

• non-critical read dashboards,
• catalog display,
• expensive search facets,
• support summaries.

Not good for:

• permission decision,
• billing account command guard,
• quote acceptance validation,
• charge creation.

When using stale response, include freshness metadata if user decision could be affected.

15. Negative Caching

Negative caching stores "not found" or "not serviceable".

Risks:

• new customer/account created after negative cache,
• serviceability changes but old not-serviceable remains,
• permission granted but negative authorization cached.

Use short TTL and include source version.

Be careful caching 404 for newly created entities in eventual-consistency systems.

16. Redis Use Cases

Redis can be used for:

• cache,
• short-lived idempotency lookup,
• rate limiting,
• distributed lock/lease,
• session/token metadata,
• feature flag cache,
• hot reference data,
• queue-like structure if intentionally designed,
• temporary workflow correlation cache.

Do not use Redis as authoritative source for:

• quote state,
• order lifecycle,
• billing charges,
• invoice/payment,
• approval decision,
• product inventory,
• audit evidence.

Unless explicitly designed with persistence, durability, backup, and recovery guarantees.

17. Distributed Lock in Redis

Redis lock is risky if misunderstood.

Use only when:

• DB constraint/lock cannot solve it,
• critical section is short,
• lock TTL is bounded,
• operation is idempotent anyway,
• failure mode is understood.

Do not rely on Redis lock as sole guarantee for financial uniqueness.

Better for many cases:

• DB unique constraint,
• DB row lock,
• idempotency record,
• queue claim with skip locked.

If Redis lock expires while operation still running, another worker may enter.

18. Search Index Model

Search index is derived projection for search.

Example index fields:

order_search_document
- order_id
- order_number
- customer_name
- account_number
- status
- created_at
- product_names
- site_city
- searchable_text
- source_version
- indexed_at

Search index should not expose fields user cannot access.

Search result must be filtered by:

• tenant,
• account scope,
• role/permission,
• data classification,
• active/deleted status.

19. Search Index Update Patterns

Patterns:

Event-driven search must handle:

• duplicate events,
• out-of-order events,
• delete/anonymize events,
• schema change,
• reindex.

Index document should include source aggregate version to avoid stale overwrite.

20. Projection Consistency

Projection consistency requires:

source event version
last processed event
projection version
projected_at
freshness status

Example projection:

order_projection_for_billing
- order_id
- status
- billing_account_id
- aggregate_version
- last_event_id
- projected_at

Consumer rule:

Apply event only if event.aggregate_version > projection.aggregate_version.

This prevents stale events overwriting newer projection.

21. Projection Lag

Projection lag should be visible.

Fields:

projection_checkpoint
- projection_name
- source
- last_event_id
- last_offset
- last_processed_at
- lag_seconds
- status

Dashboard/API can show:

projection freshness = 2 minutes behind

Critical command decisions should not rely on stale projection unless policy allows.

22. Cache and Privacy/Purge

Cache/search/projection must respect privacy deletion/anonymization.

When entity anonymized/purged:

• invalidate cache,
• remove/update search document,
• update read model,
• update analytics copy if applicable,
• remove derived PII,
• preserve allowed tombstone if needed.

Purge process must include derived stores.

Failure mode:

Contact deleted from DB but still searchable in index.

23. Cache and Authorization

Never cache authorization-insensitive response and serve it to users with different scopes.

Bad key:

quote:{quoteId}

for response containing margin.

Better:

quote:{tenantId}:{quoteId}:view:{fieldPolicyVersion}:{userScopeHash}

Or avoid caching user-specific sensitive DTOs.

Cache raw internal object only in service layer and apply authorization per request if safe.

24. Cache Invalidation Table

For important invalidations, persist record.

cache_invalidation_event
- id
- cache_region
- entity_type
- entity_id
- source_version
- reason_code
- status
- created_at
- processed_at

This can be useful when:

• invalidation is asynchronous,
• multiple cache/search/read stores need update,
• failures must be retried,
• privacy purge requires proof.

25. Materialized View Consistency

Materialized views/read summaries need refresh policy.

Options:

• refresh on schedule,
• refresh incrementally,
• refresh on event,
• refresh concurrently if DB supports,
• rebuild table then swap.

Track:

read_model_refresh
- model_name
- refresh_type
- status
- started_at
- completed_at
- source_watermark

Do not let stale materialized view drive critical command state.

26. Cache Warmup

Warmup may be useful for:

• published catalog,
• price lists,
• common reference data,
• permission metadata,
• high-traffic customer summaries.

Warmup should be version-aware and safe.

Avoid warmup that:

• overloads DB after deploy,
• caches unauthorized data globally,
• warms stale version,
• fails silently before critical traffic.

27. PostgreSQL Physical Design for Projection Tracking

Projection checkpoint:

create table projection_checkpoint (
  projection_name text primary key,
  source_name text not null,
  last_event_id uuid,
  last_offset text,
  last_processed_at timestamptz,
  lag_seconds integer,
  status text not null,
  error_message text,
  updated_at timestamptz not null
);

Search index tracking:

create table search_index_document_state (
  id uuid primary key,
  index_name text not null,
  entity_type text not null,
  entity_id uuid not null,
  source_version integer,
  indexed_version integer,
  index_status text not null,
  indexed_at timestamptz,
  error_message text,
  unique (index_name, entity_type, entity_id)
);

Cache invalidation tracking:

create table cache_invalidation_record (
  id uuid primary key,
  cache_region text not null,
  entity_type text not null,
  entity_id uuid not null,
  source_version integer,
  reason_code text,
  status text not null,
  created_at timestamptz not null,
  processed_at timestamptz,
  error_message text
);

Indexes:

create index idx_search_index_state_status
on search_index_document_state (index_name, index_status, indexed_at);

create index idx_cache_invalidation_status
on cache_invalidation_record (status, created_at);

28. Java/JAX-RS Backend Implications

Cache-aware service pattern:

public ProductOffering getPublishedOffering(String offeringId, int version) {
    String key = "catalog:offering:" + offeringId + ":v" + version;
    return cache.getOrLoad(key, () -> catalogRepository.loadPublished(offeringId, version));
}

Command pattern:

public void acceptQuote(AcceptQuoteCommand command) {
    // Do not trust cached quote for transition.
    Quote quote = quoteRepository.findForUpdate(command.quoteId());
    quote.accept(...);
    quoteRepository.save(quote);
    outbox.append(QuoteAcceptedEvent.from(quote));
}

Invalidate after committed change through outbox/event when possible.

29. Redis Data Modelling

Redis key design examples:

catalog:offering:{offeringId}:v:{version}
permission:user:{tenantId}:{userId}:v:{permissionVersion}
serviceability:{locationId}:{offeringId}:{ruleVersion}
rate-limit:{clientId}:{window}
lock:{resourceType}:{resourceId}

Store metadata:

{
  "value": {...},
  "sourceVersion": 12,
  "cachedAt": "2026-07-12T10:00:00Z"
}

Set TTL intentionally. Avoid immortal keys for mutable/sensitive data.

30. Observability

Monitor:

• cache hit ratio,
• stale read incidents,
• invalidation failures,
• Redis latency/errors,
• cache memory/evictions,
• hot keys,
• stampede events,
• search indexing lag,
• projection lag,
• search document failure count,
• stale permission cache,
• purge/anonymization propagation failures.

Example checks:

-- Search document behind source version
select *
from search_index_document_state
where index_status = 'INDEXED'
  and indexed_version < source_version;

-- Projection unhealthy
select projection_name, status, last_processed_at, lag_seconds
from projection_checkpoint
where status <> 'HEALTHY'
   or lag_seconds > 300;

-- Cache invalidation stuck
select cache_region, entity_type, count(*), min(created_at)
from cache_invalidation_record
where status in ('PENDING', 'FAILED')
group by cache_region, entity_type;

31. Failure Modes

32. PR Review Checklist

When reviewing cache/search/projection changes, ask:

• What is source of truth?
• Is this cache, projection, or search index?
• What is cache key?
• Does key include tenant/scope/version?
• What is TTL?
• What invalidates it?
• Is event-driven invalidation reliable?
• Is stale data acceptable?
• Can this be used for commands or only reads?
• Does value include source version/freshness?
• Are sensitive fields included?
• Does purge/anonymization propagate?
• Can projection/search be rebuilt?
• Is out-of-order event handled?
• Are lag/failure metrics available?
• Is Redis used only for safe responsibilities?

33. Internal Verification Checklist

Verify these in the internal CSG/team context:

• Redis/cache usage by quote/order/catalog/billing services.
• Standard cache key conventions.
• Tenant/account/user-scope cache rules.
• Permission cache TTL and invalidation strategy.
• Catalog/price cache versioning strategy.
• Whether command services ever rely on cache for critical validation.
• Search index technology and update mechanism.
• Search document security/masking strategy.
• Projection checkpoint and lag monitoring.
• Cache invalidation event mechanism.
• Purge/anonymization propagation to cache/search/read models.
• Redis persistence/HA assumptions.
• Incidents involving stale cache, wrong permission, stale search, projection lag, or Redis outage.

34. Summary

Cache and derived projections are performance tools with correctness risk.

A strong model must define:

• source of truth,
• cacheable data categories,
• cache key dimensions,
• TTL,
• invalidation,
• versioned cache,
• stale-read policy,
• Redis responsibility boundary,
• search document model,
• projection version,
• projection checkpoint,
• freshness metadata,
• security/masking,
• purge propagation,
• rebuild strategy,
• observability.

The key principle:

Cache is safe only when everyone knows it is a derived copy, how fresh it is, how it is invalidated, who can read it, and why it must never bypass source-of-truth invariants.