Series MapLesson 23 / 35
Deepen PracticeOrdered learning track

Learn Java Security Cryptography Integrity Part 023 Tamper Evident Logs Audit Trails And Evidence

22 min read4281 words
PrevNext
Lesson 2335 lesson track2029 Deepen Practice

title: Learn Java Security, Cryptography and Integrity - Part 023 description: Tamper-evident audit trails, append-only event design, hash chains, evidence packets, log signing, regulatory defensibility, and forensic readiness for Java systems. series: learn-java-security-cryptography-integrity seriesTitle: Learn Java Security, Cryptography and Integrity order: 23 partTitle: Tamper-Evident Logs, Audit Trails & Evidence tags:

  • java
  • security
  • audit
  • logs
  • integrity
  • evidence
  • tamper-evident
  • forensics
  • compliance date: 2026-06-30

Part 023 — Tamper-Evident Logs, Audit Trails & Evidence

Target: setelah part ini, kamu mampu mendesain audit trail Java yang defensible: event apa yang harus dicatat, bagaimana menjaga integritasnya, bagaimana membuatnya tamper-evident, bagaimana memverifikasi ulang chain, dan bagaimana mengubah log menjadi evidence packet yang bisa dipakai untuk investigasi, regulator, dispute, atau post-incident review.

Part ini bukan sekadar “tambahkan logging”. Kita membahas log sebagai evidence system.

Dalam sistem regulasi, finansial, enforcement lifecycle, case management, eligibility, entitlement, fraud review, atau workflow kritikal, pertanyaan yang harus bisa dijawab bukan hanya:

Apa yang terjadi?

Tetapi:

Siapa melakukan apa, atas authority apa, terhadap entity mana,
pada state/version mana, melalui channel mana, dengan input apa,
diproses oleh service/version mana, menghasilkan keputusan apa,
dan bagaimana kita tahu catatan itu tidak dimanipulasi setelah kejadian?

Core invariant:

Audit evidence is trustworthy only when event semantics are explicit, ordering is explainable, identity and authority are bound to the action, records are append-only, integrity can be independently verified, and retention/access controls preserve the chain of custody.

Referensi utama:


1. Kaufman Deconstruction: Audit Integrity Skill Map

CapabilityPertanyaan korektifOutput engineering
Event semanticsEvent ini membuktikan fakta apa?Audit event taxonomy.
Actor bindingSiapa actor efektif dan delegated authority-nya?Actor/authority model.
Object bindingEntity, version, tenant, dan business key mana yang berubah?Target object model.
Decision traceabilityKenapa keputusan dibuat?Reason code + policy version.
Append-only designSiapa yang bisa mengubah audit setelah ditulis?Write-once/append-only store.
Tamper evidenceBagaimana perubahan terdeteksi?Hash chain/Merkle/signature.
OrderingBagaimana menjelaskan urutan event lintas service?Timestamp + sequence + correlation.
ConfidentialityApakah log bocor data sensitif?Redaction/tokenization/access control.
RetentionBerapa lama evidence harus disimpan?Retention and legal hold policy.
VerificationBisakah pihak lain memverifikasi evidence?Verification tool + evidence packet.

Kaufman-style learning objective:

Bukan hafal library logging.
Bangun kemampuan melihat “audit gap” dalam desain sistem.

Audit gap adalah keadaan ketika sistem benar-benar melakukan sesuatu, tetapi tidak memiliki catatan yang cukup kuat untuk membuktikan:

  • siapa melakukan aksi;
  • aksi dilakukan terhadap object mana;
  • authority apa yang dipakai;
  • state/version sebelum dan sesudah;
  • aturan/decision model yang dipakai;
  • apakah event dapat dimanipulasi setelah kejadian.

2. Log, Audit Trail, Security Event, dan Evidence: Jangan Dicampur

Banyak sistem gagal karena semua disebut “log”. Padahal kategorinya berbeda.

Jenis catatanTujuan utamaContohRetentionIntegrity requirement
Application logDebugging/operabilityexception, latency, SQL timeoutpendek-sedangmedium
Security eventDetection/responsefailed login, token replay, suspicious IPsedanghigh
Audit trailAccountability/complianceapprove case, change entitlement, override decisionpanjangvery high
Evidence recordDispute/regulatory proofsigned decision packet, review trailpanjang/legal holdvery high + chain of custody
Business eventDomain processingcase submitted, payment settledtergantung domainhigh jika menjadi source of truth

Mental model:

Application log explains system behavior.
Audit trail proves accountability.
Evidence proves a defensible fact under dispute.

Jangan memakai application logs sebagai satu-satunya audit trail untuk aksi kritikal. Application logs sering:

  • berubah format;
  • sampling;
  • tidak lengkap;
  • bisa hilang saat rolling;
  • terlalu noisy;
  • memuat data sensitif;
  • tidak append-only;
  • tidak punya event semantics stabil.

Audit trail harus menjadi domain artifact dengan schema, invariant, owner, retention, verification, dan governance.


3. Threat Model untuk Audit Trail

Audit trail diserang karena ia mengandung bukti. Threat model-nya berbeda dari log biasa.

ThreatContohDampakControl utama
DeletionAdmin menghapus event approvalevidence hilangappend-only + external replication
Modificationdecision=REJECTED diganti APPROVEDfakta palsuhash chain/signature
Insertionevent backdated dimasukkantimeline palsumonotonic sequence + timestamp authority
Reorderingevent override dipindah sebelum approvalnarrative palsuprevious hash + sequence
Selective omissionhanya event menguntungkan yang dieksporevidence biasrange proof + completeness check
Actor spoofingservice menulis actor user tanpa proofattribution palsuactor context from trusted auth boundary
Reason spoofingreason code tidak sesuai policyinvalid decision proofpolicy version + decision input hash
Log injectionnewline/control char membuat fake eventinvestigation confusionstructured logging + encoding
Sensitive leakagePII/secret masuk logbreach/compliance riskredaction + classification
Time manipulationserver clock diubahtimeline tidak defensibletrusted timestamp/clock sync
Key compromisesigner key bocorforged audit eventsKMS/HSM + key rotation + compromise window

Security invariant:

A privileged operator may administer infrastructure,
but must not be able to silently rewrite the accountability record.

Ini tidak berarti admin tidak bisa melakukan damage. Artinya damage harus terdeteksi, terbatas, dan bisa dijelaskan.


4. Audit Event Taxonomy: Apa yang Wajib Dicatat?

Audit event harus dipilih berdasarkan risk, bukan berdasarkan apa yang mudah dilog.

4.1 Identity and access events

  • login success/failure;
  • MFA enrollment/change;
  • passkey enrollment/removal;
  • password reset request/complete;
  • privilege grant/revoke;
  • role/policy change;
  • session revocation;
  • token replay detection;
  • break-glass access.

4.2 Domain-critical lifecycle events

Untuk case management/regulatory systems:

  • case created/submitted;
  • evidence uploaded/removed/reclassified;
  • task assigned/reassigned/escalated;
  • deadline changed;
  • decision proposed;
  • decision approved/rejected;
  • manual override;
  • enforcement action issued;
  • sanction modified;
  • appeal received;
  • case reopened/closed;
  • external notification sent;
  • data correction performed.

4.3 Policy and configuration events

  • rule version published;
  • workflow definition changed;
  • threshold changed;
  • jurisdiction mapping changed;
  • SLA/escalation config changed;
  • feature flag changed for security-sensitive path;
  • signer/truststore/key changed.

4.4 Data access events

Tidak semua read harus diaudit dengan tingkat sama. Audit read ketika:

  • data sensitif/regulated;
  • bulk export;
  • privileged view;
  • cross-tenant access;
  • access by support/admin;
  • access after case closure;
  • access outside assigned team;
  • access through emergency mode.

4.5 System and integration events

  • webhook verification failed;
  • API request signature invalid;
  • inbound message duplicated/replayed;
  • external system response accepted/rejected;
  • reconciliation mismatch;
  • schema version rejected;
  • data import/export started/completed/failed;
  • integrity verification failure.

Anti-pattern:

Audit everything.

Itu terdengar aman, tetapi sering menghasilkan biaya besar, noise, privacy risk, dan investigasi buruk. Yang benar:

Audit every security/accountability decision and every mutation that changes rights, obligations, evidence, state, or money.

5. Audit Event Schema: Minimal yang Defensible

Audit event harus memiliki schema yang stabil. Contoh field minimum:

FieldTujuan
eventIdunique immutable event id
eventTypecontrolled vocabulary
occurredAtwaktu domain event terjadi
recordedAtwaktu audit record ditulis
tenantIdisolation and governance
actorhuman/service/system actor
authorityrole, delegation, policy, consent, mandate
subjectprincipal impacted jika berbeda dari actor
actionverb stabil: APPROVE, OVERRIDE, REVOKE
objectentity type/id/version
outcomesuccess/failure/denied/partial
reasonCodecontrolled explanation
policyVersionrule/policy version used
requestIdrequest correlation
traceIddistributed trace correlation
sourceservice/app/node/build version
inputHashhash dari input penting, bukan payload sensitif lengkap
beforeHashhash state before jika relevan
afterHashhash state after jika relevan
prevAuditHashlink ke event sebelumnya
auditHashhash canonical audit record
signatureoptional HMAC/digital signature
keyIdsigner key id
schemaVersionevolusi schema

Contoh event:

{
  "schemaVersion": "audit.case.v3",
  "eventId": "01JZ0K4X9BN7E1N3G7YF5QT9V8",
  "eventType": "CASE_DECISION_APPROVED",
  "occurredAt": "2026-06-30T08:00:42.120Z",
  "recordedAt": "2026-06-30T08:00:42.245Z",
  "tenantId": "regulator-id",
  "actor": {
    "type": "HUMAN",
    "principalId": "user-7421",
    "authSessionId": "sess-9f2...",
    "assuranceLevel": "PHISHING_RESISTANT_MFA"
  },
  "authority": {
    "mode": "ROLE_AND_DELEGATION",
    "role": "CASE_SUPERVISOR",
    "delegationId": "del-2026-06",
    "policyVersion": "authz-policy-83"
  },
  "action": "APPROVE_DECISION",
  "object": {
    "type": "CASE",
    "id": "case-2026-000194",
    "version": 17
  },
  "outcome": "SUCCESS",
  "reasonCode": "EVIDENCE_SUFFICIENT",
  "requestId": "req-4340ad",
  "traceId": "2e5bd7f3a5b44d1a",
  "source": {
    "service": "case-decision-service",
    "build": "2026.06.30+sha.4a76c91"
  },
  "inputHash": "sha256:...",
  "beforeHash": "sha256:...",
  "afterHash": "sha256:...",
  "prevAuditHash": "sha256:...",
  "auditHash": "sha256:...",
  "signature": "base64url:...",
  "keyId": "audit-signing-2026-q2"
}

Important distinction:

  • occurredAt: kapan domain action terjadi.
  • recordedAt: kapan audit system mencatat.
  • prevAuditHash: link chain.
  • auditHash: hash record canonical.
  • signature: bukti bahwa record dibuat oleh component yang memegang signing key saat itu.

6. Actor, Authority, Subject: Jangan Disederhanakan

Banyak audit trail hanya menyimpan createdBy. Itu tidak cukup.

Dalam sistem nyata:

  • user bisa acting as organisasi;
  • service bisa acting on behalf of user;
  • admin bisa support impersonation;
  • supervisor bisa delegated authority;
  • automated rule engine bisa membuat keputusan;
  • batch job bisa memproses data dari external authority;
  • API client bisa merepresentasikan partner institution.

Gunakan minimal tiga konsep:

Actor     = entity yang menjalankan aksi.
Authority = hak/mandate yang membuat aksi sah.
Subject   = entity yang terkena dampak jika berbeda.

Contoh:

ScenarioActorAuthoritySubject
Citizen updates profilecitizen userown-account ownershipsame citizen
Case officer edits caseofficerassigned case roleregulated party
Supervisor approves sanctionsupervisordelegated supervisor authorityregulated party
Rule engine escalates caseservicepublished workflow rule v12case
Support views customer datasupport agentbreak-glass ticketcustomer
Partner submits reportAPI clientpartner certificate + agreementreporting institution

Audit event harus menjelaskan authority, bukan hanya identity.

Anti-pattern:

log.info("approved by {}", userId);

Lebih baik:

audit.record(CaseDecisionApproved.builder()
    .actor(Actor.human(userId, sessionId, assuranceLevel))
    .authority(Authority.roleDelegation("CASE_SUPERVISOR", delegationId, policyVersion))
    .object(AuditObject.caseId(caseId, version))
    .reasonCode("EVIDENCE_SUFFICIENT")
    .decisionInputHash(inputHash)
    .build());

7. Architecture: Audit as a First-Class Subsystem

Jangan membuat audit sebagai side-effect logging di tiap controller tanpa kontrak. Buat subsystem.

Design choice:

  • audit event creation should happen close to domain decision;
  • audit storage should be append-only;
  • audit verification should be independent from writer;
  • audit export should include enough metadata to verify integrity later;
  • audit system should be observable but not expose secrets/PII unnecessarily.

8. Transaction Boundary: Audit and Domain State

Critical question:

Jika domain mutation commit tapi audit gagal, apa yang terjadi?
Jika audit commit tapi domain mutation rollback, apa yang terjadi?

Ada tiga common models.

8.1 Same database transaction

BEGIN
  update case
  insert audit_event
COMMIT

Pros:

  • strong atomicity;
  • simple query model;
  • good for monolith/modular monolith.

Cons:

  • attacker with DB write access may alter both;
  • append-only enforcement harus kuat;
  • cross-service audit sulit.

Use when:

  • domain and audit in same service boundary;
  • DB permissions can enforce insert-only audit table;
  • later replication/signing still exists.

8.2 Transactional outbox

BEGIN
  update case
  insert outbox audit command
COMMIT
publisher -> audit service

Pros:

  • reliable across service boundary;
  • decouples audit storage;
  • works for microservices.

Cons:

  • audit is eventually recorded;
  • need gap detection and retry;
  • occurredAt and recordedAt may differ.

Use when:

  • distributed systems;
  • central audit service;
  • event-driven architecture.

8.3 Synchronous external audit service

domain service -> audit service -> domain commit?

Pros:

  • central policy;
  • immediate rejection if audit unavailable.

Cons:

  • availability coupling;
  • complex rollback semantics;
  • latency.

Use only for very high-criticality flows where audit failure must block action.

Practical recommendation:

For most enterprise Java systems:
use local transactional outbox + central append-only audit service + independent verifier.

9. Tamper-Evident Design Patterns

Tamper-evident does not mean tamper-proof. It means manipulation is detectable.

9.1 Per-record hash

Each audit record stores hash of canonical content.

auditHash = SHA-256(canonicalAuditRecordWithoutAuditHashAndSignature)

Detects content modification, but not deletion/reordering.

9.2 Hash chain

Each record includes previous record hash.

auditHash[i] = SHA-256(canonical(record[i]) || auditHash[i-1])

Detects modification, deletion, and reordering within chain.

9.3 HMAC chain

Use HMAC if verifier is internal and shared secret can be protected.

auditMac[i] = HMAC(key, canonical(record[i]) || auditMac[i-1])

Pros:

  • fast;
  • good for internal audit store;
  • simple key rotation.

Cons:

  • verifier with same key can forge;
  • weaker external non-repudiation.

9.4 Digital signature

Use private key to sign record hash or batch root.

signature = Sign(privateKey, auditHash or merkleRoot)

Pros:

  • public verification possible;
  • stronger separation of signer and verifier;
  • better evidence export.

Cons:

  • slower;
  • key management harder;
  • requires certificate/trust model.

9.5 Merkle tree batch

Batch events into Merkle tree, store root, sign root.

hourlyRoot = MerkleRoot(eventHash[0..n])
signature = Sign(key, hourlyRoot)

Pros:

  • efficient verification for large volume;
  • supports inclusion proof;
  • good for anchoring.

Cons:

  • more complex;
  • completeness proof must be designed carefully.

9.6 External anchoring

Periodically anchor chain head/root outside primary system:

  • separate account/cloud project;
  • object storage immutable bucket;
  • external timestamp authority;
  • WORM storage;
  • transparency log;
  • offline export;
  • regulator-owned repository.

Purpose:

Even if primary audit store is compromised later,
attacker cannot silently rewrite history before the last external anchor.

10. Chain Granularity: Global, Tenant, Entity, or Stream?

One global hash chain is simple but may bottleneck. Multiple chains scale better.

Chain modelProsConsGood for
Global chainsimple total orderbottleneck; cross-region hardlow-medium volume
Tenant chainisolation; scalablecross-tenant ordering harderSaaS/regulatory tenants
Entity chaineasy evidence per casemany chains; gap managementcase management
Stream partition chainhigh throughputordering by partition onlyevent platforms
Batch Merkle rootscalable verificationmore complex proofhigh-volume logs

For regulatory case management, combine:

entity-level chain for case evidence
+ tenant-level periodic root
+ external daily anchor

This supports:

  • efficient case-specific verification;
  • tenant-level completeness;
  • external tamper evidence.

11. Canonicalization for Audit Hashing

Never hash arbitrary toString() or raw JSON produced by unstable serialization.

Bad:

byte[] hash = sha256(auditEvent.toString().getBytes(StandardCharsets.UTF_8));

Why bad:

  • field order may change;
  • whitespace may change;
  • timezone formatting may change;
  • map order may change;
  • locale may change;
  • null handling may change;
  • library upgrade may alter output.

Better:

1. Define canonical audit representation.
2. Use stable schema version.
3. Sort object keys deterministically.
4. Use UTC timestamps with fixed precision.
5. Normalize strings when policy requires it.
6. Exclude mutable fields such as signature from signed payload.
7. Include schemaVersion and algorithm metadata.

For JSON, RFC 8785 JCS is a useful reference. But the broader invariant matters more:

Verifier must reconstruct exactly the same bytes from the same semantic event.

12. Java Model: Audit Event as Immutable Domain Object

Use immutable records/value objects. Do not pass mutable maps around.

import java.time.Instant;
import java.util.Map;

public record AuditEvent(
    String schemaVersion,
    String eventId,
    String eventType,
    Instant occurredAt,
    Instant recordedAt,
    String tenantId,
    AuditActor actor,
    AuditAuthority authority,
    AuditObject object,
    String action,
    String outcome,
    String reasonCode,
    String requestId,
    String traceId,
    AuditSource source,
    String inputHash,
    String beforeHash,
    String afterHash,
    String previousHash,
    Map<String, String> attributes
) {}

public record AuditActor(
    String type,
    String principalId,
    String sessionId,
    String assuranceLevel
) {}

public record AuditAuthority(
    String mode,
    String role,
    String delegationId,
    String policyVersion
) {}

public record AuditObject(
    String type,
    String id,
    long version
) {}

public record AuditSource(
    String service,
    String instanceId,
    String buildVersion
) {}

Rules:

  • no setters;
  • no lazy mutation;
  • no post-write enrichment that changes signed content;
  • no raw secrets;
  • all optional fields explicitly modeled;
  • all enums controlled by schema.

13. Java Example: Canonical JSON Hashing Interface

In production, use a proven canonicalization library or an internal canonicalizer with test vectors. The key design is to isolate canonicalization as a contract.

public interface AuditCanonicalizer {
    byte[] canonicalize(AuditEvent event);
}

public interface AuditHasher {
    String hash(byte[] canonicalBytes);
}

public final class Sha256AuditHasher implements AuditHasher {
    @Override
    public String hash(byte[] canonicalBytes) {
        try {
            var digest = java.security.MessageDigest.getInstance("SHA-256");
            byte[] hash = digest.digest(canonicalBytes);
            return "sha256:" + java.util.Base64.getUrlEncoder()
                    .withoutPadding()
                    .encodeToString(hash);
        } catch (java.security.NoSuchAlgorithmException e) {
            throw new IllegalStateException("SHA-256 unavailable", e);
        }
    }
}

The canonicalizer must be deterministic across:

  • JVM versions;
  • library upgrades;
  • map insertion order;
  • locale;
  • timezone;
  • default charset;
  • null/default field behavior.

Test vector example:

Input event: audit.case.v3 CASE_DECISION_APPROVED ...
Canonical bytes hex: 7b22616374696f6e223a22415050524f... 
SHA-256: sha256:u3zB...
Expected signature: base64url:MEUCIG...

If you cannot produce stable test vectors, your audit integrity scheme is not ready.


14. Java Example: Hash Chain Builder

public record ChainedAuditRecord(
    AuditEvent event,
    String canonicalHash,
    String chainHash,
    String algorithm
) {}

public final class AuditChainBuilder {
    private final AuditCanonicalizer canonicalizer;

    public AuditChainBuilder(AuditCanonicalizer canonicalizer) {
        this.canonicalizer = canonicalizer;
    }

    public ChainedAuditRecord append(AuditEvent event, String previousChainHash) {
        byte[] eventBytes = canonicalizer.canonicalize(event);
        byte[] previousBytes = previousChainHash.getBytes(java.nio.charset.StandardCharsets.US_ASCII);

        try {
            var digest = java.security.MessageDigest.getInstance("SHA-256");
            byte[] eventHash = digest.digest(eventBytes);

            digest.reset();
            digest.update("audit-chain-v1".getBytes(java.nio.charset.StandardCharsets.US_ASCII));
            digest.update((byte) 0x00);
            digest.update(previousBytes);
            digest.update((byte) 0x00);
            digest.update(eventHash);
            byte[] chainHash = digest.digest();

            var enc = java.util.Base64.getUrlEncoder().withoutPadding();
            return new ChainedAuditRecord(
                event,
                "sha256:" + enc.encodeToString(eventHash),
                "sha256:" + enc.encodeToString(chainHash),
                "SHA-256/audit-chain-v1"
            );
        } catch (java.security.NoSuchAlgorithmException e) {
            throw new IllegalStateException("SHA-256 unavailable", e);
        }
    }
}

Note the domain separation string:

audit-chain-v1

This prevents reusing the same hash bytes across different protocol meanings.


15. Java Example: HMAC-Signed Audit Record

Use HMAC when verification remains internal and key is protected in KMS/HSM or secret manager.

import javax.crypto.Mac;
import javax.crypto.SecretKey;
import java.nio.charset.StandardCharsets;
import java.util.Base64;

public final class AuditMacSigner {
    private final SecretKey key;
    private final String keyId;

    public AuditMacSigner(SecretKey key, String keyId) {
        this.key = key;
        this.keyId = keyId;
    }

    public SignedAuditRecord sign(ChainedAuditRecord record) {
        try {
            Mac mac = Mac.getInstance("HmacSHA256");
            mac.init(key);
            mac.update("audit-record-signature-v1".getBytes(StandardCharsets.US_ASCII));
            mac.update((byte) 0x00);
            mac.update(record.chainHash().getBytes(StandardCharsets.US_ASCII));
            byte[] signature = mac.doFinal();

            return new SignedAuditRecord(
                record,
                keyId,
                "HmacSHA256",
                Base64.getUrlEncoder().withoutPadding().encodeToString(signature)
            );
        } catch (Exception e) {
            throw new IllegalStateException("Audit MAC signing failed", e);
        }
    }
}

public record SignedAuditRecord(
    ChainedAuditRecord record,
    String keyId,
    String algorithm,
    String signature
) {}

Verification must use constant-time comparison:

boolean same = java.security.MessageDigest.isEqual(expectedMac, suppliedMac);

Operational rule:

The audit writer may request signing.
It should not be able to export raw signing key material.

16. Digital Signature vs HMAC for Audit

CriteriaHMACDigital signature
Performancehighlower
Verificationrequires same secretpublic key
External evidenceweakerstronger
Key sharing riskhigherlower
Non-repudiation claimnot appropriatepossible but still legal/process-dependent
Good forinternal tamper evidenceevidence export, regulator verification

Do not overclaim.

Digital signatures do not magically create legal non-repudiation. They support technical integrity and signer authentication. Legal defensibility also needs:

  • key custody;
  • identity proofing;
  • certificate/trust policy;
  • audit process;
  • time evidence;
  • chain of custody;
  • incident handling.

17. Append-Only Storage Design

Audit integrity fails if anyone can update/delete records silently.

17.1 Database controls

Use database-level constraints:

CREATE TABLE audit_event (
  tenant_id           text NOT NULL,
  chain_id            text NOT NULL,
  sequence_no         bigint NOT NULL,
  event_id            text NOT NULL,
  occurred_at         timestamptz NOT NULL,
  recorded_at         timestamptz NOT NULL,
  event_type          text NOT NULL,
  canonical_payload   bytea NOT NULL,
  event_hash          text NOT NULL,
  previous_chain_hash text NOT NULL,
  chain_hash          text NOT NULL,
  key_id              text,
  signature_algorithm text,
  signature           text,
  created_at          timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (tenant_id, chain_id, sequence_no),
  UNIQUE (tenant_id, event_id),
  UNIQUE (tenant_id, chain_id, chain_hash)
);

Enforce:

  • application DB user can insert only;
  • no update/delete permission;
  • migration/admin path logged separately;
  • database audit enabled;
  • periodic export to immutable storage.

17.2 Immutable object storage

Write segments:

/tenant/{tenantId}/audit/{chainId}/date=2026-06-30/segment-000120.jsonl
/tenant/{tenantId}/audit/{chainId}/date=2026-06-30/segment-000120.manifest.json
/tenant/{tenantId}/audit/{chainId}/date=2026-06-30/segment-000120.sig

Manifest contains:

  • start sequence;
  • end sequence;
  • first previous hash;
  • last chain hash;
  • event count;
  • content hash;
  • signer key id;
  • service build;
  • export time.

Use immutable bucket policies/object lock/WORM where available. But remember:

Storage immutability protects after write.
It does not prove that the writer wrote correct event semantics.

You still need domain-level audit schema.


18. Ordering and Time: Be Precise

Audit systems often overtrust timestamps. Distributed systems do not have perfect time.

Use multiple ordering indicators:

FieldMeaning
occurredAtdomain event time
recordedAtaudit ingestion time
sequenceNochain-local monotonic order
previousHashcryptographic predecessor
traceIddistributed request correlation
causationIdcommand/event causing this event
correlationIdbusiness process correlation
externalTimestampTokenoptional trusted timestamp evidence

Never rely on timestamp alone for ordering critical events.

Better invariant:

Within a chain, sequenceNo and previousHash define order.
Across chains, correlation/causation explain relationship, not total order.

19. Completeness: The Hard Part

Tamper detection often proves “this record was not altered”, but not “there are no missing records”. Completeness requires additional design.

Controls:

  • monotonic sequence per chain;
  • gap detection;
  • expected event count in segment manifest;
  • source outbox reconciliation;
  • domain mutation requires audit outbox row;
  • periodic control totals;
  • independent verifier scans;
  • external anchors;
  • immutable export manifests.

Verifier checks:

for each chain:
  sequence starts from expected checkpoint
  each next sequence increments by 1
  previousHash matches prior chainHash
  canonical payload hash matches eventHash
  signature verifies
  segment manifest count matches actual count
  anchor hash matches external anchor

Completeness is an operational property, not just a cryptographic property.


20. Evidence Packet Design

When exporting evidence for a case, do not send raw logs only. Build an evidence packet.

Example packet structure:

evidence-case-2026-000194/
  README.md
  manifest.json
  audit-events.jsonl
  audit-events.canonical.sha256
  chain-proof.json
  signatures/
    segment-000120.sig
    segment-000121.sig
    signer-cert.pem
  domain-snapshots/
    case-before-decision.json
    case-after-decision.json
  policy/
    authz-policy-83.json
    workflow-definition-42.bpmn
    decision-rule-19.json
  verification/
    verify.sh
    verify.jar
    expected-results.json

manifest.json:

{
  "packetVersion": "evidence.packet.v1",
  "subject": {
    "type": "CASE",
    "id": "case-2026-000194"
  },
  "generatedAt": "2026-06-30T10:00:00Z",
  "generatedBy": "audit-export-service",
  "purpose": "REGULATORY_REVIEW",
  "eventRange": {
    "chainId": "case:case-2026-000194",
    "fromSequence": 1,
    "toSequence": 83
  },
  "rootHash": "sha256:...",
  "signature": {
    "algorithm": "Ed25519",
    "keyId": "audit-export-2026-q2",
    "value": "base64url:..."
  }
}

A useful evidence packet is:

  • self-describing;
  • minimally sufficient;
  • verifiable offline;
  • redacted according to purpose;
  • includes policy versions;
  • includes chain proof;
  • includes verification instructions.

21. Sensitive Data in Audit Logs

Audit logs often become a secondary data lake with worse access controls. Avoid that.

Do not log:

  • passwords;
  • tokens;
  • session cookies;
  • private keys;
  • OTP values;
  • full payment instrument;
  • unnecessary personal data;
  • full documents unless evidence store is designed for it;
  • medical/legal data unless purpose and controls are explicit.

Prefer:

  • stable identifiers;
  • hashes of inputs;
  • reason codes;
  • classification labels;
  • references to secured evidence objects;
  • redacted summaries;
  • tokenized values.

Example:

{
  "eventType": "EVIDENCE_DOCUMENT_UPLOADED",
  "object": { "type": "DOCUMENT", "id": "doc-882" },
  "contentHash": "sha256:...",
  "classification": "CONFIDENTIAL_REGULATED",
  "storageRef": "evidence-store://tenant/case/doc-882",
  "fileNameHash": "sha256:..."
}

Instead of logging the raw file name and content.


22. Access Control for Audit Data

Audit data is sensitive because it exposes:

  • user behavior;
  • investigation timeline;
  • policy internals;
  • system architecture;
  • PII references;
  • security events.

Use role separation:

RoleAccess
App servicewrite audit events only
Audit verifierread canonical records and signatures
Investigatorread scoped events for assigned case
Compliance officerexport approved evidence packet
Security engineerread security events; not all business evidence
DB admininfrastructure access, but actions logged and monitored

Important:

Audit store access itself must be audited.

Meta-audit event examples:

  • AUDIT_RECORD_VIEWED;
  • AUDIT_EXPORT_CREATED;
  • AUDIT_EXPORT_DOWNLOADED;
  • AUDIT_RETENTION_POLICY_CHANGED;
  • AUDIT_VERIFICATION_FAILED;
  • AUDIT_ADMIN_ACCESS_GRANTED.

23. Verification Service

Do not trust that audit chain is valid just because write path computed hashes. Run independent verification.

Verification report:

{
  "chainId": "case:case-2026-000194",
  "verifiedAt": "2026-06-30T12:00:00Z",
  "fromSequence": 1,
  "toSequence": 83,
  "status": "VALID",
  "lastChainHash": "sha256:...",
  "signatureKeys": ["audit-signing-2026-q2"],
  "anchorsChecked": ["2026-06-30-daily-root"],
  "verifierBuild": "2026.06.30+sha.a19c0d"
}

Alert on:

  • missing sequence;
  • hash mismatch;
  • signature mismatch;
  • unexpected key id;
  • expired/revoked signing key used outside allowed window;
  • segment manifest mismatch;
  • external anchor mismatch;
  • future timestamp beyond clock skew policy;
  • event schema unknown.

24. Integration with Observability

Audit trail is not the same as observability, but they should correlate.

Use shared correlation identifiers:

requestId -> traceId -> commandId -> causationId -> auditEventId

Example:

try (var ignored = org.slf4j.MDC.putCloseable("requestId", requestId)) {
    commandHandler.handle(command);
}

But do not rely on MDC for audit truth. MDC is request-scoped logging metadata, not an evidence model.

Audit subsystem should receive explicit values:

audit.record(AuditCommand.builder()
    .requestId(requestId)
    .traceId(traceId)
    .actor(actor)
    .authority(authority)
    .object(caseRef)
    .action("APPROVE_DECISION")
    .reasonCode(reasonCode)
    .build());

25. Log Injection Defense

Log injection can forge apparent events or confuse parsers.

Bad:

log.info("user={} action={}", username, action);

If username contains newline/control characters, downstream text logs may be misleading.

Better:

  • structured JSON logging;
  • encode control characters;
  • validate controlled vocabularies;
  • never concatenate raw user text into audit event type/action;
  • use schema field constraints;
  • reject invalid identifiers.

Example sanitizer for display-only fields:

public final class AuditText {
    public static String safeSingleLine(String value) {
        if (value == null) return null;
        return value
            .replace("\\", "\\\\")
            .replace("\r", "\\r")
            .replace("\n", "\\n")
            .replace("\t", "\\t");
    }
}

Do not sanitize cryptographic canonical bytes after the fact. Validate and encode at field construction.


26. Failure Modes and Policy

What happens if audit write fails?

Classify actions:

Action typeAudit failure policy
Low-risk readallow + operational log
Sensitive readfail closed or degraded approval
Business mutationretry via outbox; if outbox unavailable fail closed
Privilege changefail closed
Evidence mutationfail closed
Break-glass accessfail closed unless emergency offline process exists
Batch importpause batch and reconcile

Explicit policy:

No rights/obligations/evidence-changing mutation may commit unless the audit outbox record commits in the same transaction.

This is often the most important practical invariant.


27. Case Study: Enforcement Decision Approval

Scenario:

  • case officer proposes sanction;
  • supervisor approves;
  • system issues enforcement notice;
  • regulated party disputes the decision six months later.

Defensible audit trail must prove:

  1. The case existed at version N.
  2. Officer had assignment when proposal was made.
  3. Evidence set hash at proposal time.
  4. Supervisor had authority at approval time.
  5. Applicable rule/policy versions.
  6. Decision input hash.
  7. Approval event occurred after proposal.
  8. Enforcement notice content hash.
  9. Notification was sent through approved channel.
  10. No audit chain tampering after decision.

Mermaid flow:

Evidence packet should include:

  • CASE_DECISION_PROPOSED;
  • CASE_DECISION_APPROVED;
  • ENFORCEMENT_NOTICE_ISSUED;
  • policy versions;
  • evidence set manifest;
  • notification hash;
  • chain proof;
  • signature verification material.

28. Code Review Checklist

Ask these questions during review:

Event semantics

  • Is event type controlled and stable?
  • Does event represent a real business/security fact?
  • Is outcome explicit?
  • Is reason code controlled?
  • Is policy/rule version captured?

Actor and authority

  • Is actor derived from trusted authentication context?
  • Is authority captured, not inferred later?
  • Is delegation/impersonation/break-glass explicit?
  • Are service actors distinguishable from human actors?

Object and state

  • Is object id stable?
  • Is object version captured?
  • Are before/after hashes captured for critical mutation?
  • Is tenant boundary captured?

Integrity

  • Is canonicalization deterministic?
  • Is previousHash included?
  • Are hashes/signatures computed over the right fields?
  • Are mutable fields excluded from signed payload?
  • Is algorithm/key id stored?
  • Is verification implemented independently?

Storage

  • Is audit store append-only?
  • Are update/delete permissions removed?
  • Is external replication/anchoring configured?
  • Are retention and legal hold defined?

Privacy

  • Are secrets excluded?
  • Is PII minimized?
  • Is audit access itself audited?
  • Are exports purpose-scoped and redacted?

29. Common Anti-Patterns

29.1 Audit as log string

log.info("User {} approved case {}", userId, caseId);

Problem:

  • no schema;
  • no version;
  • no authority;
  • no object version;
  • no integrity;
  • hard to verify.

29.2 Mutable audit correction

UPDATE audit_event SET actor_id = 'corrected-user' WHERE event_id = '...';

Better:

Append AUDIT_CORRECTION_RECORDED referencing original event.

29.3 Logging final state only

case status changed to APPROVED

But not who/why/authority/policy/input.

29.4 Hash without canonicalization

Hashing unstable serialization gives false confidence.

29.5 Signature without trust model

A signature is useless if:

  • key custody is unknown;
  • verifier accepts any key;
  • key rotation state is not stored;
  • algorithm is not allowlisted;
  • signed bytes are ambiguous.

29.6 Audit event after commit but not reliable

update DB
then try audit over HTTP

If audit call fails, mutation exists without evidence. Use outbox or fail closed.


30. Practice Lab

Build a local audit subsystem for a simplified case platform.

Requirements

Implement:

  1. AuditEvent immutable model.
  2. Canonical JSON serializer with deterministic field order.
  3. SHA-256 event hash.
  4. Chain hash with previous hash.
  5. HMAC signature over chain hash.
  6. Append-only file store using JSON Lines.
  7. Verifier CLI.
  8. Tamper test suite.

Events

  • CASE_CREATED
  • CASE_ASSIGNED
  • EVIDENCE_UPLOADED
  • DECISION_PROPOSED
  • DECISION_APPROVED
  • NOTICE_ISSUED

Tamper tests

Verify detection when:

  • event content changed;
  • event deleted;
  • event reordered;
  • event inserted in middle;
  • signature changed;
  • previous hash changed;
  • sequence skipped;
  • wrong key id used;
  • canonicalization changed.

Success criteria

You are done when:

java -jar audit-verifier.jar audit-events.jsonl
VALID chain=case:case-001 events=6 lastHash=sha256:...

And every tampered fixture fails with a clear reason.


31. Decision Record Template

Use this when introducing audit integrity in a real system.

# ADR: Tamper-Evident Audit Trail for <System>

## Context
<Why audit evidence matters. Regulatory/dispute/security requirements.>

## Assets
- audit events
- signer keys
- canonical schemas
- export packets
- verification reports

## Threats
- deletion
- modification
- insertion
- reordering
- selective omission
- signer key compromise
- sensitive data leakage

## Decision
We will use:
- chain granularity: <tenant/entity/global>
- canonical format: <JCS/internal canonical JSON>
- event hash: <SHA-256/...>
- chain algorithm: <...>
- signing algorithm: <HMAC-SHA-256/Ed25519/...>
- storage: <DB + immutable object storage>
- external anchor frequency: <daily/hourly>

## Invariants
- no critical mutation commits without audit outbox row
- audit records are append-only
- verifier runs independently
- evidence packets are offline-verifiable

## Consequences
<Latency/cost/operational complexity.>

## Verification
<Test vectors, verifier, incident alerts.>

32. Final Mental Model

Audit integrity is not “logs with hashes”. It is a layered evidence system.

Domain semantics
  -> structured audit event
  -> canonical bytes
  -> hash / chain / signature
  -> append-only storage
  -> external anchor
  -> independent verification
  -> evidence packet
  -> defensible investigation

If one layer is missing, understand what claim becomes weaker:

  • no domain semantics: cannot explain what happened;
  • no actor/authority: cannot prove accountability;
  • no canonicalization: cannot verify consistently;
  • no chain: cannot detect deletion/reordering;
  • no signature: weaker origin proof;
  • no append-only storage: silent modification easier;
  • no external anchor: full rewrite by privileged attacker harder to disprove;
  • no verifier: tampering may remain undetected;
  • no evidence packet: investigation becomes manual and fragile.

33. Self-Assessment

You should be able to answer:

  1. Apa perbedaan application log, security event, audit trail, dan evidence record?
  2. Mengapa createdBy tidak cukup untuk audit defensible?
  3. Apa bedanya occurredAt, recordedAt, sequenceNo, dan previousHash?
  4. Threat apa yang dicegah oleh per-record hash? Threat apa yang tidak dicegah?
  5. Kapan memakai HMAC vs digital signature?
  6. Bagaimana mendeteksi deletion atau reordering?
  7. Mengapa canonicalization wajib sebelum hashing/signing?
  8. Bagaimana mendesain evidence packet untuk dispute enam bulan kemudian?
  9. Apa failure policy jika audit write gagal?
  10. Bagaimana memastikan audit store tidak menjadi data leakage source?

34. Ringkasan

  • Audit trail adalah accountability system, bukan debug log.
  • Tamper-evident berarti manipulasi terdeteksi, bukan mustahil terjadi.
  • Event semantics harus eksplisit: actor, authority, object, action, outcome, reason, policy version, state hash.
  • Hash chain mendeteksi modification/deletion/reordering dalam chain.
  • Signature/HMAC menambahkan origin/integrity proof, tetapi bergantung pada key management.
  • Completeness membutuhkan sequence, manifest, reconciliation, verifier, dan anchor.
  • Evidence packet harus offline-verifiable dan purpose-scoped.
  • Critical mutation tidak boleh commit tanpa audit record/outbox yang commit atomically.

Part berikutnya membahas Secure Serialization, Canonicalization & Signing: bagaimana membuat JSON/XML payload yang bisa diverifikasi dengan aman, menghindari ambiguity, signature wrapping, parser trust boundary, dan canonicalization traps.

Lesson Recap

You just completed lesson 23 in deepen practice. Use the series map if you want to review the broader track, or continue directly into the next lesson while the context is still warm.