Deepen PracticeOrdered learning track

Image Supply Chain: SBOM, Vulnerability Scanning, Signing, Trust, and Policy

Learn Docker, Containerization, Docker Compose, Docker Swarm - Part 024

Image supply chain engineering for Docker: SBOM, vulnerability scanning, base image governance, digest pinning, Docker Scout, build attestations, provenance, signing, registry promotion, and policy gates.

18 min read3599 words
PrevNext
Lesson 2435 lesson track2029 Deepen Practice
#docker#containerization#supply-chain-security#sbom+5 more

Part 024 — Image Supply Chain: SBOM, Vulnerability Scanning, Signing, Trust, and Policy

Target pembelajaran: setelah part ini, kita mampu mendesain supply chain container image yang traceable, scannable, policy-driven, dan promotable; memahami batas SBOM/scanning/signing; menggunakan digest, SBOM, provenance, Docker Scout, attestations, dan signing sebagai evidence chain; serta membuat release gate yang realistis untuk engineering organization.

Part 023 membahas secret dan sensitive data. Part ini membahas trust terhadap image artifact.

Pertanyaan utama:

How do we know this image is the artifact we intended to build, from the source we intended, with dependencies we understand, and acceptable risk for the environment where it will run?

Image supply chain bukan hanya vulnerability scan. Ia adalah gabungan dari:

  • source integrity;
  • dependency governance;
  • build reproducibility/traceability;
  • image identity;
  • SBOM;
  • vulnerability intelligence;
  • signing;
  • provenance;
  • registry promotion;
  • deployment policy;
  • runtime drift monitoring.

1. Supply Chain Mental Model

Container image adalah artifact distribusi. Supply chain adalah jalur dari source sampai runtime.

Goal:

No image reaches production unless identity, contents, origin, and risk are known and acceptable.

2. What "Trust" Means for Container Images

Trust bukan perasaan. Trust adalah evidence.

QuestionEvidence
Image apa yang dijalankan?digest
Source mana yang membangunnya?provenance / labels / CI metadata
Dependency apa yang ada di dalamnya?SBOM
Vulnerability apa yang diketahui?scan report
Siapa/apa yang membangun?CI identity / attestation
Apakah artifact diubah setelah build?signature / digest verification
Apakah policy terpenuhi?policy evaluation
Apakah image dari base yang disetujui?base image policy
Apakah deploy sesuai environment?promotion record

Top 1% engineer tidak berkata:

It came from our registry, so it is safe.

Mereka berkata:

It is digest X, built by workflow Y from commit Z, with SBOM A, scan result B, signature C, and policy decision D.

3. Threat Model

Supply chain threat meliputi:

ThreatContoh
malicious dependencypackage dependency disusupi
vulnerable dependencyCVE pada OS/library package
compromised base imagebase image mengandung malware/vulnerability
mutable tag attacklatest berubah tanpa kontrol
registry compromiseimage diganti/ditimpa
CI compromisepipeline membangun artifact malicious
secret leakagetoken masuk image layer
build context poisoningfile tidak sengaja masuk build context
cache poisoningcache dari boundary tidak terpercaya
provenance gaptidak jelas image berasal dari commit mana
signing key compromiseattacker bisa sign image palsu
policy bypassdeployment tidak memverifikasi gate

Diagram:


4. Image Identity: Tag vs Digest

Part 005 sudah membahas tag/digest. Dalam supply chain, ini menjadi governance rule.

Tag:

company/api:1.4.2

Digest:

company/api@sha256:...

Tag adalah nama manusia. Digest adalah identitas konten.

AttributeTagDigest
mudah dibacayatidak
mutablebisatidak secara konten
cocok untuk UXyaterbatas
cocok untuk verificationlemahkuat
cocok untuk promotion evidencelemah sendirikuat

Policy:

Humans may discuss tags.
Systems should deploy digests or verify that tags resolve to expected digests.

5. Production Rule: Pin What You Promote

Bad deployment:

services:
  api:
    image: company/api:latest

Better deployment:

services:
  api:
    image: company/api:1.4.2

Best evidence-oriented deployment:

services:
  api:
    image: company/api@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Trade-off:

  • digest sulit dibaca;
  • tag mudah untuk human release note;
  • registry/CI bisa menyimpan mapping tag → digest;
  • deployment manifest bisa memakai digest dan metadata comment/label untuk readability.

Example release record:

service: billing-api
version: 1.4.2
image: registry.example.com/billing/api@sha256:...
gitCommit: 3f2a9c1
builtBy: github-actions://company/billing/.github/workflows/release.yml
sbom: registry-attestation://...
scanDecision: pass-with-exceptions
approvedBy: platform-security

6. Base Image Governance

Base image adalah inherited trust.

Jika base image buruk, semua image turunan mewarisi risiko.

Base image policy harus menjawab:

  • base image apa yang boleh dipakai?
  • siapa owner base image?
  • bagaimana update base dilakukan?
  • bagaimana CVE base image dipantau?
  • apakah digest dipin?
  • apakah image berasal dari official/trusted source?
  • apakah distro lifecycle masih aktif?
  • apakah runtime image terlalu besar?
  • apakah debug tools boleh ada di production image?

Base image categories:

TypeKelebihanRisiko
full distrodebugging mudah, package lengkapattack surface besar
slimlebih kecilkadang library/tool hilang
Alpinekecilmusl compatibility issue untuk beberapa workload
distrolessattack surface kecildebugging lebih sulit
scratchminimal totalhanya cocok untuk static binary tertentu
internal golden imagestandard org kuatharus dikelola aktif

7. Approved Base Image Matrix

Contoh matrix organization:

WorkloadApproved BaseNotes
Java serviceeclipse-temurin:21-jre internal mirrorpin major runtime, scan weekly
Node servicenode:22-bookworm-slim internal mirrorno dev dependencies in runtime
Go servicegcr.io/distroless/static or internal scratch templatestatic binary preferred
Nginx edgenginx:stable-alpine internal mirrorconfig owned by platform
Debug variantfull distro debug imagenever production default

Governance point:

An approved base image list without an update process becomes stale documentation.

8. Base Image Update Strategy

Base image update harus menjadi recurring workflow.

Practices:

  • rebuild images even when app code does not change;
  • track base image digest in metadata;
  • use dependency bot/scout recommendations if available;
  • separate emergency rebuild from regular release;
  • test runtime compatibility;
  • have rollback digest.

9. Labels and OCI Metadata

Labels membantu traceability.

Dockerfile:

ARG VERSION
ARG REVISION
ARG CREATED

LABEL org.opencontainers.image.title="billing-api" \
      org.opencontainers.image.description="Billing API service" \
      org.opencontainers.image.version="$VERSION" \
      org.opencontainers.image.revision="$REVISION" \
      org.opencontainers.image.created="$CREATED" \
      org.opencontainers.image.source="https://github.com/company/billing" \
      org.opencontainers.image.licenses="Apache-2.0"

Do not put secrets in labels.

Good labels:

  • service name;
  • commit SHA;
  • version;
  • source repository;
  • build timestamp;
  • license;
  • vendor.

Bad labels:

  • token;
  • internal password;
  • customer identifier;
  • private endpoint with embedded credential;
  • deploy approval secret.

10. SBOM Mental Model

SBOM = Software Bill of Materials.

It answers:

What components are inside this artifact?

It does not automatically answer:

Is this artifact safe?

SBOM is inventory. Risk decision needs inventory + vulnerability data + exploitability + context + policy.


11. What an SBOM Usually Contains

Typical SBOM records:

  • package name;
  • package version;
  • package type/ecosystem;
  • supplier/origin;
  • license;
  • file/package relationships;
  • checksums;
  • dependency relationships;
  • metadata about generation tool/time.

Container image SBOM often includes:

  • OS packages;
  • language dependencies;
  • application packages;
  • sometimes files/binaries;
  • sometimes base image relation;
  • sometimes license data.

Caveat:

SBOM quality depends on generator coverage and package metadata quality.

12. SBOM Formats

Common formats:

FormatCommon Use
SPDXlegal/license and software package data, widely standardized
CycloneDXapplication security and dependency inventory
Syft JSONtool-specific output from Anchore Syft
in-toto/SLSA provenancebuild provenance, not SBOM itself

Do not over-optimize early. For engineering workflow, first ensure:

  1. every production image has an SBOM;
  2. SBOM is tied to image digest;
  3. SBOM can be retrieved later;
  4. SBOM is used by scanner/policy;
  5. SBOM generation is automated.

13. Docker Build SBOM Attestation

Docker Buildx/BuildKit can produce SBOM attestations.

Example:

docker buildx build \
  --tag registry.example.com/billing/api:1.4.2 \
  --sbom=true \
  --push \
  .

Equivalent explicit form:

docker buildx build \
  --tag registry.example.com/billing/api:1.4.2 \
  --attest type=sbom \
  --push \
  .

Mental model:

An SBOM attestation attaches inventory evidence to the image artifact/digest in the registry ecosystem.

Do not store SBOM only as a detached CI log artifact that disappears after retention window.


14. Provenance Mental Model

Provenance answers:

How was this artifact built?

A useful provenance record can include:

  • builder identity;
  • source repository;
  • commit SHA;
  • build workflow;
  • build arguments/parameters;
  • build start/end time;
  • materials/input digests;
  • build platform;
  • invocation metadata.

SBOM says:

What is in it?

Provenance says:

Where did it come from and how was it produced?

Signature says:

Who/what claims this artifact is authentic?

Policy says:

Is this acceptable for this environment?

15. Docker Build Provenance Attestation

Example:

docker buildx build \
  --tag registry.example.com/billing/api:1.4.2 \
  --provenance=mode=max \
  --push \
  .

Combined:

docker buildx build \
  --tag registry.example.com/billing/api:1.4.2 \
  --sbom=true \
  --provenance=mode=max \
  --push \
  .

Trade-off:

  • more provenance detail improves auditability;
  • some metadata may expose internal details;
  • review what is emitted before enabling broad external publishing;
  • internal registry can often store richer evidence than public distribution.

16. Vulnerability Scanning Mental Model

Scanner compares image inventory against vulnerability intelligence.

Scanner output is not a final decision. It is input to risk triage.

Questions:

  • Is vulnerable package actually present?
  • Is vulnerable code path reachable?
  • Is vulnerability fixed upstream?
  • Is patched base image available?
  • Is exploit public/active?
  • Is service exposed externally?
  • Is compensating control present?
  • Is exception time-limited?

17. Docker Scout

Docker Scout analyzes images, creates/infer inventories/SBOM, matches them against vulnerability data, and supports policy evaluation.

Typical commands:

docker scout quickview registry.example.com/billing/api:1.4.2

docker scout cves registry.example.com/billing/api:1.4.2

docker scout recommendations registry.example.com/billing/api:1.4.2

Policy evaluation:

docker scout policy registry.example.com/billing/api:1.4.2

Use Scout as part of engineering loop:

  • local quick feedback;
  • PR gate;
  • release gate;
  • registry monitoring;
  • base image update recommendation;
  • org policy compliance.

18. Vulnerability Severity Is Not Enough

Severity alone is insufficient.

Example matrix:

VulnerabilitySeverityRuntime ExposureDecision
critical OpenSSL in public TLS edgecriticalreachableblock release
critical package in unused debug tool not in prod imagecriticalabsentnot applicable after image fix
high CVE in build-stage only dependencyhighnot in final imageacceptable if final scan clean
medium vulnerability in internal admin toolmediumVPN-only + authfix scheduled
low vulnerability in base packagelowlowmonitor

Better gate:

Block critical/high vulnerabilities with available fixes unless exception is approved and expires.

19. False Positives and False Negatives

Vulnerability scanning has limits.

False positive examples:

  • package version patched by distro backport but scanner sees old upstream version;
  • vulnerable package installed but vulnerable feature not used;
  • scanner misidentifies binary/package;
  • dev dependency appears but not used at runtime.

False negative examples:

  • statically linked binary not detected;
  • manually copied binary has no package metadata;
  • language dependency hidden in vendored directory;
  • vulnerability database lacks advisory;
  • SBOM generator misses ecosystem.

Engineering implication:

Scanning is necessary but not sufficient.

Use:

  • minimal images;
  • dependency lockfiles;
  • SBOM;
  • base image governance;
  • regular rebuilds;
  • runtime controls;
  • security review for high-risk components.

20. Fix Strategy

When scan finds vulnerabilities, fix in order:

  1. Update base image digest/tag.
  2. Rebuild image.
  3. Update OS packages only if base-image policy allows it.
  4. Update language dependencies.
  5. Remove unnecessary packages.
  6. Move build-only tools out of runtime image.
  7. Switch to smaller runtime base.
  8. Add exception only with reason and expiry.

Bad habit:

RUN apt-get update && apt-get upgrade -y

This can make image less reproducible and drift from base image lifecycle.

Better in many orgs:

Update base image centrally, rebuild dependents, promote new digest.

21. Build Stage vs Runtime Stage Scanning

Multi-stage build reduces runtime surface, but only if final image is clean.

Example:

FROM node:22-bookworm AS build
WORKDIR /src
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM nginx:1.27-alpine AS runtime
COPY --from=build /src/dist /usr/share/nginx/html

Scan final image:

docker scout cves company/web:1.0.0

But also consider scanning build images if they are pushed/cached/shared.

Rule:

Anything pushed to a registry is an artifact and should be governed.

22. Secret Scanning in Image Supply Chain

Part 023 covered secret leakage. In supply chain gate, add secret scan.

Search final image filesystem for obvious patterns:

docker run --rm company/app:1.0.0 sh -c '
  find / -type f \( -name "*.pem" -o -name "id_rsa" -o -name "*.key" -o -name ".npmrc" \) 2>/dev/null
'

Use dedicated scanners in CI/registry where possible.

Policy:

If a real secret is found in an image, block promotion and rotate the secret.

Deleting/rebuilding is not enough.


23. Signing Mental Model

Signing answers:

Did an identity we trust sign this artifact digest?

Signing does not answer:

Is the artifact vulnerability-free?

A signed vulnerable image is still vulnerable.

A signed malicious image built by compromised CI may still verify unless provenance/policy catches the issue.

Signing is one layer:


24. Docker Content Trust

Docker Content Trust uses Notary-based signing through Docker CLI commands such as docker trust.

Example pattern:

export DOCKER_CONTENT_TRUST=1

docker push registry.example.com/billing/api:1.4.2

Manual trust command style:

docker trust sign registry.example.com/billing/api:1.4.2

Important caveat:

Content Trust is one trust mechanism, but modern supply-chain workflows often also evaluate SBOM, provenance, CI identity, and policy. Do not treat signing alone as sufficient.

Many organizations use Sigstore Cosign or registry-native signing/attestation workflows alongside or instead of older trust workflows, depending on platform compatibility.


25. Sigstore Cosign Pattern

Cosign is widely used for container image signing and verification.

Keyless signing pattern:

cosign sign registry.example.com/billing/api@sha256:...

Verify:

cosign verify \
  --certificate-identity-regexp 'https://github.com/company/billing/.github/workflows/release.yml@refs/tags/.*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  registry.example.com/billing/api@sha256:...

Conceptual value:

  • signature is bound to digest;
  • keyless flow can bind signing to OIDC identity;
  • transparency log can improve auditability;
  • CI identity can become part of trust policy.

Caveat:

  • verification policy must be enforced;
  • signing key/identity compromise remains a risk;
  • signature does not replace vulnerability management.

26. Policy Gate Architecture

A serious pipeline has gates.

Policy examples:

[ ] image has digest
[ ] image has SBOM
[ ] image has provenance
[ ] image has valid signature from approved CI identity
[ ] no critical/high CVE with fix available
[ ] no secret detected
[ ] base image approved
[ ] image runs as non-root
[ ] no privileged runtime setting in deployment spec
[ ] license policy satisfied

27. Example CI Gate Script

Illustrative script:

#!/usr/bin/env bash
set -euo pipefail

IMAGE="${IMAGE:?must set IMAGE}"

# Build and push with metadata.
docker buildx build \
  --tag "$IMAGE" \
  --sbom=true \
  --provenance=mode=max \
  --push \
  .

# Basic vulnerability check.
docker scout cves "$IMAGE"

# Policy evaluation if configured.
docker scout policy "$IMAGE"

# Optional signing after policy passes.
DIGEST_REF="$(docker buildx imagetools inspect "$IMAGE" --format '{{json .Manifest.Digest}}' | tr -d '"')"

echo "Built digest: $DIGEST_REF"

Real pipelines should avoid fragile shell parsing. Store digest as CI output from build action/tooling when possible.


28. GitHub Actions Pattern with Docker Buildx

Illustrative workflow fragment:

name: release-image

on:
  push:
    tags:
      - 'v*'

permissions:
  contents: read
  packages: write
  id-token: write

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: docker/setup-buildx-action@v3

      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - uses: docker/build-push-action@v6
        with:
          context: .
          push: true
          tags: ghcr.io/company/billing-api:${{ github.ref_name }}
          sbom: true
          provenance: mode=max

Add signing/policy according to organization tooling.


29. Registry Promotion Model

Avoid rebuilding separately for each environment.

Bad:

build dev image
build staging image
build prod image

Better:

build once -> scan/sign -> promote same digest across environments

Promotion should move trust decision, not mutate artifact.

Good release record:

staging approved digest sha256:X
prod approved same digest sha256:X

If production rebuilds from same source, you have two artifacts and need two evidence chains.


30. Registry Tag Strategy

Recommended tag set:

TagPurposeMutable?
1.4.2semantic releaseshould not mutate
1.4minor channelmay move by policy
1major channelmay move by policy
git-3f2a9c1commit traceshould not mutate
build-20260701.1234build traceshould not mutate
latestconvenience onlymutable

Production deployment should not rely on latest.

Policy:

Mutable tags are channels, not evidence.

31. Registry Immutability

If registry supports immutable tags, use them for release tags.

Benefits:

  • prevents accidental overwrite;
  • protects release evidence;
  • simplifies incident analysis;
  • makes rollback reliable;
  • reduces mutable tag attack.

Still store digest.

Immutable tag protects name.
Digest identifies content.

32. Pull Policy and Deployment Drift

Runtime can drift if deployment uses mutable tags and nodes pull at different times.

Example failure:

node A pulls api:latest at 10:00 -> digest X
node B pulls api:latest at 10:05 -> digest Y
same service version now runs different code

Mitigation:

  • deploy digest;
  • use immutable release tags;
  • record resolved digest;
  • avoid latest in production;
  • force controlled rollout after promotion.

Swarm stack file can use digest references:

services:
  api:
    image: registry.example.com/billing/api@sha256:...

33. License Governance

Image supply chain is not only CVE.

SBOM can support license review.

Questions:

  • Are licenses allowed for distribution model?
  • Are copyleft obligations understood?
  • Are commercial dependencies licensed?
  • Is license metadata accurate?
  • Are generated artifacts including third-party assets?

Policy examples:

Block GPL-3.0 in proprietary distributed client artifact unless approved.
Warn for unknown license.
Require legal review for custom license.

For backend-only services, license risk differs from distributed software, but still needs governance.


34. Non-Root and Runtime Policy as Supply Chain Signals

Supply chain policy can inspect image config.

Check default user:

docker image inspect company/api:1.0.0 --format '{{json .Config.User}}'

Check healthcheck:

docker image inspect company/api:1.0.0 --format '{{json .Config.Healthcheck}}'

Check labels:

docker image inspect company/api:1.0.0 --format '{{json .Config.Labels}}' | jq

Policy examples:

  • image must not run as root by default;
  • image must have OCI labels;
  • image must not include secret-looking env vars;
  • image must not define privileged assumptions;
  • image must expose only expected ports;
  • image must define a sane entrypoint.

35. Image Size Is a Supply Chain Signal

Large images are not automatically insecure, but size often correlates with:

  • unused packages;
  • build tools left in runtime;
  • debug utilities in production;
  • larger scan surface;
  • slower pull/startup;
  • more patch obligations.

Check:

docker image ls company/api

docker history company/api:1.0.0

Policy should not blindly block by size, but can flag anomalies:

billing-api runtime image grew from 250MB to 1.2GB in one release.

That deserves review.


36. Reproducibility Reality

Docker improves environment packaging, but does not automatically guarantee reproducible builds.

Sources of non-reproducibility:

  • unpinned package versions;
  • apt-get update against moving repository;
  • mutable base tags;
  • timestamps in artifacts;
  • random build output;
  • network downloads without checksum;
  • architecture differences;
  • dependency lockfile drift;
  • build args not captured;
  • external scripts fetched at build time.

Better patterns:

  • pin base image digest for release builds;
  • use lockfiles;
  • verify checksums for downloaded artifacts;
  • avoid curl | sh;
  • record build provenance;
  • build once, promote digest;
  • rebuild regularly under controlled process.

37. Dangerous Pattern: curl | sh

Anti-pattern:

RUN curl -fsSL https://example.com/install.sh | sh

Risks:

  • remote script mutable;
  • no checksum;
  • hard to audit;
  • build non-deterministic;
  • attacker controlling endpoint can affect build.

Better:

ARG TOOL_VERSION=1.2.3
ARG TOOL_SHA256=...
RUN curl -fsSLo /tmp/tool.tar.gz "https://example.com/tool-${TOOL_VERSION}.tar.gz" \
    && echo "${TOOL_SHA256}  /tmp/tool.tar.gz" | sha256sum -c - \
    && tar -xzf /tmp/tool.tar.gz -C /usr/local/bin \
    && rm -f /tmp/tool.tar.gz

38. Dependency Lockfiles

Lockfiles are supply-chain artifacts.

Examples:

EcosystemLockfile
npmpackage-lock.json
yarnyarn.lock
pnpmpnpm-lock.yaml
Mavenless strict by default; use dependency management / enforcer
Gradledependency locking
Gogo.sum
Pythonrequirements.txt with hashes / lock tools
RustCargo.lock

Docker build should copy lockfiles before installing dependencies to get stable cache and stable dependency graph.

Example:

COPY package.json package-lock.json ./
RUN npm ci

Not:

RUN npm install

for production build reproducibility.


39. Build Context Policy

Part 023 covered secrets. Supply chain adds additional concerns.

.dockerignore should exclude:

  • .git if provenance captured elsewhere;
  • local build outputs;
  • credentials;
  • test coverage;
  • node_modules/target/build unless intentionally copied;
  • editor/IDE files;
  • local caches.

But be careful: excluding .git can make build scripts unable to derive commit. Prefer passing commit as build arg from CI:

docker buildx build \
  --build-arg REVISION="$GIT_SHA" \
  --label "org.opencontainers.image.revision=$GIT_SHA" \
  .

The value is non-secret metadata, so ARG is appropriate.


40. Policy as Code

Manual checklist does not scale.

Supply-chain policy should be executable.

Policy examples:

deny if image tag == latest in production
warn if base image not approved
deny if critical CVE has fix and no exception
deny if image lacks SBOM attestation
deny if image lacks provenance
deny if signer identity is not approved CI workflow
deny if default user is root
warn if image size increased > 50%
deny if secret scan detects credential

Tools vary, but architecture is stable:


41. Exception Management

Zero vulnerability policy is often unrealistic. Unlimited exceptions are dangerous.

Good exception fields:

FieldExample
image digestsha256:...
vulnerabilityCVE-...
affected packagelibxyz 1.2.3
environmentstaging/prod
justificationnot reachable due to disabled module
compensating controlnetwork isolation + WAF rule
ownerplatform-security
expiry2026-07-31
approvalsecurity lead
remediation issuelink/id

Rule:

Every exception must expire.

42. Release Evidence Bundle

For every production release, store an evidence bundle.

Minimum:

service: billing-api
environment: production
release: 1.4.2
imageDigest: sha256:...
sourceCommit: 3f2a9c1
builder: github-actions/release-image
buildTime: 2026-07-01T10:15:00Z
sbom: present
provenance: present
signature: verified
scanner: docker-scout
policyDecision: pass
exceptions: []
approvedBy: release-policy

This turns release from "someone deployed a tag" into auditable event.


43. Incident Response: Vulnerability Found After Deploy

When a new CVE affects deployed images:

Important capability:

Can we answer which running services contain package X version Y?

Without SBOM/image inventory, this becomes manual archaeology.


44. Incident Response: Compromised Image

If image is suspected malicious:

  1. Freeze tag/digest promotion.
  2. Identify all deployments using digest.
  3. Pull registry audit logs.
  4. Verify signature/provenance.
  5. Compare image filesystem/layers to expected build.
  6. Rotate registry credentials if push compromise possible.
  7. Rebuild from trusted clean pipeline.
  8. Promote new digest.
  9. Roll production.
  10. Revoke old tags or mark as blocked.
  11. Scan for persistence/runtime compromise.
  12. Document root cause.

Rule:

Do not overwrite the suspicious tag and call it fixed.
Keep evidence for investigation.

45. Image Promotion Anti-Patterns

Anti-PatternFailure Mode
rebuild per environmentartifacts differ, evidence fragmented
deploy latestdrift, rollback ambiguity
scan after deploy onlyproduction becomes test gate
sign without verifysignature is ceremonial
SBOM not tied to digestinventory mismatch
exceptions never expirerisk accumulates
base image owner unclearpatching stalls
registry push token broadcompromised CI can poison many repos
no immutable tagsaccidental overwrite
no release evidenceaudit failure

46. Production Image Standard

Reusable baseline:

1. Image must be built by approved CI, not manually from a laptop.
2. Image must include OCI metadata labels.
3. Image must be tagged with immutable release tag and recorded digest.
4. Image must have SBOM evidence tied to digest.
5. Image must have provenance evidence tied to build identity/source.
6. Image must pass vulnerability policy or have approved expiring exception.
7. Image must pass secret scan.
8. Image must use approved base image or approved exception.
9. Runtime image must exclude build-only tools unless explicitly approved.
10. Production deployment must use digest or verified immutable tag mapping.
11. Release evidence bundle must be stored with deployment record.

47. Dockerfile Supply Chain Checklist

[ ] Base image is approved.
[ ] Base image version/digest strategy is explicit.
[ ] Package installs are deterministic enough for release process.
[ ] Remote downloads use pinned versions and checksums.
[ ] No curl | sh without verification.
[ ] Lockfiles are copied and used.
[ ] Build-only tools do not remain in final image.
[ ] Dockerfile labels include source/version/revision.
[ ] No secrets in ARG/ENV/layers.
[ ] Final image runs as non-root where possible.
[ ] Final image has minimal runtime surface.

48. CI/CD Supply Chain Checklist

[ ] Build runs in approved CI identity.
[ ] Build uses isolated trusted runner class for release.
[ ] Build pushes to approved registry.
[ ] Digest is captured as release output.
[ ] SBOM attestation generated.
[ ] Provenance attestation generated.
[ ] Vulnerability scan executed.
[ ] Secret scan executed.
[ ] Image signed after build.
[ ] Signature verification tested.
[ ] Policy decision stored.
[ ] Release requires protected branch/tag/environment.
[ ] Registry credentials are scoped.

49. Registry Governance Checklist

[ ] Registry has access control by team/environment.
[ ] Push access is limited to CI release identities.
[ ] Pull access is limited to runtime/deploy identities.
[ ] Release tags are immutable where supported.
[ ] Old images have retention policy.
[ ] Critical images are mirrored if external dependency risk is high.
[ ] Registry audit logs are retained.
[ ] Vulnerability monitoring is enabled.
[ ] Attestations/signatures are preserved.
[ ] Deprecated/vulnerable digests can be blocked.

50. Deployment Policy Checklist

[ ] Production deploy does not use latest.
[ ] Deployment resolves image to expected digest.
[ ] Signature/provenance policy verified before deploy.
[ ] CVE policy evaluated before deploy.
[ ] Runtime hardening config checked.
[ ] Secrets are runtime-projected, not baked into image.
[ ] Rollback digest is known.
[ ] Deployed digest is recorded.
[ ] Post-deploy verification confirms expected digest running.

Swarm verification:

docker service inspect billing_api --format '{{json .Spec.TaskTemplate.ContainerSpec.Image}}'

51. Practical Lab 1 — Generate SBOM/Provenance with Buildx

Build:

docker buildx build \
  --tag localhost:5000/demo/app:1.0.0 \
  --sbom=true \
  --provenance=mode=max \
  --push \
  .

Inspect image metadata:

docker buildx imagetools inspect localhost:5000/demo/app:1.0.0

Learning questions:

What digest was produced?
Where is SBOM/provenance attached?
Does your registry preserve attestations?
Can your deployment system see or verify them?

52. Practical Lab 2 — Compare Tag and Digest

Pull tag:

docker pull alpine:3.20

Inspect repo digest:

docker image inspect alpine:3.20 --format '{{json .RepoDigests}}' | jq

Run by digest:

docker run --rm alpine@sha256:<digest> cat /etc/alpine-release

Learning point:

The tag helps humans find the image. The digest identifies the exact content.

53. Practical Lab 3 — Docker Scout Scan

Run quick view:

docker scout quickview alpine:3.20

Run CVE scan:

docker scout cves alpine:3.20

Run recommendations:

docker scout recommendations alpine:3.20

Learning questions:

  • Which packages are present?
  • Which CVEs have fixes?
  • Does scanner distinguish base image update recommendation?
  • What would your release gate block?
  • What would your release gate warn?

54. Practical Lab 4 — Detect Root Default User

Dockerfile bad:

FROM alpine:3.20
CMD ["sleep", "3600"]

Build:

docker build -t user-bad:1 .

Inspect:

docker image inspect user-bad:1 --format '{{json .Config.User}}'

Dockerfile better:

FROM alpine:3.20
RUN addgroup -S app && adduser -S -G app app
USER app
CMD ["sleep", "3600"]

Build and inspect:

docker build -t user-good:1 .
docker image inspect user-good:1 --format '{{json .Config.User}}'

Learning point:

Supply-chain policy can inspect image config before runtime.

55. Practical Lab 5 — Secret Scan Sanity Check

Create bad image:

FROM alpine:3.20
RUN echo 'AWS_SECRET_ACCESS_KEY=example-secret' > /app.env

Build:

docker build -t secret-image-bad:1 .

Manual check:

docker run --rm secret-image-bad:1 sh -c 'grep -R "AWS_SECRET" / 2>/dev/null || true'

Expected:

The build must fail in real CI if this is a real credential.

56. Supply Chain Maturity Model

LevelBehavior
0manual build/push from laptop, mutable tags
1CI builds image, basic tags, manual deploy
2CI scan, release tags, no latest in prod
3digest promotion, SBOM, base image policy
4provenance, signing, policy gates, exception workflow
5continuous monitoring, automated rebuilds, admission enforcement, audit-grade evidence

Goal for top-tier engineering org:

At least Level 4 for production-critical services.
Level 5 for regulated/high-risk platforms.

57. What Good Looks Like

A strong image supply chain has these properties:

  1. Every artifact has immutable identity.
  2. Every artifact is built by trusted automation.
  3. Every artifact has inventory.
  4. Every artifact has origin evidence.
  5. Every artifact is scanned.
  6. Every artifact is signed or otherwise authenticity-verified.
  7. Every release has policy decision.
  8. Every exception expires.
  9. Every deployment records digest.
  10. Every vulnerable digest can be found quickly.
  11. Every production image can be rebuilt or replaced through a known process.
  12. Every base image has an owner.

58. What Not to Overclaim

Avoid these false claims:

"We have SBOM, so we are secure."

SBOM is inventory, not security.

"We scanned it, so it is safe."

Scanner can miss issues and context matters.

"It is signed, so it is safe."

Signing proves identity/integrity under policy; it does not remove vulnerabilities.

"It is in a private registry, so it is trusted."

Private registry can still host bad artifacts.

"Docker makes builds reproducible."

Docker helps package environments, but reproducibility requires discipline.


59. Decision Framework

Use this decision flow for release:


60. Engineering Handbook Standard

Reusable standard:

Production Container Image Supply Chain Standard

1. All production images must be built by approved CI pipelines.
2. Manual production image pushes are prohibited except under break-glass with audit.
3. All production images must have immutable release tags and recorded digests.
4. Deployments must use image digest or verify immutable tag-to-digest mapping.
5. All production images must have SBOM and provenance evidence tied to digest.
6. All production images must pass vulnerability and secret scanning gates.
7. All production images must be signed or verified by approved authenticity mechanism.
8. Base images must come from approved sources and have assigned owners.
9. Exceptions must be approved, justified, assigned an owner, and expire.
10. The organization must be able to answer: "where is digest X running?" and "which running images contain package Y?"

61. Mental Model Recap

Image supply chain is evidence engineering.

A container image is not trustworthy because it runs.
A container image is trustworthy when its identity, contents, origin, risk, and approval are known and verified.

Core invariants:

  1. Tags are names; digests are content identity.
  2. Build once, promote the same digest.
  3. SBOM is inventory, not safety.
  4. Scanning is necessary, not sufficient.
  5. Signing proves authenticity, not absence of vulnerabilities.
  6. Provenance connects artifact to build/source.
  7. Base image governance controls inherited risk.
  8. Exceptions must expire.
  9. Release evidence should survive log retention.
  10. Runtime deployment should record exact digest.

62. References


63. What Comes Next

Part 025 akan memulai fase Docker Swarm:

  • Swarm mode architecture;
  • manager/worker;
  • Raft;
  • desired state;
  • service/task model;
  • scheduler;
  • quorum;
  • operational invariants.
Lesson Recap

You just completed lesson 24 in deepen practice. Use the series map if you want to review the broader track, or continue directly into the next lesson while the context is still warm.

Continue The Track

Keep the momentum while the lesson is still fresh. Move backward for review or continue forward into the next concept.