Part 033 — Docker and Kubernetes Security Supply Chain

Part sebelumnya membahas private endpoint dan private connectivity.

Part ini membahas security supply chain untuk containerized Java/JAX-RS services: dari source code, dependency, build pipeline, base image, image registry, admission policy, sampai pod berjalan di Kubernetes.

Masalah supply chain tidak selalu terlihat sebagai bug aplikasi. Ia sering muncul sebagai:

• image berisi CVE critical,
• base image tidak pernah dipatch,
• dependency Java rentan,
• secret ikut masuk image layer,
• image tag bisa ditimpa,
• deployment memakai image yang tidak bisa ditelusuri ke commit,
• image diambil dari registry yang salah,
• CI runner memiliki permission terlalu luas,
• admission policy tidak mencegah image tidak sah,
• Docker socket dibuka ke workload build,
• SBOM tidak tersedia saat audit,
• artifact production tidak bisa dibuktikan provenance-nya.

CSG note: jangan mengasumsikan CSG memakai scanner tertentu, SBOM tool tertentu, image signing tertentu, admission controller tertentu, registry tertentu, atau SLSA maturity tertentu. Semua detail harus diverifikasi di CI/CD pipeline, Dockerfile, Maven build, dependency scanning report, container registry, GitOps repository, admission policy, security dashboard, dan diskusi dengan platform/SRE/security team.

1. Core Concept

Security supply chain adalah rantai kepercayaan dari source code sampai runtime.

Untuk service Java/JAX-RS di Kubernetes, rantainya kira-kira seperti ini:

Git commit
  -> Maven dependency resolution
  -> unit/integration/security tests
  -> container image build
  -> image scan + SBOM generation
  -> image signing/provenance
  -> image push to registry
  -> manifest updated with tag/digest
  -> GitOps/CI deploys to cluster
  -> admission controller validates policy
  -> kubelet pulls image
  -> pod runs under runtime security controls

Supply-chain security bertanya:

Can we prove what is running, where it came from, who built it,
what dependencies it contains, whether it was scanned,
and whether it was allowed by policy?

Jika jawabannya tidak jelas, production memiliki blind spot.

2. Why This Exists

Container membuat deployment lebih portable, tetapi juga membuat artifact lebih kompleks.

Satu image bisa berisi:

• OS packages,
• Java runtime,
• application jar/war,
• transitive Maven dependencies,
• shell utilities,
• certificate bundles,
• timezone data,
• native libraries,
• generated files,
• build leftovers,
• accidental secrets.

Kubernetes kemudian menjalankan image tersebut secara otomatis di banyak node.

Artinya:

A weak image becomes a replicated production risk.
A weak pipeline becomes a repeatable vulnerability factory.

3. Threat Model for Container Supply Chain

Supply-chain risk bukan hanya “ada CVE”. Beberapa threat yang lebih luas:

4. Dependency Scanning vs Image Scanning

Ada dua scanning yang sering dicampur:

4.1 Dependency Scanning / SCA

Fokus pada dependency aplikasi.

Untuk Java/JAX-RS:

• Maven dependencies,
• transitive dependencies,
• Jakarta/JAX-RS libraries,
• HTTP client libraries,
• database drivers,
• Kafka/RabbitMQ/Redis clients,
• JSON/XML libraries,
• logging libraries,
• test dependencies jika ikut ke runtime artifact.

Pertanyaan review:

Dependency mana yang benar-benar masuk runtime artifact?
Dependency mana hanya test/build time?
Apakah vulnerable dependency reachable dari code path aplikasi?
Apakah ada fixed version yang compatible?

4.2 Image Scanning

Fokus pada isi image final.

Yang discan:

• OS packages,
• JVM runtime packages,
• native libraries,
• package manager metadata,
• application dependency jika scanner mendukung,
• file system layer.

Pertanyaan review:

Apakah CVE berasal dari base image atau aplikasi?
Apakah package vulnerable dipakai runtime?
Apakah fixed base image tersedia?
Apakah image final terlalu besar sehingga attack surface tinggi?

5. SBOM

SBOM adalah inventory komponen software dalam artifact.

Untuk container image, SBOM membantu menjawab:

• library apa saja di image,
• versi dependency apa saja,
• OS package apa saja,
• base image apa,
• license apa yang muncul,
• apakah artifact terdampak CVE tertentu,
• apakah dependency tertentu dipakai di production.

Mental model:

SBOM is not security by itself.
SBOM is evidence and inventory for security decisions.

Tanpa SBOM, CVE triage menjadi tebak-tebakan.

Untuk Java service, SBOM harus membantu menghubungkan:

Maven dependency -> packaged artifact -> container image -> running pod

6. CVE Triage

Tidak semua CVE memiliki risiko production yang sama.

Triage harus mempertimbangkan:

• severity,
• exploitability,
• reachability,
• exposure path,
• runtime privilege,
• network access,
• data sensitivity,
• compensating control,
• patch availability,
• compatibility risk,
• production blast radius.

Contoh reasoning:

Critical CVE in unused shell utility inside distroless-like image
may be lower practical risk than medium CVE in reachable JSON parser
on public API input path.

Namun jangan menjadikan reachability sebagai alasan malas patch. Gunakan sebagai prioritas, bukan pembenaran permanen.

7. Base Image Patching

Base image adalah dependency besar yang sering dilupakan.

Review base image harus menjawab:

• image berasal dari publisher tepercaya,
• image digest tercatat,
• update cadence jelas,
• patch process tersedia,
• compatibility test tersedia,
• image lama tidak dipakai terlalu lama,
• image final tidak memakai build image penuh,
• package manager cache dibersihkan,
• shell/tooling tidak perlu di runtime image.

Anti-pattern:

FROM openjdk:latest

Masalah:

• tag mutable,
• sulit audit,
• tidak tahu update kapan,
• reproducibility buruk,
• rollback tidak deterministik.

Lebih baik:

FROM eclipse-temurin:17-jre-jammy@sha256:<digest>

Atau gunakan base image internal yang sudah distandardisasi dan dipatch oleh platform/security team.

8. Image Tag, Digest, and Promotion

Tag mudah dibaca manusia, digest akurat untuk mesin.

Production deployment idealnya bisa menjawab:

Pod ini menjalankan image digest apa?
Digest itu dibangun dari commit apa?
Commit itu melewati pipeline mana?
Scan report-nya apa?
SBOM-nya mana?
Siapa/apa yang mempromosikan ke production?

Promotion yang baik tidak rebuild image per environment.

Pattern yang lebih aman:

Build once -> scan once -> sign once -> promote same digest across environments

Anti-pattern:

Build dev image
Build staging image separately
Build prod image separately

Masalahnya: environment berbeda mungkin menjalankan binary berbeda walau source commit sama.

9. Image Signing and Provenance

Image signing membuktikan image ditandatangani oleh identity yang dipercaya.

Provenance membuktikan bagaimana artifact dibuat.

Pertanyaan penting:

• siapa builder-nya,
• pipeline mana yang membangun,
• source commit mana,
• parameter build apa,
• dependency/source apa yang dipakai,
• kapan dibangun,
• apakah build reproducible,
• apakah artifact sudah ditandatangani,
• apakah admission policy memverifikasi signature.

Mental model:

Signing without admission enforcement is mostly evidence.
Signing with admission enforcement becomes a runtime gate.

10. Admission Policy

Admission controller dapat mencegah object yang tidak sesuai policy masuk cluster.

Policy supply chain dapat mencakup:

• image harus dari registry tertentu,
• image tidak boleh memakai latest,
• image harus memakai digest,
• image harus signed,
• image harus punya scan status acceptable,
• image harus memenuhi vulnerability threshold,
• container harus run as non-root,
• privileged container dilarang,
• hostPath dilarang,
• resource request wajib,
• liveness/readiness probe wajib untuk workload tertentu.

Contoh policy intent:

Production namespace only accepts images from approved registry
with immutable digest and valid signature.

Admission policy harus punya exception process. Tanpa exception process, engineer akan mencari bypass.

11. Registry Access Control

Registry bukan hanya tempat menyimpan image. Registry adalah security boundary.

Hal yang perlu dikontrol:

• siapa bisa push,
• siapa bisa pull,
• siapa bisa delete,
• siapa bisa overwrite tag,
• apakah tag immutability aktif,
• apakah production image bisa dimodifikasi manual,
• apakah cross-account/cross-project pull dibatasi,
• apakah image pull secret dirotasi,
• apakah audit log tersedia,
• apakah retention policy aman.

Risk pattern:

CI can push to prod registry, developers can overwrite tags,
cluster accepts mutable tags, and no admission verification exists.

Itu berarti trust boundary sangat lemah.

12. Build Secret Leakage

Secret dapat bocor ke image lewat:

• ARG,
• ENV,
• file .npmrc, .m2/settings.xml, credential helper,
• copied config file,
• build logs,
• intermediate layer,
• shell history,
• package manager cache,
• test fixture,
• accidentally committed file.

Anti-pattern:

ARG MAVEN_TOKEN
RUN echo $MAVEN_TOKEN > /root/.m2/settings.xml

Masalah:

• token bisa muncul di layer history,
• bisa tersimpan di intermediate layer,
• bisa bocor di build logs.

Pattern lebih aman:

# Conceptual only: use BuildKit secret mount
RUN --mount=type=secret,id=maven_settings,target=/root/.m2/settings.xml \
    mvn -B -DskipTests package

Tetap verifikasi apakah CI/CD dan builder mendukung secret mount dengan benar.

13. Docker Socket Risk

Mount Docker socket ke container build sangat berisiko.

Contoh:

volumes:
  - /var/run/docker.sock:/var/run/docker.sock

Mengapa berbahaya?

Container yang punya akses ke Docker socket pada dasarnya bisa mengontrol Docker daemon host.

Ia bisa:

• menjalankan container privileged,
• mount filesystem host,
• membaca secret host,
• memodifikasi image/container lain,
• memperluas compromise dari CI job ke runner host.

Alternatif yang perlu dipertimbangkan:

• rootless builder,
• isolated build runner,
• Kaniko/BuildKit/buildah pattern sesuai kebijakan platform,
• ephemeral CI runner,
• permission minimal,
• no shared long-lived runner untuk untrusted workloads.

14. CI Runner Security

CI runner adalah bagian kritis dari supply chain.

Checklist runner:

• runner ephemeral jika memungkinkan,
• permission cloud minimal,
• token lifetime pendek,
• secret hanya tersedia untuk branch/environment tertentu,
• PR dari fork tidak mendapat secret production,
• build logs disanitasi,
• artifact signing key tidak bisa diakses sembarang job,
• cache tidak membocorkan dependency credential,
• runner tidak reuse workspace secara tidak aman,
• network egress dikontrol,
• Docker daemon/socket tidak diekspos tanpa alasan kuat.

Pertanyaan penting:

Jika CI runner compromise, apa yang bisa attacker lakukan?
Push image? Sign image? Update GitOps repo? Deploy to prod? Read secrets?

Jawaban atas pertanyaan ini menentukan blast radius.

15. Java/JAX-RS Specific Concerns

Untuk Java/JAX-RS service, supply chain concern sering ada di:

• transitive Maven dependencies,
• vulnerable JSON/XML parser,
• old HTTP client,
• logging framework vulnerability,
• database driver vulnerability,
• Kafka/RabbitMQ/Redis client library,
• Jakarta/JAX-RS implementation runtime,
• embedded server dependency,
• dependency shading yang menyembunyikan versi asli,
• fat jar yang sulit diinspeksi,
• native library dari image base,
• CA certificate store,
• timezone/native dependency.

Review harus bertanya:

Apakah scanner melihat dependency di fat jar?
Apakah shaded dependency muncul di SBOM?
Apakah runtime image berisi tool yang tidak perlu?
Apakah dependency test ikut packaged?

16. Kubernetes Runtime Enforcement

Supply chain tidak selesai saat image dipush.

Cluster harus enforce runtime policy:

• image source restriction,
• signature verification,
• pod security standard,
• read-only root filesystem,
• non-root user,
• capabilities drop,
• seccomp profile,
• no privileged container,
• no hostPath kecuali exception,
• no hostNetwork/hostPID kecuali exception,
• resource request/limit,
• secret/RBAC least privilege,
• NetworkPolicy where required.

Tanpa runtime enforcement, image yang “baik” masih bisa dijalankan dengan konfigurasi buruk.

17. Artifact Promotion Flow

Promotion flow yang sehat:

Developer commit
  -> CI build image
  -> tests pass
  -> scan pass or approved exception
  -> SBOM generated
  -> image signed
  -> push immutable digest
  -> update lower env manifest
  -> promote same digest
  -> deploy via GitOps/controlled pipeline
  -> admission validates
  -> runtime observed

Promotion flow yang lemah:

Developer builds local image
  -> pushes mutable tag
  -> manually updates deployment
  -> no scan
  -> no SBOM
  -> no signature
  -> no traceability

18. Failure Modes

Common failure modes:

19. Debugging Workflow

When deployment is blocked by supply-chain controls:

1. Identify where it failed
   - dependency scan
   - image scan
   - signing
   - registry push/pull
   - manifest update
   - admission
   - kubelet image pull

2. Identify artifact identity
   - image repository
   - tag
   - digest
   - commit SHA
   - build run

3. Check evidence
   - scan report
   - SBOM
   - signature
   - provenance
   - exception approval

4. Check policy
   - allowed registry
   - severity threshold
   - signature identity
   - namespace policy
   - image pull permission

5. Fix root cause
   - patch dependency/base image
   - rebuild and promote same digest
   - update policy only with approved exception
   - correct registry auth
   - correct manifest digest

Do not bypass supply-chain policy casually. Bypass becomes production debt.

20. Review Checklist

Image and Dependency

• Base image approved.
• Base image version/digest pinned.
• Runtime image minimal.
• Build artifacts not leaked into runtime image.
• Maven dependencies scanned.
• Transitive dependency risk reviewed.
• Fat jar/shaded jar scanning works.
• SBOM generated.
• CVE exception documented.

Registry and Promotion

• Image pushed to approved registry.
• Production does not use latest.
• Digest traceable.
• Tag immutability enforced where needed.
• Same digest promoted across environments.
• Registry ACL least privilege.
• Retention policy does not remove active production image.

Signing and Provenance

• Image signing enabled if required.
• Signature verified before production admission if required.
• Build provenance available.
• Commit-to-image-to-pod traceability exists.
• Signing key/identity protected.

CI/CD Security

• CI runner has least privilege.
• Production secrets not exposed to untrusted jobs.
• Docker socket risk reviewed.
• Build secrets not written to image layers.
• Logs do not expose credentials.
• Pipeline cannot silently skip security gates.

Kubernetes Enforcement

• Admission policy validates image/source/security requirements.
• Runtime security context enforced.
• RBAC least privilege.
• NetworkPolicy considered.
• Audit evidence available.

21. Internal Verification Checklist

Untuk konteks CSG/team, verifikasi:

• Registry apa yang digunakan: ECR, ACR, private registry, atau lainnya.
• Apakah production image memakai immutable digest atau mutable tag.
• Apakah tag immutability aktif.
• Apakah image scan wajib sebelum promotion.
• Tool dependency scanning/SCA apa yang dipakai.
• Apakah SBOM dibuat dan disimpan.
• Apakah image signing/provenance digunakan.
• Apakah admission controller memverifikasi image policy.
• Apakah ada CVE exception process.
• Apakah base image dikelola platform/security team.
• Apakah Dockerfile memakai BuildKit secret mount atau pattern aman lain.
• Apakah CI runner punya Docker socket access.
• Siapa yang bisa push ke registry production.
• Siapa yang bisa deploy/promote ke production.
• Apakah GitOps repo menyimpan tag atau digest.
• Bagaimana commit-to-pod traceability dilakukan.
• Bagaimana incident supply-chain dicatat dan dieskalasi.

22. Senior Engineer Mental Model

Supply-chain security bukan pekerjaan security team saja.

Untuk senior backend engineer, pertanyaan minimum adalah:

Can I prove that the code I reviewed is the code running in production?
Can I prove that the image was built by the approved pipeline?
Can I prove that known risks were scanned, triaged, accepted, or fixed?
Can I prove that Kubernetes would reject unauthorized artifacts?

Kalau tidak bisa, sistem mungkin tetap berjalan, tetapi tidak defensible.

23. Practical Summary

Hal paling penting:

• Jangan gunakan latest untuk production.
• Jangan rebuild image berbeda per environment.
• Jangan memasukkan secret ke image layer.
• Jangan membuka Docker socket tanpa threat model.
• Jangan menganggap scan report otomatis berarti aman.
• Jangan menganggap SBOM otomatis berarti secure.
• Jangan mengandalkan policy tanpa enforcement.
• Jangan mengandalkan signing tanpa admission verification.
• Pastikan artifact bisa ditelusuri dari commit sampai pod.
• Pastikan exception security punya owner, expiry, dan mitigation.

Supply-chain security yang baik membuat deployment lebih lambat sedikit di awal, tetapi jauh lebih cepat saat audit, incident, CVE response, dan rollback.