Part 083 — Streaming Client Response and File Upload/Download Client

Fokus part ini: membangun outbound HTTP client untuk file dan stream tanpa buffering berbahaya, tanpa connection leak, dan tanpa membuat JVM/container jatuh karena memory pressure.

File transfer terlihat sederhana:

download file
upload file
send multipart
read stream
write stream

Tetapi di production, file transfer adalah salah satu integration pattern yang paling sering memunculkan failure mahal:

• heap spike
• native memory pressure
• connection leak
• socket timeout
• proxy timeout
• partial upload
• corrupt download
• duplicate upload
• checksum mismatch
• unbounded retry
• stuck thread
• temp file leak
• security bypass
• audit gap

Prinsip dasar senior engineer:

Never treat file transfer as ordinary JSON API call.

File and stream integration must be designed as resource lifecycle,
not as simple request/response object mapping.

1. Core Mental Model

Outbound file integration memiliki empat resource boundary:

JAX-RS service process
  -> HTTP client connection
  -> stream reader/writer
  -> storage/downstream system

Setiap boundary perlu kontrol:

connection lifecycle
stream lifecycle
buffer size
timeout
retry safety
checksum
content type
size limit
audit trail
cleanup

Untuk payload kecil, object mapping biasa masih masuk akal:

MyResponse response = client.target(url)
    .request(MediaType.APPLICATION_JSON_TYPE)
    .get(MyResponse.class);

Untuk file besar, pattern seperti ini berbahaya:

byte[] file = client.target(url)
    .request()
    .get(byte[].class);

Masalahnya:

The whole response may be buffered into heap.
Large file -> large heap pressure -> GC spike -> OOMKilled risk.

Pattern yang lebih aman:

HTTP response body stream
  -> bounded buffer copy
  -> file/storage/output stream
  -> checksum while copying
  -> close everything deterministically

2. Standard JAX-RS/Jakarta REST vs Jersey-Specific Concern

Pisahkan konsepnya.

Standard/JAX-RS concepts

Yang umum di Jakarta REST/JAX-RS:

• InputStream sebagai entity body
• StreamingOutput untuk response streaming server-side
• Response sebagai HTTP response wrapper
• Entity<?> untuk outbound request body
• MediaType
• MultivaluedMap
• MessageBodyReader
• MessageBodyWriter
• filters/interceptors
• Client, WebTarget, Invocation.Builder pada JAX-RS Client API

Jersey-specific concerns

Yang perlu dicek jika memakai Jersey:

• Jersey Client connector yang dipakai
• Apache connector vs default connector
• multipart module Jersey
• buffering behavior
• connector timeout properties
• chunked transfer support
• entity buffering setting
• logging feature yang mungkin membuffer body
• custom MessageBodyReader/Writer
• MultiPartFeature

Internal verification checklist

Jangan asumsikan implementasi internal.

Cek:

• Apakah outbound client memakai Jersey Client, Java HttpClient, Apache HttpClient, OkHttp, Retrofit, OpenFeign, atau wrapper internal?
• Apakah file upload memakai multipart?
• Apakah ada internal file-transfer library?
• Apakah response body pernah dibaca ke byte[] atau String?
• Apakah HTTP logging filter membuffer request/response body?
• Apakah connector mendukung streaming sejati?
• Apakah ada proxy/gateway yang membatasi body size atau idle timeout?

3. Download Lifecycle

Download file bukan satu operasi. Ia lifecycle.

1. Build request
2. Attach identity/signature/headers
3. Open connection
4. Receive response headers
5. Validate status code
6. Validate content-type/content-length
7. Open response stream
8. Copy stream with bounded buffer
9. Calculate checksum while copying
10. Persist or forward output
11. Close stream
12. Close response
13. Emit metrics/log/audit
14. Handle partial failure cleanup

Contoh struktur aman:

public DownloadResult downloadToFile(URI uri, Path target) {
    Path temp = target.resolveSibling(target.getFileName() + ".part");

    Response response = client.target(uri)
        .request()
        .get();

    try (response) {
        if (response.getStatus() != 200) {
            throw mapDownloadError(response);
        }

        String contentType = response.getHeaderString("Content-Type");
        long expectedLength = readContentLength(response);

        try (InputStream in = response.readEntity(InputStream.class);
             OutputStream out = Files.newOutputStream(temp,
                 StandardOpenOption.CREATE,
                 StandardOpenOption.TRUNCATE_EXISTING,
                 StandardOpenOption.WRITE)) {

            MessageDigest digest = MessageDigest.getInstance("SHA-256");
            long bytes = copyWithDigest(in, out, digest);

            verifyLength(expectedLength, bytes);
            String checksum = hex(digest.digest());

            Files.move(temp, target,
                StandardCopyOption.REPLACE_EXISTING,
                StandardCopyOption.ATOMIC_MOVE);

            return new DownloadResult(target, bytes, checksum, contentType);
        }
    } catch (Exception e) {
        safeDelete(temp);
        throw new FileTransferException("Download failed", e);
    }
}

Core invariant:

Do not expose a partially downloaded file as final output.

Gunakan temporary path lalu atomic move.

4. Upload Lifecycle

Upload juga lifecycle.

1. Validate local/source file exists
2. Validate size and content type
3. Calculate checksum if required
4. Open stream
5. Build request body
6. Attach idempotency key if upload creates/changes remote state
7. Attach auth/signature
8. Send stream
9. Validate response
10. Close stream
11. Close response
12. Emit audit/metrics
13. Handle retry safely

Anti-pattern:

byte[] bytes = Files.readAllBytes(path);
client.target(url).request().post(Entity.entity(bytes, mediaType));

Masalah:

Large file -> heap allocation
Concurrent uploads -> heap explosion
Retry -> repeated heap pressure
Logging -> accidental body exposure

Lebih aman:

try (InputStream in = Files.newInputStream(path)) {
    Entity<InputStream> entity = Entity.entity(in, mediaType);

    Response response = client.target(url)
        .request()
        .header("Idempotency-Key", idempotencyKey)
        .header("Content-Length", Files.size(path))
        .post(entity);

    try (response) {
        if (response.getStatusInfo().getFamily() != Response.Status.Family.SUCCESSFUL) {
            throw mapUploadError(response);
        }
    }
}

Namun perlu verifikasi: tidak semua client connector mengirim InputStream secara streaming dengan behavior yang sama.

5. Multipart Client Model

Multipart bukan sekadar “file + form field”. Ia format MIME dengan boundary dan setiap part punya header sendiri.

multipart/form-data
  part 1: metadata field
  part 2: file content
  part 3: optional checksum/signature

Gunakan multipart saat downstream contract memang meminta kombinasi metadata + file.

Jangan gunakan multipart hanya karena “kelihatannya upload”. Untuk service-to-service integration, kadang lebih baik:

PUT /objects/{id}
Content-Type: application/pdf
Digest: sha-256=...

<raw bytes>

atau:

POST /uploads
{ metadata }

PUT pre-signed-url
<raw bytes>

Multipart failure modes

• boundary corrupt
• wrong content disposition name
• wrong filename encoding
• server membuffer semua part
• client membuffer semua part
• file part tidak memiliki content type
• metadata dan file tidak konsisten
• retry duplicate creates multiple remote objects
• logging filter menulis body multipart ke log

Internal verification checklist

Cek:

• Multipart library yang dipakai.
• Apakah multipart upload streaming atau buffering?
• Apakah max part size dikontrol?
• Apakah metadata divalidasi sebelum file dikirim?
• Apakah file name dari user disanitasi?
• Apakah content type dipercaya atau diverifikasi?

6. Buffering: The Hidden Killer

Streaming bisa menjadi palsu jika ada layer yang membuffer.

Application thinks:
  stream -> stream -> stream

Runtime actually does:
  stream -> memory buffer -> log filter -> byte[] -> send

Layer yang sering membuffer:

• logging filter
• retry library
• metrics interceptor
• multipart builder
• HTTP connector
• proxy/gateway
• security scanning layer
• compression layer
• tracing body capture
• generated client abstraction

Senior review question:

At which exact layer is the payload held fully in memory?

Jika jawabannya tidak diketahui, design belum production-ready.

7. Content-Length vs Chunked Transfer

Jika ukuran file diketahui:

Content-Length: 104857600

Keuntungan:

• server bisa enforce limit lebih awal
• progress tracking lebih jelas
• checksum/length verification lebih mudah
• beberapa downstream menolak upload tanpa content length

Jika ukuran tidak diketahui:

Transfer-Encoding: chunked

Risiko:

• tidak semua server/proxy menerima chunked upload
• gateway bisa buffer chunked body
• retry lebih sulit
• error sering muncul setelah sebagian body terkirim

Internal verification:

• Apakah gateway menerima chunked upload?
• Apakah downstream butuh Content-Length?
• Apakah connector client mengirim chunked secara default?
• Apakah service mesh/proxy membuffer chunked request?

8. Timeout Model for File Transfer

Timeout file transfer tidak cukup satu angka.

connect timeout
connection request timeout
TLS handshake timeout
write timeout
read timeout
idle timeout
overall deadline
proxy timeout
downstream processing timeout

Untuk file besar, read timeout harus dipahami sebagai:

maximum idle time between bytes/chunks

bukan:

maximum total download duration

Jika tidak ada overall deadline, stream bisa berjalan terlalu lama.

Pattern:

small JSON call:
  short read timeout
  short total deadline

large file download:
  connect timeout tetap pendek
  idle timeout masuk akal
  total deadline berdasarkan max size dan expected throughput

Failure mode:

read timeout terlalu pendek -> large file selalu gagal
read timeout terlalu panjang -> stuck connection lama terdeteksi
total deadline tidak ada -> hung transfer makan resource

9. Retry Safety for Upload and Download

Download biasanya lebih aman untuk retry, tetapi tidak selalu.

Aman jika:

• GET benar-benar safe
• file immutable
• range/resume supported
• checksum diverifikasi
• partial file tidak dipublish

Tidak aman jika:

• download menghasilkan audit side effect
• link single-use
• file generated on demand dan mahal
• downstream punya rate limit ketat

Upload lebih berisiko.

Retry upload hanya aman jika:

• idempotency key didukung
• object key deterministic
• remote side dedupe by checksum/key
• partial upload bisa dibatalkan/dibersihkan
• server contract jelas untuk duplicate request

Jika upload menciptakan remote object random ID tanpa idempotency:

retry can create duplicate remote files

10. Resume and Range Request

Untuk file besar, resume bisa lebih baik daripada full retry.

HTTP range request:

GET /files/quote-batch-2026-07.csv HTTP/1.1
Range: bytes=1048576-

Response:

206 Partial Content
Content-Range: bytes 1048576-9999999/10000000

Gunakan resume jika:

• file besar
• network tidak stabil
• downstream support range
• file immutable
• ETag/Last-Modified bisa memvalidasi object yang sama

Jangan resume jika:

• file bisa berubah saat didownload
• ETag tidak tersedia
• checksum tidak bisa diverifikasi
• range response tidak reliable

Invariant:

Resume must prove that bytes belong to the same object version.

11. Checksum and Integrity Guard

Checksum bukan optional untuk transfer penting.

Gunakan untuk:

• mendeteksi corrupt transfer
• memastikan file yang disimpan benar
• dedupe
• audit
• reconciliation

Header umum:

Digest
Content-MD5 legacy-style usage
x-checksum-sha256 custom/internal
ETag if semantics are documented

Jangan selalu anggap ETag adalah checksum. Di beberapa object storage, ETag bisa bukan MD5 sederhana, terutama untuk multipart upload.

Pattern:

calculate checksum while streaming
compare with expected checksum
fail if mismatch
never mark transfer successful before integrity validation

12. Storage Handoff Pattern

Sering kali service JAX-RS tidak seharusnya menjadi pipe utama file besar.

Alternatif:

Client -> Service: request upload session
Service -> Object Storage: create pre-signed URL / upload token
Client -> Object Storage: upload directly
Object Storage -> Event/Callback: uploaded
Service -> DB: mark upload complete after validation

Keuntungan:

• mengurangi beban JVM
• mengurangi network hop
• mengurangi thread occupancy
• object storage lebih cocok untuk file besar

Trade-off:

• security token harus aman
• lifecycle upload lebih kompleks
• callback/reconciliation perlu jelas
• audit harus tetap lengkap

Untuk enterprise CPQ/order system, file bisa berupa:

• attachment quote
• contract document
• generated PDF
• product catalog import/export
• pricing sheet
• bulk order file
• integration payload archive

Jangan biarkan semua itu lewat endpoint JAX-RS biasa tanpa design file lifecycle.

13. Proxy, Gateway, and Platform Limits

File transfer sering gagal bukan karena Java code, tetapi karena platform limit.

Cek:

API gateway max body size
Ingress max body size
reverse proxy buffering
idle timeout
request timeout
response timeout
TLS termination behavior
service mesh timeout
WAF body inspection limit
Kubernetes memory limit
pod ephemeral storage

Failure symptoms:

413 Payload Too Large
408 Request Timeout
499 Client Closed Request
502 Bad Gateway
503 Service Unavailable
504 Gateway Timeout
connection reset by peer
unexpected EOF
broken pipe

Senior debugging principle:

For file transfer bugs, always draw the full network path before changing Java code.

14. Security Considerations

File transfer is security-sensitive.

Checklist:

• authenticate caller
• authorize access to file/resource
• validate tenant boundary
• validate content type
• do not trust filename
• enforce size limit
• scan if policy requires
• reject executable content if not allowed
• store outside web root
• avoid path traversal
• redact file content from logs
• audit access/download/upload
• define retention and deletion policy

Never do:

Path target = Paths.get(uploadDir, userProvidedFilename);

without normalization and boundary check.

Safe pattern:

generate internal object key
store original filename as metadata after sanitization
never use user filename as filesystem path authority

15. Observability for File Transfer

Minimum metrics:

file_transfer_attempt_total
file_transfer_success_total
file_transfer_failure_total
file_transfer_bytes_total
file_transfer_duration_seconds
file_transfer_active_count
file_transfer_checksum_mismatch_total
file_transfer_timeout_total
file_transfer_cancelled_total

Useful labels:

direction: upload|download
integration: downstream-name
operation: import-catalog|download-contract|upload-attachment
status_class: 2xx|4xx|5xx|timeout|cancelled

Avoid high-cardinality labels:

file_name
object_key
quote_id
order_id
customer_id
tenant_id if unbounded/high-cardinality

Logs should include:

correlation_id
operation
integration
file_size
checksum_algorithm
result
failure_category
elapsed_ms

Logs should not include:

file content
raw multipart body
PII filename if sensitive
secret upload URL
signed URL token

16. Common Failure Modes

Heap blow-up

Cause:

read file into byte[]
logging buffers body
multipart builder buffers all parts

Detection:

GC spike
heap dump
OOMKilled
large allocation in profiler

Fix:

stream with bounded buffer
remove body logging
change connector/provider

Connection leak

Cause:

Response not closed
InputStream not closed
exception path skips cleanup

Detection:

connection pool exhausted
threads waiting for connection
increasing open sockets

Fix:

try-with-resources
centralized transfer utility
connection pool metrics

Partial file accepted as success

Cause:

no checksum
no length verification
no temp-file atomic move

Detection:

consumer fails reading file
checksum mismatch downstream
reconciliation mismatch

Fix:

checksum + content length validation + atomic publish

Retry creates duplicate upload

Cause:

no idempotency key
random remote object ID
client retries after ambiguous timeout

Detection:

multiple attachment records
multiple remote objects
billing/storage anomaly

Fix:

idempotency key
stable object key
dedupe by checksum

Proxy timeout during large transfer

Cause:

proxy idle/response timeout shorter than transfer duration
buffering proxy waits for full body

Detection:

504
connection reset
server keeps processing after client gone

Fix:

align timeout across path
use direct object storage upload
chunk progress
async job model

17. Debugging Playbook

When file transfer fails:

1. Identify direction: upload or download.
2. Identify exact payload size.
3. Identify full network path.
4. Check status code and failure source.
5. Check whether response/request was buffered.
6. Check timeout at each layer.
7. Check whether stream/response is closed.
8. Check checksum/content-length.
9. Check retry behavior.
10. Check platform limits.

Questions to ask:

Did the request reach downstream?
Did downstream receive full body?
Did client close early?
Did gateway cut the connection?
Was partial output persisted?
Was retry attempted?
Was retry safe?
Was file content logged?

Useful evidence:

• application log
• access log
• gateway log
• downstream log
• object storage audit
• network proxy log
• metrics
• tracing spans
• pod memory graph
• connection pool metrics

18. Design Alternatives

Option A — Direct JAX-RS upload/download

Good for:

• small files
• simple internal use
• strong service-side authorization
• low volume

Risk:

• JVM resource pressure
• proxy timeout
• thread occupancy

Option B — Pre-signed object storage transfer

Good for:

• large files
• external clients
• high throughput
• object storage integration

Risk:

• more complex lifecycle
• token security
• callback/reconciliation required

Option C — Async job-based file generation

Good for:

• generated reports
• export jobs
• bulk data
• long-running transformations

Pattern:

POST /exports -> 202 Accepted + job id
GET /exports/{id} -> status
GET /exports/{id}/file -> download when ready

Risk:

• job orchestration
• retention cleanup
• status model

Option D — Event-driven file handoff

Good for:

• backend integration
• batch import/export
• reconciliation

Pattern:

file placed in storage
metadata event emitted
consumer validates and processes

Risk:

• eventual consistency
• duplicate event
• missing file/event mismatch

19. PR Review Checklist

Review outbound file/stream code with these questions:

Contract

• Is file size limit explicit?
• Is content type explicit?
• Is checksum required?
• Is idempotency defined?
• Is retry behavior safe?
• Is partial success represented?

Implementation

• Is payload streamed rather than fully buffered?
• Are Response, InputStream, and OutputStream closed?
• Is temp file used before final publish?
• Is checksum calculated while streaming?
• Is buffer size bounded?
• Is content length validated?

Resilience

• Are timeout values appropriate for file size?
• Is retry bounded?
• Is ambiguous timeout handled?
• Is cancellation handled?
• Are proxy limits known?

Security

• Is authorization enforced?
• Is tenant boundary enforced?
• Is filename sanitized?
• Is body excluded from logs?
• Is signed URL protected?
• Is malware/content scanning considered if required?

Observability

• Are bytes, duration, failure category, and downstream captured?
• Are high-cardinality labels avoided?
• Is checksum mismatch visible?
• Is partial failure visible?

20. Internal Verification Checklist

For CSG Quote & Order or similar enterprise system, verify:

Client implementation

• Which HTTP client is used for file transfer?
• Is it Jersey Client, Retrofit, OpenFeign, Java HttpClient, Apache HttpClient, OkHttp, or wrapper internal?
• Does the selected client stream request and response bodies?
• Are connector-specific buffering behaviors documented?

File handling

• Are attachments, generated documents, catalog imports, pricing files, or bulk order files handled by JAX-RS services?
• Are file sizes bounded?
• Are temporary files used?
• Are checksums stored?
• Is there a canonical object key strategy?

Platform

• What are gateway/ingress max body size limits?
• What are proxy idle timeout and response timeout?
• Is chunked upload allowed?
• Is compression enabled for binary files?
• Are large files routed directly to object storage?

Security

• Are file operations tenant-aware?
• Are signed URLs used?
• Are file names sanitized?
• Is PII in filenames/logs controlled?
• Is scanning required?
• What is retention policy?

Operations

• Are file transfer metrics available?
• Are failures categorized?
• Is there a runbook for stuck/partial/corrupt transfer?
• Is there reconciliation between DB metadata and object storage?

21. Senior Engineer Heuristics

Use these rules:

If file can exceed a few MB, assume streaming is required.
If file can be retried, assume idempotency is required.
If file affects business state, assume audit is required.
If file crosses tenant boundary, assume strict authorization is required.
If file persists anywhere, assume retention and cleanup are required.
If file goes through a gateway, assume platform limit must be verified.

A senior engineer does not ask only:

Does the upload work?

They ask:

Does it work under concurrent load?
Does it avoid buffering?
Does it survive retry?
Does it detect corruption?
Does it clean up partial output?
Does it respect tenant/security boundaries?
Can operators debug it at 03:00?

22. Summary

Outbound file and stream integration is a resource-lifecycle problem.

Key takeaways:

• Do not treat file transfer like JSON API calls.
• Avoid byte[] for large payloads.
• Stream with bounded buffers.
• Close Response and streams deterministically.
• Verify checksum and content length.
• Use temp files and atomic publish.
• Define idempotency before retrying uploads.
• Align timeout across client, gateway, proxy, and downstream.
• Avoid body logging.
• Prefer object storage handoff for large files.
• Make transfer observable, auditable, and recoverable.

Production-grade file transfer is not about moving bytes.

It is about preserving correctness while bytes move through unreliable systems.