Azure/AKS Traffic Flow: DNS and Edge to NGINX and Pods
Azure and AKS Traffic Flow with NGINX
Menelusuri Azure DNS, Front Door, Application Gateway, Load Balancer, APIM, VNet controls, Services, and ingress in AKS.
Part 023 — Azure/AKS Traffic Flow: DNS and Edge to NGINX and Pods
Depth level: Production/Architecture-level
Prerequisite: Part 002, 007–009, 015, dan 018–021; dasar Azure DNS, VNet, subnet, NSG, UDR, Azure Load Balancer, Application Gateway, Front Door, API Management, AKS networking, Kubernetes Service/Ingress/Gateway, dan NGINX reverse proxy.
Scope: Azure DNS, Front Door, Application Gateway, AGIC, Application Gateway for Containers, Azure Load Balancer, API Management, public/private IP, VNet/subnet/NSG/UDR, Private Link/Private Endpoint, AKS pod networking, Kubernetes Service, NGINX ingress, source IP, forwarded headers, TLS, health probes, observability, failure attribution, Java/JAX-RS impact, debugging, PR review, dan internal verification.
Bukan scope utama: generic on-prem/hybrid—Part 024; DNS resolver internals—Part 025; WebSocket/SSE detail—Part 026; GitOps—Part 029; full incident runbook—Part 032.
Current Azure/AKS note — 11 July 2026
- Kubernetes community
ingress-nginxtelah retired pada 24 March 2026. Microsoft menyatakan AKS application routing add-on with NGINX tetap memperoleh official support untuk critical security patches dan production workloads sampai November 2026. Ini adalah temporary support horizon, bukan alasan untuk menunda migration planning. - Microsoft mengarahkan pengguna managed NGINX application-routing untuk bermigrasi ke application routing Gateway API implementation atau implementation lain yang didukung sebelum support window berakhir.
- Application routing Gateway API pada AKS saat ini menggunakan Istio control plane untuk mengelola Gateway API infrastructure, tetapi bukan full Istio service mesh. Ia tidak dapat diaktifkan bersamaan dengan AKS Istio service mesh add-on.
- AGIC memprogram Azure Application Gateway dari Kubernetes resources dan dapat mengirim traffic langsung ke private pod IP, sehingga datapath-nya berbeda dari Azure Load Balancer → NGINX Service.
- Application Gateway for Containers adalah offering terpisah yang dibangun khusus untuk Kubernetes. ALB Controller di cluster menerjemahkan Ingress/Gateway API ke Azure-managed data plane di luar cluster.
kubenetuntuk AKS dijadwalkan retired pada 31 March 2028. Azure CNI Overlay atau Azure CNI Pod Subnet harus dipertimbangkan sesuai kebutuhan address space, routability, network policy, dan integration.
Primary rule: Jangan menyebut “request masuk lewat Azure ke AKS” sebagai satu hop. Pecah menjadi naming, global edge, regional L7/L4 delivery, VNet routing, Kubernetes datapath, NGINX routing, Service/EndpointSlice, dan Java/JAX-RS processing.
Daftar isi
- Tujuan pembelajaran
- Executive mental model
- Traffic path sebagai chain of contracts
- Azure control plane versus data plane
- Reference topologies
- Topology A: Azure Load Balancer ke NGINX ingress
- Topology B: Application Gateway dan AGIC langsung ke pod
- Topology C: Front Door ke Application Gateway ke NGINX
- Topology D: Front Door Premium melalui Private Link
- Topology E: API Management ke private AKS ingress
- Topology F: Application Gateway for Containers
- Azure DNS mental model
- Public dan private DNS zones
- Split-horizon DNS
- Private DNS zone links dan custom DNS
- Azure DNS Private Resolver
- DNS failure modes
- Azure Front Door
- Front Door route, origin group, dan origin
- Front Door health probes
- Front Door WAF dan edge security
- Front Door forwarded headers
- Azure Application Gateway
- Listeners, rules, backend pools, dan HTTP settings
- Application Gateway health probes
- AGIC reconciliation model
- AGIC direct-to-pod datapath
- AGIC shared-gateway and ownership risk
- Application Gateway for Containers
- ALB Controller dan Kubernetes resources
- Azure Load Balancer
- Public versus internal LoadBalancer Service
externalTrafficPolicy- Azure health probe behavior
- API Management
- APIM as API product boundary
- APIM private endpoint dan VNet integration
- Avoiding duplicate gateway behavior
- VNet and subnet model
- Network Security Groups
- User Defined Routes
- Azure Firewall dan forced tunneling
- Private Link dan Private Endpoint
- AKS network models
- Azure CNI Overlay
- Azure CNI Pod Subnet
- Legacy kubenet considerations
- Pod IP routability
- Service datapath
- NGINX controller Service
- Managed application routing NGINX
- Multiple ingress controllers
- Source IP mental model
- Forwarded header trust chain
- TLS placement patterns
- Key Vault dan certificate lifecycle
- End-to-end TLS dan mTLS
- Health is layered
- Timeout and connection alignment
- Connection draining and rollout
- Observability evidence map
- Azure Monitor and diagnostic settings
- Network Watcher dan flow evidence
- End-to-end latency decomposition
- Common failure modes
- Debugging DNS
- Debugging Front Door
- Debugging Application Gateway and AGIC
- Debugging Azure Load Balancer
- Debugging NSG, UDR, dan firewall
- Debugging Private Link and private DNS
- Debugging NGINX and Kubernetes routing
- Debugging TLS and source identity
- Java/JAX-RS implications
- Security concerns
- Performance and cost concerns
- Safe rollout checklist
- PR review checklist
- Internal verification checklist
- Ringkasan mental model
- Referensi resmi
Tujuan pembelajaran
Setelah menyelesaikan part ini, Anda harus mampu:
- Menggambar request path Azure/AKS sampai Java/JAX-RS endpoint beserta owner setiap hop.
- Membedakan Front Door, Application Gateway, AGIC, Application Gateway for Containers, Azure Load Balancer, APIM, dan NGINX berdasarkan problem yang diselesaikan.
- Menentukan kapan traffic menuju node, Service, NGINX pod, atau langsung ke application pod.
- Menjelaskan source-IP dan forwarded-header contract pada topology yang dipilih.
- Menempatkan TLS termination, re-encryption, passthrough, dan mTLS secara sadar.
- Membedakan Azure health probe, Kubernetes readiness, NGINX health, dan application dependency health.
- Men-debug DNS, routing, NSG, UDR, firewall, health probe, private endpoint, controller reconciliation, Service, EndpointSlice, NGINX, dan Java service.
- Menghubungkan Azure Monitor evidence dengan Kubernetes, NGINX, dan application telemetry.
- Menilai migration risk managed NGINX menuju Gateway API atau platform ingress lain.
- Mereview architecture/PR yang menyentuh Azure ingress dan AKS networking.
Executive mental model
Azure inbound delivery memiliki beberapa layer yang dapat berdiri sendiri atau digabung:
Global edge:
Azure Front Door
Regional L7 edge:
Application Gateway / Application Gateway for Containers / APIM
Regional L4 delivery:
Azure Load Balancer
Cluster edge:
NGINX ingress / Gateway proxy / service mesh ingress
Application routing:
Kubernetes Service -> EndpointSlice -> Java/JAX-RS pod
Reference flow:
Diagram tersebut adalah superset. Architecture aktual mungkin:
Front Door -> Application Gateway -> pod
atau:
Azure Load Balancer -> NGINX -> Service -> pod
atau:
Application Gateway for Containers -> pod
Core invariant
Setiap proxy atau load balancer menambah protocol boundary, timeout boundary, identity boundary, health boundary, observability boundary, dan potential failure mode.
Karena itu, menambah layer harus dibenarkan oleh capability yang benar-benar diperlukan.
Traffic path sebagai chain of contracts
Gunakan tabel berikut pada architecture review:
| Hop | Contract yang harus eksplisit |
|---|---|
| DNS | hostname, zone scope, record, TTL, resolver path |
| Front Door | domain, route, protocol, WAF, origin, health probe |
| Application Gateway | frontend IP, listener, certificate, rule, backend pool, probe |
| APIM | public/private endpoint, API/base path, policy, backend URL |
| Azure Load Balancer | frontend IP, rule, backend pool, probe, source NAT behavior |
| VNet | address space, subnet, peering, route table |
| NSG/firewall | source, destination, port, protocol, direction |
| Kubernetes | Service type, selector, EndpointSlice, traffic policy |
| NGINX | host/path, TLS, headers, timeout, buffering, rate/auth policy |
| Java/JAX-RS | endpoint, context path, forwarded-header trust, readiness, downstream deadline |
Request success requires every contract to align.
Azure control plane versus data plane
Control plane
Control plane creates or updates desired configuration:
- Azure Resource Manager;
- Azure controllers/operators;
- AGIC;
- Application Gateway for Containers ALB Controller;
- AKS cloud provider/controller manager;
- application routing add-on controller;
- GitOps/CI/CD;
- Kubernetes API server.
Data plane
Data plane processes real traffic:
- Front Door edge;
- Application Gateway instances;
- Application Gateway for Containers managed data plane;
- Azure Load Balancer;
- NGINX workers;
- kube-proxy/eBPF datapath;
- Java/JAX-RS pod.
Failure distinction
Control-plane failure:
new route/certificate/backend not reconciled.
Existing traffic may continue using old state.
Data-plane failure:
real request cannot connect, authenticate, route, or complete.
Do not infer dataplane health merely from successful deployment command.
Reference topologies
| Topology | Primary use | Main trade-off |
|---|---|---|
| Azure LB -> NGINX | Kubernetes-native shared ingress | extra in-cluster proxy hop |
| App Gateway + AGIC -> pods | Azure-native regional L7/WAF | controller ownership and ARM reconciliation |
| Front Door -> App Gateway -> pods | global edge + regional WAF/routing | duplicate L7 policy risk |
| Front Door Premium -> private origin | global delivery without public origin | Private Link/DNS complexity |
| APIM -> private ingress | API product/security policies | latency, cost, duplicate auth/retry |
| App Gateway for Containers -> pods | managed Kubernetes-oriented L7 | feature/region/version compatibility |
| Front Door -> APIM -> NGINX | global API platform | highest layering and governance cost |
Topology A: Azure Load Balancer ke NGINX ingress
Typical implementation:
apiVersion: v1
kind: Service
metadata:
name: nginx-ingress-controller
namespace: ingress-system
annotations:
service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path: /healthz
spec:
type: LoadBalancer
externalTrafficPolicy: Local
selector:
app.kubernetes.io/name: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
Path variants
externalTrafficPolicy: Cluster:
Azure LB -> any healthy node -> cross-node forwarding possible -> NGINX pod
externalTrafficPolicy: Local:
Azure LB -> node with local NGINX endpoint -> NGINX pod
Advantages
- familiar Kubernetes ingress model;
- centralized NGINX policy;
- cloud-portable configuration;
- control over NGINX modules/logging/tuning depending distribution.
Risks
- Azure LB health probe and NGINX readiness must align;
- source IP may be changed/lost depending datapath;
- extra hop and in-cluster capacity requirement;
- managed NGINX support horizon must be considered;
- controller annotations are not portable across distributions.
Topology B: Application Gateway dan AGIC langsung ke pod
AGIC watches Kubernetes resources and updates Application Gateway configuration.
Important datapath property
Application Gateway can route directly to private pod IPs. In that topology:
- no Azure Load Balancer is required for the ingress path;
- NodePort is not in the request path;
- kube-proxy may not be in the request path;
- Kubernetes Service remains an intent/backend discovery object, but traffic can target pod IPs directly.
Implication
Troubleshooting must inspect:
- Ingress resource;
- AGIC logs/events;
- generated App Gateway listeners/rules/settings;
- backend pool addresses;
- VNet route/NSG reachability to pod CIDR;
- Application Gateway probe;
- pod readiness and listener.
Risk
A healthy Kubernetes Service does not guarantee Application Gateway backend pool is current. Conversely, a current backend pool does not guarantee application-level routes are correct.
Topology C: Front Door ke Application Gateway ke NGINX
This topology can be valid when:
- global anycast routing/CDN is required;
- regional WAF/private routing is required;
- NGINX provides Kubernetes-specific routing or app-specific features.
It is usually excessive when all three layers implement:
- host/path routing;
- WAF;
- redirects;
- TLS policies;
- retries;
- rate limits;
- authentication;
- header rewriting;
- access logging.
Required ownership table
| Concern | Sole owner |
|---|---|
| global failover | Front Door |
| regional private ingress | Application Gateway |
| Kubernetes service routing | NGINX |
| domain authorization | Java/JAX-RS |
| retry policy | one chosen layer only |
| external request ID | first trusted edge |
| client IP normalization | one documented trust boundary |
Topology D: Front Door Premium melalui Private Link
Front Door Premium can connect to supported origins through Private Link.
Security objective
Origin should reject direct public bypass. Otherwise, WAF and edge policy can be bypassed.
Verify
- approved private endpoint connection;
- Front Door origin state;
- private DNS mapping;
- origin host header/SNI;
- NSG/firewall path;
- certificate name expected by Front Door;
- health probe path and host.
Failure pattern
Public URL returns 502/503 from Front Door,
but direct origin test works.
Possible causes:
- Private Link connection pending/rejected;
- incorrect origin host header;
- certificate name mismatch;
- probe path requires auth;
- private DNS resolves differently;
- origin allows only wrong source/path.
Topology E: API Management ke private AKS ingress
APIM is justified when you need API product capabilities such as:
- subscription/product management;
- API lifecycle/version exposure;
- policy-based transformation;
- developer portal;
- API consumer analytics;
- centralized external authentication/quota;
- controlled publication of internal APIs.
Avoid duplicate behavior
Do not independently configure APIM, NGINX, and Java to all retry the same POST.
Do not validate JWT in three layers unless each validation has a distinct trust purpose.
Do not let APIM rewrite base paths without documenting how JAX-RS reconstructs public URIs.
Topology F: Application Gateway for Containers
Application Gateway for Containers uses an Azure-managed data plane outside the cluster and a Kubernetes controller inside the cluster.
Architectural distinction
AGIC:
Kubernetes controller programs traditional Application Gateway.
Application Gateway for Containers:
Kubernetes-oriented offering with separate Azure-managed control/data plane.
Review items
- supported region;
- controller/add-on version;
- Ingress/Gateway API support matrix;
- WAF and TLS requirements;
- frontend and association lifecycle;
- subnet delegation;
- managed identity permissions;
- backend health behavior;
- migration plan from AGIC/NGINX;
- rollback path.
Azure DNS mental model
Azure DNS can participate in different scopes:
Public Azure DNS zone:
Internet resolvers -> public endpoint
Private DNS zone:
Linked VNets/custom resolver -> private endpoint/IP
Enterprise DNS:
On-prem resolver -> conditional forwarder -> Azure DNS Private Resolver
DNS is not routing
DNS chooses an address. It does not prove:
- listener exists;
- certificate matches;
- NSG allows traffic;
- route table reaches target;
- backend is healthy.
Public dan private DNS zones
Public zone
Used for internet-visible names:
api.example.com -> Front Door endpoint
Private zone
Used for VNet-only names:
api.internal.example.com -> internal Application Gateway/IP
Internal verification
- authoritative name servers;
- registrar delegation;
- record type;
- alias/CNAME target;
- TTL;
- private zone VNet links;
- auto-registration setting;
- duplicate/conflicting zones;
- DNSSEC requirements if applicable.
Split-horizon DNS
Same hostname can resolve differently:
Public resolver:
api.example.com -> Front Door public endpoint
Corporate resolver:
api.example.com -> internal Application Gateway
Benefit
- private path for employees/services;
- controlled exposure;
- reduced public dependency.
Risks
- certificate must cover both paths;
- public and private route behavior can drift;
- incident reproductions differ by network;
- client DNS cache masks changes;
- VPN users may use unexpected resolver.
Debugging rule
Always record:
client location + resolver IP + answer + TTL + time
Private DNS zone links dan custom DNS
A private zone only resolves from networks linked to it or through an appropriate resolver chain.
Custom DNS servers in a VNet can break Azure private-name resolution unless forwarding is configured correctly.
Common mistake:
Private Endpoint created successfully,
but custom DNS still resolves service hostname to public address.
Evidence:
nslookup api.internal.example.com
nslookup api.internal.example.com <corporate-dns-ip>
dig +trace api.example.com
For private zones, +trace from a public context may not reveal the internal resolution chain. Query the actual resolver used by the workload.
Azure DNS Private Resolver
Azure DNS Private Resolver can provide inbound/outbound endpoints and forwarding rulesets for hybrid name resolution.
Use cases:
- on-prem clients resolve Azure private zones;
- Azure workloads resolve on-prem zones;
- centralized conditional forwarding without DNS VMs.
Failure modes
- forwarding rule matches wrong suffix;
- VNet link missing;
- NSG blocks DNS UDP/TCP 53;
- on-prem route cannot reach inbound endpoint;
- recursive forwarding loop;
- overlapping private zones;
- stale cached negative response.
DNS failure modes
| Symptom | Likely area |
|---|---|
| NXDOMAIN | record/zone/suffix/forwarder |
| timeout | resolver reachability/NSG/firewall |
| public IP returned internally | private zone/forwarding not applied |
| old IP returned | cache/TTL/stale corporate DNS |
| intermittent answer | multiple resolvers or inconsistent zones |
| TLS mismatch after DNS change | hostname routed to wrong edge |
Do not fix DNS by hardcoding IP
Hardcoding can temporarily mask the issue but creates:
- certificate/SNI mismatch;
- failover loss;
- hidden stale dependencies;
- unsafe production drift.
Azure Front Door
Azure Front Door is a global HTTP/HTTPS application delivery service.
Use it for:
- global anycast entry;
- multi-region routing/failover;
- edge WAF;
- caching/content acceleration;
- custom domains and edge TLS;
- origin health evaluation.
It is not a Kubernetes ingress controller.
Front Door is another reverse proxy hop
Therefore it has its own:
- route matching;
- timeout limits;
- header behavior;
- health probes;
- certificate lifecycle;
- WAF rules;
- logs/metrics;
- origin connection behavior.
Front Door route, origin group, dan origin
Conceptual mapping:
Endpoint/custom domain
-> Route
-> Origin group
-> Origin(s)
Verify:
- accepted protocol HTTP/HTTPS;
- forwarding protocol;
- patterns to match;
- origin path;
- origin host header;
- priority/weight;
- session affinity if used;
- private link state;
- health probe method/path/protocol.
Origin host header
This header influences:
- Application Gateway listener selection;
- NGINX
server_nameselection; - upstream certificate SNI/name validation;
- Java-generated absolute URLs.
Changing it is an architecture decision, not a cosmetic setting.
Front Door health probes
Health probes must answer the question:
Can this origin serve the class of traffic that Front Door may send now?
Poor probe:
/ always returns 200 from a static default page
Better probe:
/ready/edge validates router and critical application readiness,
but remains cheap, unauthenticated only from trusted probe path,
and does not execute expensive downstream checks every second.
Single-origin nuance
When only one origin exists, probe status may not provide failover value. Still verify exact platform behavior and cost before disabling it.
Front Door WAF dan edge security
Front Door WAF can reject attacks before regional infrastructure.
Possible policy placement:
| Control | Suggested owner |
|---|---|
| volumetric/edge WAF | Front Door |
| regional private exposure | Application Gateway/NSG |
| route-specific request limits | NGINX/APIM |
| domain authorization | Java service |
Avoid policy drift
If Front Door and Application Gateway both run WAF:
- document rule ownership;
- synchronize exclusions intentionally;
- test false positives at both layers;
- preserve a correlation ID through both logs;
- know which layer returned the block response.
Front Door forwarded headers
Treat all client-supplied forwarding headers as untrusted at the first trusted edge.
Expected model:
Client-supplied X-Forwarded-* removed or normalized
Front Door adds trusted forwarding metadata
Downstream trusts only known Front Door/Application Gateway sources
At NGINX:
set_real_ip_from <trusted-proxy-cidr>;
real_ip_header X-Forwarded-For;
real_ip_recursive on;
This is only safe when the source ranges and full proxy chain are understood.
Azure Application Gateway
Application Gateway is a regional L7 reverse proxy/load balancer with capabilities including:
- HTTP/HTTPS listeners;
- host/path routing;
- TLS termination;
- WAF;
- backend health probes;
- redirects and rewrites;
- end-to-end TLS;
- autoscaling depending SKU/configuration.
It is VNet-integrated and normally requires a dedicated subnet.
Listeners, rules, backend pools, dan HTTP settings
Traffic path:
frontend IP
-> listener
-> routing rule
-> backend pool
+ backend HTTP setting
+ health probe
Listener
Defines:
- frontend IP;
- port;
- protocol;
- hostname;
- certificate.
Backend pool
Defines target addresses:
- VM/IP/FQDN;
- pod IPs through AGIC;
- private endpoint/origin depending architecture.
Backend HTTP setting
Defines:
- backend protocol/port;
- timeout;
- host name behavior;
- cookie affinity;
- trusted root certificates;
- probe association.
Common mismatch
Listener HTTPS works,
but backend setting uses HTTP to a pod requiring HTTPS.
Result can be unhealthy backend or 502.
Application Gateway health probes
Probe dimensions:
- protocol;
- host header;
- path;
- interval;
- timeout;
- unhealthy threshold;
- accepted status range;
- backend TLS validation.
Probe contract
A probe should not:
- require end-user token;
- redirect indefinitely;
- depend on Host header not supplied by probe;
- return 200 from unrelated default backend;
- perform expensive deep diagnostics at high frequency.
Evidence
Use backend health view plus diagnostic logs. Do not infer health from listener state alone.
AGIC reconciliation model
Reconciliation lag
A Kubernetes object can exist before Application Gateway configuration is active.
Monitor:
- AGIC pod health;
- leader state if applicable;
- Azure identity permission;
- ARM throttling/errors;
- rejected annotations;
- backend pool updates;
- Ingress events.
AGIC direct-to-pod datapath
AGIC can configure backend pools with pod private IPs.
Consequences
- pod IP must be reachable from Application Gateway subnet;
- route tables must know pod CIDR;
- NSG/firewall must allow backend port;
- pod IP churn must be reconciled quickly;
- health probe reaches pod directly;
- NGINX is absent unless intentionally added.
Networking-model dependency
Direct pod reachability differs across:
- Azure CNI Overlay;
- Azure CNI Pod Subnet;
- legacy Azure CNI node subnet;
- kubenet;
- custom route/firewall topology.
Never copy a topology diagram without validating the actual AKS network profile.
AGIC shared-gateway and ownership risk
A shared Application Gateway may contain:
- AGIC-managed listeners/rules;
- manually managed listeners/rules;
- non-AKS backends.
Potential risks:
- controller overwrites manual configuration;
- ownership ambiguity;
- conflicting listeners/ports/hostnames;
- shared WAF exclusions;
- blast radius during reconciliation;
- application teams affect unrelated workloads.
Governance requirement
Document:
which resource scopes AGIC owns,
which configuration may be manually changed,
and how drift is detected.
Application Gateway for Containers
Key mental model:
Kubernetes resources are desired state.
ALB Controller is control plane bridge.
Azure-managed Application Gateway for Containers is data plane.
It can support Ingress and Gateway API, but exact features and resource versions are implementation-specific.
Why it matters for NGINX engineers
Migration may remove NGINX from the datapath. Re-evaluate:
- NGINX-specific annotations;
- snippets/modules;
- logging fields;
- timeout defaults;
- rewrite semantics;
- canary behavior;
- rate/auth controls;
- source IP/header contract;
- operational runbooks.
Do not assume semantic equivalence.
ALB Controller dan Kubernetes resources
Controller may watch resources such as:
Ingress;GatewayClass;Gateway;HTTPRoute;GRPCRoute;- Azure-specific custom resources.
Verify status, not only spec
kubectl get gatewayclass
kubectl get gateway -A
kubectl get httproute -A
kubectl describe gateway <name> -n <namespace>
kubectl describe httproute <name> -n <namespace>
Inspect conditions such as:
- Accepted;
- Programmed;
- ResolvedRefs;
- controller-specific errors.
Azure Load Balancer
Azure Load Balancer is L4.
It does not understand:
- HTTP host;
- path;
- JWT;
- cookies;
- CORS;
- application response body.
It operates on:
- frontend IP/port;
- load-balancing rule;
- backend pool;
- health probe;
- transport flow.
AKS usage
A Kubernetes Service of type LoadBalancer causes AKS cloud integration to create/update Azure LB configuration.
Do not manually modify AKS-managed LB resources unless the platform explicitly supports that operation.
Public versus internal LoadBalancer Service
Public example:
apiVersion: v1
kind: Service
metadata:
name: nginx-public
spec:
type: LoadBalancer
selector:
app: nginx-ingress
ports:
- port: 443
targetPort: 443
Internal example:
apiVersion: v1
kind: Service
metadata:
name: nginx-internal
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
type: LoadBalancer
selector:
app: nginx-ingress
ports:
- port: 443
targetPort: 443
Exact annotations and supported fields depend on AKS version and cloud-provider implementation.
Static IP
Prefer supported annotations/resource references over deprecated spec.loadBalancerIP patterns when possible. Validate current AKS guidance.
externalTrafficPolicy
Cluster
LB may send to any node.
Node may forward to endpoint on another node.
Pros:
- broad node availability;
- even distribution at node level.
Cons:
- possible source NAT;
- extra cross-node hop;
- health probe semantics can differ.
Local
LB should send only to nodes with a local endpoint.
Pros:
- source IP preservation is often easier;
- no cross-node forwarding for ingress endpoint.
Cons:
- uneven endpoint distribution can cause hotspot;
- nodes without local endpoint are excluded;
- rollout/scale-down requires careful draining.
Invariant
Local does not automatically guarantee correct client IP at Java. Every upstream proxy can still add or rewrite identity headers.
Azure health probe behavior
Azure Load Balancer probe behavior depends on:
- protocol;
externalTrafficPolicy;- Service annotations;
- AKS version/features;
- ingress controller defaults.
For NGINX ingress, explicitly verify the expected health endpoint, commonly /healthz, and the associated Service annotation.
Failure example
NGINX pods are Ready,
but Azure LB probe hits wrong path and marks all nodes unhealthy.
Symptoms:
- public IP exists;
- Service shows external IP;
- no inbound requests reach NGINX;
- LB probe failures visible in Azure evidence.
API Management
APIM is not merely a load balancer. It is an API management layer.
Common capabilities:
- API products/subscriptions;
- policy engine;
- JWT/certificate validation;
- quotas/rate limits;
- transformations;
- revisions/versions;
- developer onboarding;
- analytics.
Architecture question
Ask:
Is this API being exposed as a governed product, or only routed to a backend?
If only routing is needed, APIM may be unnecessary.
APIM as API product boundary
A reasonable responsibility model:
| Layer | Responsibility |
|---|---|
| APIM | consumer identity, subscription, coarse quota, API publication |
| NGINX | cluster ingress routing, transport controls |
| Java/JAX-RS | domain authorization, lifecycle invariants, transaction semantics |
Anti-pattern
APIM policy converts every backend error to 200 with custom JSON.
This destroys HTTP semantics, alerting, retry behavior, and client compatibility.
APIM private endpoint dan VNet integration
Separate inbound and outbound connectivity:
Inbound private endpoint:
consumer -> private APIM gateway endpoint
Outbound VNet integration/injection:
APIM -> private backend
They are different directions and capabilities.
DNS requirement
Private endpoint requires hostname resolution to private IP for intended clients. Incorrect private DNS is a common cause of accidental public routing or connection failure.
Avoiding duplicate gateway behavior
Create a policy ownership matrix:
| Concern | Front Door | App Gateway | APIM | NGINX | App |
|---|---|---|---|---|---|
| global failover | owner | ||||
| WAF | owner/optional | optional | optional | ||
| API subscription | owner | ||||
| Kubernetes route | owner | ||||
| domain authorization | owner | ||||
| retry | one owner only | ||||
| public URL reconstruction | support | owner |
Blank cells are intentional. More checks are not automatically more secure.
VNet and subnet model
A VNet provides Azure private address spaces and routing domains.
Relevant subnets may include:
- AKS node subnet;
- pod subnet;
- Application Gateway subnet;
- Azure Firewall subnet;
- private endpoint subnet;
- integration subnet;
- management/bastion subnet.
Subnet design concerns
- address-space exhaustion;
- delegation requirements;
- NSG association;
- UDR association;
- service endpoints/private endpoints;
- peering reachability;
- route propagation;
- availability-zone design.
Warning
Application Gateway and Application Gateway for Containers can have specific subnet/delegation requirements. Verify product documentation before reusing a subnet.
Network Security Groups
NSGs are stateful packet filters applied to subnet/NIC scope.
Evaluate both directions:
source -> destination -> protocol -> port -> priority -> action
Common mistake
Only ingress rule is checked, while an outbound deny or UDR prevents response/downstream traffic.
Evidence model
Record:
- effective NSG rules;
- NIC/subnet associations;
- service tags used;
- priority conflict;
- source IP after NAT/proxy;
- destination port actually used;
- timestamp.
User Defined Routes
UDRs override system routing for matching prefixes.
Typical use:
0.0.0.0/0 -> Azure Firewall/NVA
Risks
- asymmetric routing;
- next-hop cannot route pod CIDR;
- return path differs from request path;
- firewall SNAT changes source identity;
- health probes follow unexpected path;
- AKS control-plane dependencies become unreachable;
- private endpoint traffic is forced incorrectly.
Debugging
Do not only inspect the route table object. Inspect effective routes on the relevant NIC/subnet.
Azure Firewall dan forced tunneling
When AKS egress is forced through Azure Firewall/NVA:
- required FQDNs/service tags must be allowed;
- SNAT port capacity matters;
- TLS inspection can break certificate pinning or mTLS;
- DNS path must align with firewall policy;
- return routes must remain symmetric;
NO_PROXYmust include cluster/internal destinations where appropriate.
Although this part focuses inbound traffic, egress failure can make readiness fail and indirectly remove all ingress backends.
Private Link dan Private Endpoint
Private Endpoint
A private IP in your VNet representing a supported service endpoint.
Private Link
Underlying private connectivity service.
Common misconception
Private Endpoint created = private connectivity complete
Actually you still need:
- connection approval;
- DNS mapping;
- NSG/firewall reachability;
- route path;
- public access policy;
- correct hostname/SNI;
- consumer-side resolver.
AKS network models
Network model determines:
- pod IP allocation;
- pod IP routability;
- VNet address consumption;
- route programming;
- network policy support;
- integration with App Gateway/AGIC;
- observability and troubleshooting.
Never use “Azure CNI” as a single undifferentiated term.
Azure CNI Overlay
Conceptually:
nodes receive VNet IPs;
pods receive IPs from separate overlay pod CIDR;
Azure networking translates/routes traffic appropriately.
Benefits:
- reduced VNet IP consumption;
- scalable pod addressing;
- common Kubernetes overlay model.
Review:
- pod CIDR overlap;
- node subnet size;
- outbound SNAT;
- direct pod reachability from external Azure appliances;
- AGIC/Application Gateway compatibility for current version;
- network policy mode;
- dual-stack specifics.
Azure CNI Pod Subnet
Pods receive addresses from a delegated pod subnet.
Potential advantages:
- direct VNet-routable pod IPs;
- separate node/pod address planning;
- integration with Azure networking controls.
Concerns:
- subnet capacity;
- IP allocation strategy;
- route/NSG policy;
- peering and on-prem reachability;
- pod churn and address utilization.
Legacy kubenet considerations
Kubenet uses node-centric routing/NAT and is on a retirement path for AKS.
Migration analysis must include:
- pod/service CIDRs;
- application IP allowlists;
- source identity assumptions;
- AGIC backend reachability;
- network policy;
- UDRs;
- downtime/maintenance window;
- rollback constraints;
- test environment parity.
Do not treat network-plugin migration as a transparent implementation change.
Pod IP routability
For any component attempting direct pod access, answer:
- What is the pod CIDR?
- Who owns the route to it?
- Is the source VNet/subnet peered or connected?
- Does NSG/firewall allow it?
- Is return routing symmetric?
- Does source NAT occur?
- Is the pod IP ephemeral and how is state reconciled?
- Does the component support the selected AKS networking mode?
Service datapath
A Kubernetes Service provides stable discovery and backend selection semantics.
Depending topology, datapath can be:
Azure LB -> node -> Service rules -> NGINX pod
or:
NGINX -> ClusterIP -> Service rules -> Java pod
or direct:
Application Gateway -> Java pod IP
The Service object may exist in all three, but its role differs.
NGINX controller Service
Inspect:
kubectl get svc -A
kubectl describe svc <controller-service> -n <namespace>
kubectl get endpointslice -n <namespace> -l kubernetes.io/service-name=<controller-service>
Verify:
type;- external/internal IP;
- annotations;
- ports/targetPorts;
- selector;
externalTrafficPolicy;- health check node port;
- load balancer source ranges;
- IP family;
- events.
Managed application routing NGINX
AKS application routing add-on manages NGINX ingress controllers and related integrations.
Benefits
- managed lifecycle;
- Azure-supported configuration surface;
- integration with Azure DNS/Key Vault in supported modes;
- simplified operations.
Constraints
- controller/image/version follows managed release;
- not every upstream annotation/module is supported;
- direct customization may be restricted;
- support ends after November 2026 for the managed NGINX path according to current Microsoft guidance;
- migration requires semantic testing, not only manifest conversion.
Internal action
Create an inventory now:
IngressClasses
annotations
ConfigMap settings
snippets
custom modules
TLS/cert integration
source-IP behavior
metrics/logs
canary/sticky behavior
This becomes the migration compatibility matrix.
Multiple ingress controllers
Multiple controllers can be valid for:
- public versus private traffic;
- different trust zones;
- different teams/SLOs;
- migration canary;
- protocol-specific ingress.
They require unique:
IngressClass/controller identity;- Service/load balancer;
- DNS names;
- certificates;
- metrics and logs;
- ownership;
- admission policy.
Failure mode
An Ingress without explicit class can be processed by the wrong controller or none, depending configuration.
Source IP mental model
Track identity by hop:
| Hop | Peer IP seen | Logical client identity source |
|---|---|---|
| Front Door | client or edge-specific | trusted forwarded header |
| Application Gateway | Front Door/private source | appended forwarding chain |
| Azure LB | transport source based on topology | socket peer or preserved IP |
| NGINX | prior proxy/node | trusted header after validation |
| Java | NGINX pod/node | sanitized Forwarded/X-Forwarded-For |
Invariant
Never trust “leftmost X-Forwarded-For” without proving who was allowed to append it and where untrusted values were removed.
Forwarded header trust chain
Recommended contract:
- First trusted edge removes/normalizes spoofable headers.
- Each trusted proxy appends or rewrites according to documented policy.
- NGINX trusts only known proxy networks.
- NGINX emits one canonical identity contract to Java.
- Java does not blindly trust client-provided identity headers.
Example:
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Request-ID $request_id;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
When an earlier Azure proxy already supplies canonical values, blindly rebuilding them may lose public scheme/host. Choose and document one strategy.
TLS placement patterns
Pattern 1: terminate at Front Door, HTTP internally
Simple but weakens internal transport confidentiality.
Pattern 2: Front Door TLS -> Application Gateway TLS -> NGINX TLS
Strong transport separation but requires certificate/SNI/trust management at every hop.
Pattern 3: Azure LB TCP passthrough -> NGINX TLS termination
NGINX owns public certificate and HTTP routing.
Pattern 4: Application Gateway termination -> HTTPS backend
Application Gateway validates backend certificate/trusted root.
Pattern 5: end-to-end mTLS
Use only when workload identity and certificate lifecycle are operationally mature.
Key Vault dan certificate lifecycle
Certificates may be managed through:
- Front Door managed/customer-managed certificate;
- Application Gateway Key Vault integration;
- APIM certificate/custom domain;
- Kubernetes TLS Secret;
- cert-manager;
- AKS application-routing integration;
- internal PKI.
Inventory fields
hostname
terminating component
certificate source
issuer
renewal owner
secret/key location
rotation trigger
reload behavior
expiry alert
rollback
Common failure
Key Vault certificate renews, but consuming component remains pinned to an old version or lacks permission to fetch the new version.
End-to-end TLS dan mTLS
For each hop define:
- encryption required?;
- server certificate name/SAN?;
- trusted CA?;
- SNI value?;
- client certificate required?;
- revocation checking?;
- certificate identity propagated?;
- failure owner?;
Do not forward raw client certificate identity casually
If TLS terminates upstream and identity is sent via header:
- strip client-supplied version;
- sign or protect the channel;
- validate source proxy;
- define canonical encoding;
- avoid logging full certificate/PII;
- keep domain authorization in Java.
Health is layered
| Layer | Health question |
|---|---|
| Front Door | can origin answer expected probe? |
| Application Gateway | can backend answer configured probe? |
| Azure LB | can node/controller health port answer? |
| Kubernetes readiness | should pod receive Service traffic? |
| NGINX | is route/upstream available? |
| Java readiness | can this instance serve requests now? |
| Java liveness | is restart likely to help? |
| dependency monitor | are DB/Kafka/downstreams available? |
Anti-pattern
One deep health endpoint called every few seconds by every layer can overload dependencies and cause cascading ejection.
Timeout and connection alignment
Example chain:
client deadline: 30 s
Front Door origin timeout: < client deadline
Application Gateway request timeout: < Front Door timeout
NGINX proxy timeout: < Application Gateway timeout
Java downstream deadline: < NGINX timeout
Exact ordering depends on desired error ownership, but it must be intentional.
Failure of misalignment
Front Door aborts at 30 s,
NGINX waits 60 s,
Java keeps processing 120 s.
Result:
- client sees gateway failure;
- backend wastes capacity;
- side effect may complete after client retries;
- telemetry reports conflicting outcomes.
Connection draining and rollout
A safe rollout coordinates:
- Front Door/Application Gateway origin deregistration;
- Azure LB health probes;
- Kubernetes readiness transition;
- NGINX graceful shutdown;
- pod
terminationGracePeriodSeconds; preStopif used;- load balancer draining timeout;
- Java graceful shutdown;
- long-lived connections.
Sequence
Probe/reconciliation delay must fit the termination window.
Observability evidence map
| Layer | Evidence |
|---|---|
| Azure DNS | record/config/query result |
| Front Door | access/WAF/health logs and metrics |
| Application Gateway | access, firewall, performance, backend health |
| APIM | gateway logs, policy trace, analytics |
| Azure LB | metrics, probe state, flow evidence |
| VNet/NSG | effective rules, routes, Network Watcher |
| AKS controller | logs, events, status conditions |
| NGINX | access/error logs, metrics, generated config |
| Kubernetes | Service, EndpointSlice, pod/readiness/events |
| Java | request logs, traces, metrics, thread/connection pools |
Correlation requirement
Preserve:
- request ID;
- W3C
traceparent; - public host/path;
- source identity after normalization;
- upstream address;
- timing per hop;
- final status owner.
Azure Monitor and diagnostic settings
Azure resources do not automatically send every useful log to the destination you expect.
Verify diagnostic settings for:
- Front Door;
- Application Gateway/WAF;
- APIM;
- Key Vault;
- Azure Firewall;
- private endpoint/NSG flow-related sources where applicable;
- AKS control plane;
- managed controller logs.
Governance questions
- destination Log Analytics workspace/storage/Event Hub?;
- retention?;
- PII redaction?;
- sampling?;
- cross-subscription access?;
- alert ownership?;
- clock synchronization?;
- query/runbook tested?
Network Watcher dan flow evidence
Useful tools/features can include:
- connection troubleshoot;
- IP flow verify;
- next hop;
- effective security rules;
- packet capture;
- topology;
- flow logs where supported.
Caution
A successful TCP connection test does not prove:
- SNI/certificate correctness;
- HTTP host/path routing;
- auth;
- response buffering;
- application correctness.
Use layered tests.
End-to-end latency decomposition
Model:
T_total =
T_dns
+ T_client_to_frontdoor
+ T_frontdoor_queue_processing
+ T_frontdoor_to_origin
+ T_appgateway_processing
+ T_network_to_aks
+ T_nginx_queue_processing
+ T_service_to_pod
+ T_java_queue_processing
+ T_dependencies
+ T_response_transfer
Capture available fields at each proxy. A single 2-second “request duration” field cannot identify the slow hop.
Common failure modes
| Symptom | Likely failure domains |
|---|---|
| DNS resolves public instead of private | private zone/forwarder/VPN resolver |
| Front Door 502 | origin TLS/host/probe/connectivity |
| App Gateway backend unhealthy | probe host/path/port/TLS/route/NSG/pod |
| public LB IP exists but timeout | probe, NSG, node health, Service, route |
| NGINX 404 | host/path/class/rewrite/default backend |
| NGINX 502 | endpoint connect/TLS/reset |
| NGINX 503 | no endpoint/upstream unavailable/limit |
| NGINX 504 | upstream read timeout |
| Java sees wrong scheme | forwarded-header contract |
| all requests appear from proxy | source IP/trusted header chain |
| intermittent during rollout | probe lag/draining/pod distribution |
| private endpoint works only from one VNet | DNS link/peering/route/NSG |
| APIM returns policy error | policy expression/backend/identity |
| AGIC resource exists but route absent | reconciliation/identity/annotation/conflict |
Debugging DNS
Step 1 — identify resolver path
cat /etc/resolv.conf
nslookup api.example.com
nslookup api.example.com <expected-resolver-ip>
dig api.example.com A
dig api.example.com AAAA
From Kubernetes:
kubectl run dns-debug --rm -it --image=registry.k8s.io/e2e-test-images/dnsutils:1.3 -- sh
Step 2 — compare contexts
Test from:
- public internet;
- corporate network;
- VPN;
- AKS pod;
- jump host in VNet;
- Front Door/App Gateway configured origin context where observable.
Step 3 — validate destination
Check whether returned address belongs to:
- Front Door;
- public Application Gateway;
- internal load balancer;
- private endpoint;
- stale resource.
Debugging Front Door
- Confirm custom domain is enabled and certificate valid.
- Confirm route matches protocol/domain/path.
- Confirm origin group and enabled origin.
- Confirm origin hostname and host header.
- Confirm health probe result.
- Confirm Private Link connection if used.
- Inspect WAF blocks.
- Compare Front Door request ID with origin logs.
- Test origin directly only from authorized diagnostic path.
- Validate timeout and response-size behavior.
Diagnostic command
curl -vk --resolve api.example.com:443:<front-door-address> https://api.example.com/health
Do not rely on IP pinning for normal operation.
Debugging Application Gateway and AGIC
Kubernetes side
kubectl get ingress -A
kubectl describe ingress <name> -n <namespace>
kubectl get pods -n kube-system -l app=ingress-appgw
kubectl logs -n kube-system <agic-pod>
kubectl get svc,endpointslice -n <namespace>
Azure side
Inspect:
- listener;
- routing rule priority;
- backend pool addresses;
- backend HTTP setting;
- custom probe;
- backend health details;
- provisioning state;
- AGIC managed identity permission;
- diagnostic logs.
Controlled backend test
From a VNet diagnostic host:
curl -vk -H 'Host: api.example.com' https://<pod-or-backend-ip>:<port>/ready
Only do direct pod testing when network/security policy permits it.
Debugging Azure Load Balancer
- Verify Service external IP/internal IP.
- Verify LB rule and frontend IP.
- Verify backend pool membership.
- Verify health probe configuration and status.
- Verify
externalTrafficPolicy. - Verify controller pods exist on expected nodes.
- Verify NSG allows probe and traffic.
- Verify node health port or
/healthzendpoint. - Verify port/targetPort mapping.
- Verify no UDR/asymmetric route breaks return traffic.
Kubernetes evidence:
kubectl describe svc nginx-ingress-controller -n ingress-system
kubectl get pods -n ingress-system -o wide
kubectl get endpointslice -n ingress-system
Debugging NSG, UDR, dan firewall
Create a hop table:
| Source | Destination | Port | Expected next hop | Actual result |
|---|---|---|---|---|
| App Gateway subnet | pod CIDR | 8443 | VNet/route | ? |
| Front Door Private Link | origin private endpoint | 443 | Private Link | ? |
| NGINX pod | Java Service/pod | 8080 | cluster datapath | ? |
| Java pod | database | 5432 | firewall/private endpoint | ? |
Use:
- effective routes;
- effective NSG rules;
- Network Watcher connection troubleshoot;
- firewall logs;
- packet capture at authorized points;
- pod-level
curl/nc.
Asymmetric routing clue
SYN reaches target, response leaves through different NVA/path, connection times out without application log.
Debugging Private Link and private DNS
Checklist:
- private endpoint connection approved;
- expected subnet/IP;
- private DNS record exists;
- correct VNet linked;
- on-prem forwarder points to Azure path;
- public network access setting matches design;
- source can route to private IP;
- NSG/firewall permits;
- certificate hostname remains service hostname, not private IP;
- SNI/Host correct.
Test:
nslookup <service-hostname>
curl -vk https://<service-hostname>/health
openssl s_client -connect <private-ip>:443 -servername <service-hostname>
Debugging NGINX and Kubernetes routing
kubectl get ingress,ingressclass,svc,endpointslice,pod -A
kubectl describe ingress <name> -n <namespace>
kubectl logs -n <controller-namespace> <controller-pod>
kubectl exec -n <controller-namespace> <controller-pod> -- nginx -T
Validate:
- correct controller/class;
- host/path;
- TLS Secret;
- rewrite;
- backend Service name/port;
- ready EndpointSlices;
- NetworkPolicy;
- NGINX upstream address;
- generated configuration;
- controller events.
Controlled test from controller pod:
curl -sv http://<service>.<namespace>.svc.cluster.local:<port>/ready
Debugging TLS and source identity
TLS:
openssl s_client -connect api.example.com:443 -servername api.example.com -showcerts
curl -vk https://api.example.com/resource
Inspect every termination hop separately when possible.
Source identity:
- socket peer at each proxy;
X-Forwarded-Forchain;Forwardedheader;- Front Door/Application Gateway request IDs;
- NGINX
$remote_addrand$realip_remote_addr; - Java remote address and canonical client field.
Safe diagnostic endpoint
A restricted non-production or authenticated diagnostic endpoint can return normalized request metadata. Never expose secrets, tokens, or raw client certificates.
Java/JAX-RS implications
Public URI reconstruction
JAX-RS may see internal values:
scheme=http
host=service.namespace.svc
port=8080
path=/orders
while public URL is:
https://api.example.com/qao/orders
Define canonical forwarding metadata and trusted proxy configuration.
Redirect correctness
A redirect generated with internal host can leak topology and break clients.
Prefer relative redirects where feasible. Otherwise reconstruct from trusted external metadata.
Base path
If APIM or NGINX strips /qao, Java must know whether:
- generated links include
/qao; - OpenAPI server URL includes it;
- callbacks use it;
- auth redirect URIs use it.
Timeout/cancellation
When Azure/NGINX client disconnects, Java work may continue. Propagate deadlines and support cancellation where possible.
Readiness
Readiness should represent ability to serve new requests, not merely JVM process existence.
Client identity
Do not use untrusted forwarding headers for audit, authorization, or rate-limit keys.
Security concerns
- Public origin bypasses Front Door/WAF.
- NSG allows internet directly to internal ingress.
- Private endpoint DNS falls back to public endpoint.
- Forwarded headers can be spoofed.
- AGIC/shared gateway ownership allows cross-team impact.
- Raw Ingress annotations or snippets bypass platform policy.
- TLS terminates early and internal traffic is plaintext without acceptance.
- Managed identity has excessive resource permissions.
- APIM/NGINX logs leak tokens/PII.
- Default backend exposes diagnostic/admin endpoints.
- Certificate renewal owner is unclear.
- Multiple WAFs have inconsistent exclusions.
- Direct pod reachability expands network trust boundary.
- Route/Ingress cross-namespace references are insufficiently governed.
Performance and cost concerns
Every proxy adds
- connection setup;
- TLS processing;
- buffering;
- queueing;
- logging;
- health probes;
- data processing cost;
- operational complexity.
Analyze
- Front Door egress/caching benefit;
- Application Gateway capacity/autoscaling;
- APIM tier/throughput;
- Azure LB flow/SNAT behavior;
- NGINX CPU/memory/connections;
- cross-zone/cross-node traffic;
- log ingestion volume;
- Private Link charges;
- firewall/NVA throughput;
- duplicate compression;
- retry amplification.
Capacity invariant
The narrowest layer sets end-to-end capacity.
Front Door capacity high
App Gateway capacity medium
NGINX capacity high
Java DB pool capacity low
The database pool still governs sustainable throughput.
Safe rollout checklist
- Capture current route and dependency map.
- Verify DNS TTL and cutover plan.
- Validate certificate before traffic switch.
- Pre-create and verify health probes.
- Test public and private resolution separately.
- Validate source IP and forwarded headers.
- Compare route semantics, rewrites, and status codes.
- Test upload, streaming, WebSocket/SSE if applicable.
- Align timeout and draining windows.
- Verify metrics/logs before cutover.
- Use weighted/canary path where supported.
- Keep rollback endpoint and config available.
- Prevent simultaneous unrelated changes.
- Observe error rate, latency, backend health, and saturation.
- Confirm old ingress can be safely removed only after DNS/cache expiry and connection drain.
PR review checklist
Topology
- Is each hop necessary?
- Is control/data-plane ownership clear?
- Is direct bypass prevented?
- Does architecture match public/private requirement?
Routing
- Host/path and rewrite semantics tested?
- Controller/class explicit?
- Route conflict checked?
- Backend protocol/port correct?
Network
- VNet/subnet/IP capacity checked?
- NSG and UDR reviewed in both directions?
- Private DNS/Private Link tested?
- AKS network model compatible?
TLS and identity
- Termination and re-encryption explicit?
- Certificate source/rotation owner defined?
- SNI/Host correct?
- Forwarded-header trust documented?
Reliability
- Health probes represent real readiness?
- Timeouts ordered intentionally?
- Retry owner and idempotency known?
- Draining/termination aligned?
Observability
- Diagnostic settings enabled?
- Request/trace IDs preserved?
- Status owner distinguishable?
- Dashboard/runbook updated?
Migration
- Managed NGINX support horizon considered?
- NGINX-specific features inventoried?
- Gateway/API implementation support matrix verified?
- Rollback path tested?
Internal verification checklist
Azure subscription and ownership
- Identify subscriptions, resource groups, management groups, and owners.
- Confirm IaC/GitOps source of truth for every edge resource.
- Check Azure Policy/Blueprint-equivalent constraints.
- Identify managed identities and RBAC scope.
DNS
- Inspect public/private Azure DNS zones and records.
- Confirm registrar delegation and TTL.
- Map corporate DNS conditional forwarders.
- Verify VNet links and Private Resolver rules.
- Test split-horizon behavior from all client zones.
Front Door
- Inspect domains, routes, origin groups, origins, weights, and priorities.
- Review origin host header and probe configuration.
- Review WAF policy, exclusions, and logs.
- Verify Private Link state and origin bypass protection.
- Confirm certificate ownership and renewal.
Application Gateway / AGIC
- Inspect frontend IPs, listeners, certificates, rules, pools, HTTP settings, and probes.
- Determine whether gateway is dedicated or shared.
- Review AGIC version, identity, logs, events, and annotations.
- Confirm direct pod route/NSG reachability.
- Check manual-versus-controller ownership boundaries.
- Review migration plans toward Application Gateway for Containers if relevant.
Application Gateway for Containers
- Confirm region and feature availability.
- Inspect ALB Controller version/add-on mode.
- Check Gateway API/Ingress compatibility.
- Verify subnet delegation and managed identity.
- Inventory unsupported NGINX features.
API Management
- Identify APIs/products/policies and backend URLs.
- Check inbound private endpoint/public access.
- Check outbound VNet integration and DNS.
- Review auth, rate limit, retry, transform, and cache ownership.
- Confirm APIM logs and policy tracing access.
AKS networking
- Inspect network plugin/mode, pod CIDR, service CIDR, node/pod subnets.
- Check kubenet retirement impact if applicable.
- Review NSGs, route tables, peerings, firewall, and effective routes.
- Check address-space/IP exhaustion risk.
- Review NetworkPolicy implementation.
NGINX/application routing
- Identify self-managed, managed app-routing NGINX, F5 NIC, or other controller.
- Record controller version, image, support lifecycle, and
IngressClass. - Inspect Service annotations, health probe path, and
externalTrafficPolicy. - Inventory annotations, ConfigMaps, snippets, custom templates/modules.
- Verify support ending November 2026 migration plan for managed NGINX if used.
TLS and identity
- Map TLS termination/re-encryption/passthrough per hop.
- Inventory Key Vault, TLS Secrets, cert-manager, and internal CA.
- Verify certificate expiry alerts and rotation tests.
- Validate canonical Host/scheme/client-IP/request-ID contract.
- Confirm direct clients cannot spoof identity headers.
Observability and operations
- Enable/check diagnostic settings for all Azure edge resources.
- Locate Log Analytics workspaces and retention.
- Confirm NGINX access/error logs and metrics.
- Confirm AKS controller logs and Kubernetes events.
- Confirm Java traces/logs correlate with Azure request IDs.
- Locate runbooks, dashboards, alert rules, incident notes, and ownership contacts.
Ringkasan mental model
- Azure provides several overlapping ingress products; choose by responsibility, not brand familiarity.
- Front Door is global edge; Application Gateway is regional L7; Azure Load Balancer is L4; APIM is API product management; NGINX is cluster/application edge; Java owns domain behavior.
- AGIC and Application Gateway for Containers can route directly toward pods, producing a different datapath from LoadBalancer Service → NGINX.
- DNS, Private Link, NSG, UDR, health probe, Kubernetes readiness, and NGINX routing are separate contracts.
- Source identity must be tracked hop-by-hop and normalized at a trusted boundary.
- Health is layered; no single green indicator proves end-to-end health.
- Managed NGINX on AKS has a current support horizon through November 2026, so migration compatibility must be engineered now.
- Network-plugin choice affects pod reachability and integration; kubenet has a 31 March 2028 retirement date.
- Every extra proxy needs explicit ownership for TLS, timeout, retry, auth, headers, and telemetry.
- Production debugging succeeds when each hop has an evidence source and a named owner.
Referensi resmi
- Ingress networking in Azure Kubernetes Service
- Managed NGINX ingress with the application routing add-on
- Application routing add-on with Kubernetes Gateway API
- Configure multiple application-routing NGINX controllers
- Application Gateway Ingress Controller overview
- Application Gateway for Containers overview
- Azure Front Door overview
- Secure Front Door origins with Private Link
- Azure Load Balancer with AKS
- AKS CNI networking overview
- Azure CNI Overlay
- AKS legacy CNI and kubenet retirement
- Use API Management with AKS microservices
- API Management private endpoint
- NGINX reverse proxy module
- Kubernetes Services
- Kubernetes Gateway API
End of Part 023.
You just completed lesson 23 in deepen practice. Use the series map if you want to review the broader track, or continue directly into the next lesson while the context is still warm.
Keep the momentum while the lesson is still fresh. Move backward for review or continue forward into the next concept.