Skip to content
Edouard Topin's Blog
VKS: deploy, troubleshoot and operate an application with TOP Dev / Series 02/05

VKS Networking: Trace DNS, Services, Endpoints, and Database Paths

Trace a VKS request from DNS and TLS through Service endpoints to the stateless app and its PostgreSQL dependency.

Edouard Topin
7 min read
At the coffee corner, a colleague shows TOP the failed order on her phone while the developer listens.

At the coffee corner, a colleague shows TOP the order that fails from her phone. The internal menu correctly shows zero coffee beans, but the reorder button still does not work. TOP records this request’s context before following it through DNS and the Service, then checking whether the API can reach PostgreSQL.

Before deciding the app is wrong, he needs to follow the request: the right name, the right destination, a ready backend, and a response he knows how to interpret.

In episode one, we distinguished the management API from the workload cluster. Now we follow GET /menu to that cluster’s application. TOP Dev uses the application’s behavior to guide the investigation, then connects it to infrastructure evidence.

Revised on 27 October 2026, this article separates common Kubernetes concepts from the actual VKS network design. An NSX/VPC-backed environment and a VDS-backed one need their own supported topology. The exercise commands target a disposable workload namespace described in the next episode; they are not instructions to modify a production network.

Write the starting point on the test result

“Coffee Break works from my laptop” is incomplete evidence. Which address, port, protocol and path did the laptop use? Did it traverse the external gateway, connect through port-forward or reach a Pod directly? Was TLS verified? Each answer changes what the success proves.

TOP records the source environment, destination name, resolved address, port, release and response. He uses two paths: a request inside the application cluster and a request from the intended user network. They answer different questions.

Observation Useful next check
Name does not resolve Resolver used, search domain, DNS record and permitted DNS traffic
Name resolves, TCP times out Actual destination, route, filtering, return path and service health
TLS verification fails Certificate chain, name, trust and the real termination point
HTTP 404 Host/path routing and the handler receiving the request
HTTP 503 Which component returned it and which upstream dependency is unavailable
HTTP 200 with the wrong list Actual release, request target, configuration and data source

A timeout does not identify a firewall by itself. An HTTP status does not identify its issuer by itself. Preserve the response and relevant logs before changing rules. The upstream DNS debugging guide helps establish which resolver and name are involved.

A Service needs matching, usable destinations

Inside our workload cluster, the coffee Service selects Pods labeled app: coffee. Kubernetes records backend endpoints through EndpointSlices. Readiness influences whether a backend is normally eligible for service traffic. This links three observations: the Service’s selector and port, the Pod’s labels and readiness, and the published endpoints. Debug Services.

The coffee Service selects app=coffee Pods; EndpointSlices record their addresses and readiness. A label mismatch leaves the Service without matching backends even while the Deployment has running Pods.

Logical selection and endpoint discovery. The drawing does not prescribe a proxy, CNI implementation or physical route.

With the disposable example installed and the kubeconfig context named coffee-workload:

kubectl --context coffee-workload -n coffee-demo get service coffee -o yaml
kubectl --context coffee-workload -n coffee-demo get pods -l app=coffee --show-labels
kubectl --context coffee-workload -n coffee-demo get endpointslices -l kubernetes.io/service-name=coffee -o yaml

Check port, targetPort, labels, endpoint addresses and readiness together. A Pod in phase Running can still be unready. A selected process may listen on a different port. A Service with no suitable destinations cannot manufacture an application response.

In the small exercise, this call runs in one of the existing coffee Pods and uses the internal Service name:

kubectl --context coffee-workload -n coffee-demo exec deployment/coffee -- wget -qO- http://coffee/menu

Expected output lists coffee-beans with stock 0 and filters with stock 24. The image includes wget; another application image may not. Exec must be allowed by your role. A diagnostic Pod with different labels or identity can be treated differently by policy, so describe its context rather than assuming equivalence.

At the diagnostic board, TOP and his colleague connect their observations to the request path: DNS resolution, Service selector and EndpointSlices. One line is crossed off in the notebook: that check passed. They still need to identify the failing hop.

Standing at a mobile whiteboard, TOP follows the request steps while his colleague compares a test-result sheet.

External access adds a separate arrangement

A ClusterIP Service is an internal abstraction. For a user outside the cluster, the platform supplies an approved exposure mechanism. A LoadBalancer Service delegates provisioning to the installed integration. HTTP host/path routing requires an implemented gateway or ingress arrangement. Merely creating an API object does not install the controller that fulfills it. Kubernetes Service types.

The former article drew Avi/AKO, T0/T1 and direct Pod routing as one mandatory sequence. That is too broad. Identify the actual load-balancer provider and backend mode: it may forward through node-level targets, another supported integration or the platform’s configured routing. Do not infer direct routability of every Pod from the presence of NSX.

For NSX/VPC, document the published subnet, gateway, allocation, exposure and egress policies. For VDS, document the workload network and supported load-balancing arrangement. On either path, the application contract still needs addresses, ports, ownership and a return route. Broadcom groups these choices and policies in its VKS networking guide.

Three separate paths: user through the configured DNS/TLS/gateway to the application; workload API access for operators; application egress to DNS, registry or database through the configured network controls.

The boxes marked “configured” are questions to resolve against your environment. They are not a claim that every VKS deployment uses the same load balancer or NAT.

For the external test, use the real service FQDN from the intended client network. Verify HTTPS normally, including its trusted CA. Record where TLS terminates and whether the next hop is encrypted. A valid certificate on the gateway does not establish the database’s trust configuration.

The app’s outgoing request starts another journey

A persistent version of Coffee Break could, for example, call PostgreSQL on TCP 5432. That connection originates from its execution environment, not the user’s browser. The observed source may be translated. Ask the network team which address the database firewall actually sees and how the reply returns.

Node addresses, Pod CIDRs, Service CIDRs and external address pools serve different purposes. Reserve and validate them through the network design; check overlap with reachable corporate networks and the supported platform rules. Do not turn one example CIDR into a universal allocation plan.

A standard Kubernetes NetworkPolicy can select ingress and egress permissions when a compatible network implementation enforces it. Policies can be additive, and isolated ingress/egress must both permit the relevant path. They cannot override a denial in another firewall. See Network Policies.

TOP starts with the application needs: DNS, approved destinations and ports. The platform translates them into the right controls. Applying a default-deny policy without planning DNS and dependencies can break a perfectly healthy process. Conversely, an allow-all policy hides which flows the service actually needs.

Three tests and their limits: port-forward to a Pod, internal Service request, then the external DNS and TLS path. Success on one does not prove every path.

Keep a before/after network acceptance sheet

Before exposure or a network change, record the Service selector and ports, ready endpoints, image release, intended DNS/TLS behavior and one successful menu response. Also capture the application-to-database test if that dependency exists. The static fixture in our exercise deliberately has no database; its success cannot prove that separate dependency.

After the change, repeat the same requests from the same points. Compare the response contents as well as the status. If internal access works and external access fails, investigate the additional external arrangement with its owner. If internal endpoints disappear, start with labels and readiness before asking for a BGP change.

Port-forward is helpful to inspect a selected application process, but bypasses several parts of the normal service path. Treat its success as scoped evidence. In the next episode, we will intentionally change the demo Service selector, observe the missing backends, then restore it. The Pod count will stay reassuringly unchanged throughout the fault.

Beside the storage cupboard, his colleague shows him the empty jar: the menu’s zero stock makes sense, but they still need to place an order. TOP keeps one precise network question in his notebook: which hop between the browser, API Service and PostgreSQL is failing?

TOP stands with folded arms beside the coffee cupboard while his crouching colleague shows the team the empty bean jar.

Continue with the first cluster and first application. We will keep the two contexts explicit and make a small service work before adding a public address or a database.

Get the next one by email

New articles and series, sent when they are published. No other mail.

One click to unsubscribe, any time.

Back to blog
Share

Related articles

  1. 7 min read

    Cilium Default-Deny and DNS: TOP Troubleshoots the Festival Running Order

    Trace DNS and HTTP failures in a disposable Cilium lab as TOP helps a festival stage manager restore the running-order app.

  2. 9 min read

    Aria to VCF Automation 9.1.1: TOP redraws the address book

    Dedicated service runtimes, reused VIPs, new node IPs and internal CIDRs: understand the architecture before upgrading.

  3. 11 min read

    From factory to shop: GitOps and rolling updates with TOP

    Magical Unicorn is packaged but missing from the menu. Follow TOP through Git, Argo CD and VKS to replace a release while the shop serves.

Follow along

New articles, thoughts, and updates.