We earn commissions when you shop through the links below, at no extra cost to you. Our guides and technical recommendations are written independently of these relationships: a commission never decides what we recommend, how we diagnose a problem or what we publish, and the content does not depend on it. Affiliate disclosure

Kubernetes · Intermediate · 15 min

Fix a Kubernetes Service with no endpoints

Fix a Kubernetes Service with no endpoints: compare the selector with Pod labels, check targetPort, and find Pods that are not ready.

A Service only forwards to ready Pods whose labels match its selector, on its targetPort. This runbook reads the endpoints and EndpointSlices to tell those three causes apart, fixes each in its source, and covers what to check when the endpoints look right. Commands were run on a kind cluster with Kubernetes v1.37.0.

See it run

A short replay of real output from commands in this guide, captured on a WSL2 Ubuntu test machine (Docker 29.1.3, kind v0.33.0, Kubernetes v1.37.0, single-node kind cluster). The typing is replayed, and the host name, account and MAC addresses are redacted. Your output will differ, and a test machine is not a production server.

Text version of this demo
ops@web-01:~$ kubectl -n demo-svc get endpoints web 2>/dev/null
NAME   ENDPOINTS   AGE
web    <none>      0s

ops@web-01:~$ kubectl -n demo-svc describe service web | grep -E 'Selector|Endpoints'
Selector:                 app=frontend
Endpoints:                

ops@web-01:~$ kubectl -n demo-svc get pods --show-labels
NAME                   READY   STATUS    RESTARTS   AGE   LABELS
web-6bd858c6f9-qqjht   1/1     Running   0          10s   app=web,pod-template-hash=6bd858c6f9

ops@web-01:~$ kubectl -n demo-svc patch service web --type=json -p '[{"op":"replace","path":"/spec/selector","value":{"app":"web"}}]' && sleep 2 && kubectl -n demo-svc get endpoints web 2>/dev/null
service/web patched
NAME   ENDPOINTS       AGE
web    10.244.0.5:80   3s

ops@web-01:~$ kubectl -n demo-svc run curl --image=busybox:1.37 --restart=Never --rm -i --quiet -- wget -qO- -T 5 http://web | grep -o '<title>.*</title>'
<title>Welcome to nginx!</title>

A Service is a stable name and IP address in front of a set of Pods. It does not know about Pods directly: its selector picks them by label, only Pods that pass their readiness probe receive traffic, and targetPort says which container port to send it to. When any of those three is wrong, the Service exists and resolves in DNS, but requests are refused or time out. This runbook finds which of the three it is in about a minute.

Commands and output were captured on a single-node kind cluster (kind v0.33.0, Kubernetes v1.37.0, kubectl v1.37.1) with an nginx:1.27-alpine Deployment.

What a Service needs before it sends traffic to a Pod1. The selector matches (the Pod carries every label the Service asks for (section 2)) Then 2. The Pod is ready (it passes its readiness probe (section 4)) Then 3. targetPort is right (the container really listens on that port (section 3)) Then 4. Requests reach the Pod (the address is listed in the endpoints).The selectormatchesthe Pod carries everylabel the Serviceasks for (section 2)The Pod is readyit passes itsreadiness probe(section 4)targetPort isrightthe container reallylistens on that port(section 3)Requests reachthe Podthe address is listedin the endpoints
The Service exists and resolves in DNS even when all three conditions fail. Look at the endpoints first: an empty list points to the selector or to readiness, and a wrong targetPort breaks requests to a Pod that is listed.

0. Rules of engagement

  1. Read before you change. Sections 1 to 3 change nothing.
  2. Fix the Service or the Pod template in its source (manifest, Helm values, Kustomize overlay), not only with kubectl patch; the next deploy would undo a live patch.
  3. Do not remove a readiness probe to make endpoints appear. It would send traffic to Pods that are not ready.

1. Look at the endpoints

Since Kubernetes 1.33 the Endpoints API is deprecated and kubectl get endpoints prints a warning; it still works. The current source of truth is EndpointSlices, which also show whether each address is ready.

# Endpoints for the Service (deprecated API, still the quickest view)
kubectl -n NAMESPACE get endpoints SERVICE
# Real output from the test cluster: no Pod matched the selector
Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
NAME   ENDPOINTS   AGE
web    <none>      0s
# EndpointSlices with each address and its ready condition
kubectl -n NAMESPACE get endpointslices -l kubernetes.io/service-name=SERVICE \
  -o jsonpath='{range .items[*].endpoints[*]}{.addresses[0]} ready={.conditions.ready}{"\n"}{end}'
What you seeCauseGo to
<none>, no addresses at allThe selector matches no PodSection 2
Addresses with an unexpected port, for example 10.244.0.5:8080targetPort does not match the container portSection 3
Addresses with ready=falsePods fail their readiness probeSection 4
Correct addresses, ready=true, requests still failNot a Service problem: NetworkPolicy, the client, or DNSSection 5

From inside the cluster, each of the first two cases looked the same on the test cluster: the request was refused.

# Real output from a busybox Pod in the cluster, both for a selector mismatch and for a wrong targetPort
wget: can't connect to remote host (10.96.88.7): Connection refused

2. The selector matches no Pod

# The Service selector, then the labels the Pods really have
kubectl -n NAMESPACE describe service SERVICE | grep -E 'Selector|TargetPort|Endpoints'
kubectl -n NAMESPACE get pods --show-labels
# Real output from the test cluster: the Service wants app=frontend, the Pod is labeled app=web
Selector:                 app=frontend
TargetPort:               80/TCP
Endpoints:
NAME                   READY   STATUS    RESTARTS   AGE   LABELS
web-6bd858c6f9-6g4vf   1/1     Running   0          11s   app=web,pod-template-hash=6bd858c6f9

Every key and value in the selector must be on the Pod, letter for letter. Check:

  • Typos and case: app: Web is not app: web.
  • Extra selector keys: a selector with app: web, tier: frontend needs both labels on the Pod.
  • The namespace: a Service selects Pods in its own namespace only.
  • Helm and Kustomize often add labels such as app.kubernetes.io/instance; make the Service template use the same helper as the Deployment.

Test a selector directly: kubectl -n NAMESPACE get pods -l app=frontend must list the Pods you expect.

3. targetPort does not match the container

# Real output from the test cluster: the selector is fixed, but targetPort is 8080 and nginx listens on 80
NAME   ENDPOINTS         AGE
web    10.244.0.5:8080   12s

The endpoint exists, so this looks healthy at a glance, but nothing listens on that port inside the Pod. Compare:

# The port the Service sends to, and the ports the container declares
kubectl -n NAMESPACE get service SERVICE -o jsonpath='{.spec.ports[*].targetPort}{"\n"}'
kubectl -n NAMESPACE get pods -l app=web -o jsonpath='{.items[0].spec.containers[*].ports[*].containerPort}{"\n"}'
  • port is what clients use (http://web:80); targetPort is where the application listens in the container. They often differ (port: 80, targetPort: 8080).
  • A named targetPort (for example http) must match a ports[].name in the container spec.
  • containerPort is informational: an application that listens on another port than it declares still breaks the Service. kubectl -n NAMESPACE exec POD -- netstat -ltn (or ss -ltn) shows what it really listens on, if the image has the tool.

After setting targetPort: 80, the same request returned the nginx welcome page on the test cluster.

4. Pods are not ready

# Real output from the test cluster during a rollout with a readiness probe on a path that does not exist
NAME                   READY   STATUS    RESTARTS   AGE
web-5f46cfd98f-8gjvv   0/1     Running   0          26s
web-6bd858c6f9-6g4vf   1/1     Running   0          55s
10.244.0.5 ready=true
10.244.0.9 ready=false

A Pod that is Running but 0/1 ready is listed in the EndpointSlice with ready=false and gets no traffic. Here the old Pod is still ready, so the Service keeps working while the rollout is stuck; when every Pod is unready, the Service has no usable endpoint at all.

# Why the probe fails: the events at the end of describe
kubectl -n NAMESPACE describe pod POD | sed -n '/Readiness/p;/Events:/,$p'

Fix the probe (path, port, timing) or the application, as described in Troubleshoot Kubernetes probes and resource limits.

5. Endpoints look right, requests still fail

  • NetworkPolicy: kubectl -n NAMESPACE get networkpolicy. A default-deny policy blocks traffic until an allow rule matches the client.
  • DNS: from a test Pod, nslookup SERVICE.NAMESPACE.svc.cluster.local. Across namespaces, the short name does not resolve.
  • Headless Services (clusterIP: None) have no virtual IP; clients get the Pod addresses directly from DNS.
  • The client itself may use the wrong port, or https against a plain-HTTP application.

6. Verify

# A request from inside the cluster, through the Service name
kubectl -n NAMESPACE run curl --image=busybox:1.37 --restart=Never --rm -i --quiet -- wget -qO- -T 3 http://SERVICE
# Real output from the test cluster after the fix (trimmed)
<title>Welcome to nginx!</title>

7. Rollback and prevention

  • Rollback: reapply the previous Service manifest, or kubectl rollout undo deployment/NAME if a Pod template change broke the labels or ports.
  • Generate labels and selectors from one place (a Helm helper or Kustomize labels), so the Service and the Deployment cannot drift apart.
  • Name your ports (name: http) and use the name as targetPort, so a port change in the container follows through.
  • Alert on Services with no ready endpoints, for example with kube-state-metrics or a synthetic check through the Service name.

References

Official reference

kubernetes.io ↗

Technical steps are original explanations; consult the upstream documentation for version-specific details.

Behavior differs by distribution and tool version. Commands are examples for authorized systems; review output for secrets before sharing.

Found this useful?

Follow us on YouTube for video explainers, and find us on Facebook to keep up with what is new.