Lesson 9 of 10Diagnostics

14 min · theory and practice · Advanced

Kubernetes Service: Running is not Ready

Check labels, selectors, targetPort and readiness along the actual request path.

Explain an empty EndpointSlice and distinguish it from an application-port mismatch.

A Service connects a stable entry point to suitable backends. A running process establishes only one link in that chain. This case is explicitly a Kubernetes simulation: manifests control readiness and real HTTP requests without a separate cluster.

Start through the Service entry point

Check the Service name and business route, then inspect pods, services and EndpointSlices. This separates backend selection problems from routing to an already selected application.

Commands
curl -sS -i --max-time 5 http://orders-service/api/orders
kubectl get pods --show-labels
kubectl get services
kubectl get endpointslices -o yaml

These names belong to this exercise. In a real cluster, first verify the namespace and current context.

Compare the selector with the required labels

A multi-key selector requires all its label conditions to match. A Pod name resembling the Service name is insufficient. If no suitable Ready backends exist, inspect selection before changing network rules.

Commands
kubectl describe service orders-service
kubectl get deployment orders-api -o yaml
Example output
selector:
  app: orders-api
ports:
- port: 80
  targetPort: 8080

port is the Service entry port; targetPort is the application destination. They are separate boundaries.

Separate Running from Ready

A readiness probe checks whether an instance should receive traffic. The process can be Running while the probe calls a missing URL. Inspect the readiness reason, path and port, then compare them with the application’s actual endpoint.

Commands
kubectl describe pod orders-api-a
kubectl logs orders-api-a

Removing readinessProbe hides readiness information. Repair the mismatch between the probe and the application.

Inspect EndpointSlices and both replicas

Inspect conditions.ready and the port as well as addresses. An ordinary Service routes to ready endpoints by default. One responding replica does not establish the other’s health. In the simulation, compare the Ready list with replica responses.

Commands
kubectl get pods,endpointslices
kubectl exec probe-client -- curl -i http://orders-service/api/orders

Real Kubernetes has additional Service, EndpointSlice and not-ready publishing modes. This exercise covers the ordinary selector-based path without modeling CNI or scheduling.

Change one contract and recheck the path

After a manifest change, wait for observed state to update, recheck Ready and endpoints, then send a bounded set of requests through the Service. Preserve both ready replicas and the correct probe. Testing only a Pod IP bypasses the entry point under investigation.

From symptom to check

ObservationNext checkHow to interpret it
No selected endpointsService selector and Pod labelsCompare every key/value pair, not the Pod name.
Endpoint exists but its connection failstargetPort and application portThe Service may send traffic to the wrong port.
Running 0/1, ready: falsereadinessProbe and its resultThe application runs but is not ready for ordinary Service traffic.

Common mistakes

  • Removing readinessProbe to make a Pod look healthy.
  • Testing only a direct Pod address rather than the Service.
  • Recovering one replica without checking the other.

How to prove recovery

  1. Match labels, selector and intended namespace.
  2. Verify the correct port and readiness for both replicas.
  3. Inspect the updated EndpointSlice, including ready and port.
  4. Test a bounded request series through the Service and verify both replicas participate.

MINI INCIDENT

Why is a Running Pod excluded from traffic?

The selector matches and the process serves /health/ready, but its probe checks /health/old and receives 404. What does ready: false mean in the EndpointSlice, and how can you confirm the hypothesis?

Evidence to investigate
Pod: Running 0/1
readiness probe: GET /health/old -> 404
EndpointSlice: conditions.ready=false
Explore the diagnosis

Match the probe path and port with the application contract and describe output. An unready endpoint should not receive ordinary Service traffic. After aligning them, wait for Ready and Slice updates, then test the Service while preserving readiness checks.

WORKSHOP

Workshop: explain the evidence and choose an action

The names look plausible, but the Service selects no ready addresses. Identify the exact contract.

Sample evidence
Service selector: app=orders,release=v2
Pod labels:       app=orders,release=v1
Pod state:        Running, Ready=True
EndpointSlice:    no matching addresses
  1. Why are Running and Ready not sufficient?

    Compare your reasoning

    The Pod must also belong to the Service’s selection. The specified label requirements must match; a similar Pod name does not make it an eligible backend.

  2. How can you determine which side of the contract is wrong?

    Compare your reasoning

    Compare the intended release, Deployment template and Service definition. Blindly changing a label is unjustified: the Service may correctly expect v2 while the rollout remains on v1.

  3. What should be checked once selectors and labels agree?

    Compare your reasoning

    Verify ready EndpointSlice addresses, all selected replicas and a request through the Service from the client context. A direct Pod probe does not test the Service.

CHECK YOUR UNDERSTANDING

All Pods are Running, but the EndpointSlice has no ready addresses. Does this prove DNS is broken?

Show explanation

No. First inspect selector/label matching and readiness. A running process and DNS name do not guarantee inclusion among ready backends.

Documentation

Guest progress is stored separately in this browser.

TAKE IT TO THE TERMINAL

Practice on an incident

Restore the Service in the training simulation, preserve readiness and verify both replicas participate.

DID IT MAKE SENSE?

Was this article helpful?

Rate the article or tell us what needed more explanation. Your feedback helps improve the lessons.

Helpful 0 Not helpful 0

Sign in to rate this article and leave a comment.

Discussion

0

Join the discussion →

No comments yet. Share your first impression or ask a question about this topic.