· til kubernetes kafka

Kafka/Kubernetes: Failed to resolve: nodename nor servname provided, or not known

I’ve been trying out the Running Pinot in Kubernetes tutorial and ran into a problem trying to write data to Kafka. In this blog we’ll explore how I got around that problem.

I’m using Helm with Kubernetes and started a Kafka service by running the following:

helm repo add kafka https://charts.bitnami.com/bitnami
helm install -n pinot-quickstart kafka kafka/kafka --set replicas=1,zookeeper.image.tag=latest

I waited until the service had started and then ran the following command to port forward the Kafka service’s port 9092 to port 9092 on my host OS:

kubectl port-forward service/kafka-headless 9092:9092 -n pinot-quickstart

I then ran a script that attempted to write some data into Kafka:

python datagen.py --sleep 0.0001 2>/dev/null |
jq -cr --arg sep ø '[.uuid, tostring] | join($sep)' |
kcat -P -b localhost:9092 -t events -Kø

This hung for a few seconds before throwing the following error:

%3|1685969550.862|FAIL|rdkafka#producer-1| [thrd:kafka-0.kafka-headless.pinot-quickstart.svc.cluster.local:9092/]: kafka-0.kafka-headless.pinot-quickstart.svc.cluster.local:9092/0: Failed to resolve 'kafka-0.kafka-headless.pinot-quickstart.svc.cluster.local:9092': nodename nor servname provided, or not known (after 5002ms in state CONNECT)

I’d assumed that all I needed to do was map the port, but it seems like it tries to resolve the hostname, which isn’t known by the host OS. A workaround for this issue is to update your /etc/hosts file: kafka-0.kafka-headless.pinot-quickstart.svc.cluster.local

If your Kafka has a different host name, be sure to change the second argument appropriately.

  • LinkedIn
  • Tumblr
  • Reddit
  • Google+
  • Pinterest
  • Pocket