Kafka Single Node Container

Step-by-step instructions on how to deploy a Kafka broker with Docker container for developers

Kafka Container

Prerequisites: Install Docker.

Once Docker is installed, you can proceed to run Kafka as a container.

Create a docker-compose.yml file with the following content to run Kafka as a container:

docker-compose.yml : https://raw.githubusercontent.com/ContainerTalks/kafka-operations/main/single-node/docker-compose.yml

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.7.0
    container_name: zookeeper
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    networks:
      kafka:
  broker:
    image: confluentinc/cp-kafka:7.7.0
    container_name: broker
    depends_on:
      - zookeeper
    ports:
      - 9092:9092
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:9092
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 100
    networks:
      kafka:
networks:
  kafka:
    external: true

The experiences I’ve documented are available in the kafka-operations repository. Please refer to it for all the configurations.

Make the containers up with the following

docker network create kafka
docker-compose up -d

Check with the status of the container with docker ps -a

The above command will run a container to execute the command, and the container will be removed once the task is complete. In this case, we are using the command to list the topics.

Create the topic with temp container

docker run --net=kafka --rm confluentinc/cp-kafka:7.7.0 kafka-topics --create --topic test_topic --partitions 1 --replication-factor 1 --if-not-exists --bootstrap-server broker:9092

OUTPUT: WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Created topic test_topic

List the container topics with the following

docker run --net=kafka --rm confluentinc/cp-kafka:7.7.0 kafka-topics --list --bootstrap-server broker:9092

Describe topic

docker run --net=kafka --rm confluentinc/cp-kafka:7.7.0 kafka-topics --describe --topic test_topic --bootstrap-server broker:9092