@KafkaListener | Spring Boot Example

Key Takeaways

• The @KafkaListener annotation simplifies the process of consuming messages from Kafka topics.
• The Spring Kafka framework abstracts a lot of boilerplate code, allowing developers to focus on the core logic.
• Proper configuration of consumer and container factories is crucial for optimal performance and reliability.

You might be familiar with using Kafka as a powerful event streaming platform, and Spring Boot has made it easier than ever to integrate Kafka into your applications. Let's walk through a refreshed example of using the @KafkaListener annotation for consuming Kafka messages in a Spring Boot application.

Before starting, ensure you have a Kafka cluster running locally or remotely that your application can connect to.

Kafka Listener in Practice

Here's a quick example of how you can set up a Kafka listener using the @KafkaListener annotation:

@Component
public class KafkaConsumer {
    @KafkaListener(id = "foo", topics = "myTopic")
    public void listen(String message) {
        System.out.println("Received message: " + message);
    }
}

The @KafkaListener annotation is used to define a Kafka listener. It automatically creates a KafkaMessageListenerContainer that interacts with the specified Kafka topics.

Configuring the Kafka Consumer

To make our listener work, we need to configure a consumer factory and a listener container factory:

@Configuration
public class KafkaConsumerConfig {

    @Bean
    public ConsumerFactory<String, String> consumerFactory() {
        Map<String, Object> props = new HashMap<>();
        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "http://localhost:9092");
        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        return new DefaultKafkaConsumerFactory<>(props);
    }

    @Bean
    public ConcurrentKafkaListenerContainerFactory<String, String>
    kafkaListenerContainerFactory() {
        ConcurrentKafkaListenerContainerFactory<String, String> factory =
                new ConcurrentKafkaListenerContainerFactory<>();
        factory.setConsumerFactory(consumerFactory());
        return factory;
    }
}

This configuration sets up the necessary beans for our application to consume messages from Kafka topics.

Running the Spring Boot Application

The entry point of our application is as follows:

@SpringBootApplication
public class KafkaDemoApplication {
   public static void main(String[] args) {
      SpringApplication.run(KafkaDemoApplication.class, args);
   }
}

The @SpringBootApplication annotation enables automatic configuration and component scanning.

Important Dependencies

dependencies {
   implementation 'org.springframework.boot:spring-boot-starter'
   implementation 'org.springframework.kafka:spring-kafka'
}

You'll need these dependencies, typically defined in your build.gradle or pom.xml, to integrate Kafka with Spring Boot.

The Role of @KafkaListener

The @KafkaListener annotation defines a method that listens for messages from Kafka topics specified by the topics property. Settings like groupId, concurrency, and others can be configured directly within this annotation.

Configuration and Abstraction

Spring's Kafka abstraction handles the setup of the KafkaMessageListenerContainer classes, and you often don't need to worry about their lifecycle management that's handled by the KafkaListenerEndpointRegistry.

FAQ

What is the @KafkaListener annotation used for?

The @KafkaListener annotation simplifies the process of consuming messages from Kafka by automatically configuring the necessary listener containers.

How do I configure my Kafka consumer in Spring Boot?

You need to define a ConsumerFactory and a KafkaListenerContainerFactory in a configuration class to set up your Kafka consumer.

What is the benefit of using Spring Kafka over native Kafka clients?

Spring Kafka reduces boilerplate code and handles many complexities of consumer container management, making it more straightforward to integrate into Spring applications.

Can I listen to multiple topics with @KafkaListener?

Yes, you can specify multiple topics in the @KafkaListener annotation using the topics property or use a regular expression with topicPattern to listen to multiple topics.