How to use Spring Kafka JsonSerializer (JsonDeserializer) to produce/consume Java Object messages

In the previous post, we had setup a Spring Kafka Application succesfully by explicitly configuration Kafka Factories with SpringBoot. But the messages had been used have String type. While in the development, POJO (Plain Old Java Object) are often used to construct messages. So with the tutorial, JavaSampleApproach will show how to use Spring Kafka JsonSerializer (JsonDeserializer) to produce/consume Java Object messages.

Related Articles:
How to start Spring Kafka Application with Spring Boot
How to start Spring Apache Kafka Application with SpringBoot Auto-Configuration

I. Technologies

– Java 8
– Maven build
– Spring Boot
– Apache Kafka
– Spring Tool Suite editor

II. Overview

Spring Kafka Application - architecture

In the tutorial, we send and receive Java object messages to/from Apache Kafka, so ProducerFactory uses JsonSerializer.class and ConsumerFactory uses JsonDeserializer.class to serialize/deserialize Java objects to Json bytes.

KafkaProducerConfig:

KafkaConsumerConfig:

– Note: SpringKafka uses Jackson library to serialize/de-serialize Java objects to/from Json bytes so we need jackson-databind dependency.

III. Practice

We create a SpringBoot project with 2 main services: KafkaProducer and KafkaConsumer for sending and receiving messages from Apache Kafka cluster.

spring kafka application send Java Object - project structure

Step to do:
– Create a SpringBoot project
– Create Customer model
– Create Kafa Factories (ProducerFactory & ConsumerFactory)
– Create Services (Producer and Consumer)
– Implement Client
– Deployment

1. Create a SpringBoot project

Use SpringToolSuite to create a SpringBoot project, then add dependencies {spring-kafka, jackson-databind}:

2. Create Customer model

3. Create Kafa Factories (ProducerFactory & ConsumerFactory)

Open application.properties, add kafka configuration:

jsa.kafka.bootstrap-servers is used to indicate the Kafka Cluster address.
jsa.kafka.consumer.group-id is used to indicate the consumer-group-id.
jsa.kafka.topic is used to define a Kafka topic name to produce and receive messages.

3.1 Create ProducerFactory and KafkaTemplate

3.2 Create ConsumerFactory and KafkaListenerContainerFactory

4. Create Services (Producer and Consumer)

– Create a KafkaProducer service:

– Create a KafkaConsumer service:

5. Implement Client

6. Deployment

Start Apache Kafka Cluster:
– Start a ZooKeeper:

– Start the Apache Kafka server:

>>> More details at: How to start Apache Kafka

Build and Install the SpringBoot project with commandlines: mvn clean install and mvn spring-boot:run

-> Logs:

IV. Sourcecode

SpringKafkaSendConsumeJavaObject

By JavaSampleApproach | June 11, 2017.


Related Posts


Got Something To Say:

Your email address will not be published. Required fields are marked *

*