Spring XD Stream – How to start with Stream

Spring XD stream is a big data solution for how to ingestion. Each Stream is defined by 3 kinds of module: Source, Sink & Processor.

The tutorial will introduce Spring XD Stream.

Relate Post: Spring XD Batch Job – How to Start


I. Technology for Spring XD Stream

– Java 8
– Spring XD 1.3.1.Release

II. Concepts

A Spring XD is define by 3 kinds of modules: SOURCE | PROCESSOR 1 | PROCESSOR 2 | … PROCESSOR n | SINK. 2 modules Source & Sink are mandatory, and Processor is option.

Stream Modules are built with Spring Integration.

1. Source

A source is a message flow, contains a direct channel named output which is fed by an inbound adapter (configured with a poller, or triggered event)

List of sources supported by Spring XD, you can see at spring-xd-1.3.1.RELEASE\xd\modules\source:

We can create a customized source module using Spring Integration.

2. Processor

A processor is a message flow that contains a direct channel: input and a subscribable channel: output. It is use for transformation on the message.

List of Processors supported by Spring XD, you can see at \spring-xd-1.3.1.RELEASE\xd\modules\processor:

3. Sink

A sink is a message flow that contains a direct channel: input and an outbound adapter, or service activator used to provide the message to an external resource.

List of sinks supported by Spring XD, you can see at \spring-xd-1.3.1.RELEASE\xd\modules\sink:

III. Practices

Create a simple stream: http | transform –expression=payload.toUpperCase() | log
Relate Post: Spring XD Batch Job – How to Start

1. Start Spring XD with single-node mode

Open a CMD
Go to: ./spring-xd-1.3.1.RELEASE\xd\bin
Run xd-singlenode

start spring xd singlemode

2. Create a Stream

Go to: .\spring-xd-1.3.1.RELEASE\shell\bin
Run xd-shell

Then run: stream create --definition "http | transform --expression=payload.toUpperCase() | log" --name myprocstrem --deploy

spring-xd-stream-create-stream

Now open Browser, go to http://localhost:9393/admin-ui/#/streams/definitions
Stream is deployed:

srping xd stream-is-deployed

You also can view on Containers:

spring xd stream container

3. Check Result

Create a Post from xd-shell
http post --target http://localhost:9000 --data "hello"
On xd-singlenode console, view log:
spring xd stream log-result

For undeploy or destroy the stream, you can use commandlines or web admins:
With command line:

spring xd stream detroy undeploy


Related Posts



Got Something To Say:

Your email address will not be published. Required fields are marked *

*