Unveiling the Enterprise Integration Patterns (EIP) - Part Two

Hey there! I'm excited to have you back for the second part of our EIP journey. Last time, we covered the basics of Enterprise Integration Patterns and Message Channels. Today, I want to share something more personal - our team's experience implementing these patterns in a real-world project. Trust me, it's been quite a ride!

Theory in Practice - EIP Demo Project - Part 1

The Challenge We Faced

You know how it goes - you start with what seems like a straightforward requirement: “We need a service that can handle messages reliably”. As always, you get surprised by how many things pop up in the uncovering process, things like:

Ensure no message gets lost (our stakeholders were very clear about this!)
Process messages through different workflows (and make it easy to add new ones)
Handle messages from and to various systems
And of course, make it all bulletproof with proper monitoring, logging, and error handling

I remember our team sitting in that first architecture meeting, sketching out ideas on the whiteboard, and realizing that it would be fun to try something new for a change and start to explore, that’s when we came across EIP.

How We Tackled It: Our Architecture Journey

After much coffee and debate, we decided to break our service into three main components:

An Inbound Component (the welcoming committee)
A Core Processing Component (where the magic happens)
An Outbound Component (the farewell party)

The idea is that you can work on one component of the system independently without affecting the others. Since communication between components occurs via message channels, and since they are agnostic to the type of message channels they receive and send messages from/to, they could even be split into separate services. Without EIP, we would need to define and implement all of the message-passing functionality, which is already standardized inside EIP.

Let me walk you through how we brought each piece to life.

The Welcome Mat: Our Inbound Component

First up was the inbound component. We needed something flexible that could handle requests from a variety of sources. The first thing we pulled from our arsenal was the Message Gateway pattern which would offer a clean and elegant solution to this problem.

@MessagingGateway
interface InboundGateway {
    @Gateway(requestChannel = REQUEST_CHANNEL)
    fun <T> forward(message: Message<T>)
}

I love how clean this turned out - it’s like having a really well-trained receptionist who knows exactly what to do with every message that comes in. No matter if it’s a REST call, GraphQL query, or message from a queue - our gateway handles it all efficiently. After this bad boy does his work we then safely store this message in our DB for safekeeping and later on processing. The first point for our stakeholders is checked!

The Engine Room: Core Processing Component

Now, this is where things got interesting. We needed a reliable way to process messages without losing any of them. After a few spirited debates, we landed on using a Polling Consumer pattern:

@Configuration
class MessageQueue(
    private val messageStore: JdbcChannelMessageStore,
    private val poller: PollerSpec,
    private val point2PointChannel: MessageChannel
) {
    @Bean
    fun pollingConsumerChannel(): MessageChannel = QueueChannel(
        MessageGroupQueue(messageStore, "polling-consumer")
    )

    @Bean
    fun messageFlow(): IntegrationFlow = integrationFlow(pollingConsumerChannel()) {
        transform<Notification>(
            { "Message Receiver From Poller" },
            { poller(poller) }
        )
        channel(point2PointChannel)
    }
}

One drawback if you used a ready-made implementation of a polling consumer is limited customizability. In our case we wanted to use the LISTEN / NOTIFY feature of PostgreSQL, which required a custom implementation. We learnt this the hard way with a few sleepless nights and after contacting the person who wrote the framework for EIP in Spring.

For handling different business workflows, we implemented Service Activators. They’re like specialized workers in a factory, each knowing exactly what to do with their specific type of message:

@Configuration
@EnableIntegration
class ServiceActivatorExample {
    @ServiceActivator(
        inputChannel = "inboundRequestChannel"
    )
    fun handler(notification: Notification) {
        // This is where the magic happens!
    }
}

The Farewell Committee: Outbound Component

For our outbound component, we wanted something that could reliably deliver messages while keeping an eye on everything that goes out. Think of it as a meticulous postal service that not only delivers your packages but also keeps detailed records of every delivery. Similarly, for the inbound component, a Message Gateway will not only define the boundary against other external/internal systems, but also provide an easy way for adding new listeners to the outgoing messages that perform tasks such as logging and other concerns.

Let’s Talk About Why EIP Made Our Lives Easier

Let me tell you about the real game-changers we discovered when using these EIP patterns in our system:

First off, these patterns are amazing at keeping things separate and organized - kind of like a well-designed office building with a good office layout, in which different departments can do their work without bumping into each other. That’s exactly what Messaging Endpoints do for our code. The Marketing team doesn’t need to know how Sales processes their leads, right? Same principle! Thanks to patterns like Messaging Gateway and Service Activator, we can modify one part of the system without messing up the whole thing. Trust me, when you need to upgrade or replace something later, you’ll be thankful for this separation!

Then there’s the flexibility these patterns give you - it’s like having a universal translator for your system. Remember that time we needed to connect our system to a legacy application that felt like it was speaking a completely different language? The Messaging Gateway made it a breeze by handling all the translation work for us. And here’s a cool thing: our Polling Consumer lets us work with older systems that can’t actively push messages. It’s like having a reliable courier who checks the mailbox at regular intervals, making sure we don’t miss any important deliveries.

Here’s another thing I love: asynchronous processing. Think of Service Activator as your super-efficient personal assistant. Instead of waiting around for each task to be completed before starting the next one, it can tackle multiple tasks at once. We saw a huge improvement in how our system handles busy periods - it’s like upgrading from a single-lane road to a multi-lane highway. When you’re dealing with thousands of messages per minute, this kind of efficiency is priceless!

Lastly, let’s talk about keeping things consistent. Do you know how frustrating it is when everyone uses a different format for their reports? Well, Message Construction patterns solve that problem for our system. They ensure every message follows the same structure - like having a standardized template that everyone uses. Headers go here, content goes there, and everything is in its proper place. It makes life so much easier when you’re processing messages because you always know what to expect. Plus, it cuts down on those annoying errors that pop up when different parts of the system can’t understand each other.

I have to say, the implemented patterns may give the impression that all this is too easy. But to have something like this functioning well, you have to understand the tools you are using to build blocks. Seeing them in action gives you a different perspective!

Building Our Component Toolkit

Now, let me walk you through the process of building some of these components. Think of this as your practical guide to EIP implementation - the stuff I wish someone had shown me when I started!

Point-to-Point Channel: Your Message Highway

First up, let’s look at creating a simple message channel. It’s like building a dedicated lane for your messages:

@Bean
fun point2PointChannel(): DirectChannel = DirectChannel()

Want to use it? Here’s how we did it in our REST controller:

@RestController
@RequestMapping("api/v1/")
class MessageSenderController(
    private val point2PointChannel: DirectChannel,
) {
    @PostMapping("send")
    fun receive(@RequestBody notification: Notification): ResponseEntity<String> {
        try {
            point2PointChannel.send(MessageBuilder.withPayload(notification).build())
            return ResponseEntity.ok("Message sent successfully!")
        }
        catch (e: Exception) {
            return ResponseEntity.badRequest().body(e.message!!)
        }
    }
}

Messaging Gateway: Your System’s Front Desk

Think of this as your system’s friendly receptionist. It knows exactly how to handle incoming messages and where to send them:

@MessagingGateway
interface InboundGateway {
    @Gateway(requestChannel = "inboundRequestChannel")
    fun forward(notification: Notification)
}

Using it is super straightforward:

@RestController
@RequestMapping("api/v1/")
class MessageReceiverController(
    private val inboundGateway: InboundGateway,
) {
    @PostMapping("receive")
    fun receive(@RequestBody notification: Notification): ResponseEntity<String> {
        try {
            inboundGateway.forward(notification)
            return ResponseEntity.ok("Message forwarded successfully!")
        }
        catch (e: Exception) {
            return ResponseEntity.badRequest().body(e.message!!)
        }
    }
}

Service Activator: Your Message Handler

This is where your messages get processed. Think of it as your specialized worker who knows exactly what to do with each type of message:

@Configuration
@EnableIntegration
class ServiceActivatorExample {
    @ServiceActivator(
        inputChannel = "inboundRequestChannel"
    )
    fun handler(notification: Notification) {
    }
}

Polling Consumer: Your Reliable Message Fetcher

Last but not least, here’s how we set up our polling consumer. It’s like having a dedicated courier who checks for new messages at regular intervals:

Let me show you how we set up our polling consumer. First, let’s configure the basic building blocks:

@Configuration
class MessageStoreConfiguration {
    @Bean
    fun messageStore(dataSource: DataSource): JdbcChannelMessageStore {
        val jdbcChannelMessageStore = JdbcChannelMessageStore(dataSource)
        jdbcChannelMessageStore.setChannelMessageStoreQueryProvider(H2ChannelMessageStoreQueryProvider())
        return jdbcChannelMessageStore
    }
}

@Configuration
class PollerConfiguration {
    @Bean
    fun poller(transactionManager: TransactionManager): PollerSpec {
        return Pollers.fixedDelay(100L)
    }
}

And here’s how we put it all together in our message queue:

@Configuration
class MessageQueue(
    private val messageStore: JdbcChannelMessageStore,
    private val poller: PollerSpec,
    private val point2PointChannel: MessageChannel
) {
    @Bean
    fun pollingConsumerChannel(): MessageChannel = QueueChannel(
        MessageGroupQueue(messageStore, "polling-consumer")
    )

    @Bean
    fun messageFlow(): IntegrationFlow = integrationFlow(pollingConsumerChannel()) {
        transform<Notification>(
            { "Message Receiver From Poller" },
            { poller(poller) }
        )
        channel(point2PointChannel)
    }
}

What’s Next?

In the upcoming articles, we will continue our deep dive into EIP, exploring patterns related to Message Routing, Message Transformation, and System Management. By understanding these advanced patterns, you will be equipped to tackle even more complex integration challenges and build resilient, high-performance systems.

Thank you for joining us on this exploration of Enterprise Integration Patterns. We look forward to your continued participation and feedback as we delve deeper into this fascinating and essential area of software architecture.

Keep coding and stay curious!