How to connect Kafka with Elasticsearch?

I would suggest using Kafka Connect and its Elasticsearch sink. I actually presented on exactly this subject last night :) Here are the slides.

You can see a detailed example here.

Update May 2020: See also this tutorial video.


The empty input block for Kafka will not work, since it doesn't know where to get the data from. The minimal config for your scenario will be something like this:

input {
    kafka {
            bootstrap_servers => "localhost:9092"
            topics => "test"
    }
}