Connecting Kafka-Python with a cluster with Kerberos

If the task is to solve this problem in python, another alternative could be to use confluent-kafka-python library that internally uses librdkafka that is written in C, and supports SASL, and the use of the keytab file. That wouldn't require having a separate Java process for the communication with kafka over SASL.

For instructions also refer to the documentation of the librdkafka library:

https://github.com/edenhill/librdkafka/wiki/Using-SASL-with-librdkafka - general intro https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md - the properties that can be passed into the constructor of confluent_kafka.Producer and confluent_kafka.Consumer in python

Well Guys,

I found the issue.

The problem is that Kerberos is not supported for Kafka producer in Python using Key Tab.

To use Key Tab we need to set a java Environment Variable.

According Hortonworks we need to set the client_jaas_client to connect.

The solution was using Py4j to call the Kafka Producer in JVM.

See the example here.