Kafka Producer - org.apache.kafka.common.serialization.StringSerializer could not be found

Its issue with the version you are using. It was also suggested to version 0.8.2.2_1. Suggest you to adjust the version of kafka you are using and give a try. code wise, I cross checked many code samples in kafka dev list and seems like you have written in right way.

i.e Thread.currentThread().setContextClassLoader(null);


I find the reason by reading the kafka client source code.

kafka client use Class.forName(trimmed, true, Utils.getContextOrKafkaClassLoader()) to get the Class object, and the create the instance, the key point is the classLoader, which is specified by the last param, the implementation of method Utils.getContextOrKafkaClassLoader() is

public static ClassLoader getContextOrKafkaClassLoader() {
    ClassLoader cl = Thread.currentThread().getContextClassLoader();
    if (cl == null)
        return getKafkaClassLoader();
    else
        return cl;
}

so, by default, the Class object of org.apache.kafka.common.serialization.StringSerializer is load by the applicationClassLoader, if your target class is not loaded by the applicationClassLoader, this problem will happend !

to solve the problem, simply set the ContextClassLoader of current thread to null before new KafkaProducer instance like this

Thread.currentThread().setContextClassLoader(null);
Producer<String, String> producer = new KafkaProducer(props);

hope my answer can let you know what happend .


The issue appears to be with the class loader, as @Ram Ghadiyaram indicated in his answer. In order to get this working with kafka-clients 2.x, I had to do the following:

public Producer<String, String> createProducer() {
            ClassLoader original = Thread.currentThread().getContextClassLoader();
    Thread.currentThread().setContextClassLoader(null);
    Properties props = new Properties();

    props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,
        BOOTSTRAP_SERVERS);
    ... etc ...

    KafkaProducer<String, String> producer = new KafkaProducer<>(props);
    Thread.currentThread().setContextClassLoader(original);
    return producer;

}

This allows the system to continue loading additional classes with the original classloader. This was needed in Wildfly/JBoss (the specific app I'm working with is Keycloak).