Apache Kafka Connector Release Notes - Mule 4

Kafka Consumer and Message Listener failed after picking one message. (SE-21705)

Kafka Message Listener failed to process messages while using IMMEDIATE ACK. (SE-21070)

Kafka Connector was continuously rebalancing the loop when using multiple consumers per partition. (SE-20824)

Payload was null in the Batch component when using the Batch message listener source. (SE-20119)

Performance degraded when using the Publish operation. (SE-20124)

Kafka connector created a new connection every time a producer published a new message. (SE-20003)

Kafka Connector 4.1.0 did not allow you to pass ssl.endpoint.identification.algorithm with a blank value. (SE-16215)

Kafka Connector 4.0.0 had missing connection options. (SE-15175)

Added the SASL SCRAM connection type for connecting to a Kafka broker. This connection type works with the SCRAM-SHA-256 and SCARM-SHA-512 authentication mechanisms.

Added the Endpoint Identification Algorithm field to all connection types. Clients use this field to validate the server hostname.

Inability to set additional properties in Kafka Connector 4.0.0 (SE-15197)

Four new operations, including:

A new Batch Message Listener source

The addition of different acknowledgment modes in the Consumer configuration, including:

One plaintext connection type that supports transactions

Fixed a vulnerability related to the Guava library.

Kafka Connector versions 3.0.8 through 3.0.11 do not work with Mule 4.1.x, as previously stated by the documentation. (SE-17493)

Updated Kafka Client version to the latest to prevent internal exceptions. (SE-14104)

Fixed Kafka consumer disconnection that occurred after a period of inactivity. (SE-14221)

Upgraded the Kafka Clients library to version 2.3.0. (CONN-2857)

Implemented validation method for producer and consumer connection. (CONN-2698)

Added Scheduler Service for spawning new threads for consumers. (CONN-2849)

Refactoring to improve compliance with the Mule development best practices. (CONN-2699, CONN-2700, CONN-2701, CONN-2702, CONN-2848, CONN-2703, CONN-2847, CONN-2747, CONN-2696, CONN-2697)

Deprecated the UNKNOWN error type that will be removed in the next major upcoming release. (CONN-2703)

In Kafka versions 3.0.2 to 3.0.6, in the Publish Message operation, the Key field appeared in Anypoint Studio as required. In version 3.0.7, the Key field was no longer required.

In Kafka versions 3.0.2 to 3.0.6, in the Message Consumer source operation, the Partition Offsets field was described as "a list of offsets representing the partitions offsets configuration. For each element in the list you have to specify partition index and offset". In version 3.0.7, the Partition Offsets field is now described as "a list of objects that contain the index of the partition and the offset".

In Kafka versions 3.0.2 to 3.0.6, in the Message Consumer source operation, the Partition Offsets field is typed as an array of offsets. In 3.0.2 to 3.0.6 in the Offset array, the Partition Number and Partition Offset fields did not require values. In 3.0.7, the Partition Number and Partition Offset fields now require values.

In Kafka versions 3.0.2 to 3.0.6, in the Repeatable File Store Stream object, the Max in Memory Size field name was changed to be In Memory Size in 3.0.7.

Updated the Kafka Clients library to version 2.2.0.

Updated code to use the poll(java.time.Duration) method provided in the new library that replaces the deprecated poll(long) method. The deprecated method did not block beyond the timeout for waiting on a partition assignment. (KIP-266)

The Test Connection button in Anypoint Studio for the Kafka connector did not work in Mule 4. (SE-11817)

Fixed a performance connection validation issue relating to the same issue as in v3.0.3. (SE-11817)

Can’t stop Kafka consumer flow. See How To Stop Or Start Flows In Mule 4.x Programmatically. (SE-11337)

The Test Connection button in Anypoint Studio for the Kafka connector did not work in Mule 4. (SE-11817)