Apache Kafka Connector Release Notes - Mule 4

Support Category: Select

Anypoint Connector for Apache Kafka (Kafka Connector) enables you to connect to the Apache Kafka messaging system, enabling seamless integration between your applications and an Apache Kafka cluster.

4.6.0

October 19, 2021

What’s New

  • Added the Consumer SASL/TOKEN Connection and Producer SASL/TOKEN Connection authentication types, which use token delegation to connect to the Kafka cluster.

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0, and 2.7.0

OpenJDK

8 and 11

4.5.7

September 28, 2021

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0, and 2.7.0

OpenJDK

8 and 11

Fixed Issues

Issue ID

Kafka Connector reconnection did not work when there was an ACL access issue.

SE-22595

Kafka Message Listener did not handle duplicate headers.

SE-22520

4.5.6

September 15, 2021

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0, and 2.7.0

OpenJDK

8 and 11

Fixed Issues

Issue ID

The reconnection strategy did not work.

SE-22340

4.5.5

July 12, 2021

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0, and 2.7.0

OpenJDK

8 and 11

Fixed Issues

  • Kafka Consumer and Message Listener failed after picking one message. (SE-21705)

4.5.4

June 11, 2021

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0, and 2.7.0

OpenJDK

8 and 11

Fixed Issues

  • Kafka Message Listener failed to process messages while using IMMEDIATE ACK. (SE-21070)

4.5.3

June 8, 2021

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0, and 2.7.0

OpenJDK

8 and 11

Fixed Issues

  • Kafka Connector was continuously rebalancing the loop when using multiple consumers per partition. (SE-20824)

4.5.2

April 8, 2021

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0, and 2.7.0

OpenJDK

8 and 11

Fixed Issues

  • Payload was null in the Batch component when using the Batch message listener source. (SE-20119)

4.5.1

March 19, 2021

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0, and 2.7.0

OpenJDK

8 and 11

Fixed Issues

  • Performance degraded when using the Publish operation. (SE-20124)

  • Kafka connector created a new connection every time a producer published a new message. (SE-20003)

4.5.0

February 25, 2021

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0, and 2.7.0

OpenJDK

8 and 11

Features

Added a new configuration property to Producer connections to enable the selection of a message partition strategy from those provided by the kafka-clients library. The message partition strategy is used for routing messages to partitions when a key or a partition is not provided.

Fixed Issues

Fixed the performance degradation issue that occurred when an incorrect key was used to remove closed sessions. (SE-19638)

4.4.3

January 29, 2021

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0, and 2.7.0

OpenJDK

8 and 11

Fixed Issues

Fixed the Anypoint Studio test connection functionality for producer and consumer connections.

4.4.2

January 15, 2021

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0, and 2.7.0

OpenJDK

8 and 11

Features

Upgraded the Kafka client library to version 2.7.0.

4.4.1

January 12, 2021

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0

OpenJDK

8 and 11

Fixed Issues

On Windows, using backslashes as path separators when providing a full path for a keytab file did not work. (SE-15416)

4.4.0

November 26, 2020

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0

OpenJDK

8 and 11

New Features

Added the ability to leverage the manual offset commit option for the Consume operation.

Fixed Issues

The manual connector commit inside Mule error handling had no effect. (SE-18605)

4.3.5

November 2, 2020

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0

OpenJDK

8 and 11

Fixed Issues

Apache Kafka Connector ignored the values of the Heartbeat interval and Session timeout fields in the consumer configuration. (SE-18343)

4.3.4

October 15, 2020

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0

OpenJDK

8 and 11

Fixed Issues

The issue that caused the Kafka Connector source to fail to reconnect after multiple queue stop events is fixed. (SE-17727)

4.3.3

September 18, 2020

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0

OpenJDK

8 and 11

Fixed Issues

The issue that caused Kafka Connector to throw a KAFKA:COMMIT_FAILED error when messages took a longer time to process than what was specified in the Maximum poll interval parameter is fixed. (SE-17447)

4.3.2

July 30, 2020

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0

Fixed Issues

The issue that caused Kafka Connector to throw a No ConnectionProvider defined error when the connection expires while the client consumer is polling is fixed. (SE-16727)

4.3.1

July 14, 2020

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0

Fixed Issues

Fixed an issue where the value for the password key of the JAAS configuration was not set. (SE-16670)

4.3.0

June 30, 2020

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0

New Features

Added support for a SASL/PLAIN connection.

4.2.0

June 19, 2020

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0, 2.5.0

New Features

Added support for a Kerberos connection.

Fixed Issues

  • Kafka Connector 4.1.0 did not allow you to pass ssl.endpoint.identification.algorithm with a blank value. (SE-16215)

  • Kafka Connector 4.0.0 had missing connection options. (SE-15175)

4.1.0

May 28, 2020

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0

New Features

  • Added the SASL SCRAM connection type for connecting to a Kafka broker. This connection type works with the SCRAM-SHA-256 and SCARM-SHA-512 authentication mechanisms.

  • Added the Endpoint Identification Algorithm field to all connection types. Clients use this field to validate the server hostname.

Fixed Issues

  • Inability to set additional properties in Kafka Connector 4.0.0 (SE-15197)

4.0.0

February 25, 2020

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0

New Features

Kafka Connector 4.0.0 contains a lot of changes to improve efficiency, including new operations to enable you to better understand how to use the connector.

All of the changes included in this version break backward compatibility.

See Kafka Connector Upgrade for detailed information about all of the changes in this release.

The new features and changes in this release include:

  • Four new operations, including:

    • Commit

    • Consume

    • Publish

    • Seek

  • A new Batch Message Listener source

  • The addition of different acknowledgment modes in the Consumer configuration, including:

    • AUTO

    • MANUAL

    • DUPS_OK

    • IMMEDIATE

  • One plaintext connection type that supports transactions

3.0.12

September 9, 2020

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.4.0

Fixed Issues

  • Fixed a vulnerability related to the Guava library.

  • Kafka Connector versions 3.0.8 through 3.0.11 do not work with Mule 4.1.x, as previously stated by the documentation. (SE-17493)

3.0.11 (Deprecated)

June 12, 2020

Compatibility

Software Version

Mule

4.2.1 and later

Apache Kafka

2.4.0

Fixed Issues

Changed the Scheduler type from cpuLight to IO. (SE-16328)

3.0.10 (Deprecated)

December 17, 2019

Compatibility

Software Version

Mule

4.2.1 and later

Apache Kafka

2.4.0

Fixed Issues

  • Updated Kafka Client version to the latest to prevent internal exceptions. (SE-14104)

  • Fixed Kafka consumer disconnection that occurred after a period of inactivity. (SE-14221)

3.0.9 (Deprecated)

December 16, 2019

Compatibility

Software Version

Mule

4.2.1 and later

Apache Kafka

2.3.0

Fixed Issues

Fixed relative path issue for Kafka connector settings with Windows. (SE-14140)

3.0.8 (Deprecated)

November 25, 2019

Compatibility

Software Version

Mule

4.2.1 and later

Apache Kafka

2.3.0

Fixed Issues

When KafkaConsumer was stopped by the Mule runtime engine, Kafka threw a concurrency error. (SE-13970)

3.0.7

August 21, 2019

Compatibility

Software Version

Mule

4.1.1 and later

Apache Kafka

2.3.0

Features

  • Upgraded the Kafka Clients library to version 2.3.0. (CONN-2857)

  • Implemented validation method for producer and consumer connection. (CONN-2698)

  • Added Scheduler Service for spawning new threads for consumers. (CONN-2849)

Fixed Issues

  • Refactoring to improve compliance with the Mule development best practices. (CONN-2699, CONN-2700, CONN-2701, CONN-2702, CONN-2848, CONN-2703, CONN-2847, CONN-2747, CONN-2696, CONN-2697)

  • Deprecated the UNKNOWN error type that will be removed in the next major upcoming release. (CONN-2703)

  • In Kafka versions 3.0.2 to 3.0.6, in the Publish Message operation, the Key field appeared in Anypoint Studio as required. In version 3.0.7, the Key field was no longer required.

  • In Kafka versions 3.0.2 to 3.0.6, in the Message Consumer source operation, the Partition Offsets field was described as "a list of offsets representing the partitions offsets configuration. For each element in the list you have to specify partition index and offset". In version 3.0.7, the Partition Offsets field is now described as "a list of objects that contain the index of the partition and the offset".

  • In Kafka versions 3.0.2 to 3.0.6, in the Message Consumer source operation, the Partition Offsets field is typed as an array of offsets. In 3.0.2 to 3.0.6 in the Offset array, the Partition Number and Partition Offset fields did not require values. In 3.0.7, the Partition Number and Partition Offset fields now require values.

  • In Kafka versions 3.0.2 to 3.0.6, in the Repeatable File Store Stream object, the Max in Memory Size field name was changed to be In Memory Size in 3.0.7.

3.0.6

July 2, 2019

Compatibility

Software Version

Mule

4.0.0 and later

Apache Kafka

2.2.0

Fixed Issues

Kafka connector overrode the user-defined security.protocol=SASL_SSL to use the security.protocol=SSL value. (SE-11947)

3.0.5

June 19, 2019

Compatibility

Software Version

Mule

4.0.0 and later

Apache Kafka

2.2.0

Fixed Issues

  • Updated the Kafka Clients library to version 2.2.0.

  • Updated code to use the poll(java.time.Duration) method provided in the new library that replaces the deprecated poll(long) method. The deprecated method did not block beyond the timeout for waiting on a partition assignment. (KIP-266)

3.0.4

May 16, 2019

Compatibility

Software Version

Mule

4.0.0 and later

Apache Kafka

0.10.2.0

Fixed Issues

Fixed a performance connection validation issue relating to the same issue as in v3.0.3. (SE-11817)

3.0.3

May 16, 2019

Compatibility

Software Version

Mule

4.0.0 and later

Apache Kafka

0.10.2.0

Fixed Issues

  • The Test Connection button in Anypoint Studio for the Kafka connector did not work in Mule 4. (SE-11817)

  • Fixed a performance connection validation issue relating to the same issue as in v3.0.3. (SE-11817)

3.0.2

April 30, 2019

Compatibility

Software Version

Mule

4.0.0 and later

Apache Kafka

0.10.2.0

Fixed Issues

3.0.1

June 26, 2018

Compatibility

Software Version

Mule

4.0.0 and later

Apache Kafka

0.10.2.0

Features

None

Fixed Issues

Kafka connector was not serializing and deserializing properly the messages that were not plain text. Now the content of the messages can be sent as any type (plain, XML, JSON) as the value is serialized and deserialized properly.

Was this article helpful?

💙 Thanks for your feedback!