Contact Us 1-800-596-4880

DataWeave 2.4.0 Release Notes

November 2024

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

Apache POI folder used for temporal files is no longer deleted.

W-16899674

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

September 2024

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

DataWeave now preserves MimeType properties correctly.

W-16161736

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

July 2024

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

The Flat File module now adds the substituteCharacterAsMissingValue reader property.

W-15571377

The Flat File module now adds the fillRedefinesByMaxLength writer property.

W-15973636

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

June 2024

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

A java.io.IOException exception no longer occurs when writing the same binary input multiple times for JSON and XML formats.

W-14589732

The Excel writer now generates XLSX files using ZIP specification 2.0.

W-15534640

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

May 2024

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

Internal libraries are now upgraded.

W-15296551

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

April 2024

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

The Excel reader no longer reads furigana characters.

W-14991276

Coercion of the third parameter on the ternary function executor now works as expected.

W-15061008

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

February 2024

Patch 2.4.0-20240119-20240215

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

Performance is now improved when coercing addition and subtraction operations.

W-14927555

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

Patch 2.4.0-20240119

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

Caching now works correctly when coercing for addition and subtraction operations.

W-14591653

do scope declarations with pattern matching now properly reflect variable names when variables are optimized with common subexpression elimination.

W-14514837

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

November 2023

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

The cached function coercion now works correctly for a second time when an argument is constant.

W-13763138

The DataWeave sum function with a single numeric string arg now returns Number type instead of String.

W-14086694

Reading Avro schema with an array of enum type items no longer returns an error.

W-14252745

Inconsistent XML encoding detection no longer occurs between indexed and in-memory parsers.

W-13805995

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

October 2023

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

System property com.mulesoft.dw.xml_writer.declareXmlNamespace has been included to manage adding implicit namespaces like xml in the XML writer.

W-11965399

Temporary files are now closed for inputs bigger than 1.5 MB.

W-13868202

Milliseconds information is now preserved when converting the java.util.Date type.

W-14002706

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

September 2023

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

An invalid UUID logical type in Avro module now throws an exception as expected.

W-13943112

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

July 2023

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

Multipart reader performance is enhanced by using the proper input stream implementation.

W-13099773

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

June 2023

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

Streaming support now works correctly for JSON.

W-11790958

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

May 2023

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

Closing resources immediately after reading when using text/plain MIME type.

W-12398584

Incorrect trimming of whitespaces no longer occurs when reading XML content.

W-12528980

Fixed splitBy management of large strings so String and Regex options are consistent with each other.

W-12452407

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

April 2023

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

Added cache Java values evaluation to avoid consuming streams more than once.

W-12635909

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

March 2023

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

Fixed IndexOutOfBoundsException that occurred when combining the usage of heap memory for internal buffering with a customized buffer size.

W-12247447

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

February 2023

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

dw::core::Dates::atBeginningOfWeek now works correctly when the input DateTime is Sunday.

W-11940166

The new com.mulesoft.dw.javaSqlDateToDate system property enables mapping java.sql.Date to Date instead of DateTime.

W-12043162

The Resource already closed exception no longer occurs when using the write function with a value bigger than 1.5MB with Avro format.

W-12067821

Cacheable overloaded functions, such as then, now dispatch to the correct function definition when called multiple times.

W-12073562

The default behavior now works correctly when having a null value coming from the Mule runtime.

W-12166609

High memory usage no longer occurs when using the dw::core::Arrays::divideBy function with an in-memory Array as input.

W-12167313

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

January 2023

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

The function dw::core::Dates::atBeginningOfHour now works as expected when an hour is represented as a single digit.

W-11620156

Escaped quote characters now work as expected in CSV Module.

W-12128349

The FlatFile module now provides a reader property, retainEmptyStringFieldsOnParsing, for handling fields that lack values. When set to true, this property makes the reader retain these fields and set their values to an empty string. The default behavior of the reader is to remove those fields.

W-12036336

Data payloads without line breaks no longer work when using lenient as the recordParsing property. The other options to recordParsing support records that lack line breaks. noTerminator is preferred for fixed-length records that lack a line break.

W-12472890

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

November 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Issue Resolution ID

onNull functions now work as expected.

W-11865849

Instead of returning an empty array in some scenarios, iterating flat file arrays now work as expected.

W-11932440

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

September 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The Mule patch release addresses the following DataWeave issues:

Description Issue

Visual Studio Code extension no longer breaks with binary input format in the test scenario inputs directory.

W-11507190

Namespace is now added on elements when previously used at a deeper structure level.

W-11349764

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

August 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The August Mule patch release addresses the following DataWeave issues:

Description Issue

The message for the StackOverflow script compilation error is no longer hidden.

W-11215958

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

July 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The July Mule patch release addresses the following DataWeave issues:

Description Issue

Writing binary values with the JSON writer no longer outputs the base of the values.

W-11326859

This release fixes Avro property handling.

W-11324418

This release improves handling of streams that do not declare a size.

W-11277575

Fix write function so Avro format can be handled.

W-11047202

This release improves handling of string values in XML format.

W-11273037

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

June 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The June Mule patch release addresses the following DataWeave issues:

Description Issue

DataWeave now parses Java objects without errors.

W-10832766

The text/plain reader now correctly ignores the byte order mark (BOM). The JSON writer now uses the encoding information available in Binary values when writing them as String values instead of using the writer configuration encoding. To restore the previous behavior, enable the system property com.mulesoft.dw.decode_binaries_with_writer_encoding.

W-10696255

Multiline string indentation on YAML now works correctly.

W-10955882

The FlatFile module now includes the allowLenientWithBinaryNotEndElement property to support existing behavior.

W-10935528

Fixed ClassCastException in multipart module.

W-10943884

DataWeave now shows a descriptive error when there is an unsupported output mime type.

W-10876534

The indexed XML reader no longer adds unnecessary text fields to XML objects.

W-10942171

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

March 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The March Mule patch release addresses the following DataWeave issues:

Description Issue

A direct memory error no longer occurs when the com.mulesoft.dw.directbuffer.disable system property is enabled.

SE-24186

Duplicate declarations for namespace prefixes in XML no longer occur.

W-10648792 / DW-1081

Exceptions are no longer thrown due to use of the default operator with fromBase64.

W-10648825 / SE-23812

Avro unions with enums are now supported.

W-10706991

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

February 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The February Mule patch release addresses the following DataWeave issues:

Description Issue

An endless loop issue no longer occurs when writing a CSV file with bufferSize=0.

DW-1074

Creating sample files using the binary type no longer causes an error.

DW-390

Corrected the documentation of the`Periods::between` function.

DW-1070

Upgraded some dependencies, including avro and jackson, and removed unused AWS dependencies (#1673).

DW-1080

Upgraded the edi-parser with SLF4J.

CONN-9598

Performing transformations from avro to other formats no longer causes an exception when the enum type field is present.

SE-23802

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

January 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The January Mule patch release addresses the following DataWeave issues:

Description Issue

Flow retyping inside dynamic functions now works as expected.

DW-677

DataWeave no longer incorrectly sets autostreaming.

DW-918

The UnsupportedTypeCoercionToLiteralException error is now caught when using the similar operator (~=).

DW-932

JSON streaming now works correctly.

DW-940

The AttributeDelegate function no longer causes a stack overflow error.

SE-23494

Removing objects by property name now works as expected.

SE-23444

When using arrays, the XML writer properly keeps namespaces.

SE-23299

DataWeave now correctly parses large strings in JSON transformations.

SE-23291

A Scatter Gather issue that caused DataWeave to throw the ArrayIndexOutOfBoundsException error no longer occurs.

SE-22431

Comparison operators for dateTime type now work as intended.

SE-22025

Local variable resolution now works as expected.

SE-21847

Avoid closing resources prematurely when using the write function, which affects MUnit tests.

SE-23483

Fix OutOfMemory error using writeLinesWith function, which affected the ndjson module.

SE-23759

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

November 2021

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The November Mule patch release addresses the following DataWeave issues:

Description Issue

Fixed a ClassCastException that occurred in the Avro module by correcting its generated MIME type.

SE-21869

Comments within YAML files processed in Mule applications are no longer erased.

DW-706

You can now disable the removal of self-referencing implicit variables.

DW-893

DataWeave now allows mixed content structures (text and nodes) when reading and writing XML content.

SE-21464

Functions in the Dates module now accept one-digit numbers for months and days.

DW-894

Improved the performance of splitting functionality.

DW-776

Improved the performance of the JSON module.

DW-706

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

October 5, 2021

DataWeave is the programming language designed by MuleSoft for data transformation. It is also the expression language Mule runtime engine uses to configure components and connectors.

For more detail, see DataWeave Language.

What’s New

The 2.4.0 version of DataWeave introduces new features and enhancements.

Release notes for previous versions of DataWeave are available within Mule Runtime Release Notes.

New Features

  • Ability to read larger-than-memory strings automatically. When you are using the indexed reader strategy and processing a string with a size larger than 1.5 MB, DataWeave automatically splits the value in chunks to avoid out-of-memory issues. Related reader properties include com.mulesoft.dw.max_memory_allocation and com.mulesoft.dw.buffered_char_sequence.enabled. Refer to Indexed Readers in DataWeave.

  • New reader and writer properties for working with data formats. For example, the XML writer now enables you to define namespaces at the root level.

  • Many new modules, functions, types, annotations, and variables. Some are experimental and subject to change in future releases.

For additional detail, see What’s New in DataWeave

Enhancements

A number of DataWeave functions are overloaded to introduce new helper functions for handling null values, including:

  • Core:

    • entriesOf

    • namesOf

    • keysOf

    • valuesOf

    • replace

    • --

    • contains

    • sizeOf

    • endsWith

    • joinBy

    • scan

    • splitBy

    • startsWith

    • matches

    • match

  • Arrays:

    • countBy

    • sumBy

    • divideBy

    • take

    • drop

    • slice

    • indexWhere

    • splitAt

    • takeWhile

    • dropWhile

    • splitWhere

    • partition

    • firstWith

  • Strings:

    • charCode

    • charCodeAt

    • fromCharCode

    • repeat

The default operator no longer captures null function calls because the functions themselves now manage the null value. Uses of these functions with the default operator might behave unexpectedly for this reason.

Compatibility

The following table specifies which version of DataWeave is bundled with each Mule runtime engine release:

Mule Version DataWeave Version

4.8

2.8

4.7

2.7

4.6

2.6

4.5

2.5

4.4

2.4

4.3

2.3

4.2

2.2

4.1

2.1

3.9

1.2

3.8

1.1

3.7

1.0

Fixed Issues

The following table lists bug fixes that are specific to the initial release of DataWeave 2.4.0. This release also incorporates all fixes to DataWeave, including patch updates through the September 2021 release, that are listed in Mule Runtime 4.3.0 Release Notes.

Description Issue

Added the writeDeclaredNamespaces XML writer property to write all declared namespaces in the root element.

DW-16

Added the writeDeclaration YAML writer property to avoid writing the header declaration.

DW-162

Added a new Strings module for easier string data management.

DW-406

Added the constant folding functionality so that DataWeave replaces operations involving constant values with the actual result. For example, 1024*8 is replaced at compile time with the resulting value 8192.

DW-427

Added flow retyping to the type system, which enables better accuracy within flow-control structures featuring checks that involve typing.

DW-438

Added code inspectors so that quick fixes are suggested with best practices, such as replacing null checks with a default expression, or replacing sizeOf() == 0 with isEmpty().

DW-33/DW-632

Added an inspection to replace using with do blocks.

DW-771

Improved error messages when creating Java objects.

DW-565

Improved performance on floating point number parsing by identifying in advance if the number has a floating point or not.

SE-20614

Extended the common subexpression elimination process to work with index selectors.

SE-19934

Added functions that work with null values to the Core, Arrays, and String modules.

DW-425

Improved JSON performance by copying bytes from input to output directly during JSON-to-JSON data transformations.

DW-230

Improved YAML support of scalar types, string quotations, custom tags, and comment retention.

DW-687

Added the filterTree function to the Tree module to be able to filter any data structure.

DW-160

Added validations for index tokens in CSV format.

DW-577

Added support for the Java classes OptionalInt, OptionalDouble and OptionalLong.

DW-628

Added support for file:/example/file.txt as a valid URL.

DW-195

Added warning when using the design-time only format application/dw without dataOnly mode.

DW-873

Added experimental run and eval functions to dynamically execute scripts.

DW-170

Added the concept of optional function parameters in the type system.

DW-874

Improved CSV streaming to avoid unnecessary processing.

DW-876

Improved the mapObject function to avoid unnecessary processing.

DW-535

Improved the flatten function to avoid unnecessary processing.

DW-509

Improved code validation. Now an error returns when a variable is using the same name as an input.

DW-522

Improved the precision of math operations and added a property to retain the previous behavior: com.mulesoft.dw.math.exact_precision.

SE-16452

Improved handling of big XML files to fix potential I/O exceptions.

DW-250

Fixed error locations when scripts feature dynamic string selectors.

DW-875

Fixed a false negative error message when using multiple union types.

DW-297

The write function return types are now consistent with input types.

DW-298

The update operator no longer fails when working with repeated keys and attributes.

DW-306

Processing of XML documents with comments and streaming enabled now works as expected.

DW-307

DataWeave no longer fails to parse a single key-value pair with a conditional and parentheses.

DW-309

The type system now validates overloaded functions as parameters.

DW-316

Parallel processing on ndjson now works as expected.

DW-325

Fixed a materialization and streaming failure.

DW-368

Fixed the behavior of the slice function.

DW-409/DW-410

Fixed the ability to import namespaces from external modules.

DW-414

DataWeave now produces the error Division by Zero when dividing by 0.0 instead of an internal exception.

DW-415

DataWeave now properly validates that do blocks do not contain any reserved word declaration.

DW-416

Running flatten on flatten no longer causes loss of values.

DW-691

The update operator now works when no attribute is defined.

DW-216

Improved handling of binary data to prevent running out of memory when calculating sizes of processed data.

SE-16559

The com.mulesoft.dw.coercionexception.verbose property no longer logs data from coercion exceptions.

SE-15487

Using nested binary functions no longer produces a StackOverflow error.

SE-16092

DataWeave no longer returns UnsupportedOperationException when parsing an invalid script.

DW-294

Fixed a potential IndexOutOfBoundException when using overloaded functions with four parameters.

DW-159

Added support for dumping Flat File and Avro schemas when com.mulesoft.dw.dump_files is enabled.

DW-137

Improved code compilation. Implicit inputs are not added if a variable with the same name is explicitly defined.

DW-630

Upgrade

For upgrades to the 2.4.0 version of DataWeave, which is bundled within Mule runtime engine, see Upgrading Mule Runtime Engine (Versions 4.x to 4.n).