+

DataWeave 2.4.0 Release Notes

August 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The August Mule patch release addresses the following DataWeave issues:

Description Issue

The message for the StackOverflow script compilation error is no longer hidden.

W-11215958

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

July 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The July Mule patch release addresses the following DataWeave issues:

Description Issue

Writing binary values with the JSON writer no longer outputs the base of the values.

W-11326859

This release fixes Avro property handling.

W-11324418

This release improves handling of streams that do not declare a size.

W-11277575

Fix write function so Avro format can be handled.

W-11047202

This release improves handling of string values in XML format.

W-11273037

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

June 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The June Mule patch release addresses the following DataWeave issues:

Description Issue

DataWeave now parses Java objects without errors.

W-10832766

The text/plain reader now correctly ignores the byte order mark (BOM). The JSON writer now uses the encoding information available in Binary values when writing them as String values instead of using the writer configuration encoding. To restore the previous behavior, enable the system property com.mulesoft.dw.decode_binaries_with_writer_encoding.

W-10696255

Multiline string indentation on YAML now works correctly.

W-10955882

The FlatFile module now includes the allowLenientWithBinaryNotEndElement property to support existing behavior.

W-10935528

Fixed ClassCastException in multipart module.

W-10943884

DataWeave now shows a descriptive error when there is an unsupported output mime type.

W-10876534

The indexed XML reader no longer adds unnecessary text fields to XML objects.

W-10942171

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

March 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The March Mule patch release addresses the following DataWeave issues:

Description Issue

A direct memory error no longer occurs when the com.mulesoft.dw.directbuffer.disable system property is enabled.

SE-24186

Duplicate declarations for namespace prefixes in XML no longer occur.

W-10648792 / DW-1081

Exceptions are no longer thrown due to use of the default operator with fromBase64.

W-10648825 / SE-23812

Avro unions with enums are now supported.

W-10706991

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

February 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The February Mule patch release addresses the following DataWeave issues:

Description Issue

An endless loop issue no longer occurs when writing a CSV file with bufferSize=0.

DW-1074

Creating sample files using the binary type no longer causes an error.

DW-390

Corrected the documentation of the`Periods::between` function.

DW-1070

Upgraded some dependencies, including avro and jackson, and removed unused AWS dependencies (#1673).

DW-1080

Upgraded the edi-parser with SLF4J.

CONN-9598

Performing transformations from avro to other formats no longer causes an exception when the enum type field is present.

SE-23802

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

January 2022

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The January Mule patch release addresses the following DataWeave issues:

Description Issue

Flow retyping inside dynamic functions now works as expected.

DW-677

DataWeave no longer incorrectly sets autostreaming.

DW-918

The UnsupportedTypeCoercionToLiteralException error is now caught when using the similar operator (~=).

DW-932

JSON streaming now works correctly.

DW-940

The AttributeDelegate function no longer causes a stack overflow error.

SE-23494

Removing objects by property name now works as expected.

SE-23444

When using arrays, the XML writer properly keeps namespaces.

SE-23299

DataWeave now correctly parses large strings in JSON transformations.

SE-23291

A Scatter Gather issue that caused DataWeave to throw the ArrayIndexOutOfBoundsException error no longer occurs.

SE-22431

Comparison operators for dateTime type now work as intended.

SE-22025

Local variable resolution now works as expected.

SE-21847

Avoid closing resources prematurely when using the write function, which affects MUnit tests.

SE-23483

Fix OutOfMemory error using writeLinesWith function, which affected the ndjson module.

SE-23759

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

November 2021

DataWeave 2.4.0 is bundled with the Mule 4.4.0 release. The November Mule patch release addresses the following DataWeave issues:

Description Issue

Fixed a ClassCastException that occurred in the Avro module by correcting its generated MIME type.

SE-21869

Comments within YAML files processed in Mule applications are no longer erased.

DW-706

You can now disable the removal of self-referencing implicit variables.

DW-893

DataWeave now allows mixed content structures (text and nodes) when reading and writing XML content.

SE-21464

Functions in the Dates module now accept one-digit numbers for months and days.

DW-894

Improved the performance of splitting functionality.

DW-776

Improved the performance of the JSON module.

DW-706

For details about the Mule release and patch updates, see Mule Runtime Engine 4.4.0 Release Notes.

October 5, 2021

DataWeave is the programming language designed by MuleSoft for data transformation. It is also the expression language Mule runtime engine uses to configure components and connectors.

For more detail, see DataWeave Language.

What’s New

The 2.4.0 version of DataWeave introduces new features and enhancements.

Release notes for previous versions of DataWeave are available within Mule Runtime Release Notes.

New Features

  • Ability to read larger-than-memory strings automatically. When you are using the indexed reader strategy and processing a string with a size larger than 1.5 MB, DataWeave automatically splits the value in chunks to avoid out-of-memory issues. Related reader properties include com.mulesoft.dw.max_memory_allocation and com.mulesoft.dw.buffered_char_sequence.enabled. Refer to Indexed Readers in DataWeave.

  • New reader and writer properties for working with data formats. For example, the XML writer now enables you to define namespaces at the root level.

  • Many new modules, functions, types, annotations, and variables. Some are experimental and subject to change in future releases.

For additional detail, see What’s New in DataWeave

Enhancements

A number of DataWeave functions are overloaded to introduce new helper functions for handling null values.

Compatibility

The following table specifies which version of DataWeave is bundled with each Mule runtime engine release:

Mule Version DataWeave Version

4.4

2.4

4.3

2.3

4.2

2.2

4.1

2.1

3.9

1.2

3.8

1.1

3.7

1.0

Fixed Issues

The following table lists bug fixes that are specific to the initial release of DataWeave 2.4.0. This release also incorporates all fixes to DataWeave, including patch updates through the September 2021 release, that are listed in Mule Runtime 4.3.0 Release Notes.

Description Issue

Added the writeDeclaredNamespaces XML writer property to write all declared namespaces in the root element.

DW-16

Added the writeDeclaration YAML writer property to avoid writing the header declaration.

DW-162

Added a new Strings module for easier string data management.

DW-406

Added the constant folding functionality so that DataWeave replaces operations involving constant values with the actual result. For example, 1024*8 is replaced at compile time with the resulting value 8192.

DW-427

Added flow retyping to the type system, which enables better accuracy within flow-control structures featuring checks that involve typing.

DW-438

Added code inspectors so that quick fixes are suggested with best practices, such as replacing null checks with a default expression, or replacing sizeOf() == 0 with isEmpty().

DW-33/DW-632

Added an inspection to replace using with do blocks.

DW-771

Improved error messages when creating Java objects.

DW-565

Improved performance on floating point number parsing by identifying in advance if the number has a floating point or not.

SE-20614

Extended the common subexpression elimination process to work with index selectors.

SE-19934

Added functions that work with null values to the Core, Arrays, and String modules.

DW-425

Improved JSON performance by copying bytes from input to output directly during JSON-to-JSON data transformations.

DW-230

Improved YAML support of scalar types, string quotations, custom tags, and comment retention.

DW-687

Added the filterTree function to the Tree module to be able to filter any data structure.

DW-160

Added validations for index tokens in CSV format.

DW-577

Added support for the Java classes OptionalInt, OptionalDouble and OptionalLong.

DW-628

Added support for file:/example/file.txt as a valid URL.

DW-195

Added warning when using the design-time only format application/dw without dataOnly mode.

DW-873

Added experimental run and eval functions to dynamically execute scripts.

DW-170

Added the concept of optional function parameters in the type system.

DW-874

Improved CSV streaming to avoid unnecessary processing.

DW-876

Improved the mapObject function to avoid unnecessary processing.

DW-535

Improved the flatten function to avoid unnecessary processing.

DW-509

Improved code validation. Now an error returns when a variable is using the same name as an input.

DW-522

Improved the precision of math operations and added a property to retain the previous behavior: com.mulesoft.dw.math.exact_precision.

SE-16452

Improved handling of big XML files to fix potential I/O exceptions.

DW-250

Fixed error locations when scripts feature dynamic string selectors.

DW-875

Fixed a false negative error message when using multiple union types.

DW-297

The write function return types are now consistent with input types.

DW-298

The update operator no longer fails when working with repeated keys and attributes.

DW-306

Processing of XML documents with comments and streaming enabled now works as expected.

DW-307

DataWeave no longer fails to parse a single key-value pair with a conditional and parentheses.

DW-309

The type system now validates overloaded functions as parameters.

DW-316

Parallel processing on ndjson now works as expected.

DW-325

Fixed a materialization and streaming failure.

DW-368

Fixed the behavior of the slice function.

DW-409/DW-410

Fixed the ability to import namespaces from external modules.

DW-414

DataWeave now produces the error Division by Zero when dividing by 0.0 instead of an internal exception.

DW-415

DataWeave now properly validates that do blocks do not contain any reserved word declaration.

DW-416

Running flatten on flatten no longer causes loss of values.

DW-691

The update operator now works when no attribute is defined.

DW-216

Improved handling of binary data to prevent running out of memory when calculating sizes of processed data.

SE-16559

The com.mulesoft.dw.coercionexception.verbose property no longer logs data from coercion exceptions.

SE-15487

Using nested binary functions no longer produces a StackOverflow error.

SE-16092

DataWeave no longer returns UnsupportedOperationException when parsing an invalid script.

DW-294

Fixed a potential IndexOutOfBoundException when using overloaded functions with four parameters.

DW-159

Added support for dumping Flat File and Avro schemas when com.mulesoft.dw.dump_files is enabled.

DW-137

Improved code compilation. Implicit inputs are not added if a variable with the same name is explicitly defined.

DW-630

Upgrade

For upgrades to the 2.4.0 version of DataWeave, which is bundled within Mule runtime engine, see Upgrading Mule Runtime Engine (Versions 4.x to 4.n).

Was this article helpful?

💙 Thanks for your feedback!