Contact Free trial Login

Supported Data Formats

DataWeave can read and write many types of data formats, such as JSON, XML, and many others. Before you begin, note that DataWeave 2.0 is for Mule 4 apps. For a Mule 3 app, refer to the DataWeave 1.0 documentation set in the Mule 3.9 documentation. For other Mule versions, you can use the version selector for the Mule Runtime table of contents.

DataWeave supports the following formats as input and output:

MIME Type ID Supported Formats

application/avro

avro

Avro Format

application/csv

csv

CSV Format

application/dw

dw

DataWeave Format (dw) for testing a DataWeave expression

application/flatfile

flatfile

Flat File Format, COBOL Copybook Format, Fixed Width Format

application/java

java

Java Format, Enum Format for Java

application/json

json

JSON Format

application/octet-stream

binary

Binary Format

application/xlsx

excel

Excel Format

application/xml

xml

XML Format, CData Custom Type

application/x-ndjson

ndjson

Newline Delimited JSON Format (ndjson)

application/x-www-form-urlencoded

urlencoded

URL Encoded Format

application/yaml

yaml

YAML Format

multipart/form-data

multipart

Multipart Format

text/plain

text

Text Plain Format

text/x-java-properties

properties

Text Java Properties

DataWeave Readers

DataWeave can read input data as a whole in-memory, in indexed fashion, and for some data formats, part-by-part by streaming the input. When attempting to read a large file, it is possible to run out of memory or to impact performance negatively. Streaming can improve performance but impacts access to file.

  • Indexed and In-Memory:
    Allow for random access to data because both strategies parse the entire document. When using these strategies, a DataWeave script can access any part of the resulting value at any time.

    • Indexed:
      Uses indexes over the disk.

      DataWeave can only read the following formats in the indexed mode:

      CSV, JSON, XML (starting in Mule 4.3.0)

      Note that DataWeave does not support reading Excel (XLSX) input in the indexed mode.

    • In-Memory:
      Parses the entire document in memory.

      DataWeave can read all supported formats using the in-memory mode.

  • Streaming:
    Allows for sequential access to the file. This strategy partitions the input document into smaller items and accesses its data sequentially, storing the current item in memory. A DataWeave selector can access the portion of the file that is getting read.

    DataWeave supports streaming only for the following formats:

    CSV, JSON, Excel (XLSX, starting in Mule 4.2.2), XML (starting in Mule 4.3.0)

For details, see Streaming in DataWeave.

Using Reader and Writer Properties

In some cases, it is necessary to modify or specify aspects of the format through format-specific properties. For example, you can specify CSV input and output properties, such as the separator (or delimiter) to use in the CSV file. For Cobol copybook, you need to specify the path to a schema file using the schemaPath property.

You can append reader properties to the MIME type (outputMimeType) attribute for certain components in your Mule app. Listeners and Read operations accept these settings. For example, this On New File listener example identifies the , separator for a CSV input file:

Example: Properties for the CSV Reader
<file:listener doc:name="On New File" config-ref="File_Config" outputMimeType='application/csv; separator=","'>
  <scheduling-strategy >
    <fixed-frequency frequency="45" timeUnit="SECONDS"/>
  </scheduling-strategy>
  <file:matcher filenamePattern="comma_separated.csv" />
</file:listener>

Note that the outputMimeType setting above helps the CSV reader interpret the format and delimiter of the input comma_separated.csv file, not the writer.

To specify the output format, you can provide the MIME type and any writer properties for the writer, such as the CSV or JSON writer used by a File Write operation. For example, you might need to write a pipe (|) delimiter in your CSV output payload, instead of some other delimiter used in the input. To do this, you append the property and its value to the output directive of a DataWeave expression. For example, this Write operation specifies the pipe as a separator:

Example: output Directive for the CSV Writer
<file:write doc:name="Write" config-ref="File_Config" path="my_transform">
  <file:content ><![CDATA[#[output application/csv separator="|" --- payload]]]></file:content>
</file:write>

The sections below list the format-specific reader and writer properties available for each supported format.

Setting MIME Types

You can specify the MIME type for the input and output data that flows through a Mule app.

For DataWeave transformations, you can specify the MIME type for the output data. For example, you might set the output header directive of an expression in the Transform Message component or a Write operation to output application/json or output application/csv.

Starting in Mule 4.3.0, you can set the output directive using the format ID alone, instead of using the MIME type. For example, you might set the output header directive of an expression in the Transform Message component or a Write operation to output json or output csv. You can also use the format ID to differentiate the format of the output data from the MIME type in the output header. For example, you can write JSON data but customize the MIME type to application/problem+json by using the DataWeave directive output application/problem+json with json. See Change a Script’s MIME Type Output for examples that customize the MIME type output of a script.

This example sets the MIME type through a File Write operation to ensure that a format-specific writer, the CSV writer, outputs the payload in CSV format:

Example: MIME Type for the CSV Writer
<file:write doc:name="Write" config-ref="File_Config" path="my_transform">
  <file:content ><![CDATA[#[output application/csv --- payload]]]></file:content>
</file:write>

For input data, format-specific readers for Mule sources (such as the On New File listener), Mule operations (such as Read and HTTP Request operations), and DataWeave expressions attempt to infer the MIME type from metadata that is associated with input payloads, attributes, and variables in the Mule event. When the MIME type cannot be inferred from the metadata (and when that metadata is not static), Mule sources and operations allow you to specify the MIME type for the reader. For example, you might set the MIME type for the On New File listener to outputMimeType='application/csv' for CSV file input. This setting provides information about the file format to the CSV reader.

Example: MIME Type for the CSV Reader
<file:listener doc:name="On New File"
  config-ref="File_Config"
  outputMimeType='application/csv'>
</file:listener>

Note that reader settings are not used to perform a transformation from one format to another. They simply help the reader interpret the format of the input.

You can also set special reader and writer properties for use by the format-specific reader or writer of a source, operation, or component. See Using Reader and Writer Properties.

Was this article helpful?

💙 Thanks for your feedback!

Edit on GitHub