Contact Us 1-800-596-4880
  1. Homepage
  2. DataWeave (2.3)

  3. Newline Delimited JSON Format (ndjson) (2.3)

Newline Delimited JSON Format (ndjson)

MIME type: application/x-ndjson

ID: ndjson

DataWeave represents the Newline Delimited JSON format (ndjson) as an array of objects. Each line of the ndjson format is mapped to one object in the array.

The following parser strategies are supported by the ndjson reader:

  • In-memory

  • Streaming

For details, see DataWeave Readers.

Examples

The following examples show uses of the ndjson format.

Example: Transform ndjson to the DataWeave Format (dw)

This example shows how DataWeave represents simple ndjson input.

Input

{"name": "Leandro","lastName": "Shokida"}
{"name": "Mariano","lastName": "De Achaval"}

Source

The DataWeave script transforms the ndjson input to the DataWeave (dw) format and MIME type.

%dw 2.0
output application/dw
---
payload

Output

The DataWeave (dw) format outputs the ndjson input into an array of comma-separated objects.

[
    {"name": "Leandro","lastName": "Shokida"},
    {"name": "Mariano","lastName": "De Achaval"}
]

Example: Use the skipInvalid

This example shows that the ndjson reader ignores all lines of ndjson data that are invalid if skipInvalid=true.

Input

The input to the DataWeave source includes valid and invalid lines of ndjson data. Assume that the input is from a file myInput.ndjson.

myInput.ndjson:
{"name": "Christian"
{"name": "Mariano"}
{"name": "Tomo"
{"name": "Shoki"}

Source

The DataWeave script inputs the contents of the input file myInput.ndjson, applies the skipInvalid=true reader property, and transforms the input to the JSON format and MIME type.

%dw 2.0
var myInput = readUrl('classpath://myInput.ndjson', 'application/x-ndjson, {skipInvalid=true})
output application/json
---
myInput

Output

The JSON output is an array of the valid objects from the ndjson input.

[
  {
    "name": "Mariano"
  },
  {
    "name": "Shoki"
  }
]

Configuration Properties

DataWeave supports the following configuration properties for the Newline Delimited JSON format (ndjson).

Reader Properties

The ndjson format accepts properties that provide instructions for reading input data.

Parameter Type Default Description

ignoreEmptyLine

Boolean

true

Ignores any empty line. Valid values are true or false.

skipInvalid

Boolean

false

Skips data that is not valid ndjson. Valid values are true or false.

Writer Properties

The ndjson format accepts properties that provide instructions for writing output data.

Parameter Type Default Description

bufferSize

Number

8192

Size of the writer buffer.

deferred

Boolean

false

When set to true, DataWeave generates the output as a data stream, and the script’s execution is deferred until it is consumed. Valid values are true or false.

encoding

String

'UTF-8'

Encoding for the ndjson reader to use.

skipNullOn

String

null

Valid values are arrays or objects.

writeAttributes

Boolean

false

Valid values are true or false.

Supported MIME Types

The Newline Delimited JSON format (ndjson) supports the following MIME types.

MIME Type

application/x-ndjson

application/x-ldjson