Contact Free trial Login

Avro Format

MIME type: application/avro

ID: avro

Avro is a binary data format that uses a schema to structure its data. DataWeave relies on the schema to parse the data. Avro data structures are mapped to DataWeave data structures.

Java Value Mapping

The following table shows how Avro types map to DataWeave types.

Avro Type DataWeave Type

long

Number

int

Number

double

Number

boolean

Boolean

string

String

fixed

String

bytes

Binary

enum

String

map

Object

array

Array

null

Null

Example: Use an Avro Schema

The following example shows how to specify a schema that the writer uses to output an Avro data structure.

Input

An Avro schema looks something like this.

schema.json:
{
    "type": "record",
    "name": "userInfo",
    "namespace": "my.example",
    "fields": [
        {
            "name": "username",
            "type": "string",
            "default": "NONE"
        },
        {
            "name": "age",
            "type": "int",
            "default": -1
        },
        {
            "name": "phone",
            "type": "string",
            "default": "NONE"
        },
        {
            "name": "housenum",
            "type": "string",
            "default": "NONE"
        }
    ]
}

Source

The schemaUrl property in the header of this DataWeave script passes a schema (schema.json) to the DataWeave writer. The writer uses the schema to structure content from the body of the script and output the results in Avro format.

%dw 2.0
output application/avro schemaUrl="classpath://schema.json"
---
[{
    username: "Mariano",
    age: 35,
    phone: "213",
    housenum: "123"
},
{
    username: "Leandro",
    age: 29,
    phone: "213",
    housenum: "123"
},
{
    username: "Christian",
    age: 25,
    phone: "213",
    housenum: "123"
}]

Configuration Properties

DataWeave supports the following configuration properties for Avro.

Reader Properties (for Avro)

The Avro format accepts properties that provide instructions for reading input data.

Parameter Type Default Description

schemaUrl (Required)

String

null

The URL for the Avro schema. Valid values are classpath://, file://, or http://.

Writer Properties (for Avro)

The Avro format accepts properties that provide instructions for writing output data.

Parameter Type Default Description

bufferSize

Number

8192

Size of the buffer writer.

deferred

Boolean

false

Property for deferred output. Valid values are true or false.

schemaUrl (Required)

String

null

The URL for Avro schema. Valid values are classpath://, file://, or http://.

Supported MIME Types

The Avro format supports the following MIME types.

MIME Type

application/avro

Was this article helpful?

💙 Thanks for your feedback!

Edit on GitHub