Contact Free trial Login

Compression Module Documentation Reference

A module which provides functionality for compressing and decompressing data.

Configurations


Config

Default configuration

Parameters

Name Type Description Default Value Required

Name

String

The name for this configuration. Connectors reference the configuration with this name.

x

Operations

Archive

<compression:archive>

Compresses all the given entries into a new file in the configured format.

Each entry passed to this operation is placed inside the compressed archive with the name provided by the user in the DataWeave script. For example, for this expression:

{ file: vars.aTxtContent, 'dir/resume.pdf': vars.pdf }

The resultant archive contains two entries one named "file" at root level and another called "resume.pdf" inside a directory called "dir"

+- Archive | \- file1 | \+ dir1 | \- file2

In this example, the slash "/" in the name of an entry indicates directory separation, so all names are introspected to create directories inside the archive.

Parameters

Name Type Description Default Value Required

Entries

Object

#[payload]

Archiver

zip-archiver

x

Output Mime Type

String

The MIME type of the payload that this operation outputs.

Output Encoding

String

The encoding of the payload that this operation outputs.

Streaming Strategy

Configure if repeatable streams should be used and their behavior.

Target Variable

String

The name of a variable on which the operation's output will be placed

Target Value

String

An expression to evaluate against the operation's output and the outcome of that expression will be stored in the target variable.

#[payload]

Output

Type

Binary

Throws

  • COMPRESSION:COULD_NOT_COMPRESS

Compress

<compression:compress>

Compresses a given content using the configured format, ZIP or GZIP. If a problem occur while compressing the content a COULD_NOT_COMPRESS error will be thrown.

Parameters

Name Type Description Default Value Required

Content

Binary

#[payload]

Compressor

One of:

  • gzip-compressor

  • zip-compressor

x

Output Mime Type

String

The MIME type of the payload that this operation outputs.

Output Encoding

String

The encoding of the payload that this operation outputs.

Streaming Strategy

Configure if repeatable streams should be used and their behavior

Target Variable

String

The name of a variable on which the operation's output will be placed

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

Output

Type

Binary

Throws

  • COMPRESSION:COULD_NOT_COMPRESS

Decompress

<compression:decompress>

Decompresses a single entry of compressed content which is assumed to be in an specific format (ZIP or GZIP). If the content is not in the configured format, an INVALID_ARCHIVE error is thrown. Compressed archives can have multiple entries, this operation can handle only single entry archives because if the archive would contain more than one the operation wouldn't know which should return, if the content has more than one entry an TOO_MANY_ENTRIES error will be thrown. For multiple entry archives use the `extract` operation of this module, which will let you choose only formats that accept multiple entries.

Parameters

Name Type Description Default Value Required

Compressed

Binary

#[payload]

Decompressor

One of:

  • gzip-decompressor

  • zip-decompressor

x

Output Mime Type

String

The MIME type of the payload that this operation outputs.

Output Encoding

String

The encoding of the payload that this operation outputs.

Streaming Strategy

Configure if repeatable streams should be used and their behavior

Target Variable

String

The name of a variable on which the operation's output will be placed

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

Output

Type

Binary

Throws

  • COMPRESSION:COULD_NOT_DECOMPRESS

  • COMPRESSION:INVALID_ARCHIVE

  • COMPRESSION:TOO_MANY_ENTRIES

Extract

<compression:extract>

Decompresses a given content that represent an archive in a compression format.

Parameters

Name Type Description Default Value Required

Compressed

Binary

#[payload]

Extractor

One of:

  • zip-extractor

x

Target Variable

String

The name of a variable on which the operation's output will be placed

Target Value

String

An expression that will be evaluated against the operation's output and the outcome of that expression will be stored in the target variable

#[payload]

Output

Type

Object

Throws

  • COMPRESSION:COULD_NOT_DECOMPRESS

  • COMPRESSION:INVALID_ARCHIVE

Types

Repeatable In Memory Stream

Field Type Description Default Value Required

Initial Buffer Size

Number

This is the amount of memory to allocate to consume the stream and provide random access to it. If the stream contains more data than can fit into this buffer, then the buffer expands according to the bufferSizeIncrement attribute, with an upper limit of maxInMemorySize.

Buffer Size Increment

Number

This is by how much the buffer size expands if it exceeds its initial size. Setting a value of zero or lower will mean that the buffer should not expand, meaning that a STREAM_MAXIMUM_SIZE_EXCEEDED error will be raised when the buffer gets full.

Max Buffer Size

Number

This is the maximum amount of memory that will be used. If more than that is used then a STREAM_MAXIMUM_SIZE_EXCEEDED error will be raised. A value lower or equal to zero means no limit.

Buffer Unit

Enumeration, one of:

  • BYTE

  • KB

  • MB

  • GB

The unit in which all these attributes are expressed

Repeatable File Store Stream

Field Type Description Default Value Required

Max In Memory Size

Number

Defines the maximum memory that the stream should use to keep data in memory. If more than that is consumed then it will start to buffer the content on disk.

Buffer Unit

Enumeration, one of:

  • BYTE

  • KB

  • MB

  • GB

The unit in which maxInMemorySize is expressed

Was this article helpful?

💙 Thanks for your feedback!

Edit on GitHub