Repeatable vs. Non-Repeatable Streaming

Understanding streaming strategies and choosing the one that best suits your use case is key to maximizing flow performance:

A non-repeatable stream (read payload only once) or
A repeatable stream (read payload more than once)

Non-Repeatable Streaming

The non-repeatable strategy disables repeatable streams. If your use case requires reading the payload only once, use the non-repeatable-stream parameter in the connector. See the following example:

<file:read path="exampleFile.json">
  <non-repeatable-stream />
</file:read>

Repeatable Streaming

Alternatively, if your use case requires reading the payload multiple times or concurrently, you can choose between the following strategies:

File-store streaming
In-memory streaming

File-Store Repeatable Stream

When using the file-store repeatable stream strategy, customize the buffer size inMemorySize to your use case to optimize performance:

Configure a larger buffer size to avoid the number of times Mule needs to write the buffer on disk. This increases performance, but it also limits the number of concurrent requests your application can process because it requires additional memory.
Configure a smaller buffer size to decrease memory load at the expense of response time.

The following example sets a specific buffer size for a file-store repeatable stream for a file that is approximately 1 MB:

<file:read path="bigFile.json">
  <repeatable-file-store-stream
    inMemorySize="1"
    bufferUnit="MB"/>
</file:read>

Always run performance tests to find the optimal buffer size for your specific use case.

In-Memory Repeatable Stream

When using the in-memory repeatable stream strategy, to manage larger or smaller files, customize the buffer by setting:

initialBufferSize
The initial size
bufferSizeIncrement
The rate at which the buffer increases.
maxInMemorySize
The maximum buffer size.
bufferUnit
The unit of measurement for the buffer size value.

Adjusting the in-memory buffer size (defaults to 512KB) based on your payload size and the expected concurrency.

The following example configures an in-memory repeatable stream with an initial size of 512 KB, which grows at a rate of 256 KB and allows up to 2000 KB (2 MB) of content in memory:

<file:read path="exampleFile.json">
  <repeatable-in-memory-stream
    initialBufferSize="512"
    bufferSizeIncrement="256"
    maxInMemorySize="2000"
    bufferUnit="KB"/>
</file:read>