Parallel For Each (<parallel-foreach/>)

logo cloud IDE Cloud IDE

logo desktop IDE Desktop IDE

Process a collection of messages by splitting the collection into parts that are simultaneously processed.

After processing all messages, the component aggregates the results following the same order they were in before the split, and then the flow continues.

Parallel For Each buffers all processing routes' results in a list to return it after the scope finishes processing, which can cause out-of-memory errors when processing a large number of entries. To process large payloads, use Batch Processing instead.

Component XML

This component supports the following XML structure:

<parallel-foreach
  doc:name="Parallel foreach"
  doc:id="tbfiiz"
  collection="#[payload]"
  timeout=30000
  maxConcurrency=15 />

Parallel For Each (<parallel-foreach/>) attributes are configurable through the UI and XML.

Attribute Name Attribute XML Description

Attribute Name	Attribute XML	Description
Parallel foreach (default)	`doc:name`	Editable name for the component to display in the canvas.
N/A	`doc:id`	Automatically generated identifier for the component.
Collection	`collection`	An expression that returns a collection. Defaults to `#[payload]`.
Timeout	`timeout`	Specifies the timeout in milliseconds for each parallel route.
Max concurrency	`maxConcurrency`	Specifies the maximum level of parallelism for the router to use. + By default, all routes run in parallel.
Target Variable	`target`	Name of the variable in which you want to store message data. Names can only include numbers, characters, and underscores. For example, hyphens are not allowed in the name. See Enrich Data with Target Variables in the Mule documentation.
Target Value	`targetValue`	Value of the data to store in the target variable. By default, the value is the message payload (payload). The field accepts any value that a variable accepts: any supported data type, DataWeave expressions, the keywords `payload`, `attributes`, and `message`, but not the keyword `vars`. See Enrich Data with Target Variables in the Mule documentation.

Parallel foreach (default)

doc:name

Editable name for the component to display in the canvas.

N/A

doc:id

Automatically generated identifier for the component.

Collection

collection

An expression that returns a collection. Defaults to #[payload].

Timeout

timeout

Specifies the timeout in milliseconds for each parallel route.

Max concurrency

maxConcurrency

Specifies the maximum level of parallelism for the router to use.

+ By default, all routes run in parallel.

Target Variable

target

Name of the variable in which you want to store message data. Names can only include numbers, characters, and underscores. For example, hyphens are not allowed in the name. See Enrich Data with Target Variables in the Mule documentation.

Target Value

targetValue

Value of the data to store in the target variable. By default, the value is the message payload (payload). The field accepts any value that a variable accepts: any supported data type, DataWeave expressions, the keywords payload, attributes, and message, but not the keyword vars. See Enrich Data with Target Variables in the Mule documentation.

Examples

The following examples show how to use Parallel For each to append a string to every element in a collection and how modifying variables inside the scope does not affect the values after the execution.

Example: Appending Values

The following example appends the string "-result" to every element in the collection:

<flow name="myFlow">

  <parallel-foreach collection="#[['apple', 'banana', 'orange']]">
      <set-payload value="#[payload ++ '-result']"/>
  </parallel-foreach>

</flow>

Example: Modifying Variables

The following example demonstrates that changes to existing variables inside the Parallel For Each scope are temporary and do not persist after the execution finishes:

<flow name="myFlow">

  <set-variable variableName="var1" value="var1"/>
  <set-variable variableName="var2" value="var2"/>
  <parallel-foreach collection="#[['apple', 'banana', 'orange']]">
      <choice>
          <when expression="#[payload == 'apple']">
              <set-variable variableName="var2" value="newValue"/>
              <set-variable variableName="var3" value="appleVal"/>
          </when>
          <when expression="#[payload == 'banana']">
              <set-variable variableName="var3" value="bananaVal"/>
          </when>
          <otherwise>
              <set-variable variableName="var3" value="otherVal"/>
              <set-variable variableName="var4" value="val4"/>
          </otherwise>
      </choice>
  </parallel-foreach>

</flow>

The example produces the following output:

{
      var1: "var1",
      var2: "var2"
}

Every execution of the Parallel For Each scope starts with the same variables and values as before the execution of the block.

New variables or modifications of already existing variables while processing one element are not visible while processing another element. All of those variable changes are not available outside the Parallel For Each scope. The set of variables (and their values) after the execution of the Parallel For Each Scope remains the same as before the execution.

Error Handling

Because the scope processes every route in parallel, if an error occurs in one route, processing continues in all of the other routes until all finish processing. After that, Parallel For Each aggregates all results (and any errors) and processes them by the error handler, as shown in the following example:

<flow name="myFlow">

  <parallel-foreach collection="#[['banana', 'apple']]">
      <choice>
          <when expression="#[payload == 'banana']">
              <!-- Processor that throws error -->
          </when>
          <otherwise>
              <set-payload value="#[payload ++ '-result']"/>
          </otherwise>
      </choice>
  </parallel-foreach>
  <error-handler>
      <on-error-continue type="COMPOSITE_ROUTING">
          <!-- This will have the error thrown by the previous processor -->
          <logger message="#[error.errorMessage.payload.failures['0']]"/>
          <!-- This will be a null value -->
          <logger message="#[error.errorMessage.payload.failures['1']]"/>
          <!-- This will be a null value -->
          <logger message="#[error.errorMessage.payload.results['0']]"/>
          <!-- This will have the result of this (correctly executed) route -->
          <logger message="#[error.errorMessage.payload.results['1']]"/>
      </on-error-continue>
  </error-handler>

</flow>

Throws

MULE:COMPOSITE_ROUTING

Differences between For Each and Parallel For Each Scopes

Both For Each and Parallel For Each split the defined collection, and the components within the scope process each element in the collection. Also, in both cases, each route runs with the same initial context. The difference between these two scopes are:

For Each works sequentially, while the Parallel For Each processes in parallel. This difference affects error handling:

Because of the processing differences, the execution of For Each execution is interrupted when an error is raised (and the Error Handler is invoked), while Parallel For Each processes every route before invoking the Error Handler with a MULE:COMPOSITE_ROUTE error type.
For Each does not modify the payload, while the Parallel For Each outputs a collection of the output messages from each iteration.