Contact Free trial Login
All Connectors

To Trigger a Flow When a New File is Created or Modified

The File, FTP, and SFTP connectors provide a listener (called On New or Updated file in Studio and Design Center) that polls a directory for files that have been created or updated. A message is generated for each file that is found.

The key part of this functionality is how to determine that a file is new. There are three strategies for making this determination:

  • Set the autoDelete parameter to true. This setting deletes each file after it has been processed so that all files found in the next poll are new.

  • Set moveToDirectory parameter to move each processed file to a different directory after it has been processed. This produces the same effect as autoDelete but without deleting the file.

  • Use the watermark parameter to only pick files that have been created or updated after the last poll was executed.

  • Use a matcher to filter the files.

Listener

Listeners function similarly for the FTP, SFTP, and File connectors. However, the XML element used for them is different.

Example: File Listener (On New File Trigger)
<flow name="onNewFile">
    <file:listener config-ref="file" directory="test-data/in" autoDelete="true">
      <scheduling-strategy>
        <fixed-frequency frequency="1000"/>
      </scheduling-strategy>
    </file:listener>

    <flow-ref name="processFile" />
</flow>

Watermarking

For the File connector, the watermarking strategy can be used in two different modes: CREATION_TIMESTAMP or MODIFIED_TIMESTAMP. The strategy depends on whether you want to filter files based on their creation or modification time.

Example: Watermarking with the File Listener
<file:listener doc:name="On New File"
  config-ref="File_Config1"
  watermarkMode="CREATED_TIMESTAMP">
    <scheduling-strategy >
      <fixed-frequency />
    </scheduling-strategy>
</file:listener>

Repeatable Streams

The List operation makes use of the repeatable streams functionality introduced in Mule 4. The operation returns a list of messages, where each message represents a file in the list and holds a stream to the file. A stream is repeatable by default.

For more information on this topic, see Streaming in Mule 4.