Contact Free trial Login

Trigger a Flow When a New File is Created or Modified

The File, FTP, and SFTP connectors provide a listener (called On New or Updated file in Studio and Design Center) that polls a directory for files that have been created or updated. A message is generated for each file that is found.

The key part of this functionality is how to determine that a file is new. There are three strategies for making this determination:

  • Set the autoDelete parameter to true. This setting deletes each file after it has been processed so that all files found in the next poll are new.

  • Set moveToDirectory parameter to move each processed file to a different directory after it has been processed. This produces the same effect as autoDelete but without deleting the file.

  • Use the watermark parameter to only pick files that have been created or updated after the last poll was executed.

  • Use a matcher to filter the files.

Listener

Listeners function similarly for the FTP, SFTP, and File connectors. However, the XML element used for them is different.

Example: SFTP Listener (On New File Trigger)
<flow name="onNewFile">
    <sftp:listener config-ref="file" directory="test-data/in" autoDelete="true">
      <scheduling-strategy>
        <fixed-frequency frequency="1000"/>
      </scheduling-strategy>
    </sftp:listener>

    <flow-ref name="processFile" />
</flow>

Watermark

The SFTP connector provides a way of activating or deactivating the watermark. Here is an example:

Example: Watermarking with the SFTP Listener
<sftp:listener doc:name="On New File"
  watermarkEnabled="true"
  config-ref="SFTP_Config">
    <scheduling-strategy >
      <fixed-frequency />
    </scheduling-strategy>
</sftp:listener>

We use cookies to make interactions with our websites and services easy and meaningful, to better understand how they are used and to tailor advertising. You can read more and make your cookie choices here. By continuing to use this site you are giving us your consent to do this.