HDFS (Hadoop) Connector Reference - Mule 4

String

The name for this configuration. Connectors reference the configuration with this name.

x

The connection types to provide to this configuration.

x

Expiration Policy

Configures the minimum amount of time that a dynamic configuration instance can remain idle before Mule considers it eligible for expiration. This does not mean that the platform expires the instance at the exact moment that it becomes eligible. Mule purges the instances as appropriate.

String

The Kerberos principal. The Username is passed to the HDFS client as the hadoop.job.ugi configuration entry. The Username can be overridden by values you specify in Configuration Resources and Configuration Entries. This parameter is called Username for backward compatibility reasons.

String

Enter the path to the keytab file associated with the Username you specified. The path is used to obtain a ticket granting ticket (TGT) from the authorization server. If this value is not provided Kerberos looks for a TGT associated with the specified Username within your local Kerberos cache.

String

The name of the file system to connect to. The Name Node is passed to the HDFS client as the FileSystem#FS_DEFAULT_NAME_KEY configuration entry. The Name Node can be overridden by values you specify in Configuration Resources and Configuration Entries.

x

Array of String

A java.util.List of configuration resource files for the HDFS client to load. You can provide additional configuration files, for example, core-site.xml.

Object

A java.util.Map of configuration entries to use by the HDFS client. You can provide additional configuration entries as key-value pairs.

Reconnection

When the application is deployed, a connectivity test is performed on all connectors. If reconnection is enabled, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.

String

User identity that Hadoop uses for permissions in HDFS. When Simple authentication is used, Hadoop requires that the user is set as a system property called HADOOP_USER_NAME. If a value is not provided, Hadoop uses the username of the OS user who is currently logged in.

String

The name of the file system to which to connect. The Name Node is passed to the HDFS client as the FileSystem#FS_DEFAULT_NAME_KEY configuration entry. The Name Node can be overridden by values you specify in Configuration Resources and Configuration Entries.

x

Array of String

A java.util.List of configuration resource files for the HDFS client to load. You can provide additional configuration files, for example, core-site.xml.

Object

A java.util.Map of configuration entries for the HDFS client to use. You can provide additional configuration entries as key-value pairs.

Reconnection

When the application is deployed, a connectivity test is performed on all connectors. If reconnection is enabled, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.

Boolean

When the application is deployed, a connectivity test is performed on all connectors. If set to true, deployment fails if the test doesn’t pass after exhausting the associated reconnection strategy.

The reconnection strategy to use.

Number

How often to reconnect (in milliseconds)

Number

The number of reconnection attempts to make

Boolean

If false, the reconnection strategy runs in a separate, non-blocking thread.

true

Number

How often in milliseconds to reconnect

Boolean

If false, the reconnection strategy runs in a separate, non-blocking thread.

true

Number

A scalar time value for the maximum amount of time a dynamic configuration instance should be allowed to be idle before it’s considered eligible for expiration.

Enumeration, one of:

A time unit that qualifies the maxIdleTime attribute

Number

The amount of memory that will be allocated to consume the stream and provide random access to it. If the stream contains more data than can be fit into this buffer, then the buffer expands according to the bufferSizeIncrement attribute, with an upper limit of maxInMemorySize.

Number

This is by how much the buffer size expands if it exceeds its initial size. Setting a value of zero or lower means that the buffer should not expand, meaning that a STREAM_MAXIMUM_SIZE_EXCEEDED error is raised when the buffer gets full.

Number

The maximum amount of memory to use. If more than that is used, the STREAM_MAXIMUM_SIZE_EXCEEDED error is raised. A value lower than or equal to zero means no limit.

Enumeration, one of:

The unit in which all these attributes are expressed

Number

Defines the maximum memory that the stream should use to keep data in memory. If more memory is consumed, content on the disk is buffered.

Enumeration, one of:

The unit in which maxInMemorySize is expressed.

Number

The maximum number of times a message can be redelivered and processed unsuccessfully before triggering a process-failed message.

Boolean

Whether to use a secure hash algorithm to identify a redelivered message.

String

The secure hashing algorithm to use. If not set, the default is SHA-256.

String

Defines one or more expressions to use to determine when a message has been redelivered. This property can only be set if Use Secure Hash is false.

Object Store

The object store where the redelivery counter for each message is stored.

Check Summary

Content Summary

File Status

Boolean

Number

Number

String

Number

Number

Number

Number

Number

Number

Number

Number

Access time of file in milliseconds

Number

Replication factor of file

Number

Block size of file

Boolean

Indicates whether path is a directory

String

Group owner associated with file

Number

Length of file in bytes

Number

Modification time of file in milliseconds

String

Owner of file

String

Path name

String

Permission of file as an octal string

Boolean

Indicates whether a path is a symbolic link