salesforce Dreamforce On-Demand
Contact Us 1-800-596-4880
  1. Homepage
  2. Amazon Kinesis Connector (1.0)

  3. Using Anypoint Studio to Configure Amazon Kinesis Connector

All Connectors

Using Anypoint Studio to Configure Amazon Kinesis Data Streams Connector 1.0 - Mule 4

Anypoint Studio (Studio) editors help you design and update your Mule applications, properties, and configuration files.

To add and configure a connector in Studio:

  1. Create a mule project.

  2. Add the connector to your Mule project.

  3. Add a connector operation to the flow.

  4. Configure a global element for the connector

When you run the connector, you can view the app log to check for problems, as described in View the App Log.

If you are new to configuring connectors in Studio, see Using Anypoint Studio to Configure a Connector.

Create a Mule Project

In Studio, create a new Mule project in which to add and configure the connector:

  1. In Studio, select File > New > Mule Project.

  2. Enter a name for your Mule project and click Finish.

Add the Connector to Your Mule Project

Add Anypoint Connector for Amazon Kinesis Data Streams Connector (Amazon Kinesis Data Streams Connector) to your Mule project to automatically populate the XML code with the connector’s namespace and schema location and add the required dependencies to the project’s pom.xml file:

  1. In the Mule Palette view, click (X) Search in Exchange.

  2. In Add Modules to Project, type kinesis in the search field.

  3. Click Amazon Kinesis Data Streams Connector in Available modules.

  4. Click Add.

  5. Click Finish.

Adding a connector to a Mule project in Studio does not make that connector available to other projects in your Studio workspace.

Configure a Source

A source initiates a flow when a specified condition is met. You can configure one of these sources to use with Amazon Kinesis Data Streams Connector:

  • Listener

    Enables you to subscribe to a particular data stream. This source uses the Kinesis Client Library (KCL) to listen from a specified initial point in the data stream. It returns dynamically sized batches of records, however, the max size can also be configured.

  • HTTP > Listener

    Initiates a flow each time it receives a request on the configured host and port

  • Scheduler

    Initiates a flow when a time-based condition is met

For example, to configure the Amazon Kinesis Listener source, follow these steps:

  1. In the Mule Palette view, select Kinesis.

  2. Drag Listener to the Studio canvas.

  3. On the Listener properties window, optionally change the value of the Display Name field.

  4. Click the plus sign (+) next to the Connector configuration field to configure a global element that can be used by all instances of the Amazon Kinesis Connector in the app.

  5. On the General tab, complete the following fields:

    Field Description

    Name

    Name used to reference this connector instance

    Access Key

    Access key provided by Amazon

    Secret Key

    Secret key provided by Amazon

    Region Endpoint

    URL entry point for a web service

    Role

    IAM role, if any

  6. On the Security tab, optionally specify the TLS information for the connector.

  7. On the Advanced tab, optionally specify reconnection information, including a reconnection strategy.

  8. Click Test Connection to confirm that Mule can connect with the specified server.

  9. Click OK.

  10. In the Listener Params section of the Listener properties window, optionally enter values for the fields.

Add a Connector Operation to the Flow

When you add a connector operation to your flow, you immediately define a specific operation for that connector to perform.

You can use the following operations:

  • Checkpoint, which keeps track of the current position in the Kinesis data stream

  • Seek, which moves the reading position of the sources (listeners)

    While moving forward in the stream is fluent, moving backward causes an internal restart of the Kinesis Client Library (KCL) instance. Therefore, use this operation sparingly to move backward in a stream.

  • Put Record, which puts a record into the Kinesis data stream.

    If the Aggregation Enabled field in the global configuration Advanced tab is true (the default), then multiple records are combined and stored in one Kinesis Data Streams record.

    Records include binary data, a partition key, and an explicit hash key. The explicit hash key value is normally computed with the MD5 function of the record partition key, but you can override the value by using the Explicit Hash Key field. The operation uses the Kinesis Producer Library (KPL), which increases throughput.

To add an operation for Amazon Kinesis Data Streams Connector, follow these steps:

  1. In the Mule Palette view, select Kinesis and then select the desired operation.

  2. Drag the operation onto the Studio canvas to the right of the source.

Configure a Global Element for the Connector

When you configure a connector, it’s best to configure a global element that all instances of that connector in the app can use. Configuring a global element requires you to provide the authentication credentials that the connector requires to access the target Amazon Kinesis Data Streams system.

Amazon Kinesis Data Streams Connector supports Access Key authentication, which uses Access Key and Secret Key field values for AWS Identity and Access Management (IAM).

When you configure a global element, you can reference a configuration file that contains ANT-style property placeholders (recommended), or you can enter your authorization credentials in the global configuration properties. For information about the benefits of using property placeholders and how to configure them, see Anypoint Connector Configuration.

To configure a global element, follow these steps:

  1. Select the name of the connector in the Studio canvas.

  2. In the configuration screen for the operation, click the plus sign (+) next to the Connector configuration field to access the global element configuration fields.

  3. On the General tab, configure the following fields.

    Field Description

    Name

    Name used to reference the connector instance

    Access Key

    Access key provided by Amazon

    Secret Key

    Secret key provided by Amazon

    Region Endpoint

    URL entry point for a web service

    Role

    IAM role, if any

  4. On the Advanced tab, optionally specify proxy and reconnection information, including a reconnection strategy.

  5. Click Test Connection to confirm that Mule can connect with the specified server.

  6. Click OK.

View the App Log

To check for problems, you can view the app log as follows:

  • If you’re running the app from Anypoint Platform, the app log output is visible in the Anypoint Studio console window.

  • If you’re running the app using Mule from the command line, the app log output is visible in your OS console.

Unless the log file path is customized in the app’s log file (log4j2.xml), you can also view the app log in the default location MULE_HOME/logs/<app-name>.log. You can configure the location of the log path in the app log file log4j2.xml.

Next Step

After you configure a global element and connection information, configure the other fields for the connector.

See Also

Back to top
View on GitHub