Free MuleSoft CONNECT Keynote & Expo Pass Available!

Register now+
Nav
You are viewing an older version of this section. Click here to navigate to the latest version.

Amazon S3 Connector

Select

The Anypoint Amazon S3 Connector provides connectivity to the the Amazon S3 API, enabling you to interface with Amazon S3 to store objects, download and use data with other AWS services, and build applications that call for internet storage. Instant access to the Amazon S3 API enables seamless integrations between Amazon S3 and other databases, CMS applications such as Drupal, and CRM applications such as Salesforce.

Introduction

Amazon Simple Storage Service (Amazon S3) is storage for the Internet. You can use Amazon S3 to store and retrieve any amount of data at any time, from anywhere on the web. You can accomplish these tasks using the simple and intuitive web interface of the AWS Management Console.

The AWS SDK for Java provides a Java API for AWS infrastructure services. The Amazon S3 connector is built using the SDK for Java. For the complete list of operations supported by the connector, see Apidocs and Samples.

Prerequisites

To use the Amazon S3 connector, you must have the following:

  • Access to Amazon Web Services. 

  • To access AWS with the connector, you need the credentials in the form of IAM.

  • Anypoint Studio Enterprise edition.

This document assumes that you are familiar with Mule, Anypoint Connectors, Anypoint Studio Essentials. To increase your familiarity with Studio, consider completing one or more Basic Studio Tutorial. Further, this page assumes that you have a basic understanding of Elements in a Mule Flow, Global Elements.

This document describes implementation examples within the context of Anypoint Studio, Mule’s graphical user interface, and also includes configuration details for doing the same in the XML Editor.

Compatibility

Amazon S3 connector is compatible with:

Application/Service Version

Mule Runtime

3.5.0 or later

AWS SDK for Java

1.9.39

Installing and Configuring

The following sections describe how to install and configure this connector.

Installing

  1. In Anypoint Studio, click the Exchange icon in the Studio taskbar.

  2. Click Login in Anypoint Exchange.

  3. Search for the connector and click Install.

  4. Follow the prompts to install the connector.

When Studio has an update, a message displays in the lower right corner, which you can click to install the update.

The following updates are in version 3.0 of the Amazon S3 connector:

  • Renamed the following operations:

    • List object versions to List versions

    • Set Bucket Versioning status to Set Bucket Versioning Configuration

  • Removed the following operations:

    • Create object uri

For most of the operations the input and output attributes have been modified.

Configuring the Global Element with Studio Visual Editor

To use the Amazon S3 connector in your Mule application, configure a global Amazon S3 element for use by all the Amazon S3 connectors in the application.

  1. Click the Global Elements tab at the base of the canvas, then click Create.

  2. In the Choose Global Type window, expand Connector Configuration, and click Amazon S3: Configuration.

    ams3_01

  3. Click Ok

  4. Enter the global element properties:

    ams3_02

    Parameter Description

    Name

    Enter a name for the configuration to reference it.

    Access Key

    Alphanumeric text string that uniquely identifies the user who owns the account.

    Secret Key

    Key that plays the role of a password.

    In the image above, the placeholder values refer to a configuration file placed in the  src folder of your project. See Learn how to configure properties.

You can either enter your credentials into the global configuration properties, or reference a configuration file that contains these values. For simpler maintenance and better re-usability of your project, Mule recommends that you use a configuration file. Keeping these values in a separate file is useful if you need to deploy to different environments, such as production, development, and QA, where your access credentials differ. See  Deploying to Multiple Environments for instructions on how to manage this.

  1. Keep the Pooling Profile and the Reconnection tabs with their default entries.

  2. Click Test Connection to confirm that the parameters of your global configuration are accurate, and that Mule is able to successfully connect to your instance of Amazon S3. Read more about this in Testing Connections.

  3. Click OK to save the global connector configurations. 

Configuring the Global Element with XML Editor

First, ensure that you have included the Amazon S3 namespaces in your configuration file.


          
       
1
2
3
4
5
6
7
8
<mule xmlns:s3="http://www.mulesoft.org/schema/mule/s3"  xmlns:http="http://www.mulesoft.org/schema/mule/http"  xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns="http://www.mulesoft.org/schema/mule/core"  xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
 xmlns:spring="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.springframework.org/schema/beans" http://www.springframework.org/schema/beans/spring-beans-current.xsd

  <!-- Put your flows and configuration elements here -->

</mule>

Follow these steps to configure an Amazon S3 connector in your application: Create a global Amazon S3 configuration outside and above your flows, using the following global configuration code.


          
       
1
2
<!-- simple configuration -->
<s3:config name="Amazon_S3_Configuration" accessKey="${s3.accessKey}" secretKey="${s3.secretKey}" doc:name="Amazon S3: Configuration"/>
Parameter Description

Name

Enter a name for the configuration with which it can be referenced later by config-ref. The name in this example is Amazon_S3_Configuration.

accessKey

The access key provided by Amazon. It is required for non-anonymous operations.

secretKey

The secrete key provided by Amazon. It is required for non-anonynous operations.

------

== Using the Connector

Amazon S3 connector is an operation-based connector, which means that when you add the connector to your flow, you need to configure a specific operation for the connector to perform. The Amazon S3 connector currently supports the following list of operations:

[NOTE]
For the operations to work, you need to enable or update the subset of the overall list of Amazon S3 actions on the bucket to specify that the AWS account has access to the subset actions on the bucket.

* Abort Multipart Upload
* Complete Multipart Upload
* Copy Object
* Create Bucket
* Create Object
* Create Object Presigned URI
* Delete Bucket
* Delete Bucket Cross Origin Configuration
* Delete Bucket Lifecycle Configuration
* Delete Bucket Policy
* Delete Bucket Tagging Configuration
* Delete Bucket Website Configuration
* Delete Object
* Delete Objects
* Get Bucket ACL
* Get Bucket Cross Origin Configuration
* Get Bucket Lifecycle Configuration
* Get Bucket Location
* Get Bucket Logging Configuration
* Get Bucket Notification Configuration
* Get Bucket Policy
* Get Bucket Tagging Configuration
* Get Bucket Versioning Configuration
* Get Bucket Website Configuration
* Get Object
* Get Object ACL
* Get Object Content
* Get Object Metadata
* Initiate Multipart Upload
* List Buckets
* List Multipart Uploads
* List Next Batch of Objects
* List Next Batch of Versions
* List Objects
* List Parts
* List Versions
* Set Bucket ACL
* Set Bucket Cross Origin Configuration
* Set Bucket Lifecycle Configuration
* Set Bucket Logging Configuration
* Set Bucket Notification Configuration
* Set Bucket Policy
* Set Bucket tagging Configuration
* Set Bucket Versioning Configuration
* Set Bucket Website Configuration
* Set Object ACL
* Set Object Storage Class
* Upload Part
* Upload Part Copy

=== Adding the Connector to a Mule Flow

. Create a new Mule project in Anypoint Studio.
. Drag the Amazon S3 connector onto the canvas, then select it to open the properties editor.
. Configure the connector’s parameters:
+
image:ams3_03.png[ams3_03]
+
[%header,cols="20a,50a,30a"]
|===
|Field|Description|Example
|*Display Name*|Enter a unique label for the connector in your application.|Amazon S3
|*Connector Configuration*|Select a global Amazon S3 connector element from the drop-drown.|N/A
|*Operation*|Select the action this component must perform.|Create bucket
|*Bucket Name*|Select a parameter for the operation.|#[payload] or `${bucketName}` to pick the value using MEL expression.
|===
+
. Save your configurations.

== Use Cases

The following are the common use cases for the Amazon S3 connector:

* Store an image from a URL on Amazon S3, then retrieve and display the image.
* Create an image link in Amazon S3 and update the status in twitter along with the image link.

== Example Use Case

Create a Mule application that stores an image from a URL on Amazon S3, then retrieve and display the image.

=== Using the Studio Visual Editor


image:ams3_04.png[ams3_04]

. Begin the flow by sending a message to a bucket.
. Create a new Mule project in Anypoint Studio.
. Drag an HTTP connector into the canvas, then select it to open the properties editor console.
. Add a new HTTP Listener Configuration global element:
.. In *General Settings*, click the *+* button:
+
image:ams3_05.png[ams3_05]
+
.. Configure the following HTTP parameters, and retain the default values for the other fields:
+
[%header,cols="30a,70a",width=80%]
|===
|Field|Value
|*Name*|HTTP_Listener_Configuration
|*Port*|8081
|===
+
.. Reference the HTTP Listener Configuration global element.
. Drag an Amazon S3 connector into the flow, and double-click the connector to open its Properties Editor.
. If you do not have an existing Amazon S3 connector global element to choose, click the plus sign next to Connector Configuration.
. Configure the global element properties, then click *OK*.
. Configure the remaining parameters of the connector:
+
image:ams3_06.png[ams3_06]
+
[%header,cols="30a,70a",width=80%]
|===
|Field|Value
|*Display Name*|Enter a name for the connector instance.
|*Connector Configuration*|Select a global configuration for the connector.
|*Operation*|Create bucket
|*Bucket Name*|`${config.bucket}`
|*Canned ACL*|`PUBLIC_READ`
|===
+
Add a *HTTP Connector* to request the MuleSoft logo from MuleSoft.
+
image:ams3_07.png[ams3_07]
+
[%header,cols="30a,70a",width=80%]
|===
|Field|Value
|*Display Name*|Enter a name for the connector instance.
|*Connector Configuration*|Create a new default configuration with *Host* as *mulesoft.org* and *Port* as *80*.
|*Path*|Set the path as `sites/all/themes/mulesoft_community/logo.png`
|*Method*|GET
|===
+
. Drag another Amazon S3 connector to create the above requested MuleSoft logo in the selected Amazon S3 Bucket.
+
image:ams3_08.png[ams3_08]
+
[%header,cols="30a,70a",width=80%]
|===
|Field|Value
|*Display Name*|Enter a name of your choice.
|*Connector Configuration*|Select the global configuration that you created.
|*Operation*|Create object
|*Bucket Name*|`${config.bucket}`
|*Key*|mulesoft.png
|*Content Reference*|`#[payload]`
|===
+
. Add another Amazon S3 connector to get the newly created MuleSoft logo image object from the bucket:
+
image:ams3_09.png[ams3_09]
+
[%header,cols="30a,70a",width=80%]
|===
|Field|Value
|Display Name|Enter a name for the connector instance.
|Connector Configuration|Select the global configuration you create.
|Operation|Get object content
|Bucket Name|`${config.bucket}`
|Key|mulesoft.png
|===
+
. Finally, add another Amazon S3 connector to delete the bucket. Since delete bucket operation’s return type is void, the payload contains the object returned by the get image operation.

image:ams3_10.png[ams3_10]

=== Using XML

For this code to work in Anypoint Studio, you must provide Amazon Web Services credentials. You can either replace the variables with their values in the code, or you can provide the values for each variable in the src/main/app/mule-app.properties file.

[source,xml, linenums]
----
<?xml version="1.0" encoding="UTF-8" ?>
<mule xmlns:s3="http://www.mulesoft.org/schema/mule/s3" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core
http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http
http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ee/tracking
http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.mulesoft.org/schema/mule/s3
http://www.mulesoft.org/schema/mule/s3/current/mule-s3.xsd" >

<http:listener-config name="HTTP_Listener_Configuration" host="0.0.0.0" port="8081" doc:name="HTTP Listener Configuration" />
<http:request-config name="HTTP_Request_Configuration" host="mulesoft.org" port="80" doc:name="HTTP Request Configuration" />
<s3:config name="Amazon_S3_Configuration" accessKey="${config.accessKey}" secretKey="${config.secretKey}" doc:name="Amazon S3: Configuration" />
  <flow name="s3-example-flow" >
    <http:listener config-ref="HTTP_Listener_Configuration" path="/" doc:name="HTTP" />
    <s3:create-bucket config-ref="Amazon_S3_Configuration" bucketName="${config.bucket}" acl="PUBLIC_READ" doc:name="Create S3 Bucket" />
    <http:request config-ref="HTTP_Request_Configuration" path="sites/all/themes/mulesoft_community/logo.png" method="GET" doc:name="Get MuleSoft logo" />
    <s3:create-object config-ref="Amazon_S3_Configuration" doc:name="Create logo object in S3 bucket" acl="PUBLIC_READ" bucketName="${config.bucket}" key="mulesoft.png" />
    <s3:get-object-content config-ref="Amazon_S3_Configuration" bucketName="${config.bucket}" key="mulesoft.png" doc:name="Get Image" />
    <s3:delete-bucket config-ref="Amazon_S3_Configuration" bucketName="${config.bucket}" force="true" doc:name="Delete S3 Bucket" />
  </flow>
</mule>
----


== See Also

* Learn more about working with link:/mule-user-guide/v/3.7/anypoint-connectors[Anypoint Connectors]
* link:/release-notes/amazon-s3-connector-release-notes[Amazon S3 Connector Release Notes]
* Learn about link:/mule-user-guide/v/3.7/using-maven-with-mule[Using Maven with Mule]
* link:/mule-user-guide/v/3.7/mule-transformers[Mule Transformers]