Nav

Managing Application Data Storage with Object Stores

logo cloud active logo hybrid disabled logo server disabled logo pcf disabled

CloudHub includes application data storage for each application it manages, this data is accessible through Mule’s Object Store functionality. This storage can be used to store anything, but two of its main uses are:

  • Storing synchronization state:  Your application may need to keep track of the last synchronized record, so that the next time the application synchronizes, it can resume where it left off.

  • OAuth tokens: OAuth enabled connectors can store tokens inside an Object Store with a few lines of configuration code.

Your application can also store custom integration data as well, within the limits outlined below.

Object stores are handled in the way described in this document only for applications deployed to CloudHub workers. For applications that you deploy via the Runtime Manager to local servers, you must handle the storage of data in your own repository. See Deployment Strategies for more details.
Currently, all data in an object store is handled by servers in the US-East zone, regardless of what zone your application is deployed to. This may impact performance.

Storing and Retrieving Data

To work with application data inside a Mule flow, use the Object Store connector. The connector provides operations to store, retrieve, list, and remove data from a CloudHub Object Store.

Storing OAuth Data

Each connector contains instructions on how to use the Object Store module to store and retrieve OAuth tokens. See the "Saving and Restoring State" section in the LinkedIn connector configuration documentation for an example.

Viewing Application Data

To view application data, click your application name in Runtime Manager, click Manage Application, and click the Application Data tab. This tab contains a list of keys and values. You can view the data as plain text, or as a list.

applicationdata

To make properties secure so that they’re not visible using the Runtime Manager UI or anywhere else, see Secure Application Policies.

Semantics and Storage Limits

The CloudHub object store provides read-after-write consistency and the last write wins in terms of updates. 

Additionally, CloudHub object store places these limits on your data:

  • A maximum of 100,000 keys per application

  • A 768 byte key size

  • A 1 MB value size

  • 1 GB of data per application

Advanced Information

Internally, Mule has several different types of object stores. CloudHub provides an implementation of the user defined object store. This is for runtime information about your application, such as those use cases mentioned above. If you wish to access this object store directly, it can be found internally in the Mule Registry under the _defaultUserObjectStore. The Object Store module and connectors use this store to store all their data by default.

Object Store Example

To write data to the Runtime Manager’s Application Data object store:

  1. In Anypoint Studio, create a new Mule Project. This example names the application objectstoredemo.

  2. Create an application with at least these components:

    objectstore-flow

    • HTTP connector - Receives the data sent by a curl command.
      Settings: Host: 0.0.0.0, Port: 8081, Path (in Basic Settings) to /store, Allowed Methods set to POST

    • Object to String - Converts the input to a String.
      Settings: Leave fields empty.

    • JSON to Object - Converts the String to a Java Map object.
      Settings: Set Return Class to java.util.Map

    • ObjectStore - Stores data in the Runtime Manager Application Data area.
      Settings: Set Connector Configuration: ObjectStore_Configuration, in the plus menu leave fields empty, Operation: Store, Key: #[payload.key], Value Reference: #[payload.value]

    • Set Payload - Displays a confirmation prompt that appears after you use the curl command.
      Settings: Set Settings to KEY: #[payload.key] AND VALUE: #[payload.value] WERE SAVED OK

      The source for this example is:

      
                     
                  
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      
      <?xml version="1.0" encoding="UTF-8"?>
      
      <mule xmlns:json="http://www.mulesoft.org/schema/mule/json"
              xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:objectstore="http://www.mulesoft.org/schema/mule/objectstore" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
              xmlns:spring="http://www.springframework.org/schema/beans"
              xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xsi:schemaLocation="
      http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
      http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
      http://www.mulesoft.org/schema/mule/objectstore http://www.mulesoft.org/schema/mule/objectstore/current/mule-objectstore.xsd
      http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd">
          <http:listener-config name="HTTP_Listener_Configuration" host="0.0.0.0" port="8081" doc:name="HTTP Listener Configuration"/>
          <objectstore:config name="ObjectStore__Connector" doc:name="ObjectStore: Connector"/>
          <flow name="objstoredemoFlow">
              <http:listener config-ref="HTTP_Listener_Configuration" path="/store" doc:name="HTTP"/>
              <object-to-string-transformer doc:name="Object to String" />
              <json:json-to-object-transformer returnClass="java.util.Map" doc:name="JSON to Object" />
             <objectstore:store config-ref="ObjectStore__Connector" doc:name="ObjectStore" key="#[payload.key]" value-ref="#[payload.value]"/>
             <set-payload value="KEY: #[payload.key] AND VALUE: #[payload.value] WERE SAVED OK" />
          </flow>
      </mule>
  3. Run this application as a Mule application to test it. After the application works, you can stop the application.

  4. Export the application and save it as a zip file on your computer.

  5. Log into Anypoint Platform and click Runtime Manager. Deploy the application and use the zip file that you exported to your computer for the deployed application.

  6. Start the deployed application.

  7. From a command prompt, run this curl command, which sends a JSON-formatted key and value to the http://objectstoredemo.cloudhub.io/store example URL:

    curl -X POST -H "Content-Type: application/json" -d '{
        "key": "Test",
        "value": "Test sent to the object store"
    }' "http://objectstoredemo.cloudhub.io/store"

    The JSON for the payload to send in this example is the same as specifying:

    
                
             
    1
    2
    3
    4
    
    {
      “key”:”Test”,
      “value”:"Test sent to the object store”
    }
  8. In Runtime Manager, click the app name and click Application Data. The data from the curl command is now in the Application Data page:

    objectstore-application-data