Nav

Managing Application Data Storage with Object Stores

logo cloud active logo hybrid disabled logo server disabled logo pcf disabled

CloudHub includes application data storage for each application it manages, this data is accessible through Mule’s Object Store functionality. This storage can be used to store anything, but two of its main uses are:

  • Storing synchronization state:  Your application may need to keep track of the last synchronized record, so that the next time the application synchronizes, it can resume where it left off.

  • OAuth tokens: OAuth enabled connectors can store tokens inside an Object Store with a few lines of configuration code.

Your application can also store custom integration data as well, within the limits outlined below.

Notes:

  • Object stores are handled in the way described in this document only for applications deployed to CloudHub workers. For applications that you deploy via the Runtime Manager to local servers, you must handle the storage of data in your own repository. See Deployment Strategies for more details.

  • All data in an object store is handled by servers in the US-East zone, regardless of what zone your application is deployed to. This may impact performance.

Storing and Retrieving Data

To work with application data inside a Mule flow, use the Object Store connector. The connector provides operations to store, retrieve, list, and remove data from a Runtime Manager Object Store.

Note: You must specify a unique Partition name in the connector, and to view information in Runtime Manager > Application Data, your connector project must contain an Object Store Reference to the _defaultUserObjectStore Spring bean instance.

See:

Storing OAuth Data

Each connector contains instructions on how to use the Object Store module to store and retrieve OAuth tokens. See the "Saving and Restoring State" section in the LinkedIn connector configuration documentation for an example.

Viewing Application Data

To view application data, click your application name in Runtime Manager, click Manage Application, and click the Application Data tab. This tab contains a list of keys and values. You can view the data as plain text, or as a list.

applicationdata

Tip: To make properties secure so that they’re not visible using the Runtime Manager UI or anywhere else, see Secure Application Policies.

Semantics and Storage Limits

The Runtime Manager object store provides read-after-write consistency and the last write wins in terms of updates. 

Additionally, Runtime Manager object store places these limits on your data:

  • A maximum of 100,000 keys per application

  • A 768 byte key size

  • A 1 MB value size

  • 1 GB of data per application

Advanced Information

Internally, Mule has several different types of object stores. Runtime Manager provides an implementation of the user defined object store. This is for runtime information about your application, such as those use cases mentioned above. If you wish to access this object store directly, it can be found internally in the Mule Registry under the _defaultUserObjectStore. The Object Store module and connectors use this store to store all their data by default.

Object Store Example

To write data to the Runtime Manager > Application Data object store:

  1. In Anypoint Studio, create a new Mule Project. This example names the application objectstoredemo.

  2. Create an application with at least these components:

    objectstore-flow

    • HTTP connector - Receives the data sent by a curl command.
      Settings: Host: 0.0.0.0, Port: 8081, Path (in Basic Settings) to /store, Allowed Methods set to POST

    • Object to String - Converts the input to a String.
      Settings: Leave fields empty.

    • JSON to Object - Converts the String to a Java Map object.
      Settings: Set Return Class to java.util.Map

    • ObjectStore - Stores data in the Runtime Manager Application Data area.

      Settings:

      In the plus menu:

      • You must specify a Partion Name, which can be any unique name.

      • You must set the Object Store Reference to the _defaultUserObjectStore value.

      • If you specify a value for entryTtl, values for expirationInterval and maxEntries are also required. You can set entryTtl to 0 for never expire. For Operation: Store, Key: #[payload.key], and Value Reference: #[payload.value]

    • Set Payload - Displays a confirmation prompt that appears after you use the curl command.
      Settings: Set Settings to KEY: #[payload.key] AND VALUE: #[payload.value] WERE SAVED OK

      The source for this example is:

      
                     
                  
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      
      <?xml version="1.0" encoding="UTF-8"?>
      
      <mule xmlns:objectstore="http://www.mulesoft.org/schema/mule/objectstore"
      xmlns:json="http://www.mulesoft.org/schema/mule/json"
      xmlns:http="http://www.mulesoft.org/schema/mule/http"
      xmlns="http://www.mulesoft.org/schema/mule/core"
      xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
      xmlns:spring="http://www.springframework.org/schema/beans"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://www.springframework.org/schema/beans
      http://www.springframework.org/schema/beans/spring-beans-current.xsd
      http://www.mulesoft.org/schema/mule/core
      http://www.mulesoft.org/schema/mule/core/current/mule.xsd
      http://www.mulesoft.org/schema/mule/json
      http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
      http://www.mulesoft.org/schema/mule/http
      http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
      http://www.mulesoft.org/schema/mule/objectstore
      http://www.mulesoft.org/schema/mule/objectstore/current/mule-objectstore.xsd">
          <http:listener-config name="HTTP_Listener_Configuration" host="0.0.0.0"
           port="8081" doc:name="HTTP Listener Configuration"/>
          <objectstore:config name="ObjectStore__Connector" partition="mypartition"
          entryTtl="0" expirationInterval="42" maxEntries="42"
          objectStore-ref="_defaultUserObjectStore" doc:name="ObjectStore: Connector"/>
          <flow name="objectstoreFlow">
              <http:listener config-ref="HTTP_Listener_Configuration"
              path="/store" doc:name="HTTP"/>
              <object-to-string-transformer doc:name="Object to String"/>
              <json:json-to-object-transformer returnClass="java.util.Map"
              doc:name="JSON to Object"/>
              <objectstore:store config-ref="ObjectStore__Connector"
              key="#[payload.key]" value-ref="#[payload.value]" doc:name="ObjectStore"/>
              <set-payload value="KEY: #[payload.key] AND VALUE: #[payload.value] WERE STORED__"
              doc:name="Set Payload"/>
          </flow>
      </mule>
  3. Run this application as a Mule application to test it. After the application works, you can stop the application.

  4. Export the application and save it as a zip file on your computer. Right-click the project name in Package Explorer, click Export > Mule > Anypoint Studio Project to Mule Deployable Archive.

  5. Log into Anypoint Platform and click Runtime Manager. Deploy the application and use the zip file that you exported to your computer for the deployed application.

  6. Start the deployed application.

  7. From a command prompt, run this curl command, which sends a JSON-formatted key and value to the http://objectstoredemo.cloudhub.io/store example URL:

    curl -X POST -H "Content-Type: application/json" -d '{
        "key": "Test",
        "value": "Test sent to the object store"
    }' "+http://objectstoredemo.cloudhub.io/store+"
  8. In Runtime Manager, click the application’s name and click Application Data. The data from the curl command displays in the Application Data page:

    objectstore-application-data