Nav

DataSense

DataSense uses message metadata to facilitate application design. With this functionality, Anypoint Studio proactively acquires information such as data type and structure, in order to prescribe how to accurately map or use this data in your application.

Rather than forcing you to manually discover information about the data your internal and external resources employ, Mule intelligently discovers this information and uses it to make application design suggestions. For example, if your application connects to Twitter, DataSense captures information about the type and structure of data that Twitter employs, then uses the information to advise you of the type of data you should send to, or expect to receive from Twitter.

Prerequisites

This document assumes that you are familiar with the Visual Editor in Anypoint Studio, the Anypoint Connectors, and the DataWeave transformer that you can use to create applications in Studio. Further, this document references Connectors and Global Elements.

DataSense Functionality

Studio offers these DataSense features to support proactive discovery of metadata from external resources:

  • Perceptive Flow Design

  • DataSense Explorer

Perceptive Flow Design

Mule takes one more step to make designing your application easier with Perceptive Flow Design. If you model a flow to include an Anypoint Connector with a working configuration selected, preceded or followed by a DataWeave (Transform Message) component, Mule uses the working connection to the resource – a SaaS provider, for example, to retrieve metadata about message payload and properties. Capturing the data type and structure that a SaaS employs, Mule feeds that data into DataWeave, thereby providing DataWeave with the expected input or output. This functionality facilitates mapping of data from one data format and structure to another which saves you the development time you would otherwise spend trying to discover the structure and format yourself.

For example, imagine you are tasked with the job of connecting your organization’s Salesforce accounts with Twitter to publicize specific performance indicators. One of your first tasks would be to examine the available documentation from both SaaS providers to determine what type of data structures they use so that you can, eventually, figure out how to map one structure to the other. Rather than plod through that exhausting exercise, you can utilize Perceptive Flow Design. Drop two connectors onto your Studio canvas – Salesforce and Twitter – then configure them, making sure to test the connection for Salesforce. After confirming connectivity, simply drop a DataWeave Transformer between the connectors. Open DataWeave to find that Mule has intelligently captured the data type and structure information from each SaaS provider, and prescribed the input and output for your data mapping. With the prescription in place, all you need to do is configure the mapping, filling in the blanks of the DataWeave code.

dataweave-1

As an even simpler example, imagine you are obtaining status update data from a Twitter connector and that you want to log the tweet text but you aren’t familiar with the property names used by this connector. Instead of having to look up Twitter’s documentation to find out what arbitrary name they picked for the property you need, you can just place a Logger message processor right after the Twitter connector and write in it. If you start writing #[payload. in the logger’s Message field, and then press ctrl + space bar, you will get a list of all the properties and methods associated to the payload, including the properties returned by the request that you’re performing on the Twitter connector.

payload+autocomplete

DataSense Explorer

This handy feature in Studio allows you to preview the Message data structure at different points of the flow, while still in design time. Select a specific element in the flow, then click the DataSense Explorer icon (shown below) and explore the structure of the data that enters and leaves it.

datasense+logo

The DataSense Explorer allows you to navigate the Payload, Flow and Session Variables, Inbound and Outbound Properties, and Record Variables. It takes the known output fields of flow elements and the DataSense metadata of compatible connectors into account to predict the message contents at any given point in the flow. You can easily find what information is available where, even the variables that are implicitly created by certain flow elements which you might not be aware of.

datasense+explorer2-1

DataSense Best Practices

To take full advantage of the functionality DataSense has to offer, design-time best practice dictates that you should build an application in the following order:

  1. Configure the endpoints and connectors — and the global configurations they reference — so that, where possible, you can establish connections and use DataSense to extract any metadata about the expected input or output with the external source. If a connector itself doesn’t expose any metadata about its output, you can specify it through the custom metadata tab of the connector.

  2. Insert other elements into your flow to enrich the message, set, copy or remove variable or properties, route messages, etc.

  3. Lastly, insert any DataWeave transformers into your flow to adjust the format or structure of the message so that each connector receives the input it expects.

The objective of this design-time strategy is to set the pieces of the integration puzzle in place, then "glue them together" with DataWeave transformers. Rather than designing a flow sequentially, this type of "align, then glue together" strategy ensures that you are utilizing DataSense, wherever possible, to pre-populate the information about the structure and format of the input or output data into a DataWeave transformer.

For example, if you configure a Salesforce connector in your application, then drop a Transform Message component after it, this component uses the information that DataSense extracted to pre-populate the input values for mapping, so that you only need to drag these inputs fields over to output fields to construct the required mapping. In other words, DataSense makes sure that the Transform Message component knows the data format and structure with which it must work so you don’t have to figure it out manually. Whenever you can use DataSense, you are taking a development shortcut, removing the hassle of manually discovering the format and structure of data contained within a message.

See Also