Contact Us 1-800-596-4880

Veeva Vault Connector 1.2 Additional Configuration Information - Mule 4

Connector Configuration

image

Follow the instructions in Connection Configuration.

Create Documents

The Create Documents operation enables you to create single or multiple documents with the provided document metadata in CSV or JSON format. Make sure you have uploaded document files on the vault FTP server location with the document content to attach to the created documents.

Create Documents Configuration

veevavault connector create documents
Field Description

Metadata Format

Select CSV or JSON to specify the format for the document metadata.

Document Metadata

Document Metadata payload with the required metadata in CSV or JSON format. Ensure the payload is in the format specified in the Metadata Format field.

Input

The following shows the document metadata required to create documents on the vault. Ensure the payload has the mandatory metadata required for the vault.

veevavault connector create documents input

Output

The Create Documents operation output status or response from the vault.

veevavault connector create documents output

Note: Check the Success and Failure response status of operations in Connector Operations SUCCESS and FAILURE Response.

Get Documents

The Get Documents operation enables you to retrieve document details based on the document type, subtype and classification selected. Using the selected type, subtype, and classification it fetches document properties metadata and builds VQL queries dynamically. The VQL query is then executed on the vault to retrieve document details. Place a For-Each/Splitter element after this operation to fetch each document’s data (page) sequentially.

This operation provides a paging mechanism based on Mule standard pagination. See Streaming and Pagination for more details.

The document type, subtype, and classification are optional. If none are selected then document properties metadata is fetched using API (/api/{version}/metadata/objects/documents/properties) and the VQL Query is built based on the document metadata properties added or inserted in the Document Properties list. If metadata properties are not provided then all queryable document properties are used in VQL and executed.

Get Documents Configuration

image
Fields Description

Type (Optional)

Click the refresh icon (VeevaVault refresh icon) to retrieve Document Types from the vault. Select a document type from the list to retrieve document properties.

Subtype (Optional)

Click the refresh icon (VeevaVault refresh icon) to retrieve Document Subtypes from the vault. Select a document subtype from the list to retrieve document properties.

Classification (Optional)

Click the refresh icon (VeevaVault refresh icon) to retrieve Document Classification from the vault. Select a document classification from the list to retrieve document properties.

Document Properties (Optional)

Insert document properties per your business requirements. These properties will be used to build a VQL query to execute on the vault.

WHERE Clause

Insert a VQL WHERE clause (without using keyword WHERE) conditions as per business requirement and it will be appended to dynamically to build VQL query to execute on vault, for example: classificationv = ‘study_set_up_worksheetc’

Fetch Size

Provide the number of records per page. Default is 1000.

Batch Size

Provide the number of pages per batch. Default is 10.

Input

N/A

Output

You can find a list of queryable document properties retrieved from the vault. If a document properties list is provided, then the specified properties detail is retrieved and results in operation output. The default is a list of all queryable properties.

image

Delete Documents

The Delete Documents operation enables you to delete single or multiple documents using the bulk API with input metadata in CSV or JSON format.

Delete Documents Configuration

image
Field Description

Metadata Format

Select CSV or JSON to specify the format for the document metadata.

Document Metadata

Document Metadata payload with the required metadata in CSV or JSON format (same as the specified value in Metadata Format).

Input

The following shows the list of document metadata that’s required to delete documents on the vault.

image

Output

The delete documents operation output status or response from vault:

image

Note: Check the Success and Failure response status of operations in Connector Operations SUCCESS and FAILURE Response.

Update Documents

The Update Documents operation enables you to update bulk documents with the provided editable metadata in the payload in CSV or JSON format.

Update Documents Configuration

image
Field Description

Metadata Format

Select CSV or JSON to specify the format for the document metadata.

Document Metadata

Document metadata payload with the required editable metadata in CSV or JSON format. The payload should be in the format specified in the Metadata Format field.

Input

The following is the list of editable document metadata to update documents on the vault. The payload should have only the editable metadata for that vault.

image

Output

Create documents operation output status or response from the vault.

image

Export Documents

An Export Documents operation enables you to query a set of documents for export to your vault’s FTP staging server. It is recommended that you use the Export Documents operation in a separate asynchronous flow using Async Scope and passing the payload with document IDs.

When used asynchronously the operation executes in separate flow thread and waits for the specified polling interval to poll the Job status until it is successful while the main flow continues its process.

The exported documents are stored on the FTP staging area in the folder, with the hierarchy structure:
{root}/{userId}/{jobId}/{documentIds}/{versions}.

The following is an example of the Export Documents operation returning job results as response:

[
  {
	"file": "/41601/249051/0_1/TestDocument.docx",
	"user_id__v": 1885110,
	"id": 249051,
	"responseStatus": "SUCCESS",
	"minor_version_number__v": 1,
	"major_version_number__v": 0
  },
  {
	"file": "/41601/249050/0_1/TestDocument.docx",
	"user_id__v": 1885110,
	"id": 249050,
	"responseStatus": "SUCCESS",
	"minor_version_number__v": 1,
	"major_version_number__v": 0
  },
  {
	"file": "/41601/249052/0_1/TestDocument.docx",
	"user_id__v": 1885110,
	"id": 249052,
	"responseStatus": "SUCCESS",
	"minor_version_number__v": 1,
	"major_version_number__v": 0
  }
]

The previous example shows the following attributes:

  • file
    Has the format: "/{jobId}/{documentId}/{major-minor-version}/{filename}", which is the absolute file path location on the Vault FTP server staging area.

  • user_id__v
    Vault system user ID of a user whose credentials are configured in the connector. The folder is created on the Vault FTP server staging area root location under where the exported file is located. The folder is created with the name of the user ID prefixed with the letter ‘u’; for example:
    /uXXXXXXX/{jobId}/{documentId}/{major-minor-version}/{filename}\)

  • id
    Document ID of the exported document file.

  • major_version_number__v
    Major version number of the exported document file

  • minor_version_number__v
    Minor version number of the exported document file.

Export Documents Configuration

image
Field Description

Metadata Format

Select CSV or JSON to accept metadata for the specified document ID in either CSV or JSON format.

Document Metadata

Document Metadata payload contains document IDs in either CSV or JSON format. The payload should be in the format specified in the Metadata Format field.

Source

(Optional) To exclude source files, set to false. Default is true.

Renditions

(Optional) To include renditions, set to true. Default is false.

All Versions

(Optional) To include all versions, or the latest version, set to true. Default is false.

Polling Interval

(Optional) Poll the vault at specified intervals (in seconds) until the job is successful. Default is 30 seconds.

Input

Input document metadata required to export documents from the vault to the FTP staging server. Make sure the payload has the mandatory metadata required for the vault.

veevavault connector export documents input

Output

The export documents job status response is under the Output tab:

veevavault connector export documents output

Note: Check the Success and Failure response status of operations in Connector Operations SUCCESS and FAILURE Response.

Create Document Renditions

The Create Document Renditions operation enables you to add document renditions in bulk. Your vault must be in Migration Mode before using this operation. It takes CSV input data with a maximum size of 1 GB with a standard UTF-8-encoded value.

Create Document Renditions Configuration

veevavault connector create document renditions

Field Description

Metadata Format

Select CSV or JSON to specify the format for the document metadata.

Document Metadata

Document metadata payload with required metadata in CSV or JSON format. The payload must be in format specified in the Metadata Format field.

Input

Document metadata required in payload under the Input tab:

veevavault connector create document renditions input

Output

Operation output or response under the Output tab:

veevavault connector create document renditions output

Note: Check the Success and Failure response status of operations in Connector Operations SUCCESS and FAILURE Response.

Get Document Renditions Types

The Get Document Renditions Types operation enables you to retrieve document renditions types details.

Get Document Renditions Types Configuration

image
Field Description

Document Metadata

Document ID in the payload in either in CSV or JSON format.

Input

Document ID required in the payload is under the Input tab of the retrieve document renditions details operation:

image

Output

The following operation output or response is under the Output tab:

image

Note: Check the Success and Failure response status of operations in Connector Operations SUCCESS and FAILURE Response.

Delete Document Renditions

The Delete Document Renditions operation enables you to delete document renditions in bulk using CSV or JSON format input metadata.

Delete Document Renditions Configuration

image
Field Description

Metadata Format

Select CSV or JSON to specify the format for the document metadata.

Document Metadata

Document Metadata payload with the required metadata in CSV or JSON format (specified in the Metadata Format field).

Input

Document metadata required to create documents on the vault. Make sure the payload has the mandatory metadata required for the vault.

image

Output

Delete document renditions operation output status or response from the vault:

image

Note: Check the Success and Failure response status of operations in Connector Operations SUCCESS and FAILURE Response.

Create Object Records

The Create Object Records operation enables you to create bulk object records for the selected vault object with the provided object metadata in CSV or JSON format.

Create Object Records Configuration

image
Field Description

Object Name

Click the VeevaVault refresh icon (VeevaVault refresh icon) to retrieve the vault object list. Select the object name from the list to create records for the selected object.

Metadata Format

Select CSV or JSON to accept object metadata in the specified format.

Object Metadata

Object field’s payload with the required metadata in CSV or JSON format. Make sure the payload is in the format specified in the Metadata Format field.

Input

The following shows the list of object metadata required to create object records:

image

Output

Operation output or responses retrieved from the vault for a created object record.

image

Note: Check the Success and Failure response status of operations in Connector Operations SUCCESS and FAILURE Response.

Get Object Records

The Get Object Records operation enables you to retrieve object records details using object metadata fields to build a VQL query dynamically and execute on vault to get object details. Place a For-Each/Splitter element after this operation to fetch each object records (page) sequentially.

This operation provides a paging mechanism based on Mule standard pagination. See Streaming and Pagination for more details.

Get Object Records Configuration

image
Fields Description

Vault Object

Click the refresh icon (VeevaVault refresh icon) to retrieve a vault objects list from vault. Select an object from the list to retrieve the object records detail.

Object Fields

(Optional) Insert object fields per your business requirements. These fields will be used to build a VQL query to execute on the vault.

WHERE Clause

Insert a VQL WHERE clause (without using the keyword WHERE) conditions as per your business requirement and it will be appended to dynamically build a VQL query to execute on the vault. For example: study_country__v = ‘0SC000000000101’

Fetch Size

Provide the number of records per page. Default is 1000.

Batch Size

Provide the number of pages per batch. Default is 10.

Input

N/A

Output

The list of object records fields to add into the Object fields list. If the object fields list is provided, then the specified fields detail is returned as output. Default is a list of all fields.

image

Delete Object Records

The Delete Object Records operation enables you to delete single or multiple object records with the bulk API using CSV or JSON format input metadata.

Delete Object Records Configuration

image
Field Description

Object Name

Click the refresh icon (VeevaVault refresh icon) to retrieve vault object list. Select the object name from the list to delete object records for the selected object.

Metadata Format

Select CSV or JSON to accept document metadata in the specified format.

Object Metadata

Object fields payload with the required metadata in CSV or JSON format. Make sure the payload is in the same format specified in the Metadata Format field.

Input

List of object metadata required to delete object records:

image

Output

Operation output or response retrieved from vault for an object records deleted:

image

Note: Check the Success and Failure response status of operations in Connector Operations SUCCESS and FAILURE Response.

Update Object Records

The Update Object Records operation enables you to update object records of specified vault object.

Update Object Records Configuration

image
Field Description

Object Name

Click the refresh icon (VeevaVault refresh icon) to retrieve the vault object list. Select the object name from the list to update the object’s records.

Metadata Format

Select CSV or JSON to accept object metadata in the specified format.

Object Metadata

Object fields payload with the required metadata in CSV or JSON format. Make sure the payload is in the same format that is specified in the Metadata Format field.

Input

List of object metadata required to create object records:

image

Output

Object status of an object updated as an output or response retrieved from the vault is under the Output tab of the operation.

image

Note: Check the Success and Failure response status of operations in Connector Operations SUCCESS and FAILURE Response.

Query

The Query operation enables you to execute the specified VQL query on the vault and retrieve the result in a paginated data input stream. Place a For-Each/Splitter element after the Query operation to fetch each record in sequential order.

Note: Do not specify LIMIT and OFFSET in the VQL query. These parameters are incorporated internally.

This operation provides a paging mechanism based on Mule standard pagination. See Streaming and Pagination for more details.

Query Configuration

image
Field Description

VQL Query

Enter a VQL query to send it to the vault.

Note: Do not specify the LIMIT and OFFSET in the VQL query.

Input Parameters

The input parameters pass parameters to a VQL query dynamically. The parameters must be a name-value pair. Payload or variable values can be passed; for example:

#[
	{
		'table': 'site__v',
	}
]

Default parameter is #[{}]

Fetch Size

Provide the number of records per page. Default is 1000.

Batch Size

Provide the number of pages per batch. Default is 10.

Input

N/A

Output

You can find the specified VQL query fields in the Output tab of the operation. It returns the specified VQL query and the result contains pagination data.

image

Use Input Parameters to Protect VQL Queries

The Query operation is used to retrieve documents, object, and workflow information from the vault. The primary concept of this operation is to supply a VQL query and use DataWeave to supply the parameters.

In the previous example, input parameters are supplied as key-value pairs, which you can create by embedding a DataWeave script. Those keys are used in conjunction with the colon character (:) to reference a parameter value by name. This is the recommended approach for using parameters in your VQL query.

The alternative is to directly write <veevavault:vql>SELECT id, namev, studyv FROM documents WHERE name__v = #[payload] </veevavault:vql>, but this is a very dangerous practice that is not recommended.

The following are the advantages of using input parameters to configure the WHERE clause in a SELECT statement:

  • The query becomes immune to VQL injection attacks.

  • The connector can perform optimizations that are not possible otherwise, which improves the app’s overall performance.

Get Picklists

The Get Picklists operation enables you to retrieve all available picklist values configured on a picklist.

Get Picklists Configuration

image
Field Description

Picklist Name

Click the refresh icon (VeevaVault refresh icon) to retrieve picklists from the vault. Select a picklist name from the list to retrieve picklist values.

Input

N/A

Output

You can find picklist values with name and label in the Output tab of the Get Picklists operation.

image

Get Audit Details

The Get Audit Detail operation enables you to retrieve audit details for the specified audit type. This operation provides a paging mechanism based on Mule standard pagination.

See Streaming and Pagination for more details.

Get Audit Details Configuration

image

Field Description

Audit Type

Click the refresh icon (VeevaVault refresh icon) to retrieve audit types from the vault. Select an audit type from the list whose audit details you need to retrieve.

Start date

(Optional) Specify the start date to retrieve audit information. This date cannot be more than 30 days in the past. Dates must be in YYYY-MM-DDTHH:MM:SSZ format. Dates and times are in UTC. If a time is not specified, it defaults to midnight (T00:00:00Z) on the specified date.

End date

(Optional) Specify the end date to retrieve audit information. This date cannot be more than 30 days in the past. Dates must be in YYYY-MM- DDTHH:MM:SSZ format. Dates and times are in UTC. If the time is not specified, it defaults to midnight (T00:00:00Z) on the specified date.

Fetch Size

Provide the number of records per page. Default is 1000.

Batch Size

Provide the number of pages per batch. Default is 10.

Input

N/A

Output

The following selected audit type details are retrieved from the vault in pagination format. Make sure to use a For each/Splitter element to fetch each record.

image

Spark Validator

The Spark Validator operation validates and verifies Spark messages triggered from the vault with a signature and public key. Upon successful validation and verification, the Spark message can be further processed in the Mule flow according to your business requirements. See Spark Messaging feature and functionalities and setup for more details.

The output of the Spark Validator operation includes Spark attributes (optional) with the Spark message as it is received from Vault.

Note: Before triggering a Spark message from the Vault make sure to raise a support ticket with Veeva Vault Support to increase the time interval (for example, 10 seconds) for the Spark message to resend or retry. By default, it is configured as 3 seconds, which is less than the time required for Spark Validator to complete message validation and verification.

Note: As per Veeva Systems the public key (00001.pem) expires every two years. You receive notification in advance advising you of the expiration date. If you are using this operation in a Mule application flow, after the public key expiration date you need to remove the public key file from the path configured in the Public Key Path field. A new public key will be created during execution of the key removal. If you don’t do this, Spark message verification fails as INVALID_SPARK_MESSAGE. The Mule application deployed on CloudHub with the default path must be restarted for successful Spark Message verification.

Spark Validator Configuration

image
Field Description

Include Spark Header

Flag to include Spark headers received from the vault in Spark Validator output. Default is FALSE.

Public Key Path

The path to where the public key (00001.pem) file is saved that will be used for verifying incoming Spark messages with a signature. Default is “${mule.home}/apps/${app.name}/publickeys”

Spark Headers

The Spark headers received from the vault. Default is #[attributes.headers]

Spark Message

The Spark message body received from the vault. Default is #[payload].

Input

Inputs required for Spark Validator:

image::veevavault-connector-spark-validator-input.png[image,width=285,height=330]

Output

Output returns from the Spark Validator operation:

image::veevavault-connector-spark-validator-output.png[image,width=280,height=265]

Invoke Rest API

The Invoke Rest API operation enables you to invoke Veeva Vault Rest APIs. The response returned is either in JSON (default) or XML format, depending on the header Accept value provided.

Invoke Rest API Configuration

image

Field Description

Method

Method to invoke the Veeva Vault Rest API. The default is GET.

Path

URI path to invoke the Veeva Vault Rest API. For example: /objects/documents/{doc_id}

Body

Body to send with a request to invoke Veeva Rest API. Default is #[payload].

Headers

Headers in key-value format to send along with the request. If there are no required headers in the request, leave empty. The default is Empty list.

URI Parameters

URI Parameters in key-value format to resolve the value in the path. If there are no URI parameters in the path, leave empty. The default is an Empty List.

Query Parameters

Query parameters in key-value format to send along with the request. If there are no query parameters required in the request, leave empty. The default is Empty list.

Request Streaming Mode

Whether to use streaming mode for the request. By default, if the type of the payload is a stream, streaming is used to send the request. Select the streaming option from the drop-down list.

Input

N/A

Output

Response either in JSON (default) or XML format, depending on the specified header Accept value provided in Headers section.

A typical use of the Invoke Rest API operation is to invoke the Veeva Vault Rest API using the default GET method. By default, GET methods do not send the payload in the request; the body of the HTTP request is empty. The other methods send the message payload as the body of your request.

After sending a request, the connector receives the response either in JSON (default) or XML format, depending on the header Accept value provided. The payload is then passed to the next element in your application’s flow.

Add Parameters

By default, the Invoke Rest API operation sends the Mule message payload as the request body, but you can customize it using a DataWeave script or expression. In addition to the body of the request, you can configure:

  • Headers

  • Query parameters

  • URI parameters

Headers

In General > Request > Query Parameters, click the plus icon (+) to add headers to the request. For example, add header names HeaderName1 and HeaderName2, and header values HeaderValue1 and HeaderValue2.

You can use DataWeave expressions, for example:

#[{'HeaderName1' : 'HeaderValue1', 'HeaderName2' : 'HeaderValue2'}].

URI Parameters

You configure a URI parameter when you want to use a placeholder, such as /objects/documents/{doc_id}, in the path of your request. To configure a URI parameter, type the placeholder enclosed in curly brackets in the Path field. Select URI Parameters, and click the plus sign (+), to enter a name and value.

Query Parameters

In General > Request > Query Parameters, click the plus icon (+) to add a parameter to a request. Type a name and value for the parameter or use a DataWeave expression to define the name and value.

Sending Form Parameters in a POST Request

To send parameters in a POST request:

  1. In General > Request, select the POST method.

  2. In Body, construct the payload of the Mule message.

    Following are some examples:

    1. Enter the names and the values of the parameters to send for application/x-www-form-urlencoded:

      #[output application/x-www-form-urlencoded --- {'key1':'value1', 'key2':'value2'}]
    2. In multipart/form-data, ensure the header Content-Type value contains a boundary attribute with the same value as the Dataweave output as shown, for example: multipart/form-data; boundary=abcdefg.

      %dw 2.0
      output multipart/form-data boundary='abcdefg'
      ---
      {
      	parts: {
      		file: {
      			headers: {
      				"Content-Disposition": {
      					"name": "file",
      					"filename": attributes.fileName
      				},
      				"Content-Type": payload.^mimeType
      			},
      			content : payload
      		},
      		name__v: {
      			headers: {
      
      			},
      			content: 'Test Document'
      		},
      		type__v: {
      			headers: {
      
      			},
      			content: 'Trial Management'
      		},
      		subtype__v: {
      			headers: {
      
      			},
      			content: 'Meetings'
      		},
      		classification__v: {
      			headers: {
      
      			},
      			content: 'Kick-off Meeting Material'
      		},
      		lifecycle__v: {
      			headers: {
      
      			},
      			content: 'Base Doc Lifecycle'
      		},
      		study__v: {
      			headers: {
      
      			},
      			content: '0ST000000000301'
      		},
      		comments__c: {
      			headers: {
      
      			},
      			content: 'Test Document'
      		}
      	}
      }

Connector Operations SUCCESS and FAILURE Response

The Veeva Vault Connector operation responses are based on the Veeva Vault API success or failure response with an error.

The connector returns a SUCCESS response at HIGH LEVEL and SUCCESS or FAILURE at LOW LEVEL. This means the connector operation is successful but some document or object records failed to create or update due to some irrelevant metadata being passed in the request.

Examples

SUCCESS with SUCCESS response:

{
	"responseStatus": "SUCCESS",
	"data": [{
			"id": 239026,
			"name__v": "E22611234--38483",
			"responseStatus": "SUCCESS"
		},
		{
			"id": 239025,
			"name__v": "Kick-off Meeting Material Updated12341234--81032",
			"responseStatus": "SUCCESS"
		}
	]
}

SUCCESS with a FAILURE response:

{
	"data": [
		{
			"external_id__v": "TEST-238924",
			"rendition_type__v": "imported_rendition__c",
			"id": 238924,
			"responseStatus": "FAILURE",
			"minor_version_number__v": 1,
			"errors": [
				{
					"type": "INVALID_DATA",
					"message": "Document not found [238924/0/1]."
				}
			],
			"major_version_number__v": 0
		},
		{
			"external_id__v": "TEST-238925",
			"rendition_type__v": "imported_rendition__c",
			"id": 238925,
			"responseStatus": "FAILURE",
			"minor_version_number__v": 1,
			"errors": [
				{
					"type": "INVALID_DATA",
					"message": "Document not found [238925/0/1]."
				}
			],
			"major_version_number__v": 0
		}
	],
	"responseStatus": "SUCCESS"
}

The Veeva Vault operations throw an exception when Veeva Vault APIs return a FAILURE response, for example:

FAILURE with an ERROR response

{
	"responseStatus": "FAILURE",
	"errors": [
		{
			"type": "INVALID_DATA",
			"message": "Unknown relationship [reviewer__v]"
		}
	]
}

Upon receiving the FAILURE response from Veeva Vault APIs, the connector operations throw an exception, which is caught in the Error Handling component within the Mule flow:

**********************************************************************************
Message : An error occurred from the Veeva Vault API.
Error Code: INVALID_DATA.
Original Error Message: Unknown relationship [reviewer__v].
Error type : VEEVAVAULT:INVALID_DATA
**********************************************************************************

The following are the error codes that are caught in the Error Handling component:

  • VEEVAVAULT:API_LIMIT_EXCEEDED

  • VEEVAVAULT:ATTRIBUTE_NOT_SUPPORTED

  • VEEVAVAULT:INACTIVE_USER

  • VEEVAVAULT:INVALID_DATA

  • VEEVAVAULT:INVALID_DOCUMENT

  • VEEVAVAULT:INSUFFICIENT_ACCESS

  • VEEVAVAULT:MALFORMED_URL

  • VEEVAVAULT:METHOD_NOT_SUPPORTED

  • VEEVAVAULT:NO_PERMISSION

  • VEEVAVAULT:OPERATION_NOT_ALLOWED

  • VEEVAVAULT:PARAMETER_REQUIRED

Streaming and Pagination

All the operations in the connector (except Download Document) return an InputStream as a payload with respective results based on operation output. Because of this, by default, Mule applies Streaming Strategies. See Mule Streaming Strategies for more details. The streaming strategies configuration fields are in the Advanced tab of the connector operations.

image

The following operations in the connector provide a pagination mechanism based on Mule standard pagination.

While using the paginated operations make sure to place a For-Each/Splitter element to retrieve each object (metadata is in JSON format) at a time. The pagination operations have Fetch Size and Batch Size fields.

  • Fetch Size
    The Fetch Size is a limit number of records that can be retrieved in a single page. The operation returns the pages with the fetch size number of JSON object records.
    In some cases, Veeva APIs auto-calculate the fetch size (number of records on each page) based on record size and the calculation exceeds the standard record size. The operation returns calculated records on each page.

  • Batch Size
    The Batch Size is the number of pages to return in each batch and each page will have the fetch size number of records. The operation returns a number of records (metadata in JSON format) per batch, and is calculated like in the following example:

Fetch Size set as *1000*
Batch Size set as *10*

If the total records in the vault are *100,000*, then:

Number of pages = Total records/Fetch Size
				= 100000/1000
				= 100 pages.

Number of pages per batch 	= Number of pages/Batch Size
							= 100/10
							= 10 pages per batch.

Number of Records per batch = Number of pages per batch * Fetch Size
							= 10 * 1000
							= 10,000 records.

Therefore, the number of records returned per batch would be 10,000 records.

The repeatable streams measure the buffer size in byte measurements. When handling objects, the runtime measures the buffer size using instance counts.

In non-repeatable streams connector operations return streams as the number of records per batch. Repeatable streams return all records at once, so when calculating the in-memory buffer size for repeatable auto-paging, you need to estimate how much memory space each instance takes to avoid running out of memory.

Next Step

After you complete configuring the connector, you can try the Examples.

View on GitHub