Configuring RAG Operations

Retrieval-Augmented Generation (RAG) is a technique for enhancing AI-generated outputs by retrieving relevant content, and using it to augment AI prompts with additional context. By grounding LLMs with this additional information, they can provide more accurate and reliable responses.

Configure the RAG Load Document Operation

The RAG load document operation retrieves information based on a plain text prompt from an in-memory embedding store.

To configure the RAG load document operation:

Select the operation on the Anypoint Code Builder or Studio canvas.
In the General properties tab for the operation, enter these values:
- Data
  
  The prompt to send to the LLM and the embedding store to respond to.
- Context Path
  
  Contains the full file path for the document to ingest into the embedding store. Ensure the file path is accessible.
  
  You can also use a DataWeave expression for this field, for example:
  
  mule.home "/apps/" app.name ++ "/customer-service.pdf"
In the Context section for the operation, select the File Type:
- text
  
  Text files, such as JSON, XML, TXT, and CSV.
- URL
  
  A single URL pointing to web content to ingest.

This is the XML configuration for this operation:

<ms-aichain:rag-load-document
  doc:name="RAG load document"
  doc:id="3d3edd66-4970-4dad-a5bf-2a8eae123da4"
  config-ref="MAC_AI_Llm_configuration"
  data="#[payload.prompt]"
  contextPath="#[payload.contextPath]"
/>

Output Configuration

This operation responds with a JSON payload that contains the main LLM response. Additionally, attributes such as token usage are included as part of the metadata (attributes), but not within the main payload.

This is an example response of the JSON payload:

{
  "response": "Wakanda, a technologically advanced and environmentally conscious nation in Africa, is renowned for its unique integration of ancient traditions with cutting-edge innovations, powered by the rare metal Vibranium. With a population of 12.5 million, it emphasizes sustainable growth, quality education, and healthcare, while maintaining a zero carbon footprint through advanced eco-tech solutions. Despite its peaceful nature, Wakanda's formidable military and cultural heritage, led by King T’Challa and the Dora Milaje, ensure its resilience and unity as a symbol of progress and tradition."
}

The operation also returns attributes that aren’t within the main JSON payload, which include information about token usage, for example:

{
  "tokenUsage": {
      "outputCount": 9,
      "totalCount": 18,
      "inputCount": 9
  },
  "additionalAttributes": {}
}

tokenUsage

Token usage metadata returned as attributes
outputCount

Number of tokens used to generate the output
totalCount

Total number of tokens used for input and output
inputCount

Number of tokens used to process the input

Configuring RAG Operations

Configure the RAG Load Document Operation

Output Configuration

See Also