Contact Us 1-800-596-4880
  1. Homepage
  2. MongoDB Connector (6.1)

  3. Upgrading and Migrating MongoDB Connector

All Connectors

Upgrading and Migrating MongoDB Connector - Mule 4

Upgrade Anypoint Connector for MongoDB (MongoDB Connector) from version 5.3.x to version 6.x.x.

Supported Upgrade Paths

From Version To Version

5.3.x

6.x.x

Changes in This Release

MongoDB 6.x.x contains the following changes:

  • Changed connection types and configuration

  • Changed operation names, metadata, fields, and values

  • Changed input sources

  • Added the ability to specify which MongoDB Java driver to use

  • At runtime, you can overwrite the Write concern acknowledgement and Write concern timeout fields in the global element of the following operations:

    • Find documents

    • Insert document

    • Insert documents

    • Update Documents

    • Count documents operations

Changes in Connection Types and Configuration

MongoDB 6.x contains the following changes to connection types and configuration:

  • Removed the Connection String connection type.

    There is now a single, unified generic connection that contains the configurable properties that were previously available in the Connection String connection type.

  • Removed the Metadata configuration section.

    MongoDB automatically generates the metadata.

Config Advanced Tab

The following optional changes apply to the Advanced tab in the global configuration:

  • Added the following read concern options:

    • Read concern: Enables the control of the consistency and isolation properties of the data read from replica sets and replica set shards.

    • Read preference: Specifies the read preferences for this connection.

    • Maximum staleness seconds: Specifies how stale a secondary MongoDB cluster can be before the client stops using it for read operations.

      A value of -1 means that there is no maximum staleness threshold.

    • Read preference tags: Enables you to specify a tag set in the read preference to target those members associated with a tag in the set.

  • Added the following write concern options:

    • Write concern acknowledgment: Requests an acknowledgment that the write operation was propagated to the specified number of instances.

    • Write concern timeout: Specifies a time limit for the write concern.

    • Write concern timeout timeunit: Specifies a time unit for the Write concern timeout field.

Connection Section

The following changes apply to the tabs in the Connection section of the global configuration:

  • Added the following fields to the General tab:

    • Required Libraries: Reference to the MongoDB Driver.The version must be 3.11 or later.

    • Server Addresses: This field is now is a list of server addresses to use for the connection.

  • Added a Security tab for optional TLS context configuration.

  • Added the following optional fields to the Advanced tab:

    • Authentication mechanism: Authentication mechanism used for this connection

    • Replica set name: Name of the replica set to which to connect

    • Authentication source: Database name associated with the user’s credentials

    • Compressors: List of compressors for which to enable network compression for communication between this client and a mongod/mongos instance

    • Zlib compression level: Integer that specifies the compression level if using zlib for network compression.

      Set this field to 0 for no compression or from 1 through 9 for compression, with 1 being the lowest compression level and 9 the highest. High compression levels require more processing time than low compression levels.

    • Connection timeout: Connection timeout used when establishing the socket connections

    • Connection timeout time unit: Time unit for the Connection timeout field

    • Local threshold: Size of the latency window for selecting among multiple suitable MongoDB instances

    • Local threshold time unit: Time unit for the Local threshold time unit field

    • Server selection timeout: How long to block for server selection before throwing an exception

    • Server selection timeout time unit: Time unit for the Server selection timeout field

    • Socket timeout: Maximum time to wait for a send or receive operation on a socket before the attempt times out

    • Socket timeout unit: Time unit for the Socket timeout field

    • Min connection pool size: Minimum size for the connection pool

    • Max connection pool size: Maximum size for the connection pool

    • Max wait queue time: Maximum wait queue time for the connection pool

    • Max wait queue time unit: Time unit for the Maximum wait queue time field

    • Max connection life time: Maximum connection lifetime for the connection pool

    • Max connection life time time unit: Time unit for the Maximum connection life time field

    • Max connection idle time: Maximum connection idle time for the connection pool

    • Max connection idle time unit: Time unit for the Maximum connection idle time field

Changed Operations, Parameters, and Return Types

The following table shows changes to operation names, input parameters, and return types:

MongoDB 5.x Operation Changes in MongoDB 6.0

Count documents

Input parameters are now:

  • Collection name (String)

  • Query (JSON)

Return type is now Long with the count result.

Create Collection

Input parameters are now:

  • Collection name (String)

  • Max objects (Integer)

  • Collection size (Integer)

  • Collection size data unit (TimeUnit)

Exists collection

Operation is now called Collection exists.

Create file from payload

  • Operation is now called Create file.

  • Return type is now JSON.

Create index

Input parameters are now:

  • Collection name (String)

  • Field name (String)

  • Sort order (Enum: ASC or DESC)

Dump

Input parameters are now:

  • Output directory (String)

  • Output name prefix (String)

  • Oplog (no change)

  • Operation timeout (Integer)

  • Operation timeout unit (TimeUnit)

Return type is now a List<String> type that points to the created files. Each string is a filePath type.

Execute command

  • Input parameter is Command (JSON).

  • Return type is now a JSON type that contains the result of the command.

Find documents

Input parameters are now:

  • Collection name (String)

  • Query (JSON)

  • Fields (String)

  • Sort by (JSON)

  • Page size (Integer)

  • Limit (Integer)

This operation supports pagination. Each item returned as a JSON type.

Find files

  • Input parameters are now:

    • Query (JSON)

    • Sort (JSON)

  • Return type is now a JSON list.

Get file content

  • Input parameter is now File id (JSON).

  • Return type is now a stream with the binary file content. This operation also returns the file information as attributes.

Insert document

Return type is now an entire JSON object with the _id object of the created document.

Insert documents

Return type is now a Bulk Operation Result, which contains a JSON file that lists each created record and its status.

List collections

Return type is now a List<String> type that contains the names of the collections.

List indices

Operation is now called List indexes.

Map reduce objects

  • Operation is now called Map reduce.

  • Return type is now a JSON type that represents the output specified in the reduce function.

Remove documents

  • Input parameters are now:

    • Collection name (String)

    • Query (JSON)

  • Return type is now a Long with the count result.

Remove files

Input parameter is now File id (JSON).

Update documents

  • Input parameters are now:

    • Collection name (String)

    • Query (JSON)

    • Content to update (JSON)

    • Multiple update (Boolean: moved to the Advanced tab)

    • Upsert (Boolean)

  • Return type is now a JSON type that contains the following structure:

    • Int Matched

    • Int Modified

    • String Upserted Id

Changed Operations Metadata

MongoDB 5.x generated operations metadata when the user provided a set of documents per collection from which to take the attributes. MongoDB 6.0 generates metadata automatically, based on the latest document in each collection.

Metadata in MongoDB v6.0

Operation Name Input Metadata Output Metadata

Insert Document

Document

Resolved dynamically based on the selected value of the collection parameter. The connector adds the latest document of the given collection and uses the document’s structure as input/output metadata.

Insert Documents

Document

Resolved dynamically based on the selected value of the collection parameter. The connector adds the latest document of the given collection and uses the document’s structure as input/output metadata.

Update Documents

Not applicable

JSON that contains the following structure:

  • Int Matched

  • Int Modified

  • JSON upsertedId

Remove Documents

Query: JSON Object

Not applicable

Count Documents

Query: JSON Object

Not applicable

Find Documents

Query: JSON Object

Resolved dynamically based on the selected value of the collection parameter. The connector uses the structure of the last document in the given collection.

Create File

Not applicable

A JSON object with the following attributes:

  • JSON Id

  • String fileName

  • Long Length

  • Int ChunkSize

  • Datetime uploadDate

  • JSON Metadata

Find Files

Not applicable

List of JSON objects with the following attributes:

  • JSON Id

  • String fileName

  • Long Length

  • Int ChunkSize

  • Datetime uploadDate

  • JSON Metadata

Removed Operations

The following operations were removed from the MongoDB connector:

Removed from MongoDB 5.x Can be reproduced in MongoDB 6.0.0 through

Name

Description

Name

Description

Incremental dump

Executes an incremental dump of the database

Dump

Use the Dump operation.

Restore

Takes the output from the dump and restores it

Restore from file or Restore from directory

Restore from file or Restore from directory take the output from the dump file or directory and restore it.

Update documents by function

Update documents using a Mongo function

Not applicable

Not applicable

Update documents by functions

Update documents using one or more Mongo functions

Not applicable

Not applicable

Find one and update document

Finds and updates the first document that matches a given query

Not applicable

This operation’s functionality can be reproduced by combining other operations.

For example, you can invoke the Find documents operation with a limit of 1 document. Then you can invoke the Update documents operation using the ID of the document returned previously.

Save document

Inserts or updates a document based on its object ID

Not applicable

Not applicable

Find one document

Finds the first document that matches a given query

Find documents

Set the Limit=1 to return one document.

Execute generic command

Executes a generic command on the database

Execute command

Executes a command on the database

List files

Lists all files that match the given query and sorts them by filename

Not applicable

Not applicable

Find one file

Returns the first file that matches the given query

Find files

Specify a specific file to query

Removed Parameters

The Num To Skip field in the Find Documents operation is removed from MongoDB Connector 6.x. The skip() method in MongoDB, which is used internally by the Num To Skip field, can lead to performance issues, especially as the offset increases. This is because the server scans through the results from the beginning to the specified offset before returning any documents, which adds unnecessary load and slows down the operation.

To optimize pagination and improve performance when migrating to MongoDB Connector 6.x, the following approaches are recommended:

  • Avoid using skip() for large offsets.

    Instead of using the Num To Skip field, it is more efficient to refine query conditions to exclude unwanted documents directly.

  • Use range queries.

    Use existing indexes to perform range queries. For example, if your documents have a timestamp or a unique ID field (indexed), you can use these fields to filter results directly to avoid scanning through unwanted documents.

    { "timestamp": { "$gt": ISODate("2024-01-01T00:00:00Z") } }

  • Apply limit() for pagination.

    Use the limit() method in combination with range queries to control the number of documents returned, effectively implementing pagination without the performance issues associated with skip().

    { "timestamp": { "$gt": ISODate("2024-01-01T00:00:00Z") } }
.limit(10)

  • Combine filters and sorting.

    Consider ordering documents by an indexed field, which allows for more efficient queries when paginating through results.

Changes in Input Sources

MongoDB 6.0.0 has one input source, Object Listener, which retrieves all of the created documents that belong to a specific collection.

The Delete Sources and Update Sources input sources were removed.

Metadata in Object Listener

Input Metadata Output Metadata

Not applicable

The connector resolves output metadata dynamically based on the selected value of the Collection Name parameter. The connector reads the last document of the given collection and uses the document’s structure as output metadata.

Requirements and Limitations

Software Version

Mule

4.1.1 and later

MongoDB

MongoDB Java driver 3.11 and later

Upgrade Prerequisites

Before you perform the upgrade, you must:

  1. Create a backup of your files, data, and configuration in case you need to restore to the previous version.

  2. Install MongoDB v6.0 to replace the MongoDB operations that were previously included in MongoDB Connector v5.x.

Upgrade Steps

Follow these steps to perform the upgrade from MongoDB Connector v5.3.x to MongoDB Connector v6.0:

  1. In Anypoint Studio, create a Mule project.

  2. In the Mule Palette view, click Search in Exchange.

  3. In the Add Dependencies to Project window, enter MongoDB in the search field.

  4. In the Available modules section, select MongoDB Connector and click Add.

  5. Click Finish.

  6. Verify that the salesforce-connector dependency version is 6.0.0 in the pom.xml file in the Mule project.

Studio upgrades the connector automatically.

Verify the Upgrade

After you install the latest version of the connector, follow these steps to verify the upgrade:

  1. In Studio, verify that there are no errors in the Problems or Console views.

  2. Check the project pom.xml file and verify that there are no problems.

  3. Test the connection and verify that the operations work.

Troubleshooting

If there are problems with caching the parameters and metadata, try restarting Studio.

Revert the Upgrade

If it is necessary to revert to the previous version of MongoDB Connector, change the mule-mongodb-connector dependency version 6.0.0 in the project’s pom.xml to the previous version.

See Also

Back to top
View on GitHub