TIBCO Scribe® Online Connector For MongoDB

The TIBCO Scribe® Online Connector for MongoDB is based on the CData driver for MongoDB. Use this Connector to integrate with CRM, accounting, eCommerce, and marketing systems. If you use the Connector to store data for an application you have developed, you can integrate that application using TIBCO Scribe® Online without building a custom Connector. This Connector based on Scribe.Connector.AdoNet library and CData MongoDB ADO.NET provider.

Possible use cases for the MongoDB Connector include: 

Connector Specifications

  Supported

Agent Types

On Premise X
Cloud X

Replication Services

Source  
Target  

Integration Services

Source X
Target X

Migration Services

Source  
Target  

Maps

Integration X
Request-Reply X
Message  

This Connector is available from the TIBCO Scribe® Online Marketplace. See Marketplace TIBCO Scribe® Certified Connectors for more information.

Supported Entities

The MongoDB collections are exposed as entities.

Setup Considerations

The MongoDB Connector supports MongoDB version 3.6.

Advanced scenarios such as customizing the SSL configuration or connecting through a firewall or a proxy, fine-tuning of discovering metadata for nested JSON objects, or property types are described in the Advanced Settings section of the CData documentation and in the Metadata Notes section of this document.

Selecting An Agent Type For MongoDB

Refer to TIBCO Scribe® Online Agents for information on available Agent types and how to select the best Agent for your Solution.

Connecting To MongoDB

Note: Best practice is to create Connections with credentials that limit permissions in the target system, following the principle of least privilege. Using Administrator level credentials in a Connection provides Administrator level access to the target system for TIBCO Scribe® Online users. Depending on the entities supported, a TIBCO Scribe® Online user could alter user accounts in the target system.

  1. Select More > Connections from the menu.
  2. From the Connections page select Add to open the Add a New Connection dialog.
  3. Select the Connector from the drop-down list in the Connection Type field, and then enter the following information for this Connection:
    • Name — This can be any meaningful name, up to 25 characters.
    • Alias — An alias for this Connection name. The alias is generated from the Connection name, and can be up to 25 characters. The Connection alias can include letters, numbers, and underscores. Spaces and special characters are not accepted. You can change the alias. For more information, see Connection Alias.
    • Server — Enter the IP address or DNS name of the MongoDB server.
    • Port — Optional. Enter the port number for the MongoDB server. Must be in the 1023 to 65535 range.
    • Database — Name of the MongoDB database you are accessing.
    • User — Name of the database user with access to connect to MongoDB.
    • Password — Password for the database user.
    • Additional Parameters — Optional field where you can specify one or more connection string parameters. See the Connection String Options section of the CData documentation for a list of parameters that can be used and their default values. Note that in some cases the CData MongoDB ADO.NET Provider does not fully support all of the possible parameters.

      Syntax for the Additional Parameters field is as follows:

      • All blank characters, except those within a value or within quotation marks, are ignored
      • Preceding and trailing spaces are ignored unless enclosed in single or double quotes, such as Keyword=" value"
      • Semicolons (;) within a value must be delimited by quotation marks
      • Use a single quote (') if the value begins with a double quote (")
      • Use a double quote (") if the value begins with a single quote (')
      • Parameters are case-insensitive
      • If a KEYWORD=VALUE pair occurs more than once in the connection string, the value associated with the last occurrence is used
      • If a keyword contains an equal sign (=), it must be preceded by an additional equal sign to indicate that the equal sign is part of the keyword
      • Parameters that are handled by other fields or default settings in the Connection dialog are ignored if used in the Additional Parameters field, including: 
        • Server
        • Logfile — This parameter is not visible in the Connection dialog, but is set by the Connector. The default size is a maximum of 10MB. Any CData log files generated by this setting are stored in the default TIBCO Scribe® Online Agent Logs directory, C:\Program Files (x86)\Scribe Software\TIBCO Scribe® Online Agent\logs\. The format for log file names for CData logs are as follows: <ConnectorName><GUID of the Connection><DateTimeStamp>.log

          Note: For information on setting log file verbosity, see Verbosity in the CData Help.

        • MaxLogFileSize — This parameter is set by the Connector to a maximum of 10MB.
        • Other
        • RTK
  4. Select Test to ensure that the Agent can connect to your database. Be sure to test the Connection against all Agents that use this Connection. See Testing Connections.
  5. Select OK to save the Connection.

Metadata Notes

MongoDB is a schemaless document database. The Connector metadata provider generates the schema based on the data in each MongoDB database Collection.

Retrieving Metadata

MongoDB Naming Restrictions

As noted in the MongoDB documentation there are some restrictions for entity names.

Connection metadata must have unique entity, relationship, and field names. If your Connection metadata has duplicate names, review the source system to determine if the duplicates can be renamed.

Collection Names

Field Names

Arrays And Nested Objects

MongoDB documents containing nested objects and arrays are not supported.

Binary Data

Binary fields are not supported.

Generate And Modify Schema Files

You can generate, and then change the schema files to customize metadata by setting the GenerateSchemaFiles connection string property value to:

The location of schema files can be set using the Location connection string option.

Note: This feature is available only for On-Premise Agents.

Relationships

Relationships and hierarchical data are not supported.

MongoDB Connector As IS Source

Consider the following when using the MongoDB Connector as an Integration Solution source.

Filtering

When a datetime is configured on the Query Block on the Block Properties Net Change Tab to query for new and updated records, that configuration is treated as an additional filter. The Net Change datetime filter is applied as an AND after any other filters specified on the Block Properties Filter Tab. TIBCO Scribe® Online builds a query combining both the Net Change filter and the filters on the Filter tab. See Net Change And Filters for an example.

Some Connectors for TIBCO Scribe® Online only support one filter. For those Connectors you can use either Net Change or one filter on the Filter tab, not both.

Note: The Net Change date is ignored when previewing data on the Preview tab. Filters on the Block Properties Filters tab are used to filter the data on the Preview tab.

Net Change is supported for MongoDB columns with a date and timestamp.

For additional filtering information refer to the CData MongoDB ADO.NET documentation.

Native Query

The MongoDB Connector supports SQL queries in Native Query Blocks to create customized queries for MongoDB. The query can be as simple or complex as you need it to be; however, it should return a single result set. The native query text is sent to MongoDB exactly as it is entered without any modifications.

You can use SELECT , UPDATE , INSERT and DELETE clauses. If support for Enhanced SQL is enabled, you can use Joins, Aggregate functions, and Projection functions. For additional details, see the SQL Compliance and Support Enhanced SQL sections of the CData documentation .

After entering the SQL query, you must select Test to validate the query. Invalid queries are not accepted by the Connector. See Native Query Block and Creating Native Queries For Microsoft SQL Server for additional information.

When testing a Native Query in a Map, if the source datastore does not return any data, TIBCO Scribe® Online cannot build the schema for the underlying metadata and the Map cannot be saved. To allow TIBCO Scribe® Online to build the schema, do the following:

  1. Create a single temporary record in the source datastore that matches the Native Query.
  2. Test the Native Query and ensure that it is successful.
  3. Save the Map.
  4. Remove the temporary record from the source datastore.

JSON Functions

You can execute special functions in SQL queries directly in Native Query Blocks to retrieve various JSON structures from the document. For the full JSON Functions list and examples see the JSON Functions section of the CData documentation.

Using Native MongoDB Queries

The MongoDB Connector supports the db.collection.find() function when the Query Passthrough parameter is set to true in the Additional Parameters field on the Connection dialog.

Example

To query the Customers collection using native MongoDB syntax in the Native Query Block and return the following information:

{
 "_id":"5ac362f5ac83422dd0d25132",
 "name":"John Smith",
 "emails":[
   "jsmith@microsoft.com",
   "jsmith@gmail.com"
 ],
 "address":{
   "country":"USA",
   "state":"Texas",
   "city":"Dallas"
 }
}

Use this syntax in the Native Query Block:

db.Cusotmers.find (
	{name: 'John Smith'}
)

On the Query Block Preview tab, the fields and data for John Smith's record display.

Executing JavaScript Functions Using Stored Procedures

The Native Query Block can be used to call javascript functions with the EVAL stored procedure.

Example

EXECUTE EVAL jsFunction = 'function() {
	var i = 1;
	var j = 2;
	var k = 3;
	var result - (i+j) * k;
	return result;}'

On the Query Block Preview tab, the returned result displays in the retval field.

 

MongoDB Connector As IS Target

Consider the following when using the MongoDB Connector as an Integration Solution target.

Supports the following operations:

Batch Processing

Batch processing is not supported.

TIBCO Scribe® Online API Considerations

To create connections with the TIBCO API, the MongoDB Connector requires the following information:

Connector Name

MongoDB

Connector ID

7BCF0FCF-4825-48E6-8163-F157D67D0161

TIBCO Scribe® Online Connection Properties

In addition, this Connector uses the Connection properties shown in the following table.

Note: Connection property names are case-sensitive.

Name Data Type Required Secured Usage

Server

string

Yes

No

 

Port

string

No

No

 

Database

string

Yes

No

 

User

string

No

No

 

Password

string

No

Yes

 

ConnectionString

string

No

No

 

License Agreement

The TIBCO Scribe® Online End User License Agreement for the MongoDB Connector describes TIBCO and your legal obligations and requirements. TIBCO suggests that you read the End User License Agreement.

More Information

For additional information on this Connector, refer to the Knowledge Base and Discussions in the TIBCO Community.