TIBCO Scribe® Online Connector For Mapper

The TIBCO Scribe® Online Connector For Mapper serializes and deserializes content based on a sample data structure, such as JSON. It can be used in conjunction with other connector extensions, such as the HTTP Connector.

Connector Specifications

	Supported
Agent Types
On Premise	X
Cloud	X
Replication Services
Source
Target
Integration Services
Source	X
Target	X
Migration Services
Source
Target
Maps
Integration	X
Request-Reply	X
Message

Supported Entities

The Mapper Connector derives the entities and related entities from the entity names and samples that you provide when you configure the Connection.

Operation Details

The Query Block displays but does not return any data since the Mapper Connector does not connect to a source datastore, but instead serializes and deserializes content returned by other Connectors.

Fetch

Use the Fetch Block to deserialize a string, for example, the response to an HTTP GET request or an inbound HTTP message received from an API.

This Map illustrates using the Fetch Block to deserialize a response to an HTTP GET request.

The Fetch Block Filter tab captures the content returned by an HTTP GET request and deserializes it based on the sample for the selected entity in the Mapper connection dialog.

The results of the Fetch Block are used to Create records in another target datastore.

This Map illustrates using a Fetch Block to deserialize an inbound HTTP message received from an API.

The Fetch Block Filter tab captures the content received in the message and deserializes it.

The results of the Fetch Block are used to Create records in another target datastore.

Serialize

• Converts inbound data based on the sample for the selected entity in the Mapper connection dialog.
• The foundation of the Serialize Block is the CreateWith Block, which is typically used with hierarchical data to add a single record or a parent record that contains one or more child records. In the case of hierarchical data, the Add Block is used to Add child records for the parent record in the CreateWith Block. See CreateWith Block and Add Block.
• Selecting all of the related entities on the Serialize Block Include tab is required to be able to serialize the data correctly.

  

• Output fields are read-only and contain the results of the serialization to be used in subsequent Blocks, such as in an HTTP Connector Send Block.

  

• The HTTP Send Block sends the output of the Serialize Block as a serialized record to the designated URI.

  

Setup Considerations

API Usage Limits

Some APIs limit the number of calls in a 24 hour period, or some other sliding window. Refer to the documentation for the API you are accessing and review the limits.

Selecting An Agent Type For Mapper

Refer to TIBCO Scribe® Online Agents for information on available Agent types and how to select the best Agent for your Solution.

Connecting To Mapper

1. Select More > Connections from the menu.
2. From the Connections page select Add to open the Add a New Connection dialog.
3. Select the Connector from the drop-down list in the Connection Type field, and then enter the following information for this Connection:
   • Name — This can be any meaningful name, up to 25 characters.
   • Alias — An alias for this Connection name. The alias is generated from the Connection name, and can be up to 25 characters. The Connection alias can include letters, numbers, and underscores. Spaces and special characters are not accepted. You can change the alias. For more information, see Connection Alias.
   • Entity
     • Use the Add button to add a user-specified Entity Name for each Entity accessed by this Connection. A new row displays in the Entities column labeled EntityName. Configure the properties in the right pane, including Entity Name, for each Entity you add on the left pane.

     Note: When an entity is added in the left pane, associated configuration fields display in the right pane.

   • Select the Entity to edit it. Configuration options display in the right pane for the selected Entity.
     
   • Use the Entities Fields table below to configure the properties for the selected Entity.

   Entity Fields
   

   Field	Definition
   Entity	Name of the entity as it should display in the flow.
   Content	The format of the source schema. Formats include: Sample JSON Sample CSV
   With Header	Select the With Header checkbox to indicate that the sample contains a header row.
   Sample	Provide sample input data in the Sample window.
   Entity Preview	Select Generate Entity preview to generate a preview of the output. Select Clear to remove the preview of the output.

   Note: Sample CSV only supports comma "," as a delimiter.

   • To modify an entity, select it and edit the associated properties in the right pane.
   • To delete an entity, select Delete from the Gear menu next to the Entity name.

     Note:

     Before deleting an entity name from Mapper Connections, remove all references to the entity from any flows.

     Deleting the Entity Name permanently removes all configuration settings associated with that Entity Name.

     Note:

     • At least one Sample is required to save the Connection.

     • Best practice is to test the Connection after adding each sample to allow the Connector to check samples individually for errors.

   • To view fields for a different Entity, select a new Entity from the list in the left pane.
4. Select Test to ensure that the Agent can connect to your database. Be sure to test the Connection against all Agents that use this Connection. See Testing Connections.
5. Select OK/Save to save the Connection.

Metadata Notes

• Arrays of arrays are not supported, such as: { "arrays": [ ["foo", "bar"] ] }.
• Empty objects or empty arrays are not supported, such as { } or [ ]. An error message is generated similar to the following: A top level array of json type 'Array' is currently not supported.
• JSON fields with empty arrays are not supported and generate the JSON validation error. To resolve the error, you have to remove empty array fields or enter values in them.
• If a sample in the Connection dialog is incomplete and TIBCO Scribe® Online cannot determine the data type for a field, type string is used, such as {"contactObject":null}.
• When deseralizing, if the Connector encounters a numeric string, such as {"number":5}, the value is treated as an INT.
• When serializing, if the Connector encounters a numeric string, such as { "number":"5" }, the value is treated as a string.

Naming

Connection metadata must have unique entity, relationship, and field names. If your Connection metadata has duplicate names, review the source system to determine if the duplicates can be renamed.

Mapper Connector As IS Source

Consider the following when using the Connector as an IS source.

Filtering

• Net Change is not supported.
• The only valid filter value is mapper_input, used when deserializing content.

Mapper Connector As IS Target

Consider the following when using the Connector as an IS target.

Batch processing is not supported.

Sample Maps

Storing a variable (or state) of fields/values, multiple entities, and child records can be complex to design and even more difficult to maintain using out-of-the-box functions. In addition, you may want to store a JSON format of your data when using Connectors such as Files, HTTP, and Variables. Mapper can help you create a blueprint for your dataset, such as JSON, and provide the ability to get and set the entities and fields in that variable/state.

These simple Maps attempt to demonstrate some concepts and how they can be achieved using the Mapper Connector. Review the Group Block labels in the Maps that explain the purpose of the Blocks inside the Group.

The Serialize and Deserialize (JSON) map demonstrates the following: 

• Starting a Map with no source data for the Query Block.
• Using a Fetch Block to retrieve Account records from a Sample CRM database.
• Serializing an Account record input to output JSON. The customer entity in the Serialize Block uses the customer entity JSON sample configured on the Mapper Connection dialog to determine how to map the Account record fields to a JSON format. The objects on the Serialize Block Properties Include tab represent how TIBCO Scribe® Online interprets the objects in the JSON sample.
• Writing the results to the Agent log.
• Using an Upsert Block to create a variable that stores the JSON so it can be used later in the Map since Fetch Block results cannot be accessed outside of the Fetch Block.
• Using a Loop Exit Block to exit the Fetch Block. This forces the Fetch Block to only process one record at a time instead of all of the records returned by the Fetch. The Map can then process the record stored in the variable before processing any additional Account records.
• Using a Lookup Block to retrieve the customer record from the variable.
  

Note: Lookup block is not supported for hierarchical data.

• Using a Fetch Block to deserialize the JSON. The Filter tab captures the JSON returned from the variable retrieved in the previous Block and deserializes it based on the sample JSON in the Mapper connection dialog.
• Using an Execute Block to write the response to the Agent log.

The Working with Arrays Map demonstrates the following: 

When configuring sample JSON in the Mapper Connection dialog, the square brackets [ ] indicate that the sample is an array. The top level of this array corresponds to the object labeled Anonymous in the Include tab of the Serialize and Fetch Blocks. To reveal and access the objects within the array, you must select the Anonymous object on the Include tab. The Include tab displays the format of the JSON as TIBCO Scribe® Online interprets it.

• Starting a Map with no source data for the Query Block.
• Using the Serialize Block to establish the format of the array_of_customer JSON output by selecting the objects on the Include tab. Errors display because there are no fields mapped on the fields tab, but for this scenario no field mappings are required.
• Using a Fetch Block to retrieve Account records from a Sample CRM database.
• Using an Add Block to map Account fields to the correct array_of_custoomer JSON objects and add each customer record into the array.
• Using an Execute Block to write the output to the Agent log.
• Using a Fetch Block to deserialize the JSON. The mapper_input field on the Filter tab captures the JSON output generated by the Serialize Block and deserializes it based on the sample JSON in the Mapper Connection dialog.
• Using the For Each Child Block to retrieve one customer record at a time from the deserialized JSON.
• Using an Execute Block to write a message for each record to the Agent log.

The Working with HTTP Map is a more complete scenario and demonstrates the following: 

• Starting a Map with no source data for the Query Block.
• Using the Serialize Block to establish the format of the array_of_cusotmerJSON output by selecting the objects on the Include tab. Errors display because there are no fields mapped on the fields tab, but for this scenario no field mappings are required.
• Using a Fetch Block to retrieve Account records from a Sample CRM database.
• Using an Add Block to map Account fields to the correct JSON objects and add each Account record into the array.
• Using a Send Block with the POST HTTP verb to send the JSON array in an HTTP request. This Block uses Postman Echo to echo the HTTP request back to TIBCO Scribe® Online.
• Using a Fetch Block to deserialize the JSON in the HTTP response. The mapper_input field on the Filter tab captures the JSON response from the Send POST Block and deserializes it based on the sample JSON in the Mapper Connection dialog.
• Using the For Each Child Block to retrieve one customer record at a time from the deserialized JSON.
• Using an Execute Block to write a message for each record to the Agent log.

TIBCO Scribe® Online API Considerations

To create connections with the TIBCO Scribe® Online API, the Mapper Connector requires the following information:

Connector Name	Mapper
Connector ID	EFCAFD59-08C7-4F26-BF0E-67DD85D7290B

TIBCO Scribe® Online Connection Properties

In addition, this Connector uses the Connection properties shown in the following table.

Name	Data Type	Required	Secured	Usage
Entities	String	Yes	No
JsonFormatVersion	String	Yes	No
CSVFormatVersion	String	Yes	No

License Agreement

The TIBCO Scribe® Online End User License Agreement for the Mapper Connector describes TIBCO and your legal obligations and requirements. TIBCO suggests that you read the End User License Agreement.

More Information

For additional information on this Connector, refer to the Knowledge Base and Discussions in the TIBCO Community.