TIBCO Scribe® Online Connector For Microsoft Common Data Service (CDS)

Use the TIBCO Scribe® Online Connector for Microsoft Common Data Service (CDS) to work with the Microsoft Common Data Service (CDS) database. It supports custom fields and custom entities, relationships between entities, and supports Microsoft Dynamics 365 API’s bulk load method for higher performance.

Note: Some features are available only if you had a Microsoft Dynamics 365 / CRM organization when you created your Microsoft Common Data Service (CDS) database, such as a subset of Microsoft Dynamics 365 entities, Merge functionality, and alternate keys.

Connector Specifications

	Supported
Agent Types
Connect on-premise	X
Connect cloud	X
Data Replication Apps
Source	X
Target
On Schedule Apps
Source	X
Target	X
On Event Apps
Source	X
Target	X
Flows
Integration	X
Request-Reply	X
Message

Note: This Connector is available from the TIBCO Cloud™ Integration Marketplace. See Marketplace Connectors for more information.

Setup Considerations

Requires either a Microsoft PowerApps account and a Microsoft Common Data Service (CDS) database.

Permissions

Your Microsoft PowerApps account must have appropriate permissions to allow TIBCO Cloud™ Integration - Connect to perform Create, Read, Update, Delete, and Batch or non-Batch operations. Contact your system administrator for assistance.

Run As User

The Microsoft Common Data Service (CDS) Connector provides the ability for a user to perform all operations as if a different user were performing them. In the Connection, this alternate user is the Run As User. The user named in the User ID field is the "login user".

Assume that as the login user, you have administrator credentials, but you set the Run As User to be a salesperson. You then run an app that imports contacts. In this case, all the contacts appear to have been entered by the Run As User. In addition, all operations run in the context of the roles and privileges held by the Run As User.

Selecting An Agent Type For Microsoft Common Data Service (CDS)

Refer to TIBCO Cloud™ Integration - Connect Agents for information on available agent types and how to select the best agent for your app.

Connecting To Microsoft Common Data Service (CDS)

Note: Best practice is to create Connections with credentials that limit permissions in the target system, following the principle of least privilege. Using Administrator level credentials in a Connection provides Administrator level access to the target system for TIBCO Cloud™ Integration - Connect users. Depending on the entities supported, a TIBCO Cloud™ Integration - Connect user could alter user accounts in the target system.

There are three deployment types for the Microsoft Common Data Service (CDS) Connector that provide different authentication methods. Follow the link for the authentication method you plan to use for step-by-step configuration instructions:

Online Authentication — Select if your site is using Microsoft PowerApps and you do not want to authenticate using Active Directory.
Azure AD Authentication — Select to use OAuth authentication with Active Directory.
Client Secret Authentication — Select to use Client Secret authentication with Active Directory..

Online Authentication

Select Connections from the menu.
From the Connections page select Create to open the Create a connection dialog.
Select the Connector from the list to open the Connection dialog, and then enter the following information for this Connection:
- Name — This can be any meaningful name, up to 25 characters.
- Alias — An alias for this Connection name. The alias is generated from the Connection name, and can be up to 25 characters. The Connection alias can include letters, numbers, and underscores. Spaces and special characters are not accepted. You can change the alias. For more information, see Connection Alias.
- Deployment — Select Online if your site is using Microsoft PowerApps and you do not want to authenticate using Active Directory.
  When you select the deployment, TIBCO Cloud™ Integration - Connect displays the appropriate Connection parameters fields.

Discovery Service URL — The URL (web address) for Microsoft Dynamics 365. For Microsoft Common Data Service (CDS) you still connect to a Microsoft Dynamics 365 URL. This URL varies depending on the type of deployment selected.

Microsoft URLs

Microsoft Office 365

Location	URL
North America	https://disco.crm.dynamics.com
North America 2	https://disco.crm9.dynamics.com
Europe, Middle East, and Africa (EMEA)	https://disco.crm4.dynamics.com
Asia Pacific (APAC)	https://disco.crm5.dynamics.com
Oceania	https://disco.crm6.dynamics.com
Japan (JPN)	https://disco.crm7.dynamics.com
South America	https://disco.crm2.dynamics.com
India	https://disco.crm8.dynamics.com
Canada	https://disco.crm3.dynamics.com

Microsoft Account

Location	URL
North America	https://dev.crm.dynamics.com
Europe, Middle East, and Africa (EMEA)	https://dev.crm4.dynamics.com
Asia Pacific (APAC)	https://dev.crm5.dynamics.com

User ID/Password — A valid Microsoft PowerApps user ID and password.
Organization — The Microsoft Organization you want to access. Select Browse to have TIBCO Cloud™ Integration - Connect determine the Organization associated with these Microsoft credentials. Browse only displays for Online Deployments.
Note: The Microsoft-friendly Organization name is the Organization name that appears in the upper right hand corner of your Microsoft account. This name is case-sensitive.
Run As User ID — Optionally, specify the valid User ID of a user to serve as the Run As User.
Maximum Batch Size — Maximum number of records to include in a single call to the Microsoft API when Batch Processing is enabled for a target operation. Default is 500, minimum is 1, and maximum is 1000. Setting to 0 disables the Test button. Setting to a value higher than 1000 defaults to 1000, but continues to display the higher number.
Note: Receiving Timeout errors is an indication that the number of records per call has exceeded the default setting for your Microsoft Organization and you should decrease this number.
Modifying or requesting modification of the default setting by Microsoft or your hosting partner may allow for a higher value. See the following Microsoft Knowledge Base Article: Service Protection API Limits and the Microsoft D365 CRM/CE request limits/allocations and API Limits article in the TIBCO Knowledge Base.
Maximum Batch Threads — Number of simultaneous calls to the Microsoft API when Batch Processing is enabled for a target operation, such as Delete. Default is 2, minimum is 1, and maximum is 32. Setting to 0 disables the Test button. Setting to a value higher than 32 defaults to 32, but continues to display the higher number.
Note: Receiving any of the following errors, is an indication that the number of calls or threads has exceeded the default setting for your Microsoft Organization and you should decrease this number.
- Server busy
- The following error has occurred in the Dynamics 365 Connector: System.ObjectDisposedException: Cannot access a disposed object.
- Generic SQL errors
Modifying or requesting modification of the default setting by Microsoft or your hosting partner may allow for a higher value. See the following Microsoft Knowledge Base Article: Service Protection API Limits and the Microsoft D365 CRM/CE request limits/allocations and API Limits article in the TIBCO Knowledge Base.
Include Picklist Display Names select this option if you want to include the display names for any picklist, status, or state fields that you are replicating. When the entity is recreated, TIBCO Cloud™ Integration - Connect adds a new field in the format fieldname _displayname for each qualifying field, where _displayname is the text description rather than the numeric value.
See Picklists below for more information.
Select Test to ensure that the agent can connect to your database. Be sure to test the Connection against all agents that use this Connection. See Testing Connections. If the Test fails, an error similar to the following is displayed. Correct the issue outlined in the error and try the test again.
Select OK/Save to save the Connection.

Azure AD Authentication

The Microsoft Common Data Service (CDS) Connector supports connecting to your Microsoft Common Data Service (CDS) instance using Azure AD authentication. This authentication method requires a separate configuration to establish a connection from TIBCO Cloud™ Integration - Connect.

Select Connections from the menu.
From the Connections page select Create to open the Create a connection dialog.
Select the Connector from the list to open the Connection dialog.
Enter a Connection Name and an Alias.
In the Deployment field select Azure AD.
In the Discovery Service URL field, enter the URL of your discovery service. This is used to validate your Microsoft PowerApps Organization name. For Microsoft Common Data Service (CDS) you still validate using a Microsoft Dynamics 365 URL. See the Microsoft URLs table in the Connecting To Microsoft Common Data Service (CDS) section for a list of URLs. Example: https://disco.crm.dynamics.com
Note: Only the first part of the full service URL displayed in your Microsoft PowerApps instance is used. If you enter the full service URL an error is generated. For example, if your full service URL is https://disco.crm.dynamics.com/XRMServices/2011/Discovery.svc, only enter https://disco.crm.dynamics.com/.
In the D365 Instance URL field, enter the URL for your Microsoft PowerApps instance. Example: https://<yourcompany>.crm.dynamics.com
Note: The D365 Instance URL is case sensitive when testing the Connection, but not when authenticating. Best practice is to copy the URL from your Microsoft PowerApps instance and paste it into the Connection dialog.
Select the Authenticate button. This launches a new browser tab with a Sign in page. Enter your Sign in information and select Next.
On the Password dialog, enter your password and select Sign In.
If you have enabled multi-factor authentication in your Microsoft PowerApps Organization, the Enter code dialog displays. Enter the code you received and select Verify.
A permissions page is displayed.
Select Accept to grant permission for TIBCO Cloud™ Integration - Connect to access this Microsoft PowerApps instance. The Success page is displayed.
On the Success page select Close to return to TIBCO Cloud™ Integration - Connect. Next to the Grant Consent button the message Consent granted. is displayed. This message indicates that permission has been granted correctly and you can proceed with the remaining configuration steps.
Organization — The Microsoft Organization you want to access. You can use either the friendly name or the unique name. Names are case-sensitive, therefore a best practice is to copy the name from your Microsoft PowerApps instance and paste it into the Connection dialog.
Note: The Microsoft-friendly Organization name is the Organization name that appears in the upper right hand corner of your Microsoft account.
Maximum Batch Size — Maximum number of records to include in a single call to the Microsoft API when Batch Processing is enabled for a target operation. Default is 500, minimum is 1, and maximum is 1000. Setting to 0 disables the Test button. Setting to a value higher than 1000 defaults to 1000, but continues to display the higher number.
Note: Receiving Timeout errors is an indication that the number of records per call has exceeded the default setting for your Microsoft Organization and you should decrease this number.
Modifying or requesting modification of the default setting by Microsoft or your hosting partner may allow for a higher value. See the following Microsoft Knowledge Base Article: Service Protection API Limits and the Microsoft D365 CRM/CE request limits/allocations and API Limits article in the TIBCO Knowledge Base.
Maximum Batch Threads — Number of simultaneous calls to the Microsoft API when Batch Processing is enabled for a target operation, such as Delete. Default is 2, minimum is 1, and maximum is 32. Setting to 0 disables the Test button. Setting to a value higher than 32 defaults to 32, but continues to display the higher number.
Note: Receiving any of the following errors, is an indication that the number of calls or threads has exceeded the default setting for your Microsoft Organization and you should decrease this number.
- Server busy
- The following error has occurred in the Dynamics 365 Connector: System.ObjectDisposedException: Cannot access a disposed object.
- Generic SQL errors
Modifying or requesting modification of the default setting by Microsoft or your hosting partner may allow for a higher value. See the following Microsoft Knowledge Base Article: Service Protection API Limits and the Microsoft D365 CRM/CE request limits/allocations and API Limits article in the TIBCO Knowledge Base.
Include Picklist Display Names select this option if you want to include the display names for any picklist, status, or state fields that you are replicating. When the entity is recreated, TIBCO Cloud™ Integration - Connect adds a new field in the format fieldname _displayname for each qualifying field, where _displayname is the text description rather than the numeric value.
See Picklists below for more information.
Select Test to ensure that the agent can connect to your database. Be sure to test the Connection against all agents that use this Connection. See Testing Connections. If the Test fails, you must reauthenticate. Correct the issue outlined in the error and try the test again.
Note: The D365 Instance URL is case sensitive when testing the Connection, but not when authenticating. If the test fails, check the case on the Dynamics Instance URL to make sure it is correct.
Select OK/Save to save the Connection.

Azure AD Best Practices

If you authenticate using Azure AD, there are some timing issues to take into consideration to prevent errors.

Avoid re-authenticating an Azure AD connection while running an app using the connection. There is a narrow window where this could cause the app to fail and may generate an exception similar to the following:
```
Authorization Code already Redeemed
```
Avoid re-authenticating an Azure AD connection while resetting metadata or when metadata is being loaded in an app or flow. There is a narrow window where this could result in an exception similar to the following:
```
Key not found Exception
```
Avoid hitting Cancel on the Connection dialog after successfully testing a connection: Due to the nature of this authentication type, testing credentials modifies the credentials whether you cancel or not.

Client Secret Authentication

The Microsoft Common Data Service (CDS) Connector supports connecting to your Microsoft Common Data Service (CDS) instance using Client Secret authentication. This authentication method requires a separate configuration to establish a connection from TIBCO Cloud™ Integration - Connect.

Select Connections from the menu.
From the Connections page select Create to open the Create a connection dialog.
Select the Connector from the list to open the Connection dialog.
Enter a Connection Name and an Alias.
In the Deployment field select Client Secret.
In the URL field, enter the URL for your Microsoft PowerApps instance. Example: https://<yourcompany>.crm.dynamics.com
Note: The URL is case sensitive when testing the Connection. Best practice is to copy the URL from your Microsoft PowerApps instance and paste it into the Connection dialog.
Client ID — Enter the Client ID, also know as the Application ID, generated when you registered the Azure Active Directory (AAD) App.
Client Secret — Enter the Client Secret generated when you registered the Azure Active Directory (AAD) App.
Maximum Batch Size — Maximum number of records to include in a single call to the Microsoft API when Batch Processing is enabled for a target operation. Default is 500, minimum is 1, and maximum is 1000. Setting to 0 disables the Test button. Setting to a value higher than 1000 defaults to 1000, but continues to display the higher number.
Note: Receiving Timeout errors is an indication that the number of records per call has exceeded the default setting for your Microsoft Organization and you should decrease this number.
Modifying or requesting modification of the default setting by Microsoft or your hosting partner may allow for a higher value. See the following Microsoft Knowledge Base Article: Service Protection API Limits and the Microsoft D365 CRM/CE request limits/allocations and API Limits article in the TIBCO Knowledge Base.
Maximum Batch Threads — Number of simultaneous calls to the Microsoft API when Batch Processing is enabled for a target operation, such as Delete. Default is 2, minimum is 1, and maximum is 32. Setting to 0 disables the Test button. Setting to a value higher than 32 defaults to 32, but continues to display the higher number.
Note: Receiving any of the following errors, is an indication that the number of calls or threads has exceeded the default setting for your Microsoft Organization and you should decrease this number.
- Server busy
- The following error has occurred in the Dynamics 365 Connector: System.ObjectDisposedException: Cannot access a disposed object.
- Generic SQL errors
Modifying or requesting modification of the default setting by Microsoft or your hosting partner may allow for a higher value. See the following Microsoft Knowledge Base Article: Service Protection API Limits and the Microsoft D365 CRM/CE request limits/allocations and API Limits article in the TIBCO Knowledge Base.
Include Picklist Display Names select this option if you want to include the display names for any picklist, status, or state fields that you are replicating. When the entity is recreated, TIBCO Cloud™ Integration - Connect adds a new field in the format fieldname _displayname for each qualifying field, where _displayname is the text description rather than the numeric value.
See Picklists below for more information.
Select Test to ensure that the agent can connect to your database. Be sure to test the Connection against all agents that use this Connection. See Testing Connections. If the Test fails, you must reauthenticate. Correct the issue outlined in the error and try the test again.
Note: The URL is case sensitive when testing the Connection. If the test fails, check the case on the Dynamics Instance URL to make sure it is correct.
Select OK/Save to save the Connection.

Metadata Notes

Filter, Match Criteria, Lookup, And Target Operations

Leading and trailing spaces in string field values are trimmed when used in a filter, matching criteria, or lookup criteria, or when writing those values during target operations, such as Create or Update.

Naming

Connection metadata must have unique entity, relationship, and field names. If your Connection metadata has duplicate names, review the source system to determine if the duplicates can be renamed.

Custom Decimal Fields

When adding custom decimal fields to entities in Microsoft Common Data Service (CDS) with a precision of 0, the precision is not being honored.

Picklists

In Microsoft Common Data Service (CDS) databases, some fields are generated via a picklist. Picklists include both a text value that displays in Microsoft Common Data Service (CDS) and an internal code. By default, TIBCO Cloud™ Integration - Connect uses the text value when running an app. However, you can choose to exclude the text value that displays in the Microsoft Common Data Service (CDS) interface by disabling the Include Picklist Display Names option in the Microsoft Common Data Service (CDS) Connection configuration. When this option is enabled, an additional field is created in your Microsoft Common Data Service (CDS) datastore to contain the text value.

For example, if you have a picklist field, shippingmethod, where the picklist values include available shipping methods of DHL, FedEx, UPS, or USPS, and the Include Picklist Display Names option is enabled, both the picklist value and the picklist_displayname columns display for that entity in the database, as shown:

Picklist value	shippingmethod _displayname
1	DHL
2	FedEx
3	UPS
4	USPS

Including Picklist Display Names

When you create a new Microsoft Common Data Service (CDS) Connection, the Include Picklist Display Names option is selected by default. Picklist, state, and status fields include a _displayname column. You can choose to include or exclude the _displayname by editing the Connection.

From the Connections page, edit the Microsoft Common Data Service (CDS) Connection you want to change.
Select Include Picklist Display Names.
To exclude the _displayname, make sure this option is not selected.
Select Test and then select OK to save the changes to the Connection.

Note: TIBCO Cloud™ Integration - Connect may be considerably slower the first time you run the app after adding or removing picklist names.

Changing Picklist Entities

When you make changes in Microsoft Common Data Service (CDS) to entities with picklists, the source and target are affected as follows:

If you add a new picklist value for a particular entity definition, the new value is added to the target the next time the app runs.
If you delete an existing picklist value for a particular entity definition, no change is made in the target. Existing records in the target datastore still have the deleted value or code/display name.
If you update the definition of an existing picklist value, the change is made the next time the app runs only if:
- New records are added in Microsoft Common Data Service (CDS) that include the picklist value.
- Another field in that record is updated.
Existing records with no changes display the new updated display name in the Microsoft Common Data Service (CDS) source; but the target data contains the old display name.
New picklist custom fields are automatically included in subsequent runs of the app.

Note: If you change the value of a picklist field for any records within Microsoft Common Data Service (CDS) after a replication runs, the display name changes are not reflected in previously replicated records. To properly replicate changed picklist values, drop the table in the target database. The next time the replication runs, the table is recreated with the new display names.

Using Picklists With Apps

When using picklist values with apps, because the display names are not actual fields in the source or target entity, the following caveats apply:

Picklist display names are not available as Filters or Match Field criteria.
You can map picklist display fields from a Microsoft Common Data Service (CDS) source to any valid target field. However, you cannot map a source field to a picklist display field in the target.

Multi-Select Option Sets

Multi-select option sets are picklists that allow a Microsoft Common Data Service (CDS) user to select more than one value for a picklist for a single record. For example, a multi-select option set named Countries Visited where the user could select one or more countries instead of a single country.

Picklist display names are enabled or disabled on the Microsoft Common Data Service (CDS) Connection dialog.
Filtering and Match Criteria for multi-select values is not supported.
When writing multi-select values to a target they must be formatted as a comma separated list of values.
When querying records with multi-select values, values are returned as a comma separated list of values.
Updating multi-select values on a record, overwrites existing values. To append new values to an existing record, use a Lookup Block to get the existing values and append the new values to the existing list.
Picklist values are case sensitive.
A combination of names and numeric picklist values can be used when mapping to a multi-select field.

Note: The Formula Editor used to map fields on the Block Properties Fields tab does not support picklist option names that contain commas or quotes, such as Yell,ow or Yell"ow. When mapping these types of names, use the numeric picklist value instead.

Ignored Entities

The following entities may exist in your Microsoft Common Data Service (CDS) database if you had a Microsoft Dynamics 365 / CRM instance when you created the CDS database. However they are not available through the Microsoft Common Data Service (CDS) Connector.

Ignored Microsoft Dynamics 365 Entities
applicationfile asyncoperation attributemap bulkoperation clientupdate constraintbasedgroup displaystring duplicaterule entitymap import importdata importfile importjob importlog importmap integrationstatus	isvconfig mailmergetemplate metadatadifference new_scribechangehistory new_scribepublisherqueue pluginassembly plugintype plugintypestatistic relationshiprole report resourcespec resourcegroupexpansion savedquery savedqueryvisualization sdkmessage sdkmessagefilter	sdkmessagepair sdkmessageprocessingstep sdkmessageprocessingstepimage sdkmessageprocessingstepsecureconfig sdkmessagerequest sdkmessagerequestfield vsdkmessageresponse sdkmessageresponsefield serviceendpoint solutioncomponent usersettings webwizard wizardaccessprivilege wizardpage workflow workflowlog

Ignored Microsoft Dynamics 365 Entities

applicationfile

asyncoperation

attributemap

bulkoperation

clientupdate

constraintbasedgroup

displaystring

duplicaterule

entitymap

import

importdata

importfile

importjob

importlog

importmap

integrationstatus

isvconfig

mailmergetemplate

metadatadifference

new_scribechangehistory

new_scribepublisherqueue

pluginassembly

plugintype

plugintypestatistic

relationshiprole

report

resourcespec

resourcegroupexpansion

savedquery

savedqueryvisualization

sdkmessage

sdkmessagefilter

sdkmessagepair

sdkmessageprocessingstep

sdkmessageprocessingstepimage

sdkmessageprocessingstepsecureconfig

sdkmessagerequest

sdkmessagerequestfield

vsdkmessageresponse

sdkmessageresponsefield

serviceendpoint

solutioncomponent

usersettings

webwizard

wizardaccessprivilege

wizardpage

workflow

workflowlog

Microsoft Common Data Service (CDS) Connection As A Data Replication App Source

Microsoft Common Data Service (CDS) can be used as a data source for Data replication apps. See Managing Data Replication Apps.

Note: For information about using Microsoft Common Data Service (CDS) with TIBCO Cloud™ Integration - Connect Data replication apps, see Selecting Source Entities.

Recommended Entities

When you select the Recommended option in the Data replication app, TIBCO Cloud™ Integration - Connect replicates entities for which you have the appropriate permissions, including custom entities.

Note: Custom entities with the prefixes msdyn_, msdynsm_, and msfp._ are not replicated.

Entities In Recommended List

Microsoft Common Data Service (CDS) Entities Included In Recommended List
account annotation appointment bookableresource bookableresourcebooking bookableresourcebookingheader bookableresourcecategory bookableresourcecategoryassn bookableresourcecharacteristic bookableresourcegroup bookingstatus businessunit businessunitnewsarticle calendar campaign campaignactivity campaignresponse cardtype category characteristic competitor competitoraddress connection connectionrole contact contract contractdetail customeraddress customeropportunityrole customerrelationship discount discounttype email entitlement	entitlementchannel entitlementcontacts entitlemententityallocationtypemapping entitlementproducts equipment fax feedback goal incident incidentresolution interactionforemail internaladdress invoice invoicedetail kbarticle knowledgearticle knowledgearticleviews lead leadaddress letter list mailbox metric opportunity opportunityclose opportunityproduct opportunitysalesprocess orderclose phonecall phonetocaseprocess position post pricelevel processsession	product productpricelevel publisher publisheraddress quote quoteclose quotedetail recurringappointmentmaster reportcategory reportentity reportlink reportvisibility role salesliterature salesliteratureitem salesorder salesorderdetail service serviceappointment sharepointdocumentlocation sharepointsite site sla slaitem socialactivity socialprofile subject systemuser task territory topichistory topicmodel transactioncurrency uom uomschedule

Microsoft Common Data Service (CDS) Entities Included In Recommended List

account

annotation

appointment

bookableresource

bookableresourcebooking

bookableresourcebookingheader

bookableresourcecategory

bookableresourcecategoryassn

bookableresourcecharacteristic

bookableresourcegroup

bookingstatus

businessunit

businessunitnewsarticle

calendar

campaign

campaignactivity

campaignresponse

cardtype

Microsoft Dynamics 365 Entities Not Included In Recommended Or Selected Lists
appconfigmaster authorizationserver businessprocessflowinstance childincidentcount cloneproduct delveactionhub documentindex lookupmapping merge msdyn_odatav4ds multientitysearch	officedocument officegraphdocument orginsightsmetric orginsightsnotification partnerapplication principalobjectaccessreadsnapshot publishproducthierarchy queueitemcount queuemembercount recommendeddocument revertproducthierarchy	rollupjob savedorginsightsconfiguration sharedobjectsforread sharepointdata subscriptionstatisticsoffline subscriptionstatisticsoutlook subscriptionsyncentryoutlook syncattributemappingprofile systemapplicationmetadata userapplicationmetadata

Net Change Filters

Data replication apps have Net Change filters enabled by default to replicate only new and updated records. The source Connector automatically chooses the datetime field to be used for the Last Modified Date for each table. When the Data replication app runs it selects new or updated source records based on that datetime field, and only replicates those records. In some cases, the datetime field does not contain milliseconds. To ensure that all records are replicated, the Connector subtracts one second from the most recent Last Modified Date, which could cause a small number of records to be reprocessed.

For example, if the last record written had a Last Modified Date of 09/04/2018 10:10:20, the next Replication starts with data that has a Last Modified Date of 09/04/2018 10:10:19. Any records that were already processed between 09/04/2018 10:10:19 and 09/04/2018 10:10:20 are processed again.

Deleted Records In Microsoft Common Data Service (CDS)

When a record is deleted from the Microsoft Common Data Service (CDS) source:
- TIBCO Cloud™ Integration - Connect adds an entry to the scribechangehistory custom entity in Microsoft Common Data Service (CDS), which records the delete.
- The TIBCO Cloud™ Integration - Connect agent registers a plug-in assembly in Microsoft Common Data Service (CDS) to publish these delete operations from the scribechangehistory custom entity into the Microsoft SQL Server Server database during replication.
  Note: The Delete operation is asynchronous, and may not appear immediately.
- The next time the app runs, TIBCO Cloud™ Integration - Connect selects deleted records based on the datetime value in the scribechangehistory CreatedOn field. In the Microsoft SQL Server Server target database, TIBCO Cloud™ Integration - Connect creates an entry for that record in the Scribe_DeletedOn column that exists for every TIBCO Cloud™ Integration - Connect table in the Microsoft SQL Server Server database.
Records in the scribechangehistory table are purged 60 days after the last modified date.

Replicated Entities

TIBCO Cloud™ Integration - Connect does not replicate all entities. Replicated entities are chosen based on dependencies between entities and criteria such as, whether or not the entity has a Last Modified date. TIBCO Cloud™ Integration - Connect ensures that you replicate all of the data needed without replicating internal tables and other target-specific data. You can select additional entities to replicate from the Selecting Source Entities option on the App Details page.
If the activitymimeattachment entity exists it is not replicated due to its size. To move activitymimeattachment data into another datastore, such as Microsoft SQL Server, use an On schedule app instead of a Data replication app.
Deletes are not recorded for the activitypointer entity. For the initial replication, all activitypointer records are replicated. For subsequent replications, new and updated records are added. Delete the entire activitypointer table from the target replication datastore and rerun the Data replication app to remove the unwanted records.
For entities that do not include a LastModifiedOn field, the CreatedOn field is used to determine whether the entity has already been replicated during a prior run. Currently, only the Audit entity uses the CreatedOn field in this manner.
Rollup Fields, Calculated Fields — Values displayed in rollup and calculated fields in the interface or queried through the API are calculated dynamically, based on other field values. However, the dynamically calculated values that appear in the interface may not be picked up by the Replication process.
For example, if an OrderTotal field in one entity is calculated based on the value in an ItemTotal field in a different entity, changes made to ItemTotal do not update the ModifedOn date value in the entity that contains the OrderTotal field, and so do not trigger the record to be picked up by the Replication process. However, the next time a change is made that does trigger the record to be picked up, the query of the records recalculates the values and the recalculated OrderTotal value is picked up.

Custom Objects

During replication, TIBCO Cloud™ Integration - Connect automatically creates any corresponding tables for custom entities in the target datastore. Select the entities you want to replicate from the Selecting Source Entities option on the Data replication app Details page.

If your Microsoft Common Data Service (CDS) data contains custom entities:

The custom entities are replicated and no action is needed if they exist when the app first runs.
If you add custom entities later, TIBCO Cloud™ Integration - Connect includes the new customizations the next time the data is replicated. If you select specific entities to replicate, be sure to include the new entity. If you are replicating either Recommended or All entities, your custom entity is automatically included.
If you add a custom field to existing entities, the new field is added to the existing replicated table. However:
- The new field does not replicate until you reset the metadata for the Microsoft Common Data Service (CDS) Connection.
- When the new field is created in CDS, the corresponding field values in the target datastore are set to NULL until you update them in CDS and rerun the Data replication app.

Note: When you create a custom entity in Microsoft PowerApps, a unique default prefix is added to the entity name. However, you can change the name of the default prefix within Microsoft PowerApps. If you are using custom entities or fields, be sure you know the custom prefix name and use that name to search for your custom entities. For more information, see your Microsoft PowerApps Administrator.

Microsoft Common Data Service (CDS) Connection As An App Source

Relationships

Note: For this Connector, only one level of relationship, Parent/Child, is supported for the entity selected in the Query or Fetch block. For records with multiple levels of relationships, such as Parent, Child, Grandchild, only the selected Parent entity and related Child entities can be retrieved. Additional levels beneath Child entities, such as Grandchildren, cannot be accessed for this Connector using the Include tab. See Block Properties Include Tab — Relational Data.

Hierarchical data is not supported. See Hierarchical Data for additional information.

Query And Native Query

When using a Native Query Block with FetchXML or a standard Query Block to return related entities, only the first 1000 child entities can be returned for a single parent record. If more than 1000 child entities are returned, an error similar to the following is generated:

Unable to get the next page of data. Dynamics 365 has not advanced the page cookie for Entity: <entity name>, PagingCookie: paging cookie".

To work around this issue, modify your Query to exclude child entities, then add a Fetch Block for each result to retrieve all children for the Entity selected in the Query.

When using a Native Query Block with FetchXML or a standard Query Block to return multiple records, if those records include milliseconds in the modifiedon field, the Connector may return the same page of data over and over again without advancing to the next set and generate an error similar to the following:

Unable to get the next page of data. Dynamics 365 has not advanced the page cookie for Entity: <entity name>, PagingCookie: <paging cookie>

To workaround this issue contact Microsoft for assistance in determining the cause.

Using an Order by function for a child entity in Fetch XML in a Native Query Block limits the number of records returned to 12000. If the query exceeds 12000 records, an error similar to the following is generated:

Paging cookie is required when trying to retrieve a set of records on any high pages.

To work around this issue, modify your Query to exclude child entities, then add a Fetch Block for each result to retrieve all children for the Entity selected in the Query.

When testing a Native Query in a flow, if the source datastore does not return any data, TIBCO Cloud™ Integration - Connect cannot build the schema for the underlying metadata and the flow cannot be saved. To allow TIBCO Cloud™ Integration - Connect to build the schema, do the following:

Create a single temporary record in the source datastore that matches the Native Query.
Test the Native Query and ensure that it is successful.
Save the flow.
Remove the temporary record from the source datastore.

FetchXML

The Microsoft Common Data Service (CDS) Connector supports using the FetchXML query language in a Native Query Block to create customized queries for Microsoft Common Data Service (CDS) entity records and associated child records. The query can be as simple or complex as you need it to be; however, it should return a single result set.

After entering the FetchXML query, you must select Test to validate the query. Invalid XML is not accepted by the Connector.

You should understand the FetchXML query language before attempting to use the Native Query Block to access your Microsoft Common Data Service (CDS) data. For assistance with FetchXML, see Build queries with FetchXML on the Microsoft developer site.

Using FetchXML in a Native Query Block includes support for sorting, grouping, filtering, and relationships. See the examples below for each option.

Simple Query

The example below returns all accounts and associated accountid and name values.

<fetch mapping='logical'>
    <entity name='account'>
     <attribute name='accountid'/>
     <attribute name='name'/>
    </entity>
</fetch>

Sorting

The example below sorts returned records by the value of the creditlimit field.

<fetch mapping="logical" version="1.0">
    <entity name="Account">
      <all-attributes />
      <order attribute="creditlimit" />
    </entity>
</fetch>

Grouping

The example below groups returned records by the value of the address1_city field.

<fetch mapping="logical" aggregate="true" version="1.0">
    <entity name="account">
      <attribute name="accountid" alias="COUNT_accountid" aggregate="countcolumn" />
      <attribute name="address1_city" groupby="true" />
    </entity>
</fetch>

Filtering

The example below returns records where the name field contains Acme.

<fetch mapping="logical" version="1.0">
    <entity name="account">
      <attribute name="name" />
      <attribute name="accountid" />
      <filter>
        <condition attribute="name" operator="like" value="%Acme%" />
      </filter>
    </entity>

Alias

If you have more than one entity with the same name, you must alias the entity names to distinguish the entities from each other.

Relationships

When parent and child records are returned, they are flattened and treated as a single set of records with the parent information repeated for each child. In this case, you cannot use a For Each Child Block. When mapping parent and child records, all fields for both the parent and the child records display as a single result set in the source side of the operation Block Properties Fields tab. If the parent and child entities have the same name and/or the same field names, such as a parent Account with multiple child Accounts, TIBCO Cloud™ Integration - Connect differentiates them by adding an alias to the entity or field name in the format entityalias_fieldname.

The example below performs an inner join on the account and SystemUser entities and returns System Users where the ownerid of the account matches the systemUserID of the SystemUser.

<fetch mapping="logical" version="1.0">
    <entity name="account">
      <attribute name="name" />
      <link-entity name="SystemUser" from="systemUserId" to="ownerid" alias="s" link-type="inner" />
    </entity>
</fetch>

Query Block Filtering

When a datetime is configured on the Query block on the Block Properties Net Change Tab to query for new and updated records, that configuration is treated as an additional filter. The Net Change datetime filter is applied as an AND after any other filters specified on the Block Properties Filter Tab. TIBCO Cloud™ Integration - Connect builds a query combining both the Net Change filter and the filters on the Filter tab. See Net Change And Filters for an example.

Some Connectors for TIBCO Cloud™ Integration - Connect only support one filter. For those Connectors you can use either Net Change or one filter on the Filter tab, not both.

Note: The Net Change date is ignored when previewing data on the Preview tab. Filters on the Block Properties Filters tab are used to filter the data on the Preview tab.

Microsoft Common Data Service (CDS) Connection As An App Target

Batch Processing

The following operations support Batch Processing:
- Insert
- Update
- Delete
- Upsert
The maximum Batch size is 1000 records.
Only two concurrent Batch requests are supported. Sending more than two simultaneous Batch requests to the Microsoft Common Data Service (CDS) online server causes a server busy error to occur.

For information about using Batch processing, see Batch Processing, Create Block, Update Block, or Delete Block.

Id And Idtype

For Microsoft Common Data Service (CDS) target Connections, if you map an id field, such as ownerid, you may also need to set the corresponding owneridtype to the value of the id base field.

To determine if you need to map the owneridtype, see the Description of the field displayed by selecting the Info icon for the Entity. See Block Properties — Fields Tab.

Ownerid And Owneridtype

For Microsoft Common Data Service (CDS) target Connections, both the ownerid and owneridtype fields may need to be mapped:

Create — If you are mapping the ownerid field during a Create or a create through an Upsert, you must also map the owneridtype field. Otherwise, an error occurs and the record is not assigned to any user type.
Update — If you are performing an Update or an update through an Upsert, mapping the owneridtype field is optional. If it is not mapped, Microsoft assigns the type based on the ownerid GUID value.
Note: If you perform an Update or an update through an Upsert, provide an ownerid without an owneridtype, and the ownerid does not match any existing systemuser or team, the following error occurs:
systemuser not found.

Example

For example, in the image below, the Description indicates that if you map the ownerid field, you must also set a value for the ownertypeid field.

In the image below, the Description for the owneridtype field specifies the possible base entities and values: systemuser, team. To set this value, enter the name of the base field type in quotation marks into the Formula field.

Merge Block Functionality

The Merge Block and the virtual merge entity support the Microsoft merge functionality to merge account, contact, lead, or incident records.

When using merge, note the following:
- Only two records can be merged in each operation: one target record and one subordinate record.
- After the merge:
  - The target record is the master of the subordinate record.
  - The subordinate record is deactivated.
- The children of the subordinate record are merged with the children of the target record.
  Note: If you are merging records and performing any relationship processing on the child records, use separate flows for the merge processing and the relationship processing. Otherwise, the merged child records may not be included in the relationship processing.
- The contents of the fields in the subordinate record are not merged.
The virtual merge entity includes the following fields that can be mapped on the Merge Block Properties Fields tab:
- entityTypeName — Name of the entity type of the records you are merging. This value must be one of:
  - account
  - contact
  - lead
  - incident
- targetId — ID of the target record. After the merge, the target record is made master of the subordinate record and parent to all child records of the subordinate record.
- subordinateId — ID of the subordinate record. After the merge, this record is deactivated.
- performParentingChecks — If set, generates an error when there is a conflict merging the parents of the target and the subordinate. If not set, no error is generated.

SkipSeekUnder

Seek Under is a lookup process where the Microsoft Common Data Service (CDS) Connector uses the Matching Criteria on a target operation Block to locate a record. The Connector then returns the entire record from the data store and builds the updated record with any fields that were mapped on the Fields tab. This process provides a greater degree of accuracy in locating the correct record and provides unmapped data for use in subsequent operations. For example, if telephone number is not one of the mapped fields on the Fields tab, it is still available for use in an operation later in the flow. However, because the target datastore is accessed twice, once to return the record and again to perform the target operation, there is an impact to performance.

The skipseekunder virtual field on the Fields tab of the target operation Block is used to enable or disable the Seek Under process. Use skipseekunder when you do not need access to additional fields for the selected entity and you are sure that the fields you provided in Matching Criteria are sufficient to locate the correct record. When using skipseekunder, note the following:

Mapping skipseekunder to true, disables the Seek Under process. The default setting for this field is false, which allows the Seek Under process to run.
When set to true, you must configure a primary key id or alternate key field on the Matching Criteria tab to ensure that the correct record is found.
For the Upsert operation, when skipseekunder is mapped to true, you must map the primary key id or the alternatekeyname and corresponding alternate key fields.
The setting for skipseekunder is ignored and the Seek Under process runs when:
- There is no primary key id configured on the Matching Criteria tab.
- Multiple fields are configured on Matching Criteria tab.
- For Upsert, if the primary key id or the alternatekeyname and corresponding alternate key fields are not mapped.
Attempting to skip the Seek Under process by setting skipseekunder to true, while simultaneously trying to use an entity reference field without the corresponding type, such as parentcustomerid and parentcustomeridtype, may cause the Connector to fail with an error similar to the following:
```
Error in calling Operation Update: Operation failed. Label: Update contact, Name: contactUpdate, Message: -2147220970:An unexpected error occurred. : System.NullReferenceException: Microsoft Dynamics CRM has experienced an error. Reference number for administrators or support: #C1CE39CD 
```
See Id And Idtype for additional information.
Supported for all entities and the following operations:
- Upsert
- Update
- Insert
- Delete — The skipseekunder field cannot be mapped for this operation. The Seek Under process is automatically skipped when Batch Processing is enabled, and the primary key or a complete alternate key is configured on the Matching Criteria tab. If Batch Processing is disabled, Seek Under is always used.

Notes On Standard Entities

If you had a Microsoft Dynamics 365 / CRM Organization when you created your Microsoft Common Data Service (CDS) database, a subset of these entities may be represented in the Microsoft Common Data Service (CDS) database.

ActivityParty

Batch processing is not supported for this entity.
Running more than one flow concurrently that updates this entity can cause data loss on target ActivityParty records.

AccountLeads

This entity does not support Update operations because the LeadId and AccountId fields are read only.

CalendarRule

This entity is always a child of a Calendar.
Batch processing is not supported for this entity.
Simple CalendarRule
- Although simple CalendarRules do not appear in the PowerApps interface, these entities are seen as children.
- isSimple — When True, defines the CalendarRules entity as a simple entity.

CloneProduct

This virtual entity isused to support complex product catalog integration requirements for products.

Uses the Execute operation is to clone products.
Batch Processing is not supported for this entity.

PublishProductHierarchy, RevertProductHierarchy

These virtual entities are used to support complex product catalog integration requirements for products, product families, and product bundles.

The Execute operation is available to publish or revert publication of a product hierarchy using the following virtual entities:
- PublishProductHierarchy — Supports publishing product hierarchy.
- RevertProductHierarchy — Supports reverting a product hierarchy.
Batch Processing is not supported for these entities.

Salesorder

If you change the status of an order to Fulfilled, you can populate the datefulfilled field with a user-specified date instead of the datetime stamp of when the record was changed, which provides a mechanism for entering historical data.

Systemuser

When moving a user from one Business Unit to another, the Microsoft Dynamics 365 API requires an ID for the new owner of the user's assets. Assets include Accounts or Contacts. Two virtual fields, reassignprincipalidtype and reassignprincipalid, are available to provide the user type and ID of the new owner. To assign assets back to the original user, enter that user's ID in the reassignprincipalid field and enter systemuser into the reassignprincipalidtype field.
If you populate the reassignprincipalidtype and reassignprincipalid virtual fields, but do not change the Business Unit for the user, the user's assets are unaffected.

Miscellaneous

Errors

You may receive one of the following errors when using the Microsoft Common Data Service (CDS) Connector.

Fatal Error

If you receive a fatal error that contains either of the following messages, make sure the credentials for your target Microsoft URL and Organization are correct. In addition, make sure the system time for the computer where the TIBCO Cloud™ Integration - Connect agent resides is correct.

Metadata contains a reference that cannot be resolved: 'https://example.com'

Verify that the Connection URL is correct and that you can connect to the same URL from the system running the Scribe Online Agent.

Security Time Stamp Error

This error is the result of an issue in the way Microsoft PowerApps tracks time. The Microsoft PowerApps Server, which is set to UTC/GMT, cannot generate a timestamp if the minutes value on the computer where the TIBCO Cloud™ Integration - Connect Connect on-premise agent is installed differs by more than 5 minutes from the Microsoft PowerApps Server time.

The Microsoft PowerApps Server uses your local time zone. Therefore, this problem only occurs if the minutes, not hours, are off by more than 5 minutes known as the skew time. To ensure that you do not get this message, make sure the minutes on the computer where the TIBCO Cloud™ Integration - Connect Connect on-premise agent is installed reflect the same minutes value as UTC/GMT time.

Unable To Insert Error

If you create a flow using a Microsoft Common Data Service (CDS) database as a target and see the error shown below, it may be caused by populating a field in the target that requires an associated field to also be populated. If the second field upon which the first one depends is either not linked or is being populated by a null or invalid value, the error can occur.

Unable to insert data into the XYZ entity. An unexpected error occurred.

For example, populating the businesscustomerid field and not populating the businesscustomeridtype in the Contract entity results in this error. To resolve this issue, map both fields with valid non-null values.

TIBCO Cloud™ Integration - Connect API Considerations

To create Connections with the TIBCO Cloud™ Integration - Connect API, the Microsoft Common Data Service (CDS) Connector requires the following information:

Connector Name	Microsoft Common Data Service
Connector ID	CBEB774F-B5E6-4ED7-8ED8-58D19466A1F5

TIBCO Cloud™ Integration - Connect Connection Properties

In addition, this Connector uses the Connection properties shown in the following tables.

Online Deployment Type
ClientSecret Deployment Type

Note: Connection property names are case-sensitive.

Note: TIBCO Cloud™ Integration - Connect does not support creating Connections via the TIBCO Cloud™ Integration - Connect API when using Azure AD for authentication.

Online Deployment Type

If you are creating a Connection to Microsoft Common Data Service (CDS) using an Online Deployment Type the following fields are used in the model sent to the TIBCO Cloud™ Integration - Connect API.

Name	Data Type	Required	Secured	Usage
DeploymentType	String	Yes	No	Enter "Online".
DiscoveryServiceURL	String	Yes	No
UserId	String	Yes	No
Password	String	Yes	Yes
HomeRealmUri	String	No	No	Not supported for this Connector, but the field is required on the Connection model.
Organization	String	Yes	No
Domain	String	No	No	Not supported for this Connector, but the field is required on the Connection model.
DisplayPickListNames	Boolean	No	No	Defaults to false if empty
RunAsUserName	String	No	No	Username of the user making API requests
BulkBatchSize	String	No	No	Defaults to 500 if empty
BulkConcurrentCalls	String	No	No	Defaults to 2 if empty

ClientSecret Deployment Type

If you are creating a Connection to Microsoft Common Data Service (CDS) using a ClientSecret Deployment Type, the following fields are used in the model sent to the TIBCO Cloud™ Integration - Connect API.

Name	Data Type	Required	Secured	Usage
DeploymentType	String	Yes	No	Enter "ClientSecret"
Url	String	Yes	No	The value of this field is case sensitive when testing the Connection.
ClientId	String	Yes	Yes
ClientSecret	String	Yes	Yes
HomeRealmUri	String	No	No	Not supported for this Connector, but the field is required on the Connection model.
Domain	String	No	No	Not supported for this Connector, but the field is required on the Connection model.
DisplayPickListNames	Boolean	No	No	Defaults to false if empty
BulkBatchSize	String	No	No	Defaults to 500 if empty
BulkConcurrentCalls	String	No	No	Defaults to 2 if empty

License Agreement

The TIBCO End User License Agreement for the Microsoft Common Data Service (CDS) Connector describes TIBCO and your legal obligations and requirements. TIBCO suggests that you read the End User License Agreement.

More Information

For additional information on this Connector, refer to the Knowledge Base and Discussions in the TIBCO Community.