TIBCO Scribe® Online Text File As Source
TIBCO Scribe® Online allows you to create a source Connection to one or more text files. Source Connections can be made to text files in Local Directories, and on the following servers: FTP (unencrypted), FTPS (encrypted using SSL), SFTP (SSH File Transfer Protocol), Dropbox, and OneDrive. You can then use the text Connection to integrate data into a target datastore, such as Microsoft Dynamics CRM or Salesforce. Using text files as a source also allows you to test Solutions, Maps and formulas with a small, controllable amount of data. When you are done troubleshooting Solutions, switch to a different source Connection and process larger amounts of data.
Source and target text Connections must be configured separately. See TIBCO Scribe® Online Text File As Target.
- Install the Text File as Source Connector from the Marketplace. See Marketplace TIBCO Scribe® Certified Connectors.
- Create the source text files. These can be created manually or by exporting data from an existing database. Text files are the equivalent of tables in a database or entities in TIBCO Scribe® Online. See Source Or Target Text File Requirements.
- On the Text File as Source Edit Connection dialog, configure the tabs in order as they are displayed. See Configuring Text File As Source Connection for configuration steps.
- On the Location tab of the Connection dialog, specify the physical location where the source text files are stored. For any given location, specify the directory that contains the text files. That directory represents the database or container that holds the text files or database tables. There are several location types supported, each with a different set of required parameters. See Source Text Connection — Location Tab for details on each location type.
- If you choose to store text files on your own computer, you must edit the FoldersAccess.txt file. When text files are stored locally, TIBCO Scribe® Online uses the FolderAccess.txt file to maintain the list of directories it is permitted to access when connecting to text files. The TIBCO Scribe® Online Text Connector is limited to the directories in this list. Add directories to the default list as needed. See Providing Access To Local Text Files.
- Each text file represents an entity or database table. In the left panel of the Entities tab on the Connection dialog, create a descriptive name for each entity to be accessed by this Connection.
- For each Entity you add, use the right panel of the Entities tab on the Connection dialog to configure associated properties. Properties include: file format information for parsing the text file, associated text file name, and post processing instructions for that file. See Source Text Connection — Entities Tab.
- When the Text File as Source is configured, select the Test button to make sure TIBCO Scribe® Online is communicating with the selected text files. See Testing Connections.
Note: The Cloud Agent can only be used with the Text Connector when both the source and target text files are stored in a cloud based location, such as DropBox, OneDrive, or any of the FTP location options.
Selecting An Agent Type For Text File As Source
Refer to TIBCO Scribe® Online Agents for information on available Agent types and how to select the best Agent for your Solution.
Text File Connector As IS or MS Source
Consider the following when using the Text Connector as an Integration Solution or Migration Solution or MS Source.
When a Solution configured to process source text files cannot find any files in the configured location, the Solution executes successfully with 0 records processed. This is intentional because in situations where an external system generates text files to be processed by a scheduled Integration Solution, there may be times when no files have been generated. The Solution continues to execute according to its configured schedule, but has no records to process. For example, if the external system only generates text files when new records are created, and the prior execution of the Solution moves processed files to a different location, then the source folder would be empty.
Any row of data that begins with a hash (#) is considered a comment and ignored.
Features Not supported
The following features are not available for Maps that use a Text File as Source Connection:
- Net Change based on Most Recent Record Processed date.
- Reset Metadata — Metadata parameters are contained within the Connection configuration and must be reset there. Resetting metadata is required when field names have changed in the header row of the source text file. See Resetting Metadata For Text Connections.
- Debugging Maps is unavailable for Integration Maps, when storing data in a local directory.
Data Type Mismatch Source And Target Text Data Exception
In general, when a data type mismatch occurs between your source and target data, TIBCO Scribe® Online generates a record error. However, if you are mapping Text File as Source data to Text File as Target and there is a data type mismatch, TIBCO Scribe® Online inserts a value of NULL into the mismatched target column.
Text File as Source supports Fetch operations. In this case, Fetch does not require a filter. Use Fetch to retrieve and process multiple records, returning all records. See Fetch Block for more information.
The Lookup Block is not supported for the Text File As A Source Connector.
The TIBCO Scribe® Online API does not support creating Text File as Source Connections.
The TIBCO Scribe® Online End User License Agreement for the Text as a Source Connector describes TIBCO and your legal obligations and requirements. TIBCO suggests that you read the End User License Agreement.