TIBCO Scribe® Online Connector For Twitter

Use the TIBCO Scribe® Online Connector for Twitter to extract data from Twitter and use it in a different datastore or external system by querying and filtering tweets for specific hashtags, keywords, or handles. The Twitter Connector can only be used to retrieve Source data, but cannot integrate data back into the Twitter database.

Possible use cases for data extracted using the Twitter Connector include: 

Connector Specifications

Supports version 1.1 of the Twitter REST API.

  Supported

Agent Types

On Premise X
Cloud X

Replication Services

Source X
Target  

Integration Services

Source X
Target  

Migration Services

Source  
Target  

Maps

Integration X
Request-Reply  
Message  

Note:  This Connector is available from the TIBCO Scribe® Online Marketplace. See Marketplace TIBCO Scribe® Certified Connectors for more information.

Supported Entities

The Twitter Connector supports the following entities. Select a linked entity name for additional information when using that entity in TIBCO Scribe® Online.

Entity Query Create Update Delete Upsert

Place

X

 

 

 

 

Status

X

 

 

 

 

User

X

 

 

 

 

Setup Considerations

To use the Twitter Connector you need a Twitter User account.

API Usage Limits

Twitter limits the number of API calls in a sliding 15 minute window for the user communicating with its API. Users that exceed those limits may be blacklisted and will not be able to access the Twitter API. TIBCO Scribe® Online monitors API usage to prevent you from exceeding the API limit. TIBCO Scribe® Online tracks the number of remaining API calls in the current 15 minute window and stops the query before the limit is exceeded.

If you are running a Solution and it is close to exceeding the API usage limits, the Solution stops running and TIBCO Scribe® Online generates the following error message:

API limit reached for method <name of entity>
Where <name of entity> is the name of the metadata entity being queried when the maximum was reached.

Note: To prevent you from exceeding your API limits, TIBCO Scribe® Online restricts the number of API calls used in the Preview option for a Query. This may result in the Preview option being blank where an actual query would return results.

Note: Refreshing metadata for the Connector consumes API calls.

Refer to the Twitter documentation for additional information on per user usage limits. See API Rate Limits.

Managing API Limits

Twitter API rate limits can cause TIBCO Scribe® Online Solutions to fail depending on the number of records requested. Below are some options to help you work within these limits.

Selecting An Agent Type For Twitter

Refer to TIBCO Scribe® Online Agents for information on available Agent types and how to select the best Agent for your Solution.

Connecting To Twitter

Note: Best practice is to create Connections with credentials that limit permissions in the target system, following the principle of least privilege. Using Administrator level credentials in a Connection provides Administrator level access to the target system for TIBCO Scribe® Online users. Depending on the entities supported, a TIBCO Scribe® Online user could alter user accounts in the target system.

  1. Select More > Connections from the menu.
  2. From the Connections page select Add to open the Add a New Connection dialog.
  3. Select the Connector from the drop-down list in the Connection Type field, and then enter the following information for this Connection:
    • Name — This can be any meaningful name, up to 25 characters.
    • Alias — An alias for this Connection name. The alias is generated from the Connection name, and can be up to 25 characters. The Connection alias can include letters, numbers, and underscores. Spaces and special characters are not accepted. You can change the alias. For more information, see Connection Alias.
  4. Select Authenticate to open the Twitter authorization page:
    • If you are not already logged into Twitter, the Twitter login page displays. Enter your email address and password, and select Log in.
    • If this is the first time you are creating a Twitter connection, the Twitter Authorization page displays. Click Authorize to allow Twitter to use the data specified.

      A page with the following message displays:

    • Click Close, return to the TIBCO Scribe® Online page, and then select Test to ensure that the Agent can connect to Twitter. Be sure to test the connection against all Agents that use this connection. See Testing Connections.
    • Tip: Do not test the Connection until the message at the top of the Connection dialog reads Authentication complete. Note that it may take a few minutes for this message to display.

  5. If you are using Replication Services, enter filter information in the RS Filter field. This field is required.
  6. In the Sentiment Scoring Values table select Add to enter a specific word or phrase and either a positive or negative number. Maximum of 50 entries. This field is optional. See Sentiment Scoring Values for additional information.
  7. Select OK to save the Connection.

Sentiment Scoring Values

Use the Sentiment Scoring Values table in the TIBCO Scribe® Online Manage Connections dialog for Twitter to search tweets for keywords or phrases and give each occurrence a positive or negative score. The cumulative sentiment score indicates whether the tweet is negative or positive, which helps you determine how to act on that tweet. The score given to each keyword is user specified and should be based on your own calculation for what constitutes a negative or positive total score.

TIBCO Scribe® Online uses the data in the sentiment filter as follows: 

Scores are cumulative and are stored in the target datastore in the sentiment field. For example, if you have entered good with a score of 100 and bad with a score of -25, a tweet of this phrase, "He is neither good nor bad.", generates a sentiment score of 75 or 100 + (-25)= 75.

Sentiment Scoring Values can be used with both Replication and Integration Solutions. If the Sentiment Scoring Values table is populated, a sentiment score is always generated for tweets that match the criteria. Create additional Twitter Connections to store different sets of sentiments or to remove all sentiments for instances when you do not want any sentiment scores.

Metadata Notes

Consider the following for Twitter data fields and entity types.

Naming

Connection metadata must have unique entity, relationship, and field names. If your Connection metadata has duplicate names, review the source system to determine if the duplicates can be renamed.

Entity Types

Twitter entities, as used in TIBCO Scribe® Online, are classified as either main entities or sublists.

Twitter Connector As RS Source

Consider the following when using the Twitter Connector as RS Source.

Twitter Connector As IS Source

Consider the following when using the Twitter Connector as an IS source.

Filter, Match Criteria, Or Lookup

Notes On Standard Entities

Place

Placeattribute

Status

Contributor

Media

User

TIBCO Scribe® Online API Considerations

Note: Connectors using OAuth for authentication are not fully supported by the TIBCO Scribe® Online API at this time.

More Information

For additional information on this Connector, refer to the Knowledge Base and Discussions in the TIBCO Community.