Twitter is a popular source to extract text data related to any product, company, individual or event. Let us consider an example of the Cricket World Cup which just ended. Twitter has been a hot platform of discussion. Thousands of comments were posted from viewers and cricket fans across the world over the past few weeks. Several hash tags were used for the same viz. #CWC, #CWC19, #CWC2019. It would be interesting to identify sentiment of tweets related to a hash tag by pulling and working on a collection of tweets.
In this article we will learn how to Connect to a Twitter API and fetch tweets using R.
A snapshot below shows some of the tweets for #CWC19
Things you will learn from this topic:
- Creating a Twitter App
- Code to establish connection with the app using R
- Fetching tweets
You will have to create an account on twitter (if you are a new twitter user) and then create an individual or a team developer account to create an app. So, let us begin.
Follow the below steps:
- Next a popup will show up as below. Hit the ‘Apply’ button and go to the next page. We are applying for a developer account with these steps
- Twitter requires us to fill in some details for using its developer tools. There are several options out of which we have selected ‘Teaching’ (showing in green colour) and clicked on the ‘Next’ button on the bottom left of the screen shot to proceed. You can check which ever options suits you
- The next page confirms us of our correct account identity. It also asks us if we are going to need a ‘Team developer account’ or an ‘individual developer account’. Based on our requirement we will proceed with the individual developer account. Look at the below screen shot, and the highlights marked
- After we click the yellow highlighted link as shown in the above screen shot, the next page takes us to the individual developer account page which looks like below. We need to fill some details as an individual developer here and proceed further
- This new page is to describe our intended usage of twitter data. Fill in the details as required and move to the next page
- Here we will be asked to review our answers, account and contact details. Proceed to the next page by hitting the “Looks good” button at the bottom right
- Finally, you agree to the terms and then make a Final Submission of the application to create an individual developer account
- Simultaneously, an email is also sent to the email id for verification. A popup tells the same information as well.
- After the email is verified, we have successfully created a developer account. It takes us to the developer page where we need to hit the ‘Create an app’ button again. We must fill some details about the app. Please fill the website URL to your website or blog URL. Do enable sign in with Twitter and give the call back URL as showed in the screen shot below
- We can leave these below sections blank. Answer other required questions and then proceed to create the app
- A popup highlights some developer terms before finally hitting the “create” button.
- Phew!!! We have successfully created our app. Next comes the connection between Twitter and R. Our app contained information related to Keys and Tokens. This API Key, API Secret Key information will be required while connecting with Twitter from R
Let us now see how to establish connection between our app and R with the information of API Key and API Secret Key. The code for that is as follows:
## whatever name you assigned to your created app
appname <- "Enter your app name"
## api key
key <- " Enter API Key value here"
## api secret
secret <- "Enter API Secret here"
## create token named "twitter_token"
twitter_token <- create_token(
app = appname,
consumer_key = key,
consumer_secret = secret)
Now let’s extract tweets from Twitter. If you would have noticed, we installed a package named rtweet while setting up the connection. That is the package we will use for this task. There is another package named twitteR which can be used as well. We will stick to the prior one. (We are assuming here that the reader has got some prior understanding of R.)
search_tweets is a function in rtweet package allowing us to search and extract tweets on a search string. You can learn all the features and options about this package here https://cran.r-project.org/web/packages/rtweet/rtweet.pdf
Code to extract tweets for hast tag #wimbledon2019
wimbledon_tweets = search_tweets("#wimbledon2019", n=2000, lang='en')
There are several ways to measure the sentiment of these tweets.
- We can identify sentiments based on lexicons
- In continuation to the above, we can train a machine learning model on sentiments identified and then use it to predict sentiments of test tweets
- We can use deep learning methods to identify sentiments
We will take on these topics in the next article. Hoping till then you will surely experiment with twitter API and find out the various ways in which you would want to put use of these interesting tweets.