Social Analytics companies have been massively using twitter to get insights about whatever data they are interested in for brands, celebrities, etc. and also trending topics. In this tutorial you’ll know how to get countries that have trends on Twitter and also get insights about what topics are trending the most and be able to retrieve the url of that tweet and its volume as well.
Here I’ll walk you through how you can do that with Python and Tweepy. You can do many other things with Tweepy other than trending topics but in this tutorial I will focus on getting trends. Let’s dive in and see what Tweepy can do for us.
Note: Check out how to get trending hashtags worldwide if you’re interested.
What is Tweepy?
Tweepy is an API wrapper for twitter; you can use it to get some data from twitter, to know some information about your account and get some insights about public data.
Tweepy is in PyPy, so you can use pip to install it:
pip install tweepy
An alternative way to do it, is to install it from its GitHub repo:
pip install git+https://github.com/tweepy/tweepy.git
Tweepy supports Python 2.7, 3.5, 3.6, 3.7, & 3.8
Twitter Developer Credentials
First thing you need to do is to have a twitter developer account . If this is your first to set up a twitter developer account, it may take time from twitter’s side to accept your request. Once your account is accepted, you can click on Developer Portal tab at your developer account and hover over your name and select Apps from the dropdown. There you can create a new app, you’ll be asked some questions regarding your app. You should generate your tokens and it’s a good practice to save them at password manager like passpack for example; it’s a free software for storing and sharing passwords.
Authentication and Authorization
Now, you have four credentials for your app:
- API Key
- API secret key
- Access token
- Access token secret
Let’s understand why API keys are different from access tokens. Short answer is API keys are used for authorization, while tokens are for authentication. What does that mean?
Before we know that, we need to know what we should do first in order to do our task. What you want is to let twitter know who you are to give you access to the data that you want. Twitter gives you permissions through authorization and then asks you to generate tokens which know who you claim to be (authentication), this is like logging in twitter account after you’ve been authorized that you have the privilege to read and/or write access.
It’s recommended not to hardcode these credentials in our script so make sure to have them as environment variables. Here is the script so far:
import tweepy import os import json import sys import geocoder # API Keys and Tokens consumer_key = os.environ['API_KEY'] consumer_secret = os.environ['API_SECRET_KEY'] access_token = os.environ['ACCESS_TOKEN'] access_token_secret = os.environ['ACCESS_TOKEN_SECRET'] # Authorization and Authentication auth = tweepy.OAuthHandler(consumer_key, consumer_secret) auth.set_access_token(access_token, access_token_secret) api = tweepy.API(auth)
What we need to do is to let our script know our environment variables.
So let’s assign them through for the following cases:
MacOS or Linux
$ export API_KEY='Your API Key' $ export API_SECRET_KEY='Your API Secret Key' $ export ACCESS_TOKEN='Your Access Token' $ export ACCESS_TOKEN_SECRET='Your Access Token Secret'
- On the Windows taskbar, right-click the Windows icon and select System .
- In the Settings window, under Related Settings , click Advanced system settings .
- On the Advanced tab, click Environment Variables .
- Click New to create a new environment variable.
- Add the following environment variables:
- API_KEY variable with the value you got from Twitter Developer
- API_SECRET_KEY variable with the value you got from Twitter Developer
- ACCESS_TOKEN variable with the value you got from Twitter Developer
- ACCESS_TOKEN_SECRET variable with the value you got from Twitter Developer
- After creating or modifying the environment variables, click Apply and then OK to have the change take effect.
If you are interested in getting available locations that have trends at Twitter around the world, you can use Tweepy’s method available_trends.
Let’s see what we have until now:
import tweepy import os import json import sys import geocoder # API Keys and Tokens consumer_key = os.environ['API_KEY'] consumer_secret = os.environ['API_SECRET_KEY'] access_token = os.environ['ACCESS_TOKEN'] access_token_secret = os.environ['ACCESS_TOKEN_SECRET'] # Authorization and Authentication auth = tweepy.OAuthHandler(consumer_key, consumer_secret) auth.set_access_token(access_token, access_token_secret) api = tweepy.API(auth) if __name__ == "__main__": # Available Locations available_loc = api.available_trends() # writing a JSON file that has the available trends around the world with open("available_locs_for_trend.json","w") as wp: wp.write(json.dumps(available_loc, indent=1))
Trending Tweets for Specific Country
Let’s say we want to get the trending tweets in Egypt. Instead of hardcoding ‘Egypt’ in the script we can pass it as argument variable.
Tweepy has a method closest_trends() and get_place_ trends() – combining the two gives us very similar result to what we’re looking for, closest_trends() just needs a longitude and latitude of the country we want and then it can return the a JSON file that has WOEIDs that we’re interested in. WOEID (or Where On Earth IDentifier) is a unique identifier for any feature on earth. For example, WOEID for New York is 2459115 and WOEID for Los Angeles is 2442047. Although both reside at the United States, they both have unique WOE ID.
One note to consider, just install geocoder which is a library that helps you get location relevant information and we need it here to get the longitude and latitude.
Install with Pip
pip install geocoder
Install with Conda
conda install -c conda-forge geocoder
Make sure when you run the final script to add the argument variable of the country that you want:
python tweetstrending.py Egypt
Now, you can use Tweepy to get some the latest 50 trending topics on twitter, but make sure of number of requests you have if you’re using it extensively .. because you have a limit of 100,000 requests per day. That’s according to Twitter on June 19, 2019
Tweepy has a lot more than trending methods, I hope this tutorial is useful and maybe motivational to read more about Tweepy.
And if you’re interested in getting trending hashtags, I wrote a post here . Check it out!
- Tweepy Documentation
- 100 Scripts in 30 Days challenge: Script 18,19,20 — Getting trending topics on Twitter using Tweepy
- Tweeting with Python
- Photo by Morning Brew on Unsplash
If you want more help with your own code using Twitter, I offer Twitter data collection services and I also can help you with your customized code when you use Twitter API. Be sure to check out my Upwork project and let me know that you’ve landed on this page to get a discount.
If you want to hire me on Upwork for any specific role in the stack I use, Check out my profile and let’s do it together.
Published in medium