How to Get Trending Tweets in any Country with Python and Tweepy

Created on Sep 5, 2020
Updated on Mar 21, 2022

Social Analytics companies have been massively using twitter to get insights about whatever data they are interested in for brands, celebrities, etc. and also trending topics. In this tutorial you’ll know how to get countries that have trends on Twitter and also get insights about what topics are trending the most and be able to retrieve the url of that tweet and its volume as well.

Here I’ll walk you through how you can do that with Python and Tweepy. You can do many other things with Tweepy other than trending topics but in this tutorial I will focus on getting trends. Let’s dive in and see what Tweepy can do for us.

Note: Check out how to get trending hashtags worldwide if you’re interested.

What is Tweepy?

Tweepy is an API wrapper for twitter; you can use it to get some data from twitter, to know some information about your account and get some insights about public data.

Tweepy Installation

Tweepy is in PyPy, so you can use pip to install it:

pip install tweepy

An alternative way to do it, is to install it from its GitHub repo:

pip install git+https://github.com/tweepy/tweepy.git

Tweepy supports Python 2.7, 3.5, 3.6, 3.7, & 3.8

Twitter Developer Credentials

First thing you need to do is to have a twitter developer account . If this is your first to set up a twitter developer account, it may take time from twitter’s side to accept your request. Once your account is accepted, you can click on Developer Portal tab at your developer account and hover over your name and select Apps from the dropdown. There you can create a new app, you’ll be asked some questions regarding your app. You should generate your tokens and it’s a good practice to save them at password manager like passpack for example; it’s a free software for storing and sharing passwords.

Authentication and Authorization

Now, you have four credentials for your app:

Let’s understand why API keys are different from access tokens. Short answer is API keys are used for authorization, while tokens are for authentication. What does that mean?

Before we know that, we need to know what we should do first in order to do our task. What you want is to let twitter know who you are to give you access to the data that you want. Twitter gives you permissions through authorization and then asks you to generate tokens which know who you claim to be (authentication), this is like logging in twitter account after you’ve been authorized that you have the privilege to read and/or write access.

Environment Variables

It’s recommended not to hardcode these credentials in our script so make sure to have them as environment variables. Here is the script so far:

import tweepy
import os
import json
import sys
import geocoder

# API Keys and Tokens
consumer_key = os.environ['API_KEY']
consumer_secret = os.environ['API_SECRET_KEY']
access_token = os.environ['ACCESS_TOKEN']
access_token_secret = os.environ['ACCESS_TOKEN_SECRET']

# Authorization and Authentication
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)

What we need to do is to let our script know our environment variables.

So let’s assign them through for the following cases:

MacOS or Linux

$ export API_KEY='Your API Key'
$ export API_SECRET_KEY='Your API Secret Key'
$ export ACCESS_TOKEN='Your Access Token'
$ export ACCESS_TOKEN_SECRET='Your Access Token Secret'

Windows

  1. On the Windows taskbar, right-click the Windows icon and select System .
  2. In the Settings window, under Related Settings , click Advanced system settings .
  3. On the Advanced tab, click Environment Variables .
  4. Click New to create a new environment variable.
  5. Add the following environment variables:
  6. API_KEY variable with the value you got from Twitter Developer
  7. API_SECRET_KEY variable with the value you got from Twitter Developer
  8. ACCESS_TOKEN variable with the value you got from Twitter Developer
  9. ACCESS_TOKEN_SECRET variable with the value you got from Twitter Developer
  10. After creating or modifying the environment variables, click Apply and then OK to have the change take effect.

If you are interested in getting available locations that have trends at Twitter around the world, you can use Tweepy’s method available_trends.

Let’s see what we have until now:

import tweepy
import os
import json
import sys
import geocoder

# API Keys and Tokens
consumer_key = os.environ['API_KEY']
consumer_secret = os.environ['API_SECRET_KEY']
access_token = os.environ['ACCESS_TOKEN']
access_token_secret = os.environ['ACCESS_TOKEN_SECRET']

# Authorization and Authentication
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)

if __name__ == "__main__":
    # Available Locations
    available_loc = api.available_trends()
    # writing a JSON file that has the available trends around the world
    with open("available_locs_for_trend.json","w") as wp:
        wp.write(json.dumps(available_loc, indent=1))

Let’s say we want to get the trending tweets in Egypt. Instead of hardcoding ‘Egypt’ in the script we can pass it as argument variable.

Tweepy has a method closest_trends() and get_place_ trends() – combining the two gives us very similar result to what we’re looking for, closest_trends() just needs a longitude and latitude of the country we want and then it can return the a JSON file that has WOEIDs that we’re interested in. WOEID (or Where On Earth IDentifier) is a unique identifier for any feature on earth. For example, WOEID for New York is 2459115 and WOEID for Los Angeles is 2442047. Although both reside at the United States, they both have unique WOE ID.

One note to consider, just install geocoder which is a library that helps you get location relevant information and we need it here to get the longitude and latitude.

Install with Pip

pip install geocoder

Install with Conda

conda install -c conda-forge geocoder

Make sure when you run the final script to add the argument variable of the country that you want:

python tweetstrending.py Egypt

Enjoy!

Conclusion

Now, you can use Tweepy to get some the latest 50 trending topics on twitter, but make sure of number of requests you have if you’re using it extensively .. because you have a limit of 100,000 requests per day. That’s according to Twitter on June 19, 2019

Tweepy has a lot more than trending methods, I hope this tutorial is useful and maybe motivational to read more about Tweepy.

And if you’re interested in getting trending hashtags, I wrote a post here . Check it out!

Resources

Heads up

If you want more help with your own code using Twitter, I offer Twitter data collection services and I also can help you with your customized code when you use Twitter API. Be sure to check out my Upwork project and let me know that you’ve landed on this page to get a discount.

If you want to hire me on Upwork for any specific role in the stack I use, Check out my profile and let’s do it together.

Published in medium