Until recently, Twitter was among the most studied social media platforms in the world. Its accessibility to researchers, coupled with its role in key moments of modern history—from the Arab Spring to the COVID-19 pandemic—made it an indispensable source of data. But in March 2023, the company’s new ownership sharply restricted access to its data, […]
I recently had to learn gsutil, a Python package used to interact with Google Cloud Platform. Overall, I guess it was easy enough to learn, though there was one pain point. When I was in the GCP bucket containing the directories I wanted to download, I tried to download the directories directly using the graphical […]
Now that Twitter has released their Academic Research product, I thought I would jot down my initial thoughts. These observations are based on beta testing I was allowed to do in 2020 with the Search endpoint and reading documentation of the other endpoints. I have used Twitter’s API for about 7 years now. Most of […]
My Twitter pipeline collects the tweets with R but processes them with Python, but tor the event data class I just taught at APSA (material available here), I wanted to keep all of the code in R. I therefore needed to replace my Python code with R code that would open text files containing raw […]
My research uses Twitter to understand the dynamics of protest and state violence, which also means I think a lot about media coverage. I have a few thoughts on recent events. TL;DR: There is very little looting going on across the United States. There are lots and lots of peaceful protests and protesters. You would […]
My primary source of data is tweets I get from Twitter’s POST statuses/filter endpoint, what I believe was called the “Streaming Endpoint” when I started working with Twitter data eons ago. While it has always been straightforward to use a bounding box to get tweets with geographic information, exactly what Twitter reports and how it […]
Like almost every other academic, I have started a Covid-19 project. I think my team has a unique angle because of the kind of data I collect. One dynamic we are interested in is patterns of following, and being able to analyze that across enough accounts required me to work with Twitter endpoints I have […]
I recently spent a lot of time reviewing my Twitter data collection infrastructure in order to start some more collections. In that process, I discovered some tokens and streams I forgot about. The purpose of this post is to document what data I am collecting as of 04.29.2020 so that I have an easy reference […]
Today, I need to launch a new AWS EC2 instance to collect some more tweets. Since I have done this several times, I used my AWS console to replicate the most recent instance I created that collects tweets. Everything ran smoothly until I tried to install streamR, my go to library for collecting tweets in […]
With some free time on my hands, I sat down to update my code that extracts tweets from my tweet collection based on user-supplied keywords or locations. In doing that, however, I ended up making a major improvement, one that should have existed from day one. You see, simply trying to read a file of […]
zacharyst.com 