Understanding Subnational Variation in Tweets

My primary source of data is tweets I get from Twitter’s POST statuses/filter endpoint, what I believe was called the “Streaming Endpoint” when I started working with Twitter data eons ago.  While it has always been straightforward to use a bounding box to get tweets with geographic information, exactly what Twitter reports and how it […]

Crawling Followers with Intelligent Stopping

Like almost every other academic, I have started a Covid-19 project.  I think my team has a unique angle because of the kind of data I collect.  One dynamic we are interested in is patterns of following, and being able to analyze that across enough accounts required me to work with Twitter endpoints I have […]

My Ongoing Twitter Collections

I recently spent a lot of time reviewing my Twitter data collection infrastructure in order to start some more collections.  In that process, I discovered some tokens and streams I forgot about.  The purpose of this post is to document what data I am collecting as of 04.29.2020 so that I have an easy reference […]

The Lazy Way Around Updating my gcc Compiler

Today, I need to launch a new AWS EC2 instance to collect some more tweets.  Since I have done this several times, I used my AWS console to replicate the most recent instance I created that collects tweets.  Everything ran smoothly until I tried to install streamR, my go to library for collecting tweets in […]

Assigning the Correct Time to a Tweet

When Twitter provides a tweet, the ‘created_at’ field provides a timestamp for when the tweet was authored.  This timestamp is useful, but it cannot be used right away because it is in Greenwich Mean Time.  Unless the tweet happens to have come from that timezone, its time needs to be adjusted to account for this discrepancy. […]