Crawling Followers with Intelligent Stopping

Like almost every other academic, I have started a Covid-19 project.  I think my team has a unique angle because of the kind of data I collect.  One dynamic we are interested in is patterns of following, and being able to analyze that across enough accounts required me to work with Twitter endpoints I have […]

My Ongoing Twitter Collections

I recently spent a lot of time reviewing my Twitter data collection infrastructure in order to start some more collections.  In that process, I discovered some tokens and streams I forgot about.  The purpose of this post is to document what data I am collecting as of 04.29.2020 so that I have an easy reference […]

Cell Phone or Geolocation Datasets

I want cell phone data.  Location tracking or call detail records, both have their users and can answer research questions.  Basically nothing is publicly available, but lots of companies exist and some say they will work with academics.  This post is for me to keep track of those companies. SafeGraph – Focused on places, not […]

What I Read, 2019 Version

Starting with the 19th book review (Mao biography), I have decided to add a grade and, to counterbalance what can often seem like negative reviews, one interesting fact learned from each book.  I aim for a B- average.  The Sellout by Paul Beatty – Wow, what a novel.  My wife bought it for me for […]

Who I Saw, 2019 Edition

Tim Gunn (April 2019) – My wife always raves about the celebrities she saw while a student in NYC, so I was cautiously optimistic during a weekend we spent there.  While we did not see anyone in the Village, we did cross paths with Tim Gunn as we walked from Levain Bakery to the Museum […]

National Level Determinants of Twitter Usage

In responding to recent comments on a paper, I went looking for information on how prevalent Twitter is across the world.  Based on my work using geolocated images from Twitter, I had a sense that it tracks population and per capita GDP closely.  But I have not had the time to investigate the relationship more […]

Dynamically Changing Variables to Plot with ggplot

I recently had a large dataset where I wanted to make a series of plots with different variables on the y-axis.  I remembered that a few years ago I great frustrated that I could not do this with ggplot, so I moved on.  This time, I was successful. The key, as this Stack Overflow answer […]

Peak Design Everyday Backpack is not an Everyday Bag

That headline misleads slightly: the 30L backpack is a good everyday bag if photography is part of your everyday.  If your everyday involves traditional office work and commuting, I suggest looking elsewhere for a bag.  That is what I did, and I own happily the Nomatic Backpack. The internet world is in love with all […]