Crawling Followers with Intelligent Stopping

Like almost every other academic, I have started a Covid-19 project.  I think my team has a unique angle because of the kind of data I collect.  One dynamic we are interested in is patterns of following, and being able to analyze that across enough accounts required me to work with Twitter endpoints I have […]

My Ongoing Twitter Collections

I recently spent a lot of time reviewing my Twitter data collection infrastructure in order to start some more collections.  In that process, I discovered some tokens and streams I forgot about.  The purpose of this post is to document what data I am collecting as of 04.29.2020 so that I have an easy reference […]

What I Read, 2019 Version

Starting with the 19th book review (Mao biography), I have decided to add a grade and, to counterbalance what can often seem like negative reviews, one interesting fact learned from each book.  I aim for a B- average.  The Sellout by Paul Beatty – Wow, what a novel.  My wife bought it for me for […]

Who I Saw, 2019 Edition

Tim Gunn (April 2019) – My wife always raves about the celebrities she saw while a student in NYC, so I was cautiously optimistic during a weekend we spent there.  While we did not see anyone in the Village, we did cross paths with Tim Gunn as we walked from Levain Bakery to the Museum […]

Clearinghouse of Advice for New Assistant Professors

[Most recent update: 11.21.2019.] The purpose of this post is to catalogue advice from the internet about how to achieve tenure at a research university. When I was a PhD student, one method of calming my anxiety was to read advice from professors to PhD students; The Professor is In, Fabio Rojas, and Chris Blattman are particularly helpful.  Now […]

group_by() %>% mutate() using pandas

While I have my issues with the tidyverse, one feature I am enamored with is the ability to assign values to observations in grouped data without aggregating the data.  This assigning is done by using the mutate() command instead of summarize().  I am in the middle of some data processing in a Python pipeline where I […]