When Twitter provides a tweet, the ‘created_at’ field provides a timestamp for when the tweet was authored. This timestamp is useful, but it cannot be used right away because it is in Greenwich Mean Time. Unless the tweet happens to have come from that timezone, its time needs to be adjusted to account for this discrepancy. […]
In regression modeling, it is common to correct standard errors for natural groupings (clusters) in the data. There are various ways to calculate these values using R, from doing it manually to using one of many packages. Theoretically, Zelig is an R package that will cluster standard errors automatically. In my experience, however, it does […]
I’ve been writing some scripts to work with Twitter’s REST API. Naturally, I went to their developer documentation to refresh myself on their rate limits. As of today, the link they provide to their rate limit chart is broken. Fortunately, I clipped this page to Evernote a long time ago. I was therefore able to […]
I recently had to read an Excel spreadsheet into R. Why Excel? The original data were in a Google Sheet, and it appears that Google downloads everything to a .xlsx. (There HAS to be a way to download to .csv, but I did not feel like searching.) Opening the file – it was only 12 […]
How do you parallelize a function with multiple arguments in Python? It turns out that it is not much different than for a function with one argument, but I could not find any documentation of that online. An “embarrassingly parallel” computing task is one in which each calculation is independent of the ones that came […]
A great way of conveying regression results is through a forest plot. Widely used in meta-analyses to compare results across models, they are also a convenient way to visualize regression results. Wanting to make one for a presentation, I naturally turned to R and its seemingly infinite packages. The package the internet recommends is forestplot. […]
The chart below replicates the data presented in my earlier post about Twitter but ranked by accounts per million inhabitants. Antarctica, of course, does not really have the greatest number of accounts per capita. In this sample, however, Twitter identified 4 tweets from there, and Antarctica has a population of 0, as do the 2nd […]
How many followers does the average Twitter user have? How many accounts does the average Twitter account follow? How many times has the average account tweeted? What about the median? These questions seem simple, but it is not easy to find answers to them. Twitter only discloses how many monthly active users exist, and other […]
UPDATE: Thanks to @icews for helping me figure this out. It turns out that the CAMEO Code field is saved as a string, but Pandas interprets that column as integers and drops the leading zero. To read that column correctly, use the following line: data = pd.read_csv(/Data/ICEWS/events.2010.20150313084533.tab’, sep = ‘\t’, dtype={‘CAMEO Code’: object}) —————————————————————————————————————————————————— Wanting […]
ICEWS was released to the public on April 1st, and the event studies community has had a field day getting to know this early (or late?) holiday present. The dataset, which was created by Lockheed-Martin on behalf of the Department of Defense, appears to represent the new frontier for historic events data. I use the […]
zacharyst.com 