Tag Archives: Python

Proper Handling of Exceptions in Python

With some free time on my hands, I sat down to update my code that extracts tweets from my tweet collection based on user-supplied keywords or locations.  In doing that, however, I ended up making a major improvement, one that should have existed from day one. You see, simply trying to read a file of […]

I prefer simplejson to json

I thought I was going to spend some time on Friday analyzing tweets from Cameroon.  Instead, starting that process led me down a rabbit hold that has, I hope, culminated in me realizing I should have used Python’s simplejson library this whole time. A script of mine used a try-except sequence to enclose the section […]

How to Profile Python Code, With an Aside on Parallelism

I recently wrote a script that reads thousands of files  of tweets, transforms them, and spits out “only” hundreds of files.  Having tested the script on my computer on a few files, I was surprised to find the execution taking much longer than anticipated on my server, especially since the server’s CPUs are more powerful […]

Header for GDELT 2.0 and Phoenix

Working with machine-coded events data is cool.  What’s not cool is that the raw data from two of the main datasets, GDELT 2.0 and Phoenix, do not include headers in their files.  It is simple to create a list with the column names, but the closest I could find that already existed for GDELT 2.0 […]

Assigning the Correct Time to a Tweet

When Twitter provides a tweet, the ‘created_at’ field provides a timestamp for when the tweet was authored.  This timestamp is useful, but it cannot be used right away because it is in Greenwich Mean Time.  Unless the tweet happens to have come from that timezone, its time needs to be adjusted to account for this discrepancy. […]

Parallelize a Multifunction Argument in Python

How do you parallelize a function with multiple arguments in Python? It turns out that it is not much different than for a function with one argument, but I could not find any documentation of that online. An “embarrassingly parallel” computing task is one in which each calculation is independent of the ones that came […]