« »


My father-in-law is obsessed with Twitter poetry, or tweepetry, as he calls it. As someone incapable of succinct expression, I’m a great admirer of the masters of the 140 character quip, foremost among them the inimitable Anderson Miller and the by now overfollowed Shitmydadsays. I’m not sure anyone really knows what the point of Twitter is, but it does lend itself to recording felicitous turns of phrase, so I applaud my pop-in-law’s to up the literary ante.

As a gift last year, I made him a site that aggregates tweepetry using Seaofclouds’s nifty little Twitter-scraping javascript. The concept was super simple: tag any tweet with #tweepetry and it automatically appears on tweepetry.com. I added a couple of snippets to the code to remove the hash tag and done!

It worked great for a couple of weeks. Then the bits of tweepetry that had appeared on the site began disappearing. After three weeks, they were all gone. I did some research into the Twitter Search API and it turns out that only the last 10 days or so are indexed. Anything older than that can no longer be fetched dynamically using search. So tweepetry.com sat empty for six months gathering internet dust.

After three months of database programming, I realized I’d amassed the tools to resolve this problem so I revisited the site, rewriting everything but its visual elements from scratch.

Some recent tweeps


The site is still dynamic, but the dynamism is all backstage.

There’s now a php script running on my server that polls Twitter every 10 minutes (the maximum number of searches from one IP is limited to 150 per 24 hours). It talks to a MySQL database with two very simple tables, the first to recorded the id, username, time, and text of a particular tweet and the second to keep track of the id of the last tweet. I used two tables to keep my database queries fast. Every time the script runs, it searches Twitter for tweets that include the “#tweepetry” hash tag. If it finds some, it compares the id of the latest with the id stored in the second table. If they’re the same, nothing happens, but if they’re different, each tweet is stored in the database until the id of the last recorded tweet is reached, at which point it is updated to reflect the new additions. Then there’s a simple script that fetches all this information and displays it. That’s it!


Comments are closed.