90 days of node.js - Day 8 - Connecting to Twitter REST api

Day 8 of my node.js discovery went surprisingly well.

For one thing, I've narrowed down how much time I can reliably give to this project on a daily basis. I find a lot of node.js posts full of planned itineraries and lots of code from day 0. Seems to me that this is both unreasonable and overly optimistic for someone who has a kid and a house that needs constant attention. 2 hours is my limit. I don't want to do any exploratory stuff at work because of fairly strict legal guidelines at work. Better to publish any public work from home. I would definitely advise people to be realistic about how much time they can devote instead of just gong through sleepless nights.

I also figured that part of my time should go in developing some code and part in learning more on node. The development part is what is really appealing and stumbling through the tutorials can be maddening if I can't keep pushing out incremental changes to the application.

Day 8's goal was to connect to the Twitter API and grab lists as an application would. I had initially hoped to get all lists, but later settled for public lists only.

Authorization with Twitter

The first step is being able to access twitter and getting authorization from twitter to access public lists. I went with the approach of using Application Only Authentication. This is a "simple" mechanism for applications to obtain authorization tokens from twitter to access or modify data. I put the quotes around, because this whole thing ends up being a lot more complicated than it seems. You need to understand oauth2 flows to some extent. Twitter goes quite far to hide the details from you, so it is convenient to use. From the Twitter end, you need an application registered with Twitter to give you 2 pieces of the auth puzzle - a customer key and a customer secret. For now this app I have created is not accessible for installation, so I won't spend more time on it. From the node code perspective, turned out that the trickiest part was figuring out how to introduce these values in the workflow. I found a helpful stackoverflow comment that introduced me to using a module for declaring my config. Easy to use and conceptualize.

Calling the REST API to get tokens

The token api is a direct POST over https. This had me stumped for a while till  I figured out two things:
  1. The path parameter for the http(s) modules of node should NOT have the protocol in it. Seems obvious in hindsight, but somehow I completely missed that.
  2. There is a https module. Again, obvious in hindsight, but took me a few minutes to search the doc for.
Once those things were settled the rest was fairly easy. Posting the credentials got me back the access token and I could use it for the next step.

Calling the REST API to get public lists

This was relatively the quickest thing that I spent time on. The API endpoint is self-explanatory and pretty decently documented.

And thus I have my first reasonable working node module. You can find it on twitter-list's github home

Next steps

Simple Twitter List is the provisional name for a simple Twitter list (d'oh) editing app I plan to build to learn more about node. I love using lists to manage my TL and find the current landscape a little slim on options. The main goal is to learn about node, but scratching an itch while learning seems the right way to go. I am going to be using the twitter-list module a lot more in the days to come.

There is already a twitter node module which does a great job of working with the twitter api for node apps. I would recommend using it for a real production problem.

For next steps, I am going to do some refactoring of the code and break out the auth token generator and the calling code into separate functions. The next important change is using user credentials instead of just app only authorization. That opens up a large vista of read-write actions I can take with the app. Finally, an app that runs on cli is pretty much useless when interfacing with twitter. In order to get a simplistic UI and running I am going to invest time in ExpressWorks - a workshopper to learn express.

I am starting to quite like workshopper and will try to contribute something just to give back I think. Their cli is pretty sweet and inspite of some frustration it has been quite nice to work with.

Onward and upward.