90 days of node.js - Days 11 and 12 - Domains, Jade and Oauth

In case you don't know what this is all about, the out-of-loop link describing my 90 day journey through node land.

90 sequential days looks more and more of a pipe dream. What with work, family, books, podcasts and mowing the lawn, I've had quite a bit of a break. Days 11 and 12 ( yay, 10+ days) were more of a slog than any fun. I did pick up two interesting technologies though that I intend to use a lot more (hopefully professionally). I also find that blogging on a daily basis is down right hard, especially since I still want to write about books when I get a chance.

Let's talk days 11 and 12.

Jade isn't strictly node.js. It is a powerful templating technology. Over the years, I 've used a mish-mash of templating technologies - jsp, gsp, underscore and freemarker. Given all that, jade was a whiff of fresh air. Here's a sample from my checked in code:

h1 Showing Public lists of #{screenName}
    a(href="/twitter-authorize")Signin with Twitter
        span Lists
        span Members
        span Subscribers
        span Tweets
The output is pretty obvious - a forest of divs :). The great feature though are simple things. The markup is nice and indented, the ids and classes are applied with the least of ceremony and all html element attributes are treated as object properties. You can even inline javascript and declare custom tags (which are called mixins, but you know what they are). I'm really groking the simplicity this brings to the layout. Granted integrating it with something like bootstrap can be a bit of a pain, especially since the list of classes applied to elements can get too long, but I'm guessing the mixins will ease some of the pain.

The cool node thing that I stumbled into though was domains. According to the excellent node.js docs:

Domains provide a way to handle multiple different IO operations as a single group

Till my server started dying on errors, I didn't realise that the node processes kill themselves on uncaught errors. Listening to uncaughtException events leads to loss of context. Creating a domain and having it run the specific process can easily allow us to listen to errors and not fail miserably on errors.

I was trying to do some exception handling in my express server. Thanks to my choice in using promises and throwing explicit Errors when handling async calls, I was stuck with having to add a way to handle uncaught exceptions thrown by the async process.

Enter, connect-domain. It's a middleware which slots into the express middleware paradigm and allows a way to pass on the error all the way to a designated error handler. The sample code indicates how easy it is to use and I was able to start doing decent exception handling in the server. There is also a express-domain-middleware module that seems handy. I opted to ignore it for now, primarily because I am feeling an overdose of investigating modules. Unlike the javaland frameworks where things come bundled, having to install one module at a time is both time consuming and hard to sustain when beginning. There is documentation to read, figure out what the dependencies are and hack through a lot of code before "things just work"™

Which brings me to the final useful modules that I used. OAuth has been a most handy module. I tried to implement the OAuth provider on my own for Twitter login - probably not the smartest idea of all time. I killed a lot of cycles understanding and reading about oauth. By itself, reading and learning about oauth is a win, but taking into account the fact that I am not trying to build a simple twitter app, seemed overkill. There is also passport - the types of login it allows you to do is mind boggling. But again, I didn't need the wide range of login support that passport provides. So I reinvented a spoke of the big wheel. Definitely use passport for production apps. (unless you hate all the dependencies)

I find that my choices for technologies are highly colored by my experiences writing Java web applications (spring mvc or grails). Instead of CLI, which seems to be node's best experience, I opted for express since that is closer to my niche. Jade and templates are kin to jsps, so that flows well. Middleware makes perfect sense when you think of how many aspects and filters are involved in writing web apps. The next things in a Java app would have been wiring a real database and writing some tests. I still have to write some jasmine tests, so that will probably be my next step. After that I will probably dabble in a little mongo, go build that app using cli and do some documentation.

Fun stuff.


Book Review - If I die here who will remember me by Vedica Kant

The First World War was a tremendous event that ruptured the old order in many ways. India's role in the First War has always been downplayed and under documented for a number of reasons. India didn't gain freedom till the Second great war making the First a distant memory. Since there was no Indian nation yet, any forces sent from the sub continent tended to be Empire soldiers or soldiers of princely states - both notions making them somewhat less than Indian for later commentators. The post-war period saw nationalists agitating against the British and the great non-cooperation movements were initiated, making the memory of collaboration with the British Empire a distasteful memory. Most of the focus of history textbooks in this period has been the appearance of Gandhiji and his agitational politics.

Still, more than a million soldiers from the sub continent fought and died fighting the Central Powers. Their actions ranged the entire world - Indian soldiers fought in Europe, in Central Asia and in Africa. There hasn't been much commentary on this period of warfare that traces the fighting Indian soldier. Vedica Kant fills that gap with her excellent - and quite aptly named - If I Die Here Who Will Remember Me. The name though dramatic, brings into stark relief the lack of interest and absence of memory for the soldiers of Indian origin who fought and died in the First World War. It's less a narrative history and more of a documentation of poignant letters and photographs from the First War which in itself makes for a pleasant reading.

Kant traces the fighting Indian soldier from the trenches in France and Belgium all the way to the great battles in the Middle East which are more synonymous with the Indian effort. It narrates the inconsistency with which the Empire approached its native troops - at once exalting them for their loyalty while segregating them from the population they were defending. It is remarkable to read about - and see photos of - soldiers fighting in the great trench wars in the early years of the war. For a long time, WW1 has been synonymous with either bland regiment names or British war heroes. Seldom does one get to read about other nationalities whose presence made this a World War. Kant has been rigorous in her narration of these forgotten soldiers who predicted the outsize role that India would play on the world stage in the years to come.

The book also talks about the segregated facilities established for the care and comfort of the Indian soldiers, taking care to appear magnanimous and generous in their care. Yet, the letters that recuperating soldiers wrote talk of the hurt and the frustration of the native in a land that needed them to fight but not to mingle. These letters, missives from a war front, went a great deal in introducing a huge chunk of the Indian population to what was inevitably 'kaala pani' - the black waters which none would dare cross. These soldiers presaged a massive movement of people from the Indian shores to the rest of the world which is fascinating to read about.

The author also draws upon the narrative of the Indian prisoners of war. The order of battle would find the soldiers segregated by caste and religion, but in the narrow confines of the prison camps, these distances collapse easily. The book goes further afield finding photos and paintings of POWs celebrating their festivals, playing their games and being googled upon by German soldiers - a poignant and heartfelt passage.The Germans, prodded by their Emperor,"Haji" Wilhelm, tried hard to win the affection and loyalty of the Indian soldiers - especially Muslims who were coerced into believing that since the Ottomans where German allies, their war was a holy one - a jihad. The Germans also tried to study and record impressions of the mannerisms of these exotic soldiers which would help them publish newspapers and missives aimed at converting POWs to their cause. A few Muslims did fight for the Germans in belief that they were fighting for their religion, but their number was few, much fewer than the Germans needed to foment rebellion in India itself.

The book also spends time on the impact in the home land - the enormous economic burden, the lack of Indian voices in this great decision to wage war and the subsequent denudation of a generation of youth. The author has done a great job of documenting the great mobilization effort which spanned the breadth of the country, but drew excessively from the north. The book also narrates the effect the returning soldier had on the polity and the letters and money that the fighting soldier sent from the front. The Indian leadership at this time was more interested in establishing the notion of self-rule and their loyalty to the Empire while odd to comprehend now, makes perfect sense for the goals they had set themselves then.

While not necessarily a complete history, this book has been a hidden gem. It's photographs, interspersed with letters, recordings and memorabilia from the war do an invaluable job of shedding light on the contributions of the Indian populace to the Great War. The enormous losses in Neuve Chapelle, Kut and in the Flanders have been largely forgotten in history books for a generation. It is incumbent on us to remember and revere the sacrifice of these soldiers who fought an alien war for alien kings far away from home. Vedica Kant fills a great gap in our knowledge of India's role in the Great War.

Highly recommended, 4 stars. 


Book review : In the name of democracy by Bipin Chandra

The 40th anniversary of the Internal Emergency seems like a good time to review Bipan Chandra's In the Name of Democracy - JP Movement and the Emergency

Indira Gandhi's emergency swept aside opposition parties, emasculated people (literally) and ended up being one of the blackest spots on our young democracy. It's direct roots, though, lie in two major events of the first half of the 1970s. Firstly, there was the rotten economic conditions in India with sky-rocketing prices and high inflation. The nationalisation of major corporations was in full steam , private entrepreneurs were being systemically wiped out and unemployment was on the rise.  Many of the reforms instituted by successive Congress governments at the center couldn't arrest the corruption rampant in the polity. There was a major economic divide not only among the different classes but also the different states and castes. The book does a good job of placing this economic state in context and offers a good insight into the state of the country before the emergency.

In this volatile environment stepped in Jaiprakash Narayan with the clarion call to wipe out corruption, reduce unemployment and bring down prices. Someone once said, to be a mass leader in India you have to be worshipped as a saint. JP fit that bill really well. A freedom fighter of renown, JP also was an incorruptible Gandhian. His call was heard across the populous state of Bihar and led to a mass movement which caught all political parties off-guard. After the great linguistic movements, this was a movement which threatened to rupture the fabric of democratic institutions in India. The author does a good job of collecting newspaper articles, private correspondence and records of speeches to describe the nature of the movement. With their demands for immediate redressal, intimidation of public officials and direct challenge to duly elected assemblies, the JP movement was a idealistic and short-sighted attack on our young democracy.

Bipan Chandra does a good job of threading together multiple sources and presenting the factual positions of people in the eye of the storm on both camps. He goes further on to analyse the roles of the leadership in the two camps. A major part of the book is focussed on the leadership abilities (or lack of in the author's opinion) of JP. JP comes across as an old man who blatantly encouraged a negative attitude towards representative democracy and elected officials. Forever preaching a cleaning up of corruption and communist influences, JP comes across as a leader unable to manage or control the revolution led on his behalf. The author effectively argues that JP had misread the emotions of the nation and had persuaded himself that a revolution - which he was the obvious leader of - was nigh and in a hurry to cement his position in history, he turned to direct demands for the prime minister's resignation which precipitated the emergency. Chandra's nuanced appraisal of JP, mostly scorn mixed with some admiration, is worth the read irrespective of what side of the divide one is on.

His other appraisal of the leaders in the governments role comes across as being a little short and more speculative. One problem is the secret nature of most of the documents and conversations of the upper echelons of the government. The naming and shaming of Sanjay Gandhi as the primary cause of the excesses of emergency tries to deflect some attention from Indira Gandhi's actions. The author carefully catalogs the actions of the government and lays out a case for why Emergency was inevitable in the charged atmosphere of the 70s. In the latter half of the book, the author goes into much detail of the successes of the emergency - however fleeting - in putting the economy back on track. This was an interesting read as it went some way in explaining the mystery of why people who were in Mumbai never seemed to feel the pinch of the emergency as much as someone in Delhi.

The most interesting part of the book is of course the hypothetical debate on the nature of the leadership. JP had let his leftist movement's leadership slip from his hands and into the hands of the RSS and other right-wing organizations. The political opportunism that led these opposition parties to band with JP is neither unexpected nor unknown. The author argues that if the movement had succeeded with this leadership India would have fatally titled away from democracy and towards fascism. He also takes the speculation a step forwards and imagines the rule of Sanjay and his coterie which would also lean far right, if their few months in Indira's kitchen cabinet were any indication. It is horrifyingly fascinating to imagine a right wing India with constrained freedoms and arbitrary rule by a gang of thugs whatever their professed political affiliation. The author does a good job of painting such a picture for the reader.

In a well written book the late historian argues for a nuanced evaluation of the Internal emergency that was promulgated in India on that fateful summer day 40 years ago today. It's well worth the read to understand the flow of events that led to the imposition of Emergency and its abrupt end. The parallels to the Anna Hazare movement is surprising. It is intriguing that a generation of Indians had to deal with 2 such quick fix movements which tried really hard to strike at the very roots of Indian democracy, with idealistic leaders who initially refused to convert their role from demagoguery to governance, but nevertheless tried hard to bring down elected governments and the final crackdown of the central government. The book is written before these events, so one can only imagine how the author would have drawn parallels.

There is one mystery though the author is unable to shed any light on - why did Indira end the emergency? If, and that is a big if, the private papers of Indira (who knows if they exist) are released by the government, they would make a fascinating read. The Emergency was a black mark on India's fledgling democracy. It has to be treated as a practical lesson in how a government when pushed into a corner, can work within the framework of the constitution and take away freedoms from people. It is also another objective lesson on how the nameless bureaucrat can be a source of great distress in the name of "doing one's duty". It's a fascinating period and anyone trying to understand India's democratic institutions should read more about it.

Well worth a read. 3.5 stars for it's lack of deeper study of the Indira coterie, but its really good evaluation of .


90 days of node.js - Day 10 Promises and refactoring

In case you don't know what this is all about, the out-of-loop link describing my 90 day journey through node land.

Day 10 was a fairly short day, but gave me some time to plan out what the next few days will be like.

One of the final goals of this whole node.js for 90 days is for me to build out a couple of real apps. I ended up identifying one app which I think will be easy to build together and will also help me pick up some interesting node.js libraries in a app to manage twitter lists. I love lists, but have always found the regular app pretty cumbersome to use. Ergo and itch I can scratch. I find that a structured goal can keep you highly motivated in any unstructured learning process (especially one started on a whim)

My focus today was to clean up the twitter client I originally made and modify it so it becomes a server - an express app. Right now all this app does is spit out a hunk of JSON when you make a request like so http://localhost:PROCESS.ARGV[2]/list?username=SCREENNAME. Not a major leap here, but a simple demonstration of the easiness in integrating  express with any simple request handler. It's still missing exception handling and timeout handling, but I'll get that in. It was a nice tie-in to the work of day 9.

The major change though was using Q. Q is a simple-to-use implementation of the Promises/A+ spec. I love Promises. I use jqeury's deferred right now in all my work and Q has impressed me with the ease in which the exceptions flow thru the promise chain and the simplicity in chaining and creating promises. I plan to use more of Q as time goes on, Personally, I have used Futures in Java before and that's pretty much what Promises are. If you have had to query across sharded databases and get a coherent result back all at once, this whole paradigm is a huge win. I'll leave Jake Archibald's excellent write up on promises to be your guide ( the demos were a little flaky, but I came for the code and stayed for the style and the laughs)

I am using Q quite extensively in the code right now. The idea is that since every call to the Twitter rest endpoints is async, I can either pass in callbacks all the way down OR return a promise at every level and have the calling code wait on when the promises are resolved. This makes it easy to conceptualize the flow of the code. It also makes error handling a breeze and I can use the promise chain to centralize the error messaging and logging. The inversion of control that you get with the promise informing the caller when it gets done, instead of the caller asking it is refreshingly simple to understand and use.

Returning a Q promise from my functions is a bit of a leap. As far as I am concerned, all I am doing is returning an object implementing the Promises/A+ spec. In theory this should work with es6 when promises are delivered as part of the spec, since all Q does is return an object with a then and fail function in line with the Promises/A+ spec. This is a deviation from the design choice that I have seen in node code for the most part - callbacks. Callback hell is real and here to stay ( even though many people I've interviewed for work roles have never heard of it - odd that). If I can contain my code to be nicely organized around a when...then...fail paradigm, I think the code reads better, the intent is clear and debugging is easier.

You can take a gander at the code in the github repo. It's fairly shoddy code right now, so the next few days will be spent in making a more robust server. The next steps that I have planned are :

Another major goal of this whole exercise has been (re)learning how to write good technical documentation. I can write, but I am not sure if I can be effective in explaining why I am choosing a certain code path. It seems to me I have an opportunity to learn how to effectively describe the ideas of design that I am pursuing. I futzed around with a few ideas and eventually latched on docco. Docco is what backbone and underscore use to generate the annotated source code. The annotated source code has been such a life saver, I think it makes sense to write docs on those lines.

These steps involve a lot of building and compiling and seems like that also gives me a good opportunity to pick up grunt or gulp. I am picking gulp, only because the jsFiddle newsletter came out today (jsFiddle has a newsletter, you guys!!!) and it has a link to a tutorial for gulp. Not a workshopper I notice - might be an opportunity there.

In any case, looks like my next 5 days or so will remain packed with more node code.


90 days of node.js - Day 9 expressworks

In case you don't know what this is all about, the out-of-loop link describing my 90 day journey through node land.

I was talking to a colleague yesterday and we were yakking about node. He has been experimenting with node.js for a while now and has been trying to sneak it into our regular workflow. Battle of attrition imo, but what can you do :)

One thing we both agreed on immediately was something I have been observing in the node community in general. I think I can safely bundle the users in 2 categories:

  1. Node core developers - mostly backend folks. Comfortable in cli.
  2. Web application developers - Very comfortable with web apps on browsers, maybe not so much on backend.
Based on the documentation alone, I can see that the core node docs are very much geared towards people building server side cli tools. The extensive module development seems to be a catch-up game that regular web devs are playing so that their chosen backend framework can work just as easily as their regular favorite backend framework. This might be a generalization, but that is the only thing that explains to me the fantastic array of modules - each of which is solving a problem already addressed in other languages. It's a fascinating time to be writing ndoe code though.

Which brings me to Day 9. This was another mixed day - primarily because of lack of time. I was able to get thru expressworks - another workshopper that deals with express.js. express touts itself as a "fast, unopinionated, minimalist web framework". From the looks of it, express to me looks very much like spring-mvc. The code is fairly light to write - a little bit of boiler plate and then it just works. There is just so much functionality it exposes that my eyes basically glazed over the documentation. That said, express folks have done a stellar job organizing the documentation. The API reference is as good as the best put there and was easy to read and use immediately. I intend to use express as much as I can for the the foreseeable future as my UI layer just for that reason - less ramp up is always better.

My first impressions based on the workshopper are fairly positive. It is pretty easy to setup a basic express server and start routing both static files and responses generated from template files. express is pretty flexible about what template it supports and how to wire in that support. The most intriguing thing about express though was the concept of middleware. Middleware is the ability to plugin different actions on an incoming request before computing and returning a response. It's essentially like filters in the java world. The examples I have seen so far didn't demonstrate ay interesting deviation from the filter principle. express gets this from connect (which it doesn't use in v4.x and above) an interesting node module from sencha. Routing and param binding is the other big feature of express. I didn't find a dispatcher yet, but I can only assume something like that exists already.

The workshopper was again really good, but not great. It introduces express in somewhat big leaps. The exercises themselves were fairly easy to work out, but the instructions could use some work. Unlike the previous workshopper,s hints were exactly what they said - the entire solution was not provided and I had a more productive time going over the express documentation. Much fun, definitely recommended.

My workshopper solutions are on github. Two minor things that I didn't know, but now that I do I feel good about knowing. Using npm init to initialize a package.json AFTER you have run your npm install commands is really useful. It brings in all the node dependencies in the correct order. Especially useful when working with workshoppers that throw random modules at you. Similarity, git init helps to simplify the git checkin process once the modules are done ( don't forget to update your .gitignore file and have git ignore all the dependency cruft).

Overall a good day. Tomorrow I plan to refactor my twitter-list api and also move it to a UI home.


90 days of node.js

 Malcolm Gladwell throws around the figure of 10000 hours of practice before you can call yourself an expert. An obvious corollary is that you become an expert only in the usage of the particular tools you use. This is probably what makes it difficult to become an obvious expert when it comes to programming knowledge. Each day is spent working with more than one tool or framework or programming language. There just isn't enough time spent learning anything since you are working against deadlines and focusing on hacking your way through the requirements rather than picking up the nuances of your framework and learning more about your programming language of choice. Even architects tend to live in the rarefied realm of abstraction and can sometimes completely disregard the actual tools in pursuit of abstract design.

Which brings me to JavaScript. I've been writing JavaScript for a while now - I've even had to deal with JScript, so that gives you an idea of things. But, as someone who has written a lot of JavaScript, I am woefully under par when it comes to node.js. Mostly because I never got the opportunity to use node.js at work. It is easy to just point and say, "why didn't you go work on it at home?". I've a family to raise and other interests that I try to pursue to the fullest - reading, gaming, podcasting and writing. This leaves precious little time to devote to learning about node and anything else that catches my fancy.

This year, though, has been a year of discovery. A lot more book reviews, a podcast and I'm making some time for learning a new framework. So, about 10 days ago I started off on a 90 day journey thru nodeland. The goal is not to become a node expert - 10000 hours, remember - but to get more than passing familiarity to the node ecosystem. And that's what it really is - an eco system. It would appear that writing actual node framework code is passe and there is a rash of modules that do one thing and one thing alone. It's going to be a long haul picking up random modules and using them to finish the 90 days.

I realised - 10 days in - that I had no tracker for each day's worth of work. This page then will collect all the 90 days worth of effort and list them. Happy reading and if you learnt something from this - yay!

Day 13 - Events in node.
Days 11 and 12   - More of Jade, learning about domains and oauth
Day 10 - Refactoring with Q Promises
Day 9 - Expressworks - that is all.
Day 8 - Connecting to the Twitter REST API to grab public lists.
Day 7 - Washout :/ (not quite - I stumbled thru how-to-npm)
Days 5 and 6 - Stream adventure travails
Days 3 and 4 - More learnyounode and some IDE futzing
Day 2 - Starting with learnyounodeDay 1 - Installing node and first steps.


90 days of node.js - Day 8 - Connecting to Twitter REST api

Day 8 of my node.js discovery went surprisingly well.

For one thing, I've narrowed down how much time I can reliably give to this project on a daily basis. I find a lot of node.js posts full of planned itineraries and lots of code from day 0. Seems to me that this is both unreasonable and overly optimistic for someone who has a kid and a house that needs constant attention. 2 hours is my limit. I don't want to do any exploratory stuff at work because of fairly strict legal guidelines at work. Better to publish any public work from home. I would definitely advise people to be realistic about how much time they can devote instead of just gong through sleepless nights.

I also figured that part of my time should go in developing some code and part in learning more on node. The development part is what is really appealing and stumbling through the tutorials can be maddening if I can't keep pushing out incremental changes to the application.

Day 8's goal was to connect to the Twitter API and grab lists as an application would. I had initially hoped to get all lists, but later settled for public lists only.

Authorization with Twitter

The first step is being able to access twitter and getting authorization from twitter to access public lists. I went with the approach of using Application Only Authentication. This is a "simple" mechanism for applications to obtain authorization tokens from twitter to access or modify data. I put the quotes around, because this whole thing ends up being a lot more complicated than it seems. You need to understand oauth2 flows to some extent. Twitter goes quite far to hide the details from you, so it is convenient to use. From the Twitter end, you need an application registered with Twitter to give you 2 pieces of the auth puzzle - a customer key and a customer secret. For now this app I have created is not accessible for installation, so I won't spend more time on it. From the node code perspective, turned out that the trickiest part was figuring out how to introduce these values in the workflow. I found a helpful stackoverflow comment that introduced me to using a module for declaring my config. Easy to use and conceptualize.

Calling the REST API to get tokens

The token api is a direct POST over https. This had me stumped for a while till  I figured out two things:
  1. The path parameter for the http(s) modules of node should NOT have the protocol in it. Seems obvious in hindsight, but somehow I completely missed that.
  2. There is a https module. Again, obvious in hindsight, but took me a few minutes to search the doc for.
Once those things were settled the rest was fairly easy. Posting the credentials got me back the access token and I could use it for the next step.

Calling the REST API to get public lists

This was relatively the quickest thing that I spent time on. The API endpoint is self-explanatory and pretty decently documented.

And thus I have my first reasonable working node module. You can find it on twitter-list's github home

Next steps

Simple Twitter List is the provisional name for a simple Twitter list (d'oh) editing app I plan to build to learn more about node. I love using lists to manage my TL and find the current landscape a little slim on options. The main goal is to learn about node, but scratching an itch while learning seems the right way to go. I am going to be using the twitter-list module a lot more in the days to come.

There is already a twitter node module which does a great job of working with the twitter api for node apps. I would recommend using it for a real production problem.

For next steps, I am going to do some refactoring of the code and break out the auth token generator and the calling code into separate functions. The next important change is using user credentials instead of just app only authorization. That opens up a large vista of read-write actions I can take with the app. Finally, an app that runs on cli is pretty much useless when interfacing with twitter. In order to get a simplistic UI and running I am going to invest time in ExpressWorks - a workshopper to learn express.

I am starting to quite like workshopper and will try to contribute something just to give back I think. Their cli is pretty sweet and inspite of some frustration it has been quite nice to work with.

Onward and upward.


90 days of node.js - Day 7

Today was a washout. I tried really hard to get how-to-npm working. But once it failed on the private repo, things just went downhill from there. Still, it gave me the opportunity to do two things - get an overview of npm commands and secondly take a look at how a workshopper project works. Nothing much to report for today.

I am finding that as the life span of node modules and tools are so short, they tend to be not very stable. It's great for creating cutting edge code I suppose, but is frustrating when the learning tools get out of date so quickly. The next logical step seems to be to start working on a real problem instead of trying to debug why a learning module is out of syn with reality.

So, here's what we will do next, A simple module to work with the twitter api to retrieve lists. I dub the module twitter-lists. It will allow any app to retrieve a logged in user's twitter lists and return any tweets for the users listed in the lists. There is already a twitter-node api which does this and I am totally reinventing part of the wheel here. Even then, it promises to be a nice challenge.

On reading challenges

So, yet again, I've decided to take on a challenge of completing 25 books this year. Obviously, I have plenty of spare time and don't have a life or anything. Which is not true at all. Outside of family time and work, I am still working on my games (which I promise I WILL COMPLETE). And I still devote quite a bit of time following up on JavaScript updates, Java 8 updates and other random crap.

Usually, my reading is organic and I walk into a library or a bookstore or more often the Kindle library and pick up books at random. While this has worked great and has led to me finding some really amazing finds, I tend to ignore the books that I own which remain unread. Books want to be read, they want their words to escape from their pages (or their digital avatars) and lodge in your brains. So, for a change, this year I have a list of books that I want to read. I'm planning to finish these books for sure. Pretty positive I will be reading a little bit more than just these (javascript, le sigh) , but there are some that I would definitely try to finish (in no particular order).
What's the point of this? I've been picking up books over many years and have always managed to finish reading them once bought. Unfortunately, the past few years have seen me do less of that and more of impulse reading. While that has its merits, I would like to clean out my old reading stock first. This is my first attempt to do just that. If you're not the list making kind of guy, go to your public library or closest book store and browse till you can find a book that appeals to your tastes.

I am already working on Rajaji by Rajmohan Gandhi and have finished reading Dark Star - the Loneliness of being Rajesh Khanna - so a good start in general to this years reading.

Book review - Rajaji A Life

Chakravarthi Rajagopalachari - popularly known as
Rajaji - remains one of the stalwart leaders of the Indian freedom movement. His memory is not as firmly etched in the average Indian's mind like say Gandhi's or Nehru's or even Maulana Azad's except maybe in the South. I've always had a hankering to go find out more about this man whom Gandhi called "keeper of my conscience". My maternal grandparents were cadres of the Satyagraha movement that Rajaji spearheaded from Salem, TN. This slender personal connection was also a driver in me picking up Rajmohan Gandhi's Rajaji A Life.

The book does a good job of tracing Rajaji's life from his childhood all the way to his last days. Rajaji lived to the ripe old age of 93 and was a great national leader, a statesman of exceptional caliber and a politician with strong democratic roots. His roots were in the South as was his political base. The book does a good job of painting a detailed picture of his activism, both political and social. in the course of his life, Rajaji lead salt satyagrahas, established ashrams, spun khadi, became a Premier of a major province under British rule, was the first and only Indian Governor General of India and went on to form a major political formation as an alternative to the Nehru Congress in his old age. His politics remained that of a benevolent ruler and was more conservative than Nehru, his primary rival for the love of the Indian people. The author spends quite a bit of time going over details of Rajaji's political career and this book shines at that.

Particularly interesting to me was Rajaji's role as the premier of Madras Presidency in the years before the Second World War. The book goes a long way in describing the workings of a government elected under a viceroyalty which had to work with the people it abhorred and was trying to win freedom from. The balancing act of maintaining law and order while actively trying to kick out British rule was a tough one, but Rajaji's efforts put to the lie British claims that Indians were incapable of self-governance.

The book also spends quite a bit of time in side-stepping Rajaji's rather conservative views and the his self-righteousness in going about declaiming them. Prohibition and satyagraha jump to my mind. His rather heavy handed efforts to impose prohibition are excused away with short term gains and his efforts to impose Hindi in the Madras presidency explained by the need for unity in the national movement. I was a little put off by the conservativeness Rajaji displayed at every turn of his political career. The author is his grandson and does a good job of blending in personal narrative with letters and other documented material, but even then Rajaji's natural tendency to assume he knew best can rankle the reader.

Another mystery has been why Rajaji ended up being a regional powerhouse and not the national leader he deserved to be. Rajaji was one of Gandhi's first pupils and an ardent one at that. The book went some way in clearing up for me the roots of the power struggles that saw Rajai, once Gandhi's conscience-keeper, ousted from his primary role to be replaced by Nehru. The primary reason seems to be his belief that governance and not imprisonment was the correct way to wrest freedom - a view that I sympathize with. This saw his stature fall quite a bit among the leadership of the national movement - particularly in the north - who courted arrest during the time of Rajaji's premiership in Madras. Another struggle was also his natural tendency to play diplomat with everyone which probably rankled the more reactionary forces in the Congress leadership.

In recent days, Rajaji has received renewed attention. There is a e-magazine which titles itself SwarajyaMag and claims to follow in the footsteps of Rajaji.'s own Swarajya. The only similar features seem to be a conservative bent in the writing and a vitriolic hate of the Nehru legacy. To me this seems like a disturbing corruption of Rajaji's legacy. Granted he was opposed to the license-raj that Nehru setup - and rightly so - but to take that to an extreme personal animosity feels both short-sighted and unproductive.

This book though remains a remarkable work. It collects intimate material, highlights both through newspaper articles and other documents the exceptional life that Rajaji led. Totally worth a read. 3.5 stars.

I leave you with these words of Rajaji which first introduced me to him.


90 days of node.js - Days 5 and 6

Days 5 and 6 have not been fun. For one thing, they ended up being almost 3 days apart from each other becos of personal sadness :( For another, I just don't grok the stream adventure format.

Node's main purpose is to circumvent the issue of I/O bound calls and instead provide a event mechanism to handle blocking calls. Java NIO solves similar problems, but in a more multi-threaded environment. JavaScripts single threaded model implies Node is restricted by that. ( It also uses something called libuv to handle the OS calls - I'll probably punt on that for now.) To handle all this magic, node uses streams - effectively pipes that can be readable, writable or both. You write data to a stream and somehow it can get consumed when piped to another writeable stream and so on. This is at the heart of the evented I/O mechanism making streams pretty much the most importatn thing to grok as soon as possible.

Hence enter stream-adventure. The positives are there. It made me sift through a lot of documentation in a short time. It also made me more familiar with through and through2 - excellent modules imo. But this choice in modules lies in the heart of why I think stream-adventure failed for me. I kept getting lost in the forest of sketchily documented modules. Each module makes sense once I could see the final code, but to get it working was a pain. I ended up cribing code from stack overflow and the github forums (Thanks jeremy-w for your stream-adventure solutions) . Not my finest, but I didn't want to get stuck forever in a rut without progressing further. I am still not a 100% on board with what modules exist and how I can use them to fashion my own streams, but I know through now and that will be useful.

The next thing to tackle is how to use npm. I've been using it without understanding what it *really* does, so the next project is how-to-npm.

Avaunt, Excelsior and so on.


90 days of node - Days 3 and 4

Days 3 and 4 have been spent with learnyounode. It has been...interesting.

learnyounode is a fairly simple tutorial program. It encourages you to start using the different pieces of the node.js ecosystem. I am just not convinced it does a great job.

For starters, there is very limited discussion of what npm is and how modules work. Considering that these are fundamental parts of the node.js ecosystem, it is baffling that there was no tutorial for npm. There is a section on module writing and some notes on exporting modules, but they are basically hand wavy.

In general, the tutorial interface is easy to use and reminds me of the old pc games settings screens :). That's what it ends up being though - a game. There seems to be very little effort to teach the basics of programming or of the node ecosystem. This probably plays in well with the idea of modules in node, but felt a little hollow. It's mostly about how to navigate docs - which are really difficult to get to on a windows machine. It-felt like the tutorial was geared towards getting people running without getting to understand what they are doing. The "hints" tended to be way too explicit and kind of made things a little too obvious.

That said, it's an excellent way to get used to the environment and exploring in a limited way the api offered by node. The http tutorials were most informative. If you go hunting around the github page for the module, there is some amount of active discussion on different solutions, That, I thought, was pretty sweet. Highly recommended as one of the starting places to learn nodejs.

On a sad note, SublimeCodeIntel just didn't work for me for node.js. node api code complete didn't work out of the box and that is proving pretty elusive. The other plugins for sublime I found are basically not being maintained. I do get code complete with docs for JavaScript api, but at this point, I am not sure if I am going to stick to Sublime.

I did notice an interesting thing though - I was trying to type out the function names in any case. I would never do that in a Java IDE. My guess is this is just because of the way I learnt JavaScript. I have noticed, more experienced Java programmers type out every variable name and method call - probably because that's how they learnt it. Quirks mode :)

And emmet never got a trial, since there is no view rendering yet. All good things in their time, I suppose.

Now on to day 5 and stream-adventure and how-to-npm.


90 days of node.js - Day 2 Learn me some node

Day 2 of my 90 day node quest dawned with grandiose plans and absolutely no time to get stuff done. Having a 4 year old, a day job and a hundred odd chores at home mean less than 2 hours to get any meaningful work done.

That led me to pick learnyounode as my first pass at learning some node. I have already done this set of tutorials once before, but no harm in starting from the basics and getting a basic primer of things to come.

No github checkins today. The "BIG PLAN" ™ is to make 2 apps:

  • work with the twitter REST API and create a simple app to view, edit and create user lists. More on that later.
  • A different way of viewing and sorting your book picks and their reviews using the pinterest api
Pipe dreams, I'm sure.

Another aspect of picking up a programming language is your IDE. JavaScript is fairly well represented in the IDE space. For most of my programming career, Eclipse has been my go-to IDE. Whatever it's shortcomings - and there are many - it's a fairly stable and easy to use platform. It is extensible and is supported by a large ecosystem of plugins. For JavaScript development though, it is not a good fit. Not even an ok fit. Having done a little bit of research, I ended up trying out Sublime Text a couple of years ago and love it. The plugins make work fast and there seems to be a pretty decent ecosystem around it.

I'm starting out with three plugins in particular:
  • Package Control for easy installation of sublime plugins
  • SubmlimeCodeIntel for easy autocomplete
  • Emmet for css and html snippets -- granted, not a 100% on board with this, but what the heck, any productivity gain is a good thing.
That's about it, after than I'm onto learning me some node for much js win.


90 days of node.js - Day 1

I've been on and off the node bandwagon more frequently that I would like to admit. Node is pretty amazing in the way it frees the JavaScript browser from the confines of the browser. It helps the programmer use the language outside of browser quirks and into the realm of being able to create an end to end solution.Granted, this isn't new and the JVM has also had the ability to write JavaScript and run it in the JVM. But, that has involved a lot of burnt cycles figuring out the java code, build cycles, maven or any files... you get the drift. The biggest plus of node.js as far as I am concerned as a web developer is the ability to write JavaScript for both the client side and server side ends of the application. One language tying both your access to the datastore and the presentation layer. That is just fantastic.

That is also the reason I have never really gotten into node. By virtue of working on the Java web stack, more often than not I have to use a view layer bundled with one of the frameworks - be it jsps, gsps or the ilk. The backend also tends to be a Java framework which makes life easy. Inserting a node.js backend is not necessarily something that is easy to get away with, especially in enterprise environments.

So, in the spirit of getting myself off my ass which has been the general trend of this year so far, I am trying out different things in the node.js ecosystem for 90 days and document my work. 90 days of node.js. I am not a stickler for guidelines, so this might take more than the promised 90 days, but I will try to stick to the plan and hopefully by Septmber have enough material to push to github. The general plan is sketchy - I want to learn more about bower, express and streams in general, but the particulars are kind of sketchy.

That said, this is the first day of node. And the first step is installing node.That is as simple as clicking on the big, green Install button on the node.js site. I usually work on windows and my current build is win 8.1. All my installs usually go into some sort of sandbox folder - just becos.

Et voila and we're off to the races. Next stop, creating a simple module to stream data from a file.