Specifically though, in the web arena things have been moving fast and furious with new languages, approaches, and methods like RoR, Play!, and Lift (and many others!).  While I “get it” regarding the benefits of these frameworks, I never felt the need to give them more than an initial deep dive to see how they work.  I nod a few times at their similarities and move on back to plain REST-ful services in Java with Spring, maybe an ORM (i try to avoid them these days), and a JS-rich front-end.

Recently, two factors made me deep dive into Node.js.  First is my growing appreciation with the progress of the JavaScript front-end.  What used to be little JavaScript snippets to validate a form “onSubmit” have evolved to a complete ecosystem of technologies, frameworks, and stacks.  My personal favorite these days is Backbone.js and try to use it whenever I can.

Second was the first hand feedback I got from a friend at Voxer about their success in using and deploying Node.js at real scale (they have a pretty big Node.js deployment).

So I jumped in.  In the short time I’ve been using it for some (real) projects, I can say that it’s my new favorite language of choice.

First of all, the event-driven (and non-blocking) model is a perfect fit for server side development.  While this has existed in the other languages and forms (Java Servlet 3.0, Event Machine, Twisted to name a few), I connected with the easy of use and natural maturity of it in JavaScript.  We’re all used to using callbacks anyway from your typical run-of-the-mill AJAX work right?

Second is the community and how large its gotten in the short time Node has been around.  There are lots of useful open source libraries to use to solve your problems and the quality level is high.

Third is that it was just so easy to pick up.  I have to admit that my core JavaScript was decent before I started using Node, but in terms of the Node core library and feature set itself, it’s pretty lean and mean and provides a good starting point to build what you need in case you can’t find someone who already did (which you most likely can).

So having said all that, I wanted to share what resources helped me get up to speed in Node.js coming from a Java background.

Getting to know JavaScript

There are lots of good resources out there on Node.js and i’ve listed them below.  For me, the most critical piece was getting my JavaScript knowledge to the next level.  Because you’ll be spending all your time writing your server code in JavaScript, it pays huge dividends to understand how to take full advantage of it.

As a Java developer you’re probably trained to think in OO designs.  With me, this was the part that I focused the most on.  Fortunately (or unforunately), JavaScript is not a classic OO language.  It can be if you shoehorn it, but i think that defeats the purpose.

Here is my short list of JavaScript resources:

1. JavaScript: The Good Parts - Definitely a requirement.  Chapters 4 and 5 (functions, objects, and inheritance) are probably the most important part to understand well for those with an OO background.
2. Learning Javascript with Object Graphs (part 2, and part 3) – howtonode.org has lots of good Node material, but this 3 part post is a good resource to top off your knowledge from the book.
3. Simple “Class” Instantiation – Another good post I read recently.  Worth digesting.

Learning Node.js

With a good JavaScript background, starting to use Node.js is pretty straightforward.  The main part to understand is the asynchronous nature of the I/O and the need to produce and consume events to get stuff done.  Here is a list of resources I used to get up to speed:

1. DailyJS’s Node Tutorial - A multipart tutorial for Node on DailyJS’s blog.  It’s a great resource and worth going through all the posts.
2. Mixu’s Node Book - Not complete, but still worth it.  I look forward to reading the future chapters.
3. Node Beginner Book – A good starter read.
4. How To Node – A blog dedicated to Node.js.  Bookmark it.

I felt that going through these was more than enough to give me the push I needed get started.  Hopefully it does for you too (thanks to the authors for taking the time to share their knowledge!).

Frameworks?

Java is all about open source frameworks.  That’s part of why it’s so popular.  While Node.js is much newer, lots of people have already done quite a bit of heavy-lifting and have shared their code with the world.  Below is what I think is a good mapping of popular Java frameworks to their Node.js equivalents (from what I know so far).

Web MVC

In Java land, most people are familiar with Web MVC frameworks like Spring MVC, Struts, Wicket, and JSF.  More recently though, the trend towards client-side JS MVC frameworks like Ember.js (SproutCore) and Backbone.js reduces the required feature-set of some of these frameworks.

Nonetheless, a good comparable Node.js web framework is Express.   In a sense, it’s even more than a web framework because it also provides most of the “web server” functionality most Java developers are used to through Tomcat, Jetty, etc (more specifically, Express builds on top of Connect)  It’s well thought out and provides the feature set needed to get things done.

Application Lifecycle Framework (DI Framework)

Spring is a popular framework in Java to provide a great deal of glue and abstraction functionality.  It allows easy dependency injection, testing, object lifecycle management, transaction management, etc.  It’s usually one of the first things I slap into a new project in Java.

In Node.js… I haven’t really missed it.  JavaScript allows much more flexible ways of decoupling dependencies and I personally haven’t felt the need to find a replacement for Spring.  Maybe that’s a good thing?

Object-Relational Mapping (ORMs)

I have mixed feelings regarding ORMs in general, but it does make your life easier sometimes.  There is no shortage of ORM librares for Node.js.  Take your pick.

Package Management Tools

Maven is probably most popular build management tool for Java.  While it is very flexible and powerful with a wide variety of plug-ins, it can get very cumbersome.  Npm is the mainstream package manager for Node.js.  It’s light, fast, and useful.

Testing Framework

Java has lots of these for sure, jUnit and company as standard.  There are also mock libraries, stub libraries, db test libraries, etc.  Node.js has quite a few as well.  Pick your poison.  I see that nodeunit is popular and is similar to jUnit.  I’m personally testing Mocha.  Testing tools are a more personal and subjective choice, but the good thing is that there definitely are good choices out there.

Logging

Java developers have quite a list of choices when choosing a logger library.  Commons logging, log4j, logback, and slf4j (wrapper) are some of the more popular ones.  Node.js also has a few.  I’m currently using winston and have no complaints so far.  It has the logging levels, multiple transports (appenders to log4j people), and does it all asynchronously as well.

Hopefully this will help someone save some time when peeking into the world of Node.  Good luck!

One specific session stood out for me.  Ashish Kelkar gave a great talk about his experiences in Mergers and Acquisitions in the Consumer Internet – specifically about his 6 years at Google being part of the team that did just that.

There is lots of good information on why they buy companies, how much they pay, how the process works, and when to negotiate.

Definitely good stuff.

Here is a link to the deck or you can browse the embedded version below:

Here’s a very common scenario: Your website allows users to submit text which you save in your database and later display for other users to see. It could be comments, descriptions, names, addresses, etc.

You want to make sure that they don’t mess around with your system (security is important right?), so you want to make sure you escape any markup they include in their text. Escaping it should make sure that </body> tag they put it by “mistake” or that accidental <script>…</script> tag won’t fire when the text is displayed inline.

That brings me to my big question… when should you escape this data?

On the way in or the way out?

On the way in, like escaping the text before it’s saved in the database, might seem safer because you’re sure whichever way it leaves your system (or whichever developer uses that data), it will be escaped. But I think that’s the wrong approach.

I’ve always had the belief that your user-inputted data is sacred and should always be stored in its original form. You never know what you’ll be doing with it later.

Why does that help us here?

It’s because escaping changes for each output medium and you might not know which medium it is ahead of time!

Let’s go back to our original example. Let’s say you decided to escape any HTML tags before saving it to the database. Output should be easy now. You just dump the text on your webpage and call it a day.

But a few days later, your boss tells you a new output format should be supported in your system – for example, CSV files.

If you dump that escaped text to the CSV file, it will be escaped for HTML which is not what we want. Not only that, CSV files have their own escaping that is required.  You also have the possibility of double escaping when one medium isn’t compatible with the escaped field from another medium.

So escaping data on the way in has two problems. First, you can’t decide which medium to escape it to ahead of time. Second, escaping it could potentially lose data that is important to you by double escaping.

Doing it on the way out makes sure you are making the decision at the best possible time in the data’s lifecycle and keeps the system flexible to new requirements for data display.

The downside is discipline. Forgetting to escape something can be a problem from a security perspective and from a user experience perspective. As they say, with great power comes great responsibility.

Fortunately, popular frameworks and libraries already include escaping functionality. It just becomes the responsibility of the developers to make sure that happens.

What do you think?

Update: Some people have pointed out performance.  They’re absolutely right!  You could keep the original text and encode it as well in a separate field.  That way you have the freedom to use the original value or the pre-processed one.

If you got a full week of paid time-off to learn some new technical subject or improve an existing one, what would it be (and why)?

In in other words, what have you been dying to try out, but didn’t have time to do it at work (and still get paid for it).

I like this question because:

• It’s time-boxed.  You only have a week, so it focuses on achievable goals in the eyes of the candidate.
• There is no “spare-time” requirement and can be done during work hours.
• It gives some interesting – and sometimes surprising – insight on the candidate.

For example, were they interested in a new tool or language? That’s a great subject to discuss to understand what interests them.

How about improving some existing skill? Deep dive into it to see what they think they are missing and why.

How about to write up some prototype or POC of an idea they had? Ask them why they haven’t done it in their spare time…

Some candidates literally are surprised by the fact that they haven’t really thought about it at all!

In the end, it leads to some great conversation and is surprisingly refreshing for a job interview setting. It can also be a great way for a candidate to stand out from the others in your memory. There is a sinister side to it though…

Candidates sometimes (inadvertedly) admit to not knowing some skills you think are important. Oops. Not good for them, but better to know early right?

In all, this question has been very helpful for me.  Hopefully it will be for you too.

What would you learn if you had a chance?

Having chosen MongoDB as my somewhat trusty back-end database, it would have been convenient to shoe-horn that fat social graph into the database.  I mean, blogs and books have written about how to do this in your typical relational database.   Some are still figuring out the best way.  Some even got a really big shoe-horn.  Even MongoDB, which is a document db but similar enough, has its own solution page for this common task.

Getting it right with one of these solutions isn’t easy.  Especially when you start to scale and new features demand deeper traversals into the social graph.  Not to mention is it quite the maintenance nightmare.

So from day one, I figured i’d go all out and get myself a shiny graph database to add to my technology stack and see what a difference it would make.  Neo4J seemed like a good a choice as any, so I downloaded it and plugged away.  Luckily, I caught it right in the transition to a REST-based server which is a much better design for my needs.

The first thing I noticed, having read through all the documentation and their cheat sheet on reproducing IMDB, is that designing a graph database schema is a pretty natural thought process.  I literally grabbed a piece of paper and was planning to rewrite my whole data layer to just use a Neo4J! I mean, it has: transactions, embedded Lucene for full-text search and indexing, solves my use-cases, and a nifty back-end management tool.

Having some sanity though, I stopped myself before actually doing it and scaled it back just to dealing with the social graph.  I figured i’d give MongoDB a chance to do what I already wrote it to do – which is persist all the crap I gave it – and I added Neo4J to only contain the social graph information, i.e. user identifiers and their relationships.  Each user document in MongoDB kept the node ID of the Neo4J node – and each node in Neo4J kept the user ID of the user document in MongoDB. That way I could look up any user node in the graph and start any traversals as needed.  And when I found what I was looking for, I could look it up in MongoDB to get all the details. (Side-note: the built-in Lucene in Neo4J is also a great solution for many lookup/search use-cases but I personally didn’t have any use for it since I’m using Solr already).

Honestly, it was a painless process and works pretty damn nicely.  It’s also very fast and flexible now.  Do I want to know who your friends are? No problem.  Do I want to know your friends’ friends’ friends’ friends are?  I can do that too by changing a function parameter.  And the results will come back pretty much immediately.

Overall I think i’ll be using Neo4J – and graph databases in general – for those tasks involving non-trivial relationships.  For everything?  Probably not….but getting it to support paging is a good start.

I’ve concluded that this one gives me the best ammo to work with:

What have you developed in your spare time?

I love this question because it touches a few areas at once.  Spare time is a valuable resource which you usually dedicate to what you enjoy most.  Dedicating that to development is a huge indicator of where your passions lie. On the other hand, getting a “Huh?” or “In my SPARE time?” in return is probably a good indicator that this person isn’t what i’m looking for.

What they’ve been working on is also an interesting indicator.  Did they contribute to an open source project? Port a popular tool to a new language? Build some nifty tool to try out some new tech? Reading into the type of development they did and why can really give some insight into what motivates or challenges them.

It also touches on is their ability to stay up to date and be self-taught.  So much new tech and ideas are being made available and it’s hard to keep up.  It’s almost impossible to do so at your “day job”.

That’s why I like that question and use it as often as i can when interviewing candidates.

What is your favorite one?

Update:

I very much appreciate all the reactions and opinions regarding this specific topic.  In no way does it try to pigeon hole people one way or the other and is my humble opinion on my personal experiences.  The main point I want to stress is that I like to know where someone’s passions lie. If someone says “After working at my job all day, why should I work more at home?”  I totally agree!  It shouldn’t be considered “work”.

It also can (or should) be an occasional thing.  I try to spend 2-3 hours a week on average working on some idea, or testing out some new technology, or just reading a good book.  That’s hardly overwhelming.

I’d also argue that over time it becomes more important.  When you build a deep body of knowledge and experience in one field, it becomes your prism to view new problems.  Expanding your horizon can create some interesting (and sometimes surprising) ideas for new projects, ideas to solve stubborn problem from the past, or just some personal enjoyment.

The point is to have an itch regarding programming and feel the need to scratch it.

As part of your site functionality, you allow users to create, update, search for, and view “thingies”.  This thingie has an address.

When people look at your thingie (er, that doesn’t really sound right), you want to display a nice dynamic map of the address with a cute little red marker.  You don’t have the coordinates for that address and therefore want to do some geocoding.

Having those coordinates would allow you to support all types of fancy location-based searches to tell if someone is near your thingie.  Even better, your cute little red marker would be in the right place.

So how do you go about doing this?  Well, some possible solutions could be:

1) Send a geocode request when saving/updating your thingie

A good first start, but probably not the best.  This has a few disadvantages, but for the sake of simplicity, let’s say the main reason  is you don’t want users waiting for unnecessary synchronous steps to complete.  Factor in network uncertainly, timeouts, etc. and you have a nice set of disasters waiting as your site grows.

You also realize that these geocoding services have limits on daily usage!  Too many thingie’s a day will be great for you, but you’d be tied to the limitations on your geocoding daily allowance and throttling requirements.  What to do?

So you come up with:

2) Create a queue to batch your geocoding requests

Better idea! When you create new thingies – and assuming you don’t have the addresses cached somewhere – you queue them in your uber geocoding async job.  It chugs away in the background sending nicely throttled requests and stops after reaching the daily limit.  Unfortunately, unless you want to pay lots of cash to up your daily limit, your queue will start getting longer and longer as it can’t keep up with new thingies.  Also, the geocoding might not even be done for thingies that your users are viewing right now.

Fortunately, this is a great scenario of how you can take advantage of your users to help you grow.  And you do it by leveraging client-side JavaScript.  Each user using your site is a potential resource which you can use to offload some of your work.  Help your users help you!

So why not have the users geocode for you?

3) Have the first user who views your map geocode it for you

I personally like this solution best because it’s being lazy.  When you have to display your map and that thingie has no coordinates yet, have the user send out a geocoding request from their client machine and use the results to show the map.  More importantly, send another async request afterwards to your server to save that value for future users!  Now you both get what you want at the right time.

Now assuming the geocoding API you’re using allows anonymous requests (like Google),  you don’t have to worry about daily limits, throttling issues, and other headaches.

The point of it all

Now this was all a hypothetical scenario to stress a point.  Lots of services today have an HTTP API accessible via JavaScript and this will probably grow and grow.  Align that with the increased horsepower of modern machines and improved browser JavaScript engines and you have an army of resources waiting to be tapped if you can figure out how to do so.  The great part is this resource scales linearly with demand.

The tricky part though is changing your frame of mind when writing features to consider this new option.  Why use server bandwidth and resources to do it when the client can do it just as well?

Have you had any scenarios like this?

At any rate, we gambled, we lost, we left.  Statistically, that’s about right so no hard feelings.

One thing that I did notice while there was how queues are serviced.  I’m not normally that observant, but it was pretty easy considering that:

a) I just finished a great book on the subject called “The Principles of Product Development Flow” (the book is much better than the title suggests).

b) You spend half your time in Las Vegas standing in lines.

One particular incident involved me standing in line at the MGM Grand Concierge to pick up some show tickets.  This was how the line looked:

Queue Per Server

Each server / concierge (the dots) has its own queue of people.  It this a good solution?  Of course not!  This is the perfect situation that breeds line-choice-remorse.  It’s that well known phenomenon which occurs after choosing a line –  you soon realize the line next to you is faster while you’re stuck behind a guy that is buying tickets for twenty five people, a restaurant reservation, maps for all of Nevada, a limousine, some shoe string, an igloo, and a towel.  At least that is what it seemed like at the time.

In short, having a Queue Per Server doesn’t account for variation.  Assuming that each person has the same priority, they join the lines at random intervals, and each person takes a random amount of time to service, this is the perfect recipe for some queues to be blocked up for long periods of time while other queues might be empty.  It also doesn’t serve people in the order they joined.

Luckily, most of the other lines in Las Vegas were like this:

Single Queue, Multiple Servers

While these lines might look longer, they are actually more efficient.  These queues have one line which includes all the people in the order joined, and multiple servers each taking a person from the front of the queue.  Making the same assumptions as the previous example, this method is actually more resistant to variation.  If one person takes a much longer time, only one server will be blocked and the queue will continue to flow.  In addition, it’s better for your mental health.

The perfect example of this is the Taxi line at the airport.  The line was literally hundreds of people and if you didn’t know better you’d think it was going to take hours to get a taxi to your hotel.  In reality it took 10 minutes.  We were literally walking non-stop in the line.  They had a system where 15 taxis would drive in and predetermined spots and drive off with the soon-to-be poor vacationers.  Pure awesomeness.

These decisions happen in software development too, especially in multi-threaded environments and Message Queues.  I wonder where they got the idea from….

ORM solutions have really made a huge impact on software development with relational databases.  It’s pretty much expected from any modern object oriented language to have a built-in ORM solution and/or multiple 3rd party implementations of one.

You could even write a pretty impressive list of benefits of ORMs like productivity, avoidance of vendor lock-in (except DB specific ORMs), and maintenance.  But in spite of all this, I do find that ORMs have a very real and prevalent hidden cost:

ORMs set the bar too low for who gets to write data access code

You might have seen this happen, or you may been that person once, but time and again i’ve seen how the lack of understanding of what’s happening “behind the scenes” cause very painful performance, stability, and design issues: 1-n problems, fetching too much data, ignoring batch updates, etc. to name just a few common ones.

ORMs take the discipline of fine-tuned and precise database interaction and takes a huge warm dump on it.

Now don’t be mistaken, the elite can and will use ORMs in the best possible way and in all the right places, but let’s be honest about the majority.  I’ve seen the blank stare with eyes glazed when I asked a developer why they wrote 3 nested loops to find a specific value of the child of the child of a mapped ORM entity instead of just writing a specific query for it. Or why they retrieved and iterated over all 5 million items to set the same value of of the same field instead of writing an update statement.

Somehow people assume that writing specific queries for specific cases when using an ORM is an admission of defeat when they can easily just traverse the object graph and call their setters and getters.  It all comes crashing down though when you have enough data.

In the end using ORMs properly requires even more knowledge and ability and it should be entrusted to those people who can get it right.

Those of you have passed this hurdle should congratulate yourselves. There aren’t many of you.