Node.js for the Enterprise

Node.js server is a lightweight, scalable, high-performance web application server platform. Instead of spinning up a new thread for each request, Node.js runs in a single thread and prevents blocking by spinning up new threads for I/O operations leveraging JavaScript’s event model and notifying the main thread upon completion (Watch Ryan Dahl’s introduction to Node.js).

At Metamarkets, we are using Node.js for our application server. We have a lot of client-side Javascript code for visualizing data in the core product and using Node increases our development efficiency simply by having our client side and server side codebase in a single language. Node.js is also very fast to develop in and has been widely adopted as a prototyping tool.

Node is a young platform and many developers may question whether it has an open source ecosystem that is mature enough to provide the functionality needed for an enterprise application server. We will share our experience here getting the infrastructure in place for a stable and maintainable application.

Object-Relational Mapping in Node

For most web application developers, an ORM (object-relational mapping) tool is a necessity. Having hardcoded SQL for every database operation your application needs is a maintenance nightmare and produces a lot of duplicated code. It is also desirable to build in a layer of indirection between your database schema and your business object model. Hibernate has been the standard Java ORM tool for many years. There wasn’t any clear leader for ORM when we first surveyed the landscape a year ago, but we chose sequelize because there was good documentation, it was being actively developed, the code looked clean, and our initial tests using it succeeded.

Sequelize is very straightforward and provides basic ORM capabilities without trying to do too much. In order to get to production with sequelize, we had to make a few contributions: introducing joins for collection getters, and adding connection pooling. Often, there does seem to be a gap to bridge between what’s available from the Node open source community and what is required for an enterprise level application. But, the Node community is vibrant and helping to bring the tools towards maturity has been rewarding. Multi-statement transaction support is the most critical feature currently missing from sequelize, which we currently handle with direct SQL statements. An ORM does not have to handle every database interaction and there will always be some situations (batch processing, very complex, highly-optimized queries) where direct SQL is the best answer, but transactions are something the ORM should support.

Clustering Node Server

Node runs as a single process but servers have multiple processors. To take best advantage of CPU resources, we cluster our Node instances to share the request load across processors. The built-in cluster package is very easy to use, allowing you to simply fork new worker processes from the master. If a worker dies due to an uncaught exception, the master can log or alert us to the error and then create a new worker. Workers do not share state, so moving to clustering requires any in-memory caches to be moved into a separate cache server like memcached. It would be nice to have an administration console on top of cluster to be able to visualize worker activity and spawn or kill workers, but generally our experience with clustering has been problem-free.

Code Maintainability

With Node’s asynchronous framework, the code can get extremely nested and difficult to read as successive callbacks stack up. So, a package like step or groupie is crucial to keeping the code readable and maintainable. It allows you to execute a group of functions either serially or in parallel. The serial execution pattern prevents the awful counter pattern with continual checks to identify the last item to execute in a list of function calls (aka “Am I done yet?”). We reworked groupie quite a bit for our own use case but it provided a useful starting point.

Automated Testing

There are numerous frameworks out there for writing automated tests against your Node.js application. After evaluating and writing tests in several frameworks including jasmine and vows, we ended up selecting nodeunit. It is an asynchronous testing framework but its similarity to JUnit made it much easier to get up and running. Although there wasn’t a good standard mock httpRequest/httpResponse object package that supported the Express server framework, it was pretty easy to cobble together and is invaluable for testing RESTful APIs.

Node.js server can deliver speed and scalability but the relative youth of the platform means that you often need to contribute to bridge the gap to production level quality. It is definitely doable and there are many open source packages available to help get you started. The open source community around Node.js is engaged and friendly. So join the fun and start running Node as part of your next project! If you’re already running Node.js and have favorite modules or support tools to recommend to us, we’d love to hear about them in the comments.

Filed in Technology