CodePlease

Node.js community has not yet defined any standard or a good practice for deploying applications. And dealing with production caos can be a total pain in the butt. Fortunately after searching a lot over the web, I found some consensus and tips for deploying a Node.js app to production.

If you typically use Express like me, you can set up the configs for your deployment modes (development, production, etc) in the main file. The below config is simple but easily extensible for supporting your needs. Here I just set the exception verbosity and an initializer for the MongoDB connection which can differ between the deployment modes:

app.configure('development', function () {
  app.use(express.errorHandler({ dumpExceptions: true, showStack: true }));
  mongo.connect('development',logger); // mongodb initialization
});

app.configure('production', function () {
  app.use(express.errorHandler());
  mongo.connect('production',logger); // mongodb initialization
});

A Node.js app (without forking or cluster facilities) runs on a single process. An uncaught exception can totally tear your instance down, stopping your service from responding to client’s requests. One way for surpassing this issue is to listen to ‘uncaughtException’, log the error and gracefully shutdown the process.

process.on('uncaughtException', function (err) {
  console.error('uncaughtException:', err.message)
  console.error(err.stack)
  process.exit(1)})

However, even with this safety measure, your service is offline and doesn’t respond to any requests. First of all, you can daemonize your app by using Linux Upstart so the node process could run in background (forget about screen multiplexing or leaving the terminal open all day!).

On Debian systems (Ubuntu, etc) run this for installing Upstart:

$ sudo apt-get install upstart

Ok good, now create a new file (/etc/init/nodeapp.conf) for configuring your new daemon. I borrowed this config and made some little changes:

#/etc/init/myapp.conf
description "Node App"
author "me"
 
start on startup # it's possible to use "net-device-up IFACE=eth0" for starting the process when the ethernet adapter is up
stop on shutdown

script
        cd /home/me/nodeapp
        exec sudo -u ubuntu NODE_ENV=production /usr/bin/node /home/me/nodeapp/app.js >> /home/me/nodeapp/log/app.log 2>&1
end script

In this simple config I teel Upstart to auto-start the process on boot and log output to a specific log file. Upstart also gives you some basics commands for managing the daemon:

$ start nodeapp
$ stop nodeapp

Having your app daemonized doesn’t save yourself from waking up at 3am because of some crashing activity (uncaught exceptions, network timeouts, etc). The rescue tool is Monit. Monit is a awesome utility which monitor and manage processes, files, directories, networking connections on Unix systems. Under the hood it just runs tests in certain intervals and proactively takes some action based on configured rules.

On Debian systems (Ubuntu, etc) run this for installing Monit:

$ sudo apt-get install monit

Open “/etc/monit/monitrc” and uncomment the “set httpd” to be able to monitor on localhost:

set httpd port 2812
    use address localhost  # only accept connection from localhost

This test rule will check if the node and mongodb processes responds to HTTP requests:

 check host nodeapp with address 127.0.0.1
           start "/sbin/start nodeapp"
           stop "/sbin/stop nodeapp"
           if failed port 3004 protocol HTTP
                    request /
                    with timeout 5 seconds
                    then restart
                    if 5 restarts within 5 cycles then timeout
           if failed port 28017 protocol HTTP
                    request /
                    with timeout 5 seconds
                    then exec "/sbin/start mongodb"

The DSL is pretty self-explanatory. For the node process running on 3004, the test will check if HTTP requests are failing, wait 5 seconds, restart the process, and if after 5 restarts the instance still doesn’t respond, it will call timeout. On the other side, for the mongo instance running on 28107, the test will checks if HTTP requests are failing, wait 5 seconds, and start a fresh mongodb process.

Wait! don’t forget to restart monit:

$ sudo service monit restart

Monit has lots of configurations for dealing with every possible situation. I recommend searching over the web or just checking their website.

For managing a Apache web server:

check process apache with pidfile /var/run/apache2.pid
    start program = "/etc/init.d/apache2 start" with timeout 20 seconds
    stop program  = "/etc/init.d/apache2 stop"
    if totalcpu > 20% for 2 cycles then alert
    if totalcpu > 20% for 5 cycles then restart

Alert the sysadmin by mail is something goes wrong:

set mailserver localhost
set alert sysadmin@domain.com

   set mail-format {
       from:xxxx@gmail.com
     subject: monit alert --  $EVENT $SERVICE
     message: $EVENT Service $SERVICE
                   Date:        $DATE
                   Action:      $ACTION
                   Host:        $HOST
                   Description: $DESCRIPTION
  }

Finally use this command (or the web GUI) for checking the actual monitoring status:

$ sudo monit status
The Monit daemon 5.3.2 uptime: 2h 7m 

Remote Host 'nodeapp'
  status                            Online with all services
  monitoring status                 Monitored
  port response time                0.000s to 127.0.0.1:28017/ [HTTP via TCP]
  port response time                0.001s to 127.0.0.1:3004/ [HTTP via TCP]
  data collected                    Fri, 26 Sep 2013 19:31:00

System 'system_Server'
  status                            Running
  monitoring status                 Monitored
  load average                      [0.01] [0.02] [0.05]
  cpu                               0.1%us 0.2%sy 0.1%wa
  memory usage                      791252 kB [19.5%]
  swap usage                        0 kB [0.0%]
  data collected                    Fri, 26 Sep 2013 19:31:00

Finally the deployment process per-se.

Set your development environment by editing “~/.ssh/config”:

Host nodeapp
Hostname nodeapp-server.com

IdentityFile ~/.ssh/mykey.pem
User me

Next create a bare (empty) git repo on the server:

$ mkdir nodeapp.git
$ cd nodeapp.git
$ git init --bare

And a git hook in “~/nodeapp.git/hooks/post-receive” for automatically restarting your node process with the deployed modifications:

#!/bin/sh
 
GIT_WORK_TREE=/home/me/nodeapp git checkout -f
 
echo "Restarting node process..."
sudo restart nodeapp

Back in your development machine, add the remote repo to your local setup:

$ git remote add origin ssh://nodeapp/home/me/nodeapp.git

And finally push the code!

$ git push origin nodeapp

In this post I only presented some tools and tricks for deploying to production without fear. Eventually you will need to setup a reverse proxy as Nginx in front of your processes for load balancing between them or just for serving static assets and error pages. I will talk about Nginx in a future post. Good hacking!