node.js

Hosting A Node.js Site Through Apache

3 commentsWritten on January 15th, 2012 by
Categories: node.js

I recently lost some time trying to figure out how to host a Node.js site through Apache, so I figured I might as well write a post about how I got it working. Of course, this approach only makes sense if you already have a server that's running Apache and you want to add your Node.js site with minimal impact/changes on your server. If you're not using Apache already but just want to publish a Node.js site, you're better off using Node.js to host it directly, or to put Nginx in front of it since it's more lightweight than Apache. But anyways, here's how you get it working with Apache.

First of all, you need a way to start your Node.js process automatically when your system boots, and to shut it down when the system is shut down. This will depend on the type of server you're running on. In my case, it's a Debian server so I just went with the sysv init script from Nico Kaiser. Another popular alternative is the upstart utility, which is already preinstalled if you're using Ubuntu. Once you have a start|stop|restart script in place, you'll want something to monitor the Node.js process to restart it in case it goes down. An easy to use tool for this is monit. Nico Kaiser again has a good example script available for Node.js on Github.

Once you have your sysv init or upstart script in place, as well as monit, your Node.js process can stay running on your server. Of course, you probably have it set to listen to connections on some other port than port 80 because that's what your Apache server is listening on. So now, the only thing you have to do is configure Apache to proxy all requests coming in on port 80 through the URL of your Node.js site to your local Node.js process. You'll first need to install mod_proxy and mod_proxy_http. After that, the configuration to make it work is quite easy:

<VirtualHost 109.74.199.47:80>
    ServerAdmin davy.brion@thatextramile.be
    ServerName thatextramile.be
    ServerAlias www.thatextramile.be

    ProxyRequests off

    <Proxy *>
        Order deny,allow
        Allow from all
    </Proxy>

    <Location />
        ProxyPass http://localhost:3000/
        ProxyPassReverse http://localhost:3000/
    </Location>
</VirtualHost>

And that's it. Every request coming in at http://thatextramile.be or http://www.thatextramile.be will be forwarded to http://localhost:3000 where Node.js is listening. Note that the ProxyPassReverse is required to make sure that all HTTP response headers will contain the proxied URL instead of the real one (localhost).

If you need the raw throughput that Node.js offers, this solution is far from optimal. Every request that comes in through Apache will cause an Apache thread to wait/block until the response is returned from your Node.js process. This is essentially the same as when hosting PHP or Ruby through Apache, so it's not a problem, but it does take away one of the benefits of using Node.js. Again, this approach only makes sense if you're already using Apache to host other sites and you just want to add a Node.js site with minimal impact to your server.

Displaying Feed Items On A Web Page: My Solution

7 commentsWritten on December 20th, 2011 by
Categories: express.js, JavaScript, node.js, Performance

A couple of days ago I asked you how you'd implement showing links from an RSS feed on a web page (in this case: my new company web site). These are my requirements for this:

  • It needs to be fast
  • The fewer requests that are impacted by retrieving the feed data, the better
  • If I publish a post, the links on the company website should contain the new link within 30 minutes
  • The simpler the solution, the better

I came up with a very simple solution, which satisfies these requirements better than any other solution I could think of, or heard of from other people. It is extremely fast, doesn't delay any requests, and doesn't require me to deploy anything but the company website. I'm building the site with Express on Node.js, which means I can take full advantage of the asynchronous nature of Node.js to implement this.

Let's go over the code... in the script that starts the express server, I have the following code:

var express = require('express'),
    app = module.exports = express.createServer(),
    NodePie = require('nodepie'),
    request = require('request'),
    recentFeedItems = null;

app.dynamicHelpers({
    getRecentFeedItems: function() {
        return recentFeedItems;
    }
});

// ... some extra configuration of Express that isn't relevant to this post

var processFeed = function(callback) {
    request('http://feeds.feedburner.com/davybrion', function(err, response, body) {
        if (!err && response.statusCode == 200) {
            var feed = new NodePie(body);
            feed.init();
            recentFeedItems = feed.getItems(0, 5);
            if (callback) callback();
        };
    }); 
};

setInterval(processFeed, 1800000); // process feed items every 30 minutes

processFeed(function() {
    app.listen(3000);
    console.log('Express started on port 3000');    
});

I'll discuss the code in just a moment, but first I want to show the view code that renders the links:

<ul>
<% getRecentFeedItems.forEach(function(item) { %>
    <li><time class="date"><%= item.getDate().getDate() + '/' + (item.getDate().getMonth() + 1) %></time><a href="<%= item.getPermalink() %>"><%= item.getTitle() %></a></li>
<% }); %>
</ul>

And that's all. This is the solution in its entirety!

If you're new to Node, this code probably requires some explanation. Let's start with this part:

app.dynamicHelpers({
    getRecentFeedItems: function() {
        return recentFeedItems;
    }
});

Here I'm adding a dynamic helper to the Express application. It basically means that my views have access to the getRecentFeedItems function, which returns the value of the recentFeedItems variable. It's important to know that the getRecentFeedItems function creates a closure on the recentFeedItems variable created above it. That means that if the value of the recentFeedItems variable changes at any point in time, the getRecentFeedItems function will return that new value.

var processFeed = function(callback) {
    request('http://feeds.feedburner.com/davybrion', function(err, response, body) {
        if (!err && response.statusCode == 200) {
            var feed = new NodePie(body);
            feed.init();
            recentFeedItems = feed.getItems(0, 5);
            if (callback) callback();
        };
    }); 
};

This just creates a function that we can use later on. It retrieves the feed asynchronously, and when the result is retrieved, we parse the feed using the NodePie library and we get the 5 most recent items which we store in the recentFeedItems variable. Again, this creates a closure on the recentFeedItems variable which means that every time we assign a value to this variable, any subsequent call to the getRecentFeedItems function will return the value we just assigned to it because both functions point to the same memory thanks to the magic of closures. Finally, if a callback is provided as a parameter, the callback will be invoked.

setInterval(processFeed, 1800000); // process feed items every 30 minutes

processFeed(function() {
    app.listen(3000);
    console.log('Express started on port 3000');    
});

The call to setInterval makes sure that the processFeed function is called every 30 minutes. After that, we call the processFeed function manually, and we pass in a callback where we start the Express server. This guarantees that the feed items will be in memory before the server starts processing requests.

What makes this solution so great is that we take full advantage of some of Node's benefits. Whenever we retrieve the RSS feed, Node.JS will retrieve that data asynchronously. As soon as it has fired the request to get the RSS feed, it just goes to the next event in its eventloop so no request is kept waiting while we wait for the data to be downloaded. Until the data from the RSS feed is returned, each request will just use the items that are stored in the recentFeedItems variable. Once the data has been returned, our callback is executed which overwrites the value of the recentFeedItems variable. We don't need to do any locking here because the Node.JS eventloop is single-threaded: while our callback is running, no other code that has access to the recentFeedItems variable can be executed anyway. And the actual parsing of the RSS feed is done by NodePie, which uses expat behind the scenes, which is supposedly the fastest C XML parser available.

Looking back on my initial requirements, I think this solution matches very well.

Node.js For Dummies

15 commentsWritten on December 18th, 2011 by
Categories: JavaScript, node.js

I'm sure you've all heard of Node.js by now. Its popularity is increasing rapidly, which means it's a good idea to be aware of what Node.js is and especially how it differs from more traditional technology stacks. In this post, I'll try to give an easy-to-understand overview of what makes Node.js different and make it clear that it's more than just server-side JavaScript. Note that this overview is highly simplified and only meant to help people understand how Node.js works. This is definitely not a completely accurate description of the lower-level details of Node.js.

Evented/Asynchronous I/O

In most technology stacks, API calls for I/O operations are synchronous. As in, the thread that executes the operation is blocked for the duration of the I/O operation until that operation has completed. Once completed, execution of your code proceeds. Of course, a lot of technology stacks have asynchronous variants of those operations available as well, but generally speaking, they aren't used as often as the synchronous variants. In Node.js, it's the other way around. All I/O operations are asynchronous and there are only a few synchronous implementations available (and you're generally discouraged from using them).

This means that whenever you do an I/O operation (file manipulation, network requests, database operations, etc…), Node.js initiates the I/O operation through a lower-level C/C++ layer which will perform the operation asynchronously. Once the operation has completed, Node.js will execute your callback function that you passed as a parameter to the I/O operation's function call. The important thing here is that while the I/O operation is being executed, Node.js doesn't have to wait for the operation to complete, and is able to focus entirely on processing other events. And those events can be anything: incoming network requests, executing callbacks from other operations that have completed, or invoking whatever function that is assigned to a particular event.

Eventloop

The Node.js eventloop is what makes Node.js so interesting and powerful. Node.js basically just keeps reading from an event queue until that queue is empty. As it loops through the events to be processed, it invokes the JavaScript functions that have been assigned for those events. If any of those functions performs an I/O operation, Node.js will initiate the operation and then immediately move to the next event in the event queue. Once the I/O operation has completed, an event will be added to the event queue with a reference to your original callback. Once all preceding events have been processed, Node.js will get to the newly added event and invoke your callback. Because all I/O operations are asynchronous, this enables Node.js to maximize its efficiency as it processes events because it doesn't need to wait for slow I/O operations to complete.

Single-threaded

One thing that people don't always realize is that the Node.js eventloop is single-threaded. This has some nice benefits but there's a huge drawback as well. The biggest benefit is that you don't need to worry about concurrent access to shared state. After all, there is never more than 1 thread executing your JavaScript functions. This means you don't have to write any locking code to protect shared state. The drawback to the single-threaded eventloop is that you need to be careful not to block the event loop. If you're planning on doing heavy synchronous processing in your JavaScript code, you need to realize that no other events can be processed by Node.js until that synchronous block of code has completed. Obviously, since there's only one thread going through the eventloop, any delay you cause in your code can be very costly to overall throughput and performance. For now, it's best to execute synchronous processing routines as a child process, possibly even in a language that is more suitable for this than JavaScript. But it seems that future Node.js versions will provide a more integrated way to deal with this.

Why JavaScript?

JavaScript's support of closures and it treating functions as first class objects means it's ideally suited for the evented programming model that Node.js offers. Many people still think of JavaScript as a joke or a toy language, but it's a lot more powerful than many people think it is. Yes, it certainly has problems as well, but it's definitely worth learning. I do hope that this post has made it clear that there's a lot more to Node.js than simply being server-side JavaScript. What makes Node.js so interesting are the principles that I've tried to explain in this post. Those principles can be implemented with other languages as well, and could be made to work just as great as, or perhaps even greater than Node.js itself. But you'd be hard-pressed to find a language that's so ubiquitous, yet completely devoid of a pre-existing synchronous I/O infrastructure.

Anyways, I hope I succeeded at making it somewhat clear how Node.js works and why it's so different from most other technology stacks.

Repeated Failed Log-Ins: What’s Your Strategy?

11 commentsWritten on September 10th, 2011 by
Categories: ASP.NET MVC, express.js, node.js, Patterns, Performance, security

I've only been using the server that's hosting this blog for a week or two, so I'm still keeping a close eye on it. I check usage graphs (cpu, disk I/O and network) a couple of times a day to verify whether things are still running smoothly. This morning, I saw a noticeable increase in CPU usage and network activity that lasted for about 11 hours. I logged into the machine, checked some logs and found out that someone had conducted an 11 hour lasting brute-force SSH attack. It doesn't make much sense to try that on my server since my SSH daemon doesn't allow password authentication, and indeed there was no successful login during the attack so no harm done, right?

Even if such an attack is not successful, it does consume resources on the targeted server(s). And wasteful, unnecessary resource usage has always been a bit of a pet peeve of mine so I wanted to prevent this from happening again. For this particular scenario, it's pretty easy. I installed DenyHosts which routinely checks for repeated (configured at 5) failed log-in attempts, and adds the offending IP addresses to /etc/hosts.deny so every other attempted SSH connection from those IP addresses will be denied immediately. Each offending IP address will be purged from /etc/hosts.deny after 1 week. Then I added a firewall rule that prevents you from connecting through SSH more than 5 times in 60 seconds. If you go over 5 connections, it just starts dropping packets, and by the time the drop behavior for your IP address expires, you'll have been added to /etc/hosts.deny already. As I said, pretty easy in this scenario because there are great tools I can rely on.

But what would you do if you had to implement a strategy to deal with this yourself? The most interesting approach I've heard of is to add an incremental delay on each failed authentication attempt. If the user fails the authentication check, delay the response with 1 second. If the user fails the second time, delay the response with 2 seconds. Third failure means a delay of 3 seconds, and so on. This pretty much makes a brute-force or dictionary attack impossible. The key is though, that you can't block any of your request-handling threads because then you open yourself up to an easy DoS attack.

Implementing this for a web application built on Node.js and Express.js is incredibly easy (there's an ASP.NET MVC example later in this post btw). I took the authorization example of Express.js and made just a few minor changes. First of all, I added the delayAuthenticationResponse function:

function delayAuthenticationResponse(session, callback) {
  if (!session.attempts) {
    session.attempts = 1; 
  } else {
    session.attempts++;
  }

  setTimeout(callback, session.attempts * 1000);
}

This is the most important part of the implementation. Every time we get here, we increment the number of attempts for this user by one and store the number in the user's session. Side note: this is one of the few things you'd actually want to use a session for: session-related data. Then we schedule the callback to be executed after the number of attempts * 1000 milliseconds have passed. The important part to remember here is that Node's event loop is not blocked by this, so our ability to handle other requests is not impaired in any way. The only one who suffers here is the attacker. Note that in a real world implementation, you'd probably only want to start increasing the delay after 5 attempts or so, in order to not piss off users who're just having problems remembering their password.

Then I changed the authenticate function so that it receives a session as the first parameter, and uses our delayAuthenticationResponse function whenever something goes wrong:

function authenticate(session, name, pass, callback) {
  var user = users[name];

  if (!user) {
    return delayAuthenticationResponse(session, function() {
      callback(new Error('cannot find user'));
    });
  }

  if (user.pass == hash(pass, user.salt)) {
    delete session.attempts;
    return callback(null, user);
  }

  delayAuthenticationResponse(session, function() {
    callback(new Error('invalid password'));
  });
}

After that, it's just a matter of changing the function that is assigned to the login route:

app.post('/login', function(req, res){
  authenticate(req.session, req.body.username, req.body.password, function(err, user){
    if (user) {
      req.session.regenerate(function(){
        req.session.user = user;
        res.redirect('back');
      });
    } else {
      req.session.error = 'Authentication failed, please check your '
        + ' username and password.'
        + ' (use "tj" and "foobar")';
      res.redirect('back');
    }
  });
});

And there we go. This effectively makes it impossible to brute-force your way into this web application, and I'm sure you can agree it was rather easy to do so. Of course, this is only because Node.js is inherently non-blocking. In an environment where non-blocking is the exception rather than the rule, you have to keep a few more things into account when trying to implement this strategy.

For instance, ASP.NET MVC is a typical blocking web framework. There's a certain number of threads that are waiting to handle requests, and once they receive a request, they process that request in its entirety. That means that if your code has to wait on something, the request handling thread is blocked and can't handle any other requests. So obviously, if you'd like to implement this strategy for dealing with repeated failed log-ins, you really want to avoid doing something like this:

        [HttpPost]
        public ActionResult LogOn(LogOnModel model, string returnUrl)
        {
            if (ModelState.IsValid)
            {
                if (CredentialsAreValid(model.UserName, model.Password))
                {
                    FormsService.SignIn(model.UserName, model.RememberMe);
                    if (Url.IsLocalUrl(returnUrl))
                    {
                        return Redirect(returnUrl);
                    }
                    
                    return RedirectToAction("Index", "Home");
                }

                Session["attempts"] = Session["attempts"] == null ? 1 : (int)Session["attempts"] + 1;
                Thread.Sleep((int)Session["attempts"] * 1000);
                ModelState.AddModelError("", "The user name or password provided is incorrect.");
            }

            return View(model);
        }

(note: this is a slightly modified LogOn method from the default AccountController when selecting 'internet application' in the MVC project wizard)

While this looks like it does the same as the Node/Express example, it certainly doesn't. The experience for the attacker is the same, because each failed attempt causes the response time to be increased with an extra second. But on your server, the thread handling the request is blocking the whole time and is thus incapable of handling extra requests while you're making the attacker wait.

Luckily, you can use ASP.NET MVC's asynchronous controllers to provide an asynchronous implementation of an action without blocking the request handling thread:

        [HttpPost]
        public void LogOnAsync(LogOnModel model, string returnUrl)
        {
            if (ModelState.IsValid)
            {
                if (CredentialsAreValid(model.UserName, model.Password))
                {
                    FormsService.SignIn(model.UserName, model.RememberMe);
                    AsyncManager.Parameters["returnUrl"] = returnUrl;
                }
                else
                {
                    Session["attempts"] = Session["attempts"] == null ? 1 : (int)Session["attempts"] + 1;
                    var timeout = (int)Session["attempts"] * 1000;
                    AsyncManager.OutstandingOperations.Increment();

                    var timer = new System.Timers.Timer(timeout) { AutoReset = false };
                    timer.Elapsed += (sender, e) =>
                    {
                        ModelState.AddModelError("", "The user name or password provided is incorrect.");
                        AsyncManager.Parameters["model"] = model;
                        timer.Dispose();
                        AsyncManager.OutstandingOperations.Decrement();
                    };
                    timer.Start();
                }
            }
        }

        public ActionResult LogOnCompleted(LogOnModel model, string returnUrl)
        {
            if (!string.IsNullOrEmpty(returnUrl) && Url.IsLocalUrl(returnUrl))
            {
                return Redirect(returnUrl);
            }

            if (model == null)
            {
                return RedirectToAction("Index", "Home");
            }

            return View(model);
        }

Your controller has to inherit from AsyncController instead of Controller to make this work. Of course, it's much more complicated and requires more ceremony compared to the Node/Express approach, but then again, ASP.NET MVC isn't optimized for this kind of usage whereas Node/Express definitely is.

Either way, no matter what web framework you use, if you can add an incremental delay to the response of each failed log-in attempt without blocking a request-handling-thread, you've added a very effective and low-cost protection against brute-force and dictionary attacks.

Solving A Problem By Avoiding It

8 commentsWritten on August 15th, 2011 by
Categories: JavaScript, node.js

I spent a little time this weekend working on something that would be of use to me: a simple utility on Node.js that calls JSLint on all JavaScript files within a given path (including its subdirectories), and only produces output if errors are found. It was pretty easy to write, and you can find it here in case you're interested.

There was one problem i ran into though. On Node, all I/O calls are non-blocking. Well, there are synchronous versions of some calls available, but you should use them as little as possible because they block the Node event-loop. One of the things i needed to do was to scan a given path recursively to find all JavaScript files in that path. And i just couldn't get it working the way i wanted to. The best i could do was a function that would invoke the callback once for every folder that was found. But i really wanted one that would only invoke the callback once, once the entire tree had been searched. I actually spent a few hours trying to get it working, and even looked for some good solutions in other projects. Most solutions i found also invoked the callback multiple times, others resorted to partially synchronous implementations. For a command-line tool like this, synchronous calls aren't a big deal but i'm trying to get better at typical Node programming, so i'm trying to avoid synchronous I/O.

So i tried yet another variation on my implementation, and it too didn't work properly. I got frustrated and said to myself "how hard can this be? i just want a list of files like 'find' would give me". Then it hit me that i could completely avoid the problem by just doing this:

var exec = require('child_process').exec;

function getJsFilesRecursively(startPath, callback) {
    exec('find ' + startPath, function(err, stdout) {
        var jsFiles = [];
        stdout.split('\n').forEach(function(f) {
            if (!/node_modules\//.test(f) && /.js$/.test(f)) {
                jsFiles.push(f);
            };
        });

        callback(null, jsFiles);
    });
};

No blocking I/O on the Node event-loop, and it took about 2 minutes to write. The only downside that it has is that it's not cross-platform because it uses the 'find' command which every *NIX-based system has, but Windows doesn't. So it will fail if you run it on Windows but for now, it'll do just fine and at least it enabled me to move on to the other stuff i needed to implement to get to the goal i initially set out for this tool.

Of course, by now i'm completely obsessed with nailing the recursive asynchronous folder-walking function so i will replace this with a proper version once i finally figure it out.