Tuesday, August 30, 2011

Developing node.js applications in WebMatrix and testing in IIS Express on Windows

In my last few posts I introduced the iisnode project and talked about specific aspects of hosting node.js applications in IIS 7.x on Windows:

Hosting node.js applications in IIS on Windows
Integration with the URL rewrite module

Running node.js express applications

In this article I am showing how to use the WebMatrix development environment to develop node.js applications and test them in IIS Express using the same iisnode module. WebMatrix is a free, lightweight, Windows based development environment for web application development using a variety of web technologies.

Setting it up

First you need to download and install WebMatrix (free). The download will also include IIS Express, a lightweight version of IIS specifically designed for development purposes.

Next you need to download a recent x86 build of iisnode. Unzip it somewhere on disk.

Note: if you are running 64 bit Windows, you still need to download the x86 build of iisnode, since IIS Express only ships in 32 bit version. 

Now install iisnode by calling the installation script from the command line with administrative privileges:

  • on a 32 bit Windows system:
install_iisexpress.bat
  • on  a 64 bit Windows system:
%systemroot%\syswow64\cmd.exe /C install_iisexpress.bat

If everything goes well, you should see successful installation confirmation:

image

Take note of the samples directory to use it from WebMatrix in the next step.

Start up WebMatrix

Start up WebMatrix. When prompted, choose “Site from folder”:

image

Enter the samples path that was shown in the last step of iisnode installation:

image

You can now explore and modify all node.js samples that came with iisnode installation in WebMatrix:

image

Right click on index.htm and choose “Launch in browser”. IIS Express will host the samples web site and you will be able to access the node.js sample endpoints exposed from IIS Express:

image

What did just happen?

WebMatrix with IIS Express gives you a lightweight, free development stack for node.js applications on Windows. IIS Express allows you to exercise the same set of features iisnode exposes in a regular IIS installation, including URL rewriting module.

So where can I get iisnode again?

Everything you need to get started is at https://github.com/tjanczuk/iisnode. Make sure to check out the samples. Feedback welcome!

Monday, August 29, 2011

Hosting express node.js applications in IIS using iisnode

In my last two posts I introduced the iisnode project which allows hosting node.js applications in IIS on Windows, as well as shown how it integrates with the URL rewrite module. In this post I demonstrate how to run node.js applications that use the popular express framework in IIS.

Installing express on Windows

Node.js modules, including express, are typically installed using NPM. The bad news is that as of this writing NPM is not yet supported on Windows. The good news is that for simple cases one can use the ryppi.py script. Assuming you have Python installed, you can call:

ryppi.py install express

which will create the node_modules folder with the downloaded express library. You can check out the resulting layout here.

The code

A simple express application we will host in IIS looks like this:

   1: var express = require('express');
   2:  
   3: var app = express.createServer();
   4:  
   5: app.get('/node/express/hello/foo', function (req, res) {
   6:     res.send('Hello from foo! [express sample]');
   7: });
   8:  
   9: app.get('/node/express/hello/bar', function (req, res) {
  10:     res.send('Hello from bar! [express sample]');
  11: });
  12:  
  13: app.listen(process.env.PORT);

Two key aspects to call out that may be different from your bread & butter express app are:

  1. The path specified in app.get calls must be the full path of the request (lines 5 and 9). When your app is hosted in IIS, depending on the configuration you may not necessarily own the entire namespace over port 80. Like in the example above, your IIS hosted express application may reside in the ‘express’ folder of  the ‘node’ virtual directory, and only own the subordinate URL namespace.
  2. Similarly to a non-express node.js app hosted in iisnode, the listen port is provided by IIS through the PORT environment variable. When you start your listener (line 13), this is the port you should specify.

The web.config

I talked about using the URL rewrite module for regular node.js applications before, and URL rewriting is perhaps even more relevant in case of URL-conscious express apps. The web.config below allows the express application saved in hello.js to receive HTTP requests directed at all URL paths subordinate to the ‘hello’ path component, as configured in lines 20-27:

   1: <configuration>
   2:   <system.webServer>
   3:  
   4:     <!-- indicates that the hello.js file is a node.js application 
   5:     to be handled by the iisnode module -->
   6:  
   7:     <handlers>
   8:       <add name="iisnode" path="hello.js" verb="*" modules="iisnode" />
   9:     </handlers>
  10:  
  11:     <!-- use URL rewriting to redirect the entire branch of the URL namespace
  12:     to hello.js node.js application; for example, the following URLs will 
  13:     all be handled by hello.js:
  14:     
  15:         http://localhost/node/express/hello/foo
  16:         http://localhost/node/express/hello/bar
  17:         
  18:     -->
  19:  
  20:     <rewrite>
  21:       <rules>
  22:         <rule name="hello">
  23:           <match url="hello/*" />
  24:           <action type="Rewrite" url="hello.js" />
  25:         </rule>
  26:       </rules>
  27:     </rewrite>
  28:  
  29:     <!-- exclude node_modules directory and subdirectories from serving
  30:     by IIS since these are implementation details of node.js applications -->
  31:     
  32:     <security>
  33:       <requestFiltering>
  34:         <hiddenSegments>
  35:           <add segment="node_modules" />
  36:         </hiddenSegments>
  37:       </requestFiltering>
  38:     </security>    
  39:     
  40:   </system.webServer>
  41: </configuration>

One other aspect worth pointing out is request filtering. Remember the express application relies on the express library installed in the node_modules directory? You probably don’t want the contents of this directory to be served by IIS in any shape or form, and you can express (sic!) that desire by adding it to hidden segments list (lines 32-38).

Voila!

Your IIS-hosted express node.js application behaves like expected:

image

image

So where can I get iisnode again?

Everything you need to get started is at https://github.com/tjanczuk/iisnode. Make sure to check out the express sample. Feedback welcome!

Using URL rewriting with node.js applications hosted in IIS using iisnode

In my last post I introduced the iisnode project which allows hosting node.js applications in IIS on Windows. In this article I discuss using URL rewriting with node.js apps hosted in IIS, functionality necessary in all but the most trivial IIS hosted node.js applications.

The problem

Consider the hello world sample code, saved in the hello.js file in IIS virtual directory:

var http = require('http');

http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello, world! [helloworld sample]');
}).listen(process.env.PORT);

along with the following web.config that registers the iisnode module as a handler of the hello.js file, therefore indicating it is a node.js application:

<configuration>
<system.webServer>
<handlers>
<add name="iisnode" path="hello.js" verb="*" modules="iisnode" />
</handlers>
</system.webServer>
</configuration>

When both hello.js and web.config above are saved in the ‘node’ virtual directory in IIS, one can navigate to the node.js application using the following URL:

http://localhost/node/hello.js

As expected, IIS will realize the hello.js file maps to the iisnode handler and invoke it, and as expected a few million CPU cycles later a ‘Hello, world’ is sent back to the client.

However, when a subordinate URL path is requested, IIS returns an error, e.g:

image

The reason for this is that IIS does not understand that paths subordinate to the ‘hello.js’ component of the path should all be handled by the hello.js application. This is unacceptable for all but the most simplistic node.js applications, which typically own the entire URL space. Fortunately, it is easily remedied with the URL rewriting module.

URL rewriting module to the rescue

Fixing this problem requires configuring the URL rewriting module to indicate that the section of the request path that is subordinate to the hello.js component should be handled by the handler for the hello.js component itself. From IIS perspective the request processing occurs as if the request was made for http://localhost/node/helloworld/hello.js, but the original URL path is preserved and available for the handler to act on. URL rewriting is specified in the web.config file as follows (lines 17-24):

   1: <configuration>
   2:   <system.webServer>
   3:     <!-- indicates that the hello.js file is a node.js application 
   4:     to be handled by the iisnode module -->
   5:     <handlers>
   6:       <add name="iisnode" path="hello.js" verb="*" modules="iisnode" />
   7:     </handlers>
   8:     <!-- use URL rewriting to redirect the entire branch of the URL namespace
   9:     to hello.js node.js application; for example, the following URLs will 
  10:     all be handled by hello.js:
  11:     
  12:         http://localhost/node/urlrewrite/hello
  13:         http://localhost/node/urlrewrite/hello/foo
  14:         http://localhost/node/urlrewrite/hello/foo/bar/baz?param=bat
  15:         
  16:     -->    
  17:     <rewrite>
  18:       <rules>
  19:         <rule name="hello">
  20:           <match url="hello/*" />
  21:           <action type="Rewrite" url="hello.js" />
  22:         </rule>
  23:       </rules>
  24:     </rewrite>
  25:   </system.webServer>
  26: </configuration>

The configuration above will not only cause all URLs subordinate to hello.js to handled by hello.js node application; it also allows the hello.js file name to be completely removed from the URL path. With the configuration above the node.js service can now be accessed using any of the following URLs:

http://localhost/node/hello/foo/bar/baz
http://localhost/node/hello/1/2/3
http://localhost/node/hello/f/b/b?param=bat
...

What request URL does the node.js application see?

The request URL passed by IIS to the node.js application is the original URL from before re-writing. For example:

image

Where do I get iisnode again?

Everything you need to get started is at https://github.com/tjanczuk/iisnode. Make sure to check out the urlrewrite sample. Feedback welcome!

Thursday, August 25, 2011

Hosting node.js applications in IIS on Windows

This post is updated as of February, 2013: iisnode v0.2.3 and node.js v0.8.19
Pусский перевод (Russian translation).

In this post I am discussing hosting node.js appplications in IIS on Windows using the iisnode project.

What benefits does iisnode provide?

The iisnode project provides a native IIS 7/8 module that allows hosting of node.js applications in IIS 7/8 and IIS 7/8 Express (WebMatrix). The project utilizes the Windows build of node.exe.

Some of the advantages of hosting node.js applications in IIS using the iisnode module as opposed to self-hosting node.exe processes include:

  • Process management. The iisnode module takes care of lifetime management of node.exe processes making it simple to improve overall reliability. You don’t have to implement infrastructure to start, stop, and monitor the processes.
  • Scalability on multi-core servers. Since node.exe is a single threaded process, it only scales to one CPU core. The iisnode module allows creation of multiple node.exe processes per application and load balances the HTTP traffic between them, therefore enabling full utilization of a server’s CPU capacity without requiring additional infrastructure code from an application developer.
  • Auto-update. The iisnode module ensures that whenever the node.js application is updated (i.e. the script file has changed), the node.exe processes are recycled. Ongoing requests are allowed to gracefully finish execution using the old version of the application, while all new requests are dispatched to the new version of the app.
  • Integrated debugging. The iisnode module is fully integrated with the node-inspector debugger. Node.js applications can be debugged remotely from any WebKit-based browser without any additional configuration or server side process creation.
  • Access to logs over HTTP. The iisnode module provides access the output of the node.exe process (e.g. generated by console.log calls) via HTTP. This facility is key in helping you debug node.js applications deployed to remote servers.
  • Side by side with other content types. The iisnode module integrates with IIS in a way that allows a single web site to contain a variety of content types. For example, static content (HTML, CSS, images, and client side JavaScript files) can be efficiently handled by IIS itself, while node.js requests are handled by iisnode. A single site can also combine PHP applications, ASP.NET applications, and node.js. This enables choosing the best tools for the job at hand as well progressive migration of existing applications.
  • Minimal changes to node.js application code. The iisnode module enables hosting of existing HTTP node.js applications with very minimal changes. Typically all that is required is to change the listed address of the HTTP server to one provided by the iisnode module via the process.env.PORT environment variable.
  • Integrated management experience. The issnode module is fully integrated with IIS configuration system and uses the same tools and mechanism as other IIS components for configuration and maintenance.
In addition to benefits specific to the iisnode module, hosting node.js applications in IIS allows the developer to benefit from a range of IIS features, among them:
  • port sharing (hosting multiple HTTP applications over port 80)
  • security (HTTPS, authentication and authorization)
  • URL rewriting
  • compression
  • caching
  • logging

Hello World

Follow the installation instructions at the iisnode project site to get the module and samples installed on your Windows box with IIS7 enabled.
The hello world sample consists of two files: hello.js and web.config.
This is the hello.js file from the helloworld sample:

var http = require('http');

http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end('Hello, world! [helloworld sample]');
}).listen(process.env.PORT);  

You will notice that the only difference between this code and the hello world sample from the front page of http://nodejs.org is in the specification of the listening address for the HTTP server. Since IIS controls the base address for all HTTP listeners, a node.js application must use the listen address provided by the iisnode module through the process.env.PORT environment variable rather than specify its own.

The web.config file is required to instruct IIS that the hello.js file contains a node.js application. Otherwise IIS would consider this file to be client side JavaScript and serve it as static content. The web.config designates hello.js as a node.js application by scoping the registration of the handler in the iisnode module to that file only:

<configuration>
  <system.webServer>
    <handlers>
      <add name="iisnode" path="hello.js" verb="*" modules="iisnode" />
    </handlers>    
  </system.webServer>
</configuration>

This handler registration allows the same web site to contain other *.js files (e.g. jQuery libraries) that IIS will continue serving as static files.

What else can iisnode do?

For those familiar with self-hosting node applications, iisnode combines the benefits of cluster, supervisor, node-inspector, forever, and node-static in a single package. 

Websocket support

As of version 0.2.x, iisnode supports hosting Websocket applications in IIS 8 on Windows Server 2012 and Windows 8. You can use any standard node.js modules that implement the Websocket protocol, including socket.io.

Scalability on multi-core servers

For every node.js application (e.g. hello.js above), iisnode module can create many node.exe process and load balance traffic between them. The nodeProcessCountPerApplication setting controls the number of node.exe processes that will be created for each node.js application. Each node.exe process can accommodate a configurable number of concurrent requests (maxConcurrentRequestsPerProcess setting). When the overall concurrent active request quota has been reached for an application (maxConcurrentRequestsPerProcess * nodeProcessCountPerApplication ), the iisnode module starts rejecting new HTTP requests with a 503 (Server Too Busy) status code. Requests are dispatched across multiple node.exe processes serving a node.js application with a round-robin load balancing algorithm.

Auto-update

Whenever the JavaScript file with a node.js application changes (as a result of a new deployment), the iismodule will gracefully upgrade to the new version. All node.exe processes running the previous version of the application that are still processing requests are allowed to gracefully finish processing in a configurable time frame (gracefulShutdownTimeout setting). All new requests that arrive after the JavaScript file has been updated are dispatched to a new node.exe process that runs the new version of the application. The watchedFiles setting specifies the list of files iisnode will be watching for changes.

Changes in the JavaScript file are detected regardless if the file resides on a local file system or a UNC share, but the underlying mechanisms are different. In case of a local file system, an OS level directory watching mechanism is used which provides low latency, asynchronous notifications about file changes. In case of files residing on a UNC share, file timestamps are periodically polled for changes with a configurable interval (uncFileChangesPollingInterval setting).

Integrated debugging

With iisnode integrated debugging you can remotely debug node.js application using any WebKit-enabled browser. Read more about iisnode integrated debugging.

Access to logs over HTTP

To help in ‘console.log’ debugging, the iisnode module redirects output generated by node.exe processes to stdout or stderr to a text file. IIS will then serve these files as static textual content over HTTP. Capturing stdout and stderr in files is controlled with a configuration setting (loggingEnabled). If enabled, iisnode module will create a per-application special directory to store the log files. The directory is located next to the *.js file itself and its name is is specifed with the logDirectoryName setting (by default “iisnode”). The directory will then contain several text files with log information as well as an index.html file with a simple list of all log files in that directory. Given that, the logs can be accessed from the browser using HTTP: given a node.js application available at http://mysite.com/foo.js, the log files of the application would by default be located at http://mysite.com/iisnode/index.html.

The logDirectoryName is configurable to allow for obfuscation of the log location in cases when the service is publicly available. In fact, it can be set to a cryptographically secure or otherwise hard to guess string (e.g. GUID) to provide a pragmatic level of logs privacy. For example, by setting logDirectoryName to ‘A526A1F2-4E22-4488-B930-6A71CC7649CD’ logs would be exposed at http://mysite.com/A526A1F2-4E22-4488-B930-6A71CC7649CD/index.html.

Log files are not allowed to grow unbounded. The maxLogFileSizeInKB setting controls the maximum size of an individual log file. When the log grows beyond that limit, iisnode module will stop writing to that file and create a new log file to write to. To avoid unbounded growth of the total number of log files in the logging directory, iisnode enforces two additional quotas. The maxLogFiles setting controls the maximum number of log files that are kept. The maxTotalLogFileSizeInKB controls the maximum total size of all logs files in the logging directory. Whenever any of the quotas are exceeded, iisnode will remove any log files not actively written to in the ascending order of the last write time.

Note that this design of the logging feature allows the node.js application to be scaled out to multiple servers as long as the logging directory resides on a shared network drive.

Side by side with other content types

One of the more interesting benefits of hosting node.js applications in IIS using the iisnode module is support for a variety of content types within a single web site. Next to a node.js application one can host static HTLM files, client side JavaScript scripts, PHP scripts, ASP.NET applications, WCF services, and other types of content IIS supports. Just like the iisnode module handles node.js applications in a particular site, other content types will be handled by the registered IIS handlers.

Indicating which files within a web site are node.js applications and should be handled by the iisnode module is done by registring the iinode handler for those files in web.config. In the simplest form, one can register the iisnode module for a single *.js file in a web site using the ‘path’ attribute of the ‘add’ element of the handler collection:

<configuration>
  <system.webServer>
    <handlers>
      <add name="iisnode" path="hello.js" verb="*" modules="iisnode" />
    </handlers>
  </system.webServer>
</configuration>

Alternatively, one can decide that all files in a particular directory are supposed to be treated as node.js applications. A web.config using the <location> element can be used to achieve such configuration:

<configuration>
  <location path="nodejsapps">
    <system.webServer>
      <handlers>
        <add name="iisnode" path="*.js" verb="*" modules="iisnode" />
      </handlers>   
    </system.webServer>
  </location>
</configuration>

One other approach one can employ is to differentiate node.js applications from client side JavaScript scripts by assigning a file name extension to node.js applications other than *.js, e.g. *.njs. This allows a global iisnode handler registration that may apply across all  sites on a given machine, since the *.njs extension is unique:

<configuration>
  <system.webServer>
    <handlers>
      <add name="iisnode" path="*.njs" verb="*" modules="iisnode" />
    </handlers>    
  </system.webServer>
</configuration>

Output caching

IIS output caching mechanism allows you to greatly improve the throughput of a node.js application hosted in iisnode if the content you serve can be cached for a time period even as short as 1 second. When IIS output caching is enabled, IIS will capture the HTTP response generated by the node.js application and use it to respond to similar HTTP requests that arrive within a preconfigured time window. This mechanism is extremely efficient, especially if kernel level output caching is enabled. Read more about using output caching with iisnode.

URL Rewriting

The iisnode module composes very well with the URL Rewriting module for IIS. URL rewriting allows you to normalize the URL space of the application and decide which IIS handlers are responsible for which parts of the URL space. For example, you can use URL rewriting to serve static content using the IIS’es native static content handler (which is a more efficient way of doing it that serving static content from node.js), while only letting iisnode handle the dynamic content.

You will want to use URL rewriting for majority of node.js web site applications deployed to iisnode, in particular those using the express framework or other MVC frameworks. Read more about using iisnode with URL rewriting.  Also check a recommended bootstrap configuration for URL rewriting in this post.

Minimal changes to existing HTTP node.js application code

It has been the aspiration for iisnode to not require extensive changes to existing, self-hosted node.js HTTP applications. To that end, most applications will only require a change in the specification of the listen address for the HTTP server, since that address is assigned by the IIS as opposed to left for the application to choose. The iisnode module will pass the listen address to the node.exe worker process in the PORT environment variable, and the application can read it from process.env.PORT:

var http = require('http');

http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end('I am listening on ' + process.env.PORT);
}).listen(process.env.PORT);  

If you access the endpoint created by the application above, you will notice the application is actually listening on a named pipe address. This is because the iisnode module uses HTTP over named pipes as a communication mechanism between the module and the worker node.exe process. One implication of this is HTTPS applications will need to be refactored to use HTTP instead of HTTPS. For those applications, HTTPS can be configured and managed at the IIS level itself.

Configuration with YAML or web.config

The iisnode module allows many of the configuration options to be adjusted using the iisnode.yml file or the system.webServer/iisnode section of web.config. Settings in the iisnode.yml file, if present, take precedence over settings in the web.config. Below is the list of options (most of which were described above) with their default values. For detailed and most current description of the options check out the configuration sample.

# The optional iisnode.yml file provides overrides of 
# the iisnode configuration settings specified in web.config.

node_env: production
nodeProcessCommandLine: "c:\program files\nodejs\node.exe"
nodeProcessCountPerApplication: 1
maxConcurrentRequestsPerProcess: 1024
maxNamedPipeConnectionRetry: 100
namedPipeConnectionRetryDelay: 250
maxNamedPipeConnectionPoolSize: 512
maxNamedPipePooledConnectionAge: 30000
asyncCompletionThreadCount: 0
initialRequestBufferSize: 4096
maxRequestBufferSize: 65536
watchedFiles: *.js;iisnode.yml
uncFileChangesPollingInterval: 5000
gracefulShutdownTimeout: 60000
loggingEnabled: true
logDirectoryName: iisnode
debuggingEnabled: true
debuggerPortRange: 5058-6058
debuggerPathSegment: debug
maxLogFileSizeInKB: 128
maxTotalLogFileSizeInKB: 1024
maxLogFiles: 20
devErrorsEnabled: true
flushResponse: false
enableXFF: false
promoteServerVars: 

Read more about the configuration support in iisnode.

Configuration using environment variables

In addition to using web.config and iisnode.yml, you can also configure iisnode by setting environment variables of the IIS worker process. Every setting available in iisnode.yml can also be controlled with environment variables. This option is useful for hosting providers wishing to offer a web based management experience for iisnode. Read more about configuring iisnode with environment variables.

Integrated management experience

The iisnode module configuration system is integrated with IIS configuration which allows common IIS management tools to be used to manipulate it. In particular, the appcmd.exe management tool that ships with IIS can be used to augment the iismodule configuration. For example, to set the maxProcessCountPerApplication value to 2 for the “Default Web Site/node” application, one can issue the following command:

%systemroot%\system32\inetsrv\appcmd.exe set config "Default Web Site/node" -section:iisnode /maxProcessCountPerApplication:2

This allows for scripting the configuration of node.js applications deployed to IIS.

Feedback

The iisnode project is open source on GitHub. I hope you will find it useful. Please report bugs, share ideas and experiences by leaving a comment here or through https://github.com/tjanczuk/iisnode/issues.

Read more

Debugging node.js applications with iisnode
URL rewriting and iisnode
Developing node.js applications in WebMatrix
Deploying node.js to Windows Azure using Windows Azure SDK for node.js
Using Event Tracing for Windows (ETW) to diagnose node.js applications deployed to iisnode
Overview of the architecture of iisnode
Using IIS output caching to improve performance of a node.js application
Hosting socket.io Websocket apps in IIS using iisnode
How to use Websockets with node.js apps hosted in iisnode

My Photo
My name is Tomasz Janczuk. I am currently working on my own venture - Mobile Chapters (http://mobilechapters.com). Formerly at Microsoft (12 years), focusing on node.js, JavaScript, Windows Azure, and .NET Framework.