Sunday, July 11, 2010

Laharsub: HTTP REST pub/sub service for AJAX and Silverlight web clients

Today I would like to announce the creation of the Laharsub project on Codeplex and invite you to download, use, review, comment, discuss, share ideas, and otherwise collaborate on this interesting technology.

Laharsub project on Codeplex

The objectives and scope of the Laharsub project are captured on the project’s site; to quote:

Overview
The growing trend of application development targeting web clients and open standards (HTML, JavaScript, HTML 5) generates demand for ever richer experiences for the web platform. Many interesting web applications require more than the traditional request/response communication pattern enabled by Ajax technologies. Web based chat or collaboration applications, multiplayer online gaming, or news or financial data dissemination applications require low latency, asynchronous data transfer from the server to the web client. Communication patterns in most of these applications are using a pub/sub model, with topics that multiple clients can publish to, subscribe to, and receive notifications from.

Goals
The goal of the Laharsub project is to provide a solution that makes it easy for web applications to organize internet scale message exchange using a publish/subscribe pattern. The project is an ongoing experiment with a variety of web technologies. Current focus is on AJAX (in particular jQuery) and Silverlight clients, a REST based HTTP long polling subscription protocol implemented by a .NET WCF HTTP middle tier service, and researching middle tier and back end technologies that enable scale-out to a large number of clients.

The project is directly related to my interest in rich communication patterns for web application. It draws on the lessons learned from the HTTP Polling Duplex protocol in Silverlight, and is an attempt to experiment with alternative approaches that may improve developer’s experience in a number of scenarios of data push to a web client. In particular:

  • It focuses on backend scalability,
  • It focuses on Ajax clients as much as on RIA (Silverlight) clients,
  • It focuses on the publish/subscribe message exchange pattern as opposed to duplex messaging capability, reflecting the majority use case for the HTTP Polling Duplex protocol in Silverlight.

I am looking forward to your feedback.

12 comments:

  1. It'll be interesting to see how you get around the one connection per thread in IIS. Unless youare not using IIS to host ur services...

    ReplyDelete
  2. As a matter of fact, Laharsub only supports hosting in a Windows Service or a console application, and scalability is the key reason.

    ReplyDelete
  3. Hi,
    What advantage does Http Long Polling provide over using HTTP Polling Duplex protocol with persistant storage like in the RCA application by Aleksey?

    Thanks
    Cennest

    ReplyDelete
  4. Cennest,

    Aleksy's RCA framework and Laharsub both use a protocol based on the HTTP long polling principle. The RCA framework uses the HTTP Polling Duplex protocol that shipped in Silverlight since version 2, with a server implementation modified to allow for scale-out following the mechanism described at http://tomasz.janczuk.org/2009/09/scale-out-of-silverlight-http-polling.html. The key differences between the approach to HTTP long polling between the RCA framework and Laharsub are following:

    1. RCA protocol uses SOAP over HTTP, while Laharsub uses pure HTTP. As such, RCA works well with Silverlight clients, while Laharsub supports a broader variety of clients, including JavaScript, Silverlight, .NET, and any other technology that supports HTTP.

    2. RCA protocol allows for a single message per HTTP long poll response to be sent back to the client, while Laharsub allows multiplexing multiple messages on a single HTTP response using multipart/mixed content type. As such Laharsub has the potential to support much larger throughput of messages from the server to the client; see a similar mechanism added in Silverlight 4 to get an idea of the potential throughput gains: http://tomasz.janczuk.org/2010/03/comparison-of-http-polling-duplex-and.html.

    3. Laharsub provides a generic pub/sub semantics that allows for scale-out: it is provides a set of REST HTTP APIs; it maintains no protocol state on the server; and it allows a custom pub/sub backend to be plugged in.

    4. RCA framework targets deployment in Azure, while Laharsub currently focuses on self-hosted deployment.

    ReplyDelete
  5. Thanks Tomasz.This really helps!.
    We will be using the Laharsub model for our work and will get back to you with any further questions.
    Again, Thank you very much for your time!
    Cennest

    ReplyDelete
  6. Hi Tomasz,

    "4. RCA framework targets deployment in Azure, while Laharsub currently focuses on self-hosted deployment. "

    What changes are needed to enable deploying Laharsub on Azure? We intend to host our chat server on a scalable infratructure and were eyeing Azure.
    We do want to go the pure HTTP, multi-message mode and hence are more inclined towards Laharsub but we need deployment on Azure.

    Would appreciate your guidance of whether it will be possible with Laharsub and if so then what are the modifications needed.

    Thanks
    Cennest

    ReplyDelete
  7. Cennest,

    I have not investigated the possibility of Azure deployment of Laharsub yet. In theory you should be able to host Laharsub in IIS in an Azure web role, as it is a regular WCF HTTP service. In fact, you could probably use the Laharsub.exe as is since the WCF service type is public. One reason I have not prioritized Azure web role deployment investigations is that HTTP long polling mechanisms do not scale well when deployed in IIS due to threading model of IIS. There is currently no way to self-host a WCF service in Azure.

    ReplyDelete
  8. Tomasz, would a worker role (as opposed to a web role) in Azure work as "self-hosting a WCF service"?

    ReplyDelete
  9. Tim, to the best of my knowledge you should be able to host Laharsub in Azure's worker role without problems. In fact, I am planning to make this feature first-class going forward: check out the product backlog at http://laharsub.codeplex.com/wikipage?title=Future%20plans.

    ReplyDelete
  10. Hey Tomasz,

    I am building a pub/sub service, over HTTPS, in azure that needs to be scalable and will be using in production. I see the blog/project has not been updated since 2010 so assume the project has been deprecated. If building now, is Azure Service Bus the recommended approach? Note I am very new to Azure, but have an aws background.

    Thanks!

    Kevin

    ReplyDelete
  11. I also came across SignalR (http://msdn.microsoft.com/en-us/magazine/hh965663.aspx). This seems to overlap with a lot of the requirements of the Laharsub project.

    ReplyDelete
  12. Yes, SignalR is the way to go if you need .NET code on the backend.

    ReplyDelete

My Photo
My name is Tomasz Janczuk. I am currently working on my own venture - Mobile Chapters (http://mobilechapters.com). Formerly at Microsoft (12 years), focusing on node.js, JavaScript, Windows Azure, and .NET Framework.