Friday, October 29, 2010

WCF support for jQuery on wcf.codeplex.com

Today we have shipped the first release of WCF Web APIs on wcf.codeplex.com. WCF Web APIs is a set of open source components that make it easy to develop WCF web services that target web and HTTP clients, including JavaScript Ajax applications running in a browser.
image[1]
One of the components is WCF support for jQuery. It provides great Visual Studio experience for creating WCF services that are optimized for consumption from JavaScript clients running in a browser, in particular jQuery. This includes better support for the JSON format as well as application/x-www-form-urlencoded data. Submitting an HTML form to a WCF service with an Ajax call from jQuery has never been easier. Let’s have a look at the feature.

Installation

You must have Visual Studio 2010 to use the bits. Downloading and installing the WCF Support for jQuery 10.10.27 release from wcf.codeplex.com will install Visual Studio templates, as well as copy reference files and samples to %ProgramFiles%\Microsoft SDKs\WCF Support for jQuery.

Visual Studio project template

Create a new project and choose the “WCF jQuery Service Application” template in the “Web” group:
image
The template creates a web application project that contains a WCF service (WcfJQueryService.cs) and a sample HTML page that demonstrates how to invoke that service with an Ajax call using jQuery (default.htm):
image
The project references include two assemblies that have been installed as part of the WCF Support for jQuery release (you get get the source code of these assemblies from the wcf.codeplex.com site). System.Runtime.Serialization.Json.dll contains the JsonValue component which facilitates operating on JSON data in an untyped manner (JsonValue is to JSON what XElement is to XML). JsonValue supports serialization and deserialization to the JSON format. It has already shipped in Silverlight before, now we are porting it with some enhancements to .NET Framework.
image
The Microsoft.ServiceModel.Web.jQuery.dll contains a number of WCF components that integrate JsonValue with the WCF HTTP programming model, and add support for processing JSON and application/x-www-form-urlencoded data, which are the two most commonly used data formats for Ajax calls in JavaScript applications.
The WCF Support for jQuery package also includes an item template that allows a jQuery-enabled WCF service to be added to an existing web application project.

From jQuery client to WCF service

Let’s have a look at the WCF service to see what the programming experience is:
  1. [WebInvoke(UriTemplate = "", Method = "POST")]
  2. public JsonValue Post(JsonObject body)
  3. {
  4.     dynamic item = body;
  5.     item.id = items.Count;
  6.     items.Add(item);
  7.     return items[items.Count - 1];
  8. }
The method above is extending the WCF HTTP programming model available since .NET Framework 3.5 SP1 with support for JsonValue as the method parameter and return value type. If JSON or application/x-www-form-urlencoded data arrives in the body of the POST request, it will be deserialized into the JsonObject instance passed to the Post method. The JsonValue instance the method returns will be serialized into JSON format on the HTTP response.
The default.html file shows how this method can be called from JavaScript, in this case using the Ajax call from the jQuery framework:
  1. var person = {
  2.     Name: "John Doe",
  3.     Age: 21
  4. };
  5. $.post("./Service/", person, function (result) {
  6.     //...
  7. });
In line 1-4 a JavaScript object ‘person’ is created. Line 5 initiates a POST HTTP request to the WCF service identified with a relative ‘./Service/’ URL, and passes the ‘person’ instance to be sent to the service. By default, jQuery serializes the JavaScript object using the application/x-www-form-urlencoded format. The HTTP request looks like the one below, with line 4 containing the serialized ‘person’ instance:
  1. POST http://127.0.0.1:8326/Service/ HTTP/1.1
  2. Content-Type: application/x-www-form-urlencoded
  3.  
  4. Name=John+Doe&Age=21
The interesting aspect of WCF support for application/url-form-encoded is that JavaScript applications can use that format to serialize not only simple lists of key/value pairs traditionally associated with an HTML form submission, but also arbitrarily complex JavaScript objects. JsonValue supports deserialization of data from application/x-www-form-urlencoded format following the jQuery’s $.param() serialization conventions. To see this format in action, let’s send a more complex object to the server by modifying the ‘person’ instance to:
  1. var person = {
  2.     Name: "John Doe",
  3.     Age: 21,
  4.     Children: [
  5.         {
  6.             Name: "Jessica",
  7.             BestToy: "Fish"
  8.         },
  9.         {
  10.             Name: "Jeff",
  11.             BestToy: "Donkey"
  12.         }
  13.     ]
  14. };
The corresponding HTTP request with this data is shown below, with line 4 containing the complex ‘person’ structure serialized into an application/x-www-form-urlencoded format using the $.param() convention from jQuery:
  1. POST http://127.0.0.1:8326/Service/ HTTP/1.1
  2. Content-Type: application/x-www-form-urlencoded
  3.  
  4. Name=John+Doe&Age=21&Children%5B0%5D%5BName%5D=Jessica&Children%5B0%5D%5BBestToy%5D=Fish&Children%5B1%5D%5BName%5D=Jeff&Children%5B1%5D%5BBestToy%5D=Donkey
Although jQuery Ajax calls use application/x-www-form-urlencoded format by default, it is very easy to serialize the request payload in JSON format instead. Using Douglas Crockford’s json2 serializer (line 4), one can make a jQuery call like this:
  1. $.ajax({
  2.     type: "POST",
  3.     url: "./Service/",
  4.     data: JSON.stringify(person),
  5.     contentType: "application/json"
  6. });
Which results in the following HTTP request:
  1. POST http://127.0.0.1:8326/Service/ HTTP/1.1
  2. Content-Type: application/json
  3.  
  4. {"Name":"John Doe","Age":21,"Children":[{"Name":"Jessica","BestToy":"Fish"},{"Name":"Jeff","BestToy":"Donkey"}]}
A great feature of the JsonValue programming model in WCF is that regardless of the client’s choice of JSON or application/x-www-form-urlencoded as the serialization format for the request, WCF will deserialize and normalize the data into an instance of JsonValue. Given that, the application code may stay decoupled from the protocol.
So how does one access the data the client sent from the WCF service? JsonValue offers dictionary-like programming model. For example, to determine the favorite toy of the second child of the person, one could write the following code:
  1. [WebInvoke(UriTemplate = "", Method = "POST")]
  2. public JsonValue Post(JsonObject body)
  3. {
  4.     string favoriteToyOfSecondChild = (string)body["Children"][1]["BestToy"];
  5.     return favoriteToyOfSecondChild;
  6. }
If the return type of a WCF method is JsonValue, the response content type is always application/json. Note that in line 5 above we are actually returning a string instance, but an implicit cast exists between a number of primitive data types and JsonValue. In this case the specific HTTP response would look as follows:
  1. HTTP/1.1 200 OK
  2. Content-Type: application/json; charset=utf-8
  3.  
  4. "Donkey"
Surely enough, donkey is the favorite toy of John’s second child.

JsonValue 201: dynamic properties

JsonValue supports a few more interesting constructs that facilitate operating on untyped data.
First of all, there is support for dynamic properties, which is more than just a syntactic improvement over array indexers. The two lines below extract the same value from a JsonObject:
  1. JsonObject body;
  2. string favoriteToyOfSecondChild1 = (string)body["Children"][1]["BestToy"];
  3. string favoriteToyOfSecondChild2 = (string)body.AsDynamic().Children[1].BestToy;
The notation in line 2 is closer to the code one would expect to write to navigate an object in JavaScript, and some people find it more natural than using indexers in line 1.
Given that the data received from the client has not been validated against a specific schema of a CLR type, one cannot be sure the JsonObject really contains the data the server application expects, or whether the object even has the correct structure. For example, a malicious client could try to send an arbitrary string in the “Age” field instead of a number, or could completely omit the Children array. In that case, the indexer APIs (line 1 above) would throw an exception as soon as the code attempts to access a non-existing property (which is consistent with the behavior of indexers in the .NET Framework in general). One would therefore have to write a lot of validation code before accessing the data in the JsonValue object using indexers, especially if the data hierarchy is deep.
The dynamic properties of JsonValue offer a much more convenient validation experience. One can “dot into” any level of a complex object without any exceptions being thrown, only to perform a test of the value’s existence and CLR type match at the very end, with a very Fluent-like experience.
  1. JsonObject body;
  2. string aValue;
  3. if (!body.AsDynamic().Children[7].BestToy.TryReadAs<string>(out aValue))
  4. {
  5.     aValue = "None"; // assume a default value
  6. }
You can imagine how this programming model is reducing the amount of code you need to write when navigating deep data hierarchies:
  1. body.AsDynamic().I.May.Exist.Or.Maybe.Not.TryReadAs<string>(out aValue)

JsonValue 301: LINQ to JSON

Here is another useful feature JsonValue offers: LINQ to JSON. Using the ‘person’ object introduced above, let’s assume you are writing a method that is supposed to return the favorite toys of those children of the person whose name begin with ‘J’ (does this sound like a test from your SQL class?). Here is the code you could write:
  1. JsonObject body;
  2. string[] favoriteToys =
  3.     (from child in (JsonValue)body.AsDynamic().Children
  4.      where child.Value.AsDynamic().Name.ReadAs<string>(string.Empty).StartsWith("J")
  5.      select child.Value.AsDynamic().BestToy.ReadAs<string>("No favorite toy")).ToArray();
There are several interesting properties of the code above:
  1. You don’t have to worry whether the ‘Children’ property exists on the ‘body’ JsonObject (line 3), or whether it is a JSON array. If it does not, the LINQ expression will just not return any matches. No exception is going to be thrown.
  2. You don’t have to worry whether the ‘Name’ property exists on a child object (line 4). Instead, you can provide a default name in the ReadAs method that will be returned in the absence of the property.
  3. Similarly, in case a child whose name starts with ‘J’ has been found, you don’t have to worry whether the child has a ‘BestToy’ property (line 5) – if she does not, you can just fall back to returning a ‘No favorite toy’ string.
In other words, writing compact and robust LINQ expressions against JSON data with unknown structure is easy.

Read more

I have only touched on the key features of the WCF support for jQuery release. Find out more in the dedicated documentation section on wcf.codeplex.com. Let us know what you think and help us shape the direction of this feature going forward.

17 comments:

  1. The dynamic and LINQ additions for JSON look highly productive. Great stuff. Thanks for the detailed post.

    ReplyDelete
  2. Finally, I really need to implement this kind of services. Happy that microsoft is waking up.

    ReplyDelete
  3. Hi Tomas,

    The project template has a class called WcfJQueryService, and it doesn't implements an interface that define the service contract.

    Service contract interface always was required as best pratice, including services that are being building based on System.ServiceModel.Web API.

    So, the question is: from now on, this requirement is not necessary anymore?

    ReplyDelete
  4. Anonymous, the WcfJQueryService included in the template indeed implements the service contract in-line instead of separating it out into an interface. There is not hard guadiance on whether service contracts should be defined in-line vs as separate interfaces, and you are certainly welcome to choose an approach that works better in your application. In case of WCF HTTP services there is perhaps less of a reason to separate the service contract into its own interface: there is no standardized metadata/description format for HTTP/REST services, and as such a CLR construct that corresponds directly to such a description oftentimes is not needed in an application.

    ReplyDelete
  5. This is very nice, but is there a way to serialize a POCO to a JsonValue?

    ReplyDelete
  6. Anonymous, you can convert from a POCO type to JsonValue using a CreateFrom extension method for JsonValue:

    Foo aPocoType;
    JsonValue v = JsonValue.CreateFrom(aPocoType);

    Note that currently this conversion is implemented by serializing aPocoType using DataContractJsonSerializer and then deserializing it back to JsonValue. We are working on a more efficient transformation mechanism between CLR types and JsonValue.

    ReplyDelete
  7. Thanks Tomasz!

    I also have a question about WCF HTTP. You guys have open sourced the two libraries: Microsoft.ServiceMode.Http and Microsoft.ServiceModel.Http.Client.

    However, both of these open source libraries are dependent on Microsoft.Http.dll, Microsoft.Http.Extensions.dll, and Microsoft.QueryComposition.dll which are not open source.

    Are these three binaries part of another SDK or open source toolkit that will always be available and supported?

    ReplyDelete
  8. In your article you mention : "The WCF Support for jQuery package also includes an item template that allows a jQuery-enabled WCF service to be added to an existing web application project."

    Is there a way to explain if this is also applicable to e.g. PHP existing web applications?

    ReplyDelete
  9. With regards to the question on open source. Currently bulk of the source and binaries on Codeplex are under a permissive license similar to that of ASP.NET MVC. This is not open source, bu it does allow you use it within your organization and to fork it if necessary. The QueryComposition.dll we did not release the source for yet due to time constraints on cleaning up the code. We are planning to get the source out there though.

    On the larger question of our licensing, we are in the process of working toward releasing under an OSS license. Is having the source under OSS a key requirement for you?

    Thanks

    ReplyDelete
  10. Anonymous, you can certainly call into jQuery-enabled WCF services from within client side JavaScript rendered from PHP running on the server.

    ReplyDelete
  11. @Glenn Block
    Having the source under OSS for the dependencies isn't a key requirement. I'm more concerned about their long-term support. If this project happens to be abandoned and I have a lot of projects invested in them, I have the option of maintaining the source myself for the WCF HTTP and WCF jQuery libraries. But that wouldn't be the case for their dependencies. If the dependencies aren't part of an SDK, there's no published documentation for them, and I wouldn't have the source code either as an alternative to the documentation. That would make maintenance of the WCF HTTP/jQuery source kind of tough.

    ReplyDelete
  12. I couldn't use JsonValue.CreateFrom. Do I need to add any other reference?

    ReplyDelete
  13. Why call it wcfjqeury service application(in vs) or wfc jquery support? The method of submitting the right data format for wcf to process has nothing to do with jquery. Per quote in this post "The Microsoft.ServiceModel.Web.jQuery.dll contains a number of WCF components that integrate JsonValue with the WCF HTTP programming model, and add support for processing JSON and application/x-www-form-urlencoded data, which are the two most commonly used data formats for Ajax calls in JavaScript applications.". I can write a javascript ajax framwork called x or use asp.net ajax to simulate the same request jquey does and wcf wouldnt care what framework I use so please please please, dont confuse people by naming the technogy/dll something that has nothing to do with it. People are going to read the name and automatically assumes this is only for jquery. Unless, Microsoft.ServiceModel.Web.jQuery.dll contains logic specific only to jquery, lets try not to confuse the public. Great agnostic http wcf implementation, now lets not tigh it to a particular ajax/scripting framework. Thanks for the post and good job to MS WCF team.

    ReplyDelete
  14. I agree with emmanuel's post. I was a bit confused at first with how jQuery was related in any way to server-side code or technologies.

    ReplyDelete
  15. @Anonymous (12/29/2010)

    It's actually JsonValueExtensions.CreateFrom, not JsonValue.CreateFrom found in System.Runtime.Serialization.Json.

    ReplyDelete
  16. I just tried this and I'm getting the following exception:

    Server Error in '/' Application.

    To be XML serializable, types which inherit from IEnumerable must have an implementation of Add(System.Object) at all levels of their inheritance hierarchy. System.Json.JsonValue does not implement Add(System.Object).

    Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

    Exception Details: System.InvalidOperationException: To be XML serializable, types which inherit from IEnumerable must have an implementation of Add(System.Object) at all levels of their inheritance hierarchy. System.Json.JsonValue does not implement Add(System.Object).

    Am I missing something? I got WCF WebApi from Nuget... I see all the assemblies referenced on this post being included in my project.

    Thanks!
    Raciel

    ReplyDelete
  17. I created a function like folowing to work with JQuery Datatable. But it alwsys failed with server failure. Do you have any idea?
    [WebGet(UriTemplate = "GetTableData")]
    public JsonValue GetTableData()
    {
    ...
    dynamic retData = new {
    sEcho = sEcho,
    iTotalRecords = filteredCompanies.Count(),
    iTotalDisplayRecords = filteredCompanies.Count(),
    aaData = filteredCompanies
    } ;

    return retData;
    }

    ReplyDelete

My Photo
My name is Tomasz Janczuk. I am currently working on my own venture - Mobile Chapters (http://mobilechapters.com). Formerly at Microsoft (12 years), focusing on node.js, JavaScript, Windows Azure, and .NET Framework.