Agile Ajax

Comet 2008: The State of Play in Reverse Ajax

Just as with advertising measurement for Ajax, I continue to follow the technology known as Comet (open up an HTTP request from browser to server, then keep it open, pushing content down periodically) to see if it is ready for prime time. I thought I'd share my reading list from the last few months:

• 20,000 Reasons Why Comet Scales - Greg Wilkins achieved sub-second latency using Dojo Cometd/Bayeux and Jetty. Yes, it's a benchmark, and benchmarks can't be swallowed whole, but it's still quite impressive to see just two load balanced servers manage this kind of load.
• Comet: Reverse Ajax for streaming data from the server - Daniel Rubio of TheServerSide take an introductory look at Comet, but from the Enterprise/SOA perspective. If you're looking to convince your CTO to adopt Comet for your enterprise, you can adapt some of Daniel's arguments.
• A Comet Implementation for Google Web Toolkit - somehow I missed this one from last July. Does the usual hidden iframe implementation for Comet on the client. Simple design with lots of code.
• Comet Daily - Yep. The Comet technology now has its own blog (though it isn't quite daily). Lots of juicy articles, reviews and tutorials.
• Comet Support in GlassFish - Jean-Francois Arcand writes a compact little article on how to do Comet with GlassFish. Has links to lots of little demos.
• Comet Ruby on Rails - not to be left out, the Rails folks are trying their hand at Comet with the Juggernaut plugin.

While there is some progress, I have to say that Comet still seems to me like climbing up the down escalator. If you really want this kind of functionality in the browser, develop a new protocol and lobby for built in browser support.

Technorati Tags: ajax, comet, gwt, rails, dojo

GWT and Comet - Using GWT with Jetty Continuations

Let's review: Comet basically means keeping an HTTP connection open between the server and the browser so the former can stream messages to the latter without polling (see the Ajax Patterns page on HTTP Streaming for info on how it's done). This poses a problem for commonly used web infrastructure, like app servers, which usually associate one thread per connection. If you have lots of open connections, let's say 10,000, then you will need that many threads that are doing very little but sitting around waiting for the next message to be sent. Regardless of OS, 10,000 threads puts a pretty heavy load on a system. As a solution, Greg Wilkins proposed an extension of the Servlet standard to allow "continuations." What are they?

A java Filter or Servlet that is handling an AJAX request, may now request a Continuation object that can be used to effectively suspend the request and free the current thread. The request is resumed after a timeout or immediately if the resume method is called on the Continuation object.

Continuations, while not a part of the Servlet standard, are available in Jetty 6.

GWT, or the Google Web Toolkit, is the client-side framework that let's you write and debug your application in Java, then deploy it as Javascript.

Whew! Now that the explanations are out of the way, I can tell you that Greg Wilkins (of Jetty Continuations fame) has been noodling about how to get GWT to do Comet, specifically using Jetty's continuations.

Unfortunately GWT has not made it easy to use continuations within their RPC mechanism. Firstly they catch Throwable, so he Jetty RetryException is caught. Secondly they have made most of the methods on the GWT servlet final, so you cannot fix this by extension.

Luckily GWT is open source under the apache 2.0 license, so it was possible to do a cut/paste/edit job to fix this. The OpenRemoteServiceServlet recently added to Jetty is [a] version of GWTs RemoteServiceServlet without the final methods and a protected method for extending exception handling.

Once the GWT remote service servlet has been opened up, it is trivial to extend it to support Continuations...

There's enough code and pointers in the article to make it an effective, if short, howto. Apparently http://www.gpokr.com/ uses these extensions to GWT in its app, so it's proven in production.

Update: I see that there is an entry entitled "RemoteServiceServlet refactoring to be pluggable" in the GWT 1.4 dev plan. Is this what I think it is?

Technorati : ajax, comet, continuations, gwt, jetty

Is It Time To Stop Pushing the Browser Around?

In my first ever real grownup consulting engagement, back in 1990, I was called in to improve the data cleaning operations of a small testing company. This was back in the days when even companies heavily dependent on computer technology relied on a single "computer guy." This particular guy had rigged up a data cleaning system on a set of state-of-the-art 80386 machines. He had set up a folding table in one corner of the office ("Machine room? What's a machine room?") and taped "Do not touch!!!" signs to the monitors. The guy would take the tapes containing maybe 20-30MB of data and run then for 6 days through his cleaning environment. If something blew up or a new defect was identified in the source data, boom! another 6 days.

When I probed a little bit about what he was using to perform this magic, I found out, to my horror, that he had implemented the whole data cleaning shebang as a set of WordPerfect macros! Ouch! Talk about the wrong tool. I recommended they buy a little Sparcstation (sorry, x86 unix was kind of expensive and unreliable back then) and solved their problem with a modest amount of Perl4.'

So what does WordPerfect have to do with Javascript in the Browser? As people have been testing the limits of Ajax, we see more sophisticated SOA and messaging concepts and abstractions making their way into the picture. Coach Wei, a clever and insightful man, has an article up now on the Nexaweb's IMB (Internet Messaging Bus).

It's a clever concept of how to introduce MOM/ESB into a browser environment (Nexaweb also supports Java and Desktop clients, as you can see in the article above). I've implemented a number of distributed apps using JMS as well as a few using ESB's such as WebMethods, Tibco and Mule. It's an eleagant way to design apps, but requires a somewhat different approach and outlook than OOP (not that you can't include OO software in the mix as a component). See Enterprise SOA: Service-Oriented Architecture Best Practices for a pretty good read on what's involved in building SOA systems.

I checked out a few of the applications mentioned in his article, and they we slick, responsive, elegant. Nice work by the Nexaweb folks.

So again, what does this have to do with WordPerfect macros? Using a technology in an inappropriate way will come to bite you. All that infrastructure and encapsulation and tunneling comes at a price, espectially in a still somewhat wobbly execution environment like Javascript on the browser. Sure enough, those Nexaweb apps start out nice, but over time start to peg my CPU at 100% on both IE and Firefox. Don't even think of running more than one of these apps on your machine at one time. Maybe you can put a "Do Not Touch!!!" graphic over all of the other application icons on your desktop. ;-) I don't think this is Nexaweb's fault. Javascript in the browser still has a long way to come before you can write truly reliable, complex and layered systems.

Further, lots of these boundary pushing technologies make use of characteristics of the browser that are more side-effects than features. Look at Comet, for example. Common Comet techniques such as keeping the response from the server to the browser open (in an Iframe, etc.) or streaming a multipart HTTP message via XHR don't work the same in all browsers, are not part of any browser requirements, and could change with the next release. This is no way to implement serious applications.

Now I'm not saying that Comet is a bad idea, or that Nexaweb is peddling SOA/MOM junkware. Quite the contrary. I like what I see and hope to see more. What I object to is that all of this is being done via the back door. If we are going to implement fancy communication layers in the browser, let's do it by the front door, by requesting a more robust Javascript execution environment perhaps with threading and better networking control built into the browser.

Today's browsers are not that far removed from the Netscape Navigator with Livescript (later renamed to Javascript) that I used in 1995 to toggle a few heirarchical checkboxes. We are asking them to do things they were not designed to do. As we push the envelope of what a web app is and can do, we should do so in a smart way. Otherwise we are just like that guy who took WordPerfect macros -- a tool designed to perform simple textual and menu operations -- and wrote an ETL system. Yes, it can be done, but why would you?

Technorati : ajax, coach wei, esb, jms, mom, nexaweb, soa

Roundup - Comet Tutorials, Examples and Resources

I've been collecting resources recently for Comet and Cometd. Just a reminder on what Comet is:

Fundamentally, [Comet applications] use long-lived HTTP connections to reduce the latency with which messages are passed to the server. In essence, [Comet applications] do not poll the server occasionally. Instead the server has an open line of communication with which it can push data to the client.

Cometd (originally call "Shortbus") is an attempt to standardize Comet as a protocol with a reference implementation. The protocol, Bayeux, runs on top of the server push to allow pub sub subscribe to events, sort of like a JMS for Javascript.

I've tried to focus here on resources that explain the nitty-gritty of streaming to the browser. Unfortunately there aren't a whole lot of tutorials of this sort. There are, however, a ton of libraries, tools and frameworks out there that abstract the basics, and I'll cover those in a future post.

Tutorials and Examples

• gCometd: an example introducing Cometd, Bayeux using Grizzly, the Glassfish NIO HTTP listener.
• Pageview counter Comet example. Probably the most understandable of all the tutorials.
• Michael Mahemoff's post on portable Comet using Iframes.
• java.net tutorial on doing "Reverse Ajax" with DWR, which includes Comet as one of three approaches.

Resources

• Of course the Cometd home page. Still a little light on information.
• Michael Mahemoff's page from his ajaxpatterns Wiki explaining HTTP Streaming
• Alex Russell's (of Dojo) intro to Comet.
• An analysis by Alex Russell of the GTalk interface in GMail. Good analysis of the low level support for streaming HTTP in the browser.
• An old but still good blog entry by John Resig on HTTP Streaming and how it works.
• Pushlets, an early form of comet.

Technorati : ajax, comet, cometd, examples, tutorials

Bayeux - A JSON Protocol For Publish/Subscribe Event Delivery

The Dojo Foundation has certainly been very active in the last few weeks. In addition to the inclusion of cometd in their stable of projects, they're involved in putting together a specification for sending events to browsers -- async update, in other words, kind of like a JMS for JSON, Comet and browsers. The spec looks like a solid effort to turn the ad-hoc comet efforts into something more robust. From the draft intro:

Initial connection setup specifies to the system a message envelope type for all communications on a particular channel, but inside this envelope a normalized JSON structure is used to communicate with all endpoints. The event router is channel based, with rules based on longest-prefix match dispatching for registered listeners.

The "/meta/*" channel is reserved for communications with the event router itself (including connection setup, tear-down, and re-connection), and all conformant clients and servers must implement the following meta-channel verbs:

• handshake
• connect
• reconnect
• disconnect
• status
• subscribe
• unsubscribe
• ping

The full draft specification for the Bayeux protocol can be found here. Also, Alex Russel has blogged about Bayeux and Cometd here and here.

It certainly looks interesting. Of course, the key to the success of any standard is wide adoption. Given the tendency of folks in the Ajax community to roll their own, I have my doubts right now on whether this standard gains enough traction. Maybe it would help if they got the Open Ajax Initiative involved.

Curious name for a protocol. I wonder if it's related to the Bayeux wide-area data disemination technology that was developed for streaming media applications by Shelley Q. Zhuang, et. al.? No info about the name choice from Dojo.

Technorati : ajax, bayeux, comet, cometd, dojo

Cross Domain Comet

I hate to be a blog echo chamber, but there's a post from Alex Russell of Dojo Toolkit that is worth spreading around a bit more.

First, some background. There are two technologies that Alex and Co. have been working on. One is called JSONP, which stands for JSON with Padding. The essential idea is that you can insert a script tag dynamically in a document and have it pull a script from a server in a different domain. This URL for this script takes a parameter that is prepended to the JSON which is also wrapped in parentheses. In the case where the prefix is null, we have just a JSON object. Otherwise, it's a function call. This gives us the ability to do a callback when the script loads; you want to do a callback, otherwise, how will you know that the thing loaded? Obviously, this requires some cooperation from the server -- you can't just do this with any old JSON service. A discussion by Alex of this technology can be found here. Dan Theurer has a somewhat clearer explanation, IMHO, of a similar technology used by Yahoo.

The other technology is Cometd. What is Comet? Without getting too technical, when a web application is to update a browser using AJAX, the browser "polls" the client, i.e. it keeps opening up connections to the server to ask "is it ready?" With Comet, the browser opens up a connection to the server and keeps it open until the server has something to say. There are all sorts of issues in doing this, from limitations of the HTTP protocol, to the Servlet specification, to the consumption of resources on servers and network appliances that are expecting short connections. Cometd is a server that tries to solve some of the issues.

What Alex has done is combine Cometd and JSONP to allow cross domain Comet. You can read Alex's post here...

Is Cross Domain AJAX/Comet a 'Good Thing?'

There's a school of thought that AJAX apps should really be the SOA orchestrator of various backend services from various domains -- a little Amazon S3 here, a little Google maps there, a little Yahoo mail there -- and voila!, the ultimate mashup. While it is seductive to think you can assemble applications in this way, it makes me nervous for some reason that I can't quite articulate. Having been the victim of bad SOA governance within companies, I have to think that these types of applications will be very brittle at the least, and vulnerable to security exploits at the worst.

Grizzly - Infrastructure for COMET and AJAX

Other than being the black hole into which the JavaMail API has disappeared, the Open Source Glassfish project -- a J2EE app server building on the Toplink and J2EE code donated by Oracle and Sun respectively -- has some interesting stuff under the hood. They have a new NIO-based HTTP Connector named Grizzly. See their Webtier page; down a little on the page, you'll see a heading entitled "HTTP Connector." Along with a nice high level diagram, the section describes Grizzly as

Grizzly is an HTTP Listener using Java's NIO technology and implemented entirely in Java. A re-usable NIO based framework that can be used for any HTTP related operations (HTTP Listener/Connector) as well as non-HTTP operations, thus allowing the creation of any type of scalable multi-threaded server.

With NIO, as anyone who has ever used the Unix select system call, you can have a single thread operating on a bunch of connections instead of one thread per connection. This can save tremendous overhead and can achieve a surprisingly high degree of performance. See the old single-threaded HTTP servers Boa and Tiny httpd for evidence that this is not a new concept.

To understand why Grizzly helps in one piece of the COMET puzzle, we look at Jean-Francois Arcand's blog entry, Can a Grizzly run faster than a Coyote. He ran ApacheBench against Tomcat and Glassfish and came up with the following result: Tomcat needs 500 threads where Grizzly needs only 10 to handle a large benchmark test.

The results are a little controversial -- nobody likes their app server trashed -- and I don't want to get into a discussion about the relative performance of Tomcat vs Glassfish, but the ability to handle the IO of an application server with a small number of threads bodes well for COMET. COMET, remember, keeps a connection open to the browser while the Servlet is performing some long running calculation or waiting on a message, and that means we'll have more connections open at one time. If we needed one thread per connections, we'd be hosed.

What NIO doesn't solve, however, is the that the waiting Servlet still chews up a thread. As we've mentioned previously, Jetty 6 has a continuation mechanism for Servlets that allows waiting Servlets to give up their threads. That's the other part of the puzzle for COMET. Jetty 6, BTW, also has an NIO HTTP Connector to go with its Servlet Continuations, so this may be the way to go for COMET early adopters.

 

Not There Yet: COMET with Apache and Jetty

I had intended to marry the nice Apache2 event MPM and Jetty 6 with Continuations in order to achieve a thrifty, thread-sparing COMET capable Java app. The idea is that the Event MPM module in apache frees up a thread when an open connection to the browser is snoozing and Jetty frees up a thread when the backend servlet is snoozing. There are two problems, however.

1. The Event MPM module seems to only snooze between requests. With COMET the need to snooze happens during a requests, i.e. we are expecting something to come back from the server.
2. The request handler module -- in this case mod_jk or mod_proxy_ajp -- and not the Event MPM module handles the socket connections to the servlet container. From my reading of the most recent SVN branch, the modules are using blocking I/O and not polling.

It seems there's still a bit of work to be done to make Apache and Jetty do the COMET dance.