Agile Ajax

Mash note: Remember the Milk

You've got to love a Web 2.0 startup manned by a dev team of 2 that manages to add every feature on your wish list just before said feature's omission really starts to bug you. That's the case with me and Remember the Milk, a to-do-list webapp developed by an Australian company and supplemented by awesome new features with astonishing regularity.

Although they're a commercial entity, they've got a beta API that's allowed the development of a handful of really cool mash-ups. They've had iCal integration and a gCal plug-in for ages. They got on the Twitter bandwagon really early. They were one of the first non-Google companies to port their application offline with Gears. Last but not least, they've got a powerful (though still not perfect) user interface. (More on that later.)

A little background: I've been an obsessive to-do-lister since high school. GTD is my mantra. During my years at Microsoft-centric companies, I used Outlook to manage my entire life. But these days I find myself on Windows, 'nix and OS X machines in disparate locations for hours or days at a time. A webapp is clearly the only way to go for my to-do needs. But after years as an Outlook power user, I need something that will slice and dice my many lists (work, play, home, shopping, whatever) with exacting precision.

I gave Apple's iCal a shot, but its list functionality was way too primitive. I need multiple lists, categories, tags, flexible sort criteria - you get the picture. Ta-Da Lists from 37signals was even more stripped-down than iCal - plus I found its interface surprisingly clunky for something developed by a well-regarded Ajax shop. I thought about todo.txt, but I'm not enough of a command-line purist to completely abandon the GUI - even after years of suffering through Outlook's hideous hidden menus. Then, about a year ago, I stumbled on Frank Gruber's Do More: Online To Do Lists Compared. I didn't agree with his conclusions, but at least he gave me several more options to explore. Luckily for me, Remember the Milk was the first one I tried.

There's a lot to love about RTM, especially its UI, so just let me gush for a minute.

• Flexible organization: You can create multiple tabbed lists, apply arbitrary tags to your tasks, and create saved searches based on any criteria.
• Keyboard navigation: Except for a few advanced functions, such as moving items from one list to another, you can do almost anything from the keyboard. Create tasks, set priorities and due dates, apply tags, edit multiple items ... most functions take only a single keystroke.
• Natural language entry: You can set due dates and repeat intervals using some pretty flexible natural language. It ain't perfect, but with a little training the syntax becomes second nature.
• SMS, IM and email reminders: The service can nag you quite effectively via a wide range of communication protocols.
• Email, gCal, Atom and Twitter integration: You can create tasks or lists of tasks using a special email address; add or edit tasks directly from gCal and Yahoo plug-ins; and use Twitter or Atom as syndication services. The public API means that mash-ups and cross-pollination will only proliferate.
• Built-in collaboration capabilities: There are a host of features dedicated to assigning, sharing and collaborating on tasks.
• Serious accessibility: The main app is fast, powerful Ajax all the way, but the mobile version works just as well in a variety of more specialized user-agents.

There are a few things to hate:

• Poor multi-list view: The main Overview page list all of your tasks that are due, overdue or due tomorrow regardless of which list they're on. But it separates the Today, Tomorrow and Overdue tasks into separate tabs. The gCal plug-in does a much better job of showing all of your most important uncompleted tasks in a single, unified view.
• Recurring task weirdness: Delete a completed instance of a recurring task, and you've just deleted the master version of that task. After any previously spawned instances are completed, the task will no longer recur. This is a very strange user experience, especially for folks who like to purge their completed tasks every once in a while.
• Poor sortability: Tasks are sorted by their priority. Period. This shortcoming doesn't account for the fact that a low-priority task that's a week overdue is probably a high-priority task by now.

See the strikethrough text above to see exactly what I meant about the RTM team rolling out your must-have features before the lack thereof becomes too annoying. I've been dying for flexible sortability on RTM since I started using it. But no sooner did I start this post than I noticed an RTM blog post announcing just such a feature. It would be nice if you could set your sort preference globally as well as one list at a time, but that's a minor quibble.

So what can we learn from my ramblings, besides the fact that I really, really love this platform? I think there are a few powerful lessons for developers of any RIA:

• You don't have to sacrifice brawn for the sake of simplicity: Pleasing power users doesn't mean hopelessly confusing everybody else - at least not with smart UxD. (Apple could stand to learn that lesson, but I digress.)
• You don't have to sacrifice accessibility in the name of Ajax: Let your mobile app function as your accessible app, too.
• You don't have to fear Web 2.0 trendiness: Rapid adoption of emerging protocols such as Twitter can only help you differentiate your product and find new users.
• You don't have to lock your app down, even if you're a commercial enterprise: Public APIs - and the cross-pollination they enable - can only strengthen your foothold in the marketplace. In our Googlized world, this one's kind of a *duh*, but it bears repeating.
• You don't have to hide from your users: An active blog and well-maintained user forums are far more powerful marketing tools than terse release notes on Google Code.

I can only speculate as to the future of GTD's business model or exit strategy. Their logo carries the ubiquitous "Beta" tag, and their pages carry no ads. Where's the funding? It seems like somebody's waiting to get snapped up by a big player....

Lord knows it would be nice to have the old  Outlook/Palm OS quintet of email, calendar, contacts, notes and to-do lists available on a single web-based platform; personally, I would prefer that platform to be Google, though I can't really see them acquiring a start-up just for this one capability. Still, it would be a shame if a product as solid as Remember the Milk got lost in the shuffle or edged out by inferior offerings from bigger players. Only time will tell....

JavaFX - Another Ajax Killer

Everyone is trying to kill Ajax. Apollo, Project Flair, Silverlight, on and on and on...but..Ajax...just...won't...die. Sun is giving it another try with it's JavaFX.

JavaFX Script takes advantage of the Java Runtime Environment's (JRE) ubiquity across devices and enables creative professionals to begin building applications based on their current knowledge base. It also uses Java technology's "write once, run anywhere" capability to help realize a future where consumers can access content whenever and wherever on any Java-powered device. JavaFX applications will run on JavaFX Mobile, Sun's software system for mobile devices also previewed at JavaOne, as well as desktop browsers (see separate announcement).

I think it must be something about Silicon Valley. I've been at enough events out there to know that people actually talk like this (as opposed to writing press releases like this). The doodoo gets especially deep around the time of JavaOne.

What is at the heart of JavaFX? A new scripting language "based on Java." Hmmmmm, dynamic languages good, statically typed languages bad. I guess when Richard Stallman embedded Lisp in GNU Emacs, he really started a trend.

Scripting is all the rage these days, with Groovy, BeanShell, Javascript and others filling an important niche. But I have to feel it's just a fad. There are good reasons to use statically typed languages, especially when developing large, complex systems over long periods of time. I just hope this doesn't mean we are going into a scripting craze ("We are so past Java and C#...") only to have the pendulum swing the other way ("We tried writing our system in Perl...never again. No scripting here.").

More on the bestiary of Ajax Killers later.

Technorati : ajax, javafx, javaone

Is It Time To Stop Pushing the Browser Around?

In my first ever real grownup consulting engagement, back in 1990, I was called in to improve the data cleaning operations of a small testing company. This was back in the days when even companies heavily dependent on computer technology relied on a single "computer guy." This particular guy had rigged up a data cleaning system on a set of state-of-the-art 80386 machines. He had set up a folding table in one corner of the office ("Machine room? What's a machine room?") and taped "Do not touch!!!" signs to the monitors. The guy would take the tapes containing maybe 20-30MB of data and run then for 6 days through his cleaning environment. If something blew up or a new defect was identified in the source data, boom! another 6 days.

When I probed a little bit about what he was using to perform this magic, I found out, to my horror, that he had implemented the whole data cleaning shebang as a set of WordPerfect macros! Ouch! Talk about the wrong tool. I recommended they buy a little Sparcstation (sorry, x86 unix was kind of expensive and unreliable back then) and solved their problem with a modest amount of Perl4.'

So what does WordPerfect have to do with Javascript in the Browser? As people have been testing the limits of Ajax, we see more sophisticated SOA and messaging concepts and abstractions making their way into the picture. Coach Wei, a clever and insightful man, has an article up now on the Nexaweb's IMB (Internet Messaging Bus).

It's a clever concept of how to introduce MOM/ESB into a browser environment (Nexaweb also supports Java and Desktop clients, as you can see in the article above). I've implemented a number of distributed apps using JMS as well as a few using ESB's such as WebMethods, Tibco and Mule. It's an eleagant way to design apps, but requires a somewhat different approach and outlook than OOP (not that you can't include OO software in the mix as a component). See Enterprise SOA: Service-Oriented Architecture Best Practices for a pretty good read on what's involved in building SOA systems.

I checked out a few of the applications mentioned in his article, and they we slick, responsive, elegant. Nice work by the Nexaweb folks.

So again, what does this have to do with WordPerfect macros? Using a technology in an inappropriate way will come to bite you. All that infrastructure and encapsulation and tunneling comes at a price, espectially in a still somewhat wobbly execution environment like Javascript on the browser. Sure enough, those Nexaweb apps start out nice, but over time start to peg my CPU at 100% on both IE and Firefox. Don't even think of running more than one of these apps on your machine at one time. Maybe you can put a "Do Not Touch!!!" graphic over all of the other application icons on your desktop. ;-) I don't think this is Nexaweb's fault. Javascript in the browser still has a long way to come before you can write truly reliable, complex and layered systems.

Further, lots of these boundary pushing technologies make use of characteristics of the browser that are more side-effects than features. Look at Comet, for example. Common Comet techniques such as keeping the response from the server to the browser open (in an Iframe, etc.) or streaming a multipart HTTP message via XHR don't work the same in all browsers, are not part of any browser requirements, and could change with the next release. This is no way to implement serious applications.

Now I'm not saying that Comet is a bad idea, or that Nexaweb is peddling SOA/MOM junkware. Quite the contrary. I like what I see and hope to see more. What I object to is that all of this is being done via the back door. If we are going to implement fancy communication layers in the browser, let's do it by the front door, by requesting a more robust Javascript execution environment perhaps with threading and better networking control built into the browser.

Today's browsers are not that far removed from the Netscape Navigator with Livescript (later renamed to Javascript) that I used in 1995 to toggle a few heirarchical checkboxes. We are asking them to do things they were not designed to do. As we push the envelope of what a web app is and can do, we should do so in a smart way. Otherwise we are just like that guy who took WordPerfect macros -- a tool designed to perform simple textual and menu operations -- and wrote an ETL system. Yes, it can be done, but why would you?

Technorati : ajax, coach wei, esb, jms, mom, nexaweb, soa

Application Watch - Jotlet

OK, maybe there are too many Ajax calendars out there. Jotlet stands apart, in my opinion, in terms of it's usability and simplicity. With Ajax, there is the temptation to introduce too many whizbang behaviors, and this Calendar doesn't fall into that trap. The interface is simple and uncluttered.

It's missing some features, like exporting to Outlook, and I'm not necessarily recommending this calendar over Google, etc.. I'm offering it as an object lesson in how less is more in interfaces.

Google Pitching Services to Small and Medium Sized Businesses

This has been such a rolling press release that I'm not sure it qualifies as news anymore, but it's in the New York Times so it must be news. Google announced today that it is providing a set of hosted applications for small to medium sized businesses. The beta service will be free for now, and the premium service is under development.

What comes with the application suite? From the overview FAQ:

You can currently choose from Google Mail, Google Talk, Google Calendar, and Google Page Creator. Also, you'll soon be able to add a Personalized Start Page for your domain.

I think everyone knew that this was coming, and there's been speculation about Google offering its office-like applications -- including Writely and Spreadsheet -- as a pay service, for several months. The last two paragraphs of the Times article, however, point toward these applications being offered as appliance-based software to larger companies.

Providing technology to corporations and large organizations accounts for less than 2 percent of Google's revenue, but the business is increasingly critical, Mr. Girouard said. Most of that involves selling "server appliances," large computers that take on the job of conducting searches of large databases and company records.

"We are a very small part of Google's overall business, but we're growing quickly," he said.

If Google starts to cut into Microsoft's market share, this could prove the software as a service (SaaS) business model and might trigger a land rush by online companies into areas heretofore the exclusive domain of desktop application vendors.

Update: just saw this Red Herring article on the 17 competitors to MS Office. A thorough article that covers more than just the usual handful of Web 2.0 startups, it is well worth reading if for nothing else than these sorts of heartwarming quotes:

WriteBoard can seem almost bare-bones in its features, but Mr. Fried is betting on simplicity. "The problem with the traditional software industry is that they have to bloat their products," he said. "They have to add more and more so they can get more money out of their users next year, but we don't want to follow that model."

Yes. Bring back the simple 64k application. My Commodore 64 is still somewhere in a closet in my parents' house. ;-)

Technorati : ajax, google, microsoft, saas

Blummy - Bookmarklets with A Whole Lot of Ajax

Another way to add Ajax functionality to a third party web page is a bookmarklet -- little snippets of Javascript that can modify the current page. Blummy takes this concept a step farther by allowing you to combine many different bookmarklets into a single dropdown, saving precious space on your browser toolbar.

You can drag and drop "blumlets," the individual bookmarklets that Blummy aggregates, into your drop down via the blummy.You can even write your own blumlets, if the madness takes you. As one might expect, many of the blumlets available are one for social bookmarking sites.All of your configuration information is stored with the blummy service, so in theory you can carry your blummy configuration to a different browser.

Technorati : ajax, bjax, blummy, bookmarklets

Application Watch - Noovo Home Page Editor

Noovo has a preview of their new Ajax home page editor available. From their blog:

Noovo is an Ajax-based web application for homepage editing and we will also provide hosted homepage service as well. During the demo phase, only main features of editor are available. You may get a hint as to what kind of service we have in product plan if you look at the top menu. Those features for hosted homepage service will be introduced here before long.

The fact that there are no user interface or usability guidelines in the Ajax world is apparent from the claustrophobic toolbar and somewhat unusual behavior of some of the controls. So, powerful but clunky.

Technorati : ajax, application, home page editor

Application Watch - Kool, a Meebo IM Competitor

Via TechCrunch, it's getting crowded in some of these AJAX application categories. For a while, Meebo was the sole player in the AJAX IM arena, with only local chat apps playing at its ankles. No longer. A new competitor is entering the fray: Kool IM. While I was napping, a number of others have also crept up on Meebo:

In addition to Silicon Valley based Meebo and Amsterdam based eBuddy, ILoveIM, Germany based Mabber and other competitors are out there vying for the same user base. Luckily, users tend to rack up a lot of time (if not page views) with the service, allowing the companies to generate reasonable revenue from advertising - eBuddy claims to be profitable.

"Too early is the same as wrong," as the old Venture Capital maxim goes. I guess all these competitors are validating Meebo's business model, just as they're getting ready to eat it's lunch.

RSS and Social Bookmarking - The Drosophila of Web 2.0

Via Micro Persuasion, Earthlink has launched an RSS reader and a social bookmarking app. Steve correctly points out that both Earthlink and AOL cannot be left behind when their users are leaking to other sites. Still, is a me too strategy sufficient in the fast moving world of Web 2.0? Has Google changed the name of the game from "distribution and content" to "applications, baby!"?

Top 10 AJAX Applications - How Have Things Changed?

"Too early is the same thing as wrong." -- Old Venture Capitalist Proverb

Most Venture Capitalist (VC) proverbs are actually stolen from investment banking, but I think this one is actually home grown. (Full disclosure: my wife is a VC and I count many VC's among my friends and acquaintances.) I thought it would be instructive and amusing to look at the predictions of one VC and see if he was right, too early, or just plain wrong.

Dan Grossman of Venrock was kind enought to publish a top ten list of AJAX apps last September. I would classify the picks in his list into four categories:

1. More or less useless "Oh, cool, it's AJAX type apps." Amazon Zuggest is probably foremost among them, but the periodic table is also showing it's age. Any kid with Dojo and a few minutes on their hands could whip that up now.
2. Desktop style applications that are enabled by AJAX technology. These are Kiko, Writely and Time Tracker. Of course Google's Calendar app has turned off the oxygen for Kiko. Look for Writely and it's pygmy competitors to suffer the same fate shortly.
3. Improvements on existing apps -- portals, news readers, etc. -- through AJAX technology. Everyone has a portal and a news reader, so slathering on the AJAX was pretty easy. Now everyone has an AJAX portal and RSS reader -- Pageflakes and Bloglines come to mind.
4. Proto Web 2.0 applications. These combine the usability enhancements of AJAX with collaboration and network effects. Backpack is probably the best of these. I'm not sure whether to put Del.icio.us Director into this category or not. Social bookmarking sites like Digg certainly seem to dominate this category right now.

I think if I put together a list of my top ten AJAX applications, two things would be true. First, at least four of those applications would be from Google. Second, in one years time, at least four of those applications would still be from Google, but everything else on the list would have changed. Maybe "AJAX" is not a specific enough category on which to base a top 10 list. (Should we say Web 2.0 and then proceed to define what Web 2.0 means?)

Thanks to Dan for sticking out his neck and making the list. I'm not sure any of these applications other than Backpack make it onto the list now. What does your list look like?

Eyespot - AJAX App for Video Editing

We've see AJAX bringing typical desktop applications like digital image editing (ala Photoshop) to the web. Video editing may seem like an upload too far, but as long as you keep the upload clips small, you can do some useful things with the Beta release of Eyespot. You can upload your clips, tag them, search for other clips. Trim them, assemble them via drag and drop with other clips into a timeline, apply effects and transitions, add a soundtrack and add title slides. You won't be able to edit The Godfather with this thing, but for some quick editing of a camera phone clip, it does the trick.

The application is a mix of multi-page and single-page, i.e., each major function -- selecting a clip, editing a clip, etc. -- is a seperate page, while actions within that page happen in a single-page AJAX manner. Under the hood, the application seems to be based on Rico, a venerable AJAX framework that I've neglected so far (I guess I figured everyone already knew about it).

Firefox Extensions and AJAX - A Model for Web 2.0?

Let's face it, cross site AJAX is a real pain. If you're going to provide your components as a service, sort of like Google maps, you're task is not impossible yet still challenging. If you want to mash up your app with other services at the browser level, you've got to proxy those services. Put any way, it's a pain. How to simplify creating truly powerful, cross-site AJAX apps?

Greasemonkey showed the way, demonstrating how you could manipulate pages with Javascript. Book Burro was one of the first AJAX/Greasemonkey hybrids that allowed you to search for a book at your favorite online book merchant, then click on a semi-transparent drop down menu and see book prices at competing retailers. The magic of AJAX! Book Burro has since made the transition from Greasemonkey script to Firefox extension in its own right.

Another example of this curious Firefox Extension/AJAX hybrid caught my eye the other day: MyStickies. This extension will allow you to insert AJAX-DIV stickies onto any web page. The extension handles this insertion and also includes a Javascript library via a Javascript file from mystickies.com and another via chrome://mystickies/ for the behavior.

The sticky location is persisted to the mystickies site where you can also use a webapp to search, manage, etc., your stickies. It's a direct manipulation bookmark that doesn't reside just on your local computer.

Now I happen to think mystickies is pretty cool, especially if they implement some of the features on their roadmaps, like sticky sharing (how web2.0!). But what interests me here is the application architecture. Firefox Extension plus AJAX = a service that can be applied across other web sites. There is clearly a limit to this opportunity -- you have to really like a service and truly, how many of us are willing to keep 20 extensions running, each of which hogs a piece of browser toolbar realestate? Many of these players are already planning IE, Safari and Opera ports.

What shall we call this thing? Browser Extensions and AJAX? BEJAX?

More Killer App - OpenRecord, a Wiki/Spreadsheet

I've been promoting the idea of the collaborative spreadsheet as the killer app for AJAX for a while now. First there was WikiCalc, then Google spreadsheet. The folks over at the Dojo Foundation -- yes, the Dojo Toolkit folks -- have been working on a Wiki app that fits into this category as well. From their about page:

The goal of the project is to create an open source software product. The product we're making is a wiki engine, but where most wiki engines are geared toward pages of text, OpenRecord is geared toward loosely-structured database content.

It's still in an early development stage, but you can take a peek at some prototypes here. Click around to the various pages and try adding and editing cells and rows.

Speaking from experience, these types of applications will need professional support if they are to make a dent in the corporate world. Look for the document and knowledge management folks (OpenText, EMC/Documentum, etc.) to start introducing these "Office Lite" functions in their products.

GWT Libraries Released - XML, SOAP, Crypto, Amazon, etc.

Openfount has released a beta of a set of libraries and tools for use with GWT. The web services parts of these work in conjunction with their queued server. This queued server basically acts as a proxy for XMLHttpRequest's. So, if you want to access web services from Amazon but you're loading your Javascript from another server, you can proxy the request through your server. There's a bit more to it, but that's the jist of it.

Looking at their Open Source license, I'm not sure it smells that good, but IANAL.

Update 1: I've had a closer look and have even more second (or should I say third) thoughts on this package.

Open Laszlo DHTML Milestone

Ever since Laszlo Systems announced that they would be targeting DHTML as well as Flash, I've been peeking into their source repository from time to time, hoping to catch a glimpse of the elusive "Legals" (horrible name), the runtime that supports DHTML. Well, this past Thursday, on the Open Laszlo Project blog, they announced a milestone release of this mythical beast. From the entry:

Late last week, the OpenLaszlo project reached a huge milestone: release of the first source snapshot of our multiple-runtimes architecture, code named "Legals". The purpose of this snapshot is to deliver infrastructure, tools, and architecture sufficient to allow broad community participation in the project.

We began Legals back in January because we felt it was finally time to invest in OpenLaszlo's potential as a multi-runtime application framework. Adobe had released an initial beta of Flash 9 (then called Flash 8.5), and it was clear to us that it would be essentially an entirely new VM: new bytecode set, many improvements to the ActionScript language, and substantially revised APIs. In order to support Flash 9 we would need to build a new compiler backend and new runtime libraries.

If you're viewing an lzx file from the OpenLaszlo server, you'll need to tack a parameter onto it to get it to display the DHTML, i.e. test.lzx?lzr=dhtml.

So far, I've only been able to get it to display two text components. Adding in a button or anything else gives you a big nothing. Definitely, as it says, a work in progress on the DHTML side.

OpenLaszlo Developments Afoot

One way that AJAX frameworks are going to be embraced is by providing lots of widgets. Another is by signing up other companies to develop applications using their framework. Nothing spells credibility than working software. To that end Laszlo Systems has announced that someone's going to be adding more oomph to their email software product built using OpenLaszlo:

Laszlo Systems, developer of OpenLaszlo, the leading advanced open source platform for building and deploying Ajax applications, and Goodmail Systems, creator of the CertifiedEmail system for trusted email, today announced that Laszlo Mail will support the recognition and presentation of CertifiedEmail messages. Offered for license to businesses and communications service providers, Laszlo Mail delivers the functionality and responsiveness of desktop email without requiring any client software installation. Now with CertifiedEmail, Laszlo Mail will have the added benefit of protecting users from phishing and other forms of email-based fraud.

Hmm, the former Flash product is now "the leading advanced open source platform for building and deploying Ajax applications." Last time I looked at their source tree it didn't do DHTML yet.

The Hazards of Exposing Business Logic on the Client

Via Ajaxian we get an object lesson in the dangers of exposing business logic in the browser:

Beau Hartshorne of Snipshot (formerly Pixoh) says "massive chunks" of Cellsea code are identical to Snipshot. "This is not an accidental inspiration. Check out the cropping code, the resizing code, and so on. We've also noticed that portions of their website are also stolen directly from ours ... We are contributors to MochiKit open source project. However, the code in question is proprietary and was taken directly from out site."

Can I say "I told you so?" I've blogged about the danger of Ajax and Leaky Business Logic before. What is the danger here and the lesson the be learned?

• Don't put your crown jewels in the browser. See if you can't lock a fair bit of your business logic on the server side by using a server-side component framework like Echo2 or ZK.
• If you're going to deploy meaningful applications in Javascript, do what people have been doing with script source code for decades: obfuscate. You can do something simple like renaming all of your variable to one or two character names, or you can use a code generation framework like GWT or Morfik that produce unreadable code from the beginning. Be forewarned that code generators can be vulnerable to decompilers -- in short, if someone knows how GWT generates it's code, you could possibly reverse the process and produce the original source code.
• Build in anti-theft mechanisms. This could be something as simple as a method that checks to see that the url the application is running on is the correct one, otherwise display a nasty message. You could make this as tricky and complicated as you like, all the way to encrypting big chunks of code with the website url and only decrypting them at runtime.
• Hold out for "byte compiled" Javascript.

A combination of all of these may be what we end up with. I'm not sure I'd want to run any script in my browser, though, that I can't understand. Still, you would hate to be the victim of code-theft as has apparently happened to Snipshot.

How similar are these two programs? Well, Cellsea offers a bunch more functions, but they do look very similar, both in terms of the UI and the underlying code. The original Snipshot is here and the knock off here. You be the judge.

If you are really hungry for a good AJAX image editing app, the best of the bunch of them may be Phixr, which gives you preview, the ability to marquee select for certain operations, etc. Slick, even if the UI is a bit jumpy.

An Interview with ZK Creator Tom Yeh

Anyone who has been following the development of AJAX frameworks has heard of ZK, the server-side component GUI framework that allows you to write applications like you would a desktop app. It has become one of the most popular projects on sourceforge and has been nominated for a 2006 Sourceforge.net Community Choice Award.

Recently I talked with one of ZK's creators, Tom Yeh of Potix, about the Framework's popularity and future.

ZK Background

DK: How did you come up with the idea for ZK, i.e. an AJAX framework that allows you to write webapps like you would a desktop app?

TY: As I mentioned in http://zk1.sourceforge.net/faq.html#Why, it is the result of frustration.

I was a fan of thin-client computing since leading a wonderful team to develop Network Computer and Window Terminal in 1995. I truly believe in the so-called utility computing. Accessing applications should be as simple as using tap water.

It is crazy that someone should carry tons of water when traveling, since tap water is available everywhere. However, we still travel with notebooks, even though Internet connectivity is everywhere. Similarly, the Web edition of MS Outlook has been available for years, but we are still using the desktop version. Why? Frustrated user experiences and excessive development costs. In other words, it is too costly to develop a Web UI from scratch or add-on to 'legacy' apps, and people won't use the Web UI even if it is provided.

After trying different ways to apply Ajax in the projects on which we consulted, we found that applying Ajax at the client side, as most frameworks did, only solved the half of the problem -- though it did result in a lovely interface. That is why ZK was born.

DK: Who is Potix?

TY: Potix is a consulting firm providing expertise in Web development and project management. We also develop applications on an ODM-basis. ZK is the consequence of this work. However, due to strong demand, we mainly focus on ZK now.

Potix is also a house of old dogs. Most of us have more than 15 year of experience, ranging from developing Windows/Linux/Web applications, to embedded OS and GUI frameworks. ZK is my second OSS project; the first one, called OpenPam (GUI for embedded devices), was not very popular.

DK: How are you planning on making money with ZK? Nextapp, the makers of Echo2, sells an Eclipse plugin for around $400. Is that a business model you are examining?

TY: Dual license (as MySQL did). You might also take a look at one of my posts: http://sourceforge.net/forum/message.php?msg_id=3605162

Other Frameworks

DK: Echo2 seems to be the only other server-side AJAX framework out there. How would you compare yourself with Echo2?

TY: Echo2 assumes UI designers are Swing programmers, while ZK assumes many of them are not programmers.

Markup languages, like HTML, XUL and XAML, scripting languages, like PHP and Ruby, and the object oriented approach are the three most important developments since GUI was introduced. Unfortunately, Echo2, WebOnSwing and similar frameworks only focus on the object-oriented approach.

In addition to BeanShell, we are looking at the possibility of using Ruby and PHP in zscript. It looks to be, so far, quite feasible.

DK: There's lots of talk now about the Google Web Toolkit. Do you see it as a competitor to ZK or a complement?

TY: Both.

From a technological viewpoint, it is a complement. GWT is a client-side solution and quite good for developing Ajax components. On the other hand, it is never a good idea to replicate the business logic to the client, which eventually brings us back to the maintenance headache of fat clients. In addition, loading huge JavaScript files into the client and executing them there is not fun at all. At the end, you need a server-side solution to handle the business logic and presentation tier.

It would be great if we can leverage GWT's power to simplify the effort of component development (as we did with DOJO and FCKeditor).

From a perception level, users might see it as a competitor. Google is now at a sweet spot with plenty of resources and a good reputation, as Microsoft was in the 1980s. Developers love to explore any kit Google puts out there and exaggerate what it can do as if Google was the first to invent it.

DK: Do you envision developers writing ZK components using GWT?

TY: Sure, GWT does a good job for non-JavaScript programmers who want to develop Ajax components. I expect that some results of this will show up in next few months.

However, what makes Ajax programming difficult is not JavaScript itself. Rather, it is the incompatible and buggy API (to communicate with DOM). The ability to do Java-to-JavaScript is important, but I am also hoping for some benefits from the standards efforts of OpenAjax.

The true value of ZK is its architecture. We look forward to adapting components to ZK as decent (client-side) Ajax components become available.

DK: What other AJAX tools, frameworks or technologies do you think are noteworthy or interesting?

TY: DOJO has an amazing set of components, though it is too heavy (one of the reason we didn't build the ZK Client Engine upon it). Atlas has great integration with Visual Studio. Anyone who wants to provide an authoring tool should take it as an example. Google Spreadsheets is an excellent Ajax application. You should unplug the network connection and see how it reacts, though. It's cool nonetheless.

Strengths and Weaknesses

DK: What would you say are the strengths and weaknesses of the ZK framework?

TY: Strengths: boundless. But seriously, great richness and excellent productivity, if we can put in a sentence.

Many of ZK users appreciate that they can design a rich Web application without programming. Many have thanked us for the intuitive event model and feature rich components that have saved them countless hours. Many enjoy the power of prototyping so they can change the look and function right in front of their customers. For more information, you could take a look at the executive summary (http://zk1.sourceforge.net/wp/ZK-exesum.pdf).

Weaknesses:

1. Stops working if the connection is broken. It would be better if a subset of functions still worked (such as read only viewing).
2. Round-trip latency, though hardly observable. To minimize the effect of this, we plan to, in addition to Client Side Action, add visual effects that execute solely at the client and notify the server only when necessary.

One other weakness shared by all HTTP-based solutions is the lack of Server-side push. Though someone has implemented so-called "reverse Ajax", it introduces too much overhead on the server. This -- server-side push -- is the next important step beyond Ajax.

DK: There aren't any production applications out there using ZK. Do you see that as a problem and, if so, what are you doing to address it?

TY: According to the profile of our customers, adopting ZK is still in the development (and evaluation) phase. By and large, adopting Ajax for most companies is still in an early stage. It is just a matter of time. I'm not really that concerned about it as long as the growth of ZK's acceptance is strong.

DK: Right now, the framework doesn't seem to allow for easy customization of the look and feel (window color, font, etc.). First, is that correct and, if so, do you have any plans to address it?

TY: Not correct. The look and feel are well separated by CSS and molds. What prevents users from customizing the look is the lack of documentation and samples. Also, the customization of the sophisticated components is sometimes not intuitive. It forces users to look at DSP files (the templates used to generate HTML tags). We realize this is an issue and we will provide documentation in the Developer Reference guide.

DK: I've noticed in the code that you spawn another thread to handle events. Can you explain the design to me? Also, how does this fit in with the Servlet standard and doesn't this make it difficult to cluster a ZK application if the user session has non-serializable information and context?

TY: It is common that an application has to confirm with users about, say, whether to delete a file. It is unbelievably costly for Web developers to implement such functions. Why? Users get no response until the servlet has completed, while the servlet requires user's confirmation to decide whether to proceed further.

There are many solutions to this problem, but the most elegant one is modal dialogs. ZK is one of the first Web framework that really enables the server-side modal dialog. The magic is to process events in another thread. Then, users can suspend and resume the processing anytime they like -- including but not limited to modal dialogs.

It does make clustering difficult if there is a pending thread in a session. We are working on the serialization issue now. A basic idea is to send the application a notification, and it can decide how to handle pending threads. Of course, if there is no pending thread, there is no notification at all.

DK: What is your favorite ZK feature?

TY: Macro components, the threading models, and zscript.

DK: Are there some cases where you wouldn't use ZK to develop an AJAX-based web application?

TY: Hard to imagine unless clients want the application to be functional even if the connection is off. Oh, action games, but it might not be a good idea to use Ajax there at all.

DK: It doesn't appear that you are using JUnit in the ZK code. Is this true? And if so, why?

TY: Only common utilities are tested with JUnit (refer to pxcommonTest in SVN). I am still thinking about which is the best way to test ZK components. We will come out with something in the near future.

Future Plans

DK: What is the biggest feature enhancement request you are getting from the users of ZK?

TY: Safari support and design patterns. By design patterns I mean information on how to really write an application with ZK. An illustration with Pet Store is a common request.

DK: On your road map you mention plans to provide a mobile version of ZK. First, what is the time frame for this? Second, how are you planning to do this architecturally? Can users write one application and dynamically re-target their apps to a different front end?

TY: We originally planned to provide a J2ME-based client engine, but we are keen on developing a Flash-based solution now. Architecturally, we have to do three things to support a new client: client engine, server engine and components. Users have to do two things accordingly: use a different set of components and adjust their UI to fit into small screen. We will keep the component API very similar (and also provide common interfaces that make developer's code polymorphic).

It is possible to let users use the same set of components, where components would be smart enough to detect the client and behave differently. However, we're not considering this approach because it makes the development and maintenance of components too complicated. Of course, we can use the factory pattern to chose the correct implementation, but it makes the instantiation of a component too complex for most users.

DK: You've stated that ZK is at the interface layer. Do you plan to introduce components that punch through to the data layer, sort of like the .NET datagrid?

TY: Yes. We will add more data-layer utilities, such as zkplus's DelegatingVariableResolver, too. After all, most ZK applications will have to access a database. However, we may not do a similar deep integration as Ruby on Rails did -- I don't like to make so many assumptions about how applications might be built.

DK: What do you see as the major challenges facing the AJAX community today?

TY: There are too many frameworks, such that many users prefer to wait. There isn't even a well-recognized categorization for them (such as server vs. client). A comparison chart or something similar would be very useful.

A standard like OpenAjax will help (at the client side), but I've never seen an alliance really work.

DK: Care to share any other plans you have for ZK?

TY: We are interested in adding mega-components, such as spreadsheet, forum and wiki. Part of the success of PHP is because the availability of plenty of off-of-shelf stuff. We won't develop all of these components ourselves. Rather, we prefer to help other developers deliver them either as OSS or commercial components. We will have a ZK.forge website to promote such collaborations.

ZK - Documentation & Tools

Documentation

The quality of the ZK documentation is very high by Open Source standards. The documentation includes:

• A 13 page PDF Quickstart guide - shows how to set up the demo application and a hello world app.
• A 169 page PDF Developer's Guide - steps you through the ZK framework with lots of small examples illustrating components and concepts.
• A 43 page PDF Developer's Reference. Documents component properties and behaviors. This document is only about 40% complete.
• A 1 page PDF Executive Overview.
• A 16 page PDF Product Overview - gives information on the motivations behind ZK and the architecture of the product.

The reference guide will need to be completed, otherwise developers will be guessing at specifics of components and the ZUML. Also, the documentation on CSS and "molds," i.e. the templates for components, is spotty. This will make it difficult for developers to change the look and feel in any significant way.

Tutorials

While there is a ZK demo application that shows off various components, there appear to be no tutorials or example/reference applications available to the would-be ZK developer. There is a wiki with a how-to/cookbook page, but just like the developer's guide, it consists of short examples, not a substantial application. There is also a port of a struts application over to ZK, but it seems to make use of so little of the AJAX capability of the framework that it can't really be considered a tutorial or reference app. For now the forums are probably the best source of information and support. Also, there is a "small talks" section of the project site that has some bits of information on integrating ZK with things like the Spring Framework.

Usability

Overall the framework is a joy to use. The ZUML makes it easy to do quick iterations -- edit, test, edit, etc. -- without a long compilation step. The ZUML is also easy enough so that non-programmers can compose or modify a UI. Exposing effects such as drag-and-drop, async update, etc., as components or attributes further eases the development of complex user interfaces for non-programmers and programmers alike.

Tools

As of this writing there is no IDE integration for ZK. This is really a crying shame, since the ZUML makes a WYSIWYG UI layout tool a natural fit.

Product Watch - Nusearch.com and Zohoshow

 

Two new AJAX flavored products launched today: Nusearch.com, a search engine with AJAX whizbang, and Zohoshow, the last piece in the Zoho Office Suite, a replacement for MS Office that includes Zohowriter.

As of this writing, it looks that Nusearch, with all it's AJAX innovations, has succumbed to the first day onslaught. I'll try and update this post when I manage to get through.

As for Zohoshow, it's basic, allows you to import existing Powerpoint files. Don't expect to do any charts or graphs or animations, though.

I've said before that presentations are just about the last thing for which I would want to depend on network connectivity. How many conference rooms have you been in that didn't give you access to the outside world?

Update 1: Nusearch came back up. Yes, there's some AJAX there, but in my opinion it's rather annoying. The search engine will let you preview their cached version of a page just by mousing over it's entry. It's so twitchy, though, because the table rows are close together. Also, the preview is really no better than a frame or iframe. Snooze. From a pure search perspective, it doesn't seem to do a very good job of sifting and ranking. The search results have a somewhat spammy feel to them.

 

I like dzone's little popup screenshot much better.

Ajax Forms - Enhancing the user experience

The web is filling up quickly with examples of Ajax powered sites that allow us to do things we’ve never been able to do before on the web. From note taking apps, to spreadsheets, photo editors, and online games, Ajax apps like these are continually pushing the boundaries of what is possible, and in the process changing the way we work live and play on the internet.

However, we can also use Ajax to build better user experiences for the tasks we’ve already been performing on the web.  One kind of task crying out for Ajax attention is form submittal, in which a user fills in a set of fields and presses the submit button. 

Almost all forms today use the POST action in the SUBMIT button to upload data to the server, where processing gets done, and an entirely new page is sent back to the user.  Depending on the results, this might be a “Thank You” message, or it could specify an error in the form submission (more likely when the form is lengthy).  Either way, the experience is a poor one. 

First, the navigational requirements are increased, as at least two pages are required-one for form and one for result.  The result page usually contains no more that a couple of sentences displaying the results of the form submission, and possibly instructions on what next steps to take.

Second, the user has to wait until the end of the form submittal process to receive any error messaging.  This increases the likelihood that there will be unnecessary typing and mouse movement/clicking, since incorrectly entered information into one input field may require a user to reenter fields down the entire form. 

By using Ajax to asynchronously send the information to a server for processing, form entry and submission can be much more quick and painless. 

Information can be uploaded processed and returned as the user enters data into an input field.  Since not all the input needs to be sent at one time upon submit button on click, the user is told instantly, say, that the username he just entered is already taken.   

Once all the required form data is submitted, the result can be displayed on the same page, allowing the user to move on to other, more enjoyable, tasks more quickly.  If the form is small, and shares screen real estate with other content on one page, the user can continue on that page after the form is successfully submitted.  No need to hit the back button.

Here is an example of what can be done to build a better Newsletter Signup form.

Ajax continually provides us with new and exciting web based products and services.  But is also is a powerful tool for restructuring the way information is presented and manipulated on the web.

MXGraph - Direct Manipulation Graph Component

Via Ajaxian: JGraph has release MXGraph, a direct manipulation visio-like beast. The web site says it's a thin-client component on top of JGraph (if you're not familiar with it, it is a -- shocking -- Java desktop visio-like beast). It has some nice things in it, such as a nice marquee selection tool and draggable everything. There are some small defects, like drag from the pallette not working quite right. Still, a very impressive piece of code (even if the code is obfuscated).

JGraph is open source, but some of the cooler stuff, like advance automatic layout, comes in the commercial Pro version. It'll be interesting to see what the licensing terms for the MXGraph component ends up being.

Max Does it Again - 50 Great AJAX Reference Sites

Max Kiesler continues his list making. His latest? 50 AJAX Reference Websites From Around the World. He doesn't just restrict himself to English; Italian, Japanese, German, Korean and Spanish sites are also listed. Watch the comments as people chime in with their own worthy sites.

Google Spreadsheet Available for Limited Test

Holy collaborative spreadsheets Batman! Looks like Google is getting on the killer app bandwagon with their new spreadsheet offering. It's only available in their lab environment to select invities. From the announcement and the google blog post, it looks exciting. I especially liked this bit:

Real-time collaboration: Google Spreadsheets can be shared, updated and edited by several users at the same time, in real-time, saving users from the hassle of manually consolidating multiple spreadsheets from others. Users can also chat while editing or viewing the same spreadsheet and control who may edit and view their shared spreadsheets by listing the specific people by email address.

This has me feeling almost prophetic. If google comes out with a private label version, I may never use Excel again.

Quick Update: Collaborative Spreadsheet

Ajaxian alerts us that Jotspot has partnered with Salesforce.com to produce a collaborative spreadsheet-like application.

Check out the inner popups, formulas, security selections, etc. This really shows how you can build an application that looks like a desktop app but also has the mashup ability shown off too.

In this case the Salesforce.com application provides the backend model for the collaborative spreadsheet application. It does more than just provide a spreadsheet, however. It includes task tracking and calendaring.

What are Jotspot's plans for the application going forward? From the original article:

The salesforce.com/Tracker partnership is a good example of Jotspot's strategy to "embrace and extend Microsoft", which Jotspot CEO Joe Kraus and I spoke about previously in my post in March. With JotSpot Tracker integrated into salesforce.com, there is no need to download Excel files to the desktop - because everything is available on the Web via the Salesforce service. Which of course is one of the steps towards a Web Office suite!

Embrace Microsoft? That had better be a massive bearhug. Good luck.

 

AJAX OS's

We saw an announcement a few days ago of AjaxOS, an application suite bundled in an AJAX "desktop." So now we can all use productivity apps online and pretend we are using our own computers. It turns out that the concept of an AjaxOS isn't exactly new. Over at webbys.world we have a more extensive overview of other AjaxOS's. The best of these seems to be eyeOS.

I get a kick out of opening another AjaxOS in the eyeOS web browser. Down the rabbit hole.

Collaborative Spreadsheets: The Killer AJAX App?

Management abhors a vacuum. That's why when there are no workable information systems in place, spreadsheets and email chains spring up to fill the gap. Anyone who has tried to tame one of these organic sneaker-nets knows how hard they are to uproot.

One of the reasons they are so resilient is that they work pretty well for the users involved. The user interface is familiar to all of them, and writing a spreadsheet is a heck of a lot easier than writing reports in crystal and entering data in a disconnected and unwieldy forms interface.

The downsides for the larger business are, of course, the reason why senior management typically tries to eradicate theses ad hoc systems is that their business data is stuck, unversioned and unverified, in employees inboxes and file-systems. They break down as they grow and as spreadsheets are invariably modified, often in incompatible directions, in response to changing and growing business requirements.

The response is to implement a variety of replacement solutions: Knowledge Management Systems, data entry and reporting solutions, work-flow and OPM systems. These solutions tend to be expensive, slow to implements, and ultimately unsatisfying replacements.

Why not embrace the spreadsheet and integrate it with the KM/work-flow/reporting systems? Collaborative spreadsheets, with roll-ups, entitlements, work-flow -- the killer App of the AJAX era? There are already a few AJAX spreadsheets -- Num Sum has a good one -- and in a blast from the past, Dan Briklin, one of the creators of VisiCalc, is working on WikiCalc. Don't get too excited. WikiCalc doesn't have any of those necessary things -- work-flow, roll-ups, reporting -- that would make this a killer app.

Bit even if the perfect solution comes along in a few months, that's no guarantee of success. One secret to implementing information systems that are successfully adopted is this: eliminate the competition. In the case of a collaborative spreadsheet system this means eliminating email attachments, desktop spreadsheet applications, shared file-stores, etc. The AJAX collaborative spreadsheet may be a little ways away from being able to replace the current desktop solutions, but their time may come.

 

At any rate, these office apps are unlikely to be Microsoft's Pearl Harbor or anyone elses.

Google Calendar: Not As Fat as Other Ajax Apps

Again a non-scientific look at an Ajax application and whether it delivers on the "many smaller requests" advantage of Ajax. Last time we looked at the old and new Yahoo Mail. There the traffic looked pretty bulky for Ajax, i.e. the new Yahoo Ajax Mail was sending large amounts of data with XHR and making more requests than the old non-Ajax app.

This time we don't have an old app to compare it to, and yes it's apples and oranges, i.e. we're talking about calendaring, not email, so the information sent back and forth is likely to be smaller. Again, the disclaimer: not scientific. Just accessing the app and trying to excercise as many of the features as possible. Here are the numbers:

(The count is the number of hits. All the other figures are in bytes. The figures are for response data sent from the server and doesn't include the size of the request data sent from the browser.)

                               

Count	93
Average	4393.108
Median	110
10th	44.2
25th	66
75th	581.5
90th	2385.6
Max	331424

At first glance, the communication seems pretty bulky, but if you subtract out the honking Javascript file at 331424 bytes, your average comes down to 838. So, overall, this Ajax app seems pretty efficient. In particular, I performed many more actions on the browser than are reflected in the measely 93 hits in my proxy log. That suggests that plenty of the business logic is in the browser.

One more thought: as Ajax apps become more sophisticated, we may want to have web servers and proxies and such start logging both the response and request sizes.

Note: I'm not sure there's any help for the ugly logo, no matter how well the app performs.

Ajax Translator App with Ad Targeting

Just came across a semi cool Ajax translation application. I'm not sure how useful a translate-as-you type app is, but this version will pull down google ads based on what sort of text you are typing. Perhaps a taste of advertising future?

Zohowriter - an Ajax based word processor

These word processors are springing up like weeds. YAAWP (Yet Another Ajax Word Processor) is Zohowriter.  By only beef so far is that it has a few defects and doesn't seem to behave quite like a desktop Word Processor. In particular, the text properties in the formatting bar don't always change when selecting text in the document.

So, are all of these word processors actually useful, or is the word processor to Ajax applications what the chess program was to AI, aka "the drosophila of AI?"