Agile Ajax

6 GWT Resources for the iPhone

We're starting to do some work at my firm for the iPhone (see my colleague Noel Rappin's recent release of a Rails Plugin for iPhone development), so I've begun some investigations and experimentations of using GWT for iPhone development. The sad truth is that, despite the initial hoopla around GWT on the iPhone, not much has been happening.

1. GWT + iPhone == Pretty Tasty - a nice discussion of the development of the GWT Feed Reader for the iPhone and what went into building it. A great starting point if you're just getting started with GWT on the iPhone.
2. GWT App Dev for the iPhone - from the GWT Blog, a slightly different take on the GWT Feed Reader project discussed above.
3. GWT Feed Reader source code - open source. If you learn best by doing, read the code. See how it's done.
4. Fake the iPhone User Agent - testing the iPhone specific versions of sites and apps is easy if you have Mac OS X, but if you don't this quick little tutorial shows you how to fake the User Agent in Firefox.
5. iPhone Debug - an attempt to make debugging Ajax on the iPhone easier, sending debugging info back to the server. Uses Jetty with its support for COMET.
6. Firebug for the iPhone - the idea that inspired the above. A bridge between your iPhone and the Firebug console in your desktop browser. Now you can see those debug messages in the comfort of your laptop instead of having to do it in the little iPhone window.

I'm going to put together a little bit of a iPhone/GWT howto/tutorial/investigation in the next couple of weeks, starting with a close analysis of the Feed Reader above. Stay tuned.

My IBM developerWorks series, part 2: Tooltips, lightboxes and more jQuery goodness

IBM developerWorks recently published my second "Ajax overhaul" tutorial. This series teaches intermediate-level developers how to layer Ajax features atop old-fashioned CRUD applications. My colleague Dietrich Kappe calls this the "Christmas tree" approach to Ajax development, and it's a valid choice for many companies. It's not the sexiest take on Ajax, but it often provides a lot of value for relatively little cost.

"Ajax overhaul" tackles a fictional shopping site for its use cases. Each installment examines a particular deficiency in the existing application's user experience and improves it using Ajax and progressive enhancement. Because I focus on the interface layer, the code examples feature only client-side code. I've completed four installments so far, each one employing some combination of jQuery, custom JavaScript and custom CSS.

I previously posted about the first installment. Here's the direct link to the second:

• Ajax overhaul, Part 2: Retrofit existing sites with jQuery, Ajax, tooltips, and lightboxes

Looking for a front-end jedi/ninja/warrior/whatever

Pathfinder is recruiting! Although it's not yet listed on our career portal, we're looking to beef up the Ajax practice at our Chicago office with a talented front-end programmer. This person should be a jack of all trades, master of several, and flexible about which warrior metaphor to adopt for professional use.

Please give me a shout if the following describes you:

• You enjoy hand-coding complex DHTML and Ajax applications. JavaScript is your passion ...
• ... but not your only passion. You also make room in your heart for standards-compliant HTML markup and expertly crafted CSS.
• Speaking of CSS, you're chomping at the bit for CSS 3. Table-free, CSS-based layout is old hat to you.
• You've used more than a single browser and a single operating system in the last five years. You can debate the merits of Gecko vs. Webkit till the cows come home. You can rattle off browser bugs like the names of old high-school friends. The phrase "Opera 9.5 beta 2 on 10.4 Tiger" doesn't sound like nonsense to you.
• You've written production code using more than one open-source JavaScript toolkit and, more importantly, you know how to code without one. You have opinions about why Dojo is better than GWT (or vice-versa) and can intelligently discuss the pros and cons of jQuery vs. Prototype vs. MooTools.
• You're passionate about front-end developement as a discrete category of software engineering, but you can jump in on the server-side stuff when you have to. SQL, Apache and the command line don't scare you.
• You've worked with templating systems in JSP, Rails, PHP or some other framework.
• If you don't have direct experience with Flash, Flex, Adobe Air or Silverlight, you're at least willing to give such competing UI technologies a shot.
• You aren't necessarily a visual designer, but you don't need somebody who went to art school to swoop in and Photoshop a rounded corner or a background pattern for you.
• You have experience working for an Agile shop - or you desperately want to.
• You may not be an information architect, but you understand the world of user experience design. You're comfortable reading wireframes and requirements documents and participating in the design of complex software systems.
• You're not scared of acronyms like XML, XSL and XSLT.
• In short, you know how to put the "V" in MVC.

If this sounds like you, I'd like to take you out for lunch and a little chat about Pathfinder Development. See our jobs page for all the boilerplate about relocation, etc.

Photos (Creative Commons Attribution License from Flicker): PhillipWest | R'eyes' | dizznbon.

Whitepaper - Ajax Roadmap: How to tranform your website without starting from scratch

My colleague Brian Dillard has written a rather decent whitepaper entitled Ajax Roadmap: How to tranform your website without starting from scratch, that lays out an approach to transforming your website or application without having to reengineer the whole darned thing. It offers a number of case studies, rationals and justifications for launching Ajax projects, pitfalls, quick wins -- in short, everything you need to start an Ajax transformation in your own company.

We're quite pleased with it. Come on by and have a look.

My Web 2.0 Expo slides now available online

Photo: James Duncan Davidson

Web 2.0 Expo is closing up shop. I think my talk went pretty well. Despite being the last session of the last day, I had about 100 people, most of whom stayed till the bitter end. Not a lot of questions, but quite a few folks stopped by to chat afterwards - including Mark Meeker, a friend and former colleague from Orbitz who's no stranger himself to the conference circuit.

I made a conscious decision not to focus on code, but rather to provide tools and advice to developers who want to be a part of the conversation between standards bodies, browser vendors and library authors about the future of the web. I'm not sure how much of it makes sense without my narration, but I've posted my slides to Slideshare.

Now, out for Afghani food with old friends. More about Web 2.0 Expo next week!

• My slides (embedded)
• My slides (slideshare)
• My abstract
• My Crowdvine page

GWT, Remote Models and the Need for Better Replication Algorithms

PESSIMISM, n.

A philosophy forced upon the convictions of the  observer by the disheartening prevalence of the optimist with his  scarecrow hope and his unsightly smile.

-- Ambrose Bierce, The Devil's Dictionary

I frequently crawl through the various GWT project hosted on Google Code, as well as the few that aren't (like the almost venerable GWT Widget Library). One type of solution that is emerging is what I would call model synchronization libraries. These build on top of the GWT-RPC mechanism and allow a synchronization of an object model on the client-side with a corresponding one on the server-side.

Right now there are two major choices available:

• hibernate4gwt - Number one with a bullet. Elegant and powerful library that allows you to ship domain objects back and forth between the GWT client and the Java server, persisting changes, all at the cost of extending one class: LazyGwtPojo. There are two basic flavors -- stateless and stateful -- with a couple of different subflavors to support a variety of development use cases.
• Gwittir - Developed by the two dudes who wrote GWT in Practice, this library provides domain object bindings without forcing a dependency on Hibernate (should we add "Hibernate infected" to our lexicon along side "Maven infected?").

It's clear why these libraries are gaining in popularity. The whole idea of writing applications in GWT is to have a commonality of language and domain model. You want to be consistent and not have to navigate the OO/RPC boundary every time you modify or retrieve your client-side model. (An additional alternative worth mentioning is IT Mill Toolkit 5, which as a server-side ajax framework with it's widgets realized in GWT, already has the client-server synchronization baked in.)
 

Web 1.0 to 2.0: What Kind of Application do you Have?

Of each particular thing ask: What is it in itself, what is its nature?

-- Marcus Aurelius

We get asked quite often to convert Web 1.0 applications into Web 2.0 applications. We do it so often that we've developed tools, written frameworks, published articles and whitepapers and given conference talks on the topic.

So far, there are and remain three different approaches:

1. The Christmas Tree Approach - decorate an application with lots of Ajax widgets.
2. The Reboot - reengineer your application from scratch.
3. The Resurfacing - leverage the MVC nature of your application by turning your controllers into web services, your views from HTML into XML or JSON, and deploying a single page JavaScript client to access your new backend.

Which one to chose? That depends on what kind of application you have. Is it a Web 2.0 application masquerading as a Web 1.0 application, or is it really a rather dull Web 1.0 application that you need to tart up with a few widgets?

 

Ajax Testing: Doubling Down with Selenium and JMeter

This idea comes from my colleague John McCaffrey, but since he's having a bad case of blogger's block, I get to post about it.

For anyone who has developed tests for web applications with a tool like Selenium (or Watir), you know it is a sysiphean task, rolling the boulder of automated tests up the hill of a constantly changing application. The DOM changes, the text changes, the url's and parameter's change, iframes and onload events don't always play nice with your test recorder. Depending on the framework you are using, id's may change in unpredictable ways, forcing you to hack together brittle xpath expressions. Still, functional testing is important, so you persevere, spending countless hours in making those tests run clean.

 

Book review: Advanced Ajax by Lauriat (Part 2 of 2)

Back in February, I reviewed the first half of Shawn M. Lauriat's "Advanced Ajax: Architecture and Best Practices" (Prentice Hall, 2008, 360p). The first four chapters of Lauriat's book, which focused almost exclusively on client-side technologies, impressed me considerably. But it's taken me several weeks to get through the remainder of the book, and there's one reason why: PHP.

The server-side portion of "Advanced Ajax" uses PHP code to illustrate its many and varied lessons about Ajax architecture. It's not that I have anything against the popular web-development framework and scripting language. It's just that, after spending my career in the ASP Classic and JSP trenches and slowly ramping up on Rails in the last year, I'm not the ideal target audience for these code samples. Adding "PHP" to the title of the book might have limited its potential audience, but it also would have been more accurate.

That said, there's a lot of value here for adherents of any server-side framework. Lauriat discusses each topic from a general perspective before diving into the code. The technical approach to a given problem would obviously differ by framework, but the high-level approach wouldn't. If you don't mind skimming past the content that doesn't apply to you, Lauriat's advice about developing stable, scalable, accessible and secure Ajax applications transcends framework allegiance.

Two Years of Agile Ajax: Web Killed Off GUI Techniques Just in Time for Ajax to Need them Again

I launched this blog just a little over two years ago, on March 21st, 2006. Appropriately enough, my first post was about User Experience (UXD) and Ajax. The blog has come a long way since then -- we've added another full-time blogger (Brain Dillard), published nearly 700 articles of original Ajax and Agile related content, and covered the major new innovations in Ajax and Web 2.0 -- but in many ways Ajax technology is still struggling with the same issues that I wrote about back then:

As it stands, Ajax is still in its infancy (or in its wild west phase -- pick your metaphor), and Bill's simple three part "patterns" are emblematic of this.

 

jQuery Form and jQuery UI Tabs: Two great tastes that taste great together

I spent last week holed up writing part 4 of Ajax Overhaul, my series of articles for IBM developerWorks. Aimed squarely at Ajax beginners, the series shows how to progressively enhance Web 1.0 sites with jQuery and Ajax. Each installment starts with the pre-Ajax version of an example e-commerce application and takes readers through the steps of retrofitting it to improve and modernize the user experience. The tagline for this installment is "Streamline multi-step processes with tabs and Ajax forms," a topic that allowed me to employ two of my favorite plug-ins for the jQuery open-source Ajax toolkit:

• jQuery Form, which gives jQuery several methods for serializing form data and submitting the results via Ajax.
• jQuery UI Tabs, which turns a series of divs and unordered lists into a tabbed interface.

I feel like I've run on and on about my enthusiasm for jQuery on this site, but I can't help it. One of the cool thing about its plugin ecosystem is the ease with which you can cross-pollinate a couple of plugins to create novel effects. In this case, I wanted to take a series of web forms - the checkout process for my example shopping site - and turn them into a single-page, tabbed interface in which each tab represented one step of the process. The biggest additional requirement was progressive enhancement; with JavaScript absent or disabled, the checkout process has to work like it did before I retrofitted it. All it took to accomplish these goals was a judicious mix of my two plugins.

Checking in from AjaxWorld East

Just a quick update from the conference (which is now over). I gave my talk at the AjaxWorld East 2008 and got some interesting reactions. The talk really had three parts.

1. What is GWT and how do you use it?
2. How do I resurface an existing J2EE app?
3. Where are all of the alligators?

There is always a risk that someone or other feels that too little or too much time is spent in one part or the other. In fact, while converting a JSP to produce JSON so it could be consumed by the GWT client app, I had one person raise their hand and as "what does this have to do with GWT."

So, in the end, I had half the people nodding their heads, yes, and the other half shaking their heads, no. I was peddling a very pragmatic approach, rather than a product or service, and that might have thrown some folks off.

Met a lot of cool folks, put some faces to names, got to sit on a panel with Douglas Crockford. Cool beans. More next week.

Comet 2008: The State of Play in Reverse Ajax

Just as with advertising measurement for Ajax, I continue to follow the technology known as Comet (open up an HTTP request from browser to server, then keep it open, pushing content down periodically) to see if it is ready for prime time. I thought I'd share my reading list from the last few months:

• 20,000 Reasons Why Comet Scales - Greg Wilkins achieved sub-second latency using Dojo Cometd/Bayeux and Jetty. Yes, it's a benchmark, and benchmarks can't be swallowed whole, but it's still quite impressive to see just two load balanced servers manage this kind of load.
• Comet: Reverse Ajax for streaming data from the server - Daniel Rubio of TheServerSide take an introductory look at Comet, but from the Enterprise/SOA perspective. If you're looking to convince your CTO to adopt Comet for your enterprise, you can adapt some of Daniel's arguments.
• A Comet Implementation for Google Web Toolkit - somehow I missed this one from last July. Does the usual hidden iframe implementation for Comet on the client. Simple design with lots of code.
• Comet Daily - Yep. The Comet technology now has its own blog (though it isn't quite daily). Lots of juicy articles, reviews and tutorials.
• Comet Support in GlassFish - Jean-Francois Arcand writes a compact little article on how to do Comet with GlassFish. Has links to lots of little demos.
• Comet Ruby on Rails - not to be left out, the Rails folks are trying their hand at Comet with the Juggernaut plugin.

While there is some progress, I have to say that Comet still seems to me like climbing up the down escalator. If you really want this kind of functionality in the browser, develop a new protocol and lobby for built in browser support.

Technorati Tags: ajax, comet, gwt, rails, dojo

Stop, Reload, Error Loading Page 404: Converting Web 1.0 to 2.0

OK, onwards in our effort to convert Web 1.0 apps to Web 2.0. Today we'll focus on some of the differences between Web 1.0 and 2.0 from the user experience perspective. When we think about our web interface, we usually think about the links, forms and controls we've put in place. But the reality is that often things go wrong with our app -- sometimes network problems, system problems or application errors -- and we get errors like 404 or 503, or a "page did not load." How does the user respond to these issues?

Usually they make use of the Stop, Reload, Back and Forward buttons. Most times application developers see the use of these browser buttons as a problem, to be mitigated ("Please don't submit this form twice..."; "Don't use the back button or your order will be submitted twice..."). If you think about it a little more carefully, you'll find that these buttons form a vital safety net for web applications, without which users would be crying in frustration at the "Unable to connect..." pages and spinning cursors that have brought your app to a halt.

 

Put on Your X-Ray Specs: Converting Web 1.0 to 2.0

I've blogged in the past about converting Web 1.0 to Web 2.0. There were and still are three options:

1. The Christmas Tree Approach - decorate your existing Web 1.0 application with lots of snazzy Ajax widgets. It's quick, but it's a bear to maintain.
2. The Version 2.0 Approach - scrap your existing app and develop a Web 2.0 version from scratch. This can take a while and suffers from the same issues that all version 2 projects do, i.e. if version 1.0 has evolved over time, capturing full requirements may be challenging.
3. The Resurfacing Approach - keep your backend logic, but rewrite the interface as a single page, desktop-like GUI.

This last approach has many advantages: it's quicker and less expensive than #2, but easier to maintain and gives a more unified user experience than #1. It also has some of it's own pitfalls. Mostly, the issues are around some of the hidden assumptions of a Web 1.0 applications that expects all interaction to occur through a postback.

So, what's the plan? How do we go about converting our 1.0 app via resurfacing? To do this you have to put on your X-ray specs, so to speak, and look at the bones of your application. If you're already using an MVC framework, your ahead of the game. So, what are the analytical steps? 

My IBM developerWorks series makes it debut

The first installment in my tutorial series "Ajax Overhaul" is now live at IBM developerWorks. The full title, and it's a mouthful, is "Ajax overhaul, Part 1: Retrofit existing sites with Ajax and jQuery: Improve your user experience and simplify your navigation with modal dialogs." Targeted at Ajax beginners, the piece walks through the steps of progressively enhancing a Web 1.0 shopping site with Ajax and DHTML using nothing but client-side code.

The hook for the entire series is this: "The user experience of most web sites suffers from some common, fundamental flaws. Don't compound those mistakes by layering Ajax on top of them. Instead, use Ajax to fix those existing problems - and do so in a way that degrades flawlessly."

This first installment focuses on modal dialogs. The forthcoming second installment will tackle tooltips and lightboxes, while the third installment will delve into navigational tabs and image slideshows. I'm hard at work on the fourth installment, which will cover Ajax-powered forms.

As a whole, the series is about 40% user-experience stuff and 60% step-by-step technical instruction; for this first installment, that those proportions are probably flipped.

My goal with this series was to show an introductory-level audience not just how to plug cool new technologies into existing web applications, but when and why to do so. A lot of technology tutorials teach developers how to play with shiny new technologies for their own sake. Instead, I wanted to show them how to meet user needs with whatever technology gets the job done. I'm looking forward to feedback from the IBM developerWorks audience so I can see how well my message comes across. There's a feedback form at the bottom of the article.

More jQuery plug-ins: jCarousel and the dreaded window.alert()

Photo: Joe Shlabotnik

 

In addition to the jQuery UI Tabs plug-in I mentioned in yesterday's post, I've been playing with jCarousel, a fairly mature component for building DHTML and Ajax slideshows out of images or arbitrary markup. Overall, I was impressed enough with jCarousel to build half of my latest piece for IBM developerWorks around it. But despite the component's many configuration options, I found that I had to do some hacking to bend it to my will. jCarousel can get a little confused when you load dynamic content, especially if that dynamic content includes blocks of markup rather than just individual image files. It miscalculates some widths in its behind-the-scenes DHTML trickery, causing images to load but never appear in the carousel. To get around this problem, I used the brute-force approach of applying a hard-coded width to my carousel:

//build the carousel
jQuery('#imageCarousel').jcarousel({
	itemLoadCallback: itemLoadCallback,
	scroll: 1
});

//now recalculate the width of the unordered list
//that makes up the carousel items
jQuery('ul.productImages').css("width","3012px");

jQuery UI in action: Thumbs up on tabs, thumbs sideways on themes

The first two installments haven't even been published yet, but I spent most of last week hammering out the third entry in the series I'm writing for IBM developerWorks. The first two installments tackled things like modal dialogs, tooltips and lightboxes. But for the latest piece, I delved a little deeper into full-scale Ajax by showing readers how to load page fragments and images on the fly into tabbed interfaces and image carousels. As before, I enjoyed writing for a more beginner/intermediate audience. And I especially enjoyed the chance to play with jQuery and all of its shiny new plug-ins.

Inspired, perhaps, by the success of YUI and Ext, all the big JavaScript frameworks are releasing companion libraries of packaged, themable UI widgets. jQuery is no exception. My latest IBM piece uses jQuery UI Tabs, one of many components in the new jQuery UI framework. UI Tabs began its life as a standalone plug-in. With the launch of jQuery UI, it has become more tightly integrated with the other components. Ultimately, that's the lure of an umbrella concept such as jQuery UI: fewer far-flung extensions of unknown provenance; more plug-and-play efficiency.

Book review: Advanced Ajax by Lauriat (Part 1 of 2)

Because Ajax moves so much application logic from the server to the client, it forces many developers to master a wider range of web technologies than ever before. To work effectively on Ajax projects, front-end developers have to concern themselves with database performance, business logic and other server-side concerns. Back-end and middleware developers, meanwhile, have to make friends with XHTML, CSS, JavaScript and a wide range of browsers. Sure, it's possible to develop Ajax apps in a siloed team environment. But it's not the easiest way, and it rarely provides the strongest results.

Shawn M. Lauriat's "Advanced Ajax: Architecture and Best Practices" (Prentice Hall, 2008, 360p) bridges the gap between developers with exclusive client- or server-side skills. By exploring tools, technologies and best practices for every layer of the Ajax programming model, this solid new programming manual promises to plug the holes in any developer's resume. Lauriat's tops-to-tails approach offers something for almost any developer, but it also guarantees most readers will find some sections remedial. As this two-part review will demonstrate, that's not necessarily a liability.

Agile User Experience: If you don't have a user, invent one of those too...

Following up on Brian's point...

A customer proxy of one kind or another is also important in integrating Agile practices with User Eexperience Design.

One of the long standing issues in combine Agile development with User Experience Design is just the differing time cycles between Agile developers, who are running on a test, develop, refactor cycle that could be mere minutes long, and the UXD designer, who is working on a longer, perhaps more traditional, cycle.

The problem comes in when the developers start making coding decisions for the interface. Without immediate feedback from the UXD designers, the developers are often left to their own devices for the initial versions, with revised designs coming weeks or months later.

Our goal is to keep the development team moving forward with current feedback from the design team. For exactly this reason, the original XP book listed Onsite Customer as a core XP practice. However, that's not always feasible, and in any case for User design you want the user's feedback, which may not be the same as the customer's.

What we try to do is use the idea of a customer or user proxy to allow us to have incremental refinement of design, and allow the development team to get quick feedback and continue with the next fine-grained task.

When a developer has a usability question, the developer asks a team member acting as a user proxy. This is either an analyst on the project or one of the designers. The user proxy makes a decision on the question -- if not a final decision on all details of the problem, at the very least a consensus on some aspect of the problem that the developer can do next. While the developer does that task, the user proxy goes off and consults with the design team and the customer.

Ideally, by the time the developer is ready for the rest of the task, the user proxy has more detailed requirements and design for the developer. The key is making sure the developer's next step is always covered.

This does require some effort from your team -- the developers have have awareness of usability issues and concerns and the design team has to have the ability to respond to developer concerns quickly. Again, though, the design team doesn't need to have all the answers at once, they just need to be able to give incremental details to the developer.

(The image of the floor appearing directly underfoot is from the ad campaign for Gatorade G2)

---

Have I mentioned my upcoming book Professional Ruby on Rails, yet? Available mid-February wherever fine computing books are sold.

Agile User Experience: If you don't have a user, invent one of those too...

Following up on Brian's point...

A customer proxy of one kind or another is also important in integrating Agile practices with User Eexperience Design.

One of the long standing issues in combine Agile development with User Experience Design is just the differing time cycles between Agile developers, who are running on a test, develop, refactor cycle that could be mere minutes long, and the UXD designer, who is working on a longer, perhaps more traditional, cycle.

The problem comes in when the developers start making coding decisions for the interface. Without immediate feedback from the UXD designers, the developers are often left to their own devices for the initial versions, with revised designs coming weeks or months later.

Our goal is to keep the development team moving forward with current feedback from the design team. For exactly this reason, the original XP book listed Onsite Customer as a core XP practice. However, that's not always feasible, and in any case for User design you want the user's feedback, which may not be the same as the customer's.

What we try to do is use the idea of a customer or user proxy to allow us to have incremental refinement of design, and allow the development team to get quick feedback and continue with the next fine-grained task.

When a developer has a usability question, the developer asks a team member acting as a user proxy. This is either an analyst on the project or one of the designers. The user proxy makes a decision on the question -- if not a final decision on all details of the problem, at the very least a consensus on some aspect of the problem that the developer can do next. While the developer does that task, the user proxy goes off and consults with the design team and the customer.

Ideally, by the time the developer is ready for the rest of the task, the user proxy has more detailed requirements and design for the developer. The key is making sure the developer's next step is always covered.

This does require some effort from your team -- the developers have have awareness of usability issues and concerns and the design team has to have the ability to respond to developer concerns quickly. Again, though, the design team doesn't need to have all the answers at once, they just need to be able to give incremental details to the developer.

(The image of the floor appearing directly underfoot is from the ad campaign for Gatorade G2)

---

Have I mentioned my upcoming book Professional Ruby on Rails, yet? Available mid-February wherever fine computing books are sold.

Agile User Experience: If you don't have a user, invent one of those too...

Following up on Brian's point...

A customer proxy of one kind or another is also important in integrating Agile practices with User Eexperience Design.

One of the long standing issues in combine Agile development with User Experience Design is just the differing time cycles between Agile developers, who are running on a test, develop, refactor cycle that could be mere minutes long, and the UXD designer, who is working on a longer, perhaps more traditional, cycle.

The problem comes in when the developers start making coding decisions for the interface. Without immediate feedback from the UXD designers, the developers are often left to their own devices for the initial versions, with revised designs coming weeks or months later.

Our goal is to keep the development team moving forward with current feedback from the design team. For exactly this reason, the original XP book listed Onsite Customer as a core XP practice. However, that's not always feasible, and in any case for User design you want the user's feedback, which may not be the same as the customer's.

What we try to do is use the idea of a customer or user proxy to allow us to have incremental refinement of design, and allow the development team to get quick feedback and continue with the next fine-grained task.

When a developer has a usability question, the developer asks a team member acting as a user proxy. This is either an analyst on the project or one of the designers. The user proxy makes a decision on the question -- if not a final decision on all details of the problem, at the very least a consensus on some aspect of the problem that the developer can do next. While the developer does that task, the user proxy goes off and consults with the design team and the customer.

Ideally, by the time the developer is ready for the rest of the task, the user proxy has more detailed requirements and design for the developer. The key is making sure the developer's next step is always covered.

This does require some effort from your team -- the developers have have awareness of usability issues and concerns and the design team has to have the ability to respond to developer concerns quickly. Again, though, the design team doesn't need to have all the answers at once, they just need to be able to give incremental details to the developer.

(The image of the floor appearing directly underfoot is from the ad campaign for Gatorade G2)

---

Have I mentioned my upcoming book Professional Ruby on Rails, yet? Available mid-February wherever fine computing books are sold.

Book recommendation: Ajax Security by Hoffman and Sullivan

Reviewers overuse the phrase "required reading," but no other description fits the new book "Ajax Security" (2007, Addison Wesley, 470p). This exhaustive tome from Billy Hoffman and Bryan Sullivan places the specific security concerns of the Ajax programming model in historical perspective. It demonstrates not only new security threats that are unique to Ajax, but established threats that have gained new traction in the Web 2.0 era. It then details both the specific technical solutions and - more importantly - the mindset that are necessary to combat such threats.

Because so many developers have historically overlooked the importance of security, the authors approach their topic for what it is: a remedial subject. They take pains to explain the basic mechanisms by which hackers have exploited insecure web applications over the last decade: cross-site request forgeries, denial of service attacks, cross-site scripting and SQL injection. Then they explain how those mechanisms have changed thanks to the rise of xmlHttpRequest, public APIs, mash-ups and aggregators. If you've ever read a Douglas Crockford rant about the "brokenness" of the web security model and wondered why the guy was such an alarmist, Hoffman and Sullivan are only too happy to provide you with a much-needed wake-up call.

20 useful Facebook/FBJS developer resources

The Facebook application platform is less than a year old, and Facebook JavaScript (FBJS) made it out of beta less than four months ago. It's therefore unsurprising that comprehensive developer resources for both are a little thin. Most of the real content comes from official Facebook properties. Still, a little digging helped me uncover some decent tutorials and a wealth of open-source wrapper libraries for Facebook's RESTful API. Interestingly, I didn't uncover any big attempts at Prototype-style FBJS toolkits - just a few scattered examples of utility classes. It seems that Facebook development experience is still enough of a competitive advantage that the experienced few aren't rushing to create helper libraries for their hapless peers.

New Year's Resolutions 2008

As I've posted before, I'm pretty leery of prognostication. My colleageus Noel and Dietrich have already made their predictions for 2008, anyway (here and here) so who am I to join the fray? Instead, let me dust off another hoary device and share my programming resolutions for 2008.

2007 in review: Five humble suggestions for better programming and writing

I began 2007 as a front-end tech lead on a multi-million-dollar software project for a global travel company with a massively distributed waterfall development model. I ended it working in small, agile teams on R&D projects at a small outsourced software shop. I got involved in open source, became (yet another) tech blogger, and set in motion lots of other writing and speaking projects for 2008. It's been quite a year.

I'm sure everybody is sick of year-end lists at this point, so let me present a slightly more personal and highly subjective list of the five most important things I learned this year about software development and technology writing:

Ajax Predictions for 2008

I'm a little early this year for my 2008 predictions. No matter. I seem to have been a bit early with my 2007 predictions as well, and as they say in the venture capital biz, "too early is the same thing as wrong." Some of my predictions did come true -- Ajax is no longer such a big buzzword; a number of framework specific books (GWT, jQuery) have been published at the end of 2007; Microsoft's Ajax stack continues to limp behind the rest. A few others did not, notably the ho-hum release of IE7 and the delay in FF3. So, let me try my hand at some more prognostication:

1. Elegant, good looking frameworks like Ext and myGWT will gain traction. The more out-of-the-box good looks you can give them with CSS and bundle images and icons, the more acceptance they will gain.
2. The JavaScript framework chaos will continue. The OpenAjax alliance will continue to be ineffectual in promoting things like widget interchangeability.
3. Security will become a big headache, as less sophisticated developers start to venture into the wonderful world of Ajax, littering the web with state and control logic on the client side.
4. Towards the end of 2008, FF4 and IE8 will start to change the landscape of Ajax and Web 2.0 with an update of JavaScript and new browser features.
   
5. MS Volta will do nothing. It is just a FUD shot across the bow of GWT.
6. Everyone except Craig's List will have some form of Ajax on their site.
7. Desktop RIA's through Google Gears, Adobe AIR, etc., will start to make an impact in the second half of 2008. Look for desktop/web hybrids of the office productivity tools, such as word processors and Powerpoint clones, to see greater use in the corporate IT space.
   
8. GWT's compiler will produce more efficient code than 98% of JavaScript developers can do by hand.
9. With the new browsers, a cross platform Canvas/SVG will be a reality by the end of the year.
10. IE8 will still leak memory like a sieve.

Have any of your own predictions? Feel free to add them in the comments.



Technorati Tags: ajax, 2008 predictions

IA and agility

A friend of mine recently left a position at a supposedly Agile shop where developers wrote code based on raw requirements rather than formal IA. When he described their process to me, I really felt his pain. I can't imagine trying to capture the complex requirements of Ajax applications without the tools of modern IA.

Good information architecture provides not only detailed blueprints for developers, but also the raw material for test cases. If your only documentation of a component is just a high-level business document or a working example of that component, then it's difficult to identify deviations from the spec or prove that an unexpected behavior is a bug. Sure, on Agile projects the first-round IA sometimes gets reverse-engineered from working demo code. But IA and UxD should be a central component of Agile development.

Technorati Tags

• information architecture
• agile
• JavaScript
• Ajax

Ajax and Browsers: Recapitulating the Early Days of Personal Computers

I guess its my turn to spin out historical analogies. Does anyone remember GEOS? That's right, a GUI OS for the Commodore 64. In those early days, everyone rolled their own GUI  (character mode windowing...yum). Some were good, some were bad, some were downright ugly. GEOS was an attempt to bring the unifying influence of an OS that provided the basics of a GUI to the C64, something that Windows brought to the PC and the Mac OS brought from the beginning to the Macintosh. GEOS and the C64 faded away, but the days of the custom crafted GUI were numbered. All that stuff is and should be baked into the OS.

We're in a very similar situation when it comes to the browser and Ajax/Widget libraries and frameworks. Everyone is rolling their own widgets, developing their own DOM and Ajax utility libraries and frameworks. I think the next step should be obvious. Bake an Ajax library into the browser. Can you imagine having Ext already present and available in your JavaScript runtime? Imagine the boon this would be to standardization, code reuse, etc.?

 

IA in the age of Ajax

On Monday, Richard F. Cecil of UX Matters published an interesting piece titled Documenting the Design of Rich Internet Applications: A Visual Language for State. In it, he discusses the challenges facing information architects in the Ajax era. Full of practical, how-to advice for IAs and developers, it's already prompted some interesting comments. I'd recommend it to anyone interested in the process of developing RIA specs.

A few months back, some former colleagues of mine - both of them top-notch IAs - approached me about putting together a panel discussion about just this topic. The panel never happened, but our initial discussions got me thinking about the role of IA in RIAs. As web-based user interfaces grow more complex and modular, many of the traditional approaches to IA fall by the wayside. Steve Hansen, a commentator on Cecil's piece, nails the problem:

As someone who designs and builds complex RIAs, with complex non-linear work flows, I have been struggling to find the ideal modeling vocabulary. Things get even more interesting when you have complex server back-end functions. I currently use a hodge podge (technical term) of wireframes, UML activity, sequence, and communication diagrams and whatever else seems to fit the bill. This is surely an area ripe for discussion and exploration.

It's clear to anyone who's ever worked on even a moderately complex Ajax application that screen-based specs don't cut it. Unless documentation breaks things down into individual components, controls and user actions, it will fail to capture all of the possible conditions. Even a component as commonplace as an image carousel or a rich-text editor includes more discrete points of user interaction than an entire webapp did five years ago. Throw in something like keyboard commands - a topic Cecil's piece doesn't broach - and the complexity multiplies. And let's not even talk about the need to document accessibility modes.

I'd be interested to hear from people with more of a desktop software background - the same folks who are flocking to GWT - about their approach. I don't know much about IA in the desktop world, but I imagine that webapp IAs will be looking there for inspiration. Desktop GUI frameworks don't offer the free-for-all that HTML, CSS and JavaScript do. There's less freedom, but there's less of a burden to reinvent the world over and over.

That's why the GWT or Ext model offers productivity gains not just for developers, but also for IAs and designers. If your interface is built out of standardized widgets rather than build-your-own components, much of the documentation can live at the widget level. Application IA then benefits from a shared vocabulary in which common interactions have already been described and it's only corner cases and the interplay between components that need individual documentation. Users win, too, because they don't have to figure out how, for example, your date-picker works; it's just an instance of the same date-picker widget they've used in other applications on the same platform.

As Ajax UI toolkits mature, it will be interesting to see how their authors balance the desire for completely customizable interface components with the productivity and usability gains of componentization. At what point do bells and whistles cost more in complexity than they deliver in novelty?

Technorati Tags

• information architecture
• usability
• JavaScript
• Ajax

GWT Testing With Groovy? (Heck yeah!)

Speaking of GroovyTestCase, I experimented a bit with using both it and JMockit as an alternative to GWTTestCase.  This may sound strange at first-- after all, the point of GWTTestCase is to exercise your code in the way it will eventually execute as JS at runtime.  What I've effectively done is to run my tests from within Groovy against GWT classes compiled-as-Java.  First I will go through an example of what I'm talking about, then I will summarize some of the thoughts I had as a result of going through this exercise.

Below I have an implementation of KeyboardListener which modifies the contents of a TextArea based on keypress.  I want to test the behavior of this listener.  Here's what I did:

1) Write a test.  I did this using Groovy, but you could just as easily write it in Java using JUnit TestCase if you prefer.  This particular test verifies that my listener will append an equal amount of whitespace to the next line when the <enter> key is pressed, and position the cursor at the end of that new line.  Very simple stuff, but worth testing even outside of GWTTestCase.  I find that the Groovy style keeps this test case a lot shorter than it would otherwise be in Java.

class SampleTest extends GroovyTestCase {
    private static final char KEY_ENTER = (char) KeyboardListener.KEY_ENTER
    private static final int NO_MODIFIERS = 0;

    protected void setUp() throws Exception {
        Mockit.redefineMethods(GWT, MockGWT)
    }

    public void testEnterKey() {
        def keyListener = new MyKeyListener()
        def textArea = new TextAreaFixture(text: "    abcd", cursorPos: 8)

        keyListener.onKeyPress(textArea, KEY_ENTER, NO_MODIFIERS)

        assertEquals "Content is incorrect", "    abcd\r    ", textArea.text
        assertEquals "Cursor position is incorrect", 13, textArea.cursorPos
    }
}

class TextAreaFixture extends TextArea {
    String text;
    int cursorPos;
}

2) Mock out com.google.gwt.core.client.GWT.  This ultimately comes down to mocking out the 'create' method.  Notice below that we make a special case for 'DOMImpl.class'.  The reason for this is that the actual DOMImpl is decided at runtime.  It is also referenced in the constructors of the widget we are testing (i.e. 'TextArea()').  Note that this class needs to be a Java class (i.e. non-Groovy) due to the way in which JMockit redefines methods on existing Java classes.  As for why I return 'null' for everything else, I do this to fail fast in any cases I'm not testing.  As it turns out, this is all I need for the testcase above.

public class MockGWT {
    public static Object create(Class classLiteral) {
        try {
            if (classLiteral.equals(DOMImpl.class)) {
                return new MockDOMImpl();
            }
            return null;
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

3) Subclass com.google.gwt.user.client.impl.DOMImpl.  Ah-hah!  Here's the catch...  This step provides us just enough code to satisfy construction of our Widgets.  For my purposes, I let my IDE generate this subclass for me, and stuck with the default implementations it generated.  I return an instance of it above.

public class MockDOMImpl extends DOMImpl {
    public Element createElement(String tag) {
        return null;
    }
    [...]
}

The approach to this style of testing intercepts any calls on the Widgets themselves before anything is allowed to hit the underlying DOM implementation.  What I provide above is just an example of how this could be done, specifically when testing behavior of custom classes that happen to inherit or reference GWT classes.  Testing things like layout or UI concerns is another matter, but one which I would not consider 'interesting' enough for unit testing, since most of the behavior on the UI can be easily inspected by viewing Widgets as simple beans.

So why would you want to attempt this anyway?  First, I happen to like working in Groovy, so I crave more reasons to do exactly that.  Second (and more pertinent), much of behavior I find interesting in GWT apps revolves around simple message passing.  I trust GWT enough to do its job once I leave the Java space.  Beyond this are some philosophical differences I have with GWTTestCase, particularly since GWTTestCase makes it hard to maintain a larger body of unit tests, requires additional configuration in your module file (or as I've done before, constructed a different "test-only" module file that inherited from two module files), did not let me set up tests the way I would like to, etc.  I could go on.  When unit testing, very little of the logic I write is made less interesting or less relevant outside of a JS runtime environment, so where possible I would like to gain some benefits from treating everything as Java.

There are other benefits.  There's the noticeable speedup in execution time when running entirely within the JVM.  There's also less restrictions on the test class itself with regards to instance variables, language (i.e. Groovy), dependencies in outside packages, etc.  Why wouldn't I want such flexibility in my test code?  I really want my test code to be as concise and expressive as possible, particularly as my apps get larger and more complex.  I think having alternative ways of approaching this kind of testing brings a lot of these types of benefits.

GWT as developer tool

Test driven development has advantages all around-- one of them being that it saves you time.  And what's a good use of this time?  Why, coming up with novel ways of testing, of course.  I had one of those experiences the other day thinking of ways to go about profiling memory usage of a web application (by which I mean, the server-side component of a GWT-based application currently in development).

At it's heart, the problem was simple enough: execute a few thousand service method calls with semi-random data, and see what happens to memory over time.  I considered writing a standalone Java application, or culling through existing testcases for some kind of integration test which provided enough coverage to be "interesting", and decided that what I really wanted to do was exercise the system through its front door.  I pulled out JMeter but soon had a realization: "we're writing a GWT app.. why not just write another GWT app to exercise the same services methods as we otherwise do, but in a manner I need?"  So that's what I did.

The merits of alpha-geekdom

Jeff Atwood of Coding Horror recently picked up a thread from Ben Collins-Sussman (of SVN fame) about the distinction between alpha geeks and the majority of vocational programmers. According to Collins-Sussman:

There are two "classes" of programmers in the world of software development: I'm going to call them the 20% and the 80%.

The 20% folks are what many would call "alpha" programmers — the leaders, trailblazers, trendsetters, the kind of folks that places like Google and Fog Creek software are obsessed with hiring. These folks were the first ones to install Linux at home in the 90's; the people who write lisp compilers and learn Haskell on weekends "just for fun"; they actively participate in open source projects; they're always aware of the latest, coolest new trends in programming and tools.

The 80% folks make up the bulk of the software development industry. They're not stupid; they're merely vocational. They went to school, learned just enough Java/C#/C++, then got a job writing internal apps for banks, governments, travel firms, law firms, etc. The world usually never sees their software. They use whatever tools Microsoft hands down to them -- usually VS.NET if they’re doing C++, or maybe a GUI IDE like Eclipse or IntelliJ for Java development. They've never used Linux, and aren't very interested in it anyway. ... They know exactly enough to get their job done, then go home on the weekend and forget about computers.

Say what you will about Collins-Sussman's generalizations; his original post, Atwood's response and Collins-Sussman's follow-up have generated hundreds of comments, many of them wildly entertaining. I think most people would agree that the two extremes exist but that there's a broad range of in-betweeners. Ignore the hyperbole and it's an interesting theory.

It's also a theory that can be applied to many different disciplines. Information architects and visual designers have their own spectrum between the vocational majority and the obsessive minority. So do project managers. Even within the programming field, nobody can be an alpha geek about every aspect of software development. A lot of people who fall into the 20% camp when it comes to writing, say, Java code fall into the 80% when it comes to the user interface. As for me, I'm far more vocational about server-side technologies than I am about Ajax. I'm obsessive about the things I find interesting and merely competent at the rest.

Software engineering is by its very nature multi-disciplinary. Even self-professed "20%-ers" can only master so many discrete technical and design skills. They key to building great software is to recognize the specific sub-disciplines that inspire you to alpha-geekdom. Then, search out opportunities that will reward you for obsessing over them.

• JavaScript
• Ajax

Thinking of alternatives to GWT-RPC

What would alternatives to GWT-RPC look like?, it would be interesting to see these come about, and I think it would be worthwhile to do some experimentation in this area.

I believe rather than tweaking what's there, we can see radically different approaches go different directions, either by building on top of GWT-RPC, or providing a drop-in replacement. 

A few issues to consider: while we are working in a richer client-side environment, we still define and build our server-side interfaces in the same fashion as before:  Remote interface, implementation, and server-side configuration.  GWT requires an additional client-side (i.e. 'Async') interface, target definition & invocation code (casting the GWT-generated proxy to the async type), editing the module file definition for the service, making sure any data objects implement GWT's 'isSerializable' interface, and the server-side component extending GWT's RemoteServiceServlet.

The client/server contracts are generally the same-- "save this", "retrieve this", "search for this", etc.  So a a little bit of convention might go a long way between what the client is likely to send and what the server generally provides.  Might this reduce the list above?  How about putting the async behavior where it belongs on the GWT components rather than using the 'onSuccess/onFailure' callback mechanism?  It would make the code clearer, and would probably be much more testable.

And then there's testing-- conventional wisdom suggests having a separate, 'mock' version of the server-side component when running in hosted mode.  GWT provide a way of detecting 'hosted' mode from 'non-hosted' mode, effectively allowing your code to switch between server-side implementations.

Frameworks such as the GWT Server Library aid in the server-side configuration pain, allowing you to write POJO services while using Spring configuration to wire into GWT-RPC.  But the net effect hasn't changed-- there's still all that configuration and, I put forth, intrusive code required even to get us there.

As Java developers, our pain points are generally high when it comes to this sort of thing-- and the tired and true aphorism generally surfaces: "well, I've seen worse..."  Let's not settle our technology choices just for that reason alone.

Now, I know there's nothing in GWT that requires you to use GWT-RPC.  Because any solution would require JS bindings for any Java code, it is not a trivial problem to solve, but not out of the question either.  Alternatives could happy live side by side with GWT-RPC.  Will an alternative surface?  Can this be made better by using another paradigm?

ZK on Android

I'm glad that my Comp Sci classes were taught differently from my Art History courses, otherwise all I would remember is the hum of the slide projector and the taste of sleep in my mouth. Instead I remember the Bridge Pattern and the wonderful things it can do for you. Specifically, you can decouple an abstract component GUI from it's concrete implementation (see AWT). This makes the task of retargeting the GUI framework to a new platform comparatively quick and easy.

Now server-side Ajax frameworks (Echo2, ZK, etc.) have been using the Bridge Pattern for a while, but with the exception of OpenLaszlo (Flash and Ajax) and WidgetServer (Ajax, Swing Applet, Swing Desktop), they only target the browser, i.e. one platform.

Now ZK Mobile, an extension of the ZK Framework for mobile devices, is showing the power of the Bridge Pattern in the mobile world. On 11/12/2007 Google released the Andriod SDK and two days later, on 11/14/2007, the ZK team released an extension to ZK Mobile that allows you to run your ZK apps on Android. How is that for quick turnaround? Good design can be good for you and good to you.

The ZK "smalltalk" demos a simple Twitter application for the Android SDK.

Amazing how little work is necessary to get this app on a mobile phone.

Technorati Tags: ajax, android, mobile, zk

Upcoming No Fluff Just Stuff conference

The Great Lakes Software Symposium (aka No Fluff Just Stuff) is coming to Chicago November 16-18th, and if you are a java/ajax/agile developer or manager you should really check it out. I went for the first time last year and it was great. With the amount of relevant and interesting sessions they have, I could have gone to the Milwaukee one the next week and still not made it to every session I wanted to.

  Why is it good?

1. No Fluff, Duh! - This means there are no B.S. artists there trying to pitch their half-baked, proprietary, "silver-bullet", junk.
2. Best of the Best - the people that present are top-notch practitioners, and more importantly top-notch, well practiced presenters. (Its one thing to be a smart guy who knows how to write great code, but to present it and really "Teach" the audience is a different skill set)
3. Fair price- Compared to other conferences, the NFJS series is very reasonably priced (particularly because its close by). Also a $100 discount for alumni!! combine that with the early registration deal (which has passed already for this year), and you are down to a very good price.
4. Great networking - I've seen a fair amount of some of Chicago's finest developers at NFJS, and found it to be a great way to meet people from other companies (its good to find out which companies sponsor the cost, and which ones don't)
5. Relevant topics - The latest on Ajax, Agile, Spring/AOP, Groovy, Ruby/Rails, and overall development best practices.
6. Door prizes - last year a friend of mine won an Ipod. This year they have more iPods, iPhones, Sony PS3.

All in all its a good conference at a good price, and even in the cases where it covers a topic you know well, there are always good things that come out of the discussion. There is a cool energy in the air when you go to a conference like this. Amazing ideas are thrown around, groups of people huddle around a laptop trying out new ideas, and you leave the place really excited about development and eager to try out some new ideas.

Hope to see you there!
-John