800 on Your Math SAT, Software Development and Bugs

Why don't more people get a perfect score on the math portion of the SAT? I mean its dead simple -- just simple arithmetic. And there are plenty of bright young things that understand the problems cold.

I can understand stubbing your toe on the verbal; after all, there may be a word in the test that you just don't know or remember. But the math? There aren't more than a half dozen problems, and though the numbers may change, the concepts don't. So why don't more people get a perfect 800 on their math portion?

I ask this question every time someone asks me why there are bugs in software.


Continue reading "800 on Your Math SAT, Software Development and Bugs" »

Whitepaper - Ajax Roadmap: How to tranform your website without starting from scratch

My colleague Brian Dillard has written a rather decent whitepaper entitled Ajax Roadmap: How to tranform your website without starting from scratch, that lays out an approach to transforming your website or application without having to reengineer the whole darned thing. It offers a number of case studies, rationals and justifications for launching Ajax projects, pitfalls, quick wins -- in short, everything you need to start an Ajax transformation in your own company.

We're quite pleased with it. Come on by and have a look.

Agile Business, Microsoft and the Threat of Cloud Computing

Competition is the keen cutting edge of business, always shaving away at costs.

-- Henry Ford


I've been working with Java and Microsoft technologies -- .NET most recently -- in one form or another for quite some time. My company, now headquartered in Chicago with an office in NYC, was actually founded in Seattle by a group of four developers that had met around developing an Exchange-based bulk email system to replace the sendmail-based ones that Microsoft was using at the time. In that span, despite all of the food fights about total cost of ownership (TCO), etc., I haven't seen any evidence that Linux, Windows, Mac, Java, .NET, etc., puts you at a significant business advantage one way or the other. Until now.

Continue reading "Agile Business, Microsoft and the Threat of Cloud Computing" »

My talk at Web 2.0 Expo this Friday

Web_20_expoI have arrived in San Francisco for Web 2.0 Expo. Over the next couple of days, I'll post about my favorite sessions and all that jazz. But here's the obligatory plug for my session, which takes place at 3.50 p.m. Friday in room 2003. I posted about my talk when I was first accepted, and I'm really pleased with the way it's come together. I did a dry run of it with my peers at Pathfinder on Monday and got a lot of excellent feedback.

As previously reported, my session is called Do Try This at Home: Ajax Bookmarking, Cross-site Scripting, and Other Web 2.0 Browser Hacks. In it, I'll examine the various draft WC3 specs that are already coming alive in the latest crop of browsers - and how you, the everyday developer, can help road-test the future of the web by employing the new tools sooner rather than later.

I'm part of the development track, but given that I'm speaking in the last session on the last day, I'm going to mix the technical details with a lot of strategy and DIY cheerleading. The best thing about my R&D job here at Pathfinder is the chance it affords me to play with all the latest toys. I want to encourage other developers to do the same, even if they can't find a way to work such forays into their day-to-day projects.

Hope to see you there!

BDD: Book Driven Development

recipes.jpgJay Fields, who has been posting a very nice sequence of nuts-and-bolts Ruby and Rails guidelines, pauses to talk about creating examples. It's a topic I've wanted to write about here for a while, and this is as good a lead-in as any. Plus, I'm generally interested in how principles of software development apply or don't apply in odd cases, and software being developed specifically for example purposes certainly qualifies as an odd case.

For Professional Ruby On Rails, I knew that I wanted to run a single example application through the book. I had some grand visions of it being a "real" application or at least a real example of coding best practices. For best practices, I think it's pretty good on a method-by-method bases, but has some weaknesses as an entire app, for reasons that I think will become clear.

Continue reading "BDD: Book Driven Development" »

"Do Try This at Home: Ajax Bookmarking, Cross-Site Scripting and Other Web 2.0 Browser Hacks" - my talk at Web Expo 2.0 in April

O'Reilly has booked me to speak at Web 2.0 Expo, which runs April 22-25, 2008 at San Francisco's Moscone West convention center. I'm in the process of changing the session title, so ignore what it says on the website. The new title will be something along the lines of Do Try This at Home: Ajax Bookmarking, Cross-Site Scripting and Other Web 2.0 Browser Hacks.

Webtwopointohexpo

Regardless of the slug, my talk will focus on how to push the latest and greatest browsers to their limits by peering under the hood and finding the quirks, bugs, hidden APIs and partially implemented draft specs you can use and abuse in your Ajax apps. My thesis is that instead of waiting for the Dojo or Prototype or Moo folks to exploit all the hidden goodies in today's browsers, you can go on a hunting expedition yourself if you have the right tools. With Firefox 3 and IE8 on the horizon, now's the time to start hacking.

Ajax history management will, of course, inform much of my talk, thanks to my stewardship of Really Simple History. But I'll also delve into cross-site-scripting hacks, offline storage and other on-the-bubble technologies.

My talk is scheduled for the final time slot on the final day of the conference: 04/25/2008 at 3:50 p.m. (conference schedule here). Any Agile Ajax readers who are attending the conference and plan to stick around till the bitter end should come and check me out.

Gmail, agile development and user experience design

Ionut Alex Chitu of Google Operating System posted yesterday about Gmail's evolution from internal beta to public beta to today's constantly-evolving-but-still-beta version. Gmail's Humble Beginning never uses the phrases "agile software development" or "user experience design." (Nor, for that matter, does the original post, by Gmailer-turned-FriendFeeder Paul Buchheit, from which Chitu liberally quotes.) Regardless, the evolution of Gmail provides a case study in the combination of agility and UxD.

Sample quote from Chitu's post:

Gmail got a delete button after many months of requests from users, even if Gmail's philosophy was "archive, don't delete". Gmail will also add some functionality from folders to its labels, most likely drag and drop.

The key step is to build a product that's interesting enough to a attract an audience and learn from people who use the product. "The sooner you can start testing your ideas, the sooner you can start fixing them," explains Paul.

Continue reading "Gmail, agile development and user experience design" »

Agile Development: Pipelining

Pipeline Sometimes you can roll into an iteration or sprint with a handful of high-level user stories and refine as you go. But if you have a very complex system with lots of interdependencies, or are trying to incorporate a high level of user experience design, or your domain experts aren't readily available to you on a daily basis, then your requirements have to be a little more refined up front.

The answer isn't to go to a waterfall process and bang everything out months in advance. The answer, rather, is to pipeline requirements and development. In practice, the functional team -- made up of Business Analysts and Interaction Designers -- focuses on the requirements for sprint n+1 during sprint n, while the development team focuses on developing the features for sprint n. You can, and should, extend this further: the QA testing team should work on building the automated tests for the features developed during sprint n-1 in sprint n. Anther way of saying this is that the development team is looking at now, the function team is looking one sprint ahead, and the testing team is looking one sprint behind.

While the feedback isn't as tight as if you were to do all of this in one sprint, it's still pretty far away from waterfall. One thing worth pointing out is that this doesn't mean that you do no testing or requirements gathering for sprint n in sprint n, just that the bulk of it happens in the sprint preceding and following the current one. Also, the development team will be involved in the requirements gathering somewhat, but they won't be forced to context switch between what they are working on now and what they will be working on in the next sprint. And they'll definitely get to ask all of their questions in the next sprint planning meeting.

Anyhow, if you find yourself wrestling with trying to jam requirements, development and testing all into one sprint, give pipelining a try.



Technorati Tags: , ,

Agile planning tools.. paper or 'plastic'?

How often have you heard, "Yeah it's not great, but it's the best thing out there", or maybe "We spent two months investigating this tools and there's no turning back. End of discussion. We're not going to switch again!" Does anyone ever hear anything like that when your only tools are whiteboards, paper, pencil and a wiki? What's there to change? The color of paper? Illegible handwriting? How long the walk is from the kitchen to your story wall? These are cheap to fix, and anyone can affect such change more easily when the materials you have to work with don't require logins, passwords, user permissions, mouse clicks, overhead projectors, VGA cables, and a fully charged laptop in a meeting room.

I have used many kinds of planning/tracking tools throughout the years. A recurring belief that crops up all the time is one that suggests that the biggest factor preventing a team from being more agile is the absence of a "proper" agile planning or tracking tool that will satisfy everybody's desire to (you guessed it) be more agile. I find this strange to say the least.

Ron Jeffries said it very well here:

When you're working with a tool, someone owns the keyboard, and everyone else is an observer. It's easy to develop the habit of "check the database" instead of "talk with each other". It's easy for a manager to think that he's managing the group when he's really looking at his screen.

Might a team need a tool like these? Well, yes, they might. But to a team just learning XP or Agile, I'm very concerned that the tool will get in a way.

And now, a few words of advice...

Continue reading "Agile planning tools.. paper or 'plastic'?" »

Ideal team size for your next Facebook project

I recently worked on a Facebook application for one of our clients.  This turned out to be our first collective experience building for Facebook, and it involved a mixture of re-using existing web services and building new ones for use by the same infrastructure.  All in all, a challenging thing to build and deliver given a relatively short deadline.

While there were many architectural considerations at hand, one of the interesting aspects of the project for me dealt with how we structured our team and went about tackling the problem.  As a result, I have a bit of advice on team size for those of you considering similar projects with an existing team at your company.  But first, a little history...

Continue reading "Ideal team size for your next Facebook project" »

Agile User Experience: If you have a silo, knock it down

noSilos.jpg

In many development processes, team members are organized by functional group. A project might have a team of developers building the code, and then down the hall or across the globe, a separate team of designers working on the visual and usability aspects of the application. The two sub-teams are walled off in their own silos, and communication between the two sides is minimal.

This structure is not compatible with an Agile software process -- it's too hard to get rapid feedback between the teams that make a process truly Agile. Worse, if the two teams are separated, the odds that they will have a common conception of the project, or even a common set of terms for discussing the project.

The most important step in tearing down the silos in your project is just switching your mindset from function-based teams to project-based teams. Continuous integration spreads from being just something the developers do to a way of making sure that all aspects of the project are in synch.

At the most basic logistical point, the designers and developers should share a common code repository. There's a certain amount of overlap in the tools used for web applications on both sides. If designers can see the latest version of the application when making CSS or HTML changes, then developers can see those changes and work with the most current design as quickly as possible.

In order for this structure to work well, each side needs to move out of its silo and toward the other. Developers need to have an awareness of usability and design issues, and be able to discuss potential problems with the designers. Developers also need to be willing to do any rework that will be needed as the design changes.

Designers need to be willing to work at least some of the time in the developer toolkit, working with source control, able to make CSS or HTML changes directly in the code files (even better if the designers and developers can make it easy to make tweaks at the JSP or ERB level). Not everything that a designer does can be captured this way, but using common tools and files where available increases the team's ability to work together. Sharing knowledge makes the team more able to notice potential problems, and makes teams more stable over time.


Last week, I wrote about how a user proxy can help integrate Agile and User Experience Design.


Once again, let me mention the upcoming book Professional Ruby on Rails. Available mid-February.

Agile Development, Documentation and Bringing Projects back from the Dead

Books One of the things that comes up quite frequently when talking about Agile development with newcomers to the topic, is the subject of documentation. They hear about "just enough" documentation, "low ceremony" process, and Wikis. To many it seems like "not enough" documentation.

When pressed on what they mean by documentation, they usually respond that what they want is a document that represents all of the specifications and decisions about specifications, the architecture, the design, and the tests that completely describe the system being developed. That may seem like a reasonable request, and I think that a typical agile project, with it's User Stories and various other Wiki artifacts that are produced in each sprint, constitute just such a record.

Continue reading "Agile Development, Documentation and Bringing Projects back from the Dead" »

Agile R&D: If you don't have a client, invent one

I'm working on an internal project at Pathfinder: a cool little Rails app originally developed by one team and now being dusted off and extended by another team. Because this is an R&D project, specifications are a little vague. My colleagues and I spent several hours talking about which direction we should take the app we'd inherited. Each person's idea, of course, was subtly different from everybody else's. By the time we were done, we had a backlog the size of the monster from "Cloverfield" and no clear idea how to turn it into actionable user stories for our first iteration.

Cloverfieldmonsterpicture3

Then our CEO stepped in to act as a proxy client. At our iteration kickoff, we briefly summarized all of the directions in which we thought the application could be taken. He immediately reframed the discussion for us, prioritizing one particular feature that would differentiate our project from other applications in the market we're targeting. Suddenly, it was glaringly obvious what needed to be done first and what could wait until later - or never. Within an hour, we had a wall full of assigned and estimated user stories.

The obvious lesson here is that client engagement is central to the agile method of software development. You can have all of the great ideas in the world, but without somebody to cut through the jumble, you'll end up paralyzed. The less obvious lesson is that minimalism is the key to successful iteration planning. Once you're focused on a specific feature set, pare away anything that's not absolutely essential to getting that feature set out the door. Don't focus on the platonic ideal of your application, but rather on the concrete version of your application that you plan to deliver in the current iteration. You'll probably discover that seemingly essential functionality is actually low-priority - or totally unnecessary.

"Every Meeting I Have Ever Attended.."

A few months ago, I started writing down some thoughts on the purpose of meetings, spurred on by The Laptop Herring.  I went back and re-read my initial comments and realized that I must have just walked out of a particularly unsatisfying meeting when I wrote them down.  My words started out like this:

The purpose of any meeting involving three or more participants falls into
one of two categories:

    1. The dissemination of nuanced information
    2. The agreement on a given course of action

[...]

I then go on to describe what I mean by 'nuanced information'.  I'll spare you the original brain dump, but I think I was on to something here.  Let's tackle that first category, 'the dissemination of nuanced information', as category #2 is really implied in #1 (after all, if you are not all in agreement over something, that should become apparent right away).

Put simply, meetings are not the place to recite facts that can be referenced elsewhere.  They are also not the place to expect attendees to collectively remember things in the hope that something great will eventually come out of it.  They are also not good brainstorming sessions(*).  Instead, meetings should be used as a place to describe things whose usefulness depends entirely on nuance.


nu-ance |ˈn(y)oō,äns|
noun
a subtle difference in or shade of meaning, expression, or sound : the nuances of facial expression and body language.

ORIGIN late 18th cent.: from French, ‘shade, subtlety,’ from nuer ‘to shade,’ based on Latin nubes ‘cloud.’


For example, I always take it as a bad sign when the last words spoken after a presentation are some variant of "can you put this up on the wiki for those who weren't here?"  This tells me that the pertinent information, in electronic format, can be fully understood without the group being present-- and therefore that 90% of the meeting was not necessary in the first place.  It would have been better to write up the reference portion directly on the wiki before the meeting, and meet to cover only the more nuanced parts of the material in question.

Take stand up meetings as another example.  There are a few reasons why I work to keep these as short as possible in any team I am part of, but the most important reason is to help discipline ourselves when it comes to valuing other people's time. Productive meetings don't just happen when you want them to happen.  Timing them is an art, and knowing what bits of information are worth discussing in front of the entire team, and what can be pushed off or condensed down to three words or less requires a few different skills.  A daily ten minute stand up meeting eventually brings that out of you, and if you do the math, you will choose your words carefully in a five person team when you have only two minutes of everyone else's time to say them.

A well-run stand up provides some additional benefits.  A few team members (particularly the ones who learned to hate all meetings early on in life) start wondering how to make other meetings as short and concise as those daily stand up meetings-- good things can come out of this too.

(*) Everything I wrote above applies to what I call large meetings (i.e., more than three participants).  I'm much more flexible when it comes to smaller groups, mostly because I feel that there is less likelihood of wasting time in a group of 2-3 people, and greater "shared responsibility" to keep things on track.  This is why brainstorming sessions or design sessions are best in smaller groups.

2007 in review: Five humble suggestions for better programming and writing

I began 2007 as a front-end tech lead on a multi-million-dollar software project for a global travel company with a massively distributed waterfall development model. I ended it working in small, agile teams on R&D projects at a small outsourced software shop. I got involved in open source, became (yet another) tech blogger, and set in motion lots of other writing and speaking projects for 2008. It's been quite a year.

I'm sure everybody is sick of year-end lists at this point, so let me present a slightly more personal and highly subjective list of the five most important things I learned this year about software development and technology writing:

Continue reading "2007 in review: Five humble suggestions for better programming and writing" »

IA and agility

A friend of mine recently left a position at a supposedly Agile shop where developers wrote code based on raw requirements rather than formal IA. When he described their process to me, I really felt his pain. I can't imagine trying to capture the complex requirements of Ajax applications without the tools of modern IA.

Good information architecture provides not only detailed blueprints for developers, but also the raw material for test cases. If your only documentation of a component is just a high-level business document or a working example of that component, then it's difficult to identify deviations from the spec or prove that an unexpected behavior is a bug. Sure, on Agile projects the first-round IA sometimes gets reverse-engineered from working demo code. But IA and UxD should be a central component of Agile development.

Technorati Tags

The Confluence of UXD and Agile

User experience design is integrated into software development and other forms of application development in order to inform feature requirements and interaction plans based upon the user's goals. The benefits associated to integrating these design principles include:

  • Reducing excessive features which miss the needs of the user
  • Improving the overall usability of the system
  • Expediting design and development through detailed and properly conceived guidelines
  • Incorporating business and marketing goals while catering to the user

--Wikipedia


Consultants lie. They tell you that software X can be developed in time Y for cost Z. That is a lie, plain and simple. The consultant has developed software before, and you know your business, but the two of you have never developed this particular piece of software before, otherwise you would be using it now instead of developing it. So, by definition, in software development you are doing something for the first time, otherwise you would be performing system integration.

This may explain why study after study finds that Software Engineering is notoriously poor at estimating, and why software projects typically run over in terms of cost and time. Enter Agile. Agile doesn't so much eliminate risk as admit that it is there. You don't bother putting together a 24 month project plan because you realize that:

  1. The estimate will be off my 400% or more.
  2. The client thinks they know what they want, but really doesn't.
  3. You'll know way more and make better estimates after 3 months of working on the project.

Continue reading "The Confluence of UXD and Agile" »

GWT as developer tool

Test driven development has advantages all around-- one of them being that it saves you time.  And what's a good use of this time?  Why, coming up with novel ways of testing, of course.  I had one of those experiences the other day thinking of ways to go about profiling memory usage of a web application (by which I mean, the server-side component of a GWT-based application currently in development).

At it's heart, the problem was simple enough: execute a few thousand service method calls with semi-random data, and see what happens to memory over time.  I considered writing a standalone Java application, or culling through existing testcases for some kind of integration test which provided enough coverage to be "interesting", and decided that what I really wanted to do was exercise the system through its front door.  I pulled out JMeter but soon had a realization: "we're writing a GWT app.. why not just write another GWT app to exercise the same services methods as we otherwise do, but in a manner I need?"  So that's what I did.

Continue reading "GWT as developer tool" »

Agile Publishing

I'm currently working on my third technical book (Professional Ruby On Rails, available in February, wherever fine technical books are sold). Each one has had a slightly different process, in chronological order...

  1. I submitted each chapter on completion directly to the Development Editor (DE) in charge of the book. Within a few days, she returned a well-commented revision.

  2. I submitted each chapter on completion, as before. However, the DE did not turn around an immediate review. Instead, at the 1/3, 2/3, and finish points, the partial manuscript was sent to external people for review.

  3. I submitted each chapter on completion. There is no intermediate review. Upon completion of the whole thing, it's sent to a technical editor to review code and a copy-editor for content.

Version three is a classic waterfall model -- all development is complete before any testing happens, and all revision happens in the next phase before the book moves on to the final production phase. Version one is about as close to agile as you'll get in the publishing world (with the possible exception of Pragmatic), and version two is somewhere in the middle.

Well, I got to thinking about these experiences as a kind of controlled experiment, with me as the common element and trying to measure what, if any, difference the process makes to the eventual outcome.

  • The quality of the feedback makes a huge difference. In the first process, feedback came directly from the editor and was detailed and useful. In the current process, feedback is coming from copy and technical editors contracted by the publisher -- it's not quite as detailed about issues of style, but they are quite thorough about, say, when a reader might be confused by the text. In the other process, review was handled by outside volunteers, and we got a lot of "I didn't quite make it through the entire manuscript, but here are some thoughts on chapter one" -- there was very little in these reviews that was useful.

  • I'm not sure whether a more agile process makes a difference in the final quality of the book, but it definitely makes a difference in the efficiency of the review. For example, the copy editor had an issue with a stylistic thing that I did. If I had gotten that feedback early, I could have incorporated it into later writing. As it is, the copy editor is going to have to flag this issue throughout the manuscript. There are similar issues with the technical editor and misinterpreted code instructions. The ending point will be the same, but the editors and I wind up doing some extra work.

  • That said, there's a clear efficiency win purely from the publisher's side in that they only have to manage the review editors for a period of about six weeks, rather than the about six months they'd be on the hook if chapters were being reviewed on turn-in. I don't think it's much of a financial difference -- they get paid for the same amount of work either way, but it's easier for the publisher and editor to schedule in the waterfall model.

  • From my perspective, the waterfall model is terrifying. Since there's no formal feedback on the manuscript until it's complete, I put in months of work before finding out whether it's acceptable. That's a huge potential risk, both for me and for the publisher. As it happens, the feedback has been quite good and there don't seem to be any showstopper problems. Still, I think it would have been nice for everybody to confirm that the road being walked down was, in fact, the right road.

I'm not sure the sample lends itself to big conclusions, but I think it's fair to say the faster iteration process is more manageable and efficient from the author point of view.

Agile Publishing

I'm currently working on my third technical book (Professional Ruby On Rails, available in February, wherever fine technical books are sold). Each one has had a slightly different process, in chronological order...

  1. I submitted each chapter on completion directly to the Development Editor (DE) in charge of the book. Within a few days, she returned a well-commented revision.

  2. I submitted each chapter on completion, as before. However, the DE did not turn around an immediate review. Instead, at the 1/3, 2/3, and finish points, the partial manuscript was sent to external people for review.

  3. I submitted each chapter on completion. There is no intermediate review. Upon completion of the whole thing, it's sent to a technical editor to review code and a copy-editor for content.

Version three is a classic waterfall model -- all development is complete before any testing happens, and all revision happens in the next phase before the book moves on to the final production phase. Version one is about as close to agile as you'll get in the publishing world (with the possible exception of Pragmatic), and version two is somewhere in the middle.

Well, I got to thinking about these experiences as a kind of controlled experiment, with me as the common element and trying to measure what, if any, difference the process makes to the eventual outcome.

  • The quality of the feedback makes a huge difference. In the first process, feedback came directly from the editor and was detailed and useful. In the current process, feedback is coming from copy and technical editors contracted by the publisher -- it's not quite as detailed about issues of style, but they are quite thorough about, say, when a reader might be confused by the text. In the other process, review was handled by outside volunteers, and we got a lot of "I didn't quite make it through the entire manuscript, but here are some thoughts on chapter one" -- there was very little in these reviews that was useful.

  • I'm not sure whether a more agile process makes a difference in the final quality of the book, but it definitely makes a difference in the efficiency of the review. For example, the copy editor had an issue with a stylistic thing that I did. If I had gotten that feedback early, I could have incorporated it into later writing. As it is, the copy editor is going to have to flag this issue throughout the manuscript. There are similar issues with the technical editor and misinterpreted code instructions. The ending point will be the same, but the editors and I wind up doing some extra work.

  • That said, there's a clear efficiency win purely from the publisher's side in that they only have to manage the review editors for a period of about six weeks, rather than the about six months they'd be on the hook if chapters were being reviewed on turn-in. I don't think it's much of a financial difference -- they get paid for the same amount of work either way, but it's easier for the publisher and editor to schedule in the waterfall model.

  • From my perspective, the waterfall model is terrifying. Since there's no formal feedback on the manuscript until it's complete, I put in months of work before finding out whether it's acceptable. That's a huge potential risk, both for me and for the publisher. As it happens, the feedback has been quite good and there don't seem to be any showstopper problems. Still, I think it would have been nice for everybody to confirm that the road being walked down was, in fact, the right road.

I'm not sure the sample lends itself to big conclusions, but I think it's fair to say the faster iteration process is more manageable and efficient from the author point of view.

Agile Publishing

I'm currently working on my third technical book (Professional Ruby On Rails, available in February, wherever fine technical books are sold). Each one has had a slightly different process, in chronological order...

  1. I submitted each chapter on completion directly to the Development Editor (DE) in charge of the book. Within a few days, she returned a well-commented revision.

  2. I submitted each chapter on completion, as before. However, the DE did not turn around an immediate review. Instead, at the 1/3, 2/3, and finish points, the partial manuscript was sent to external people for review.

  3. I submitted each chapter on completion. There is no intermediate review. Upon completion of the whole thing, it's sent to a technical editor to review code and a copy-editor for content.

Version three is a classic waterfall model -- all development is complete before any testing happens, and all revision happens in the next phase before the book moves on to the final production phase. Version one is about as close to agile as you'll get in the publishing world (with the possible exception of Pragmatic), and version two is somewhere in the middle.

Well, I got to thinking about these experiences as a kind of controlled experiment, with me as the common element and trying to measure what, if any, difference the process makes to the eventual outcome.

  • The quality of the feedback makes a huge difference. In the first process, feedback came directly from the editor and was detailed and useful. In the current process, feedback is coming from copy and technical editors contracted by the publisher -- it's not quite as detailed about issues of style, but they are quite thorough about, say, when a reader might be confused by the text. In the other process, review was handled by outside volunteers, and we got a lot of "I didn't quite make it through the entire manuscript, but here are some thoughts on chapter one" -- there was very little in these reviews that was useful.

  • I'm not sure whether a more agile process makes a difference in the final quality of the book, but it definitely makes a difference in the efficiency of the review. For example, the copy editor had an issue with a stylistic thing that I did. If I had gotten that feedback early, I could have incorporated it into later writing. As it is, the copy editor is going to have to flag this issue throughout the manuscript. There are similar issues with the technical editor and misinterpreted code instructions. The ending point will be the same, but the editors and I wind up doing some extra work.

  • That said, there's a clear efficiency win purely from the publisher's side in that they only have to manage the review editors for a period of about six weeks, rather than the about six months they'd be on the hook if chapters were being reviewed on turn-in. I don't think it's much of a financial difference -- they get paid for the same amount of work either way, but it's easier for the publisher and editor to schedule in the waterfall model.

  • From my perspective, the waterfall model is terrifying. Since there's no formal feedback on the manuscript until it's complete, I put in months of work before finding out whether it's acceptable. That's a huge potential risk, both for me and for the publisher. As it happens, the feedback has been quite good and there don't seem to be any showstopper problems. Still, I think it would have been nice for everybody to confirm that the road being walked down was, in fact, the right road.

I'm not sure the sample lends itself to big conclusions, but I think it's fair to say the faster iteration process is more manageable and efficient from the author point of view.

Software Development Is Like A Tortured Metaphor..

Let's kill all the metaphors.  You know which ones I'm talking about.

I was reading through Neal Ford's comments on "Building Bridges without Engineering" and started to groan.  I enjoy reading Neal's blog, and his comments are generally insightful but this rekindled a conversation I had a few days ago on this very topic.  And it seems I keep having this conversation all too often, but from now on, I'm done with it.  As for why, refer to an earlier post by Neal from several years back which I quote below (emphasis mine):

It seems like every time someone talks about the job/craft/calling of writing software, they tend to end up with a bunch of tortured metaphors (engineering, woodworking, rock climbing, gardening, etc.). None of these analogies hold up past just superficial scrutiny. I think that it's a reflection on how different it ultimately is.

I completely agree with Neal here, but he doesn't go far enough in calling us out.  As practitioners, we often groan about the use of such metaphors, but even those who try to negate them still hold on to the belief that, given enough time, the discipline of software development will eventually mature to fit those metaphors.  For something such as software which is more pervasive than bridges, more influential to our daily lives than the automobile, it really says something about the immaturity not of our discipline, but of the software development community that we feel the need to reach for these types of metaphors at all.  Even in dismissing or diminishing the usefulness of the metaphor, Neal, like many others, ends up going right back to the well in his more recent post:

We're currently at the level in software where bridges builders [sic] were when they built a bridge, ran a heavy cart across it, and it collapsed.  "Well, that wasn't a very good bridge. Let's try again".

Whereas before I used to strive to explain software in terms of other engineering disciplines (if for no other reason than to show how they differ) I don't do this anymore.  I don't hold to the teleological belief that, 50-100 years from now, we will finally figure out how the process of developing software somehow conforms to other, more established engineering disciplines.  There is no "figuring out" to do.  A more likely scenario for the future (as sobering as it may be) is that generations of individuals who were not born into a life where software concepts were so pervasive (myself included) will senesce and eventually pass on, and the original need for these metaphor will become irrelevant for future generations.  For all we know, other areas of engineering may just as likely start using software engineering as their metaphor, and argue amongst themselves over why 'X' is not like 'Y' where 'X' is some aspect of study, and 'Y' represents software concepts like iterative development, TDD, the First Law of Distributed Objects, YAGNI, etc.  Strange to think that such a "nascent" discipline as ours could be used to guide people across the next divide, but it could happen.

Metaphors exist for a reason- to bridge the gap between two seemingly disparate realms of thought.  As such, they are still useful and I employ them quite a bit when when mapping software solutions to a non-software problem. But the process by which software itself is developed does not lend itself to any metaphor that I've ever seen put to use.  I've come to find that using other engineering disciplines as approximations is a stale idea.  To try and mold the software process to another form will almost always end in tears for the customer who needs it explained to him in such ways, and is a disservice to those who will ultimately follow in our footsteps.  As far as metaphors go, they probably won't need it anyway (i.e. 'TAGNI').

The Importance of Pair Programming

It turns out I've been following agile principles for longer than I realized. Back in the early 90's, I was living in a grad student coop at the University of Chicago. The house manager -- a 50-year-old physics Ph.D. who handled all of the plumbing, electrical and other assorted house maintenance -- had decided to leave. In a house populated with humanists and social scientists, all eyes turned to me, the lone "scientist." I didn't bother telling them that computer "science" isn't really a science, and to top it all off, I was studying mathematical comp. sci., which meant I was a fair hand at Combinatorics, but I couldn't tell the business end of a soldering iron from a ground wire. Still, I had helped my dad install some drywall back when I was in junior high (mostly I held the drywall in place, while my dad did all the work), so I agreed to be the new house manager.

I set two conditions, however. First, I was not going to live in the house until I was 50, and second, we would have two house managers, whose terms would overlap by 6 months. An early form of pairing. As a result, I and my pair house manager did not feel as overwhelmed as we might have if each of us had been on our own. Further, we shared information, documented it and now, over 15 years after I introduced pair house managing, the coop still uses the practice to good effect.

I know I have mentioned my 4 commandments of Agile development: iterative development; pair programming; test driven development; continuous improvement. It is hard to point to a "most important" one of the commandments -- as far as I'm concerned this is really the bare minimum of what is necessary to practice good agile development -- but pair programming seems to be the one that is most often left on the cutting room floor. The excuses are endless: "we don't have enough people," "we have an odd number of developers," "we have too many tasks to pair." The list goes on and on. For anyone that has practiced pair programming, the only objection that holds water is the "odd number" one. It is hard to pair with 3 developers, and rotating pairs or three-somes don't work so well -- someone is always the odd man out. ;-)

So what does pair programming give you and why and how does it work? In their hilarious article All I really need to know about pair programming I learned in kindergarten (May 2000, Communications of the ACM, Volume 43 Issue 5), Laurie Williams and Robert Kessler describe some of the reasons for its success. First, in a survey of developers they found that overwhelming majorities of developers were more confident of their solutions and enjoyed their jobs more. Happier programmers produced better solutions, no surprise there. But the fact that stereotypically introverted developers would enjoy a more social, collaborative work environment seems counterintuitive.

According to the article, another key benefit of pair programming is the constant code review that it provides. As it turns out, developers working in pairs think of twice as many solutions and work more than twice as fast as two developers on their own. Anyone who has seen the parabola of bug fix expense in action, i.e. bugs are much more expensive to fix the further you get toward deployment, will appreciate how valuable this constant code review is.

The article goes on to describe some best practices for pair programming, couched in kindergarten term -- "share everything," "clean up your mess," "play fair," "don't hit people." One principle that I adhere to, and that the article mentions, is that it is permissible to work alone for up to 50% of the time -- "nap time." I'm not really an advocate of the "one monitor, keyboard and mouse approach," but rather prefer the shared desk model. That way you can collaborate, but are not exhausted by the constant engagement that the shared terminal forces on you.

One additional benefits of pair programming that I've observed is that developers reach their potential more quickly, i.e. they move from junior to senior to architects or leads more quickly. They learn from others (got to keep switching the pairs from iteration to iteration), they get to make decisions on design and architecture instead of taking dictation from a tech lead, and they feel empowered. No wonder they progress and learn more quickly than developers working along.

Back in the mid to late 90's when I first tried to introduce agile methods on Wall Street, pair programming was the practice that garnered the most pushback. I usually convinced skeptical managers by convincing them that developers would spend less time surfing the web. They nodded knowingly and agreed to give it a try. I got my way, but for the wrong reasons -- managers didn't trust their developers. That points out another agile principle that fits hand in glove with pair programming: trust your people. Or, as the Agile Manifesto puts it

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

These days, I have a much easier time of selling pair programming to clients, which is all to the good.


Technorati Tags: ,

Layered architectures & other abstractions.. revisited

Speaking in terms of abstractions can cause a peculiar kind of paralysis to occur by letting developers avoid the specifics longer than necessary.  You might have been part of a discussion in the past where your team struggled to define the names of each layer in a system or wrestled with the impedance caused by conflicts between the logical versus physical aspects of layers in an application.  These are symptoms of what I am referring to.

Suppose, for example, you have the following system-- business logic contained in a service layer, and a less well-defined set of classes that communicate with some external system(s).  The fact that the classes in this latter group serve a common function may let you talk about these classes as comprising a distinct layer in the system-- but it is just as likely that concerns such as transaction management, persistence or packaging considerations drive the need to make exceptions to an otherwise clear-cut boundary between these two layers.

As the code matures, the service layer may start to know too much about specific components in the newer layer, and this new layer may start to rely on knowledge of (and code for) certain paths through the service layer.  This may sound like a recipe for disaster, but often this kind of coupling is just an unintended consequence of developers realizing that the most expedient way of working with the system runs against the grain of the layered architecture-- which itself is an odd thing to confess, since one measure of a layered architecture's worth is in how well it provides a path of least resistance to normal development.

Two camps emerge when this kind of problem surfaces.  On the side of expediency, team members begin to suspect that maintaining a strict separation between these layers requires too much overhead either during the design of new features, or in refactoring of the existing code base-- they push to allow the layers to co-mingle.  In the other camp are developers who believe such co-mingling should not happen (to take the purists perspective), or that it is inherently unsafe (future-proofing the architecture against factors they expect to change, such as transaction management, clustering strategies, security considerations, etc).  Oddly enough, the one thing everyone seems to agree on is that the distinction between these layers should still exist-- at least on paper.  So instead of changing their definition of what constitutes a "layer" proper, they either rationalize the need for exceptions to the rule, or enforce the separation of layers despite the added cost of maintenance.

Rather than discuss all the kinds of arguments that arise in these cases, I believe it suffices to use the example above as in introduction to something more fundamental.  This more fundamental issue deals with why and how developers place such a high level of importance on abstraction in the first place, even to the detriment of otherwise good design.

Abstractions reflect the way we like to think about the construction of a system once it is in place, but I don’t believe they always reflect how teams most effectively work and live within those systems once they are built.

Take object-relational mapping frameworks for example.  We like to think that a good object-relational mapping framework shields developers from having to think about the database-- and, during the inception of a project, this is normally true.  But as the application matures, the number of caveats in the system increase-- changes to the relational side occur in lockstep with changes to the object model, and quirks on one side need to be understood, even if they don't directly manifest themselves on the other end as problems. The less an object model is explicitly bound to a database implementation, the more likely the team itself needs to become well-versed in how the underlying ORM works, and must therefore retain this knowledge throughout the life of the application.  So, with this additional level of abstraction, you have effectively changed what used to be a code maintenance tax for the life of the application to a more subtle, but (in some cases) costly mental tax on the developer for the life of the application.

Don’t get me wrong-- this is not to say that abstraction itself is a bad idea, nor that layered architectures are inherently bad.  Rather, I believe these kinds of abstractions work best when the cost on the development team for the life of the project (which, unless you are lying to yourself, is never ‘zero’) is less than the hypothetical cost of maintaining a larger code base by reducing the number of abstractions.

Rails, Ajax, RJS, and Testing

As promised, today’s entry is Agile and Ajax. Also, it’s a dessert topping.

Ajax in Rails

The initial support for Ajax calls in Rails was centered on two framework methods called link_to_remote and remote_form_tag. The basic functionality of these methods is to allow a remote call to be triggered by a link or form submit and have the result of that call be used to update a DOM element somewhere on the page.

Moving outside that simple behavior, however, quickly got messy. Both methods specify a series of callbacks where arbitrary JavaScript can get implemented at various points in the call life-cycle. However, writing dynamic JavaScript in an ERB template is awkward, and the resulting method calls could get ugly.

Enter Ruby JavaScript (RJS), sometimes described as “training wheels for JavaScript”. RJS allows a very simple subset of JavaScript to be written in Ruby and translated to JavaScript as the result of a Rails Ajax Request.

Here’s what a sample RJS template looks like:

page.visual_effect(:fade, 'recipes_to_show',  :duration => 0.5)
page.visual_effect(:fade, 'category_being_shown', :duration => 0.5)
page.delay 0.5 do
  page.replace_html("category_being_shown", 
    "Recipies For: #{@category.capitalize}")
  page.replace_html("recipes_to_show", :partial => "recipes")
  page.visual_effect(:appear, 'recipes_to_show', :duration => 0.5)
  page.visual_effect(:appear, 'category_being_shown', :duration => 0.5)
end

The page variable is automatically provided by Rails and represents the page that is going to receive the JavaScript. There are about a dozen or so instance methods that page can receive, mostly having to do with the basic Ajax-y features of replacing HTML in a DOM element and calling Scriptaculous visual effects. This particular snippet fades out two elements, changes their text, and fades them back in for a crossfade effect.

(There’s a certain similarity of concept with Google Web Toolkit, but RJS is going after a much smaller and more focused piece of functionality, optimized towards what you would do in a single Ajax call in an otherwise standard interface. GWT, on the other hand, is trying to be the entire application on both the client and server side.)

How useful this is depends on your relative level of comfort with Ruby and JavaScript. I’m personally much more comfortable in Ruby, so I think this is just great. It’s especially nice since I can use Ruby blocks to abstract this crossfade function into something I can use generically.

def crossfade(page, *dom_ids)
  dom_ids.each do |dom_id|
    page.visual_effect(:fade, dom_id, :duration => 0.5)
  end
  page.delay 0.5 do
    yield
    dom_ids.each do |dom_id|
      page.visual_effect(:appear, dom_id, :duration => 0.5)
    end
  end
end

The generic function takes a list of DOM ids and a block — the assumption is that all the content changing will be managed in the block. So the original snippet would now be changed to:

crossfade(page, 'recipes_to_show', 'category_being_shown') do
  page.replace_html("category_being_shown", 
      "Recipies For: #{@category.capitalize}")
  page.replace_html("recipes_to_show", :partial => "recipes")
end

If you’re feeling more adventurous, you can patch the crossfade method directly into the JavaScriptHelper class and change the call to page.crossfade, making it more consistent with the other calls performed in an RJS template.

Testing RJS

Now for the Agile portion — how do you test this thing? RJS templates can be tested pretty thoroughly based on the content of the JavaScript code being generated. It’s much harder at the moment to test based on the actual results of the JavaScript — for example, it’s hard to test that the DOM elements being referenced actually exist in the client page.

There are two mechanisms for testing RJS that I’ve found useful. The first is mock object testing. Using the flexmock package to set up the page as a mock object, a sample test for the crossfade method looks like this:

def test_crossfade
  page = flexmock("page")
  page.should_receive(:visual_effect).with(:fade, "dom_1", :duration => 0.5).once.ordered(:first)
  page.should_receive(:visual_effect).with(:fade, "dom_2", :duration => 0.5).once.ordered(:first)
  page.should_receive(:delay).and_yield.once.ordered
  page.should_receive(:replace_html).once.ordered
  page.should_receive(:visual_effect).with(:appear, "dom_1", :duration => 0.5).once.ordered(:last)
  page.should_receive(:visual_effect).with(:appear, "dom_2", :duration => 0.5).once.ordered(:last)
  crossfade(page, "dom_1", "dom_2") do
    page.replace_html()
  end
end

Without going into the details of flexmock because, hey, future blog post, the idea is that the mocked page object keeps track of the method calls it receives and checks them against an expected set of method calls. In this case, I’m telling the mock object that it should get two calls to the visual_effect method with various arguments, and that those calls should come before the other calls.

The interesting thing about this method is that on the face of it, it contains no assertions. Implicitly, though, each should_receive call sets up an assertion about the messages coming to the object that is validated at the end of the test.

The other mechanism uses a plugin called ARTS (Another RJS Testing System), which defines a method called assert_rjs, which checks the outgoing JavaScript for a method call matching a set of parameters.

Using ARTS, the following method tests whether an individual DOM element is in the crossfade:

def assert_crossfade(dom_id, replacement)
  assert_rjs :visual_effect, :fade, dom_id, :duration => 0.5
  assert_rjs :visual_effect, :appear, dom_id, :duration => 0.5
  assert_rjs :replace_html, dom_id, replacement
end

And a full test of the example would look like this:

def test_one_crossfade
   assert_crossfade 'recipes_to_show', /Recipe/
   assert_crossfade 'category_being_shown', /Category/
end

The replacement argument can be either a string, in which case the output needs to match exactly, or a regular expression, in which case a Regex match is performed. In general, assert_rjs works by recreating what the JavaScript looks like from the assert call, and checking the actual output for the existence of that call. Again, this is most helpful when you can bundle calls together in a single assertion.

ARTS is a nice little plugin, and it does let you do some syntactic testing of your RJS templates, but semantic testing along the lines of, is the recipes_to_show element visible at the end of the RJS and what text is actually in the element is still elusive. I think that’s doable, but it would require a much more complex mock object representing the page, one that keeps track of a pseduo-DOM tree and can manage at least some of the effects of RJS calls. Another project for another day…

Lessons Learned From Domain Modeling

There are many lessons I've learned as a result of working in a more domain-driven way.  My recent work on the UI front has highlighted a few ideas that are just as applicable to other areas of application development (outside the sphere of, say, modeling a complex domain).  Here are three things I believe developers should always keep in mind when designing or implementing code.

  1. Hierarchies require coordination between their roots.
  2. Uniqueness adds complexity.
  3. Null holds meaning, but it's meaning will never be consistent.

Like learning how to write a good unit test, these are simple concepts to explain, but each requires patience and experience to understand really well. (a good pairing partner never hurts either!)

Hierarchies Require Coordination Between Their Roots
A graph of objects often needs to interact with another graph of objects, and while it is initially faster to build direct dependencies between members of each group in an ad hoc fashion, in the long run you benefit from controlling access to those members through the root of each group/hierarchy.  I see this as an extension to the principle of encapsulation applied to hierarchies of objects (ala: Law of Demeter). 

Consider the general case of one object acting as an observer of another.  If there are notable side effects in the handling of an event up or down the graph, it makes sense to put the root in control of the event.  Refactoring across the hierarchy become easier, adding additional behavior introduces fewer edge cases (thus reducing the need for more complicated testing), behavior more predictable etc-- the only exception (as I see it) would occur when the type of messages being passed around do not affect state at all.  Which brings us to our next point:

Uniqueness adds complexity
Stateless systems are invariably easier to grasp and comprehend than stateful ones.  While stateful systems have a temporal dimension to them which is largely absent in stateless ones, this is not the only source of such complexity.  It is the interaction between two or more stateful systems where we begin to see the huge cost in terms of managing complexity (recall your first experience debugging issues in badly written multi-threaded code to remind yourself of this).

This is why value objects are so important as a concept, and why their use is so prevalent in domain-driven design.  While the identity of a value object is determined by it's state, it's state is invariant, so it does not have to be managed the way an object with variable state needs to be managed.  But the use of VO's in domain-driven design is just one manifestation of the broader concept: always strive to reduce the number of parts in an application which rely on uniqueness (even if this requires a few more parts added to the whole), whether you are writing domain logic, UI behavior, etc.

The Meaning Of Null Is Never Consistent
It is easy to miss this point, as witnessed by how often I see code which uses null as a substitute for "I Don't Know...", "Shouldn't Really Happen...", "Just Satisfy The Compiler..." or "This Is A Placeholder".  In reality, null (or nil) is often employed as a mechanism for passing the buck to someone else.  Try as you might to ascribe a single meaning to it for all cases in your API, client code will feel free to redefine this meaning on a case-by-case basis, and its meaning will evolve (or devolve, as the case may be) into code which looks defensive, but behaves unpredictably.  Avoid returning null when possible-- strive for clarity instead.  While in other languages (such as Ruby) it is possible to add behavior to the concept of null, thereby making it's contract with the rest of the code a bit clearer, I'd still choose the alternative, and use something else.

That's it-- three little things that come in handy just about anywhere. My initial intent was to write a line or two for each section, but as you can tell, there is so much to say on each topic, and all of this just scratches the surface.

Agile, The Control Paradox, and the Boring Software Manifesto

I'm probably not getting the quote quite right, but supposedly at one point during the C3 payroll project where XP was created, a developer said that it was so easy to make changes to C3 that they could turn it from a payroll system to an airline scheduling system without any strain.

Okay, it's not much of a story, but the idea resonates with the core of what Agile development means: driving the cost of incremental change so low that you can handle whatever the client or customer throws at you at any point in the process.

Gaining Control

There's two parts to that statement, of course. The first is that it's great to be able to easily fix bugs and make changes to the software over time. The second is that it's even better to be on a project that you feel is under control.

This is the paradox of being Agile. Agile dispenses with many of the artifacts that traditionally symbolize control in a development process -- big requirements documents, detailed UML diagrams, long-term Gantt charts, that kind of thing. At the same time, I've never felt more in-control of the actual day-to-day work then I do when in the midst of an Agile process. By accepting the inevitability of change in a software project, Agile allows the development team a structure by which the effect of changes can be mitigated.

The Boring Software Manifesto

Several years ago, I coined the phrase The Boring Software Development Process, in response to a former employer where project management really didn't think anything was happening unless we were trying to solve seven crises simultaneously.

The manifesto goes like this:

  • Boring software projects favor tacking exciting problems, and focusing our energy on the most interesting and valuable parts by avoiding wasting time on avoidable problems.

  • Boring software projects favor automated test suites over the excitement doing all your testing at the last minute and finding bugs after the project is "done".

  • Boring software projects favor frequently integrating work from the whole team as opposed to the excitement of finding out whether you changes work well with the rest of the team after you have been working on them for six weeks.

  • Boring software projects favor always having a working, if incomplete, system over crossing our fingers and guessing whether the newest build would compile.

  • Boring software projects favor incremental design rather than spending six months crafting a UML document that is thrown out after one week of development. (I'm not exaggerating on this one, I reviewed a project that happened on)

  • Boring software projects favor understanding that requirements change over being "surprised" when late change requests come.

  • Boring software projects favor putting working, if incomplete, systems in customer hands early to get feedback as early in the cycle as possible rather than finding misunderstandings after the project is done.

I hope that wasn't too boring... Next time, something a bit more nuts and bolts.

Pair Programming with VNC

A little Agile to go with the Ajax...

My four commandments of Agile Development are:

  1. Short iterations
  2. Pair programming
  3. Test driven development
  4. Continuous improvement (sprint reviews, etc.)

I have strong opinions about continuous integration and related agile practices, but those can be negotiable. Not all methods are appropriate for all teams, projects and clients. But the four above allow each team to find its own velocity and process.

Offshore Pair Programming

But how to do pair programming with an offshore team? There are a number of inherent problems with making this work:

  • Not enough schedule overlap with places like India and the Philippines
  • Uneven network performance
  • Language difficulties
  • Telecom difficulties

None of these issue is insurmountable. If you are offshoring significant software development, you likely operate at a scale where you can afford decent bandwidth, firewalls and telecom systems. Don't skimp here by trying to run Skype on already overloaded laptops. Give each onshore and offshore developer their own direct dial number.

For the schedule overlap, I've found that a 4 hour overlap is sufficient, though 6 is better. For a twelve hour time difference, one team can stay late while the other gets in early. Onshore and offshore can alternate on who gets in early, etc. Or you can go with the former Soviet Union (FSU), which has less of a time difference.

Finally, language difficulties can be a problem whether you are practicing pair programming or not. Rather than exacerbating the issue by placing an emphasis on verbal communication, in my experience it actually improves communication  between team members by getting them more used to one another's accents. Immersion courses work for teaching foreign languages, why should it work for the less difficult task of understanding a foreign accent?

How to Pair

Now in an onshore setting, my company rarely practices the "one mouse, one monitor, one keyboard" brand of pair programming. Tasks are shared, the developers sit at one desk and collaborate, but they have their own laptops. This differs a bit from the traditional definition:

Pair programming requires two programmers to participate in a combined development effort at one workstation. Each member performs the action the other is not currently doing: for example, while one types in unit tests, the other thinks about the class that will satisfy the test.

The person who is doing the typing is known as the driver while the person who is guiding is known as the navigator. It is often suggested for the two partners to switch roles at least every half-hour or after a unit test is made. It is also suggested to switch partners at least once a day.

But in an onshore/offshore situation, where the partners are tied together by a phone line and a VPN, the shared workstation is really the only feasible way to make pair programming work. Anything else, phone, IM, email, even video conferencing, is awkward and unproductive.

In my firm, we've tried various approaches to shared workstation pair programming, such as Windows Remote Desktop, but we've come back to the one that performs the best and presents the fewest obstacles: VNC (Virtual Network Computing). VNC basically (see here for a version that is efficient for slow connections) gives you remote control of a desktop. That's from and to pretty much any Windows and Linux/Unix/X11 machine.

We've tried the approach of having one developer hooking up to the other developer's laptop, but that leads to poor performance. A far better approach is to use a powerful server class machine and have both developers access it via VNC's screen sharing. (See here for one group's VNC setup, though they do it locally.) The idea of using a powerful server pairing station is nice. It recognizes that running something like Eclipse or Visual Studio, and application server and VNC is somewhat too heavy a task for a humble laptop.

Combined with periodic offshore/onshore personnel visits -- which build up trust between developers -- the shared workstation model of pair programming is something I would encourage anyone combining onshore and offshore development teams to try.

References


Technorati Tags: , ,

Consider Each Refactoring a "Tiny Experiment"

On the cab ride into work this morning, I read a post on the very excellent Language Log regarding a talk by Jon Bentley on "Tiny Experiments for Computing and Life".  While I have never had the opportunity to hear Jon give a talk, I did find some slides from an older presentation on "Tiny Experiments for Algorithms and Life" (ppt).  The slides are worth the quick read.  It gave me a few ideas for more tiny experiments of my own, but it also brought me to consider ways in which I perform these kinds of experiments when working with software-- and that led me to how I refactor.


Mapping The Problem

I paired with a developer a while ago who was surprised to find that, when tackling an especially ugly piece of code, I tend to "program/test/revert" several times within a matter of minutes before finally settling on the right approach for tackling the problem.  The problem was a particularly nasty one, and I'm sure you've been there before-- adding a small feature and disliking everything about the experience.  You quickly dig yourself into a hole shoring up every poorly named method, duplicate snippet of code, hard-coded configuration parameter and obsolete comment you encounter along the way.

Personally, I dislike juggling a large number of changes at one time.  When faced with a difficult coding problem, I start with the assumption that I will dig myself into a few holes-- therefore I try to find these holes as quickly as possible.  My real intent is to map the problem at hand by performing a few coding experiments.  If successful, the result will be a sequence of independent refactorings that, applied sequentially, provide a path of least resistance for my new piece of functionality.

Write Once, Revert Often

Like a "Tiny Experiment", this phase of development is less refactoring than leading up to the process of refactoring.  It usually does not last long either:  anything from a few minutes to an hour at most.  If it takes longer than an hour, you should stop to rethink your assumptions.

Since your first attempts at mapping the problem may break a large number of tests or highlight bigger issues in the architecture, it should be very cheap and easy to revert your entire workspace at a moment's notice and start over.  Because you have not changed much code or stopped to add things like comments at this point, you are not heavily invested yet.  Neither you nor your pairing partner should even blink at the thought of rolling back all your changes when you start encountering issues.

I can't say enough about being able to revert your changes quickly-- navigating menus take too much time.  I have a two-keystroke shortcut configured in my IDE which is quick, painless and blissfully unforgiving.  To show why this is so important, imagine that you have made some progress mapping the problem, and find that the solution is actually in reach.  At this point, however, you find several unrelated tests failing after you have changed the code.  Do you stop and fix the tests to conform to your changes (thereby increasing your investment in this particular solution), do you circle back to double-check your changes so far were correct (still increasing your investment in this particular solution), or do you revert the changes and investigate the tests first?  I would revert all changes and investigate the "seemingly" unrelated tests and how the two pieces of code (your changes and the code under test) are coupled.  Without understanding the dependencies in the pre-existing code, your entire work estimate may need to be reconsidered, and that could stop your progress right there, so it's better to know that now.  Moreover, you might discover that decoupling the code is a separate task that might be worked on in parallel.

And don't underestimate how important it is to be able to revert on a dime-- to stop in mid-stream and be selective about what changes to revert takes too long and sucks any fun out of the process (the same goes for commenting out or uncommenting code to get back to a safe state-- who has time for that?)  Make use of those keyboard shortcuts!  Don't let your pair sit idle while you right-click your way to sub-menu purgatory.  I'm serious!

Testing Your Hypothesis

Each refactoring itself should be thought of as a kind of tiny experiment.  Form hypotheses around questions such as:

  • "What will this piece of code look like if I rename this method?"
  • "Is the introduction of a parameter object really necessary for this method, or is the method signature clear enough as is, and unlikely to evolve?"
  • "If I promote this method up to the interface, how will that impact dependencies on other implementations of the same interface?"

You get the idea.  While some such refactoring experiments may fail, even these failed attempts can be considered constructive if they serve to highlight some facet of the code base which previously went unnoticed, or larger architectural changes which need to be formally scheduled and worked on in future iterations.

The Rails Edge: Quotes and Notes

Last week I attended Pragmatic Studio’s “The Rails Edge” conference in Skokie, IL.  I’m going to do a couple of posts on this. For the first post, I thought I’d present some of the best quotes that I noted from the speakers over the three days.

“Java is the blunt scissors of programming”
  — Dave Thomas, Pragmatic Dave himself, in a talk about Metaprogramming Ruby, and alluding to the way Java was explicitly designed to prevent a programmer from changing the program context..

But, if a Ruby programmer can change existing classes, can’t you shoot yourself in the foot? That’s not a Ruby problem…

“That’s a people problem. It’s solved by four feet of rubber hose in the car park”
  — Dave Thomas, who presumably was kidding. Kind of.

Later…

“The key to metaprogramming is understanding self. Isn’t that the key to life”
  — Dave Thomas, exploring the parallels between metaprogramming and metaphysics.

Moving on…

“I hate using the words ‘best practice’”
  — Chad Fowler

“Unless you understand the SQL code that Rails produces, your app will suck.”
  — Chad Fowler

“Metaprogramming + DSLs is the Ruby equivalent of Design Patterns in the Java world”
  — Chad Fowler. Fowler’s point here was more about the buzz and hype, just like there was a time in the early 2000s when every Java programmer wanted Design Patterns whether or not they were needed, Fowler sees a similar rush to add DSLs to Ruby programs.

“Model/View/Controller apps are like GOTO for the web”
  — Avi Bryant, quoted by Fowler.

In general, the shortcomings of Rails view support were an ongoing theme,

“I think it’s sad that Rails is still using ERB”
  — Chad Fowler

One of the best talks was by Ezra Zygmuntowicz about Rails deployment. The talk was introduced thusly:

“Rails deployment made you feel dumb. It was hard and then it still didn’t work”
  — Mike Clark. To be clear, Clark was emphasizing the past tense in that statement.

Zygmuntowicz, quickly moved to establish his credentials…

“I’m running, like 7000 Mongrels right now”  
— Ezra Zygmuntowicz

Later, Justin Ghetland presented on JRuby, proclaiming that it has

“The elegance of Java, the speed of Ruby”
  — Justin Gehtland

“In order to win companies need to unleash the creativity of open source, without fetters”  
— Justin Gehtland, discussing the difference between Sun’s handling of JRuby and Microsoft’s handling of IronRuby

“Any time you try to impose a hierarchical order on the world, you’re bound to be disappointed”
  — Dave Thomas, commenting from the stands during a later talk on REST and its strengths and weaknesses

One of my favorite talks was from Stuart Halloway about Agile, Enterprise, Ruby, and Rails. A boatload of good quotes here, I’m going to skim off my favorites.

“The right process is always ‘not quite enough process’”
  — Stuart Halloway

Stuart continued by saying that if you are using note cards for tasks and everybody kind of things that maybe they should use an Excel spreadsheet, then you’re probably in the right place, but if it’s causing real pain, you need to change.

“Do the dumbest, simplest thing that almost works”
  — Stuart Halloway, on process

Dave Thomas chimed in from the cheap seats…

“The traditional view, with sixteen pounds of documentation, introduces a single point of failure in the process, understanding the problem domain”
  — Dave Thomas

“Getting a specification involves bullying the customer”
  — Dave Thomas

Halloway then talked about enterprise development:

“What makes something enterprise? My favorite definition is to add two zeros at the end of the price”
  — Stuart Halloway

Halloway went on to justify the increased cost because of the increased amount of security and specification that an enterprise customer needs.

“What does scalable mean? [Pause. Nobody answers.] Ha. Ha. Rails programmers   don’t know what scalable means”
  — Stuart Halloway

The definition of scalable Halloway used was a linear increase in complexity as size increases. Therefore:

“Rails is slow. But it’s scalable.”