I like programming. When I don’t get to program for a while, I feel a certain restlessness. It’s the same with reading books, and physical exercise: when I get too little of it, I feel it.

Most of the time my work fills that need. When I get home and Ingrid’s gone to bed, I’d rather blog, read a book, etc. My hobby programming projects languish untouched. I haven’t done any work on my Rails app for over a month.

But this week I have barely touched a line of code at work. Instead I’ve been installing a new build-cum-version control-cum-database server, migrating all of those things from the old server to the new one, plus getting our new developer up to speed, doing various administrative tasks, and handling customer support issues.

By Thursday I was definitely feeling the pangs of abstinence, and my Rails app finally got some attention again. Luckily it’s a small app so it isn’t too hard to pick it up again. The hard part is remembering whether I had some unfinished task lying around, or whether I can just start on the next interesting-looking piece. I’ve taken to checking in a “next step.txt” in order to keep track of where I was.

An unconference similar to the previous ones I have described has been scheduled for September 12th. Read more here and join us if you can.

We’ve started work on a new help system at work. The old one uses a CMS from 2001 and we all loathe it. “Updating the help files” is the least popular task of each release. We’ve had enough; we’re starting afresh with Drupal.

So I’ve spent today setting up a Linux VPS, configuring the web server, installing Drupal and all sorts of plugin modules for it. And it’s a painful experience. Every step I take is based on something Google tells me. I have no feeling for what I’m doing, no intuition as to whether it’s dangerous or not. So with every other step, I’m wondering – if I get this wrong, how much of the system will blow up?. How wrong am I allowed to get my changes in httpd.conf before the web server will stop responding at all?

It gives me an all new perspective on what it must feel like to be an inexperienced computer user. Afraid to click on anything, afraid to change a setting, just in case they make the computer blow up.

Another perspective on this is if it hurts do it more often. It was not meant quite in this sense originally but it still makes sense. Unfortunately it’s very rare that I have a meaningful need to tinker with Linux sysadmin tasks or web server settings.

Out of all the talks and discussions at Agila Sverige 2009, the most useful tidbit I took with me was this:

Evolution = Variation + Selection

The equation is obvious, and yet it was useful to have it spelled out like that. The moment I saw it on the screen, I knew it was just what our Agile project needed right now.

I’d been thinking for some weeks now that our project felt a bit stagnant. Everything was going well, we were delivering good stuff, the process was flowing smoothly, but there was something missing: change. And if there’s no change, there’s no improvement, and I get restless.

We’d had our regular retrospectives, and we’d always come up with some points for improvement. But I felt that we were often saying the same thing every month, and that any adjustments we made were small, on the margin. Looking back (with that equation in front of me) the reason is obvious. We had a good deal of selection (critical evaluation) but far from enough variation. We need to experiment more, change our routines and our processes more (more often, more extensively, more deeply, more creatively) and see what happens.

The first thing we’re going to change (after we all get back from our vacations) is try 1-week sprints/iterations instead of 4-week ones. And I’m going to spend a lot of time during the summer doldrums (when we’re between iterations) thinking up more variations to try.

I spent Monday and Tuesday at Agila Sverige (Agile Sweden). A lightweight and agile event, as befits the topic, somewhere in the continuum between the ALT.NET unconferences and “real” conferences. There was a small attendance fee, and meals were provided, and there were printed programmes, but the structure was like an unconference – lots of scheduled lightning talks in the mornings, and Open Space sessions in afternoons.

As with the ALT.NET events, I came home not so much with facts or techniques or tips, but with feelings of inspiration and energy. While it would have been nice to find answers, it is also nice to at least know that others are struggling with the same questions (such as fitting testing into short iterations, or adjusting Agile & Scrum for use in a very small team). The term mutual admiration society may be derogatory, but sometimes a bit of mutual admiration is what we need to energize us.

The most interesting discussion was a philosophical-political one. I cannot recall the original label for the session, but one of the topics we ended up discussing was the parallels between some dynamics of the Agile movement and “real-world” revolutionary movements. Liberation movements themselves are often very illiberal, even autocratic, and it can be hard to shift from leading a liberation movement to leading a liberated population. Very different qualities and behaviours are required.

The Agile movement is a kind of liberation movement, aiming to free developer teams from the burden of excessive management, procedure, planning and documentation. But when the message trickles through to the masses, the original leaders think the message has been distorted, lament the degradation that their lofty ideas have undergone, and start talking about the need to restart the process under a new name and a new manifesto. And the masses get upset that their leaders seem to be abandoning them.

The Agile movement is a principles-based movement. It’s not so much a method as a philosophy. (Aren’t real-world revolutions the same?) And for the leaders of the movement, principles suffice. But when the idea spreads (or when you try to spread the idea) from innovators and early adopters on to the early majority, it starts to seem that principles are no longer enough. The target audience now includes people who do not think much about principles – because if they did, they would be leaders or early adopters themselves. So processes and methods and rules start to agglomerate. “If you’re not doing TDD then you’re not doing Agile,” and so on. So there is an inherent conflict between the foundation of Agile software development (trusting people), and the desire to spread it to the bulk of the developer population (whom you don’t trust to do the right thing on their own).

(This, of course, raises a new question: should we even try? Would it be desirable to have the majority of projects run in an Agile manner? Or, as another speaker put it, are we running out of Agile developers? Can any developer work in an Agile manner, or does it take a certain kind of developer, of which there is a finite number?)

The issue of leaders abandoning a movement once it reaches some level of maturity is also an interesting one. (Lean development is one of the new new things that people orient towards when they get enough of Agile.) There are at least two forces conspiring to make it so. Firstly, thought leaders are rebels and/or innovators. In order to feel happy they need to feel that what they’re doing is new and fresh, and not mainstream. When their idea becomes mainstream, they must find something even newer. Secondly, the leaders will naturally continue to develop and refine their own ideas. They won’t stop just to wait for the rest of us to catch up. So by the time we get to Agile, the people who started Agile will again be ahead of us.

The Rails learning project continues. I’ve got an actual application to work on, one that will be useful when it’s done: a web-based version of our expense tracking application. The old one was done in Access and has served us well over many years. But file sharing has always been a pain since we often have different OS versions (and now different OSes) on our two computers, and can’t be bothered with hooking up external hard drives etc. So we want a solution that we can both access more easily. Thus, web, and Rails.

It’s fun and educational but it is also painful. The trivial tasks are trivial, but the moment I try to do something other than plain CRUD with strings and numbers, I get stuck. Every task takes me a whole evening. At this rate the application won’t be done until Christmas.

Today’s stuckness: time zones. I want to be able to migrate the data from our existing database, and CSV files seemed like a sensible medium. Exporting from Access to CSV went easily. Importing text went well enough, once I figured out that I needed to open and resave the file with different encoding. But importing dates did not. My data file had an entry for 1998-01-01 and after importing it somehow became 1997-12-31 23:00. When I got that fixed (by changing the application’s time zone to UTC) the data looked right but whenever I re-imported the data file, the dates were not recognized as matching what was already in the database, so the records were appended rather than updated. It turns out that in SQLite a date without time is not the same as a datetime with time = 0. Who comes up with this stuff?

In a way, Working Effectively with Legacy Code does what it says on the tin: gives advice on how to work with legacy code. But while the title is technically correct, it is also misleading enough to be harmful.

After all, I have been working with legacy code without any help, and making progress cleaning it up, too. So why would I need a book? What can it possibly do to help, other than commiserate and advise me to refactor? For a long time I dismissed the book as “probably won’t help me much in practice”.

Well, it turns out that the book is not really about how to work with legacy code. Michael Feathers has his own, very specific, definition of legacy code: for him it means code without tests. And he also has a very specific definition of working with legacy code: for him it means adding no more untested code. Whenever a change or a new feature is needed, the affected parts of the code base need to be made testable first.

Redefining the problem from “working with legacy code” to “making untestable code testable” suddenly makes the topic appear very useful indeed, and the book turned out to be excellent. Like numerous other reviewers, I wish I had read it a year ago!

It’s full of practical advice, with lots of great examples, rather than high-level principles. Very pragmatic, where other software design veer off into idealism: “you often need to make your design worse before you can make it better” was a recurring theme here that I really sympathise with. It was a very inspiring rbook, making my fingers itch to refactor. On top of that it’s interesting, well-written, and very obviously written by an experienced, knowledgeable expert.

The book is structured sort of like a cookbook, question-answer style: “I need to make a change. What methods should I test?” I cannot imagine using the book to look up answers to such specific, disjointed questions. No, the book needs to be read and digested, and then re-read and re-digested, repeating until you’ve fully internalized all its ideas. I’ve read it once from cover to cover (skipping only the C/C++ specific advice) and I know I will be browsing it for inspiration many times. The numerous cross-references make it a very browsable book.

However. I like the general idea of making all legacy code testable, but in practice it’s not always practicable. GUI code is the most obvious example: it’s hard to test in the best of cases, and in a legacy code base, making it testable is a hopeless battle. I’m not going to refactor an entire 1500-line aspx page so that I can test my one-line change which makes a certain element invisible in certain circumstances. And more generally, making legacy code “non-legacy” is a worthwhile goal, but sometimes the change really is small and safe enough, and entangled enough, that making it testable is just not worth it. But I guess the book needs to be more categorical than reality: aim for the stars, get to the treetops.

Amazon US, Amazon UK.

Started learning RoR with the help of Rails for .NET Developers. Spent well over an hour just getting through the first two pages of creating my first app. After typing in the 3rd instruction as per the book, I got errors.

C:\dev> rails book_tracker
C:\dev> CD book_tracker
C:\dev> ruby script/generate scaffold book title:string author:string on_loan:boolean
  wrong constant name Title:stringController

Fine, I’ll google. Via Google the Ruby community tells me that I have an old version of Rails and need to upgrade – while Mac OS X comes with Ruby and Rails pre-installed, it’s not exactly the latest version.

I update Ruby and I update Rails and still I get the error. I uninstall and reinstall rails, confirm that I have the right version, reboot the computer, and still I get the error.

Back to Google. Perhaps I somehow still have the old version somewhere. I google for “multiple versions of rails” and find a brief blog post about Using Multiple Versions Of Rails that gives me this useful piece of information:

your individual rails projects store the version number they expect

Aha! I performed steps 1 and 2 of the tutorial before upgrading Rails, so step 3 was still hitting the old version rather than the new version. I delete the entire folder, redo steps 1 2 and 3 and bingo, it works.

Over an hour to perform a 5-minute task. Couldn’t rails -v report all versions of rails that I have, rather than just the topmost one in the list?

Another Open Spaces event today. This time it was officially not organized by the ALT.NET community because some of the people behind the ALT.NET movement have had a bit of a disagreement (“the movement is stagnating”, “don’t be so elitist”) about what ALT.NET is and should be. And since Scott Bellware, who’s on the “ALT.NET has degraded and we need a fresh start” side of the battle, joined our discussions, we were officially using his preferred name, which was “progressive software development”. In the discussions we kept slipping back to using ALT.NET since most of us, I think, never felt that we are part of any movement, and knew very little about the controversy. We just want to meet developers who think like we do and work with the same technologies, and really don’t care what they call it.

Today’s event was a bit different from the previous one. The presence of Scott Bellware (who’s not exactly a superstar in the developer world but still rather well-known in the .NET circles) changed the character and the focus of our discussions. Scott’s got a forceful personality (as revolutionaries and firebrands tend to) so some of the sessions veered from discussion into lecture. But they were good lectures, inspiring and interesting. A lot of talk about the meaning of quality, and what is needed to change the world of software development to focus more on quality and learning. I came home with a whole reading list about lean thinking and lean development, and a renewed desire to improve myself and my work. I’ve sort of relaxed for the last half-year and stagnated, not learned as much as I could. Time to pick up the pace again.

As a first step, I will make a new effort to learn Ruby. Why Ruby? Because the other thing I came home with was a book titled Rails for .NET Developers. Scott told us about a project he’s planning, which involves developing a Rails-based web site, and invited us to contribute. I liked his idea, but had to admit I would be of little use as a developer since I knew no Ruby. That’s no excuse! he said and offered me the book. What the heck, said I, took the book and decided to have a go. So not only have I met a celeb developer up close, I now also own a book that used to belong to a celeb developer, ha ha.

The code base I work with is in a pretty bad state. Much better than it was a year ago, but that’s still faint praise. At times I can avoid thinking about it, especially when we’re focusing on some new piece of functionality that is mostly independent of the legacy code. Other times I have to wade through all this old muck to get something done. That really makes me want to clean up, go in with a big broom and sweep all the dirt away. Whenever we’re ahead of schedule I take a day or so to focus on refactoring.

A day is a very short time to spend cleaning a big code base, so I want to make sure I spend it well. But the code is so messy that it’s hard to decide where to begin. It’s like when you have a house that hasn’t been cleaned for years: whichever part you start cleaning, something else grabs your attention because from another angle that looks even worse than the part you’re working on.

Do I start from the parts I need to touch most often?
From the most important parts?
From the dirtiest parts?
From the parts where I can get the most done in the shortest time?
From the riskiest, most complex parts?

Every time I face this decision I make a different choice, but in general I tend to pick the first of these strategies. The files I need to work with regularly get cleaner, but there are many places that I have never even looked at, and I’m sort of hoping that I never will, although I’m sure the day will come.