Alignment Trap

Test Driven Development is helping the development effort to be aligned with the needs of the customers, but is it always good to be aligned with these needs? What about the risk of developing features that are too customized to local needs of a specific user, and not a general product solution that a software company should provide.

Why to be Aligned?

The concepts of alignment with the business are easy to understand, you want your product to be aligned with the business of your company (especially if all your business is building a software product) and of your customers (why else should they buy your product?).

How to be Aligned?

The concepts of alignment is also very clear if you read any of the sources of agile development:
Lean Software Development of Tom and Mary Poppendieck.
All you need to do is to work closely with your customer, preferably having them on the development site. Get their requirements from first hand, show them your progress on a daily (and even hourly) basis, modify your software according to their comments etc. But even if you are doing everything right, you might find yourself in an alignment trap (the term is taken from IT research done by Bain & Company)

What is the trap?

As you can see in the diagram above giving full control to the customers, you might find yourself implementing features that are either too small (not in the eyes of the specific customer you talk to, of course) or too complex technically. Having too many small features or not spending enough time to implement them correctly, leads directly to the alignment trap. If I had 1$ every time that I heard the sentence "I don't have time to write tests to the feature...", I would probably be able to buy many cups of coffee and pizza trays to the sleepless night trying to fix these features.

When to be Aligned?

Knowing when to be aligned is the crucial point to avoid the dangerous alignment trap. If you can't adopt changes quickly enough, if you are afraid of changing your code (as it might break), or if you have no ability to know if you are aligned with the market needs or not, then you are stuck in a classic alignment trap.
Therefore you must build beforehand into your organization the following productivity components:

• Automatic Tests - you must be able to change your code with confidence.
• Refactoring Methods - you should be able to "pull up" and utilize other refactoring recipes to allow quick generalization of code and other structural cross module changes.
• Measurements - You have to be able to measure how much a feature is used, and how it is used.

If you have these essentials in place, you are ready to work to be aligned. You can quickly (quick and dirty style) build prototypes and initial versions of your features, throw away some of them (if they are not used), and improve and expand the ones that are mostly used.

The Bottom Line

From my experience one of the main reasons that agile development is failing in an organization is lack of productivity in the development team. The little credit that agile methods gets from the management will disappear, once they discover the alignment trap. "We did everything right, and still we are trapped - agile development sucks...", they will say. You will have only to be sorry that you didn't invest in improvement of your team's basic productivity methods before business alignment.

Debug Addicts

"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it."

Brian W. Kernighan

One of the signs of a problematic software environment is that you see many developers in "Debug" mode. If the code is complicated and have no tests, the developer has no other option than to go into the code with a debugger, trying to understand what is wrong with the code (if he is trying to solve a bug), or how can he extend the code (if he is trying to write a new functionality to an existing component). Since "Debugging" is a very slow process, it takes a lot of time and the chances of finding a developer "debugging" is even larger.

What is wrong with Debug?
If debug is so wrong, then why do we have many debug tools. Sometimes the ability to debug is the main attraction of a tool (for example, MyEclipse writes JavaScript Debugger first in his feature list right after its WebSphere tools). In Java you can find many tricks to make the debugging process faster as; Hotswap (change the code while you debug and see the change instantly), Conditional break point (stop the debugger in a specific line only when a specific condition is happening), Remote debugging (connecting to a remote running application to debug) and many others. Debugging was never easier!

After all these enhancements "Debugging" is still a manual, slow, error prone that gives a very limited insight into the code. It should be used as a last option, where nothing else is working. 

What else can we do?

I see "Debugging" as a symptom to a sickness of complex and untestable code. The alternatives that a developer has to understand a code flow are:

• Code Review - when the code is written in a simple way, short and unique (doing exactly one thing) methods, no Singeltons etc., you can understand the code by simply reading it. I would also suggest that you will try to read your code as English sentences. If you succeed, it means that you used meaningful names for your variables and simple straight forward logic. If the code is messy and you need to "understand" it with a debugger, you can try to document it. It will help you to understand, and also help future generations, as they will probably have to spend a lot of time in this messy code.
  
• Good Logging - Using Log4J in a meaningful way of debug levels (Error, Warn, Debug, Trace...) and good messages can replace any debug session. Good logger can be found for every language including java script (Firebug for Firefox is great). 
  
• Good Tests - Good tests are like a tutorial, which explains how to use a class, a method or a component. It gives example to its input, its usage and expected output. Usually it is enough to understand what it does and how to do it. More than that, a developer can add input samples to a test to check for additional behavior, instead of debugging it.

I believe that tests are the best tools for code understanding, but it is not the only one. 

Teaching the Rules of the Game

How to Introduce the Change?

One of the first tasks introducing a change in an existing team as an outsider, is to teach the new rules. If you are coming from inside the organization, having the knowledge of the people and the backup of the rest of the management, you can probably have a better plan than what I had when I joined the team.
I decided not to wait until I discover the fine balance of power inside the organization, but to set up the stage in a form of game. The game is educational, as it give first hand experience of the concepts, and more importantly, it is fun and causes much less defense from the participants.
The first concept I wanted to change in the Agile or SpeeDevelopment was the waterfall or serial stream of deliverable from the field to the lab and back. To be able to speed up the cycle of development for real customer, it is important to break the walls or gates between the teams.

The "Good Old" Way

The process used to start when the product manager wrote a long and detailed general requirement document, based on his discussions with sales and real customers. The requirements were analyzed by the development team, designed and developed. Then it was tested by the QA, who learned what are the requirements from the developers. The process was considered "Agile" as each such cycle or iteration was of one month.
The first phase "Requirement Writing" seems to me as the root for most of the problems; if the requirement are too general and the development is done far away from the real customer, the chances for success are slim and the effort wasted is fat.

The Simulation Game
To teach this lesson in a fun and quick way, I found a nice simulation called "Offing the Off-Site Customer". Just before the beginning of the first iteration that used the methodology of writing requirements by the developers with the help of the product and actual users, I ran this simulation on all people involved (development team, product team, professional services team...). I divided the group (about 30 people) into pairs and gave them the tasks of copying diagrams in two ways;

1. One writing description and the other using the description to copy the original diagram
   
2. One watching the diagram and describing it orally while watching the copied diagram being drawn by the other.
   

I think that the message got through quite effectively, as the results of the second method were dramatically better than the results of the first way.
I still had to make some modifications of the ways of requirement writing collaboration, but the idea of writing it together (developers and customers) was accepted without much objection.

Problems in Introducing SpeedDevelopment

I recently started to work with a new development team. The team is very experienced and has all the right tools to adopt very quickly to SpeedDevelopment methods, or so I thought.
SpeeDevelopment is all about speed and all about development, two things that every developer would like to have. Why, then, should it be difficult to get developers do it?

Problem #1 - "I'm not QA"
QA and writing tests are considered to be the lowest rank of software developers. When you are a student you start working as QA, gather some experience, and then you are "promoted" to become a developer. Now it is the time for the new "greenies" to write tests, and you can focus on the fine art of writing "real" code.

Problem #2 - "I can't write tests for this code"
If you managed to overcome problem #1 and try to write tests for your code, you see that it is not easy. You don't know which testing framework to use (JUnit, TestNG, JMock, EasyMock...), you don't know how to use the framework well, you don't know how to write good tests (as you are costumed to think positively and not like a tester), and the worst thing is that your code is not testable at all. All you want to do it call a simple method and check its output, and you find yourself writing mountains of code of constructors, initializers, DB connections and other "minimal requirements" for this single method to work.

Problem #3 - "I don't have time for testing"
When the task of the developer is to write code and the task of the QA is to write and run tests, the developer has no time to write tests. You can explain that the development work in not done until all tests are "Green" and that working against a specific and well defined set of tests, decreases the development time and other logical arguments. But the bottom line is that the feeling in the beginning is that the tests are an overhead that only takes a lot of time and gives nothing in return.

Don't despair, all these problems have solutions, which we will discuss in the next posts.