The Agile missing point and the Waterfall Illusion.
November 25th, 2009
The Agile missing point and the Waterfall Illusion.
Published on November 25th, 2009 @ 09:43:26 pm , using 1324 words, 7873 views
What if everybody is wrong? Well, not everybody, but majority.
Reading Kelly Water’s Post about “Agile Project Management: Avoiding The Waterfall” I came to think about this. Almost nobody claims today that they are following waterfall, because of the very bad connotation that word now has, but not actually because they understand what are they doing. Same thing for Agile followers.
From the post, I rescue that any process that has phases reassembles a waterfall. From actual presentations, blogs and discussions, Agile often means just following methodologies like Scrum, Lean or XP. Now I question: are we right about this? Let me see if we are, or not.
Waterfall is a term, it seems nobody knows who coined it, related to a phased methodology. Dr. Winston W. Royce wrote about it (without given that name), describing an ideal form that was not so good. The phases were the requirements gathering, analysis on those requirements, system design, coding, testing and operation. The ideal was each step may iteratively cycle with the next down and the next up, in little steps. The flaw was it was usually the case that testing will not just go back to coding, but it may go up till design, and from there, jump back to requirements gathering again. That flaw (the large back jump) was caused not because the phases, but because the testing phase is where the actual product is finally validated against design and requirements.
The problem is, there is no solution to this. We can dilute or minimize the impact of the testing findings, using several techniques, but we cannot avoid it. At any level, team or individual, the same steps are repeated: I see what it is needed, I analyze what is possible, I think on how to do it, I do it and finally I check if it works. That can describe the process of developer creating a simple three line method! That is, phased development is there, everywhere, and that is not the problem. The problem is when that phased process is done in a way that the impact of facing reality with needs is too big that multiplies the work and the cost of the back jump.
So, the waterfall illusion is when you think that rain is not a waterfall, simply because the raindrops are so tiny you don’t see them fall individually. But they fall anyway. It means, people that think eliminating the analysis and design steps and jump into coding breaks the phased process, I’m sorry, it doesn’t, it may avoid a large body of water fall into one place, but will make millions of little bodies fall everywhere. See the point?
Ok, there is another idea into this, and that is to make the water flow up. You start coding, with no idea of what you need, and then the phases are completed backwards. (Hey! If you want to start really backwards, then start by testing! Oh, yes, we have TDD). The code will make a design “emerge”. The next logical steps if to create requirements that match the design. Well, I guess that does not work. In that case, the requirements are still presented before coding. So the new phases are: requirements, coding, design, testing. Or, requirements, testing, coding, design, testing, coding, design, etc. Anyway, we always have phases.
PermalinkTrackback address for this post
8 comments
I agree with you people were doing Agile long before the term got coined... I found this sentence disturbing tough:
"so coders wont need to worry about those things....."
The paper by Hirotaka Takeuchi and Ikujiro Nonaka that led to Scrum talks about an holistic approach to product development where all the phases overlap. Everything happens at once, by everyone working closely together, all at the same time, just like how a rugby team passes the ball to each other in a Scrum (well a Maul to be accurate :)).
So no phases, no hand-offs. Everyone worries about everything, there are experts yes, but everyone works together:
http://apln-richmond.pbworks.com/f/New%20New%20Prod%20Devel%20Game.pdf
This is the essence of Agility and is what separates it from other ideas like Lean (yes Lean and Agile are not the same thing).
People like David Snowden talk about the science behind this idea which stems from chaos theory and complex adaptive systems theory. The most applicable theoretical basis for Agility today is social complexity theory.
So Agile is much understood, just like most things that are given a label, marketed and sold to death :)
Paul.
"No, agile does not mean following a methodology and getting rid of good guys like Architects and PMs,"
Agile does mean reducing hand-offs. It also means self-organised cross functional teams.
So whilst specialists are acceptable, they can result in hand-offs, and impede self organisation. So an architect that hands-off to a development team is not Agile, and a PM that thinks it is his job to organise the team is not Agile.
Many Agile teams have found that the best way to reduce hand-offs and to promote self organisation is to have multi-skilled team members. So Agile team members tend to be more then mere "coders". They would describe themselves as "developers", and good developers can architect and project manage for themselves, without having to rely on specialists.
I'm not saying that specialist are a good thing or a bad thing, but I believe it is a false statement to say the Agility doesn't favour cross-functional teams.
Paul.
Hello Paul!
Ok, let's clarify the sentences. The idea is that architects and project managers are not people you need to get rid off simply because you only need coders in agile. Note that is different of what you understood. As Hirokata and Ikujiro said, you need all that, just that at the same time. There is no problem if an architect codes and also keeps an eye on the higher view of the system. There is no problem if the project manager codes and also keeps numbers and takes care of resource problems. There IS a problem is no one does that, see the difference?
Now, still doesn't make sense to have all people knowing a little of some specialty, against having someone expert on that in the team, sharing his knowledge. Working all together shouldn't mean everybody doing the same thing at the same time. Self organizing teams are not the ones that synchronize to do the same thing at a particular moment. Architects play a role, as project management. PM role is not to dictate the lines of code to be written, that is a fallacy, PM is there for a needed task of control (which is not a bad word).
Take this two examples:
a. A team where all are the same age, same experience. All know how to code in java, and all have taken the Software Architecture fundamentals, had read the testing book, and already read the PMBOOk. Great, they all work doing the same, trying to code, control and work from top and down to code.
b. A team where you have a set of java experts, a testing expert, an architect and a project manager. Everyone works at the same time, providing their knowledge to the development.
I do believe both can be agile, if well organized. I do believe both may fail being agile if not organized correctly.
Also, of course, I do think there are phases. Even micro ones. There may be coders that open their minds and start coding without even thinking on what is needed, and produce something useful. But most of them will actually ask what is needed, then think how to do it, then do it and finally test it (testing may me moved upwards like in TDD, still it is a phase). So, the idea is not to remove phases, it is to keep away the focus on them and put it on the actually delivered business value.
Finally, just to mention it again, I feel following, say, SCRUM, does not make you Agile automagically. SCRUM is a way, not the only one, not the surefire one.
Cheers.
Thanks for clarifying. When I develop I move from anyalsis to specification, to design to coding to re-design (refactor), back to analysis all within minutes, and I continue like this in small cycles. So I guess if you count a few seconds as a phase then there are always phases :) I find it more descriptive however to say that "phases happen all at once".
As for specialists, I agree that they can be useful. Personally I find a good spread of technology expertise useful (web design, database design, networking etc) rather then trying to split the development role (analysis, architecture/design, leadership/management, communication, coding etc).
I see all these things as part and parcel of development, and I believe good developers should be able to do all these things. I tend to try and keep to a ratio of "journeymen" developers to "apprentices" of 1 : 3. And of course you need a Lead developer of some sort to have the final say when the team can't reach consensus.
Of course this is just opinion, and I agree that there is no fixed formula to Agility. And yes, it definitely doesn't mean following a prescribed methodology like Scrum :)
Paul.
Sorry. I said a ratio of journeymen developers to apprentices of 1:3. I should have said 1:1. And of course ideally all developers would be master crafstmen :)
Your post has got me thinking about what makes a team Agile, and on reflection I believe it comes simply down to skill. Highly skilled developers who can run with a concept, collaborate with customers and do what it takes to deliver quickly and often, whilst producing a codebase which is a clean, pleasant and sustainable place in which to work.
I think that's it. Kent Beck, Ron Jefferies and Ward Cunningham all qualify here as Agile developers, and the original chrysler C3 project team would be my example of an archetypal 'Agile" team.
Agility to me is recognising the importance of internal safeguards and risk mitigation strategies. Developers that know what they are doing and can travel light and be responsive to change. A recent blog post made the analogy with skiing:
http://availagility.co.uk/2010/01/08/process-safeguards-and-ski-slopes/
Agile developers are the skiers that are competent on the black slopes. Sure black runs are harder, but if you know what you are doing you'll get down the mountain a hell of a lot quicker :)
An environment where there are a lot of external safety factors, like architects, project managers etc assume that the developers need safeguards. This is akin to the green slopes. Wide pistes, gentle slopes, etc.
So to me Agility is an attitude and the ability to back it up. One without the other doesn't work. As Kent Beck put it "turning the volume up to 11".
BTW, Agile as I describe it here is not for everyone :) As in skiing, some teams are better off sticking to the red, blue and green slopes, and other teams shouldn't be on the mountain at all :)
Paul.
I do agree with you. Agile is not about extreme performance, where you need all geniuses locked up in a small room. Agile is about being able to deliver even in not as safe environments, with juniors, changing requirements, changing resources and even unknown outcomes.
So, as my example mentions, if you cannot gather a knowing all, doing all group of people, doesn't mean you cannot be agile. You can actually demonstrate how agile you are if you can get the team going and wining the game with only moving backwards pawns.
Cheers!
Hi William,
I disagree. Most teams are woefully under-skilled in my opinion. Why? Because over the years management have chosen to under value "coders". You get what you value. We seem to value tools over developers, experts over developers. All on the assumption that a army of cheap coders, led by one or two experts and utilizing the best tools makes the most economic sense. Agile presents another alternative. Save the money on the experts and tools, and hire a few good developers (5-7), and they will deliver as much if not more in less time. At the end of the day Agile is not a prescriptive set of practices, it is a movement.
I see your belief in architecture as a movement "The architect Movement". What I am describing is the original "Agile Movement", which started in around 1999 with XP. Since then Agile as been marketed as "good" and "best practice", I disagree with all this. Agile is one approach amongst many. I've come across very good waterfall teams in my time, and I've seen very bad Agile teams :) IMO the Agile movement is more or less dead anyway, killed off by salesmen and consultants. The original goals have now mostly moved to the "Craftsmanship Movement" and of course there is the 'Artisanal - Retro Futurism..." Movement of Brian Marick
I see your description of Agile as a watering down of the original goals, what I describe as "Me too Agile". I guess this is inevitable as Agile goes main stream :) I just wanted to point out what Agile was originally all about, and still is for those who are still pushing people over tools. We still have a long way to go, and as Brian points out in many ways we have gone backwards... Paul.
Comment feed for this post
