Saturday, July 29

How to select a right Agile methodology for your project - Part 2

In my last post, we talked about what are the popular choices for the agile software development process, and what is the difference at high level between RUP and the others. In this post we are going to examine the characteristic differences such as team size and iteration, length, and roles, as well as the reasoning and the driving forces behind the differences.

Team size: 2-12 Person (Although there are claims that XP has been successfully implemented with teams had over 60 engineers)
Iteration Length: 1-3 Weeks
Roles: Customer, Programmer, Tester, and Manager
Team size: 5-10 (Some scrum practitioners recommend a fixed team size 7 or at least less than 7)
Iteration Length: Fixed 30 days
Role: Product Owner, Scrum Master, Programmer
Team size: 3-10 (You can actually structure a RUP team that has as many members as you want, but as you increase the size your team the higher ceremony level is required which will basically deminishes the agile nature effectively)
Iteration length: 2-8 weeks
Role: Customer, Business Analyst*, Architect*, Programmer, Tester, Manager

Note: Roles with * are optional

Now you probably are going to ask the question: Ok, we have these numbers, thats great but it does not seems to help too much since they all look the same. Yes you are right, in order to pick the right process, you not only need to know these superficial numbers, but also understand where these numbers come from and what are the implications you have to deal with when you decide to change them.

The most controversial and successful element of XP, which pretty much dictates almost every aspect of XP as well as differentiates itself from other methodologies, is the way that Software Change Request gets managed or maybe I should say unmanaged. XP allows customer to add new requirement (user stories) in the middle of a iteration, because the people who created XP believe that user requirement is a ever changing reflection of the ever changing world hence a successful software development process should be nothing but adaptive to this kind of change. Wait a minute, so how XP survives this kind of harsh environment which once believed almost impossible to work with by the traditionalists. It is simple, just stick to the following rules:

  • Spread the knowledge

    • Reasons - When you have a team that needs to cope with constant changes and literally rework of the existing features every day at any given time, it becomes absolutely crucial to maintain the expertise within the team, you just simply can't effort lose a couple of weeks just because some one on your team is on vacation or sick. When a customer change request (a new user story) comes in some one on the site need to sit down and talk with the customer to come up with a simple but effective design to achieve the requirement in merely a couple of hours if not less. To preserve the knowledge you need to spread them so there will be no single point of failure in your team. Several best practices were introduced over the years to encourage this behavior, such as Move People Around, Coding Standards, Pair Programming, Collective Code Ownership.

    • Implications - This practice has two major implications, first the team size. Because XP uses a decentralized team structure, meaning there is no architect(s) or even team lead who acts like a center point or a hub of communications to reduce the communication paths, therefore a large size team will create a huge number of communication path which will effectively reduce the productivity, since people will just get moved around too much and lose their focus and proficiency. That's why XP fits well with a smaller team. The second implication is the scope of the project. Because XP requires spreading the knowledge among the team members as well as collective code ownership, in other words a perfect XP team will have no component owner but having every team member be the owner/expert of every component instead. Now you can imagine, if we have a over complicated project, and since there are so many modules and features you just can't move people around enough to work on all the modules, and as a result people either get forced to settle down on a few less components than they should and effectively roll back to the component owner model, or even worse, every body in the team become Jack Of All Traits and expert of none.

  • Test first

    • Reasons - When you are working with constant incoming change request without a formal change management system or process in place, how are you going to know whether all the user requirement including the originals and the new ones are implemented properly, and no outdated implementation gets left over in your code base? Although we do have user stories that we can manually go through each time during acceptance test stage to verify the completeness of the system, but nothing is better than using code to verify the code. High coverage automated tests serve as both a contract between requirement and the implementation to ensure the completeness and correctness, and a safety-net that allow us to make changes rapidly without worrying about accidentally breaking other parts of the system. Many best practices are available to address this aspect, such as Testable Requirement, Unit Test First, Unit Test, Test Guard.

    • Implications - Some people believes that XP is just one step away from chaos, and I believe this is THE step. In order to be successful when implementing XP, you just have to follow all these best practices, actually I should not call them best practice but mandatory practice since they are THE only practice. As the consequence, some of the overhead that is saved in XP by simplifying the process and reducing the level of formality were balanced out by the overhead created by demanding high coverage tests. A common newbie mistake will be reducing the test coverage or the effort required to automate test execution thinking it will not have a major impact to the project, and your project will be thrown back into the realm of chaos before you know it. (In other posts I will talk about the common mistakes and mis-assumptions that people make when implementing Agile process in more details)

  • Keep it simple

    • Reasons - In XP process people believe nothing is static, so why would you waste your time and energy to write sophisticated code for something that might get thrown away or rewrite any time soon. That's why in this model we encourage Just In Time Design so no time-consuming grand up-front design is required, and Just Enough Implementation so never write anything extra for future requirement or the "what-if" scenario. This practice also works as a catalyst for the other two, when you keep everything simple, it become easier for people to understand which in turns speed up the spreading of the knowledge, plus when you make it simple, it become easier to verify and test against. Furthermore, your team should even routinely refactor the code to make the complicated ones simple, and the simple ones simpler. Here is some best practice that can help you to keep things simple, Simplicity, Never Add Funcationality Early, Refactor Mercilessly, Optimize Last.

    • Implications - No specific implications or side-effect. As a matter of fact, this is one practice I recommend to all the projects, and it does not matter what process you are following. (In other posts I will talk about the recent trend and rethinking in architecture world about the new evolving adaptive architecture vs. up-front predictable architecture)

Wow this post is much longer than I originally thought. I hope it gave you some insight to what dictates these numbers for XP, and what you need to consider when you choose to use XP in your project.

Wednesday, July 26

How to select a right Agile methodology for your project - Part 1

Recently some of my friends and colleagues asked me the question that:

Though there are various types of Agile methodologies XP, SCRUM, and RUP available right now, but when you start a project which one do you choose and what criteria do you apply to assess whether the methodology is the best match for a specific project. A short answer will be pick the one you feel most comfortable with, but if you really want to compare them and pick the best one for your project then having a solid understanding of the difference between theses processes is a must-have. I am writing this post in the hope of shedding some light to this tough question. Perform a quick search you can find introductory articles such as this one from Agile Alliance, but it does not sound too practical or useful for a real life project.

As an independent consultant, I have been working with different companies with different sizes who endorse different development processes ranged from Waterfall to XP, iteration length ranged from 18 months to simply a week. Recently and fortunately I finally had a real-life hands-on experience on setting up a SCRUM team and working with it to deliver some rather complicated product under tight deadline which you can think of it as a performance and stress test for the SCRUM process, hence so far I have worked with all three of the most popular Agile methodologies therefore I want to share the tips and pearls I discovered and the lessons I learned, and paid dearly for it.

Before I can start, some people might ask how come RUP is also categorized as an Agile methodology here? Isn't it a heavy weight process that is CMMI compatible and only used for over weight project and teams? The answer to that is "Yes you are right but only partially right". RUP - Rational Unified Process; If you strip down all the fluff that's created around this methodology for obvious marketing reasons from IBM, and if you look deep enough under the hood RUP is nothing but a set of guide lines and principles - so called The Spirit of the RUP:

  • Attack major risks early and continuously ... or they will attack you.

  • Ensure that you deliver value to your customer

  • Stay focused on executable software

  • Accommodate change early in the project

  • Baseline an executable architecture early on

  • Build your system with components

  • Work together as one team

  • Make quality a way of life, not an afterthought

As long as you are following these essential principles, you can consider your project is following RUP process. So RUP process can have an iteration as long as a year or no iteration at all (it is possible to structure a RUP process with waterfall model although it is usually not a good idea) or as short as a couple of days. A RUP team can contain as many as 200 engineers or just one person (yourself working on your personal website in your basement). Based on that you can see actually RUP is considered more higher level methodology comparing to XP or SCRUM since it does not have strict rules or practices that you need to follow, all the artifacts and disciplines are optional so you can pick and choose as long as you know what you are doing, hence you can really consider RUP as a super set of XP or SCRUM. As matter of fact, you can effectively follow XP and RUP at the same time, there is no conflict between them at all.

But to avoid confusion, in this post I will follow the widely spread belief and treat RUP as a separated methodology with its own flavor.

Its 10 o'clock now, time to go to bed since I have a 7:30 SCRUM meeting every morning .... Don't be shocked, this meeting has actually nothing to do with SCRUM process and in fact it is a very valuable lesson I learned, and I will explain the details in another post :)

Say something about myself

I often get asked to say something about myself, so here we go:

I am a worrior in search for the Holy Grail of software development,
Keyboard is my sword, in hand I conquer,
Unittest is my shield, in danger it protects me,
Refactory is my steed, in its company I travel far,
Simplicity is my religion, in heart it makes me strong,
User story is my compass, showing me directions while I am lost ;)

My First Post

I was convinced last Saturday by some of my friends, they probably got tired of my constant mumbling, to write down my ideas and thoughts here in my blog so at least they don't have to be forced listening to them on the phone or in the cafeteria :)