Wednesday, 16 April 2014

Productive Developers are Smart and Lazy

When I use the terms Smart, Lazy, and Developer; I mean the following:
  • Developer as in energetic and focused on building real-world code solutions
    • Not a dreamer who never gets around to writing anything practical
  • Smart as in able to think things through (i.e. not smart-ass)
  • Lazy as in lazy-loading, that is taking time to write code (i.e. not couch potato)
Good development is lazy development a.k.a. patient development; it happens when the developer spends the time necessary to think through all the pathways of the solution that he is developing BEFORE writing the code. That is lazy-writing of code, i.e. not writing code before the problem is understood.  The more due diligence a developer does to make sure that he is writing the correct code will reduce the amount of code that needs to be written.

This due diligence takes the form of:
  • Really understanding the requirements and getting product management (business analysts) to be clear on what are the ACTUAL requirements
    • They often are not given time to gather requirements
    • They often don't have access to the right subject matter experts
    • They sometimes have very poor abilities to synthesize consistent and complete requirements (see When BA means B∪ll$#!t Artist)
  • Really making sure that you understand how you are interfacing to other developers on your team and other teams, this involves:
    • collaboration in front of a white board
    • producing diagrams (UML and Visio)
It takes time to do this due diligence to make sure that you have consistent requirements and to make sure that you will have consistent interfaces with your peers.  However, developers are eager to start banging out code and spend hours at their desks banging out code.

In reality, less than 5% of that time is spend productively (see The Programmer Productivity Paradox).  If you see developers spending 100% of their time staring at their screens with no human interaction then you are looking at some of the worst developers.

It is a bad sign if developers are always coding

Productive developers are constantly checking their understanding of the requirements and making sure that they are staying in sync with their team's code.  Productive developers are in regular contact with the product managers/business analysts and can often be seen white boarding with their peers and architects. There are definitely developers who use their years of experience to become more productive, in fact among the best developers:
  • the ratio of initial coding time was about 20 to 1
  • the ratio of debugging times over 25 to 1
  • program execution speed about 10 to 1
  • program size 5 to 1
However, in the aggregate, developers do not become more productive over time (see No Experience Necessary!), i.e. over thousands of developers there is no correlation between years of experience and productivity.  In fact, we have measured productivity regularly 8 times over the last 50 years and years of experience do not correlate (in the aggregate) with productivity.

Why is lazy writing of code so important?

Code is often written before the requirements are understood or gathered.  In addition, quickly written code often fails to fit with everyone else's code; often, it is only during integration that this problem is discovered.   Good developers are patient and realize that there is a cost to writing code quickly.

Developers become psychologically attached to their code 

Bad developers are reluctant to change poorly written code.  Rather than rewrite suboptimal code, bad developers will simply add more code to make up for deficiencies. Even worse, they tend to blame everyone else for having bad code.  What you end up with is band-aid after band-aid that lead to a severely buggy and unstable system.

Don't get me wrong, good developers can find themselves in a situation where they have written sub-optimal code.  The difference is that a good developer will recognize a problematic section of code and:
  • Refactor the code if the code is largely doing the right thing
  • Rewrite the the code otherwise
When developers produce and maintain sub-optimal code, it becomes harder and harder to change this code as time goes on.  That is because their peers will need to write code that interfaces with the suboptimal code and build clumsy interfaces or work-arounds to make the code work. As the code base grows, too many later code units rely on the functionality of this initial code.  Of course, the later code can do little to increase the stability of the code and bugs multiply when simple changes are made; in short, development becomes slower and slower.

When in doubt, be lazy and write code late

Other relevant articles:

Sunday, 9 March 2014

The Programmer Productivity Paradox

Programmers seem to be fairly productive people.

You always see them typing at their desks; they chafe for meetings to finish so that they can go back to their desks and code. When asked, they will say that there is not enough time to produce the code, and the sooner they can start coding, the sooner they will be done.

So writing code must be the most important thing, correct?

If the average programmer writes about 50 lines of production code a day.  A 50,000 line program would take 1,000 man days to produce.  The 50,000 line listing can be entered by a programmer at about 1,000 lines a day or about 50 man days.

So what the heck are the developers doing for the other 950 days?

Before addressing that issue, lets make a simple observation. Capers Jones has compared many methodologies (RUP, XP, Agile, Waterfall, etc) and programming languages over thousands of projects and determined that programmers write between 325 and 750 lines of code (LOC) per month, which is less than the 1,000 LOC per month suggested above1.  Even if programmers do not average 50 lines of code per day, the following is clear2.
  • Methodology does not explain the apparent productivity gap
  • No language accounts for the apparent productivity gap
The reality is that only a fraction of a developer's time is actually spent writing production code. If a developer is typing in code all the time then they are really trying different combinations of code until they finally find the combination of code that works.

Or more correctly, the combination that seems to match the requirements until either QA or the business analyst comes back and lets them know there is a problem.

That is why developers that plan their code before using the keyboard tend to outperform other developers. Not only do only a few developers really plan out their code before coding but also years of experience do not teach developers to learn to plan.  In fact studies over 40 years show that developer productivity does not change with years of experience. (see No Experience Required!)

Years of experience do not lead to higher productivity

Interestingly enough, there are methodologies that have been around for a long time that emphasize planning code.  Watts Humphrey is the created of the Personal Software Process (PSP)3.  Using PSP has been measured to:

PSP can raise productivity by 21.2% and quality by 31.2%

If you are interested there are many other proven methods of raising code quality that are not commonly used (see Not Planning is for Losers).

If your developers at their keyboard and not planning at a white board then odds are that your productivity is not as high as it could be.

Bibliography

1 The The Mythical Man Month is even more pessimistic suggesting that programmers produce 10 production lines of code per day
2 Jones, Capers and Bonsignour, Olivier.  The Economics of Software Quality.  Addison Wesley.  2011
3 Watts, Humphrey.  Introduction to the Personal Software Process, Addison Wesley Longman. 1997

Other articles in the "Loser" series

Want to see more sacred cows get tipped? Check out:
Moo?
Make no mistake, I am the biggest "Loser" of them all.  I believe that I have made every mistake in the book at least once :-)

Monday, 17 February 2014

When BA means B∪ll$#!t Artist

BA  means Business Analyst, but what makes for a good BA?  When do you have a good BA and when do you have someone who isn't? Many projects fail at the beginning due to incomplete, inconsistent, and overly verbose analysis that then leads to incorrect project plans and projects heading in the wrong direction.

Business analysis consists of all facets of solving business problems. Business analysis is a role executed by project, product, and engineering managers as well as other people.  However, there are people that do business analysis as their main role and we simply label them as business analysts or product managers.

We shall stick to the term business analyst for simplicity. Business analysts perform a range of tasks including:
  • Gathering requirements
  • Writing business cases and project charters
  • Performing gap analyses to incorporate new products
Are your BAs any good?  

The success of many projects depend on the ability of the BA do correctly identify the problem you are trying to solve.  If your BA is not competent then you are doomed before you start.

Ever suspect that the people responsible for performing business analysis for you are not up for the challenge?  Here are three simple questions; good business analysts will get all three correct and not take longer than 30 seconds.

Context Question
#1 Which of these two lines is longer?
#2 What does this sign say?
#3 I have two products whose prices add up to $1.10 and one product is $1 more than the other. How much is the more expensive product?

The answers are:
  1. The lines are the same length
    • Take a ruler if you are not convinced
  2. Paris in the the spring
    • Notice that the occurs twice
  3. The more expensive product is $1.05
    • If the more expensive one had been $1 then the cheaper one would be $0.10 but then the expensive one would only be $0.90 more expensive than the cheaper one.
These three questions illustrate that the mind can jump to conclusions that are often wrong. The mind can be tricked because it assesses things quickly, i.e. jumps to conclusions.  Only some people have the instinct to double check their results and check for personal bias.

Most BAs are well educated; however, the interesting thing is that studies show that educated people are less able to see their biases and they jump to conclusions more readily than other people.1 Even worse, well educated people have less of a tendency to check their conclusions than other people.

A good business analyst realizes that during analysis there will be situations where the mind will jump to conclusions.  Many business analysts are asked to resolve conflicting requirements, recognize missing requirements, and deal with biases coming from many different sources.  Unless you have the reflex of checking facts for consistency and eliminating bias then you won't make a good business analyst.

So what is the quality of your BAs?

Other articles

Bibliography

1 West, Richard F. and Meserve, Russel J. Cognitive Sophistication does not Attenuate the Bias Blind Spot. Journal of Personality and Social Psychology.

Thursday, 16 January 2014

No Business Case == Project Failure

A business case comes between a bright idea for a software project and the creation of the software project. 

  • To - idea to have a project is born
  • Tcheck - formal or informal business case
  • Tstart - project is initiated
  • Tend - project finishes successfully or is abandoned
Not all ideas for software projects make sense.  In the yellow zone above, between idea and project being initiated, some due diligence on the project idea should occur.  This is where you do the business case, even if only informally on the back of a napkin.

The business case is where you pause and and estimate  whether the project is worth it, i.e. will this project leave you better off than if you did not do it. For those who want precise definitions the project should be NPV +ve.  In layman's terms, the project should leave the organization better off on it's bottom line or at least improve skill levels so that other projects are better off.

Projects that do not improve skills or the bottom line are a failure.

Out of 10 software projects (see Understanding your chances):
  • 3 are successful
  • 4 are challenged, i.e. over cost, over budget, or deliver much less functionality
  • 3 will fail, i.e. abandoned
This means that the base rate of success for any software project is only 3 out of 10. Yet executives routinely execute projects assuming that they can not fail, even though the project team may know that the project will be a failure from day 1.

Business cases give executives a chance to stop dubious projects before they start. (see Stupid is as Stupid Does) Understanding how formal the business case needs to be comes down to uncertainty. There are three key uncertainties with every project:
  • Requirements uncertainty
  • Technical uncertainty
  • Skills uncertainty
When there is a moderate amount of uncertainty in any of these three areas then a formal business case with cash flows and risks needs to be prepared.

Requirements Uncertainty

Requirements uncertainty is what leads to scope shift (scope creep).

The probability of a project failing is proportional to the number of unknown requirements when the project starts (see Shift Happens).

Requirements uncertainty is only low for two particular projects:
  1. re-engineering a project where the requirements do not change
  2. the next minor version of a software project. 
For all other software projects the requirements uncertainty is moderate and a formal business case should be prepared.

Projects new to you have high requirements uncertainty.

Technical Uncertainty

Technical uncertainty exists when it is not clear that all requirements can be implemented using the selected technologies at the level of performance required for the project. Technical uncertainty is only low when you have a strong understanding of the requirements and the implementation technology.  Uncertainty and risk are two different animals (see  Uncertainty Trumps Risk in Software Development)

When there is only a moderate understanding of the requirements or the implementation technology then you will encounter the following problems:
  • Requirements that get clarified late in the project that the implementation technology will not support
  • Requirements that can not be implemented once you improve your understanding of the implementation technology
Therefore technical uncertainty is high when you are doing a project for the first time and requirement uncertainty is high.  Technical uncertainty is high when you are using new technologies, i.e. shifting from Java to .NET or changing GUI technology.

Projects with new technologies have moderate to high uncertainty.

Skills Uncertainty

Skills uncertainty comes from using resources that are unfamiliar with the requirements or the implementation technology.  Skills uncertainty is a knowledge problem.

Skills uncertainty is only low when the resources understand the current requirements and implementation technology.

Resources unfamiliar with the requirements will often implement requirements in a suboptimal way when requirements are not well written.  This will involve rework; the worse the requirements are understood the more rework will be necessary.

Resources unfamiliar with the implementation technology will make mistakes choosing implementation methods due to lack of familiarity with the philosophies of the implementation libraries.  Often after a project is complete, resources will understood that different implementation tactics should have been used.

Formal or Informal Business Cases?

An informal business case is possible only if the requirements, technical, and skills uncertainty is low.  This only happens in a few situations:
  • Replacing a system where the requirements will be the same and the implementation technology is well understood by the team
  • The next minor version of a software system
Every other project requires a formal business case that will quantify what kind of uncertainty and what degree of uncertainty exists in the project.  At a minimum project managers facing moderate to high uncertainty should be motivated to push for a business case (see Stupid is as Stupid Does).

Here is a list of projects that tend to be accepted without any kind of real business case that quantifies the uncertainties:
  • Change of implementation technology
    • Moving to object-oriented technology if you don't use it
    • Moving from .NET to Java or vice versa
  • Software projects by non-software companies
  • Using generalists to implement technical solutions
  • Replacing systems with resources unfamiliar with the requirements
    • Often happens with outsourcing
Projects with moderate to high risks and no business case are doomed to fail.

Related articles

  • No Experience Necessary!
    • Surprisingly there is no correlation between years of experience and productivity.  This has been verified over 40 years.
  • Stop it! No, Really Stop It!
    • 5 worst practices that software organizations commonly practice that they need to stop right away