Why Management Declared Deadlines Lead to Disaster

Senior management, in the face of market pressure, will sometimes initiate a project to address a business need that has a management declared deadline.

Management will do this when they feel market pressures to create a new software product or version because of some impending business event in the market.

These market driven deadlines occur for a variety of reasons:

• Competitor is releasing a new product/version that will eat your market share
• Laws have changed that require compliance by your software system
• Merger with another organization requires software systems to be integrated
• You are a start-up and need to build version 1.0 to obtain financing
• Sales has sold vapor-ware and engineering now needs to produce it

Some organizations (not many) are enlightened enough that they determine project size in function points and use automated sizing tools to estimate time and cost.  Large organizations have measured automated tools against actual results and determined that automated sizing tools estimate projects to within 5% of time and cost.

Counting function points will increase productivity by 17.5% and code quality by 25.8%.

Automated sizing tools will increase productivity by 16.5% and code quality by 23.7%.

However, on a regular basis automated sizing tools do not produce estimates that management likes.  So management will be adamant that the deadline is fixed and must be met at all costs. Some management groups are more dysfunctional than others about insisting that deadlines because of business reasons and not by estimates.

Rejection of estimates for business reasons will decrease productivity by 15.7% and code quality by 21.7%.

The most dysfunctional management will expect all existing project not to be impacted by the addition of a high priority project. They will say silly things like 'You will just have to multi-task' or something equally ridiculous.

Anyone who has worked in a multitasking mode is familiar with  Little’s law or that productivity drop-off that comes from trying to do multiple things at the same time.

Less dysfunctional management will realize that the high priority project will cause other projects to be late, but they will continue to expect that everything progresses even though resources will be committed to multiple projects. Project deadlines typically get set in the short term, which means that the resource for the project will be fixed.

Even if the funding is available to hire resources, the new resources will not be familiar enough with the corporate context to be of much use. The scope and quality of the project will be dictated by the market pressure that generated the project.

This project will have fixed scope, quality, time, and resources which will lead to a feasible project only once in a million projects.

Those familiar with project management know that you can only set 2 parameters of time, resources, and scope and then the 3^rd parameter must be allowed to vary, if you want a feasible solution.

If you specify the resources and scope then the time must be allowed to vary; if you specify resources and time, then the scope must be allowed to vary. When management sets up a project like this the only practical variable that can stretch is time through resources working overtime.  Overtime = excessive schedule pressure.
 
Excessive schedule pressure will decrease productivity by 16.0% and code quality by 22.5%.

If an organization is well run and a management declared project occurs rarely then the extra stress due toregular basis. Such senior management teams are unaware that they are increasing risk by an order of magnitude and are always mystified when they are unable to achieve the results that they want.

working long hours can be handled.

Unless the senior IT manager has the courage to point out the problems that such a project will cause, the IT department is doomed to become a pressure cooker. Even if the IT manager has the courage he needs to have the disciple to refuse the pressure from the other senior managers until a feasible project is defined.

Unfortunately, most senior IT managers have neither the courage of their convictions nor the discipline to weather the storm.

The end result will lead to a pressure cooker in engineering, where the engineers are being asked to perform super human efforts for which they will get very little in return.

This kind of environment if prolonged will lead to your better engineers quitting which will cause all the rest of your projects to get even later. With the loss of productivity that comes from trying to multi-task, the code that is produced is guaranteed to have more bugs than normal, which will then cause the problem to flow into QA.  Even more insanity will follow when QA time is reduced or eliminated, after all, who needs QA?

Loss of key personnel will decrease productivity by 15.7% and code quality by 21.7%.

Needless to say, when faced with such time sensitive projects the work done on the requirements is minimal and so the end product (even if delivered) will generally not satisfy the market pressure the project was designed to alleviate. In short, when senior management responds to market pressures by declaring a project and its deadline it is virtually guaranteed to fail.

How should senior managers deal with market pressures?

When faced with market pressure senior management must be absolutely certain that the pressure is real. There are many situations where managers have declared that a project needs to be finished by a given time, puts too much pressure on the project, and then the project is late or canceled.

Some managers make everything an emergency in order to get their way, they often say 'We have no choice'.

In these situations the manager will push for a ridiculous date that will be missed.  They will then turn around and accept a much later date for the the project; if this is the case then the project was not real and management was not responding to a real pressure. This happens regularly in environments where politics is more important than getting things done.

If the project must be done (i.e. compliance project) then it will be critical to assess complete and consistent requirements for the project. Those requirements need to be estimated by the project managers for the deadline and allow the resources to vary.  The best projects use formal requirements analysis so that they start with complete and consistent requirements; think about it, would you ever build a house without a solid blueprint?

Formal requirements analysis will increase productivity by 16.3% and code quality by 23.2%.

If the resources exist in the organization then they need to be dedicated to the high priority project and project scope trimmed if possible and the quality of the features need to be weakened until a feasible project is defined. If trimming the scope and bringing quality to a minimum will still not result in a feasible project then you need to consider what it means for the project to be late (because it WILL be late).  Your best chance of trimming scope correctly depends on formal scope management.

Formal scope management will increase productivity by 13.5% and code quality by 18.5%.

Any project which increases the overtime by the IT department must be done on a sustainable basis. The more time you demand from the IT department the less sustainable it is, i.e. don’t expect your IT department resources to do 80 hour weeks for 6 months.  Heavy overtime will lead to friction among team members.

Friction among team members will decrease productivity by 10.0% and code quality by 15.0%.

With high priority projects make sure that the engineers get some kind of bonus on a regular basis (free lunches, dinners) as well as a financial bonus on completion of the project. If no bonuses are available don't be surprised to see IT resources jump ship at or before the project completion date.

References

• Capers Jones, " Scoring Good and Bad Practices ", 2010
• T. Wayt Gibbs, "Software's Chronic Crisis", Scientific American, September 1994

Defects are for Losers

A developer is responsible for using any and all techniques to produce defect free code.  The average developer does not take advantage of all of the following opportunities to prevent and eliminate defects:

1. Before the code is written
2. As the code is written
3. Writing mechanisms for early detection
4. Before the code is executed
5. During unit testing
6. After the code is tested

---

The technique that is used most often is #6 above and will not be covered here.  It involves the following:

1. Code is delivered to the test department
2. The test department identifies defects and notifies development
3. Developer's fire up the debugger and try to chase down the defect

Like the 'rinse and repeat' process on a shampoo bottle, this process is repeated until the code is cleaned or until you run out of time and are forced to deliver.

The almost ubiquitous use of #6 leads to CIOs and VPs of Engineering assuming that the metric of one tester to two developers is a good thing.  Before assuming that #6 is 'the way to go' consider the other techniques and statistical evidence of their effectiveness.


Before the Code is Written

Before writing code, a developer has the most options available to him. The developer has an opportunity to plan his code, however, there are many developers who just 'start coding' on the assumption that they can fix it later.

How much of an effect can planning have?  Two methodologies that focus directly on planning at the personal and team level are the Personal Software Process (PSP) and the Team Software Process (TSP) invented by Watts Humphrey.


PSP can raise productivity by 21.2% and quality by 31.2%
TSP can raise productivity by 20.9% and quality by 30.9%

Not only does the PSP focus on code planning, it also makes developers aware of how many defects they actually create.  Here are two graphs that show the same group of developers and their defect injection rates before and after PSP training.

Before PSP training

After PSP training

The other planning techniques are:

• Decision tables
• Proper use of exceptions

Both are covered in the article Debuggers are for Losers and will not be covered here.

As the Code is Written

Many developers today use advanced IDEs to avoid common syntax errors from occurring   If you can not use such an IDE or the IDE does not provide that service then some of the techniques in the PSP can be used to track your injection of syntax errors and reduce those errors.

Pair Programming

One technique that can be used while code is being written is Pair Programming.  Pair programming is heavily used in eXtreme Programing (XP).  Pair programming not only allows code to be reviewed by a peer right away but also makes sure that there are two people who understand the code pathways through any section of code.

Pair programming is not cost effective overall (see Capers Jones).  For example, it makes little sense to pair program code that is mainly boiler plate, i.e. getter and setter classes.  What does make sense is that during code planning it will become clear which routines are more involved and which ones are not.  If the cyclomatic complexity of a routine is high (>15) then it makes sense for pair programming to be used.

If used for all development, Pair Programming can raise productivity by 2.7% and quality by 4.5%

Test Driven Development

Test driven development (TDD) is advocated by Kent Beck and stated in 2003 that TDD encourages simple designs and inspires confidence.  TDD fits into the category of automated unit testing.

Automated unit testing  can raise productivity by 16.5% and quality by 23.7%


Writing Mechanisms for Early Detection

Defects are caused by programs either computing wrong values, going down the wrong pathway, or both.  The nature of defects is that they tend to cascade and get bigger the further in time and space between the source of the defect and the noticeable effects of the defect.

Design By Contract

One way to build checkpoints into code is to use Design By Contract (DbC), a technique that was pioneered by the Eiffel programming language   It would be tedious and overkill to use DbC in every routine in a program, however, there are key points in every software program that get used very frequently.

Just like the roads that we use have highways, secondary roads, and tertiary roads -- DbC can be used on those highways and secondary roads to catch incorrect conditions and stop defects from being detected far away from the source of the problem.

Clearly very few of us program in Eiffel.  If you have access to Aspect Oriented Programming (AOP) then you can implement DbC via AOP. Today there are AOP implementations as a language extension or as a library for many current languages (Java, .NET, C++, PHP, Perl, Python, Ruby, etc).


Before the Code is Executed

Static Analysis

Most programming languages out there lend themselves to static analysis.  There are cost effective static analysis for virtually every language.

Automated static analysis can raise productivity by 20.9% and quality by 30.9%

Inspections

Of all the techniques mentioned above, the most potent pre-debugger technique is inspections. inspections are not sexy and they are very low tech, but the result of organizations that do software inspections borders on miraculous.

The power of software inspections can be seen in these two articles:

• Inspections are not optional
• Software Professionals do Inspections

Code inspections can raise productivity by 20.8% and quality by 30.8%
Design inspections can raise productivity by 16.9% and quality by 24.7%

From the Software Inspections book on p.22.

In one large IBM project, one half million lines of networked operating system, there were 11 development stages (document types: logic, test, user documentation) being Inspected.  The normal expectation at IBM, at that time, was that they would be happy only to experience about 800 defects in trial site operation.  They did in fact experience only 8 field trial defects.

Evidence suggests that every 1 hour of code inspection will reduce testing time by 4 hours

During Unit Testing
Developers have learned that improving code and getting rid of code "smells" can be accomplished through refactoring.   Unknown (or untrusted) by developers is the idea of using automated refactoring tools.

Automated refactoring can raise productivity by 8.0% and quality by 11.7%
Manual refactoring can raise productivity by 4.3% and quality by 6.7%

Conclusion
Overworked developers rarely have time to do research, even though it is clear that there is a wealth of information available on how to prevent and eliminate defects. The bottom line is that if your are only using technique #6 from the initial list, then you are not using every technique available to you to go after defects.

My opinion only, but:
A professional software developer uses every technique at his disposal to prevent and eliminate defects

---

Other articles in the "Loser" series

Moo?

Want to see more sacred cows get tipped? Check out

• Comments are for Losers
• Efficiency is for Losers
• Debuggers are for Losers
• Testing departments are for Losers

Make no mistake, I am the biggest "Loser" of them all.  I believe that I have made every mistake in the book at least once :-)

References

• Gilb, Tom and Graham, Dorothy. Software Inspections
• Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.
• Radice, Ronald A. High Quality Low Cost Software Inspections.

No Experience Required!

Did you know that we have never found a relationship between a developer's years of experience and code quality or productivity?

The original study that found huge variations in individual programming productivity was conducted in the late 1960s by Sackman, Erikson, and Grant (1968).  This study has been repeated at least 8 times over 30 years and the results have not changed! (see below) 

Sackman et al studied professional programmers with an average of 7 years’ experience and found that:

• the ratio of initial coding time was about 20 to 1
• the ratio of debugging times over 25 to 1
• program execution speed about 10 to 1
• program size 5 to 1

They found no relationship between a programmer's number of years of experience and code quality or productivity.  That is there was NO correlation between experience and productivity (i.e. ability to produce code) and there was NO correlation between experience and quality (i.e. minimizing defects) .

Think about that for a minute...

That is the worst programmers and the best programmers made distinct groups and each group had people of low and high experience levels.  Whether training helps developers or not is not indicated by these findings, only that years of experience do not matter.  If training does matter then very few developers are getting trained.

Without considering legality, this means that it is simpler to get rid of expensive poor performers with many years of experience and hire good performers with few years of experience!

Results Have Been Confirmed for 30 Years!

There were flaws in the study, however, even after accounting for the flaws, their data still shows more than an order of magnitude difference between the best programmers and the worst, and that difference was not related to experience.  In years since the original study, the general finding that "There are order-of-magnitude differences among programmers" has been confirmed by many other studies of professional programmers (full references at the end of the article):

• Curtis 1981
• Mills 1983
• DeMarco and Lister 1985
• Curtis et al. 1986
• Card 1987
• Boehm and Papaccio 1988
• Valett and McGarry 1989
• Boehm et al 2000

Technology is More Sophisticated, Developers are not

You might  think that we know much more about software development today than we knew in 1968, after all today: 

• we have better computer languages
• we have more sophisticated technology
• we have better research on effective software patterns
• we have formal software degrees available in university

It turns out that all these things are true, but we still have order of magnitude differences among programmers and the difference is not related to years of experience.  That means that there is some other x-factor that drives productive developers;  that x-factor is probably the ability to plan and make good decisions.

The bad news is that if you are not a productive developer writing quality code  then you will probably not get better simply because of years of experience.

Developers face making decisions on how to structure their code every day.  There is always a choice when it comes to:

• laying out code pathways
• packaging functions into classes
• packaging classes into packages/modules

Because developers face coding decisions, many of which are complex, the best developers will plan their work and make good decisions.  Bad developers just 'jump in'; they assume that they can always rewrite code or make up for bad decisions later. Bad developers are not even aware that their decision processes are poor and that they can become much better by planning their work.

Solution might be PSP and TSP

Watts Humphrey tried to get developers to understand the value of estimating, planning development, and making decisions in the Personal Software Process (PSP) for individuals and the Team Software Process (TSP) for teams, but only a handful of organizations have embraced it.  Capers Jones has done analysis of over 18,000 projects and discovered that¹:

PSP can raise productivity by 21.2% and quality by 31.2%

TSP can raise productivity by 20.9% and quality by 30.9%

All of these findings should have a profound effect on the way that we build our teams.  Rather than having large teams of mediocre developers, it makes much more sense to have smaller teams of highly productive developers that know how to plan and make good decisions.  The PSP and TSP do suggest that the best way to rehabilitate a poor developer is to teach them how to make better decisions.

Be aware, there is a difference between knowledge of technologies which is gained over time and the ability to be productive and write quality code.

Conclusion

We inherently know this, we just don't do it.  If the senior management of organizations only knew about these papers, we could make sure that the productive people get paid what they are worth and the non-productive people could seek employment in some other field.  This would not only reduce the cost of building software but also increase the quality of the software that is produced.

Unfortunately, we are doomed to religious battles where people debate methodologies, languages, and technologies in the foreseeable future.  The way that most organizations develop code makes voodoo look like a science!

Eventually we'll put the 'science' back in Computer Science, I just don't know if it will be in my lifetime...

Bibliography

Boehm, Barry W., and Philip N. Papaccio. 1988. "Understanding and Controlling Software Costs." IEEE Transactions on Software Engineering SE-14, no. 10 (October): 1462-77.

Boehm, Barry, et al, 2000. Software Cost Estimation with Cocomo II, Boston, Mass.: Addison Wesley, 2000.

Card, David N. 1987. "A Software Technology Evaluation Program." Information and Software Technology 29, no. 6 (July/August): 291-300.

Curtis, Bill. 1981. "Substantiating Programmer Variability." Proceedings of the IEEE 69, no. 7: 846.

Curtis, Bill, et al. 1986. "Software Psychology: The Need for an Interdisciplinary Program." Proceedings of the IEEE 74, no. 8: 1092-1106.

DeMarco, Tom, and Timothy Lister. 1985. "Programmer Performance and the Effects of the Workplace." Proceedings of the 8th International Conference on Software Engineering. Washington, D.C.: IEEE Computer Society Press, 268-72.

¹Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.

Mills, Harlan D. 1983. Software Productivity. Boston, Mass.: Little, Brown.

Valett, J., and F. E. McGarry. 1989. "A Summary of Software Measurement Experiences in the Software Engineering Laboratory." Journal of Systems and Software 9, no. 2 (February): 137-48.

Death March Calculus

In the previous post Stop It! No… really stop it. we talked about 5 worst practices and their impact on a software project.  However, all 5 worst practices will be present in a Death March.

Ed Yourdon, one of the pioneers in software engineering, wrote "Death March" to describe the common practice of setting up a software project which is not feasible.

That is, between the time, resources, and functionality (quality) desired for the project, there is no way to complete the project successfully.   Ed states that a death march is one in which one of the project parameters exceeds the norm by at least 50%.

A death march project is often known by the entire project team to be virtually impossible to succeed at.  To understand some of the motivation of why senior management creates death marches or why people participate in them you would need to read Ed's book.


In the previous article we showed this table:

Worst Practice	Productivity	Quality
Friction/antagonism among team members	-12.0%	-15.0%
Friction/antagonism among management	-13.5%	-18.5%
Inadequate communications with stakeholders	-13.5%	-18.5%
Layoffs/loss of key personnel	-15.7%	-21.7%
Excessive schedule pressure	-16.0%	-22.5%

Since senior management has asked for specific functionality that can not be produced by the current team in the given time frame, the first problem is the presence of excessive schedule pressure.  So right at the start of the project your productivity will be down 16% and quality will be down 22.5%.

With a compressed time schedule there will not be enough time to capture all the requirements from the stakeholders.  You will have to settle for partial requirements and hope to interpolate the missing requirements.  This will cause inadequate communications with stakeholders and drop the productivity 13.5% and the quality by 18.5%.  Unfortunately, these effects are cumulative with the excessive schedule pressure and you will now be down 29.5% productivity and 41% on quality.

There is no shortage of opinions in a software team; in the best teams they combine their opinions to make the best decisions.  When time is compressed we don't make good decisions in our teams and end up with friction at the management and team level. This will cause us to have friction/antagonism among management because they will have different opinions on how to execute the death march;  antagonism in management will lead to friction/antagonism among team members. These two issues will combine to drop productivity by 27% and quality by 37%.  The cumulative effect of all practices will have productivity down 64.5% and quality down by 78%.

When the project is obviously not going well, people will quit and/or be fired.  In particular, key personnel have a tendency to leave because they are the ones that have the easiest time finding new employment. If this happens then the final numbers will be productivity down 86.2% and quality down by 98.7%.

If you don't believe the numbers then you have never been on a death march!

The problem is that excessive schedule pressure will inevitably cause the other worst practices to come into existence.  Starting a project where excessive schedule pressure is present is virtually a guarantee that failure will occur.  In fact, the only death marches that have a chance at success are small teams with projects that are 3-6 months long.

Possible Way Out

The only way to solve the problem of excessive schedule pressure is to balance the three elements (time, resources, and functionality) of the project until you have a feasible project.  Since project teams are relatively fixed this means:

• reducing the functionality that needs to be produced
• moving back the project end date
• both reducing functionality and moving back the project end date

Experienced project managers on a death march will be aggressively reducing scope as soon as the project starts.  But this is no guarantee that they will get rid of enough functionality to bring the project into a feasible state.

If your organization has put you on a death march then the only possible solution is to find another job.  Hoping that somehow a successful project will result when you start off with excessive schedule pressure is futile, the mathematics shows that project failure is a virtual certainty.

You have more chances of winning big in the lottery and retiring than of completing a death march successfully.

Stop It! No… really stop it.

There are 5 worst practices in software development that if stopped immediately will  improve your productivity by a minimum of 12% and improve quality by a minimum of 15%.

These practices are so common that people assume that they are normal -- they are not, they are silent killers wherever they are present.

We often hear the term best practices enough to know that we all have different definitions for it. Even when we agree on best practices we then disagree on how to implement and measure them. A best practice is one that increases the chance your project will succeed.

How often do we talk about worst practices? More importantly, what about those worst practices in your organization that you don't do anything about?

---

When it comes to a worst practice, just stop it.

---

If your company is practicing even one worst practice in the list below it will kill all your productivity and quality. It will leave you with suboptimal and defective software solutions and canceled projects. To make matters worse, some of the worst practices will cause other worst practices to come into play. Capers Jones had statistics on over 18,000 projects and has hard evidence on the worst practices¹.

The 5 Worst Practices

The worst practices and their effect on productivity and quality are as follows:

Worst Practice	Productivity	Quality
Friction/antagonism among team members	-12.0%	-15.0%
Friction/antagonism among management	-13.5%	-18.5%
Inadequate communications with stakeholders	-13.5%	-18.5%
Layoffs/loss of key personnel	-15.7%	-21.7%
Excessive schedule pressure	-16.0%	-22.5%

Excessive Schedule Pressure

Excessive schedule pressure is present whenever any of the following are practiced:

• Senior management declares the project deadline
  • See Why Senior Management Declared Deadlines Lead to Disaster
• Formal estimation methods are used but senior management ignores them
• Informal estimation methods (i.e. SWAG) are used

Excessive schedule pressure causes the following to happen:

• Poor and incomplete requirements are captured causing extreme scope creep
  • See Shift Happens
• Sub-optimal architecture is built
• Highly defective code is written
• Little quality control is done during testing
• Few, if any, artifact inspections are done
  • See Inspections are not Optional
  • See Professionals do Software Inspections

This alone can create a Death March project and virtually guarantee project failure.

Effect of excessive schedule pressure is that productivity will be down 16% and quality will be down 22%

Not only is excessive schedule pressure one of the worst practices it tends to drive the other worst practices:

• Friction amongst managers
• Friction amongst team members
• Increases the chance that key people leave the organization

---

If your organization has a habit of imposing excessive schedule pressure -- leave!

---

Friction Between People

Software development is a team activity in which we transform our intangible thoughts into tangible working code. Team spirit and collaboration is not an option if you want to succeed. The only sports teams that win championships are those that are cohesive and play well together.

You don't have to like everyone on your team and you don't have to agree with all their decisions. However, you must understand that the team is more important than any single individual and learn to work through your differences.

---

Teams only work well when they are hard on the problem, not each other!

---

Friction among managers because of different perspectives on resource allocation, objectives, and requirements. It is much more important for managers to come to a consensus than to fight for the sake of fighting. Not being able to come to a consensus will cave in projects and make ALL the managers look bad. Managers win together and lose together.

Effect of management friction is that productivity will be down 13.5% and quality will be down 18.5%

Friction among team members because of different perspectives on requirements, design, and priority. It is also much more important for the team to come to a consensus than to fight for the sake of fighting. Again, everyone wins together and loses together -- you can not win and have anyone else lose.

Effect of team friction is that productivity will be down 12% and quality will be down 15%

---

Any form of friction between managers or the team is deadly.

---

Inadequate Stakeholder Communication

Inadequate stakeholder communication comes in several forms:

• Not getting enough information on business objectives
• Not developing software in a transparent manner

If you have insufficient information on the business objectives of a project then you are unlikely to capture the correct requirements. If you are not transparent in how you are developing the project then you can expect excessive schedule pressure from senior management.

Effect of inadequate stakeholder communication is that productivity will be down 13.5% and quality will be down 18.5%

Loss of Key Personnel

To add insult to injury, any of the other four worst practices above will lead to either:

• Key personnel leaving your organization
• Key personnel being layed off

The loss of key personnel through turnover is not really a worst practice, but creating the conditions under which people leave is a worst practice. Badly managed organizations and projects will cause the most competent people to leave the organization, simply because they can more easily get a job in another organization.

When organizations experience financial distress from late projects they will often cut key personnel because they are expensive. The reality is that laying off key personnel will sandbag your ability to get back on track; the correct thing to do is to find your least effective personnel and let them go.

Effect of layoffs/loss of key personnel is that productivity will be down 15.7% and quality will be down 21.7%

---

The loss of key personnel has a dramatic effect on team productivity and morale and a direct effect on product quality.

---

Conclusion

Any of the worst practices mentioned above will cause a project to be late and deliver defective code. Even worse, the worst practices tend to feed each other and cause a negative spiral. If you are in an organization that habitually practices any of these worst practices then your only real option is to quit. The most deadly situation is when there is the following cascading of worst practices:

• Excessive schedule pressure (leads to)
• Management and team friction (leads to)
• Loss of key personnel

---

If you are in senior management then none of these practices can be allowed if you want to avoid canceled projects or highly defective products.

---

¹Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.

Testing Departments are for Losers

I understand that this is a very strong statement and I'm definitely not trying to insult anyone; so apologies in advance.  I'm trying to challenge the belief that testing is mandatory and that there should be one testing resource for every two developers.

Quality development is about quality assurance and zero defects not about having testing departments. Developers are the ones who inject defects into the code and therefore they are the best line of defense to remove them.  The developer has the best information on what needs to be tested in his code at the time that he writes it.  The longer it takes for testing or a customer to discover a code defect the longer the developer will spend in a debugger chasing down the problem.

One testing resource for every two developers is not a good solution

Quality Assurance (QA) is about making a positive statement about product quality.  QA is about positive assurance, which is stating,  “We are certain that there are few, if any, defects in this product.”  Airplanes are a product with quality assured, the manufacturers will stand by their quality and the statistics back them up.  Contrast this with this article or this article which wonders what would happen if Microsoft made airplanes -- would you fly in them?  This is not to say that software needs to have the same quality level as an airplane, only that whatever quality level you aspire to should be assured.

The reality is that most testing departments simply discover defects and forward them back to the engineering department to fix.  By the time the software product gets released we are basically saying, “We got rid of as many defects as we could find before management forced us to release this product, however, we really have no idea how many other defects are in the code”.  This is not assuring quality; at best you get negative assurance out of this.

Everyone understand that buggy software kills sales (and start-ups :-)), however, testing is often an after thought in many organizations.  When software products take longer than expected, the testing department is often expected to test and bless code in less time than allocated.

To compound problems, many testing departments don't even receive proper requirements against which to test the code and/or sufficient tools to work with. Large testing departments and/or large amounts of manual testing are not healthy or efficient.

Humphrey Watts was emphatic that calling defects “bugs” trivializes the issue and downplays the negative impact that defects cause on a development organization.

Calling defects "bugs" trivializes an important issue

Defects are not introduced into software by goblins and elves.  Defects are injected into the code by developers that:

• don't understand the requirements or architecture
• misunderstand how to use their peer's components
• misunderstand 3rd party libraries
• having a bad day because of home troubles or work environment
• are careless because someone else will test their code

Defects are injected by the team

No one is more aware of how code can break down than the developer who writes it.   Any line of code that is written without concentration and planning becomes a potential defect.  It is impossible for testers to understand every pathway through the code and make sure that every possible combination of variables is properly taken care of.

There are many techniques that can increase code quality and dramatically reduce the amount of testing that is necessary:

• test driven development (TDD)
• database driven testing
• design by contract (DbC)
• inspections
• pair programming
• minimizing cyclomatic complexity
• using static and dynamic code tools
• proper code planning techniques

Test Driven Development

Properly written tests require a developer not only to think about what a code section is supposed to do but also plan how the code will be structured.  If you know that there are  five pathways through the code then you will write five tests ahead of time.  A common problem is that you have coded n paths through the code when there are n+1 conditions.

TDD is white box testing and can reach every pathway that the developer codes.  TDD is proactive and can test pathways from end to end, it does not just have to be used for unit testing.  When TDD is hooked up to a continuous integration engine then defects are located and fixed before they make it to testing.

Database Driven Testing

Using actual test data to test existing routines during development is an excellent way to make sure that there are fewer production problems.  The test data needs to be a copy (or subset) of production data.

Database driven testing can also be hooked up to a continuous integration engine and prevent defects from getting to testing.

Design By Contract

The Eiffel programming language introduced design by contract (DbC).  DbC is orthogonal to TDD because its goal is to ensure that the contract defined by the preconditions and postconditions for each function call is not violated.  DbC can be used in virtually any language for with their is an Aspect Oriented Programming (AOP) solution.

During development, the minute a developer violates the expected contract of any function (his or a peers) then the developer will get feedback to fix the problem before it gets to testing.

Inspections

Since the 1970s we have statistical evidence that one of the best ways to eliminate defects from code is through inspections.  Inspections can be applied to the requirements, design, and code artifacts and projects that use inspections can eliminate 99% of the defects injected into the code.  Se Inspections are not Optional and Software Professionals do Inspections.

Each hour of inspections will save you 4 hours of testing

Pair Programming

Pair programming can be selectively used to prevent and eliminate defects from code.   When developers work in pairs they not only review code as quickly as possible but also learn productivity techniques from each other.  Pair programming should only be done on complex sections of code. Pair programming not only eliminates defects but allows developers to get enough feedback that they can prevent defects in the future.

Minimizing Cyclomatic Complexity

There is evidence that routines with high cyclomatic complexity will have more latent defects than other routines.   This makes sense because the number of code pathways goes up dramatically as cyclomatic complexity increases and increases the chance that the developer does not handle all of them.   In most cases, testing departments can not reproduce all of the pathways in routines of high cyclomatic complexity.

Use Dynamic and Static Code Checking

There are many code problems caused by a careless use of pointers and other powerful language constructs.  Many of these problems can be detected by having the development team use dynamic and static code checking problems.

Proper Code Planning Techniques

There are developers that try to write code at the keyboard without planning, which is neither efficient nor effective.  This is like having to do errands in 5 locations and driving to the locations randomly -- you might get your errands done, but odds are it won't be efficient.

Watts Humphrey talks directly to the idea of planning in the Personal Software Process.  In addition techniques like diagramming with UML or using decision tables can go a long way to thinking through code structure before it is implemented.

Conclusion

Developers are the ones who inject defects into the code and therefore they are the best line of defense to remove them.  The developer has the best information on what needs to be tested in his code at the time that he writes it.  The longer it takes for testing or a customer to discover a code defect the longer the developer will spend in a debugger chasing down the problem.

Developers need to be trained in preventing and eliminating defects.  Developers who learn to get the code correct the first time will reduce and eliminate effort in testing.

One goal of development should be to catch defects early; this is the only way to assure quality.  Hence quality assurance starts and finishes in the development department, not the testing department.

Software Professionals do Inspections

Are you a software professional or not?

I'm not talking about having some kind of official certification here.  I'm asking whether creating high quality code on a repeatable basis is your top priority.

Professionals do everything possible to write quality code. Are you and your organization doing everything possible to write quality code?  Of course, whether you are a professional or not can only be answered by your peers.

If you are not doing software inspections then you are not doing everything possible to improve the quality of your code.  Software inspections are not the same as code walk throughs, which are used to inform the rest of the team about what you have written and are used mainly for educational purposes.  Walk throughs will find surface defects, but most walk throughs are not designed to find as many defects as possible.  

How do defects get into the code?  It's not like there are elves and goblins that come out at night and put defects into your code.  If the defects are there it is because the team injected them.  Many defects can be discovered and prevented before they cause problems for development.  Defects are only identified when you go looking for them, and that is typically only in QA.

Benefits of Inspections

Inspections involve several people and  require intense preparation before conducting the review. The purpose of inspections is to find defects and eliminate them as early as possible.  Inspections apply to every artifact of software development:

• Requirements (use cases, user stories)
• Design (high level and low level, UML diagrams)
• Code
• Test plans and cases

Inspections as a methodology have been around since the 1970s and certainly well codified since M. E. Fagin wrote a paper in the IEEE in 1986.  The idea behind inspections is to find defects as early as possible in the software development process and eliminate them.  Without inspections, defects accumulate in the code until testing when you discover all the defects from every phase of development simultaneously.

This diagram from Radice shows that defects will accumulate until testing begins.  Your quality will be limited by the number of defects that you can find before you ship your software.

With inspections, you begin to inspect your artifacts (use cases, user stories, UML diagrams, code, test plans, etc) as they are produced.  You attempt to eliminate defects before they have a chance to cascade and cause other phases of software development to create defects.  For example, a defect during requirements or in the architecture can cause coding problems that are detected very late (see Inspections are not Optional). With inspections the defect injection and removal curve looks like this:



When effective inspections are mandatory, the quality gap shrinks and the quality of the software produced goes up dramatically.  In the Economics of Software Quality, Capers Jones and  Olivier Bonsignour show that defect removal rates rarely top 80% without inspections;  but with inspections you can get to 97%.

Why Don't We Do Inspections?

There is a mistaken belief that inspections waste time.  Yet study after study shows that inspections will dramatically reduce the amount of time in quality assurance.  There is no doubt that inspections require an up-front effort, but that up-front effort pays back with dividends. The hidden effect of inspections is as follows:

 

The issue is that people know that they make mistakes but don't want to admit it, i.e. who wants to admit that they put the milk in the cupboard? They certainly don't want their peers to know about it!

Many defects in a software system are caused by ignorance, a lack of due diligence, or simply a lack of concentration.   Most of these defects can be found by inspection, however, people feel embarrassed and exposed in inspections because simple problems become apparent.

For inspections to work, they must be conducted in a non-judgemental environment where the goal is to eliminate defects and improve quality.  When inspections turn into witch hunts and/or the focus is on style rather than on substance then inspections will fail miserably and they will become a waste of time.

Professional software developers are concerned with high quality code.  Finding out as soon as possible how you inject defects into code is the fastest way to learn how to prevent those defects in the future and become a better developer.  Professionals are always asking themselves how they can become better, do you?

Conclusion

Code inspections have been done for 40 years and offer conclusive proof that they greatly improve software quality without increasing cost or time for delivery.  If you are not doing inspections then you are not producing the best quality software possible

---

Bibliography

• Bonsignour, Olivier and Jones, Capers; The Economics of Software Quality; Addison Wesley, Reading, MA; 2011; ISBN 10: 0132582201.
• Fagan, M.E., Advances in Software Inspections, July 1986, IEEE Transactions on Software Engineering, Vol. SE-12, No. 7, Page 744-751
• Gilb, Tom and Graham, Dorothy; Software Inspections; Addison Wesley, Reading, MA; 1993; ISBN 10: 0201631814.
• Radice, Ronald A.; High Qualitiy Low Cost Software Inspections; Paradox icon Publishing Andover, MA; ISBN 0-9645913-1-6; 2002; 479 pages.
• Wiegers, Karl E.; Peer Reviews in Software – A Practical Guide; Addison Wesley Longman, Boston, MA; ISBN 0-201-73485-0; 2002; 232 pages.

---