Friday, 22 August 2014

Infeasible Projects: Executive Ignorance or IT Impotence?

DohDoh2Infeasible software projects are launched all the time and teams are continually caught up in them, but what is the real source of the problem?

There are 2 year actual projects for which the executives set a 6 month deadline.

The project is guaranteed to fail but is this due to executive ignorance or IT impotence? InfeasibleTimeline

There is no schedule risk in an infeasible project because the deadline will be missed.  Schedule risk only exists in the presence of uncertainty (see Schedule Risk is a Red Herring!!!)

As you might expect, all executives and IT manager share responsibility for infeasible projects that turn into death marches.  Learn about the nasty side effects Death March Calculus. The primary causes for infeasible projects are:
  • Rejection of formal estimates
  • No estimation or improper estimation methods are used

Rejecting Formal Estimates

This situation occurs frequently; an example would be the Denver Baggage Handling System (see Case Study).

The project was automatically estimated (correctly) to take 2 years; however, executives declared that IT would only have 1 year to deliver.

Of course, they failed1.

The deadline was rejected by executives because it did not fit their desires.  They could not have enjoyed the subsequent software disaster and bad press.

When executives ignore formal estimates they get what they deserve.  Formal estimates are ignored because executives believe through sheer force of will that they can set deadlines.

If IT managed to get the organization to pay for formal tools for estimating then it is not their problem that the executives refuse to go along with it.


Improper Estimation Methods

The next situation that occurs frequently is using estimation processes that have low validity.  Estimation has been extensively studied and documented by Tom DeMarco, Capers Jones, Ed Yourdon, and others.

Improper estimation methods will underestimate a software project every time. Fast estimates will be based on what you can think of, unfortunately, software is not tangible and so what you are aware of is like the tip of an iceberg.

None of this prevents executives demanding fast estimates from development.  Even worse, development managers will cave in to ridiculous demands and actually give fast estimates.

Poor estimates are guaranteed to lead to infeasible projects (see Who needs Formal Measurement?) Poor estimates are delivered by IT managers that:
  • Can't convince executives to use formal tools
  • Give in to extreme pressure for fast estimates
Infeasible projects that result from poor estimates are a matter of IT impotence.

Conclusion

Both executive ignorance and IT impotence lead to infeasible projects on a regular basis because of poor estimates and rejecting estimates; so there is no surprise here.

However, infeasible projects are a failure of executives and IT equally because we are all on the same team. It is not possible for part of the organization to succeed if the other one fails.

Possibly a greater share of problem is with IT management.  After all, whose responsibility is a bad decision -- the guys that know what the issues are or the ones that don't.

If a child wants ice cream before they eat dinner then whose fault is it if you cave in and give them the ice cream?

Unfortunately, even after 60 years of developing software projects, IT managers are either as ignorant as the executives or simply have no intestinal fortitude.

Even when IT managers convince executives of the importance of estimating tools, the estimates are routinely discarded because they do not meet executive expectations.

Rejection of automated estimates: productivity -16%, quality -22%

Until we can get a generation of IT managers that are prepared to educate executives on the necessity of proper estimation and be stubborn about holding to those estimates, we are likely to continue to have an estimated $3 trillion in failures of software projects every year.


End Notes

1For inquiring minds, good automated estimation systems have been shown to be within 5% of time and cost on a regular basis. Contact me for additional information.

References

Tuesday, 12 August 2014

Schedule Risk is a Red Herring!!!

We often hear the term schedule risk, however, it is generally a Red Herring. Stating that the schedule might stretch is about as useful as saying that eating can cause you to gain weight.

You may be correct but it gives you no leverage to solve the problem

Schedules slip as a result of a problem, if you want to solve the problem then you must identify the root cause.  Any problem will result in a task taking longer than expected and potentially affecting the schedule.

Risk and uncertainty are two sides of the same coin. Without uncertainty there is no risk.

No Uncertainty = No Risk

A risk is a contingent liability; a risk is a future event that is uncertain that has consequences. The key words are future and uncertain.

If 6 months remain and the deadline is in 2 months then there is no schedule risk because there is no uncertainty.

6 months late means that the earliest that the critical path items can finish is in 6 months. Just because the project has not hit the deadline and senior staff doesn't understand the project is late does not qualify the team to talk as if the outcome is uncertain.

It is disingenuous and cowardly to suggest to senior staff that a deadline is possible when you know that it is not.

When the team knows that they are late, they often talk about tasks as being risky simply because they hope that miracles can happen1.

Hope is not a strategy

In fact, Kahneman points out all of us are wired to bet (pray?) on unlikely outcomes when faced with certain losses, i.e. we double down when faced with a loss.  Team members know about the negative consequences of failure and make projects seem possible simply because they want to delay the pain. Even worse, as the situation gets more desperate people will take bigger and bigger risks.

Using the term schedule risk when a project is not feasible essentially robs the managers of making a course correction until the point where very little can be done.

At a minimum, money can be saved by winding the project down. Few people have the intestinal fortitude to speak out when they know that a project is late. Unfortunately, cowardice is very common.

If you take a paycheck then you have an obligation to your organization to tell them when a project is late.

So it makes no sense to talk about schedule risk when:
  • The project is late and you know it
  • The project is not late but you see schedule items slipping
In the latter case you are much better to talk about why things are slipping rather than using the term schedule risk.  By talking about the root cause of the slippage, especially early in a project, can lead to you either solving the problem or adjusting the project deadline.  Either way, you will have a greater chance of ending up with a feasible project.

Related Articles

References

Saturday, 12 July 2014

User Stories are Rarely Appropriate

All tools are useful when used appropriately, and User Stories are no different.

User stories are fantastic when used in small teams on small projects where the team is co-located and has easy access to customers.

User stories can quickly fall apart under any of the following situations:
  • the team or project is not small
  • the team is not in a single location
  • customers are hard to access
  • project end date must be relatively fixed
User stories were introduced as a core part of Extreme Programming (XP). Extreme Programming assumes you have small co-located teams; if you relax (or abandon) any of these constraints and you will probably end up with a process out of control.

XP, and hence user stories, works in high intensity environments where there are strong feedback loops inside the team and with customers:
  • Individuals and Interactions over Processes and Tools
  • Customer Collaboration over Contract Negotiation

User stories need intense intra-team / customer communication to succeed

User stories are a light-weight methodology that facilitates intense interactions between customers and developers and put the emphasis on the creation of code, not documentation. Their simplicity makes it easy for customers to help write them, but they must be complemented with timely interactions so that issues can be clarified.

Large teams dilute interactions between developers; infrequent communication leads to a lack of team synchronization. Most organizations break larger teams into smaller groups where communication is primarily via email or managers -- this kills communication and interaction.

Larger projects have non-trivial architectures. Building non-trivial architecture by only looking at the end user requirements is impossible. This is like having all the leaves of a tree and thinking you can quickly determine where the branches and the trunk should go.

User stories don't work with teams where intense interaction is not possible. Teams distributed over multiple locations or time zones do not allow intense interaction. You are delusional if you think regular conference calls constitute intense interaction; most stand-up calls done via conference degrade into design or defect sessions.

When emphasis is on the writing of code then it is critical that customers can be accessed in a timely fashion. If your customers are indirectly accessible through product managers or account representatives every few days then you will end up with tremendous latency.

Live weekly demos with customers are necessary to flush out misunderstandings quickly and keep you on the same page

User stories are virtually impossible to estimate. Often, we use user stories because there is a high degree of requirements uncertainty either because the requirements are unknown or it is difficult to get consistent requirements from customers.

Since user stories are difficult to estimate, especially since you don't know all the requirements, project end dates are impossible to predict with accuracy.

To summarize, intense interactions between customers and developers are critical for user stories to be effective because this does several things:
  • it keeps all the customers and developers on the same page
  • it flushes out misunderstandings as quickly as possible

All of the issues listed initially dilute the intensity of communication either between the team members or the developers and customers. Each issue that increases latency of communication will increase misunderstandings and increase the time it takes to find and remove defects.

So if you have any of the following:
  • Large or distributed teams
  • Project with non-trivial architecture
  • Difficult access to customers, i.e. high latency
  • High requirements uncertainty but you need a fixed project end-date
Then user stories are probably not your best choice of requirements methodology. At best you may be able to complement your user stories with storyboards, at worst you may need some light-weight form of use case.

Light-weight use case tutorial:

Other requirements articles:

Tuesday, 8 July 2014

Seriously. The Devil Made me do It!

Just as eternal as the cosmic struggle between good and evil is the challenge between our two natures. Religion aside, we have two natures, the part of us that:
  • thinks things through; makes good or ethical decisions a.k.a. our angelic nature
  • reacts immediately; makes quick but often wrong decisions a.k.a. our devil nature
Guess the powers that be left a bug in our brains so that it emphasizes fast decisions over good / ethical decisions.

Quite often we make sub-optimal or ethically ambiguous decisions under pressure

You decide...

Situation: Your manager comes to you and says that something urgent needs to be fixed right away. Turns out the steaming pile of @#$%$ that you inherited from Bob is malfunctioning again. 

Of course Bob created the mess and then conveniently left the company; in fact, the code is so bad that the work-arounds have work-arounds.

Bite the bullet, start re-factoring the program when things goes wrong.  It will take more time up front, but over time the program will become stable.  Start code inspections on the critical sections to proactively reduce defects (see Inspections are not Optional)

Find another fast workaround and defer the problem to the future.  Find a good reason why the junior member of the team should inherit this problem.

Situation: You've got a challenging section of code to write and not much time to write it.

Get away from the computer, think things through.  Get input from your peers, maybe they have seen this problem before. 

Then plan the pathways out and write the code once cleanly. Taking time to plan seems counter intuitive, but it will save time. (see Not Planning is for Losers)

Naw, just sit at the keyboard and bang it out already.  How difficult can it be?

Situation: The project is late and you know that your piece is behind schedule. However, you also know that several other pieces are late as well.

Admit that you are late and that the project can't finish by the deadline.  Give the project manager and senior managers a chance to make a course correction.

Say that you are on schedule but you are not sure that other people (be vague here) will have their pieces ready on time and it could cause you to become late.

This situation is also known as Schedule Chicken...

Situation: You have been asked to estimate how long a critical project will take. You are only been given a short time to come up with the estimate.

Tell the project manager that getting a proper estimate takes longer than a few hours. Without proper estimates the project is likely to be severely underestimated and this will come back to bite you and the project manager in the @$$.  (See Who needs Formal Measurement?)

Tell the project manager exactly the date that senior management wants the project to be finished by.  You know this is what they want to hear, why deal with the problem now? This will become the project manager's problem when the project is late.

The statistics show that we often don't listen to our better (angelic?) natures very often.

So when push comes to shove and you have to make a sub-optimal or less than ethical decision, just remember:
The devil made you do it!
Run into other common situations? email me