Friday, 7 October 2016

Why Outsourcing Fails

Every year, we see corporations outsource operations only to pull back back some, if not all, of the operations. When this happens, the cost of pulling back operations or splitting things over multiple countries can leave you with significant operational issues and similar or worse costs.

Successful corporations understand the need to control costs in operations. The "spare no expense" philosophy of John Hammond in Jurassic Park is a one way ticket to disaster.

But, a focus on cost reduction is what leads corporations to face-fault when they outsource.

Corporations are seduced by the idea of potentially cutting costs dramatically; why pay expensive resources to do repetitive things when overseas people can do the same work for pennies?

The cardinal rule when outsourcing is:

Only operations that you completely understand and control can be outsourced 

To understand operations is to understand the costs of exceptions as well as how they are processed and escalated. Local resources deal efficiently with exceptions because they have informal relationships in the organization and use them to get things done efficiently; those relationships are not available to outsourced resources.

When operations gets outsourced, not only do foreign employees not understand how to deal with exceptions but also:

  • They are often in another time zone 
  • They don't have full access to local resources 
  • They don't know who to contact inside the corporation 
  • They don't understand local business rules 
  • The network configuration can make it impossible to communicate properly 

Understanding an operational function means that you capture all details, including exceptions, in your information systems. Often, legacy information systems capture exception information as unstructured fields (i.e. memo fields) and off shore resources will not understand this.

99% of your operations might be straight forward, but the 1% of exceptions can at best lead to longer implementation times.  Best case scenario is that you will look bad to your customers and at worst, it will kill your bottom line.

If your information systems contain exception information in a structured way then at least you can eventually train foreign resources to resolve them. But unless exception information is captured so you can control it (i.e. not in general text fields) then it is the same thing as not having it.

When outsourced resources are under strict performance guidelines, they will do anything to clear their work queues. This often involves kicking back work they perceive as an exception as improperly specified; they will claim 'garbage in, garbage out'.

This will result in miscommunications and longer lead times to implement your services. Not only will delivery times, and errors go up, and your customers will get mad.

By the time you have fired the local employees, all the knowledge of the function has left the building and you are left with a broken system.

Key questions to get answers to:

  • What does each exception in operations cost? 
  • How often do they happen? 
  • Can the information systems record exceptions in a way that you can track and control them? 
  • Are the mechanisms that local resources use to resolve exceptions be used by outsourced resources? 

If you don't have a clear answer to each of these questions then outsourcing is likely to be a disaster. When you have a clear answer to the 3 questions and understand how you will address each exception then you will have much more success outsourcing.

Wednesday, 10 August 2016

Project failure is due to bad requirements

Projects can fail for a number of reasons, but at the root of most failures is a failure to gather correct and consistent requirements.  We've all laughed at some variant of the above diagram, but these issues are all because:
  • We fail to capture correct and consistent requirements
  • We play "broken telephone" when we are communicating requirements
Let's take a concrete example.  Suppose we have an orienteering challenge where you need to go from the start point below to the finish point, this is the actual requirement.  

But, there is a gap between what the actual requirements are and what is actually written down.

Good Requirements

As long as the written requirements don't diverge too much from the actual requirements, you will have time to adjust the requirements during project execution and still get to the finish point.  

So as long as the initial written requirements are in the green zone, you can still complete the project on time because the requirements point you in the correct direction.

Mediocre Requirements

Now suppose one of the following happen:
  • You don't have all the core requirements because insufficient people were consulted
  • You have conflicting requirements from different sources
When you have less than accurate requirements you will be in trouble.  The initial code and architecture of the project will be created based on the quality of the requirements.  If those requirements are suspect, then there will be rework as code needs to be adjusted as the scope of the project seems to shift.

So now you will find yourself in one of the yellow zones below

Even with the best execution, the project will be challenged.  Deadlines will be missed as you attempt to bring the requirements back to what they need to be.  This is like trying to change a tire on a car and discovering the jack is missing.

It is important to realize that adjusting poor requirements is not Scope Creep.  Fixing incorrect and inconsistent requirements is necessary and it is pointless for a project manager or executive to disallow these changes.  It would be worthwhile for project postmortems if the project manager tracked whether a requirement was missing or inconsistent.

Challenged projects are typically declared as successes, but only after massive compromises, burned out resources, damage to reputations, and loss of revenue.  When all the damage is taken into account, this is hardly a victory.

Bad Requirements

The last situation occurs where you only have vague requirements before you start a project.  This situation happens where the executives need a project done quickly and over-estimate the teams familiarity with the subject domain.

These projects start with requirements in the red zone below.  You don't have a prayer of completing this project and it will turn into a Death March with all of its characteristics (see Death March Calculus).

Making core course corrections to bad requirements are like trying to change a tire on a car when you are going down the highway at 100 mph.  It won't happen.


It is human nature to assume that the sooner you start a project that the sooner you will finish.   That assumption is only correct if you have good requirements to point you in the correct direction.  Good requirements are consistent and correct and include at a minimum the core requirements for the primary users.

Starting a project with mediocre or poor requirements is simply a recipe for project failure.  Mediocre or poor requirements are incomplete and inconsistent.

If you have been part of a project failure then you will discover that despite the other factors that went wrong, requirements failure was at the root of it.

Monday, 8 August 2016

Specifically, what is complexity?

People involved in software projects would say that software development is about understanding complexity.

What is complexity?

What can we do about it?

Complexity is easy to define at a high level, but people get vague when pressed for a precise definition.  Let's see if we can:

  • quantify the problem
  • define it
  • suggest how to address it

2 Brooks, Frederick Phillips. “The Mythic Man Month”, 1995. ISBN 0-201-00650-2

Wednesday, 20 July 2016

Hidden Assumption of Agile

Agile cures common problems that we experience in software development, however, there are limitations to Agile.  It may seem like a silver bullet, but there are circumstances under which Agile is not the best choice for development, or at a minimum, not to develop the entire project.

The problems with the waterfall model are well known and understood by most by now.  The waterfall methodology at a high level appeals to executives because it provides a clean way of thinking about a problem.

After all:
  • you can't design something before you have the requirements
  • you can't code something before you have the design
  • you can't test something before you do the coding
These facts are true in Agile as well.  The only difference is that you don't try to do all of any phase as a big bang.  We understand that unless we use some kind of iterative process we can't return to a previous phase and fix anything.  So the Agile process looks as follows:

That is, instead of doing all of one phase as a big bang, we do a bit at a time.  Some methodologies call the code that results from each sprint as a 'brick', and through building bricks, we eventually complete the wall.  So Agile is essentially slicing up the waterfall methodology, but all of the work that needs to be done is still done.
So why on earth would we use a waterfall process for ANYTHING?

Well one of the hidden assumptions of Agile is that you can build all software projects one brick at a time.  But, when was the last time that you saw contractors show up and just start building a skyscraper without a plan?

I mean, isn't it just a matter of building the skyscraper one layer at a time, why bother with planning?

Even though a skyscraper will be built one floor at a time from the bottom up, there still needs to be some architectural planning that happens first.  The materials in the support structure of the building together with the common components that provide common services to each floor (plumbing, electricity, air conditioning) need to be planned out in advance. 

If insufficient planning is put into the architecture, then insufficient plumbing is put in place to drive water to the top of the building, insufficient breakers are purchased to support the electrical needs, insufficiently strong materials might be put in place to hold the weight of the building.

To some degree software is malleable and you can add things after the fact.  But there is often a cost to add things after the fact which is much higher than if the element was designed up front.  For example, you could add add a parking garage to the Empire State building, but it would be cost prohibitive. If they had wanted a garage, it should have been built when the building was constructed.

So any project that will require serious architecture should pause and plan for the structural and connective elements that will be required by the project.  The requirements that are tied to the architecture should be developed first so that the architecture of the project is developed first.  Of course, the architecture can be developed using Agile or the traditional waterfall approach.

Once the architecture is in place, then the rest of development can be done using Agile knowing that all the structures and connective elements are in place.  This is akin to building the skyscraper one floor at a time once the architecture is planned.

The alternative to building out the architecture first is that you find yourself in the position of having architected a building of 10 floors and then realize that you need to have 30 floors. The initial architecture which came together quickly will be snowed under by the work arounds that you need to as you get above 10 floors.  You might get to 15 or 20 floors, but you will never finish the project.

Things that would be expensive to add after you have built a few floors:

  • Needing an extra elevator
  • Needing an extra floor in the parking garage
  • Needing the entire building cabled with fiber optics
Most projects do not require advanced architectures, just like most 2 story houses don't need to have architects.  But if your project is large and has serious architectural considerations then you are best suited to plan for the architecture up front.

Make sure to plan the architecture before doing Agile development