PROJECT
SUCCESS AND FAILURE:
WHAT IS SUCCESS, WHAT IS FAILURE, AND HOW
CAN YOU IMPROVE YOUR ODDS FOR SUCCESS?
Robert Frese
Systems Analysis
Dr. Vicki Sauter
UM-St. Louis
December 16, 2003
We
know why projects fail, we know how to prevent their failure – so why do they
still fail?”(2)This
statement could be applied to the recent Space Shuttle disaster, or the 2003
collapse of a large portion of the U.S. electrical grid. But the author was talking about
Information Technology and Information System project failures, as they existed
in 1994. Information Technology and
Information System failures have been the topic of many articles, conferences,
symposiums, studies, and research initiatives. The literature of the IT and IS
community is rife with articles and commentary about project failures. But just how bad is the
situation? Do a large percent of
projects really fail or do we only hear the bad news? What is failure and what is
success? And lastly, what can you
do to improve your success quotient?
Let’s start by looking at project failure rates and why projects
fail.
There
are many writers who tell us why projects fail. For instance, Field(12) tells us that
“projects fail too often because the project scope was not fully appreciated
and/or user needs not fully understood.”
Hulme(13)
tells us that “MIS projects and associated procurements take place in an
environment characterized by the following: Lack of management continuity and an
incentive system that encourages overly optimistic estimates of the benefits
that can be attained from doing the project.” And Leicht(5) tells us that high
user expectations can actually be the cause of project failure. Hoffman(15) tells that
projects fail because of poor alignment between IT departments and business
users. And in another article Hoffman(9) tells us that
project managers too often act as “process cops and report compilers and loose
sight of what they’re supposed to be doing – to make sure projects are running
effectively”. Hodgson(23)
style='mso-bidi-font-weight:normal' tells us that “projects fail – that’s the
fact of life. Too many fail because
the average project is like an iceberg – 9/10ths of it lay hidden from
view”. All of these writers are
correct. But none of these authors
are reporting systematic research of the mechanisms that cause project success
or failure. And none of them
provide insight into the rate of project failures.
How
Often Do Projects Fail and How Can This be Measured?
In
a 2003 article Julia King(10) reports, “At
companies that aren’t among the top 25% of technology users, three out of 10 IT
projects fail on average.
Translation: for these
companies an amazing 30% of IT projects fail. Now if you are an extremely optimistic
person you might conclude the good news is that 70% of these projects
succeed. But note that King does
not tell us how many of the 70% of the “successful” projects were over budget,
over time, or defective in function upon completion. There are many ways to measure success
and failure, but there is no strict dividing line between the two. Baker(20) concludes, “Like
everything else, the definition of project failure is in a state of flux.” And
O’Brochta(18)
tells us that “the big problem with assessing project success is that it is not
precise.” O’Brochta
continues, “This dynamic can often be the Achilles heal for a project. Without a dependable understanding of
what constitutes success, the project is placed in the untenable position of
being judged against differing criteria, and invariably becomes one more failure
statistic reported by research firms such as Standish, Gartner, Forrester, and
others.”
And
so, Lewis(7) tells
us that “On average, about 70% of all IT-related projects fail to meet their
objectives.” In this case
Lewis includes not only projects that were abandoned (failed), but also those
that were defectively completed due to cost overruns, time overruns, or did not
provide all of the functionality that was originally promised. Amazingly, Lewis does not consider this
70% failure to meet objectives a serious issue, and compares it to the batting
average of a major league player.
Lewis contends major league coaches are happy with a 30% at-bat success
rate for a player, and Lewis thinks the same standard should apply to IT
projects. There seems to be only
one problem with Lewis’ outlook, and that is that the customer, the bill payer,
may not share this sentiment.
There
are other reports of project failure rates. A 1997 seminar paper(3) states that “In
1992 the Unites States General Accounting Office (GAO) reviewed Management
Information Systems (MIS) projects and concluded: Developing and modernizing
government information systems is a difficult and complex process. Again and again, projects have run into
serious trouble, despite hard work by dedicated staff. They are developed late, fail to work as
planned, and cost millions – even hundreds of millions – more than
expected”. In the same article we
are told that research by the Standish Group indicates only16.1 percent of all
MIS projects are completed on time and within budget. Translation: 83.9 percent of projects
fail either to some extent or fail completely. So this leads to several questions. Regardless of measurement semantics, why
do projects fail? Is there one
cause or are there many causes? If
the overall failure rate is going to remain high, then how can you, the reader,
become the exception to this rule of failure and achieve a much higher success
rate for your projects?
Your career may well depend on it.
Let’s
look at the Standish Group’s CHAOS Report(1) and see what it
has to say. For the Standish Group
not only published failure and success rates, but also pointed to indicators for
success and failure. Their original
report was done in 1994 and published as THE CHAOS Report. The Standish Group
studied 365 companies with a total of 8,380 Information System applications
under development. The resultant report divides projects into three distinct
outcomes –which they called Resolutions.
Resolution
Type 1
is a “Project Success” – it completed on time and budget, with all
features and functions as specified.
Only 16.2% of projects fell in this category.
Resolution
Type
2
is “Project Challenged.”
These were completed, but were over cost, over time, and/or lacking all
of the features and functions that were originally specified. 52.7% of all studied projects fell
into this Resolution Type 2 (Challenged) category.
Resolution
Type 3
is termed “Project Impaired/Failed.” These projects were abandoned or
cancelled at some point and thus became total losses. A disturbing 31.1% of all studied
projects fell into this category.
For
the purposes of this paper we will use the above three Standish Group measures
of project outcome: A successful project must be on time, on budget, and deliver
quality (features and functions) as promised. Anything less will be either a failed
project or a challenged project.
The
disturbing conclusion from this Standish report is that only 16.2% of projects
were successful by all measures, and that of the 70% of projects that were not
successful, Over 52 percent were partial failures and 31% were complete
failures. This should certainly
give project managers both food for thought and motivation to action.
So
now that we have information about project success and failure rates, are there
any significant differentiators found between successful and failed
projects? According to the 1994
Standish CHAOS Report, there are.
The
top 5 factors found in successful projects are:
1. User
Involvement
2. Executive Management
Support
3. Clear Statement of
Requirements
4. Proper
Planning
5. Realistic
Expectations
These
are the top 5 of 10 listed in the report.
The report concludes that these were the elements that were most often
pointed to as major contributors to project success. Will these elements alone guarantee
success? Never. But if these are
done well, a project, according to the Standish Group, will have a much higher
probability of success.
The
next category of differentiators from the Standish report deals with projects
that proved to be “Challenged;” that is they were completed buy were over
budget, over time, or did not contain all functions and features originally
required.
The
top 5 indicators found in “Challenged” projects
are:
1. Lack of User
Input
2. Incomplete Requirements &
Specifications
3. Changing Requirements &
Specifications
4. Lack of Executive
Support
5. Technical
Incompetence
And
finally a list of all the top factors found in “Failed”
projects
1. Incomplete
Requirements
2. Lack of user
involvement
3. Lack of
Resources
4. Unrealistic
Expectations
5. Lace of Executive
Support
6. Changing Requirements &
Specifications
7. Lack of
Planning
8. Didn’t Need it Any
Longer
9. Lack of IT
management
10. Technical Illiteracy
Notice
in the above three project outcomes, user involvement is listed as the first or
second item in each.
ANOTHER
PERSPECTIVE TO SUCCESS AND FAILURE:
The
results of a research paper(6), presented at a
1991 PMI symposium, found that “there are areas that should be emphasized by
project managers who are committed to the success of their projects.” The research method used Content
Analysis of showcase articles featured in pmNETwork and Project
Management Journal. The
researchers studied 24 areas of project management and found that 3 of the 24,
if done well, clearly indicated that a project had a high probability of
success. The paper states “it may
be inferred that these three variables (Good Planning, Clear Responsibility and
Accountability, and Schedule Control) in particular have the greatest impact on
the performance of the project.”
Do not, however, conclude that all of the other areas were then not
important to project success.
The paper also tells us “the data suggests that many different variables
are needed to accomplish a successful project. Let’s look at the three key areas that,
if done well, point to a successful project completion.
Good
Planning
The
first indicator, Good Planning, requires excellent forward planning, which
includes detailed planning of the process implementation stages, task
timeliness, fall-back positions, and re-planning. Notice that initial planning is not
enough. Projects often take wrong
turns, or initial solutions prove unfounded. The project manager who does not prepare
to re-plan, or has not considered and planned fall-back positions when initial
plans fail, will often find that the project first stalls, and then fails. We must remember that project management
is not a straight-line process, but an iterative process that requires agile
rethinking as the known environment changes before your eyes.
Clear
Responsibility and Accountability of Team Members
This
requires that all team members have a clear understanding of their roles and
duties in the project. They must
understand how expectations vs. achievements will be measured and graded. It is left to the project manager to
properly implement the communication of these responsibilities, to provide
feedback, and to assure all understand that for which they will be held
accountable.
Schedule
Control
This
requires the continual monitoring and measurement of time, milestones, people,
and equipment schedules. Properly
done schedule control will also give the first hint that initial planning may
not be going according to schedule.
If you pickup on these hints, you have an opportunity to implement a
fallback position and/or re-plan to get back on track.
The
same paper finds two attributes that appeared equally for projects that
succeeded or failed. These two
were: Use of Consultants, and Well
Qualified Personnel. Equal numbers
of successful and failed projects used consultants, and the same was true for
well-qualified personnel. It is
perhaps disappointing that these two attributes did not portend project
success. Obviously there are many
other variables that hold great weight in determining the ultimate outcome of an
IT project.
Lastly
this same study listed four things that foreshadowed a failed project. There were: Lack of Efficient Internal
Communication Links, Lack of Efficient External Communication Links, Lack of
Responsive Decision Making, and Lack of Effective Teamwork. These appeared most frequently in a
negative context in failed projects.
So
at this point we have several lists of things that might indicate project
success and others that might indicate project failure. But there is one thing primarily that
you must recognize in all of these lists.
There are no stock answers, and there is no one list that will guarantee
success. IT and IS projects are
complex by nature, and each is unique.
It is very difficult to plan with complete certainty something that has
never before been done. Every
single factor in all of these lists is important and must be considered for each
project. The most difficult part
may be prioritizing the factors.
Which are most important? Which must be done first? Hopefully the lists will help you answer
these questions. But in each case
you must ultimately make the decisions based upon the unique circumstances of
your immediate project.
COMMUNICATION
B.
Elenbaas(17) tells
us that “projects are about communication, communication, communication.” In 1973 I was speaking with a gentleman
who had recently retired from Monsanto Chemical Company. His career with Monsanto began back in
1933. As we spoke, the subject of
communication entered our conversation and this fine gentleman related the
following story: “In 1933 when I
was brand new to Monsanto, my boss told me that the biggest problem at Monsanto
was (the lack of) communication.
And 40 years later when I retired, the biggest problem at Monsanto was
still the lack of communication.”
This gentleman emphasized the fact that a lack of communication is very
costly to a company. Sure, a
company may still succeed, but without good internal and external communication
I submit that the cost of success will be much higher than necessary. Another consequence is that success
often takes much longer than necessary to achieve. Sometimes success never arrives.
Lack
of good communication can easily turns a corporate strategy, or an IS project,
into a modern day Towel of Babel.
Lavine(8)
tells us that “The warring factions in Africa have a better chance of
communicating with each other than many of the user and technology groups that
‘work together’ in today’s project development environment.” Lavine also relates that some years ago
he was hired as a developer for a large bond-processing bank. He was told on his third day that the
development team was no longer allowed to speak to anyone from the business
community. It seems that relations
between the two groups had become so bad that communication had come to a
complete halt. In fact,
negotiations had begun in an attempt to find an acceptable liaison to work
between the two groups. This
may seem like an extreme example, but this happens in projects. Sometimes it if overt, but all too often
it is on a subtle level. Subtle dysfunction is probably harder to
correct because it is more difficult to pinpoint. You know something is wrong, but it’s
difficult to tell exactly what it is or to pinpoint the root cause of the
problem. This often makes the
problem intractable.
Kirksey
(3) tells us that
one predictor of project success is when communications are kept honest and open
between customer and vendor. His
major indicator of project failure in this area is when an IS project manager
fails to correctly read warning signs that communication is breaking down. The result is a missed opportunity to
correct the situation before it becomes too late.
Wixom(4) argues that User
Participation and Team Skills are two of seven imperative implementation factors
that determine project success or failure, and that these two are essentially
communication skills. “User
Participation occurs when users are assigned project roles and tasks, which lead
to a better communication of their needs, and help ensure that the system is
implemented successfully”.
This is what Wixom(4) has to say about
Team Skills: “People are
important when implementing a system and can directly affect its success or
failure. Team skills include both
technical and interpersonal abilities”.
These interpersonal abilities include, without exception, interpersonal
communication skills. Who do you
know that communicates effectively?
Watch them and determine why their communication is effective. Also watch those who do poorly at
communicating, and make every effort to avoid their bad habits. There is one last point that involves
communication and how it must be used to put user expectations into
perspective. Hayes(11) provides
additional insight by noting,
“executives expect sales efforts and product development efforts to fail,
but not IT projects. He also tells
us that we must convince managers that system development today is a gamble, but
one the may have a big payoff.
Hayes tells us that we must “market” our efforts and manage user
expectations. If the user
understands that there is risk, then “you’ll have a better chance of delivering
what the users expect.” And if you want psychological insigits into people and
projects, review Wastell's(16) insight into
learning dysfunction and its association with project success or
failure.
AVOID
AT ALL COST
Here
are a few things that should never happen to you but probably will at one point
or another. If you ever hear
yourself tell a client “No, you’re wrong, that was never part of the scope! It’s clearly a scope expansion!”(21), then you have
violated one of the cardinal rules about controlling scope expansion. If you
find yourself talking more than listening, then “stop talking”(22) “because
the outcome of having your view prevail” may not ultimately be wise. Two or three heads are often better than
one, so listen to the others – you might learn something. “If a project manager wants to fail,
operate in a vacuum”(24). Operating in a vacuum, intentionally or
not, is a sure way to make sure communication dies on your project. And lastly there is this favorite quote,
“The Lone Ranger had Tonto, Yogi Bear had Boo-Boo, so why do some PMs run
without adequate help”?(25) Think about this next time you don’t
want to share the load, perhaps because no one can do it better than
you.
JIANG’S
LIST
One
concise literature study by Jiang, et al(19) produced a list
of 13 success factors. Jiang’s
conclusion at the end of this study was that “the literature suggests that IS
users and IS professionals are remarkably identical in their importance rankings
of success factors. The similarity
extends to comparisons across experience level and gender. The similarity across these demographic
considerations allows a confidence in the rankings obtained. These rankings can thus be used as
guidance in establishing policies and procedures.” Here is the list that I will call
Jiang’s 13.
1.
Clearly
defined goals
(including the general project philosophy or general mission of the project, as
well as commitment to those goals on the part of the team
members).
2.
Competent
project manager. The importance of initial selection of
skilled (interpersonally, technically, and administratively) project
leader.
3.
Top
Management Support.
Top or divisional management support for the project that has been conveyed to
all concerned parties.
4.
Competent
project team members.
The importance of selecting and, if necessary, triaging project team
members.
5.
Sufficient
resource allocation. These are Resources in the form of
money, personnel, logistics, etc.
6.
Adequate
communication channels. Sufficient information is available on
the project objectives, status, changes, organizational coordination, clients’
needs, etc.
7.
Control
Mechanisms.
(Including planning, schedules, etc.). Programs are in place to deal with
initial plans and schedules.
8.
Feedback
capabilities. All parties concerned with the project
area able to review project status, make suggestions, and corrections through
formal feedback channels or review meetings.
9.
Responsiveness
to client. All potential users of the project are
consulted with and kept up to date on project status. Further, clients receive assistance
after the project has been successfully implemented.
10.
Client
consultation. The project team members share solicited
input from all potential clients of the project. The project team members understand the
needs of those who will use the systems.
11.
Technical
tasks. The technology that is being implemented
works well. Experts, consultants,
or other experienced project managers outside the project team have reviewed and
critiqued the basic approach.
12.
Client
Acceptance. Potential clients have been contacted
about the usefulness of the project.
Adequate advanced preparation has been done to best determine how to sell
the project to the clients.
13.
Trouble-shooting. Project team members spend a part of
each day looking for problems that have surfaced or are about to surface. Project team members are encouraged to
take quick action on problems on their own initiative.
The
Future is Getting Brighter Every Day
What
does the future hold? According to Johnson(14) the success rate
for projects has actually increased since the original Standish CHAOS
report. Johnson attributes this
increased success rate to more project people using the Standish “Recipe for
Success” that was established in 1998.
Johnson tells us that the overall project success rate has increased from
16% in 1994 to 28% in 2000. What
then are the top 5 factors that have caused this significant increase? According to Johnson’s report the top 5
are:
1)
Executive Support: This is now the No. 1 factor in project failure. Lack of executive support can and does
jeopardize projects. Positive
Executive support positively influences project outcome.
2)
User Involvement: Lack of user involvement traditionally has been the No. 1
reason for project failures.
Although it is now No. 2, its importance has not decreased. Johnson feels that project
professionals better handle and solve this major problem.
3)
Experienced Project Manager: Johnson reports that ninety-seven percent of
successful projects have an experienced project manager at the
helm.
4)
Clear Business Objectives: Better control of objectives is attributed to
experienced project managers.
5)
Minimized Scope: Do not allow your scope to grow. Johnson claims that minimized scope has
replaced small milestones.
Conclusion
There
are many things that lead to project success and many that lead to failure. Jiang’s list of 13 is a good list to use
as a starting point for your projects.
But like any of the lists, it is not enough in and of itself. Good project management is a process of
continuous improvement. It is a
process of making mistakes and learning from those mistakes. It is a process of continuous study and
learning. For those who cannot
devote themselves to this never-ending process, there will be few
successes.
REFERENCES
(1) THE CHAOS Report (1994), The Standish Group, http://www.standishgroup.com/sample_research/chaos_1994_1.php
Viewed
Nov 17, 2003
(2) Martin Cobb, (1996). “Unfinished Voyages: a follow-up to the
CHAOS Report”,Standish Group Report,
http://www.standishgroup.com/sample_research/unfinished_voyages_1.php
Viewed
Nov 17, 2003
(3) Kirksey, Kirk A, (1990). “Storm Warning: Danger Signs During
Software Implementation”, Health Management Technology, 11, 6; pg 35
(4) Wixom, Barbara H. (2001). “Am Empirical Investigation of the
Factors Affecting Data Warehouse Success”, MIS Quarterly, Vol. 25 No.1, pp 17 –
41.
(5) Leicht, Michael. (1999).
“Managing User Expectations.” University of Missouri St. Louis
e-publication. http://www.umsl.edu/~sauter/analysis/user_expectations.html,
viewed
11/12/2003
(6) Anil, Iyer and Thomasson, David (1991). “An Empirical
Investigation of the Use of Content Analysis to Define the Variables Most
Prevalent in Project Successes and Failures”, Proceedings of the 1991 PMI Annual
Seminar/Symposium, Sept 27 – October 2, 1991
(7)) Lewis, Bob. (2003).
“The 70-percent failure”, InfoWorld.
http://archive.infoworld.com/articles/op/xml/01/10/29/011029/opsurvival.xml
Viewed
12/12/2003
(8) Levine, Mordy (1994). “Why Technology Uses and Developers don’t
get along”, Wall Street and Technology, Dec 1994, vol. 12, 7: pg.
56
(9) Hoffman, Thomas “Value of Project Management Offices Questioned”,
Computerworld, July 21, 2003. http://www.computerworld.com/printthis/2003.0,4814,82345,00.html,
Viewed 11/7/2003
(10) King, Julia. “Survey shows common IT woes”, Computerworld, June
23, 2003,
http://www.computerworld.com/managementtopics/management/story/0,10801,82404,00.html. Viewed Dec 13,
2003
(11) Hayes, Frank. (1997). “Managing User Expectations”,
Computerworld, Nov 3, 1997, 31,44.
(12) Field, Tom.
(1997). “When bad things happen to good projects”, CIO magazine, Oct 15,
1997, Vol. 11, 2; pg. 54, 6 pgs.
(13) Hulme, Martyn R.
(1997). “Procurement Reform and MIS Project Success”, Journal of Supply
Chain Management, Winter 1997; 33, 1; pg 2.
(14) Johnson, Jim, et al.
(2001). “Collaborating on Project Success.” http://www.softwaremag.com/L.cfm?Doc=archive/2001feb/CollaborativeMgt.html
Viewed
11/17/2003
(15) Hoffman, Thomas. 2003.
“Corporate Execs Try New Ways to Align IT with Business Units.” Computerworld. October 27, 2003
http://www.computerworld.com/printthis/2003/0,4814,86466,00.html
(16) Wastell, David G.
“Learning Dysfunctions in Information Systems Development; Overcoming the
Social Defenses With Transitional Objects”, MIS Quarterly, Dec 1999; 23, 4; pg
581
(17) Elenbass, B.
(2000). “Staging a Project – Are You Setting Your Project Up for
Success?”, Proceedings of the Project Management Institute Annual Seminars &
Symposiums, September 7 – 16, 2000,
Houston, TX.
(18) O’Brochta, Michael. (2002). “Project Success – What Are the
Criteria and Whose Opinion Counts?”, Proceedings of the Project Management
Institute Annual Seminars & Symposiums, October 3 – 10, 2002, San Antonio, TX.
(19) Jiang, James J.
Gary Klein, and Joseph Balloun (1996). “Ranking of System Implementation
Success Factors”, Project Management Journal, December 1996, pp 49 –
53.
(20) Baker, Bud. (1997). Great Expectations: Turning Failure into
Success – and Visa Versa. PM
Network, May 1997, pp. 25 – 28.
(21) PMTalk Newsletter, (2001), http://www.4pm.com/pmtalk03-19-02.pdf viewed
(22) Macomber, Hal (2003), “Reforming Project Management”, http://weblog.halmacomber.com//
Viewed
(23) Hodgson, Ian. (2002). “Keeping Your Head Above Water”, http://www/conspectus.com/2002/november/article19.asp viewed
(24) “The Eight Keys to Project Management Failure”, (2003), http://workstar.net/library/pm1.htm
viewed
(25) “Project Failure Warning Signs”. http://www.bluejeansplace.com/ProjectManagementFailureWarningSigns.html
viewed