Abstract: Software development is a highly complex and unpredictable activity associated with high risks. With more and more organizations investing substantial resources in software development, risk management becomes crucial. The present paper tries to ask these three questions: what are the most frequently used risk management approaches? What is the process of risk management? How to apply risk management in practice? An analysis of previous literatures identifies four types of risk management approach: risk-list, risk-action list, risk-strategy model, and risk-strategy analysis. A continuous process is introduced based on the dynamic system theory. A framework for applying risk management approach in practice is also proposed from action research perspective. Finally, the relevant practical implications for software development project are discussed.

1.Introduction

McDonalds "Innovate" project: conceived in January 2001, Innovate was the most expensive (planned to spend one billion dollars over five years) and intensive IT project in company history. Executives in company headquarters expected this enterprise software will enable them to see how soda dispensers and frying machines in every store were performing at any moment. However, two years later, with $170 million invested in this project, the fast food giant gave up this project (Source: http://www.baselinemag.com/paper2/0,3959,1173624,00.asp; Nelson 2007)

The above story of McDonald tells us that software development is a complicated and unpredictable activity (Han and Huang 2006). Unfortunately,  this example is not unique. The CHAOS Report in 2004 indicates that 53% of software projects failed to deliver on schedule, within budget, and with the required functions (Standish Group International, Inc. 2004, Han and Huang 2006). Based on the survey data  from IEEE Software’s 2006 , there is a high rate of software project cancellation  (Emam and Koru 2008). The figure 1 summarizes the existing evidence.

Figure 1 (Adapted from  Emam and Koru 2008, P85)

Project failures are associated with the risks embedded in software project environment  (Kwak and Stoddard 2004). Previous researches have investigated the relationship between risk and project failures. For instance, McFarlan (1981) has concluded that failure to assess individual project risk and adapt management methods accordingly is  a critical source of software project failure. Different types of risks will affect budget, user satisfactions and system performance  (Jiang and Klein 1999).

Nowadays, as more and more organizations invest substantial resources and efforts in software development, managing risks associated with software project becomes crucial (Kumar 2002). However, several questions need to be explicitly understood before organizations implement software risk management.

1.What is risk management in software development?

2.What are the approaches frequently used to manage risks?

3.What is the process for implementing risk management?

4.How to apply risk management approach to software project?

This paper dedicates to answer these questions. The contents are structured as:  I give brief iintroduction in section 1. In section 2, I present the basis concepts of software risk management. In section 3, I describe four types of risk management approach.  In section 4, I discuss a continous approach for software risk management in details. I propose a practical model for applying risk management approach to software development project in section 5. Finally,  I draw some conclusions and trace future researches in software risk management.

2.Risk management in software development

The term risk is associated with many human activities such as exploration, nuclear reactor construction, company acquisition, security of information systems and software development (Barki, Rivard and Talbot 1993). In this paper, I focus on risk management in software development. Ideas of risk management have been applied to software development over the past decades (Iversen et al., 2004).

2.1. The definition of risk impact

A software risk indicates a particular aspect of a development task, process, or environment, which, if ignored, will increase the probability of project failure (Lyytinen, Mathiassen and Ropponen 1998).  In essence, there are two ways to define risk, one is in quantitative and the other one is in qualitative. The quantitative definition of risk comes from Boehm (1989):

where RE refers to risk exposure, Prob(UO) is the probability of an unsatisfactory outcome, and Loss(UO) is the loss to the parties affected if the outcome is unsatisfactory. The software development projects involve several classes of participants (customer, developer, user and maintainer). Each of them has specific satisfaction criteria. Thus, the definition of "unsatisfactory outcome" is multidimensional (Boehm 1991). I summarize the definition of "unsatisfactory outcome" from different perspectives in table 1.

Table 1: The definition of "unsatisfactory outcome" based on stakeholders

(Adapted from Boehm 1991)

In qualitative terms, Barki et al. (1993) defines the degree of risk as the uncertainty surrounding a project and magnitude of potential loss associated with failure of the project.

2.2.Software risk management

As shown in figure 2 and 3, there are more and more software development projects are adopting risk management approaches. Software risk management is one promising approach to deal with system failures.  In addition, software project managers view risk management as a key to success (Barki et al., 1993). Using risk management in software projects has several advantages, including helping the practitioners focus on problematic aspects, emphasizing potential causes of failure, linking potential threats to possible actions, and facilitating a shared perception of the project among its participants (Iversen et al. 2004, Lyytinen et al. 1998). Risk management approaches have been developed to identify, analyze, and tackle project portfolio risks, system development risks, requirement risks, and implementation risks (Iversen et al., 2004).

Figure 2: Trend of software development and risk management based on the average worldwide traffic in 2008.

(Source:  Google Trends)

Figure 3: Software development and risk management data in 2008 by regions and cities. The data is ranked by software development

(Source:  Google Trends)

3.Four types of risk management approac

Risk management approaches are typically represented using the concepts of risk items, risk resolution techniques, and heuristics (Lyytinen, Mathiassen and Ropponen 1998).  Lyytinen et al. suggest that risk items are used to detect risky incidents, resolution actions help identify possibly relevant actions, and different kinds of heuristics help link risk items with possible resolutions. Based on Lyytinen et al. (1998) research, Iversen et al. (2004) identify four major types of approaches for software risk management which include risk list, risk-action list, risk-strategy model, risk-strategy analysis.

3.1.Risk list

Risk list approach provides a list of prioritized risk items(Iversen et al., 2004). This list of risk items helps a project manager focus on possible source of risk. However, it does not contain related information about appropriate resolution actions(Iversen et al., 2004). Usually, this risk list is easy to use in assessing risks. Likewise, it is also easy to build, drawing upon literatures on risks or experiences within a particular context. Organizations can modify risk lists according to specific organizational context. Typically, risk items can be classified into two categories: system items and specific items. System items are those items occur in most software projects. Specific risk items are related to project characteristics and organizational context. Keil et al. (1998) provide a list of risk factors ranked by relative importance (See Figure 4).  Keil et al. (1998) assembled three panels of experienced software project managers from different parts of the world-Finland, HongKong and United States. They asked the managers to identify and rate risk factors. One of the findings is very interesting that all three panels coherently viewed 11 factors as important. This finding suggests that there is a universal set of risks with global relevance.

Figure 4: A common set of 11 risk factors ranked by scale 1-10 (Adapted from Keil et al. 2004, P78).

While this approach offers strong supports to help project managers assess risks, it does not help identify relevant resolution actions and does not provide a strategic oversight of the risk profile and relevant strategies for action(Iversen et al., 2004).

3.2.Risk-action list

Risk-action list is an approach which offers a list of prioritized risk items with related resolution actions(Iversen et al., 2004). This list contains prioritized risk items, with one or more related resolution actions for each risk item. Similar to risk lists, risk action lists are also easy to use in assessing risks; they are easy to build; and they are easy to modify when needed. However, to build the resolution actions, they require additional knowledge of the potential effects of different types of actions(Iversen et al., 2004).

This approach offers the same supports to appreciate risks as risk list does. In addition, risk action list uses a heuristic to identify possible actions that will help resolve specific risks. An example comes from Boehm's (1991) paper. He identifies two primary steps for software risk management: risk assessment and risk control. For the first primary step, risk assessment, he identifies three subsidiary steps: risk identification, risk analysis, and risk prioritization. For the second primary step, risk control, also involves three subsidiary steps: risk management planning, risk resolution, and risk monitoring.

Risk identification: produce lists of the project-specific risk items related to a project’s success

Risk analysis: assess the loss probability and loss magnitude for each identified risk items and it also analyzes compound risks in risk-item interactions.

Risk prioritization: produce a ranked ordering of risk items identified and analyzed.

 Risk-management planning: help prepare you to address each risk item (e.g. via information buying, risk avoidance, risk transfer, or risk reduction), including   coordinating individual risk-item plans with each other and with the overall project plan.

 Risk resolution: generate a situation in which the risk items are eliminated or resolved.

 Risk monitoring: track the project’s progress in order to resolve its risk items and take corrective action where appropriate.

Based on these six steps, Boehm developed a top-ten list of software development risks with three to seven actions per risk (See Figure 5).

Figure 5: Top 10 software risk items with corresponding actions (Adapted from Boehm 1991, P35)

Risk-action list is easy to use and also easy to build. It is also helpful to resolve specific risks. However, this approach focuses on isolated pairs of risk items and resolution actions. It does not emphasize on a strategy for addressing the risk profile as a whole(Iversen et al., 2004).

3.3.Risk-strategy model

Risk-strategy model is a contingency model that relates aggregate risk items to aggregate resolution actions as said by Iversen et al. (2004) that risk-strategy model combines comprehensive lists of risk items with resolution actions. First, this approach abstracts categories of risks to arrive at a risk profile. And then abstracts categories of actions to arrive at an overall risk strategy. This approach uses a simple scale such as high or low to assess the risk profile along the risk categories. Through this way, it is possible to classify the project into one of a few possible situations. Then, for its situation, the model offers a risk strategy with several detailed resolution actions.

The best known of this approach is McFarlan’s (1981) portfolio model.  McFarlan identifies three important dimensions which influence the risk inherent a project, including project size, experience with the technology, and project structure.

Project size: the larger project has the greater risk.

Experience with the technology: if the project team and the IS organization have less experience about the project, the project risk will increase. In addition, large systems development group have a very high risk than a smaller, less technically advanced group since the latter group can reduce risk through purchase of outside skills for an undertaking involving technology.

Project structure:  McFarlan defines the projects which have stable and fixed outputs as highly structured projects. For those projects whose outputs are more subject to the manager’s judgement and vulnerable to change as lowly structured.

Based on these three dimensions, he creates the risk profile of project portfolio.

Figure 6: risk profile (Adapted from McFarlan 1981, p146)

Based on the risk profile, McFarlan classifies the project into eight types (see table 2).

Table 2:  Risk profile (McFarlan 1981)

Furthermore, McFarlan considers that there may be a general purpose set of tools, the contribution each device can make to planning and controlling the project varies widely according to the project’s characteristics. He proposes four general methods for managing projects:

External integration tools include organizational and other communication devices that link the project team’s work to the users at both the managerial and the lower levels.

Internal integration devices ensure that the team operates as an integrated unit.

Formal Planning tools help structure the sequence of tasks beforehand and estimate the time, money, and technical resources the team will need to execute them.

Formal control mechanisms help managers assess progress and identify potential discrepancies so that corrective action can be taken.

The last step for this approach, McFarlan (1981) combined the risk profile with the different strategy as the Figure 7.

Figure 7: risk-strategy model (Adapted from McFarlan 1981, P149)

This example from McFarlan (1981) tells us that a different resource distribution between project management, system development, and quality assurance, depends on a project’s risk profile. This model is easy to use because of the simplifying contingency model. Likewise, this contingency table can help managers appreciate risks, identify relevant actions, and build an overall understanding of the risk profile directly related to a strategy.  However, this approach still has some shortcomings.  The risk-strategy model is difficult to build and modify. Only when you completely understand the factors which influence risk profile, can you abstract the risk profile and build the model.

3.4.Risk-strategy analysis

Risk-strategy analysis is a stepwise process which links a detailed understanding of risks to an overall risk management strategy (Iversen et al. 2004). This approach is very similar to risk-strategy model. It offers detailed and aggregate risk items and resolution actions, but it applies different heuristics. The difference between this approach and the risk-strategy model is that there is no model linking aggregate risk items to aggregate resolution actions (Iversen et al. 2004). Rather, this approach adopts a stepwise analysis process. In the process, the involved actors such as customers, developers, managers, link risks to actions to develop an overall risk strategy. In contrast with risk-strategy model, the relationship between the aggregate risk items and aggregate resolution actions is looser.

Davis (1982) offers an example of risk-strategy analysis approach. He provides a stepwise approach for information requirements determination. The overall level of risk was evaluated and associated with four different strategies to deal with requirements uncertainty (see Figure 8).

(Adapted from Davis 1982, P20)

Compared with the former two approaches, this approach is difficult to build as the risk-strategy model. However, this approach is easier to modify than risk-strategy model because of the loosely coupled relationship between risk items and resolution actions (Iversen et al. 2004).

3.5.Strength and Weakness of these four risk management approaches

Based on above discussion of these four types of approach, the advantages and disadvantages of each approach are summarized as below (see Table 3).

Table 3: Comparison of four types of risk management approach based on Iverson et al. (2004)

Because each risk management approach has its own strengths and weakness, we should be cautious to choose our risk management approach depending on the specific project context and the expected outcomes of our projects.

4.A continuous process of risk management

According to Von Bertalanffy’s (1968) general system theory, each system has input, output, process, feedback, control, environment, and goal. Based on this definition of system, we can say that software development is a system. In addition, there are so many factors affect software development project. The process of software development is dynamic and involves many evolutionary and revolutionary changes(Sabherwal, Hirschheim and Goles 2001).

4.1.Basic concept of continuous process of risk management

The paradigm of risk management proposed by Software Engineering Institute (SEI) is very similar to the six steps proposed by Boehm (1991).  Figure 9 shows the paradigm from SEI. This paradigm illustrates a set of functions that are identified as continuous activities throughout the life cycle of a project.

Figure 9: Adapted from SEI (http://www.sei.cmu.edu/risk/paradigm.html)

Dorofee, et al. ( 1996) defined continuous risk management as a software engineering practice with process, methods, and tools for managing risks in a project. It provides an environment for decision makers to assess continuously what can go wrong, to determine which risks are most important, and to implement strategies to deal with these risks.

4.2.Benefits and costs of continuous risk management

Implementing continuous risk management will give organizations benefits. Simultaneously, some costs will occur. The benefits of continuous risk management include preventing problems before they occur, improving product quality, enabling better use of resources, and promoting teamwork. The costs that will occur during continuous risk management include infrastructure costs, risk management cost, mitigation costs(Dorofee, et al. 1996).

Cost-benefit value is difficult to determine when some costs and benefits cannot be quantified. Thus, the project managers need to balance the costs against the expected benefits and the cost of not doing risk management(Charette 1989).

5.A practical model for software risk management

Iversen et al. (2004) used action research to develop risk management approach for software performance improvement. The following model is modified from Iversen et  al. (2004 ) model and Dorofee et al.( 1996) model.

Figure 10: A practical model for software risk management

In section 3, I compare different software risk management approaches proposed by prior researchers. Through this section, we can have some ideas about methods that can be used to manage software risks. In section 4, I introduce a continuous software risk management process supported by the SEI.  However, we may still wonder how to apply these risk management approaches to software development projects.

In this section, based on Iversen et  al. (2004) and Dorofee et al. (1996), I develop a practical model to apply software risk management approach. This model adopts the action research processes used by Iversen et al. (2004).  Action research connects theory and practice in a cyclic process(Baskerville and Myers 2004). Based on the action research cycle proposed by checkland (1991), I combine the four risk management approaches(theory) with software development project(practice) and create a framework for implementing software risk management.

Iversen et al. (2004) identify three major stages for developing risk approaches for software performance improvement, including initial stage, iterating stage,  and closing stage.

5.1.Initialing stage

In the initial stage, stakeholders assess the problem situation based on the area of concern such as users with negative attitudes, system requirements not adequately identified, project involved the use of new technology, lack of an effective project management methodology, team members lack specialized skills required by the project, employees turnover, organization undergoing restructuring during the project. Goals of risk management is another factor should be considered when assessing the problem situation. Goals of risk management usually refer to the success of software development  (Boehm 1991).

After evaluating the problem situation, project manager and other stakeholders need to select a risk approach based on exsiting risk documents. If organization use risk-action list more often and have extensive documents, they can consider to utilize the previous method. This doesn’t mean that they have to follow the previous risk management approach. They can explore new approach based on the context and the project characteristics.

5.2.Iterating stage

In this stage, it is a continuous process including a repeating set of activities. The first iteration is to develop a risk framework. For instance, you select risk list approach. You need to identify the risk items in it, and also think about how to avoid, mitigate or transfer this risk items. The second iteration is to design the risk process.  This iteration can include reformulating risk items and introducing a first step in which the software development team should interpret the risk model in their particular context. The third iteration is to apply the risk approach based on the first two iterations and the lessons learned from first two iterations. The iteration step in this stage is to evaluate experience and then the risk approach was used again without any changes(Iversen et al., 2004). The core of iterating stage is communication between stakeholders. Through the communication, software development team and project managers can get feedbacks from internal and external, in turn can improve and refine risk management approach.

5.3.Closing stage

Two situations can exit software risk management cycle. One is that software development project is finished. The other one is that software development project is stopped in the middle way. Both two situations can provide valuable experience to software risk management since organizational learning is a dual loop.  In this stage, we need to update the risk management documents either from successful experience or failed experience which can help organization better manage software risk management in the future.

6.Conclusion

The software field has had its share of project disasters. CHAOS Report in 2004 indicated that 53% of software projects failed to deliver on schedule, within budget, and with the required functions (Standish Group International, Inc. 2004, Han and Huang 2006). Most previous studies have shown that some of these disasters would have been avoided or strongly reduced if there had been an explicit early concern with risk management(Boehm 1991).

In this paper, four types of risk management approaches proposed by previous researchers are studied in details. I find that each approach has its own advantages and disadvantages.Risk list approach is suitable for small projects or projects with high structure outcomes or in the beginning stage of a project. Risk-action approach works better for projects with less people involved. Risk-strategy model is very similiar to risk-strategy analysis. The difference between those two models is that risk-strategy model connects risk categories with resolution actions based on organizational strategy, while risk-strategy analysis doesn't. Based on this study, I recommend risk-strategy analysis approach for a complex project among those four approaches. However, they can be combined or used in different stages of software development since they are interlinked.

In addition, software risk management is a continous process rather than sequential acitivity (Dorofee, et al. 1996). By taking this continous process into consideration, I develop a model for applying software risk management approach in practice using action research method. The main contribution of this model is that it gives a brief picture of the risk management process such as how to start, how to proceed, and how to exit. It also emphsizes that software risk management is a continous process and the communication between different stakeholders. In this model, project managers can choose different types of risk management approach during different stages. Moreover, joining risk management with different stakeholders, such as customer, suppliers and subcontractors is also very critical.The importance of participant involvement in software risk management have been emphasized by many scholars (Alter and Ginzberg 1978, Li, Slyngstad and Morisio 2008). Top management support is also very important for the success of software risk management. Some researchers say that we only need to do risk management in system analysis and design stages in system development life cycle. However, because risk mangement is a continous process, each stage of development needs to consider risk management.

This model is only a theoretical model. Practical verification is needed beyond this research. To verify this model, we can need to take action research method.

References

Alter, Steven, and Michael Ginzberg. "Managing Uncertainty in MIS Implementation." Sloan Management Review, 1978: 23.

Barki, Henri, Suzanne Rivard, and Jean Talbot. "Toward an Assessment of Software Development Risk." Journal of Management Information Systems, 1993: 203-225.

Baskerville, R., and M Myers. "Special Issue on Action Research in IS—Forward." MIS Quarterly, 2004: 329-335.

Bertalanffy, Von. General Systems Theory: Foundations. New York: George Braziller, 1968.

Boehm, Barry W. "Software Risk Management: Principles and Practices." IEEE Software, 1991: 32-41.

Charette, Robert N. Software Engineering Risk Analysis and Management. McGraw-Hill Companies , 1989.

Checkland, P. "From Framework Through Exoerience to LearningL The Essential Nature of Action Research." Information Systems Research, 1991: 387-403.

Davis, G. B. "Strategies for information requirements determination." IBM SYST J, 1982: 4-30.

Dorofee, Audrey J., Julie A. Walker, Christopher J. Alberts, Ronald P. Hignera, Richard L. Murphy, and Ray C. Williams. Continuous Risk Management Guidebook. Carnegie Mellon University Software Engineering Institute, 1996.

Emam, Khaled El, and A. Gunes Koru. "A Replicated Survey of IT Software Project Failure." IEEE Software, 2008: 84-90.

Google. Google Trends. 2008. http://www.google.com/trends (accessed November 18, 2008).

Han, Wen-Ming, and Sun-Jen Huang. "An empirical analysis of risk components and performance on software projects." The Journal of Systems and Software, 2006: 42-50.

Iversen, Jakob H., Lars Mathiassen, and Peter Axel Nielsen. "Managing Risk in Software Process Improvement: An Action Research Approach." MIS Quarterly, 2004: 395-433.

Jiang, J.J., and G. Klein. "Risks to different aspects of system success." Information and Management, 1999: 263-272.

Keil, Mark, E. Paul Cule, Kalle Lyytinen, and Roy C. Schimidt. "A framework for identifying software project risks." Communications of the ACM, 1998: 76-83.

Kumar, R. "Managing risks in IT projects: an options perspective." Information and Management, 2002: 63-74.

Kwak, Y.H., and J. Stoddard. "Project risk management: lessons learned from software development enviornment." technovation, 2004: 915-920.

Li, Jingyue, Odd Petter N. Slyngstad, and Maurizio Morisio. "A state-of-the-Practice Survey of Risk Management in Development with Off-the-Shelf Software Componenets." IEEE Transactions on Software Engineering, 2008: 271.

Lyytinen, K., L. Mathiassen, and J. Ropponen. "Attention Shaping and Software Risk-A Categorical analysis of Four Classical Risk Management Approaches." Information System Research, 1998: 233-255.

McFarlan, , F.W. "Porfolio approach to information systems." Harvard Business Review, 1981: 142-150.

Nelson, Ryan R. "IT Project Management: Infamous Failures, Classic Mistakes, And Best Practices." MIS Quarterly Executive, 2007: 67-78.

Sabherwal, Rajiv, R. Hirschheim, and T. Goles. "The Dynamics of Alignment: A Punctuated Equilibrium Model." Organization Science, 2001: 179-197.

Software Engineering Institute. http://www.sei.cmu.edu/ (accessed November 21, 2008).

Standish Group International, Inc. "2004 Quarter Research Report." 2004.