How project managers can recover from worst case scenarios

Learn how to recover when a software project goes horribly wrong, due to problems with budgets, clients, stakeholders, schedules or technical failures.

Lawrence Oliva
Lawrence Oliva, PMP

At some point in our careers, we have all been involved in a worst case scenario. Problems that were small became large; large problems became giant problems -- problems like running out of disk space at 2 a.m. during a scheduled production software upgrade. Or trying to locate software defects that suddenly shut down the building network router on Friday afternoon at 4 p.m., just when a thousand people tried to submit their electronic timecards.

Winston Churchill said, "Success is the ability to go from one failure to another with no loss of enthusiasm." Often, the solutions to worst case scenarios are not pretty or elegant. In some cases, solving the problem will require doing things the organization would normally not approve. Sometimes in order to save the project, the project team has to lose professional credibility or team members. However, project recovery is important work that needs to be done.

A primary program management responsibility is to overcome problems and complete the project. What approaches can project managers take to resolve and overcome different types of worst case scenarios? Here are some guidelines to consider.

First, evaluate the seriousness of the problem.

Is this a life and death problem?

Will people's lives be at risk if the problem is not solved? If so, the project management approach is clear: Utilize every technical resource available to solve the problem as quickly as possible. Ignore standard work processes and normal approval timeframes. Don't worry about the budget; just get the problem resolved safely. After the crisis has passed, determine if project recovery is technically or financially possible. Depending on the situation, the client or management may decide to permanently stop the project. Going forward, project recovery efforts may include additional public relations outreach to explain what happened, as well as new tasks that expand quality assurance and training activities.

More project management tips
Why a PM is the CEO's best friend

How to handle IT project management in a recession 

Other types of worst case scenarios may match your project:

Worst case technical scenario

Has technology suddenly failed to work as expected, or work at all? If so, the recovery approach involves rolling back software and hardware versions to the most recent version that worked, at least most of the time. Using that as a base to test against, add simulated or actual conditions (capacity, performance, data interdependencies) that caused the newer version of software to fail. Determine the breakage points, rebuild, retest and reinstall. Hardware problems are usually faster to resolve using circuit board and storage device replacements.

Worst case budget scenario

What happened to the budget? Did the project over-consume its budget, or did the bank remove a credit line? Knowing how much money is available helps prioritize work activities. Focus on using the remaining funds to finish the core parts of the project. Uncompleted tasks should be ranked for priority using a return on investment model, or based on the overall impact to society (if it's for a government program). Completing at least some of the deliverables is preferable to none, unless the budget loss is completely unreplaceable.

Worst case client scenario

Has the client completely changed the scope or technology of the project? If so, will the changes prohibit completion or delivery of the project? If so, recovery may be limited to salvaging investments in hardware and software for reuse on another project. Save database schemas and other design data for reference use on future projects. Determine if any of the current development work can be used for other projects, to reduce their cost or schedule time. If the development organization is being sold, the new owner may be interested in the project's technical approach or its completed software modules.

Worst case schedule scenario

Was the schedule unrealistic when originally estimated, or did unplanned events or issues affect progress? Once a schedule becomes clearly impossible to achieve, recovery involves performing a worst case schedule replan, using a "bottom up" methodology. That will reveal the accuracy (or errors) in the original schedule. Having an accurate schedule will enable more precise costing of resources required to finish the project, which may be useful in resolving budget issues.

Worst case stakeholder scenario

Did a large stakeholder pull their political support from the project due to an internal scandal? Did that event cause public or private funding contributions to collapse? Emerging from the cloud of suspicion is a lengthy process that requires a project team to re-evaluate next steps, especially if financial support has dropped substantially. Recovery requires visible separation of the project from the scandal situation, clear identification of its future benefits and tight linkage to at least one of the organization's publicly supported key objectives.

Worst case services supplier scenario

Did a key supplier fail with their assignment? Did the work exceed their skillsets, or did a process fail? Project recovery requires rapidly replacing the people that failed with proven and tested resources, execution of the unperformed work, and implementation of new processes. The project manager must perform a failure and recovery analysis to ensure progress can continue forward without interruption, possibly allowing the project to regain lost schedule time.

Recovering from unexpected worst case scenarios requires a technical aptitude, strong process knowledge and prior crisis experience in order to quickly understand the scope of the problem. That type of expertise often resides in senior engineers and project managers -- professional leaders who have been trained in the heat of battle. They have supported (or led) mission-critical projects that had to recover from worst case scenarios due to important user needs or mission objectives. Failure was not an option, leading to innovative approaches that caused people and equipment to work well past their expected operational limits.

Some notable worst case scenarios with successful outcomes include the return of Apollo 13 from space orbit, rebuilding the Pentagon in one year after the 9/11 attacks, and containing the global SARS (coronavirus) epidemic in just a few weeks.

Project complexity and interdependencies continue to increase, triggering the unexpected convergence of unplanned situations. As technical and project managers encounter, recover and learn from worst case scenarios during their careers, doing the impossible with enthusiasm becomes a well-developed skill. Winston Churchill would be proud.

About the author: Lawrence Oliva is a senior consultant and program manager with CH2M HILL, a global engineering and program management company. Based in the Washington, D.C., area, he currently leads midsized to large IT programs for federal government and commercial clients with unique technical challenges that require risk management, earned value analysis, cost/schedule forensics, biometric security and green IT implementation experience.

Dig Deeper on Topics Archive