How developers could have avoided HealthCare.gov technical problems

Software project manager Kay Diller identifies the Healthcare.gov technical problems and advises developers and managers on how to avoid them.

The launch of the HealthCare.gov Web application is now an infamous example of Web development and project management...

malfunction. The fault does not lay in the health insurance being offered on the site, but in the technical decisions made during the software development cycles. As a software project manager, when I first read about this release, I thought of all the different decisions the developers, project managers and their executive administrators could have made to avoid the Healthcare.gov technical problems they experienced.

This article identifies the key problems experienced by the project management and development teams, what is being done to correct the problems, and offers a prediction of what will happen and what it will mean to the people who use the site. I have also added relevant advice for project managers throughout the article.

Where HealthCare.gov went wrong

Two high-level goals of President Obama's HealthCare.gov website are to simplify the way people can comparison-shop for health insurance and to allow them to purchase their preferred health insurance coverage online. For example, in June 2013, the HealthCare.gov project decided to deploy an upgrade to the site to promote the upcoming October 1 launch. The goal of the June site was to prepare people for what they could expect in October when they began ordering their health insurance on the site. The June site was designed, developed and launched in a 90-day time frame.

This short turnaround, pretty much unheard of for a government IT project, was successful because experts from the private sector were brought in to help. This is particularly true with the homepage, developed by Development Seed, a small firm that laid much of the groundwork for the homepage that millions of Americans still use today. The site was built in public and iteratively, using project managers and developers who leveraged Agile development and cutting-edge, open source technologies. The June version of the site was lean and proved to be successful due to the strong collaboration across the government and private sectors.

Advice

Always bring in experts during the design and development phases if you know your project team lacks the required expertise.  These experts will know how to leverage and apply cutting-edge technology to the most complex of projects.  The government did just this for the June release and it worked.  Makes one wonder what happened with the October 1 release.

The October website, which had been under construction for a few years when the June website went live, had been using the Waterfall methodology. Although I cannot find the reasoning why the government chose to stay with Waterfall after the successful June release, I am assuming changing to an Agile methodology three months before the big launch would have proven too great of a challenge with so much code already written.

The major challenges the project team faced with the October release included fixing hundreds of bugs weeks before the release, overworked testers, and a site that kept crashing when performing simple tasks, such as setting up an account. There were also concerns about the accuracy of the data.

As we all know, fixing hundreds of bugs right before a release can have a crippling effect on software and is guaranteed to create additional bugs that will not be caught in the final testing phases. That is, unless you have seasoned testers who know how to expand upon the documented test cases during the final integration testing. Unfortunately, the testers for the October release involved 200-300 government and insurance employees who tested only a few days before launch.  Load testing also appears to have been performed by these same testers over various time periods while working at their desks. Add this to a very complex technical specification and architectural design, and you have a site that was obviously not going to be ready for prime time.

Advice

As project managers, we owe it to our developers to document the problems they are having. In addition, we must document that our executives have received timely notice of the problems. Without this documentation, our developers can be accused of not having the required expertise to work on the project and of sloppy work.  I have worked with a lot of developers over my time and I have yet to meet one that is not skilled enough to write solid code given a reasonable time frame. However, they need time to fix bugs and cannot be held responsible for problems that occur with scope changes that come in at the last minute. Be a great project manager and protect your developers with strong documentation.

HealthCare.gov developers complained to each other about the problems they were experiencing, but it is still unclear if their concerns made their way up to the government's administrators who were responsible for the overall program. CBS News reported that testers were never able to successfully browse the site, and one unidentified source stated it was "unequivocally clear from testing … [that it] wasn't ready." However, government officials, such as Marilyn Tavenner, administrator for Centers for Medicare and Medicaid Services (CMS), have gone on record saying they were unaware of any problems prior to launch.

What is the government doing today to help this crippled website?  Since its first viewing in October, hundreds of the bugs have been fixed and the site has shown marked improvement with stability through additional hardware.  Resources from the private sector, such as Red Hat and Oracle, are now involved to assist with reliability, stability and scalability, sharing their Agile expertise. In addition, President Obama met with other key Silicon Valley executives in December to rally their support and expertise.

An area of improvement we can hope to see in the near future is having the development process become more agile for leaner and quicker release cycles. There is also talk about creating a new federal IT project management office (PMO) unit dedicated to larger technical projects. The new IT PMO would effectively remove the ability for each government organization to act as its own general contractor on projects such as CMS did with HealthCare.gov.  President Obama's administration is also keenly aware of the difficulty to hire skilled technical staff quickly, and is looking into how it can reduce the amount of time it currently takes for the hiring process.

Advice

If your organization is decentralized when it comes to IT projects, think about centralizing your projects under one PMO umbrella. PMOs allow standardization across projects and leverage expertise and resources.  However, be sure to also look at how a PMO would be able to lower overall costs, as this is not always the case, per a survey conducted by PMI.  When it comes to hiring practices for a PMO team, be sure the candidates have the ability to work across the board on different platforms and teams. This is no time for a project manager who will only work on one project at a time.

With these recent bug and hardware enhancements, along with the talk of leveraging Agile methodology, the new PMO unit, and improving hiring practices for skilled IT personnel, the future looks positive for HealthCare.gov. 

As seasoned project managers, we know software is often released even when we are aware of critical bugs impacting the user experience. After a disastrous release, we quickly evaluate the impact, prioritize the change requests and bug fixes, and move forward at a frantic speed until the project is stable, supportable and feature-rich. Fortunately, memories prove short when it comes to releasing software before its time.  For example, Medicare Part D had similar rollout problems when it was initially released in 2006.  Users complained that the site was too slow, their data wasn't accurate, and there were possible security breaches.  Sound familiar?  And yet, today, eight years later, the Medicare Part D website provides a popular prescription drug program among seniors. 

With the attention the HealthCare.gov technical problems are getting from the press, as well as the political implications, highly skilled resources are being applied to the site. There are now top executives from Silicon Valley on the case. They are experts when it comes to releasing and maintaining websites that have millions of hits each day. How long will it take for HealthCare.gov to become feature-rich, stable and popular like Medicare Part D?  With the progress we've seen to date, certainly not eight years, but we will need to give the project managers and developers at least a year to 18 months to work through the hundreds of remaining bugs, to enhance the security, and to ensure the data is accurate.  Not an easy task, but with the teams currently under consideration, we will one day forget about the initial release and will enjoy comparing insurance policies and purchasing healthcare from HealthCare.gov.

Dig Deeper on Software Project Management Process

PRO+

Content

Find more PRO+ content and other member only offers, here.

Join the conversation

26 comments

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

Do you think the Healthcare.gov technical issues will soon be forgotten?
Cancel
there are too many opponents of th ACA... they will use this as fodder for why the concept won't work.. even though it is two different things.
Cancel
People have a long memory of failure. It will take a couple years after the system is stabilized before it is trusted. In addition, the product, in many cases, does not contain a wide variety of choices, further leading to distrust.
Cancel
The points about lack of experience and testing are valid ones. As I understand the June Website was a fancy FAQ. The October release was a full blown applications that was intended to capture, process and distribute large volumes of data. There's a considerable difference between the functionality provided by the two sites. If you wish to compare them then you should be comparing apples to apples.

Now I could be wrong but some Agile techniques were used for the October

With respect to experts the government already has lots of in house resources who are experts at dealing with large applications, e.g. the IRS
Cancel
It won't be forgotten because it was the centerpiece of a very unpopular law.
Cancel
The Healthcare.gov debacle has now become a major event in IT history. It will probably be featured as a milestone event in history books 50 years from now.
Cancel
The tools selected do not facilitate automated testing. This capability is generally an industry-wide weakness. Systems must be designed to be tested, as well as run in a production mode. Automated testing architecture allows the development of testing scenarios for all of the system and user requirements in a REPEATABLE framework with expected documented results. Repeating the same body of tests with predictable results helps any project continue their focus on the expected solution.
Cancel
AND, by the way, the law itself is fatally flawed.
Cancel
The problem is with the healthcare law, mainly state exchanges. The website is only the face that the public sees. Pro or Con ACA, American's health should be the focus.
Cancel
Many of the issues and defects have not been discovered. This should have worked. For many it still doesn't work
Cancel
Too much publicity, too much political gain to ignore
Cancel
because the NRA (National Republican A$$Oles) will not let go... they know best, Obama care is useless and what is better is the NRA keeps saying no and don't have any other alternative
Cancel
Responding to BobMuma's comment on Agile being
used in the October release - I, too, have read many articles and claims
that Agile was used in the front-end and the hub and that they had less
problems than the overall website that used Waterfall - but then I've
also read that the true underlying methodology was really still
Waterfall and the team just used Agile terms.  For example, if they used
Agile, their testing would have caught more much earlier as they did
their short sprints.  Given this, what do you think?  Have you found
articles that can prove the government actually did sprints for the
front-end and hub? I'd love to hear back from you on this.  Thanks!
Cancel
HEALTHCARE.GOV TECHNICAL ISSUES WILL SOON BE FORGOTEN? -N-E-V-E-R- !!! OBAMA NEEDS HEALTHCARE AND MORE. GOVERN SOLUTION IS A POLITICAL ACTION. THE #1 PATIOTIC ACTION ALWAYS IS: PATRIOTIC UNION. OBAMA IS THE PRESIDENT. REPUBLICAN NEEDS DO A PATRIOTIC SUPPORT TO U.S.A. PRESIDENTIAL WORKS. REPUBLICANS OPPOSITIONS IS ALWAYS A DEEP AND UNPATRIOTIC DULL & STUPID ACCTION. THAT' ALL. LUIZ ROBERTO FORTES FURTADO, BRAZILIAN, 80 YO, ENGINEER & URBANIST, WHO KNOW WELL THE AMERICAN WAY OF LIFE.
Cancel
This a politically charged issue. Factions of the Republican party are too invested in getting the federal government out of a previously industry-run cash cow--industry lobbyists will keep funding the coffers of those willing to harp on getting oversight out of the picture. Historically, industry does not regulate itself. The federal government alone has the reach and scope to keep industry within bounds. The industry wants things back the way they were, and the lobbyists will keep funding the political allies who promise to do just that.
Cancel
The failed IRS system replacement is still regularly noted many years after the IRS admitted failure. Plus many taxpayers, many who believe in free markets, were adversely impacted by this abysmal failure.
Cancel
we have only seen the tip of the iceberg, there are many more modules to be delivered and interfaced with, IRS, financial, payments...it is only going to get worse...
Cancel
First and foremost it is far too valuable a political weapon for the Republicans so it will certainly be kicking around at least until the next presidential elections. It is also rich fodder for B-school case studies so no doubt HBR will immortalize.
Cancel
Things will get sorted out....this is a glitch seen many times when a robust application is released to production. Benchmark testes to small production users should have been made first, followed by rolling access to other uses in phases; i.e per state possibly.
Cancel
The failure was too political; it may cost the Democrats their chance at next election. How could the PMO not fully inform the stakeholders that the project was risky for an October launch?
Cancel
@ both jbob and GabelD- I think you both make good points. It will probably get sorted out somehow eventually, but it's just as likely to get worse before it gets better. From a project management stance it makes sense to roll out new features in chunks like state-by-state. But because it's a government job, politics will have to come into play to pick what state goes first, right? Or does it? 
Is there any way to make a government project Agile and successful in the face of all the necessary red-tape? Are there ethical ways to side-step the politicians (from both parties) and just make the software work?
Cancel
I think that there could be some positive effects from all this, in that it shines a light on the inefficient processes of government IT (and it's obviously not just limited to government). More transparency earlier in the process may have helped to avoid some of these issues.
Cancel
Certainly improved project management would have been a great help. However there are lessons to be learned from other industries - where a "cascade" of outsourcing has led to product failure as a result of differences in supplier cultures, incentives, attachment to intellectual property, etc. The interfacing roles require very careful design and consistent monitoring. The factors involved are not unlike those that led to the numerous problems associated with the "dreamliner."

Eugene Schneller, Ph.D.
Health Sector Supply Chain Research Consortium
W.P. Carey School of Business
Arizona State University
Cancel
This article really doesn't discuss what developers could have done as the title suggests
Cancel
Article ignores several facts. The site did not have years to develop, per congressional testimony, the requirements weren't finalized enough to start the design until March 2013, and they continued to change through October. There were 55 vendors on the project, coordinated by CMS (not a single vendor). The project was too big, and too complex for Agile - look at all the back-end interfaces that were required, along with multiple state and federal agencies that had to activate interfaces from their end. This article shows a distinct lack of experience with large projects, or in working with government agencies.
Cancel
Meaningless drivel. The author never mentions the large number of change requests the Feds added to the project weeks before rollout,. The author also has minimal real world experience if they think "agile" saves the day. Talent and common sense save the day, not some latest "buzzword'. " President Obama's administration is also keenly aware of the difficulty to hire skilled technical staff quickly, and is looking into how it can reduce the amount of time it currently takes for the hiring process." And this is related to the project how? Did the author have a target for lines of words/sentences for the article?
Cancel

-ADS BY GOOGLE

SearchSOA

TheServerSide

SearchCloudApplications

SearchAWS

SearchBusinessAnalytics

SearchFinancialApplications

SearchHealthIT

Close