The Top 7 HealthCare.gov UX failures caused by poor forecasting of usage
Executive Summary: HealthCare.gov is experiencing a UX Failure due to inaccurate forecasting of usage, but this has deeper impact on attitudes about government health care in general.
A recent Healthcare.gov UX study highlights the importance of forecasting simultaneous user sessions to ensure applications are accessible.
Failure in forecasting hurts user adoption and in this case increases already uneasy attitudes about the site specifically, and government-provided health care in general.
HealthCare.gov User Experience:
HealthCare.gov is the new government health care plan website that is now offered to U.S. citizens seeking health care insurance in the U.S. The site opened to the public on October 1st, 2013. Unfortunately, as many news stories about overloaded servers have already testified, there are serious and significant UX failures with the site.
The most grievous UX failure currently is the inability for users to access the site due to overloaded servers. As the following UX test reveals, this is potentially causing great harm in opinions about the government health care website specifically and the government provided health care plan in general.
Better simultaneous user session forecasting coupled with simple usability fixes prior to launch could have made for a much better user experience and resulting attitude about the site specifically, and the government health care plan in general.
The Healthcare.gov UX Test:
I was specifically interested in testing the Federal site Healthcare.gov, and not State specific sites such as the ones available in California, New York or other States that have their own websites.
To test the user experience of Healthcare.gov I created a simple usability test with three tasks that a typical person interested in the health care plans would need to perform. The questions the testers were presented with are as follows:
- Use this website to find the health care plan you think is best for you. Which plan is it?
- Use this website to determine how much the plan you think is best for you will cost. Can you clearly identify the differences in price, and the differences in what is covered?
- Use this website to start the application process, it is NOT necessary to complete the application process. Is this process easy or difficult?
I recruited testers from usertesting.com with the following demographics:
- Testers should be located in one of the following states and should NOT have already signed up for a Healthcare.gov plan: Florida, Georgia, Illinois, Indiana, Texas, New Jersey
- Age: 18 – 65+
- Country: U.S. only
- Gender: Any
- Income: $0k – $150k+
- Operating Systems: Any
- Web Browsers: Any
- Web Expertise: Any
Testing occurred across multiple days, States and times, from October 2nd through October 4th of 2013.
UX Test Results:
The results of the UX test demonstrated that even the simple processes of trying to find information about plans and costs were virtually impossible. This was due to the website being overloaded and thus incapable of providing users with the ability to perform the sign up process, the first step prior to finding plan and pricing information.
As this video below demonstrates, the ‘waiting page’ that testers landed on did not provide any information about how long they had to wait. Because of this lack of information, most testers abandoned the process after several minutes.
Inadequate planning of usage by the team at HealthCare.gov is the prime culprit.
According to WikiPedia as of January 1, 2013, the United States had a total resident population of 316,783,000.
If just 1% of those people wanted to access this application after the October 1st launch of the program, that would be about 3,167,830 trying to use the site at more or less the same time.
Even if just one tenth of that 1% tried to access the site simultaneously, that would still be about 300,000 simultaneous site visitors.
Yet according to the USA Today news story, the HealthCare.gov team planned for peak sessions not even close to that number:
“U.S. Chief Technology Officer Todd Park said the government expected HealthCare.gov to draw 50,000 to 60,000 simultaneous users, but instead it has drawn as many as 250,000 at a time since it launched Oct. 1.
More than 8.1 million consumers visited the site from Tuesday through Friday, according to the White House.”
Website and Chat UX Failure Causes Consumer Confidence Issues:
Because of the website and chat system failure, most of the testers had concerns about the government’s abilities to administer health care insurance at all. This is a good example of the old saying that ‘you only have one chance for a good first impression.’
As this highlight video demonstrates, testers assumed that if the website and chat system were not working, the phone lines would be even worse:
HealthCare.gov Receives a 2 out of Possible 10 Score:
Testers were overwhelmingly in agreement that the bad user experience of not being able to find basic plan information and pricing due to the non-working site earned HealthCare.gov a very low satisfaction rating. As the video below highlights, the user experience was rated extremely difficult:
Other HealthCare.gov UX Issues:
HealthCare.gov has other usability issues that cause an overall bad UX score. Listed below are just a few of the other findings of this study:
1. Slow response time with no information about how long the queue is causes confusion, which leads to very high abandonment rates and re-tries, and thus adds additional server load.
2. Chat also experiences slow or no response time with no information about how long it will be before the person can receive an answer to a question.
3. All testers stated they were confident the phone line will be even worse, and do not attempt to use it.
4. Login instructions are confusing:
“The username is case sensitive. Choose a username that is 6-74
characters long and must contain a lowercase or capital letter, a
number, or one of these symbols [email protected]/-“
Several users stumbled on these instructions, and received multiple errors due to their User Name not complying with the instructions. Some wondered if one of the symbols could replace a number, because of the use of “or” for lowercase or capital letter, and a number “or” a symbol.
5. One tester who needed a User Name and Password reminder from the system received an email that had her user name in all caps, but she had entered initial cap and lower case. The system and thus the user’s password is case sensitive, the incorrect all caps User Name in the email caused the user to enter her User Name incorrectly as all caps, this caused the user to be unable to login to the system.
6. There is no detailed information letting the user know the steps in the process to create a login, or the specific set of tasks associated with creating an account and choosing a plan, or where the user is in that process.
7. All testers assumed they would be able to find plan information and pricing without having to apply. Several referred to how other health insurance companies handle plan and pricing quotes, using basic demographic data to provide coverage information prior to applying for coverage.
From the sampling of usability issues the site is already experiencing, we can only wonder about the other user experience issues awaiting those who successfully complete the application process.
This is a reminder that really smart people (and I’m sure the designers and developers of the Healthcare.gov user experience are some of the best out there) can still stumble when creating applications for large sets of users.
The best way to avoid this is with constant testing and evaluation throughout the process, and even then UX issues will still surface. Hopefully the UX team responsible for HealthCare.gov is addressing many of these issues as we read this.
Conclusion: HealthCare.gov UX Failures
Considering the impact and far reaching consequences HealthCare.gov will have on millions of Americans, it is disturbing that so many severe UX failures are present in the system at this late stage.
Unfortunately, the bad user experience the HealthCare.gov site is currently providing is impacting attitudes and perceptions about the site specifically, and about the government’s ability to adequately administer health care insurance in general.
This should be a strong reminder to everyone in the UX community that the user experience is the Brand. Without adequate planning and testing prior to launch, a bad first impression of a system will result in bad perceptions about the Brand. And sadly in this case, that Brand is the United States Government.