blog


What is an Online User Survey?


What is a user survey?

An online user survey is similar to a structured interview with users but completed online and without an interviewer. A list of questions is displayed online and users’ responses are recorded.

 When should it be used?

Online user surveys can be used anytime during a product or website design and development lifecycle. They are a fast and affordable way to collect large amounts of data from a large number of participants. However, it is helpful to have very specific questions that need to be answered and you should be more interested in collecting the who and what such as demographics and quantifiable opinions, preferences, and/or behaviors as opposed to learning the why.

Results can be used to help inform product and website design based on users’ or groups of users’ responses. Online user surveys are used to determine demographic information to better detail the user population. They can also focus on users’ opinions and overall impressions or be used as benchmark studies.  For example, the same online user survey can track opinions over time by being launched every year.

 What do you get?

The output from a user survey typically includes a report that details:

  • The primary goals of the user survey
  • Questions asked to achieve those goals
  • Findings in terms of answers to the survey questions, common themes, and trends
  • Descriptive statistics as well as tabular and graphical displays to help clarify findings
  • Recommendations based on findings

What is a User Interview?


What is a user interview?

A user interview is a semi-structured or structured interview that may be conducted in person or over a phone to collect in-depth information about user needs, goals, experiences, attitudes, and opinions.

 When should it be used?

User interviews are typically performed at the beginning of the design phase. Although rich data from small sample interviews is not generalizable to the larger population, it can be used as a starting point for developing quantitative studies or to explain quantitative results. User interviews can also be used to collect demographic information to better detail the user population. Results can be used to help inform the design based on users’ or groups of users’ opinions and overall impressions.

 What do you get?

The output from a user interview typically includes a report that details:

  • The primary goals of the interview
  • Questions asked to achieve those goals
  • Participant demographics
  • Statistics on the quantitative data from rating, ranking, or other closed-ended questions
  • Common themes or trends from open-ended questions
  • Recommendations based on findings

If desired, user interviews can be recorded and/or transcribed.

What is a Focus Group?


What is a focus group?

A focus group is a guided discussion where a moderator leads an interactive group of representative users through a series of questions and activities focused on specific product or topic.

When should it be used?

Focus groups are typically performed early in the product or website planning process to inform requirements definition and design. Focus groups are great for gaining a better understanding of user opinions, preferences, attitudes, and reactions to a particular topic, concept, or prototype.

What do you get?

The output of a focus group typically includes findings and recommendations that inform requirements, feature offerings, and/or the design of a new or available product or website. Outputs also include insight into who the users are, their characteristics, environments of use, and how the product or website fits (or does not fit) into their lifestyle.

What is a Card Sort?


What is a card sort?

Card sorting is a usability evaluation method used to design and/or evaluate the information architecture for a product or website. Card sorting can also be used to identify logical groupings for toolbars and menus. During a card sort, representative users are provided cards containing items to sort into meaningful groups or categories.

There are different types of card sorting methods:

  • Open Card Sort – Representative users to sort cards into groups that they label.
  • Closed Card Sort – Representative users sort cards into pre-labeled groups.
  • Inverse/Reverse Card Sort or Tree Testing – Representative users are asked to find specific information or topics that have already been sorted into labeled groups.

 When should it be used?

Card sorting is typically performed during the design phase for a product or website when the information architecture is still being developed. However, it can also be used during the development phase or after a product or website has been released to identify if there are potential usability issues associated with how information is grouped or labeled.

What do you get?

The output of a card sort typically includes findings and recommendations associated with groupings and/or labels that can be used to define or validate the information architecture of a product or website and improve discoverability of information.

What is a Heuristic Review?


What is a heuristic review?

A heuristic review is a usability evaluation method where one or more trained usability/human factors experts review a product (hardware and/or software user interfaces) or website and compare it against established human factors criteria and principles for design to identify potential usability issues.

 When should it be used?

Heuristic reviews can be conducted at any time during the design and development phase of a product or website. However, the earlier a heuristic review can be performed (preferably early in the design phase), the more likely it is that identified potential usability issues can be resolved prior to product release.

 What do you get?

The output of a heuristic review typically includes a list of potential usability issues that are categorized and assigned a severity rating based on perceived usability risk and overall impact on the user experience.

Actionable and prioritized recommendations for resolving or mitigating potential usability risks are provided for all issues identified as part of the heuristic review. Visual depictions (e.g., annotated screen captures) are often included to help illustrate findings.

Results of heuristic reviews are presented to product teams in an easily digestible format that can be used internally to track status, add comments, and assign priorities.

Safety-Enhanced Design (Usability) Reports Available…Now What? Finding Reports


I am so distracted today…

First, the build up to the snow and ice event in NC is exhausting. Just let the snow and ice fall; and let the power go out already.

Second, ONC made the Safety-Enhanced Design (Usability) reports public. Thanks @HealthIT_Policy for the link.

Now what?

Well, I know I immediately went to look at some reports. Good luck with that. It seems others must have been having issues as well. Which might have led @Farzad_MD  to repost the original link and to post a link directly to one of the reports.  Thanks for the links.

We are a consulting company and have conducted User Centered Design (UCD) activities and summative tests on several of the certified products. So, I let our clients know the reports are now public. Some of those clients already emailed me back asking how to find the actual reports for their own products and for other products.

If you are having trouble finding UCD and summative test reports on the site… here is a cheat sheet:

 

Here is what I have learned:

  • It seems that products certified through Drummond are most reliable for providing both the UCD report and the Summative Test report.
  • It seems that no products certified through CCHIT have associated reports.
  • Products certified through ICSA Labs have some products with and some products without associated reports.
  • Products certified through InfoGard have some products with and some products without associated reports.

 

Now what…let’s get the discussion started:

  • How can these reports be used in a productive way?
  • How can the data in these reports be used to make EHRs safer and more efficient in support of providing quality care?
  • How can the data in these reports inform future policy?

 

More to come on these topics.

Usability Testing: 5 Strategies to Achieve Meaningful Use Stage 2 Certification


I was recently invited to join Lorraine Chapman and Anneliis Tosiine from Macadmian to be a guest contributor on a podcast about Meaningful Use Stage 2 usability testing. Lorraine and Anneliis are fellow HIMSS Usability Task Force members. The podcast discusses 5 strategies to employ when conducting usability testing for Stage 2 certification. Each of the five strategies stem from User-View’s and Macadamian’s experiences and best practices employed while supporting EHR vendors through the Safety Enhanced Design certification processes.

Usability Testing: 5 Strategies to Achieve Meaningful Use Stage 2 Certification

  1. Planning is Key
    • Obtain prompt access to the EHR that will be tested as part of summative testing. Use this access to familiarize the usability test team with the application.
    • Identify the most efficient click path and all alternative paths that can be used by end users to successfully complete tasks.
    • Work with expert users to identify the optimal completion time for each of the tasks used in the summative test session.
    • Plan for at least 4-6 weeks for a summative test when you have 2 -3 user groups.
  2. Determining Types and Number of Users
    • Start by identifying who your end users (or intended end users) are for specific usability test tasks.
    • Recruit end users who would do the task in the clinical world.
    • Determining number of participants for a summative test is driven by the User Centered Design process your team follows. Human factors best practice is 15 participants per user group are sufficient if you have been doing User Centered Design throughout design and development.
  3. Conducting Remote Usability Testing
    • Usability testing can include remote testing.
    • Remote usability testing (where the facilitator and the participant can be in different locations) can be a great way to reduce cost and expedite the recruiting and data collection process.
    • Keep in mind that if you choose to do remote usability testing, you will need a reliable web conferencing tool so you can view (and record) the sessions in real time.
  4. Importance of Risk Assessments
    • Avoid jumping right into summative testing with end users – do a risk assessment first.
    • Safety enhanced design requires a risk assessment both on the actual tasks/user interfaceassociated with the task you will carry out in your summative test and on the findings in yoursummative test report.
    • A goal of ONC and also for Human Factors / UX Specialists is to ensure the system protectsthe provider and the patient from any errors that might occur due to the user interface. The risk analysis will help identify and mitigate these kinds of problems.
  5. Preparing to Submit to ONC
    • In order to be considered for certification, teams must submit information from summative usability testing. As part of the summative test report, teams are to report the findings of the risk analysis of the use, tested performance, error rates, and satisfaction ratings. Teams are also required to report any critical use errors and specify how the product will rectify these errors. Critical use errors do not need to be reported if the product is redesigned, retested with another summative test, and retesting no longer finds the critical use error is present.
    • An objective, trained human factors researcher / UX specialist should be deeply involved in the preparation of the report for ONC. This is important because summative test reports for ONC will be released publically and you want to be sure that the study is represented accurately and the integrity of the study is maintained when it is shared with the industry.
    • NISTIR 7742 is an important resource to use for reporting.

Listen Now: Macadamian | Usability Testing: 5 Strategies to Achieve Meaningful Use Stage II Certification

You will be required to register with Macadamian to listen to the podcast.

To register, scroll to the bottom of the webpage and click the “Register” link.

The Value Comparing Similar User Needs Across Industries


While every industry has its unique user needs, you can get a lot of value from comparing different industries that on the surface wouldn’t seem to have much in common.

Two independent user-centered design projects were carried out to inform new product development. One project targeted medication inventory management of controlled substances. The other project targeted inventory management of lottery products like scratch tickets and lottery tickets.

Reflecting on the observation and interviews from each project, it became clear the two industries shared user roles, user needs, and workflow. In addition, the two industries shared terminology in how they described their needs, tasks, and workflows.

Medical inventory management

Administrators of pharmacies tasked with inventory management of controlled substances describe that a major problem is loss of product (narcotics) through losing products (e.g., pill slipped into a crevice and cannot be retrieved), damage (e.g., the blister pack is torn), and theft by employees.

Because of the high stakes of loss product (narcotics), pharmacy administrators have a need to carefully track inventory. In fact, tracking must occur at the end of each shift in order to identify any product loss and to begin the reconciliation process as soon as possible.

Throughout a shift, workers must account for each access to the product (narcotics). Employees are required to record missing inventory immediately. In the case of pills fall into crevices and damaged inventory, employees have specific procedures regarding retrieving, storing, and reporting each pill. At the end of each shift, a reconciliation process takes place.

Should a discrepancy in actual versus projected count (based on information recorded) occur, then a reconciliation process begins which includes steps such as identifying each employee who had access to the system throughout the shift, double checking counts and medication orders taken throughout the shift, interviewing each employee who worked during the shift, etc. Sometimes interviews are delayed because an employee is off shift and my not return to work for a day or more.

If discrepancies are not immediately resolved, additional activities take place. Losses are tracked over time and patterns of discrepancies are noted so as to inform future reconciliation processes.

Lottery product inventory management

Turning to stores that sell lottery products, a similar story emerged during the user-centered design process. Store owners tasked with inventory management of lottery products describe that a major problem is loss of product (scratch tickets, lotto tickets, and cash) though loosing products (e.g., a scratch ticket was mistakenly detached from the roll and stuck under the cash drawer), damage (e.g., a scratch ticket was torn), and theft by employees.

Because of the cost of loss product to the store owner, store owners carefully track lottery inventory. In fact, tracking occurs at the end of each shift in order to identify any product loss and to begin the reconciliation process as soon as possible.

Throughout a shift, workers must account for each access to the product (lotto tickets, sale of scratch ticket). Employees are required to record missing inventory immediately. In the case of tickets being removed from the roll by mistake or damaged inventory, employees have specific procedures regarding retrieving, storing and report each ticket. At the end of each shift, a reconciliation process takes place.

Should a discrepancy in actual versus projected count (based on information recorded) occur, then a reconciliation process begins which includes steps such as identifying each employee who had access to the tickets throughout the shift, double checking counts and ticket sales from throughout the shift, interviewing each employee who worked during the shift, etc. Sometimes interviews are delayed because an employee is off shift and my not return to work for a day or more.

If discrepancies are not immediately resolved, additional activities take place. Losses are tracked over time and patterns of discrepancies are noted so as to inform future reconciliation processes.

Comparing industries

As you can see, both industries display similarities.

[what is the value of making this comparison?]

Usability Is a Measurable Outcome


Consumers say, “I want a product that is easy to use. Companies say, “Our product surpasses the competition in ease of use.” Human factors specialists apply knowledge and use methods to assure products achieve usability.

But what is “easy to use” exactly? And how do you measure it?

The International Organization for Standardization (ISO) defines usability as the “extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” [ISO 9241-11 (1998)]

Getting more specific, those terms mean:

  • Effectiveness: The accuracy and completeness with which users achieve specified goals
  • Efficiency: The resources expended in relation to the accuracy and completeness with which users achieve goals (productivity)
  • Satisfaction: Freedom from discomfort and positive attitude to the use of the product
  • Context of use: Characteristics of the users, tasks, and the organizational and physical environments

Through those criteria, usability is a measureable outcome. It’s the result of a process that optimizes effectiveness, efficiency, and user satisfaction.

One such process is the user-centered design process.The distinguishing characteristic of the user-centered design process is the focus on the people who will use the product, service, or website: knowing the user, knowing the user’s needs, knowing how the user works, and evaluating if the product, service, website truly meets identified user needs.

Throughout the user-centered design process, there are appropriate methods to use that are aimed at addressing specific questions and aimed at supporting specific design objectives.

To learn more about user-centered design processes and the appropriate methods, contact us.

Defining a Human Factors Specialist


When someone says, “I am a teacher” or “I am an accountant,” folks get it. People know what those jobs are.

But when I say, “I am a human factors specialist,” I might as well say I am an alien from Mars—except even then folks would still have a better idea of exactly what I was.

In explaining what a human factors specialist is, I try to start with something most people already know: “Have you heard of the term ergonomics before? Like when an office worker gets his or her computer, desk and chair set up to prevent carpal tunnel syndrome?”

That’s a job people are familiar with. So I build on their knowledge, explaining that ergonomics is a sub-specialty of human factors. If you think of ergonomics as designing products to work with people from the neck down, then think of human factors as designing products to work with people from the neck up—the thinking part.

That’s what I do. I am a human factors specialist and I work with companies to assure them that their products, services and websites work for people.

On the human factors side (the cognitive side) I make sure:

  • Folks can easily find a phone number in the cell phone.
  • Medical devices are safe and easy to use so doctors and nurses can think about patients instead of trying to figure out how a medical device works.
  • The ATM gives you back your ATM card before giving you your money. Because once you get your money, nothing else matters and you will drive away from the ATM.

On the ergonomic side (the physical side) I make sure:

  • The back hatch of the SUV can be easily opened even when someone is wearing gloves
  • The radio and heating / air conditioning controls are within reach of the driver even when the driver is wearing a seatbelt.
  • Medication packaging takes into account decreased hand and finger strength of patients, so that patients can open the packaging

My explanation drives some of my colleagues crazy. Among experts the terms human factors and ergonomics are frequently used interchangeably. But when I separate the physical versus the cognitive aspects of human factors, folks understand what it is I do. And making things easier to understand is also part of what I do.