blog


Safety-Enhanced Design (Usability) Reports Available…Now What? Finding Reports


I am so distracted today…

First, the build up to the snow and ice event in NC is exhausting. Just let the snow and ice fall; and let the power go out already.

Second, ONC made the Safety-Enhanced Design (Usability) reports public. Thanks @HealthIT_Policy for the link.

Now what?

Well, I know I immediately went to look at some reports. Good luck with that. It seems others must have been having issues as well. Which might have led @Farzad_MD  to repost the original link and to post a link directly to one of the reports.  Thanks for the links.

We are a consulting company and have conducted User Centered Design (UCD) activities and summative tests on several of the certified products. So, I let our clients know the reports are now public. Some of those clients already emailed me back asking how to find the actual reports for their own products and for other products.

If you are having trouble finding UCD and summative test reports on the site… here is a cheat sheet:

 

Here is what I have learned:

  • It seems that products certified through Drummond are most reliable for providing both the UCD report and the Summative Test report.
  • It seems that no products certified through CCHIT have associated reports.
  • Products certified through ICSA Labs have some products with and some products without associated reports.
  • Products certified through InfoGard have some products with and some products without associated reports.

 

Now what…let’s get the discussion started:

  • How can these reports be used in a productive way?
  • How can the data in these reports be used to make EHRs safer and more efficient in support of providing quality care?
  • How can the data in these reports inform future policy?

 

More to come on these topics.

Usability Birds of a Feather Flock Together


I attended my first annual HIMSS Conference in 2005. I initially attended to network with other human factors professionals and was surprised that I could not find others discussing how human factors methods were being employed to improve health IT. How things have changed!

When I first got involved with HIMSS, usability was not part of the conversation. I remember I could not find anyone who was talking about how to impact the usability of a Health IT products or process. This year at HIMSS14 (February 23 – 27) the usability conversation is prevalent and maturing as evidenced by the sessions focused on the broader context of human factors and usability applied to applications and processes. Don’t miss:

PreConference Workshops

  • Understanding Usability in Organizational Strategy. I will be leading the session; Growing into the Health Care Usability Maturity Model.
  • Pre-Conference Symposium; Patient Safety: Making Healthcare Safer – IT Challenges and Solutions, colleagues will be discussing the topic of applying human factors principles and usability to your EHR implementation.

 

Education Sessions

  • How Workflow and Usability of EHRs Impact Patient Safety
  • Five Proven Methods for Multidisciplinary Teams to Create Clinical User Experiences
  • Measuring Nursing Usability of Electronic Health Records

 

Networking

  • Spotlight on Usability
  • Usability Community Get Together

 

I can’t wait to connect with the many colleagues I have worked with (actually volunteered with) from the Usability Task Force; now the HIT Usability Community. Read how involvement in the HIMSS Usability Task Force provided me great opportunities on the Meet Our Members section of the HIMSS website. Usability birds will be flocking together. Come check out HIMSS usability and get involved!

Usability Testing: 5 Strategies to Achieve Meaningful Use Stage 2 Certification


I was recently invited to join Lorraine Chapman and Anneliis Tosiine from Macadmian to be a guest contributor on a podcast about Meaningful Use Stage 2 usability testing. Lorraine and Anneliis are fellow HIMSS Usability Task Force members. The podcast discusses 5 strategies to employ when conducting usability testing for Stage 2 certification. Each of the five strategies stem from User-View’s and Macadamian’s experiences and best practices employed while supporting EHR vendors through the Safety Enhanced Design certification processes.

Usability Testing: 5 Strategies to Achieve Meaningful Use Stage 2 Certification

  1. Planning is Key
    • Obtain prompt access to the EHR that will be tested as part of summative testing. Use this access to familiarize the usability test team with the application.
    • Identify the most efficient click path and all alternative paths that can be used by end users to successfully complete tasks.
    • Work with expert users to identify the optimal completion time for each of the tasks used in the summative test session.
    • Plan for at least 4-6 weeks for a summative test when you have 2 -3 user groups.
  2. Determining Types and Number of Users
    • Start by identifying who your end users (or intended end users) are for specific usability test tasks.
    • Recruit end users who would do the task in the clinical world.
    • Determining number of participants for a summative test is driven by the User Centered Design process your team follows. Human factors best practice is 15 participants per user group are sufficient if you have been doing User Centered Design throughout design and development.
  3. Conducting Remote Usability Testing
    • Usability testing can include remote testing.
    • Remote usability testing (where the facilitator and the participant can be in different locations) can be a great way to reduce cost and expedite the recruiting and data collection process.
    • Keep in mind that if you choose to do remote usability testing, you will need a reliable web conferencing tool so you can view (and record) the sessions in real time.
  4. Importance of Risk Assessments
    • Avoid jumping right into summative testing with end users – do a risk assessment first.
    • Safety enhanced design requires a risk assessment both on the actual tasks/user interfaceassociated with the task you will carry out in your summative test and on the findings in yoursummative test report.
    • A goal of ONC and also for Human Factors / UX Specialists is to ensure the system protectsthe provider and the patient from any errors that might occur due to the user interface. The risk analysis will help identify and mitigate these kinds of problems.
  5. Preparing to Submit to ONC
    • In order to be considered for certification, teams must submit information from summative usability testing. As part of the summative test report, teams are to report the findings of the risk analysis of the use, tested performance, error rates, and satisfaction ratings. Teams are also required to report any critical use errors and specify how the product will rectify these errors. Critical use errors do not need to be reported if the product is redesigned, retested with another summative test, and retesting no longer finds the critical use error is present.
    • An objective, trained human factors researcher / UX specialist should be deeply involved in the preparation of the report for ONC. This is important because summative test reports for ONC will be released publically and you want to be sure that the study is represented accurately and the integrity of the study is maintained when it is shared with the industry.
    • NISTIR 7742 is an important resource to use for reporting.

Listen Now: Macadamian | Usability Testing: 5 Strategies to Achieve Meaningful Use Stage II Certification

You will be required to register with Macadamian to listen to the podcast.

To register, scroll to the bottom of the webpage and click the “Register” link.

The Value Comparing Similar User Needs Across Industries


While every industry has its unique user needs, you can get a lot of value from comparing different industries that on the surface wouldn’t seem to have much in common.

Two independent user-centered design projects were carried out to inform new product development. One project targeted medication inventory management of controlled substances. The other project targeted inventory management of lottery products like scratch tickets and lottery tickets.

Reflecting on the observation and interviews from each project, it became clear the two industries shared user roles, user needs, and workflow. In addition, the two industries shared terminology in how they described their needs, tasks, and workflows.

Medical inventory management

Administrators of pharmacies tasked with inventory management of controlled substances describe that a major problem is loss of product (narcotics) through losing products (e.g., pill slipped into a crevice and cannot be retrieved), damage (e.g., the blister pack is torn), and theft by employees.

Because of the high stakes of loss product (narcotics), pharmacy administrators have a need to carefully track inventory. In fact, tracking must occur at the end of each shift in order to identify any product loss and to begin the reconciliation process as soon as possible.

Throughout a shift, workers must account for each access to the product (narcotics). Employees are required to record missing inventory immediately. In the case of pills fall into crevices and damaged inventory, employees have specific procedures regarding retrieving, storing, and reporting each pill. At the end of each shift, a reconciliation process takes place.

Should a discrepancy in actual versus projected count (based on information recorded) occur, then a reconciliation process begins which includes steps such as identifying each employee who had access to the system throughout the shift, double checking counts and medication orders taken throughout the shift, interviewing each employee who worked during the shift, etc. Sometimes interviews are delayed because an employee is off shift and my not return to work for a day or more.

If discrepancies are not immediately resolved, additional activities take place. Losses are tracked over time and patterns of discrepancies are noted so as to inform future reconciliation processes.

Lottery product inventory management

Turning to stores that sell lottery products, a similar story emerged during the user-centered design process. Store owners tasked with inventory management of lottery products describe that a major problem is loss of product (scratch tickets, lotto tickets, and cash) though loosing products (e.g., a scratch ticket was mistakenly detached from the roll and stuck under the cash drawer), damage (e.g., a scratch ticket was torn), and theft by employees.

Because of the cost of loss product to the store owner, store owners carefully track lottery inventory. In fact, tracking occurs at the end of each shift in order to identify any product loss and to begin the reconciliation process as soon as possible.

Throughout a shift, workers must account for each access to the product (lotto tickets, sale of scratch ticket). Employees are required to record missing inventory immediately. In the case of tickets being removed from the roll by mistake or damaged inventory, employees have specific procedures regarding retrieving, storing and report each ticket. At the end of each shift, a reconciliation process takes place.

Should a discrepancy in actual versus projected count (based on information recorded) occur, then a reconciliation process begins which includes steps such as identifying each employee who had access to the tickets throughout the shift, double checking counts and ticket sales from throughout the shift, interviewing each employee who worked during the shift, etc. Sometimes interviews are delayed because an employee is off shift and my not return to work for a day or more.

If discrepancies are not immediately resolved, additional activities take place. Losses are tracked over time and patterns of discrepancies are noted so as to inform future reconciliation processes.

Comparing industries

As you can see, both industries display similarities.

[what is the value of making this comparison?]

Usability Is a Measurable Outcome


Consumers say, “I want a product that is easy to use. Companies say, “Our product surpasses the competition in ease of use.” Human factors specialists apply knowledge and use methods to assure products achieve usability.

But what is “easy to use” exactly? And how do you measure it?

The International Organization for Standardization (ISO) defines usability as the “extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” [ISO 9241-11 (1998)]

Getting more specific, those terms mean:

  • Effectiveness: The accuracy and completeness with which users achieve specified goals
  • Efficiency: The resources expended in relation to the accuracy and completeness with which users achieve goals (productivity)
  • Satisfaction: Freedom from discomfort and positive attitude to the use of the product
  • Context of use: Characteristics of the users, tasks, and the organizational and physical environments

Through those criteria, usability is a measureable outcome. It’s the result of a process that optimizes effectiveness, efficiency, and user satisfaction.

One such process is the user-centered design process.The distinguishing characteristic of the user-centered design process is the focus on the people who will use the product, service, or website: knowing the user, knowing the user’s needs, knowing how the user works, and evaluating if the product, service, website truly meets identified user needs.

Throughout the user-centered design process, there are appropriate methods to use that are aimed at addressing specific questions and aimed at supporting specific design objectives.

To learn more about user-centered design processes and the appropriate methods, contact us.

Defining a Human Factors Specialist


When someone says, “I am a teacher” or “I am an accountant,” folks get it. People know what those jobs are.

But when I say, “I am a human factors specialist,” I might as well say I am an alien from Mars—except even then folks would still have a better idea of exactly what I was.

In explaining what a human factors specialist is, I try to start with something most people already know: “Have you heard of the term ergonomics before? Like when an office worker gets his or her computer, desk and chair set up to prevent carpal tunnel syndrome?”

That’s a job people are familiar with. So I build on their knowledge, explaining that ergonomics is a sub-specialty of human factors. If you think of ergonomics as designing products to work with people from the neck down, then think of human factors as designing products to work with people from the neck up—the thinking part.

That’s what I do. I am a human factors specialist and I work with companies to assure them that their products, services and websites work for people.

On the human factors side (the cognitive side) I make sure:

  • Folks can easily find a phone number in the cell phone.
  • Medical devices are safe and easy to use so doctors and nurses can think about patients instead of trying to figure out how a medical device works.
  • The ATM gives you back your ATM card before giving you your money. Because once you get your money, nothing else matters and you will drive away from the ATM.

On the ergonomic side (the physical side) I make sure:

  • The back hatch of the SUV can be easily opened even when someone is wearing gloves
  • The radio and heating / air conditioning controls are within reach of the driver even when the driver is wearing a seatbelt.
  • Medication packaging takes into account decreased hand and finger strength of patients, so that patients can open the packaging

My explanation drives some of my colleagues crazy. Among experts the terms human factors and ergonomics are frequently used interchangeably. But when I separate the physical versus the cognitive aspects of human factors, folks understand what it is I do. And making things easier to understand is also part of what I do.

What is Human Factors


Human factors plays a role in every way that we engage with objects in the world.

It is the science. Human factors is the study of human behavior, human capabilities, and human limitations in the context of the environment where humans work, play, and live.

Human factors is an applied science. It is the application of theories, models, and data regarding human performance and interactions people have with their work place, work tools, toys, and more.

Human factors specialists are experts in human behavior as it relates to cognitive processing, in terms of physical interactions, and as human behavior is influenced by social and cultural interactions. We work with teams to assure products, services, and websites help end users achieve their goals given human capabilities and limitations.

As human factors specialists, we are also experts at research methods. We understand how to select the appropriate research methods and data to address the design or development question at hand.

In companies that focus on designing and developing products, services, and websites, human factors specialists can be found within any number of multidisciplinary teams.  Often human factors specialists are members of the user experience team.

User experience teams highlight the multidisciplinary nature of product design and development where specialists with backgrounds in visual design, information design, industrial design, human factors and other areas work to assure the product service, or website meets the end user’s need, be it at work, home and play.