User Validation

38 prog-user-validation

User Validation within the Program segment is an essential step to validate an approach to a Feature to ensure that it meets the needs of the target users.

Explore More.

Related Mindset:

Experience

Segment:

Program

Inputs:

Wireframes, low fidelity designs, and/or a functional prototype

Outputs:

The output of User Validation are findings obtained from users that ultimately drive revisions to wireframes and designs and help set direction for the design of a solution.

An organization in its ideal state chooses to validate proposed functionality with end users instead of making assumptions.

This step occurs at the Feature level within the Program segment. This validation is an essential step at this phase of the process to ensure that adjustments are identified here, where the cost is relatively low.

Validation Criteria

The goal of validation is to determine whether a Feature as it has been defined fares on the following criteria:

  • Learnability: How easy is it for users to accomplish basic tasks the first time they come across the design?
  • Efficiency: Once users have learned the design, how quickly can they perform tasks?
  • Memorability: When users return to the design after a period of not using it, how easily can they reestablish proficiency?
  • Errors: How many errors do users make, how severe are these errors, and how easily can they recover from the errors?
  • Satisfaction: How pleasant is it to use the design?

Validation Plan

The first step for an organization is to develop a validation plan. This validation plan will outline the subject, approach, and goals for the validation session. The plan should cover the following items:

  • Test objectives: The plan should clearly define what the end goal of the validation session is.
  • Subject of the test: The plan will outline exactly what is being defined and what state it is currently in. It will detail what medium will be used for the validation (wireframe, prototype, paper prototype).
  • Methodology: The plan should detail the methodology that will be used for validation.
  • Participants and recruiting: The plan should identify the individual(s) responsible for recruiting as well as criteria for participants in the session.
  • Tasks: The plan will detail each task that the participants will be asked to perform and evaluate. For workflow validations, this may also be a tree-based list.
  • Usability goals: The plan will also detail criteria for success. This can take the form of a desired completion rate or a percentage of sessions which were error free.

For an organization adopting this process for the first time, this may take additional time. For follow-on validation sessions for the same product, the time will be reduced.

Roles

There at least three roles defined for the validation process. As a part of the validation plan, individuals should be identified for each of these roles:

  1. Recruiter: The recruiter has the responsibility of identifying target users and managing the schedule of validation sessions.
  2. Facilitator: The facilitator will drive the session with a target user. They will guide the user through the tasks and solicit feedback from them during the session. The facilitator will document findings during the session, but their primary responsibility is to focus on the flow of the validation session.
  3. Observer: The observer will be the primary documenter during the session. This individual may ask questions as well.

Validation Documentation

Feedback should be gathered from both the facilitator and observer and combined into a document detailing the validation learnings. This document should include:

  • For each participant:
    • The participant’s reaction to each task in the process
    • The time required to complete each task
    • A note for each task indicating if the participant was able to complete the task
    • If there was any terminology that the user had a question about or that they user clearly misunderstood, there should be notes detailing the stumbling blocks
    • Each issue should be documented and a severity assigned to it:
      • High-severity issues: an issue that prevents the user from completing the task
      • Moderate-severity issues: an issue that causes some difficulty, but the user can ultimately complete the task
      • Low-severity issues: a minor problem that doesn’t affect the user’s ability to complete the task
  • Summary of all participants
    • A listing of all issues that occurred across multiple participants. Each item should contain a percentage of sessions in which it occurred.

Common Pitfalls

Even in organizations that adopt User Validation, there are problems that can occur in the implementation which cause a deviation from the ideal implementation:

  • Substituting for actual users: It is imperative that actual users are leveraged for the validation session. Some organizations have chosen to use internal resources as a substitute for actual users. True validation is not occurring if real users are not being used.
  • Validation occurring too late in the process: The goal of performing validation at the Feature level within the Program segment is to reduce the cost to the organization of making changes to the functionality. If validation occurs later in the process, the cost will increase exponentially.
  • Validating with too many users: There are many factors that go into determining the number of users that should be used. The Nielsen/Norman Group recommends “The best results come from testing no more than 5 users and running as many small tests as you can afford.”1 The benefits of validation are realized more when continual validation across multiple features is executed instead of extensive validation on a single feature.
  • Validating without a plan: As with many parts of the digital value delivery process, it can be tempting to jump to execution without a plan. Validation should not be performed without clearly documenting the objectives and approach.
  • Not leveraging an observer: Some organizations have reduced the resource demand for a validation session by removing the observer role. In most validation sessions, the observer will be the one who documents the most pertinent issues. This is due to the fact that the facilitator is responsible for driving the session, while the observer can spend all of the time observing the participant.
  • Inefficient documentation: The summary documentation should be created within a few hours of all of the sessions being completed. It may be tempting to spend extra time creating more comprehensive or compelling documentation, but the goal is to make the validation process as efficient as possible so that validation can be accomplished for as many features as possible.

Tools

There are a multitude of tools for User Validation and facilitating user testing.

At Universal Mind, we would recommend the following tools:

References