Usability Testing: A Pivotal Point in Product Development

Photo by David Travis on Unsplash

Last week, I stated, “In User Experience (UX) Design, creating a simplified version of a product before its full development is called Prototyping.” Prototype is the fourth stage in the UX Design Thinking Process. This week, I tested my prototype with two users. Although Design Thinking is an iterative process, the final stage of the 5 stage-model is called Test.

Test is the fifth stage with the UX Design Thinking process.

Stage 5: Test

Testing is a crucial component of product development that involves real users. Users interact with a prototype and provide honest feedback. This stage helps UX designers determine whether their product or solution is valid and works effectively or if it needs reworking. Amid a testing session, alterations and refinements may occur as designers observe how users experience and interact with their prototype. Furthermore, the test results may lead back to prior Design Thinking stages, or even as far back as identifying additional needs users may have and redefining the problem. 

User Testing vs. Usability Testing

There are many different methods of testing including concept testing, usability testing, tree testing, etc. Photo from CareerFoundry

User testing and usability testing take place in the testing stage of Design Thinking. The two sound similar, have similar end goals, and are UX terms often used interchangeably (even by me). However, CareerFoundry makes it clear that they are distinctly different. 

Here are the similarities between the two:

  • Both tests seek to design a product or solution that meets user needs
  • Both tests take note of user interactions, experiences, and challenges while utilizing the product or solution
  • Both look for ways to combat and address the identified challenges

Here are the differences:

  • User testing focus on the user. A product or solution is presented to a user to determine whether the user would actually want, need, or make use of the product. User testing asks the user if this product would be of any benefit to them.
  • Usability testing focuses on the product. It “looks at users’ needs in the context of an existing product.” The goal is to observe a user’s interactions with the product to see if the product works for them. There are often tasks for the user to complete which help UX professionals identify how users might journey through the prototype to accomplish their tasks. 

Usability Testing in Practice

A photo of my updated prototype using POP. This screen serves as a confirmation dialog box after submitting a service request using the app.

This week, I performed two usability test sessions using a prototype of an app I am designing for the city of Baltimore called Baltimorean. I’ll take you through the preparation of my usability tests as well as my test results.

Creating Tasks Prompts

Every usability test is performed by observing participants through activities. Within a test session, users have specific tasks to complete. According to Leow Hou Teng, writing good task scenarios for usability tests allows us to “test our products more robustly to collect the insights we need.” 

I wanted my tasks to be realistic and relatable for a potential resident of the city of Baltimore. I recently created user flow diagrams, detailing three specific user journeys within Baltimorean. For my usability test sessions, I converted these three user stories into task prompts to observe how these user flows would take place with real users. 

  1. User Flow 1: Finding Relevant Articles
  2. User Flow 2: Finding a Local Event
  3. User Flow 3: Submitting a Service Request
  • Task 1: You would like to read one of the most recent news articles published by the City of Baltimore. After reading the story, favorite or save it for later reference.
  • Task 2: You have noticed a pothole developing on a road you frequently use. You would like to view all services and notify the city by submitting a service request. 
  • Task 3: You would like to view upcoming events within a specific event category. Once you find an event you are interested in, add it to your calendar. 

Drafting a Script

To prepare for my test sessions, I created a script. My script assisted me with guiding and coaching my users through each session. Following an example test script, I drafted my own script to read aloud during my test sessions and keep each session controlled and focused. The session script helped me do the following:

  • prepare each user for the session
  • ask for their consent to record the session
  • inform them that we would be testing the product and not them
  • ask questions concerning their daily device usage
  • walk them through each task
  • inquire about any additional questions, comments, and feedback they have

You can read my full script within the full usability test document linked below. 

Prototyping with POP

Although I created my first prototype for the app last week, this week I took it one step further by using a prototyping app and web platform called Prototyping on Paper (POP). POP allows designers to turn basic prototypes into interactive ones. Here’s an example of how the app works.

POP App Tutorial by Monica Fabiola Castaneda provides a step-by-step explanation of how to utilize the Prototyping on Paper app to add interactivity to preexisting paper prototypes.

I took photos of my screens from last week’s prototype and formatted them for POP’s iPhone X. Once all screens were formatted correctly, I uploaded them using the POP app and then took to the web platform to link pages and add in interactivity. After testing my digital prototype a few times to ensure the interactivity worked as intended, it was time to get testing! Here is a link to my POP prototype

The process of formatting my original paper prototype for the POP platform.

Although usability tests are traditionally performed in person, I performed my tests virtually via Zoom. Whether remote or in person, test sessions are typically recorded for later reference. 

Usability Test 1

Participant Background

Aideh O., 26

  • Maryland resident, Baltimore County
  • Engineer, Illustrator/Designer/Visual Artist
  • “Looks at computers all day”
  • Spends about 12 hours a day using devices to access the internet
  • Mobile devices include smartphone, tablet, and smartwatch, all Apple-branded
  • Utilizes mobile devices and desktop devices comparably
  • Favorite app: Procreate

Usability Test 2

Participant Background

Tolani B., 26

  • Maryland resident, Prince George’s County
  • Cyber Security Analyst, Videographer
  • “Works all day”
  • Spends about 10 hours a day using devices to access the internet
  • Mobile device: Apple-branded smartphone
  • Utilizes mobile device more than her desktop/laptop
  • Favorite apps: Twitter, HBO Max

Test Results

As you may have noticed while watching my testing sessions, the usability test sessions gave great insight into what works well within the Baltimorean app and what needs revising. Both participants mentioned that the app was very straightforward and clear. 

The following potential problem areas were highlighted during user testing sessions: 

  • The app needs a back button in case users want to return to a previous step within their journey.
  • The icons on the event pages should be revisited to represent and communicate “add to calendar,” “register,” “turn on notifications,” and “contact event planner” more clearly. 
  • The language selector and online payment icons need to be substituted with other icons or text. Language selection can potentially take place on a screen before the app’s home screen.
  • The “more” page may not be necessary because users can access more information on Baltimore city’s website. Additionally, the bottom icon navigation feels more solid and direct with five icons. 
  • On the “services” page, “all services” can be read as “view all services by category.” Options on the “services” page can be labeled with descriptions 
  •  for clarity. Pages within “services” could also use description blurbs so users 
  •  are aware of what they are viewing and what options they have. 

Altogether, users were able to utilize the app and perform tasks seamlessly, finding what they were looking for quickly. They stated that the bottom icon navigation worked well and the app was easy to navigate. 

Final Thoughts

All great products have been tested and tested again. Apps update, websites are refreshed, products are upgraded, etc. The testing does not stop, even after a product, solution, software, or service has been released to the public. Testing is a critical part of product development because improvements can always be made. The more a product is refined and perfected to meet a user’s needs, the happier the user will be. So if you have identified pain points within your first prototype, rework it. If at first you don’t succeed, try again!

Paper Prototyping: A Small Investment that Produces Valuable Results

My paper prototype for Baltimorean, surrounded by some of my materials and alternate screens, fields, and dialog boxes.

Whether publishing a book, selling a product, releasing a new medication, advertising software, selling perfume, you name it, it is commonplace to create a “rough draft” of sorts to ensure that your product has been tested effectively and proved tried and true. In User Experience (UX) Design, creating a simplified version of a product before its full development is called Prototyping. Prototype is the fourth stage in the UX Design Thinking process.

Design Thinking: A Brief Explanation

The five stages of the UX Design Thinking process are Empathize, Define, Ideate, Prototype, and Test.

There are five stages in the UX Design Thinking process: Empathize, Define, Ideate, Prototype, and Test. Individually, each stage is vital, unique, and significant. Although we will focus on prototyping, here is a brief blueprint for all five stages. Keep in mind, “these stages are not always sequential, and teams often run them in parallel, out of order, and repeat them in an iterative fashion.”

  • Empathize: This stage seeks to understand your target audience and the problem(s) they face. This is achieved through immersing yourself in the environment of your audience. Ask questions, take surveys, seek experiences, and dig deep.
  • Define: After gaining insight into your user’s perspective, you can use what you learned to define the problem. In this stage, the goal is to formulate a problem statement. The problem statement focuses on the user’s core needs. It is a guiding point for all subsequent stages. You can view it as a mission or vision statement.
  • Ideate: The third stage involves generating ideas. This is where you brainstorm ways to address your user’s needs. Viewing your problem in alternative ways and thinking outside the box is encouraged. In this stage, it is important to come up with multiple feasible ideas. Quantity wins over quality. 
  • Prototype: Prototyping involves selecting the best of your ideas and creating an inexpensive version of your solution(s). This experimental stage is a chance to create something tangible. Prototypes are often created using whatever materials may be available, including paper, tape, paper clips, play-doh, popsicle sticks, etc. The product is a simple first draft of your main idea. 
  • Test: In the testing stage, your user will interact with your prototype and provide feedback. This stage helps you determine whether your solution is valid or if it needs reworking. Your results may lead you back to prior stages, or even as far back as identifying additional needs and redefining the problem.

Prototyping Explained

In UX Design it is imperative to ensure that a product works before releasing it to the public. Prototyping creates the space to “determine whether or not the design (or changes) work the way you intended them to—before they’re out in the world and in the hands of your users.”

A prototype is a scaled-down version of a product, used to sample or simulate ideas and designs. Prototypes are typically inexpensive and easy to create. They often start with paper, pencils, sticky notes, and other stationery items. Prototypes provide a quick way to investigate your ideas before investing a great deal of time and money into developing a product. Prototypes come in all kinds of shapes, sizes, and versions, “ranging from simple paper models to fully functional, interactive digital prototypes.”

Why Prototype?

Many UX professionals would agree that prototyping is a crucial and valuable step within Design Thinking. Here are a few reasons why DIY prototypes prove fruitful. 

  • Prototyping is universal. Most people are capable of creating rough sketches of their ideas. Prototyping is not limited to designers as any individual can make a paper prototype.
  • Prototyping is affordable. Paper and printed prototyping materials are inexpensive. Additionally, prototyping saves time and money in the moment and the long run.
  • Prototyping warrants honest feedback. People freely offer constructive criticism when they are aware that they are not interacting with a finished product. In this case, paper prototypes may encourage honest feedback more than a polished prototype.
  • Prototyping identifies usability issues and design flaws. While users interact with your product, you will gain insight in real-time. Furthermore, you will notice challenges users may have way before it is too late. “Prototypes enable you to fail early and cheaply.”
  • Prototyping helps designers make informed decisions. Prototyping allows you to test multiple designs, layouts, and solutions to determine the best approach. In every way, prototyping informs you of how a user may experience your product and gives you room to refine your work.

My Paper Prototyping Exercise

What is Paper Prototyping? by Interaction Design Foundation

In recent weeks, I proposed an idea for an app to accompany the City of Baltimore website, called Baltimorean. Following my proposal, I created user flow diagrams to explore how Baltimore residents might find the content and resources they need while using the app. This week, I took my app one step further through low-fidelity paper prototyping. You can view my prototyping exercise in the PDF below. I’ll explain my prototyping process here. 

Remember the User

Before crafting anything, I wanted to ensure that I kept my users at the center of my process. I revisited the following things determined in prior activities:

Target Audience

Baltimorean’s target audience would include all Baltimore residents, especially those familiar with the city website. Residents who frequently visit Baltimore’s site would benefit from an app that gives them easy access to information and resources without having to peruse the city’s site.

App Purpose

Baltimorean is a companion app, created to accompany the City of Baltimore website. This app would focus on the services and resources offered by the city. The goal of the app is to keep users in the loop, providing them with easy access to the city’s latest news and upcoming events. The app would also allow residents to access the city’s directory, submit and review service requests, and make online payments conveniently.

User’s Needs

Baltimorean will meet the following resident needs:

  • Residents will be more informed about city happenings
  • Residents will feel more connected with their community and the city
  • Residents will browse relevant city-related content in a more manageable way
  • Residents will be able to make payments and submit requests more conveniently

App Information Architecture (IA)

A diagram of my companion app for the City of Baltimore’s website. While on the go, users would likely want to access News, Events, Services, and the city’s Directory the most.

User Flows

The following flows are displayed through my paper prototyping exercise:

  • Finding Relevant Articles
  • Submitting a Service Request
  • Finding a Local Event

Gather Materials

An example of low-fidelity prototyped screens, created using some of the materials pictured, including pencils, pens, an eraser, and scissors. Photo by Pixabay on Pexels

At the very least, paper prototyping requires paper and a writing utensil. I looked around my home office and gathered the following stationery items:

  • notepad with a dot grid to ensure well-aligned elements.
  • Sticky notes in varied colors and sizes to point out interactive buttons and fields, identify user selections, and symbolize windows and dialog boxes.
  • Pencils with erasers for tracing and sketching.
  • black fineliner pen for outlining static areas.
  • Scissors to scale screens to size.
  • Double-sided tape to adhere and reuse buttons, fields, and screens. 
  • Cardboard to reinforce my prototype.
  • An old phone case to frame my screens and make the experience feel more concrete.

Determine Approach

Following Ben Coleman’s lead, I focused on small portions of my prototype that work together to make up the larger whole. These small portions include devices, screens, elements, and interactivity. 

Devices

Generally, prototyping requires designers to determine which device they will be designing for. Available user research and data typically inform what devices a specific audience prefers to use. The device determines the size of the prototype, which affects the product layout and the size of the elements within the prototype. 

In this case, I am creating a mobile app. I designed my prototype using screens crafted for an average-sized smartphone. I utilized an old phone case to frame my prototype so that it would feel realistic to myself and my users in the testing phase.

Screens

Here, you select and define the screens that are necessary to communicate the design to your audience. Screens are informed by steps a user may take in their journey to locate the content they need. A list of screens can be developed by referring back to existing UX works including sitemaps, IA, user flow diagrams, and more. 

Thankfully, I previously produced flowcharts to examine the journey of three users of the Baltimorean app. I returned to my flowcharts to develop a list of screens necessary to include in these three journeys. Some screens were recyclable while others were unique to their specific journeys. 

Elements

Like a website, the elements within an app screen can either move or change from screen to screen or remain anchored on every page. Furthermore, items housed within one area of the page can directly impact what the user sees in other areas. It is important to consider how these elements contribute to the screens within the prototype and the user’s overall experience. 

With the Baltimorean app, I chose to maintain the same header on every screen. I treated the app’s bottom icon navigation the same. Maintaining the header and navigation throughout the app assisted me with ensuring my user feels comfortable with navigating the app. It keeps them from getting lost and lets them know what to expect regarding changes from screen to screen.

Interactivity

Next, I thought about how users would potentially interact with the screens of my app? What would they click? What text fields would they need to fill in? Are there pop-ups or dialog boxes? How would I differentiate between static and interactive areas? How would I indicate the user’s selections so that the journey was communicated clearly?

This is where my sticky notes came in handy. I cut up light blue sticky notes, utilizing them for links, buttons, and fields. Light pink signified dialog boxes and overlays. Orange page-marker sticky indicated the user’s specific selection on each screen. 

Create

UX prototyping tutorial: Paper prototyping techniques by LinkedIn Learning. This video provided a great introduction to paper prototyping techniques I utilized in my process.

Once I had a sheet of notepad paper sized to screen, stuck to cardboard, and fit within the phone case, I began creating. I cut out multiple sheets of notepad paper to slide into my phone case to transition between screens. Next, I sketched, outlined, and placed my static header and bottom icon navigation within my prototype. Utilizing the Baltimore city site, my flow charts, and my app IA for reference, I went through each screen on my list. 

I started each screen by sketching the general layout, placing example content on every page. I followed with sketches on light blue sticky notes for interactivity and cut them out to stick onto portions of my screens. Once I had a good idea of how I would place everything, I outlined my sketches, erased the pencil markings, and used double-sided tape to piece everything together. Finally, I added page-marker stickies on the edges of my prototype to indicate user selections and overlaid pink stickies for dialog boxes where necessary. When I felt that my screen was complete, I took a few photos to document the current step in the journey before moving on to the next screen. In my final step, using my photos, I pieced each journey together in a PDF document and added notes to help effectively communicate what is taking place within each step of the journey to the viewer.

Paper Prototyping Takeaways

One of my screens of my Baltimorean paper prototype housed within an iPhone X phone case for a realistic experience.

As I crafted each screen within my prototype, I quickly learned that there are many questions to ask and aspects to explore. Should I use icons? Should I use cards? Does this page need a photo? Is there too much content on this page? Will my user understand what to do here? Are there too many steps? Is this process complicated? Is this clear and concise? Is this meeting my user’s needs? The list goes on.

Altogether, this exercise taught me that paper prototyping is an effective way to test endless possibilities and land on a workable and useful reality. Although user journeys and flowcharts and other tools are highly useful, creating a prototype that fits in your user’s hands gives designers deeper insight. Prototypes clearly indicate what is working and what may need improvements. They highlight imperfections and help designers make calculated decisions. Prototyping produces messy rough drafts that lead to appreciable finished works.

“I’m writing a first draft and reminding myself that I’m simply shoveling sand into a box so that later I can build castles.”

– Shannon Hale

A Basic Introduction to User Flow

Flow is a concept identified by psychologist Mihaly Csikszentmihalyi. According to Csikszentmihalyi, flow is “a state in which people are so involved in an activity that nothing else seems to matter; the experience is so enjoyable that people will continue to do it even at great cost, for the sheer sake of doing it.” Flow happens when an individual is highly focused or in the zone. In User Experience design, flows help designers foresee their user’s cognitive patterns. This helps designers develop products that foster the state of flow.

What are User Flows?

An example of the elements and process housed within a flowchart diagram. Example source: CareerFoundry

In UX, User flows, sometimes called UX flows or flowcharts, are diagrams depicting the path a user follows when using a product. Users can take numerous pathways when utilizing a product. Flows display possible routes a user can take while using a website or application from beginning to end. 

The user flow lays out the user’s movement through the product, mapping out each and every step the user takes—from entry point right through to the final interaction. – Camren Browne, CareerFoundry

How are User Flows Useful?

User flows detail each step in a user’s journey to achieving a goal. A flowchart “begins with the consumer’s entry point on the product, like an onboarding screen or homepage, and ends with the final action or outcome, like purchasing a product or signing up for an account.” Documenting this process within a visual representation assists designers, allowing them to observe, understand, and evaluate the user’s experience. Designers can then update the flow and improve the experience. 

Flowcharts benefit product designers as well as product users. They consider the user, ensuring that the user’s journey is straightforward and their needs are met. If the product meets the user’s needs effectively and the experience is satisfactory, the user will continue to utilize the product.

Creating a Flowchart 

Although flowcharts are a useful UX tool, individuals within many fields and professions utilize flowcharts to communicate the steps and decisions needed to perform a process. Flowcharts are widely used and recognized. Thankfully, there are many flowchart resources online, including guidestutorials, templates, and how-tos. Following are a few tips to make use of before you jump into creating a flowchart. 

User Flow Diagram Tips

Before creating a user flow diagram, start with these steps.

  • Determine the purpose and objective: What will your diagram depict? Which user path do you want to explore? What insight are you hoping to gain? Who will benefit from viewing this diagram? 
  • Assess user and user needs: What kind of user would utilize your product? What goal is your user seeking to achieve? What problem do they need to be solved? Are there specific features that are important to them? How will your product make their experience seamless and convenient? 
  • Create an outline: Draft a list of steps within your flow diagram from beginning to end. Start with an entry point, followed by steps to completion. Will the user have any decision points? If so, what are the results of each decision? Conclude your outline with a final interaction or endpoint. 
  • Define your flowchart elements: All flowcharts utilize shapes to depict starting points, endpoints, decisions, and more. They also utilize arrows to depict the direction of the process. There are many standard symbols incorporated in flow diagrams that communicate varying meanings. Ultimately, the diagram designer determines what elements to use, creating a key so viewers can easily understand what is taking place within each step.

My Flowchart Exercise

I recently wrote about Information Architecture (IA), completing an analysis exercise of the current structure of the website of the City Baltimore. I created a site map for the city’s current IA, followed by a site map proposing a new structure and flow. 

Simply put, Information Architecture is the science of organizing and structuring content in a logical user-friendly way. IA focuses on the organization of information within digital products, such as applications, software, and websites. 

My next task involved proposing a companion app for the website. Here is the IA structure I created for the app.

 

A diagram of my companion app for the City of Baltimore’s website. While on the go, users would likely want to access News, Events, Services, and the city’s Directory the most.

This week, I named the app Baltimorean and completed a flowchart exercise to observe how three Baltimore residents might journey through the app to access specific resources. You can view my complete exercise below. In the meantime, here is my process for one of my users, Savannah.

Baltimorean Overview

Baltimorean would serve current residents of the city of Baltimore, focusing on the city’s services and resources available to residents.

Purpose

This app would focus on the services and resources offered by the City of Baltimore. The goal is to keep residents in the loop, providing them with easy access to the city’s latest news and upcoming events. The app would also allow residents to access the city’s directory, submit and review service requests, and make online payments conveniently.

Functions

  • Make online payments
  • View Baltimore’s most recent news stories
  • Find local event information
  • Submit or view the status of a service request
  • Gain more information about the Mayor 
  • Connect with the city through social media
  • View directory information concerning each 
  •  branch of the city’s government

Audience and Needs

Target Audience

Baltimorean’s target audience would include all Baltimore residents, especially those familiar with the city website. Residents who frequently visit Baltimore’s site would benefit from an app that gives them easy access to information and resources without having to peruse the city’s site.

User Needs

Baltimorean will meet the following resident needs:

  • Residents will be more informed about city happenings
  • Residents will feel more connected with their community and the city
  • Residents will browse relevant city-related content in a more manageable way
  • Residents will be able to make payments and submit requests more conveniently

Savannah’s Story and Scenario

Savannah is a local artist seeking to become more engaged in Baltimore’s art community. My third flowchart depicts Savannah’s process of finding local art events using the Baltimorean app. Photo by Burst on Pexels

Story 

As a local artist, I want to find upcoming art-related events so that I can network and connect with other creative professionals in my city.

Scenario

Savannah is a recent graduate and a local artist seeking a community. She wants to become more connected to Baltimore city’s art community. She is interested in upcoming art events in the city in which she can hopefully share her work and network with other local creative professionals. Savannah wants to get information about all of Baltimore’s upcoming art events and exhibitions. She desires to find these events quickly and conveniently, register, and add them to her calendar. She needs to stay informed because attending these events will result in many things. Savannah will be inspired and encouraged, and hopefully, she will develop connections and find opportunities to showcase her work. 

Flow Chart Elements

Here are the elements I used to represent steps within a user’s process. Additionally, I utilized dashed arrows to indicate the flow direction.

  • Rounded Rectangle: Starting point and endpoint
  • Hexagon: Decision point
  • Rectangle: Action point
Each of my user flow diagrams utilized three shapes— rounded rectangles, hexagons, and rectangles.

Savannah’s User Flow Diagram

Final Thoughts

Creating user flows for the Baltimorean app granted a closer look into the app pages and functionalities. My flowcharts served as testing methods, helping me examine the strength of the IA I developed for the app. Baltimorean is intended to assist city residents with locating and utilizing the city’s resources and services promptly and conveniently. Thanks to my user flow diagrams, I can revisit and revamp Baltimorean’s IA to better meet my user’s needs and improve their experiences with the app. 

User flow diagrams are practical tools that designers use to challenge and assess the information architecture and interface of an application or website. These diagrams communicate every step in a process or task, helping individuals envision what it takes to complete a process and strategize for improvements.