Skip to main content
Loading Explore guides…
Usability Study Basics · Acedra
Home
Explore
Computer Science
Usability Study Basics
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
153 segments
Copy
Hide video
0:03
SPEAKER 1: You know all about the first step in a UX research study, which is planning the study. It's time to move to the second step, which is really exciting-- conducting the research. So let's get started. A usability study is a research method that assesses how easy it is for participants to complete core tasks in a design. During a usability study, researchers follow participants as they interact with a product. The user's feedback helps the design team
0:31
make important improvements to the user experience. Usability studies can take place at various points in the design process. You can conduct a usability study when you have an early idea, like a lo-fi prototype that is only somewhat interactive. Often, this is called concept testing. You can also conduct a usability study when you have an interactive prototype. This is the most common time to conduct a study because it gives the design team insight on what
0:56
needs to be revised or added before the product launches. You can even conduct a usability study with a product that is complete. You may want to change a feature of the product or test if the product is usable with a specific group of people. Imagine this scenario. A local bakery has a website, but customers can't place their orders online. The bakery has asked you to create a new feature that will allow customers to order on their website. You've created a prototype of the website that
1:24
includes this new feature to place an online order. You decide to conduct a usability study to understand how easy it is for users to complete an order. As part of the study, participants navigate the prototype from the landing page to checkout, acting as if they're real customers. As a researcher, you collect feedback as you watch the users interact with the prototype. In some usability studies, you can even interview participants after they're done interacting with the prototype
1:50
in order to get more feedback. You might be wondering, how much feedback about my design should I get? We recommend recruiting a handful of participants for a usability study. For our examples, we'll have five participants in the usability study. This sample size is large enough to uncover major user issues but small enough to keep the cost down. Let's focus on one of our usability study participants Alex. As part of the study, Alex tries to order a birthday
2:18
cake for their son on the prototype of the bakery's website. By following Alex through the ordering process, you discover that Alex's user journey hits a problem. First, the birthday cake that Alex wants to order for their son is pretty specific. They want a rectangular rainbow cake topped with rainbow sprinkles and a unicorn placed in each of the four corners of the cake. Alex selects a rectangular rainbow cake without any trouble. And Alex adds the rainbow sprinkles
2:46
by selecting them from a dropdown menu labeled Decorations. Alex notices that unicorns are also on the decorations list, so they are selected easily. But then Alex hits a problem and veers off the happy user path. When Alex adds the unicorn decoration, there's no quantity option, so Alex can't order four unicorns. There's also not a section for notes, so Alex is not able to specify that the unicorn should be placed in each corner of the cake. To add to Alex's frustrations, the prototype site
3:17
does not include a Help button. During the usability study, Alex is open and honest about these problems. Thanks to Alex's feedback, the design team can now work out solutions to improve the bakery's website. Keep in mind usability studies give UX designers a way to test their designs and get feedback without much risk and a lot of reward. While the user feedback might not all be positive, it will always, always make the product better.
3:44
Coming up, we'll get to know the two types of usability studies, moderated and unmoderated. Let's go.
3:54
Now that we know the basics of usability studies, we'll get into some specifics. There are two types of usability studies, moderated and unmoderated. Let's check them out. In moderated usability studies, a person guides participants through the study in real time. The person who guides participants through the study is known as the moderator. The moderator's goal is to help participants interact with the product and collect their feedback along the way.
4:20
On the other hand, unmoderated usability studies do not have a designated moderator. In unmoderated usability studies, participants test out the prototypes without human guidance. Usually, the study is recorded on video, and the UX team reviews the video footage after the study. Moderated and unmoderated usability studies have benefits and limitations depending on the scope and goals of your study. To compare these two types, let's walk through the benefits and limitations for each in detail.
4:51
We'll start with some of the benefits of moderated usability studies. First, in moderated usability studies, the moderator guides the participant through the study. For example, the moderator might show the participant where to click and ask them about their experience when taking that action. This is beneficial because participants take the exact actions you want to get feedback about. Second, moderated studies allow the moderator to ask specific questions and follow up in real time
5:20
to learn more. Similarly, the moderator can rephrase a question that a participant is not understanding. This is beneficial because it allows you to collect more information that you can act on to improve your design. Third, moderated usability studies allow for rapport building between the moderator and participant, which can help the participant open up and share more feedback about the design. Building rapport is especially important if the design being tested deals with
5:47
sensitive or personal issues. OK. So moderated usability studies sound pretty great, right? Well, they also have some limitations. The moderator could influence or bias the participants. Because a person is guiding the study, there's a chance that person can accidentally let their own thoughts or feelings come into the study. Also, moderated usability studies are less flexible. It's harder to reach populations like single parents or night shift workers who may be unavailable
6:16
during the daytime hours. The study can also be difficult to reschedule if a participant does not show up. In addition, there's a chance that the participant may not identify with the moderator. This could make the participant less comfortable and less open with their feedback. There are many more benefits and limitations to moderated usability studies, which you can learn about in the upcoming reading. All right. Let's shift gears and think about unmoderated usability studies.
6:43
Remember, unmoderated studies do not have a person guiding the participants. Let's discuss the benefits of this type of study. First, it's easier to see how a participant uses the product in the real world. In unmoderated usability studies, you give the participant a list of tasks to complete on their own. This helps you see how the participant would experience your product without a moderator guiding each step. Second, unmoderated usability studies
7:10
allow participants to complete tasks on their own time and in their own space because there is no need for a moderator. This also makes it easier to schedule unmoderated studies. Third, depending on the subject matter, especially if it's sensitive, participants may feel more comfortable giving feedback without others around. As you might have guessed, there are also limitations to unmoderated usability studies. In an unmoderated usability study, participants have no human guidance.
7:38
Participants get a list of fixed questions and tasks. But if they have issues or need technical assistance, there's often no one to help them. Next, unmoderated usability studies don't give the UX team an opportunity to ask the participant real-time follow-up questions. This can be limiting if the participant does not explain their problem in detail because there's no way to follow up once the UX team watches the recording. Lastly, there is little to no control over the environment
8:06
to ensure you have the participants' full attention. Without a moderator present, participants could multitask and not be focused on the key activities in the study. Ultimately, to decide which type of usability study to conduct, the UX team has to consider the scope and goals of the study and the types of participants you are trying to reach. Next up, we'll talk about how usability studies are structured.
8:33
Now that you know the basics of usability studies, it's time to watch one in action. Earlier, you learned that usability studies can be moderated or unmoderated. Join me as I conduct an unmoderated usability study so you can get a feel for how this works in the real world. Let's continue to imagine that we are on a UX team that is creating an app to help people find and schedule dog walkers. The design team wants to learn how easy it is for users to go through the main user
9:02
experience of finding and scheduling dog walkers in the app. To figure this out, I've been asked to lead an unmoderated usability study for the dog walking app. I'll have participants test our low-fidelity prototype. I've recruited five participants to provide feedback from the comfort of their own homes. I'll need your help to watch the recordings of the five participants. As a researcher, your goal is to pay attention to how each user responds to the tasks they are asked to perform.
9:30
As you watch each video, you'll see the participants try to complete a series of tasks that the design team wants to get feedback on. The participants will also share what they are thinking as they try to complete the tasks. Since this is your first usability study, just get a feel for how the process works as you watch each participant. For future studies, you'll take notes during the study, but let's keep it simple for now. It looks like the recordings for each participant just got uploaded.
9:57
Let's get started with our first one.
10:03
JORDAN HILL: Hi there. I'm Jordan Hill. I'm going to be completing this usability study on labels parts of the Gmail product. This is my first time I've ever done a study like this. So let's pull up the prompts and see what I need to do here. Prompt 1. If I said, let's set up Gmail labels for your email inbox, what do you think you might use this for? All right. Labels for my inbox. Well, my inbox isn't the most organized, to be honest. I guess I'd label some of the emails
10:30
under certain categories, like finance, work, or kids. But I don't really know if I see how helpful this would be. My inbox is so full of emails, I'm not sure how this would make it more organized. Prompt 2. Let's open Gmail in a web browser on the desktop computer. Now try to create a new label called Shopping. How would you go about doing that? OK. Open up Gmail. Got it. Create a new label. Hmm. OK.
11:04
Ugh, this is honestly super confusing. I don't even know where this would be.
11:12
OK, honestly, I'm going to give up and google it. This is so confusing.
11:45
Prompt 2 follow-up. Was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? Yeah. This was super challenging. I basically needed to google how to find it. It's confusing to me why Gmail has this feature and then hides it. Not a fan. Now that I'm able to find this feature, I can see how you might use labels sort of like a folder. Maybe they should rename it folders. Prompt 3. Let's say you now wanted to change
12:16
the color of the shopping label to your favorite color. How would you do that? Got it. OK. Try to change a color. Let's try clicking on this thing.
12:29
OK. This is confusing because I just clicked on the color of the label, but it just goes to the emails. There isn't any way to change the label's color. Let me go back.
12:41
Let's try these little dots. Ah. Yes. Cool. Well, that wasn't exactly obvious, but I can see here this is where you would go to change the color. Prompt 3 follow-up. Was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? This was just pretty challenging altogether. Normally, you'd think to change a color right where you would click on it. Instead, I had to hover and find these secret dots. Not ideal, but I'd eventually find it. Prompt 4.
13:13
Finally, try to make a sublabel named receipts. Receipts in a sublabel? Hmm.
13:28
Let's try those three dots again. Those seemed useful even though they're basically hidden.
13:40
This is confusing. I don't see anything here about sublabels. Ugh, it has to be here somewhere. This is super annoying. Honestly, I'm just going to skip this one. Sorry. Prompt 4 follow-up. Was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? I eventually figured it out, but I don't really understand why you'd use a sublabel or what I'd really want it for. I also don't know if nesting and sublabel mean the same thing. Prompt 5.
14:12
How do you think you'd improve setting up of a label? Absolutely. I think creating new labels should be in front and center. And I also think changing the color would work a lot better if you could just click on the actual colored area next to the label. OK, so wow. This is definitely not an intuitive situation. I don't understand why Gmail hides the label feature. I mean, honestly, it's pretty much buried under Settings. I don't think you can even see it. But I think that's it.
14:39
I've gotten through these tasks, so that's it. Signing off.
14:46
ALEX: Hello. I'm Alex. I'm going to be completing this usability study on the labels part of the Gmail product I've never used labels before, but I love being organized I'm excited to see if I can use this feature. So let's pull up the prompts and see what I need to do here. If I said, let's set up Gmail labels for your Gmail inbox, what do you think you might even use this feature for? Let's see. I'd imagine this is some kind of folder I could use to label categories in my email.
15:16
I'm not entirely sure how that helps overall, but I'm definitely interested in it. If it does work like a folder system, I can imagine it would be a great way to archive old emails. So let's open Gmail in a web browser on a desktop computer. Now try to create a new label called Shopping. How would you go about doing that? Oh. Honestly, I have no idea. I'm going to guess labels are in the Settings area somewhere. All right. Let's take a look. OK. So here we are on Gmail.
15:49
Labels labels labels labels. We have a gear icon. That's always promising. Settings. That's good. Density Theme. Maybe See All Settings. Well, this is weird. This is definitely Settings. I'm not sure where it would be. There's categories here, but nothing about labels. Let me try this Help button. Oh great. A search box. Let me go ahead and search for labels. Create labels.
16:24
How do I create labels in Gmail? Create labels to organize Gmail. There we go. That's what I want. Let me just read this really quickly. Create a label. On the left-- oh, there we go. Look at this. Scroll down and then click More. OK. More. Where is that? Here it is. All right. Well, let's click on More. All the way at the bottom of More-- hey, hey, check that out. Create a New Label. I see it now. Now let's create a new label. If you remember, our label is going to be Shopping.
17:03
Oh, I see. It's way down hidden on the left. OK great. Got it. Done. There it is. Was this a particular feature or functionality useful? Why or why not? What was easy and what was challenging? Well, honestly, my biggest problem was just getting started. I had no idea where to start. I already knew the gear icon was the Settings icon inside of Gmail, so it was confusing that the labels weren't there. Now that I've got the label created, I can honestly see why they put it in the main inbox left
17:36
window pane. It's a lot more accessible there with the other labels. Let's say you now wanted to change the color of the Shopping label to your favorite color. How would you do that? OK. Change the color. Let's start. Click on Shopping. Let's go back down to Managing Labels. Oh wow. Yeah, well, this doesn't seem to be it either. Nothing about colors, anyway. Well, I'm just going to google it. Honestly, that's often worked for me in the past. Let's try it. So let's go to google.com.
18:19
And let's look at change the color of a Gmail label. Let's change it to red. So let's go ahead and search that.
18:35
I see. There's three hidden dots and a dropdown menu. Well, let's go back into Gmail, find our Shopping label. Where did you go? Look at that. Three little dots as I hover over slowly. Click on that. And let's change the label color to red so I never miss an email. The label for Shopping was adjusted. Nice. Was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? Well, changing the colors would definitely be useful
19:14
and a great way to stay organized. I'm a very visual person. But I can't imagine having found that hidden dropdown menu without first searching on the web. OK. Finally, try making a sublabel named Receipts. OK. Now that I know where the colors area is, I can see this thing called sublabels. This doesn't seem too difficult. Let's go ahead and add a sublabel called Receipts.
19:43
And look at this. Nest the label under Shopping. Perfect. Click Create. Look at that. Now I have a label for shopping and a label for receipts. Nice. Well, was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? Absolutely, this would be useful. I definitely see myself using sublabels quite a bit, actually, like for the family. We have all these different areas to organize-- doctors' appointments, school meetings.
20:14
I could see using it for that. And it really wasn't hard at all to get set up since I did the other tasks first. How do you think you'd improve the setting up of a label? Well, if Gmail wants people to use labels, I'd recommend making it more prominent where to start. I had no idea, and it took me a long time to figure that out. They could move up the Create Label area so people can see it without digging first. Other than that, I think the rest of it makes a lot of sense. OK. Sweet.
20:45
It looks like I've completed the tasks. Thank you.
20:52
CHRIS: Hi. I'm Chris. I'm going to be completing this usability study on the labels part of the Gmail product. So the first prompt says, if I said, let's set up Gmail labels for your email inbox, what do you think you might use this feature for? I could see myself labeling some emails so I can find them again. Probably mostly I'd want to label things VIP and Respond so I remember to get back to people. So prompt number two says, let's open Gmail in a web browser on a desktop computer.
21:22
Now try to create a new label called Shopping. How would you go about doing that? Since Gmail organizes folders on the left, I'm going to take a guess somewhere down here is the label spot. Yep. It's right here. OK. I'm adding this label and calling it Shopping. OK. So the prompt says, was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? Definitely not challenging. I think I've used these in the past maybe.
21:54
It was pretty intuitive to find the label area and make a new one. So prompt 3 says, let's say you now wanted to change the color of the shopping label to your favorite color. How would you do that? Well, I did see some sort of label settings next to creating a new label, so let's give that a try.
22:17
This isn't very clear. Hmm.
22:25
I'm having trouble finding what I'm looking for. Oh, what about this? These three dots look like a submenu.
22:34
Got it. Great. Now it's blue. So the prompt 3 says, was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? I'm definitely a big fan of color coding, so I could see this as being useful. I actually organize my bookshelf by color, so it's cool to use this in email, too. These three dots are pretty hard to find, but I've used email products like this before, so it made sense to me where it might be. So the next prompt says, finally,
23:02
try making a sublabel named Receipts. Sublabel OK. Hmm.
23:12
Well, that clearly isn't right. This is for editing the label name. Let me go back to that menu of options.
23:21
Oh, now I see it. Add Sublabel.
23:31
OK. So prompt number 4. Was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? It was a little hard to find the sublabel area, but I could definitely imagine using this. I have a lot of little categories like receipts that actually might be nice to organize under bigger areas. So yes, this could definitely work. OK. Prompt number 5. How do you think you'd improve the setting up of a label? Overall, I think the product should highlight labels
24:00
more prominently. I'd also love some of the settings to all be in one place instead of so many different windows. Well, I think that covers it. Bye.
24:13
SAGE: Hey. I'm Sage. I'm going to be doing this usability study on the labels feature of the Gmail product. I'm not sure what to expect, but let's see what happens. Prompt 1. Let's set up a Gmail label for your email inbox. What do you think you might use this feature for? Well, if you get a lot of emails every day, maybe labels can help you organize everything in a better way, I mean, to put everything in different categories. Prompt 2. Let's open Gmail in a web browser on a desktop computer.
24:41
Now try to create a new label called Shopping. How would you go about doing that? Yeah. So I open up my Gmail, and then I look for it.
24:58
OK. I guess this is the right place. I'm creating the label now.
25:10
Prompt 2 follow-up. Was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? Yeah. I think this was easy to do after a little experimenting. Prompt 3. Let's say you now wanted to change the color of the shopping label to your favorite color. How would you do that?
25:32
This works pretty easily. And there are not too many steps. Let's try this, maybe changing the settings or going to the Help menu.
25:46
Ah. Yeah. This task seems easy to do. Yes. It seems not clear how to make the labels, but if you play with the mouse in that area, you can figure it out. Prompt 3 follow-up. Was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? For me, this idea is OK, but what about colorblind people? Did Google think about that? It's not practical for everybody. Prompt 4. Finally, try making a nested label named Receipts.
26:18
I can try making a sublabel now that I know about the labels feature. This should be easy.
26:37
Prompt 4 follow-up. Was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? These features work well, I guess. And they keep you organized. I'm good with Gmail, so I don't have too many problems. Prompt 5. How do you think you'd improve the setting up of a label? Maybe a little video to explain. A lot of people will ask how a label is different from a folder. And I still don't understand why I should use labels.
27:04
I think I finished everything I'm supposed to do. Cool. Bye.
27:11
TAYLOR: Hi, everybody. I'm Taylor. I'm joining a usability study about creating labels in Gmail. So excited to get started. And I'm sure that I'll learn a lot of useful information. OK. Let's start the prompts and see how this labels feature works. Prompt 1. Let's set up Gmail labels for your email inbox. What do you think you might use this feature for? I work in marketing. And I have people who work above me that I'm emailing every day. And I have a long list of clients,
27:36
so definitely something like this would help. Maybe one list for the clients, one for my team, and, if I like, labels for promotions about new products. I have two work Gmail accounts and one personal Gmail account, so I wonder if labels is a feature that's for professional or personal use. But maybe when I try out the feature more, I can understand what it's going to be used for, right? Prompt 2. Let's open Gmail in a web browser on a desktop computer.
27:58
Now try to create a new label called Shopping. How would you go about doing that? I would open up my Gmail again, then scroll down and click More. Is that the right way? Labels would have to be one of the options after that, right? This part is not easy. OK. There it is. I see Create New Label on the left side.
28:23
So far, so good. I always panic a little when I have to use new features in Gmail. I actually used another email client for the last 15 years. I used it at my job in construction management. Did I tell you about that? So I'm really super used to that old system, and I only started using Gmail about two months ago. So you might say that changing to new systems and learning new features can definitely make me feel a little bit stressed. Prompt 2 follow-up.
28:47
Was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? This was really useful, but I'm not sure how you would make the appearance of the labels different. I mean, if you have only a few labels, it's not important. But if you have many labels and they all look the same, it's not efficient. Google should think about adding more options to change the appearance, like fun colors or fun animation. Prompt 3. Let's say you now wanted to change
29:12
the color of the shopping label to your favorite color. How would you do that? Yes. Manage Labels seems like the best place to start if I want to edit any of the details of my labels. Let's see if it works. Let's try this out. OK, the steps seem pretty straightforward. Yes. This seems pretty easy to follow. But how do I choose the color? That part is not clear at all.
29:35
OK. Got it. Now this is the fun part. My favorite color is red, so I'll choose red. But then again, red is not such a professional color. Let's make it blue. My job can be pretty stressful, so I need a relaxing color for sure. Prompt 3 follow-up. Was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? Colors are such a good idea for making categories for the labels and to make them easy to find.
29:58
But one of my Gmail accounts has a black background, so if I add in different label colors, that might be hard to see. Maybe Google should design an app that we can use to customize the colors of our Gmail folders and labels. Prompt 4. Finally, try making a nested label named Receipts. Let's see if this task is easier than the last one. I hope it is. OK. Scrolling down from Show to Hide, to Edit, to Remove Label, to Add Sublabel. That's it. OK. This word nest just threw me off.
30:29
I'm always afraid that I'll check some feature that I want and then not be able to change it later. Maybe if there was a little question mark icon next to the word nest, it would help. Prompt 4 follow-up. Was this particular feature or functionality useful? Why or why not? What was easy and what was challenging? This feature was easy to navigate, but I don't think it will be very helpful for me to create all these sublabels. It could be extra work.
30:49
Instead, I would probably just make labels and forget about that sublabels part. Prompt 5. How do you think you'd improve the setting up of a label? Google could create an instructional video that explains all of these steps. It seems like all of these steps would be easy. But when you open your Gmail account and see all these options and folders in front of you, it can get a little confusing. Already, there are so many buttons you see and different places to access features
31:11
when you go into Gmail. And all of this really adds to the stress of how to find what you're looking for. But overall, the experience was positive. And I'll consider ways to use labels for my work Gmail accounts in the future. I guess I finished everything. Awesome. Thank you for the opportunity. OK. Thanks.
31:30
SPEAKER 1: How was watching your first unmoderated usability study? Hopefully, you now have a feel for how an unmoderated usability study is conducted in the real world. Pretty cool, right? As we discussed earlier, there are two types of usability studies, moderated and unmoderated. We're going to shift our focus to moderated usability studies now. In this video, you'll learn how to moderate a usability study yourself. We'll start from the beginning of the study
31:56
and focus on how to connect with participants before the study actually begins. Ready to get started? First, it's really important to build a rapport with participants. Ask general questions about how they're doing or what their day has been like. Doing this before the study starts can go a long way to make the participants feel comfortable. You want to establish a professional but friendly rapport with the participants right from the start. Second, thank participants for coming.
32:23
Don't skip over this step. Make sure participants know how grateful you are that they're taking the time to participate in the study. Even after the study is conducted, it's important to show how grateful you are for their participation. You can do this by sending them off with a small thank-you gift. It's also helpful to remind participants that their feedback on the product has the potential to help millions of users. It's important to be open and honest about their experience
32:49
so the design team can make improvements. If a participant is worried about offending the design team, you won't get the useful feedback you're after. Emphasize that nothing they say will offend you or hurt your feelings and that any issues that they have navigating your design will probably come up for many other participants, too. Let the participant know that you're here to find out how to improve the design, so constructive criticism is more than welcome.
33:15
Next, it's time to have participants fill out paperwork. You'll likely have participants complete a nondisclosure agreement, or NDA, which we've discussed earlier. This might sound intimidating, but your job as the study moderator is to frame this in a casual and approachable way. Since the designs they'll provide feedback on might not have launched yet, you want to ensure participants don't share any details about the designs with the outside world. But it's not all about us.
33:43
If the NDA is set up correctly, it helps build trust with participants and increase rapport. This is especially important with participants from historically marginalized communities or at-risk populations, who have often been taken advantage of in research. Once the NDAs are signed, explain the focus of the study and what participants will give feedback on. Give the participant a roadmap or a preview of what's to come. It's always a good idea to let them
34:08
know what to expect and then to ask, does this sound good? Make it a practice to check in with the participant to make sure they're comfortable with what you've planned. Now it's time to establish the ground rules for the study. For example, you might want participants to use the think aloud method, meaning you want them to explain what's on their mind while they're doing it. This is the time to let participants know what you'd like them to do and give them
34:33
specific examples of how, for instance, a think aloud might work. Also, remind participants that they are not being tested. There are no right or wrong answers to any of the questions you ask. If the participant can't complete a task, that isn't a reflection of their personal abilities. It's a reflection of the design's usability. You want this to be a low-pressure experience. No participant should leave a study feeling like they failed somehow.
34:59
The goal is to understand how the user is experiencing your product, so there's no right or wrong answer. There's one more thing to keep in mind when moderating a usability study. Keep your emotions neutral. Your mood should not impact your judgment or influence your participants' feedback. Make time to reflect on your emotional state before the usability study begins. This helps you separate your emotions from what goes on during the study. And if you have a hard time keeping
35:26
your emotions neutral during the study for some reason, this reflection can help you identify where your emotions might have influenced your research. And that's it. You now know how to get started at the beginning of a moderated usability study. You may be noticing a theme. In UX design, you want to keep the user front and center in all of your work. And in UX research, you want to keep the participant front and center each step of the way. The best practices we just shared
35:52
are all about the participant and their experience during the usability study. Up next, we'll move into how to effectively communicate with participants during the actual usability tasks. Let's keep going.
36:08
You're doing a great job learning about moderating a usability study. You've set the stage at the beginning of the study to make your participants feel comfortable and confident. Now let's focus on how to effectively communicate with participants during the usability tasks. When you're the moderator of a usability study, your main goal is to pay attention to and understand your participants. To conduct a successful moderated usability study,
36:31
you need to know what kinds of questions to ask participants and how to ask them. So let's go through a few techniques for moderation during the usability study itself. The first technique for moderation is to ask open-ended questions. Open-ended questions are questions that cannot be answered with a simple yes or no response. This gives participants more opportunities to provide feedback. For example, after participants complete a task in the usability study, ask,
36:58
how do you think that went, instead of, did you like it? As you interview participants, there might be times when you have trouble understanding their feedback or you may not be sure if you're correctly interpreting what the participant is saying. That's totally fine. All you need to do is ask some follow-up questions so the participant can clarify what they mean. This is our second technique for moderating a usability study. Ask follow-up questions.
37:25
Ask questions like, tell me more about that, or, can you expand on that? These questions will help you avoid making assumptions about what the participant might have meant. If you're still unsure what the participant's feedback means, try to summarize their answer for confirmation. Repeat back what you think the participant means and allow them to correct or confirm the statement. Be careful not to summarize their feedback for confirmation
37:50
right away, or you'll run the risk of leading the participant to a certain answer. Always ask the participant open-ended and follow-up questions first. Then use this technique if you're still unclear. All right. Now you know a few techniques for moderation during the usability study itself. Making progress. As you're moderating the usability study, keep in mind that you'll have a diverse group of participants. As a researcher, you come into the study with biases you need to be aware of.
38:18
Next step, we'll learn about biases that are likely to come up when you're moderating usability studies.
38:28
You might remember when we discussed biases in UX research in an earlier course. In this video, we'll focus on biases that might occur when you moderate usability studies. Remember, a bias is favoring or having prejudice based on limited information. It's important to call out that all humans are susceptible to bias. Yes, even you and me. The key is to identify and become aware of these biases so that you can guard against them. Identifying your own personal biases can be challenging.
38:57
Because bias is usually unconscious, a lot of times we don't even realize we're experiencing it. But if you know what biases you have, you can take steps to minimize their effects. Let's go over some common biases you or your participants might have during usability studies. Let's start by discussing a couple of biases that you might have as a moderator during a usability study. The first bias is one we've already discussed, implicit bias or unconscious bias.
39:22
Remember implicit bias is the collection of attitudes and stereotypes we associate to people without our conscious knowledge. These attitudes and stereotypes are often negative, exclusionary, or disempowering. When recruiting participants for usability studies, any implicit biases you have against a particular user group might impact your expectations for how those participants will interact with the product. Another bias you might have as a moderator during a usability
39:49
study is serial position effect, which we've discussed in a previous video. Serial position effect is a psychological bias that states that when given a list of items, people are more likely to remember the first few and the last few, while the items in the middle tend to blur. Be aware that when you're interviewing participants, the first and last things they reveal might stand out to you more than the feedback in the middle because of the serial position effect.
40:15
Similarly, your participants might experience the serial position effect. If you give participants a list of instructions, they're more likely to remember the first and last items on the list. Next up, another bias often experienced by participants is friendliness bias. Friendliness bias describes the tendency of people to agree with those they like in order to maintain a nonconfrontational conversation. Think about this in the real world. Has your friend ever asked you if you
40:41
want to eat at their favorite restaurant? Even if you hate the food, you might agree to go to their favorite restaurant in order to avoid confrontation. While this might have benefits in the social world, in the world of usability studies, it can present a bit of an obstacle. As a moderator, if you are too friendly and develop too strong a rapport with participants, there's a chance participants will want to agree with you in order to avoid confrontation.
41:04
This can stop participants from giving honest feedback. Your goal is to improve the product's design, so remind participants that you actually need honest feedback in order to improve the product. The last bias moderators need to guard against is the social desirability bias, which describes the tendency for people to answer questions in a way that will be viewed favorably by others. You've probably seen this happen in your own life. Imagine you're with a group of friends who are talking
41:30
about a movie they enjoyed. One of them asks you, did you like the movie? And everyone turns to see your response. You're likely to say, yes, I loved it, to be viewed favorably by the group, even if you didn't actually love it. Similarly, in usability studies, the social desirability bias can cause participants to focus on the positive aspects of their experience with the product and minimize the negative aspects. One way to guard against this bias
41:56
is to provide participants with a series of statements that came from other users. Ask participants which statement they most relate to and emphasize that there is no correct answer. OK. You're becoming a pro when it comes to usability studies. You now know the bias to watch out for when you're conducting a usability study and interacting with participants. Coming up, we'll continue this conversation and share some strategies for reducing your own bias during usability studies.
42:27
You now know a few common types of bias that can come into play during usability studies. Keep in mind that being aware of biases like these is actually the first step in reducing them, so we're already on our way. In this video, we'll explore strategies for reducing your own bias when conducting usability studies. The first thing you can do to overcome bias is to identify and admit your own biases. As part of our work, UX designers make a lot of assumptions.
42:55
But we also test them out and revise them when we get new information. By acknowledging your own biases, you can be more objective and question your assumptions as you conduct a usability study. For example, imagine you're designing a website that sells handbags. Before we design the website, let's start with a little introspection or soul searching. Make sure you have a piece of paper and a pen ready. Pause the video if you need to grab supplies.
43:22
Think about handbags and answer the following questions. We'll give you 10 seconds to write down ideas for each one. Who uses handbags?
43:42
How are handbags used?
43:54
What other thoughts do you have when you think about the word handbags?
44:10
Let's review what you've written down. When asked the question, who uses handbags, did you list a certain group of people, like women? Did you consider that men might also carry handbags? If you primarily thought about handbags being used by women, you may have a bias that men are not interested in handbags. In some cultures, it's common for men to use handbags every day. And when asked the question, how are handbags used, did you list, to carry lipstick, keys, or a wallet?
44:39
Based on what you listed, you may have a bias that handbags are used to carry certain items. Sometimes handbags are used to carry dogs or your lunch. Learning about all of your biases will take a lot of work and reflection. It might even be hard and uncomfortable. This exercise is just one simple example to get you thinking about what your biases might be. Being aware of your own biases can help you avoid forming any opinions before you have results from your research.
45:06
When conducting usability studies, be sure to phrase questions in a way that allows for a diversity of responses. The next thing you can do to reduce bias when conducting usability studies is to find participants from a representative sample. The small group of participants you conduct your usability study with should represent your key user group as well as user groups that are often marginalized. Next, define the research criteria beforehand.
45:33
Remember when you created a plan for your usability study? A couple of the key elements focused on the research goals and research questions. By establishing research goals with your team before the usability study begins, you reduce the chance of designer biases leading you down the wrong path. Also, phrase interview questions thoughtfully. If we ask a participant, do you like the design experience, we might not get all the information we need. Stick to open-ended, non-leading questions
46:02
like, how did it go, what was your experience like, or, what worked and what didn't work for you and why? Remember, the goal of usability studies is to get honest feedback from the participants. Next, let participants express themselves fully and in their own words. During the usability study, make sure you understand exactly how participants feel and why. As we discussed earlier, if you need more clarity on what a participant says, ask a follow-up question.
46:30
Finally, be mindful of your body language. Remember the friendliness bias? A lot of us are people pleasers. If you react favorably to something when you are the moderator by nodding or smiling, this might encourage participants to share feedback that will get them more nods or smiles. Body language like this can bias the feedback that participants provide. Identifying your own biases and how your actions can bias participants is a long process that takes a lot of practice.
46:59
You've learned some of the most common biases in UX research and effective strategies for combating bias, so you're already off to a great start. Coming up, we'll learn how to take notes during a usability study.
47:16
Now that you know how to conduct your study and avoid biases in the process, it's time to think about another important part of a usability study, note taking. In this video, we'll explore why you need to take notes during a usability study and who takes notes and ways to take effective notes. You might be wondering why we take notes during a usability study, especially if the session is being recorded. Well, there are a lot of reasons. First, note taking lets you capture any thoughts
47:44
you have during the usability study. For example, you can indicate what you notice about each participant's body language or the way that they're talking. Are they smiling? Do they seem frustrated? These are all useful observations to put into your notes. Remember to keep what we just learned about biases in mind whenever you make assumptions on how a participant is feeling based on body language or tone of voice. One way to avoid this bias in a moderated study is to ask the participant how they
48:13
felt at a point in the study where they appeared to show an emotion. Think back to the mock usability study from earlier. When conducting a usability study live or watching the recordings, you can take notes on the tone of the participant's voice or even their word choice when describing the tasks. Let's review how participant A completed a task in the example usability study.
48:37
SPEAKER 2: Prompt 3. Confirm booking of dog walker and complete the checkout process. Got it. OK. I'm going to click Book right here. This seems fairly simple. What the heck? It looks like this is a page with details about my booking and the total cost. The app didn't even give me a page to confirm the dog walker I selected or the date and time I need. I'm not even sure if I had all the right details, but it went right to confirming the appointment. That's so annoying.
49:12
I feel like most apps let you review what you're booking before you actually finish paying.
49:20
SPEAKER 1: Now let's review how participant D completed the same task. SPEAKER 3: Prompt 3. Confirm booking of dog walker and complete the checkout process. Is booking the same thing as completing the checkout process? Is there a cart or something? Let me go back. Maybe I missed an Add to Cart button. I'd expect to see an Add to Cart or something here. I'm not sure on this one. Let me go to the Learn More page I was on earlier. This must be somewhere. OK. So it looks like I'm back here again.
49:57
My only option seems to be click on the Book button, so I'll try it. Oh wow. Seems like that worked, although I really don't know what happened. The screen says congrats and total cost, but I don't really know what I booked or clicked. Like, there was no confirmation page to tell me about the dog walker, the time, or price I was scheduling. How strange. I don't think I ever had this happen before. Hmm. SPEAKER 1: What do you notice about the word choice these participants used?
50:28
Did you catch that their words made it sound like the task was not easy? If you can hear their voices, you may have noticed that their tones were very different from one another. Participant A sounded frustrated, but Participant D spoke in a positive tone. Even when they found the task difficult, they still remained upbeat. This is why note taking is so important during usability studies. It helps you capture thoughts that the participants may or may not verbalize.
50:56
Second, you might take notes to summarize a participant's experience during the study. Let's review how Participant C felt about the whole experience. SPEAKER 4: Prompt 5. Would you use the dog walking app? Yeah. I'd definitely use this, for sure. It really seems super simple to use. I'd just love to see a little more customization and, of course, some more actual details of the content. After that, I think it's pretty much ready to go. Thanks for letting me try it out today.
51:27
SPEAKER 1: For example, you may write, Participant C felt the app was simple to use and would love to see more customization. Third, your notes can highlight really compelling quotes to include in your research report. Let's review how participant D completed a task in the example usability study. SPEAKER 3: Prompt 3 follow-up. Did you find these tasks easy or difficult to complete? Is there anything you would change? Considering I had to go back and forth
51:54
between screens of the app, that was not simple. Usually, with any sort of checkout process, there's an Add to Cart function. I was just clicking on different buttons to try to figure it out. SPEAKER 1: For example, you may write something like this. "Considering I had to go back and forth, yeah, that was not simple. Usually, with any sort of checkout process, there's an Add to Cart function." Now that you know why notes are important, let's discuss who takes them.
52:22
As an entry-level UX designer, you won't run a usability study on your own or be expected to take all the notes. If you have a UX researcher on your team, they will likely run the study and take the notes. Even if there is a dedicated researcher for the study, it's helpful for other team members involved in the project to attend and take notes. Why? Because each team member is most likely to notice issues that impact their work. For example, a UX writer might notice
52:48
that a participant pressed the wrong button on the app because the wording was confusing. On the other hand, a UX designer might notice how long it took a user to find a button. At Google, we take thorough notes. In fact, whether the study is conducted remotely or in person, many team members will observe the study. Once the study is over, everyone consolidates their notes and experiences. Then, all these notes are distilled into key insights,
53:12
which inform the designs in the next stage of the design process. Now that you've learned why you take notes, you'll learn how to take notes. There's no right way to do it. And everyone has a favorite note-taking method. Our usability study notes were messy, on one sheet of paper. You might also put each observation on a sticky note so your notes are easier to organize later. If you want to be even more organized, you can use a method called spreadsheet note taking, which
53:38
we'll cover in the next video.
53:44
Let's continue our conversation about taking notes during usability studies. The notes you take will help you understand the user's point of view and firsthand experience. A simple and easy method to keep your notes organized is called spreadsheet note taking. You can use this spreadsheet to take notes about the entire usability study in one place. So what is spreadsheet note taking? And how can it keep your notes organized during a usability study? Let's investigate.
54:11
Start by setting up the structure of your spreadsheet. Each participant from the usability study is listed in their own column across the top of the spreadsheet, starting in column B. Each row of the spreadsheet contains a behavior or observation in column A. Before the study begins, the team can fill out this observation column with a list of anticipated participant behaviors. For example, select the date and time to schedule a dog walker, which is
54:39
based on the first question we're asking participants. On the other hand, the team can start the study with an empty observation column and add to the list in real time when behaviors are observed. Even if you choose to start with a list of observations, it's OK to add additional ones during the study. If the team chooses to fill out the observation column in advance, it's helpful to organize it by categories. You can even add color or borders to separate each category.
55:07
When taking notes, you will match each participant to a corresponding observation. You'll enter a 1 in the cell if the observation is true for that participant. You'll leave the cell blank if it is false. OK. Now let's try to take some notes together. We'll rewatch part of participant B's recording and take notes as we go. First, let's make sure our note-taking spreadsheet is set up correctly. Looks like we have five participants across the top in the first row. Great.
55:37
Now let's check the observations column. Our research team decided to fill it out before the study begins. It looks like they even organized the observations into three categories-- observations about how easy or difficult it was for the participants to complete the assigned tasks, observations about participants' opinions on the product's usefulness, and observations about the participants' tone of voice or attitude while completing the tasks in the study.
56:03
They also left some extra rows where we can add additional observations. Now we're ready to take some notes. Let's start participant B's recording. SPEAKER 5: Prompt 1. Pick a date and time to schedule a dog walker. Note-- the product is only an example, so you will not be able to change the date or time on the screen.
56:27
OK, so to pick a dog walker, I don't really know if Dog Walkers Near You is how to schedule or if this Schedule rectangle thing is the way to go. I'm a bit confused. I'll try the Schedule thing. OK. Cool. It seems like that worked because the app went to the next screen. I'm going to assume if this was clickable, I'd see a calendar at this point. Or it looks like maybe these two rectangles are both boxes that I would type a date and time into. Interesting. One thing I'm thinking about for myself
57:00
and my own dogs is that it doesn't seem like there's a way to schedule a re-occurring dog walker, like if I want someone to come walk the dog every week at the same time. This scheduling page seems like it's only a one-time appointment for this one date and one time. Anyway, I'll go ahead and click the Submit thing, which I think is a button.
57:24
Prompt 1 follow-up. Did you find this task easy or difficult to complete? Is there anything you would change about the process of scheduling a dog walker? I mean, it was a bit hard to figure out which button to press on the home screen, but that's OK. That could be fixed, but I still figured it out. Also, I think there needs to be a way to do a re-occurring booking. Almost everyone I with a dog doesn't schedule individual appointments like this.
57:54
They want someone to walk their dog at the same time or on the same day every week, or something like that. SPEAKER 1: Let's pause here. Participant B just completed the first task, which tells us whether or not they can select a date and time to schedule a dog walker. Participant B said, "I'm going to assume if this was clickable, I'd see a calendar at this point. OK. Done." Based on this, I'm going to put a 1 in cell C2 to show that participant B was able to select a date and time
58:25
to schedule a dog walker. Participant B also said, "I don't really know if Dog Walkers Near You is how to schedule or if this Schedule A Walk thing is the way to go. I'm a bit confused." So I'm also going to add a 1 in cell C3 because participant B was confused by how to get to the screen to select a date and time to schedule a dog walker. This is an important part of the user flow. Let's start the recording again and make more notes about our observations. SPEAKER 5: Prompt 2.
58:58
Select a dog walker. OK. So it looks like there's a lot of options here. And I can scroll through even more dog walkers. Any of these work for me, so I'm going to click one of these rectangles that say Learn More.
59:15
Looks like I'm in the right place. Cool. This seems like more details about a dog walker named Jane Doe. Prompt 3. Confirm booking of dog walker and complete the checkout process. Got it. OK. I'm going to click Book right here. This one seems obvious since it's big at the top of the screen. Oh. That was super easy. Looks like I'm all set. I have this congratulations page, which I guess means my booking of a dog walker is all good. Prompt 3 follow-up.
59:51
Did you find this task easy or difficult to complete? Is there anything you would change? Wouldn't really change anything. That was pretty easy. I got it done quickly.
1:00:06
SPEAKER 1: OK. What did you observe? Was participant B able to choose a dog walker from the list? You might have noticed when participant B said, "Any of these work for me, so I'm just going to click one of these rectangles that said Learn More." I'd say they were able to select a dog walker, so I'm going to put a 1 in cell C4. Next, was participant B able to complete the checkout process? Participant B said, "Oh, that was super easy. Looks like I'm all set on that one," so I'll put a 1
1:00:37
in cell C6. Let's revisit the recording and keep building our notes. SPEAKER 5: Prompt 4. From the home page, figure out where you would go to edit your address. Got it. Well, let me see if I can go back here. I think I need to press this X to get out of this screen. Ah, yes, I'm back on the home screen of the app. Cool. So if I'm looking for a place to edit my address, it's probably here under the logo for a person. Usually, in other apps, there is a profile page
1:01:10
with all of your personal details like address, so I wonder if this app will be the same. Great. Yes. Found it. It says address right here. SPEAKER 1: OK. Was participant B able to find the user profile to edit their address? Participant B said, "Cool, so it's probably here under the logo for a person. Great. Found it." Participant B was able to easily find the user profile screen, so I'll put a 1 in cell C8. Back to the video one more time, where
1:01:43
we're going to look for some clues about participant B's tone of voice. Roll the tapes. SPEAKER 5: Prompt 5. Would you use the dog walking app? I don't know if that's useful, especially since you can't book a re-occurring time. I don't really know if I would just book a dog walker one date at a time. Seems like a lot of effort. To be honest, I don't think I'd really use it the way it's currently set up, but thanks for letting me try it out. SPEAKER 1: What did you notice about how participant
1:02:18
B answered this question? One thing they said was, "I don't know if it's that useful, especially since you can't book a recurring time." So I'll add a 1 in cell C11. There's one important part of the spreadsheet that we need to fill in. What did you notice about participant B's tone of voice and attitude? Which cell do we want to add a 1 to in this section? I observed that participant B sounded indifferent, so I'm going to put a 1 in cell C13. Hopefully, you're getting a feel for how
1:02:51
to take notes in a spreadsheet. You're off to a great start. After the study is completed and you've added all of your notes to the spreadsheet, it will be easy to identify the most common participant behaviors and the tasks participants found easy or difficult. And there you have it. You know how to use spreadsheet note taking during a usability study. Now it's your turn to practice this skill during your next activity. Good luck. DAVE: Congratulations on finishing this course
1:03:18
from the Google UX Design Certificate. You can access the full experience, including job search help, and start to earn your certificate by clicking on the icon or the link in the description below. Watch the next video in the course by clicking here, and subscribe to our channel for more from upcoming
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat