Usability Testing & Final Prototype

INTRODUCTION 

The usability test for YoDream aimed to evaluate the app’s effectiveness, ease of use, and overall user satisfaction. The focus was to identify areas for improvement to enhance the user experience. This was achieved through the Think-Aloud Protocol and a survey technique.

Rationale of Usability Testing:

User-Centered Design: The purpose of this testing is to ensure our prototype meets user requirements after collecting information and feedback from users.

Identify Usability Issues: Through usability testing, participants were asked to perform specific tasks using the Think-Aloud Protocol and a USE questionnaire to observe whether they could complete the tasks successfully or encountered any problems.

Improve Effectiveness and Efficiency: We aimed to observe how easily participants could complete the tasks and identify any areas where efficiency could be improved.

Enhance User Satisfaction: Post-testing, we asked participants for their feedback using the USE questionnaire to understand their satisfaction with the app’s performance, including features like setting up neighborhood groups, inviting neighbors, posting updates, organizing meetings, and reviewing feedback.

Validate Design Choices: We also sought to determine if participants were satisfied with the interface design, assessing whether it was engaging and suitable for the app’s purpose.

Key Terms and Concepts:

Usability: The ease of use and learnability of a product.

Think-Aloud Protocol: A technique where participants verbalize their thought process while completing tasks, providing insights into their experience and challenges.

USE Questionnaire: A standardized survey designed to measure Usability, Satisfaction, and Ease of use.

In-Person Testing: A testing method where one or more observers evaluate participants in a controlled environment, guiding them through specific tasks.

Facilitator: The person who guides participants through the usability testing process.

Moderator: The person who conducts the usability test, leading participants through tasks and observing their behavior.

Participants: Individuals with similar backgrounds to the target user group.

Satisfaction: Participants’ comfort and positive attitude towards using the product.

Effectiveness: The accuracy and completeness with which participants achieve specified goals.

Efficiency: The resources (time) expended in relation to the accuracy and completeness with which participants achieve goals.

 

METHODS & PROCEDURES 

Participants:

User 1: Female, 21 

User 2: Male, 20 

User 3: Female, 22 

Main Tasks:

1. Create a new neighbourhoods group for safety update and community announcements within the app.

2. Use the app to invite neighbours to join the group via in-app invitations or SMS.

3. Post an update about a safety concern, local crime report or suspicious activity and community feed.

4. Schedule a virtual or in-person neighbourhoods watch meeting, send out invitations and track attendance.

5. Collect feedback from group members using the app’s feedback feature and review the responses. 

  Testing Metrics:

• Task Success: Measured by whether users could complete tasks without assistance.
• Time on Task: Recorded the time taken to complete each task.
• User Satisfaction: Collected feedback using the USE questionnaire.

Tools and Equipment:

• Smartphones: Used for testing the YoDream app and recording user interactions.
• Computer: Used for testing the YoDream app
• Google Forms: For collecting USE questionnaire responses.
• Figma: Used to run the clickable YoDream app prototype during the usability test.

Overall Procedures:

1. Briefing: Participants were briefed on the purpose of the usability test and the tasks they would be performing.
2. Task Performance: Participants performed each task while being observed, allowing for the identification of any challenges or issues.
3. Observation and Recording: Observers noted participant behavior and recorded interactions using an additional smartphone device.
4. Post-Test Survey: After completing the tasks, participants filled out the USE questionnaire to provide feedback on their experience.
5. Data Analysis: The data collected from observations and surveys were analyzed to identify usability issues and gather insights for improvement.

 

RESULTS

Task Completion Results:

Task 1: Set Up a Neighborhood Group for Safety Updates and Community Announcements – 90%

• User 1: Successfully completed the task without issues, demonstrating familiarity with the system.
• User 2: Did not attempt this task, due to lack of interest or not recognizing its relevance.
• User 3: Completed the task effectively, indicating they understood the feature but faced minor challenges.

Task 2: Invite Neighbors to Join via In-App Invitations or SMS – 85%

• User 1: Completed the task seamlessly, showing they had no trouble navigating the interface.
• User 2: Did not locate the button, they struggled with the interface and its design for this function.
• User 3: Initially struggled to find the button but managed to complete the task, indicating persistence despite usability issues.

Task 3: Post Updates About Safety Concerns, Local Crime Reports, or Suspicious Activities – 80%

• User 1: Found the button easily but had difficulty understanding how to use the feature, suggesting a gap in clarity or instruction.
• User 2: Did not attempt this function, possibly due to lack of understanding or interest.
• User 3: Faced significant challenges in both locating and comprehending the feature, highlighting potential design or instructional shortcomings.

Task 4: Organize Virtual or In-Person Neighborhood Watch Meetings and Track Attendance – 90%

• User 1: Completed the task effectively, showing they had no difficulty understanding or using this feature.
• User 2: Successfully completed the task but experienced difficulty understanding certain elements, possibly due to insufficient guidance.
• User 3: Could not find the attendance record, suggesting the feature's location or labeling needs improvement.

Task 5: Collect and Review Feedback from Group Members to Improve Communication and Engagement – 95%

• User 1: Completed the task successfully and efficiently, indicating strong familiarity and understanding of the function.
• User 2: Also completed the task without issues, showing good comprehension of the feature.
• User 3: Was unable to add members or view member information, pointing to usability issues with these specific functionalities.

Evidence: The recorded usability testing can be viewed at the following link: .

Time on Task: More experienced users were faster than less experienced users.

Observations & Feedback (Think-Aloud Protocol):

Task 1: Set Up a Neighborhood Group for Safety Updates and Community Announcements

• Observation: User 1 successfully completed the task, demonstrating familiarity with the system. User 2 did not attempt the task, possibly due to lack of interest or understanding of its relevance. User 3 successfully completed the task, indicating an understanding of the feature, although they encountered minor challenges.
• Feedback: Suggest a need to improve user understanding of the feature's value and to refine the instructions or the process itself for greater ease of use.

Task 2: Invite Neighbors to Join via In-App Invitations or SMS

• Observation: Some users struggled to understand how it works, even when they found it. Others didn't even try it, possibly because they didn't understand its purpose.
• Feedback: Add a short guide or pop-up highlighting this feature for first-time users.

Task 3: Post Updates About Safety Concerns, Local Crime Reports, or Suspicious Activities

• Observation: Some users successfully completed the task with ease, while others encountered challenges ranging from difficulty understanding the feature to locating specific elements.
• Feedback: Place the "Post Updates" button in a more intuitive location on the interface.

Task 4: Organize Virtual or In-Person Neighborhood Watch Meetings and Track Attendance

• Observation: Some users found the task easy, while others faced challenges. User 2 had trouble understanding parts of it, possibly because the instructions weren't clear enough. User 3 couldn't find the attendance record, suggesting the feature might be hidden or poorly labeled.
• Feedback: Users mentioned difficulty finding and viewing attendance records. Streamlining this feature by consolidating meeting details and attendance tracking into one location would improve accessibility.

Task 5: Collect and Review Feedback from Group Members to Improve Communication and Engagement

• Observation: Two users (User 1 and User 2) successfully completed the task, demonstrating good understanding of the feature. User 3 encountered usability issues specifically with adding and viewing member information.
• Feedback: Some users faced challenges with member management, such as adding or viewing members. Simplifying these processes and ensuring error messages are clear and actionable would enhance the experience.

Survey Results (USE Questionnaire):

Overall Experience: Users generally found the YoDream app effective and efficient. They appreciated its ability to help them meet their needs while being easy to learn and use.

Liked Most: The most appreciated features were its ease of use and user-friendly design. These qualities enabled users to quickly learn and efficiently utilize the product, leading to increased productivity and satisfaction.

Liked Least: Users expressed dissatisfaction with certain functionalities, such as inconsistency in achieving desired outcomes and the lack of steps optimization, which occasionally led to a less seamless experience.

Surprises: Users were pleasantly surprised by how quickly they became skillful with the product, highlighting its intuitive design and helpful features that enhanced their overall experience.

Frustrations: A significant source of frustration was the difficulty in performing tasks due to limited flexibility in navigation and occasional challenges in recalling how to use certain features.

Conclusion: While all users completed their tasks, various challenges were observed. Less experienced users struggled more with navigation, understanding, and locating features, leading to delays and incomplete functionality usage. More experienced users generally performed better, completing tasks faster and with fewer issues.

 IMPROVEMENTS/ACTIONS TAKEN

Task 1: Set Up a Neighborhood Group for Safety Updates and Community Announcements
Before:	After:
Users did not have clear instructions or guidance on how to create a neighborhood group, leading to confusion and lack of engagement.	Improvement: Added a pop-up toolkit tip with orange color to contrast against the turquoise background interface for new users. Action Taken: Placed the pop-up tip at the "create new group" symbol in the group message list and labeled it as "Create Group."
Principle Applied: Jakob Nielsen's 10 Usability Heuristics - Visibility of System Status: By adding the pop-up toolkit tip, we ensured that users are informed about the availability and function of the "Create Group" feature, enhancing the visibility of system status and guiding them more effectively through the app.

Task 2: Invite Neighbors to Join via In-App Invitations or SMS
Before:	After:
The share app symbol location was not easily discoverable by users, leading to confusion and underutilization of the feature.	Improvement: Moved the share app symbol to the top corner of the main page and added a toolkit tip to highlight this feature for first-time users. Action Taken: The new location makes the share app symbol more prominent and accessible, while the toolkit tip provides guidance to first-time users.
Principle Applied: Jakob Nielsen's 10 Usability Heuristics - Recognition Rather than Recall: By relocating the share app symbol and adding a toolkit tip, we reduce the cognitive load on users. They can easily recognize the share feature without having to remember its previous location, enhancing overall usability and user experience.

Task 3: Post Updates About Safety Concerns, Local Crime Reports, or Suspicious Activities
Before:	After:
The "Post Updates" button was located on the bottom navigation bar, similar to other social media apps like Instagram and TikTok, which some users found difficult to locate and understand.	Improvement: Added a new icon function for "Post Updates" on the menu search page's shortcut interface.  Action Taken: The additional icon makes the "Post Updates" feature more accessible and intuitive for users.
Principle Applied: Gestalt Principles - Law of Proximity: By placing the "Post Updates" icon in close proximity to other shortcut icons, it becomes more easily discoverable, enhancing users' ability to locate and use the feature effectively.

Task 4: Organize Virtual or In-Person Neighborhood Watch Meetings and Track Attendance
Before:	After:
After a user booked a meeting, the pop-up message said "Submitted! Meeting booked successfully." Users had to manage their booked meetings in the profile section, but this was not mentioned, leading to confusion.	Improvement: Added a red-colored text to alert users where to manage their meetings. Action Taken: Updated the pop-up message to "Submitted! Meeting booked successfully. You can manage your meetings in the 'Profile' section."
Principle Applied: Jakob Nielsen's 10 Usability Heuristics - Visibility of System Status & Shneiderman’s 8 Golden Rules - Informative Feedback: By providing clear, visible feedback with the updated message, users are immediately informed about the next steps, reducing confusion and enhancing the overall user experience.

Task 5: Collect and Review Feedback from Group Members to Improve Communication and Engagement
Before:	After:
Users had difficulty finding the "Add Member" button, as it was located at the bottom of the name list and scrolled with the parent content, leading to usability issues.	Improvement: Changed the "Add Member" button to a fixed position, making it constantly visible regardless of scrolling. Action Taken: Users can now easily locate and access the "Add Member" button without needing to scroll through the entire list
Principle Applied: Shneiderman’s 8 Golden Rules - Minimize User Memory Load & Gestalt Principles - Law of Continuity: By fixing the position of the "Add Member" button, we minimize the user's memory load and create a more continuous and predictable interaction flow, enhancing overall usability and efficiency.

CONCLUSION

Our usability testing for YoDream revealed significant areas for improvement, leading us to implement actionable changes based on user feedback and established HCI principles. By enhancing feature visibility, providing clear instructions, and simplifying processes, we improved the overall user experience and satisfaction. The final prototype now offers a more intuitive, efficient, and user-friendly platform. Watching the recorded usability testing on YouTube showcases our journey and the positive impact of these refinements. Overall, the usability testing has been crucial in transforming YoDream into a robust and user-centric app.