Boy With A Ball UX Research Report

RESEARCH OVERVIEW

Introduction

ChadCo. Studio conducted User Experience Research on the Boy With A Ball website (boywithaball.com) in August of 2019. This presentation and research report represent the intersection of the organization’s needs and their key users’ expectations and desires. Below you will find a video overview of our research findings and recommendations, followed by a detailed outline of the research process.

Problem Statement

Boy With A Ball has unique and specific needs, and so do their supporters and volunteers. Not only do communities and residents need to become aware of their program, they need to be educated on the potential to join BWAB in effectively developing under-served youth around the world. Furthermore, they need to quickly understand their ability to take action and personally contribute to the organization’s key needs.

Research Question

Understanding the funnel of their platform’s user base will help determine design decisions that need to be made in order to best serve the BWAB organization. We decided to focus on the funnel of potential donors and volunteers, who may be able to extend the reach of Boy With a Ball’s platform, and increase the opportunity for impact around the world. Initial questions we chose to address were: Who are the core users of Boy With a Ball? How intuitive do they find the donation process? How easy is it for them to volunteer for the program? What would improve these experiences?

Justification

We view design as an ongoing and iterative process, not necessarily delivered and done. As such, we see the importance of including users throughout our entire effort. This includes the research, conception, design, development, and enhancement processes. Including users from the start, allows them to shape our next steps, and ensures that we are delivering value in every decision we make. This helps us create a truly user-centered product. Knowing who the Boy With a Ball users are, and examining their key goals and motivations allow us to build an experience that is right for them, increasing the help that the organization receives overall, and lengthening the reach of BWAB’s global impact. We have repeatedly seen that this type of research and insight are invaluable, and offer immediate return on your design and development investment.

RESEARCH METHODOLOGIES

Method 1: Stakeholder Interviews

Description:
Within these interviews, ChadCo. Studio conducted 1-on-1 discussions with Boy with a Ball (BWAB) leadership, directly addressing our research problem statement, with a focus on uncovering organizational goals and expectations. We also wanted to know what the leadership thought of their users.

Interview Protocol:
ChadCo. Studio hosted virtual interview sessions with Boy With A Ball team members and volunteers, allowing for ease of coordination without travel budget or complexity.

Each session was predetermined for an appropriate time, and conducted utilizing Google Hangout. The sessions were recorded and uploaded to YouTube for reference, and future viewing opportunities, by both researcher and the BWAB organization. Each session was expected to last around 30 minutes, with buffer for extended conversation where necessary.

Within each session, the moderator followed a series of predetermined questions utilizing the attached Stakeholder Interview Moderator’s Guide. In addition to the video recording, notes were taken by each moderator, highlighting key findings and surprising discoveries.

Analysis Protocol:
In order to effectively analyze the data collected from our stakeholder interviews, ChadCo. Studio utilized In Vivo Coding to derive like themes, and present a concise user story.

1. First we reviewed our stakeholder interview notes, and highlight like terms or phrases.

2. We clustered these terms or phrases to find any regular patterns that emerge.

3. These patterns were then gathered into categories and allow us to analyze connections.

4. We have presented these connections as a table, highlighting the most commonly recurring themes or categories. We also included a visual word cloud, allowing for quick understanding of the most common words or phrases.

To optimize the data collected during our stakeholder interviews, ChadCo. Studio implemented Affinity Mapping to derive key insights.

1. First we reviewed our stakeholder interview notes, and wrote down key observations. We searched for patterns, and organized them into groups.

2. Next we gathered them into themes by drawing comparisons between key observations. Themes describe a group of related observations and develop a foundation for insights.

3. Next, we drew insights from key themes and observations in/across each group. An insight is a summary of similar data points in a clear and concise statement.

4. Finally, we dot voted on the most important insights to guide us as we focused on creating the most successful experience for Boy With A Ball users.

Tools:
The data collection instruments we will use for this research method include:

• Video Recordings of Interviews
• Stakeholder Interview Moderator’s Guide
• Moderator Notes
• Charting Key Goals of Website

Method Two: In-Person Task Analysis & Follow-Up Interviews

Description:
ChadCo. Studio scheduled and moderated these in-person research sessions. 3-4 users were selected based on screening criteria, and invited to participate in this portion of the study. Observation of two predetermined tasks – Register to Volunteer and Submit a Donation – were coupled with contextual and post-task interview questions, allowing us to understand the deeper “whys” of a user’s actions and comprehension.

Interview Protocol:
We asked participants to complete two timed tasks, focusing on making a donation, and signing up to volunteer with the organization. We also asked participants a series of interview questions to gauge their reactions to the experiences they just completed.

The sessions were recorded and uploaded to YouTube for reference, and future viewing opportunities, by both researcher and the BWAB organization. Each session lasted roughly 30 minutes, with buffer for extended conversation where necessary. Task analysis and interviews took place at ChadCo. Studio’s Design Research and Usability Lab.

The researcher followed the attached predetermined script for Cognitive Task Analysis and Follow-Up Interview Questions, but may have needed to deviate from the script within the Task Analysis portion of the research, in order to appropriately ask or address any clarifying questions.

• Participants were be offered water or coffee to drink.
• Each participant was comfortably sat at a table, with a computer in front of them.
• The computer had a web browser open (Safari for consistency) with http://www.boywithaball.com open to the homepage.
• The researcher also sat in a position to observe tasks, take notes, ask follow up questions, and record the session with documented permission from the participant.
• The researcher took notes on items validating hypotheses, invalidating hypotheses, or are of interesting/surprising nature.

Analysis Protocol:
For the Task Analysis portion of the research, we timed users’ ability to complete the entire task from start to finish. This information was coupled in a table including analysis on the user’s ability to complete the requested task, and the time taken to finish the task-at-hand. Any pertinent notes have been paired to the appropriate task heading within the table.

To analyze data collected in the Follow-Up Interviews, ChadCo. Studio has again utilized In Vivo Coding to determine similar themes based on the user’s language.

1. First we reviewed our stakeholder interview notes, and highlight like terms or phrases.

2. We clustered these terms or phrases to find any regular patterns that emerge.

3. These patterns were then gathered into categories and allow us to analyze connections.

4. We have presented these connections as a table, highlighting the most commonly recurring themes or categories. We also included a visual word cloud, allowing for quick understanding of the most common words or phrases.

Tools:
The data collection instruments we utilized for this research method include:

• Video Recordings of Interviews
• Task Analysis & Follow-Up Interview Moderator’s Guide
• Moderator Notes
• Timing Notation for Charting Task Time on Graph
• Bar Chart for Displaying Difficulty of Task(s)

RECRUITMENT

Target User Profiles

For the research methods selected, we determined two appropriate user groups (in addition to the Boy With A Ball leadership) we recommend targeting within our recruiting processes.

The first being a younger group, mainly:

• College Students.
• Aged 17-24.
• Interested in Mentorship and or Community Service.
• Ideally have Served in the Last 12 Months.
• Ideally Avid Web or Mobile Users.

The second is a more mature group, mainly:

• Aged 45-65.
• Have Made Monetary Donations in the Last 12 Months.
• Interested in Mentorship and or Community Service.
• Ideally Avid Web or Mobile Users.

Contacting Potential Participants

Potential participants were gathered and selected from a combined pool of ChadCo. Studio and Boy with a Ball contacts and prospects. Potential participants were initially contacted by email and invited to take part in our research process.

Willing candidates were then screened according to the attached screening process, in order to provide the most value for the research being conducted and to ensure proper data is collected.

Eligible respondents (who passed screening questionnaire) were notified via email of their selection.

Respondents were required to agree to participating in two concurrent activities:

1. Task-Analysis: we will ask you to perform a certain task, while we observe and take notes. We may ask a follow-up question or two in order to make certain we understand your process.
2. Interview Questions: we will ask you a series of interview questions based on the tasks you recently completed.

Respondents had to also be available for these activities the week of July 22, 2019, and physically able to attend at ChadCo. Studio’s Design and Research Facility.

Research Session Logistics

We asked all participants to complete two timed tasks, focusing on making a donation, and signing up to volunteer with the organization. We then asked participants a series of interview questions to gauge their reactions to the experiences they just completed. These combined activities took roughly 30 minutes to complete. Task analysis and interviews took place at ChadCo. Studio’s Design Research and Usability Lab.

Each participant was comfortably sat at a table, with a computer in front of them. The participant was shown the homepage of boywithaball.com, and instructed to perform various tasks, while timed by the researcher. The researcher was also seated in a position to observe tasks, take notes, ask follow up questions, and record the session with documented permission from the participant.

Required materials for Research Sessions:

• Computer or Mobile Phone Connected to Internet Connected to boywithaball.com.
• Table.
• 2 Chairs.
• Video & Audio Recording Capability.
• Adequate Lighting.
• Bottled Water or Coffee.
• Notebook or Computer for Researcher to Take Notes.
• Interview Guide.

FINDINGS & RECOMMENDATIONS

Data Collection Methods

Research Method 1: Stakeholder Interviews
ChadCo. Studio met with local chapter and organizational leaders of the Boy With A Ball (BWAB) team to conduct lightly structured 1-on-1 discussions, covering questions that address our research problem statement and focused on uncovering organizational goals and expectations. We also asked these leaders their perspectives on their volunteers and donors, who are core partners alongside BWAB and a driving force behind the program’s effectiveness.

Data Collection Process:
In order to understand the discussions and help visualize the data, ChadCo. Studio utilized In Vivo Coding to break down the discussions into key themes and phrases. In order to complete this exercise, each of the recorded interviews was transcribed and divided into sections, by question, with both participants responses placed within a cell. Next, the responses were broken down into smaller themes, phrases or sentences. Then each sentence or phrase was deduced to a simple one to two word theme. This allowed ChadCo. Studio to quickly see themes emerging from the recorded data, and understand the most important issues facing the organization. (see fig. 1.1)

Visualization of Data:
The full coding document is included in the appendix, but to quickly orient to the data, ChadCo. Studio has created a table of the most frequently used codes that rose to the top. The table below shows the most recurring codes in prioritized order.

In-Vivo Coding (fig. 1.1)

Leadership Word Cloud (fig 1.2)

Figure 1.2 is a word cloud derived from the full-length leadership interview transcriptions. This visual allows us to see which specific words were repeated most frequently. Comparing fig. 1.1 to 1.2 allows us to cross-reference themes and provides a unique lens that helps focus our recommendations and design efforts.

Research Method 2: Task Analysis and Follow-Up Interviews
ChadCo. Studio moderated three in-person research sessions. These prospective users were selected based on strict screening criteria, and invited to participate in this portion of the study having no prior knowledge of the BWAB program or organization. Observation of two predetermined tasks – Register to Volunteer and Submit a Donation – was documented and coupled with contextual and post-task interview questions, allowing us to understand the deeper “whys” of a user’s actions and comprehension.

First, users were asked some simple questions to help open them up in responding to the additional research. Next, users were presented with the boywithaball.com homepage, and asked their initial thoughts while exploring the page. Secondly, users were asked to complete a simple task while their time was tracked. Finally, several follow-up questions were asked, to further understand their actions and thought process.

Data Collection Process:
In order to understand the discussions and help visualize the data, ChadCo. Studio again utilized In Vivo Coding to break down the discussions into key themes and phrases. To complete this exercise, each of the recorded interviews was transcribed and divided into sections, by question, with all participants responses placed within a cell. Next, the responses were broken down into smaller themes, phrases or sentences. Then each sentence or phrase was deduced to a simple one to two word theme. This allowed ChadCo. Studio to quickly see themes emerging from the recorded data, and understand the most important user experience issues. (see fig. 2.1)

Additionally, for the timed task analysis, success or failure was notated and each participant’s time to completion was recorded.

Visualization of Data:
The full coded document is available in the appendix, but the table below shows the most frequent codes for the combined user interviews. (see fig. 2.1)

In Vivo Coding (fig.2.1)

After coding the data, the interview transcripts were combined to create a word cloud from all utterances. This visual allows us to see which specific words were repeated most frequently and compare to the words mentioned by BWAB leadership in fig. 1.2.

Participant Word Cloud (fig. 2.2)

Finally, we took all five interviews and combined them to form a single word cloud in order to highlight any themes that rose to the top. This also allows us to view which words remain consistent throughout. (see fig. 2.3)

Combined Interviews Word Cloud (fig. 2.3)

Key Research Findings & Recommendations

Process for Determining Insights:
To optimize the data collected during our stakeholder and user interviews, ChadCo. Studio performed an exercise known as Affinity Mapping to derive our key insights, and created an informed action plan for our design recommendations. Below you will find a description and visualization of that process, and how we developed the items we are highlighting to you (fig. 3.1).

Affinity Mapping Overview (fig. 3.1)

From our affinity mapping, we have derived six key insights and corresponding recommendations that we believe would facilitate a better experience for BWAB donors and volunteers, while aligning to the goals outlined by BWAB leadership.

1. Ease of Use Does Not Equal Understanding: Through our task analysis and follow-up questioning, we found that users were very easily able to accomplish the tasks of registering to volunteer, and making a donation. Users were given a few moments to explore the site’s homepage, then asked to complete the task. After the task was completed, each user was questioned about the ease of the task, and whether or not the felt they had enough information to either volunteer or donate on their own. While all users were able to complete the task in under 45 seconds (fig. 4.1 and 4.2), and rated the difficulty of the task as a 1 – Not Difficult at All (see fig. 4.2), none of them felt that they had a full or desired understanding of the efforts comprising a volunteer opportunity or how their donation might be used.

Task Completion Time (fig. 4.1)

Task Difficulty (fig. 4.2)

2. Most People Are Willing to Give of Themselves, They Just Need to Know Why: All users questioned were willing to either volunteer in person, provide financially to a charitable organization, or both. After spending time on the BWAB homepage and completing the task analysis, all users said they did not feel they had the desired amount of information necessary to make a proper decision as to whether or not to give of their personal time, effort, or finances, to this specific organization. (see fig. 5.1)

Participant Preferences & Understanding (fig. 5.1)

It should also be noted that at this point, all users still felt that they were unable to clearly describe what the organization did, what their goals and intentions are, and how they utilize the funding provided via donation.

3. Well Written Copy Doesn’t Negate the Need for Precise Content in the Appropriate Hierarchy: Users were repeatedly engaged by headlines and compelling copy throughout the homepage. Many users mentioned the word “Hero” used in the header of the page, calling it “strong,” “empowering,” and “intimidating.” There were other specific mentions of enjoying the content, and their desire to “dig deeper.” However, when it came time to describing what the organization does and who it serves, all users were left at a loss. Each was confused by inconsistent imagery or misaligned verbiage, and felt there was an overall lack of the information they desired, in the places they expected to find it. More information about the “what” and “how” of Boy With A Ball, was desired at a higher level of the homepage, and “high-level summaries” of these items were called out as the appropriate level of information. Additionally, while links for Donate and Volunteer are highlighted prominently within the mobile environment of the web page, the desktop experience requires users to scroll down before being able to observe these actions. (Called out specifically by participant #1 during the interview sessions.) Furthermore, all users were derailed by the multiple entry points leading to similar outcomes for Volunteering and Donation. ChadCo. Studio would recommend repurposing some of the redundant sections to highlight additional detail about the organization, and clean up the overall information architecture and navigation of the site.

4. Users Desire Stories, Not Statistics, Through Engaging Communication: As mentioned above, all users specifically noted the quality of the writing throughout the homepage. What was lacking for them was a sense of understanding, a clearly communicated vision for the organization, a bit about its background, and some stories from the field. This also aligns directly to the responses provided by the BWAB leadership, in their desire to tell compelling stories. Several users mentioned that imagery would be helpful in tying stories to real people. Furthermore, it was mentioned that calling out the specifics about how BWAB helped in these situations (without having to read the entire story) would create deeper understanding of the organization’s purpose, within a storytelling framework. ChadCo. Studio recommends the inclusion of story-specific imagery, paired with carefully curated copy, highlighting the key strengths of BWAB, paired with their desire to tell meaningful and engaging stories.

5. In Context Information Makes It Easier for Users to Partner: Because each of the participants echoed their lack of understanding and desire for additional and specific information, we recommend updating the key experiences of Registering to Volunteer and Making a Donation to include detailed information about these opportunities.

Within the Volunteer Registration Form, include a checklist of the types of opportunities a user might want to sign up for, or include specific dates that they might register to participate. While date specific opportunities might take a bit of extra site maintenance, it eliminates some of the potential back-and-forth correspondence that the organization has to follow-up on.

On the Donation form page, it is recommended that BWAB include specific information about how the organization will be using the money. This increases transparency and builds trust for those considering making financial contributions to the organization. Additionally, including an opportunity for donors to select a specific chapter of the organization, where they would like for their money to be distributed, provides an additional prompt for local-focused charitable givers.

6. Location, Location, Location: While we’ve already touched on the importance of specific locations pertaining to copy and content, another confusing element for users, seemed to be a lack of understanding of the specific locations of the program. Where is this organization based? Where do they perform their activities? Is there a chapter in my city? Not knowing the answers to these types of questions left users unaware of opportunities for them to participate. One participant noted that because of the imagery and content, they thought the program was focused solely on “third-world” or “non-American” locations, a clear misperception and departure from the intended goals of the website. For these reasons, ChadCo. Studio recommends displaying a visual map of BWAB’s local and global reach. A visual of this sort helps to quickly ground users to the vast expanse of the program’s impact.

In addition to these locations, a third location that was discussed was where users might expect to learn that a program such as this one exists within their community. Two of the three users mentioned Facebook Ads and Events as specific ways they’ve seen and engaged with other non-profit programs in the past. A third participant mentioned partnerships with other organizations as a way to increase knowledge of their existence. Because of these two factors, ChadCo. Studio would recommend leveraging social media and strategic partnerships for advertising and exposure opportunities. As an example, during our interviews, one leader mentioned that BWAB is listed as an “evidence based cross-age mentorship program” by the Department of Justice. There may be opportunities to highlight this information within the site, to help build the trust desired by potential partners and participants.

RESEARCH DISCLAIMER

About Qualitative Research

The data collected and presented is qualitative in nature and is not statistically projectable across the Boy With A Ball volunteer or donor population. Findings should be considered directional and indicative of user attitudes, perceptions and motivations. Business decisions requiring statistical validation should be supported by supplemental quantitative research; although findings of this research were consistent across participants, and with projected hypotheses.

APPENDIX

Boy With A Ball Research Screener (PDF)
Stakeholder Interview Moderator’s Guide (PDF)
Participant Interview Moderator’s Guide (PDF)
Moderator’s Toolkit for Analysis & Follow-Up (PDF)
In-Vivo Coding Annotations (PDF)