Defining user attitudes towards the ‘Profile’ feature within Handshake.com
Handshake is a job-searching platform geared towards college students (think: LinkedIn for students). Used by over 1,400 higher education institutions, Handshake provides students with the ability to connect with employers, attend virtual career fairs as well as to search for and apply to job postings.
Research goals
This project aimed to answer three main research questions tasked by our client company itself, namely:
1. What is the student's perspective of their profile? What is it used for and when? Why do they update their profile or not update their profile?
2. General feedback about the profile page - user priority of information, what's missing, what's extraneous?
3. Visibility & Privacy: Do students understand what employers, schools, and other students can see?
We aimed to give comprehensive answers to each of these questions, backed by qualitative and quantitative data obtained from user research. Our user testing aimed to evaluate the efficacy and usability of the Handshake site, specifically the profile feature. This is to determine whether there are any addressable user experience flaws and pain points during interactions with the profile feature of the Handshake site that would cause user confusion or dissatisfaction.
Process
-
The purpose of the competitive analysis was to evaluate the strengths and weaknesses of Handshake’s competitors and to identify potential opportunities for Handshake to outperform its competitors by creating a differentiated experience
-
We crafted a survey via Qualtrics to collect data on user attitudes and opinions on Handshake’s various features as well as its main competitors. This helped provide comprehensive quantitative data & qualitative insights which helped inform our plan of action later on.
-
In-depth interviews were conducted with users/stakeholders to further understand insights garnered during previous research steps. By doubling down on exploring trends observed during competitive analyses, we were able to understand the reasonings behind our survey results as well as uncover details we might not have been able to with a wider scoped survey.
-
We conducted moderated usability testing to determine any technical/design flaws that exist within the profile feature of the Handshake site. Participants were given tasks to complete and post-task measures such as the system usability survey (SUS) were administered to collect quantitative data, in addition to the qualitative information obtained from testing itself.
Competitive analysis - analogous experience
A competitive analysis was performed on platforms analogous to the Handshake experience, including direct competitors like LinkedIn and Indeed, as well as indirect competitors like Mint. The purpose of the competitive analysis was to evaluate the strengths and weaknesses of Handshake’s competitors and to identify potential opportunities for Handshake to outperform its competitors by creating a differentiated experience.
Competitive analysis - survey
We crafted a survey via Qualtrics to collect data on user attitudes and opinions on Handshake’s various features as well as its main competitors. This helped provide comprehensive quantitative data & qualitative insights which helped inform our plan of action later on.
With Qualtrics, we were able to implement display logic as needed as well as have it perform data analysis to obtain high-level insights. This helped us identify Handhsake’s strengths and areas of improvement in comparison to its competitors. Questions asked in the survey about Handshake’s profile feature also helped inform the user flow we wanted to test in our next steps.
Stakeholder/User interviews
Our next step was to conduct in-depth interviews with stakeholders, who were concurrently the target demographic of the Handshake platform - which would be students. We had eight participants in total, divided roughly in half by gender and education status (graduate/undergraduate) with a variety of academic years and majors, to ensure that we were obtaining as wide a range of user opinions as possible. Our participants all had online job-seeking experience, and at least basic familiarity to the Handshake platform along with other competitor sites.
The first section of our interviews consisted of more general questions pertaining to students’ job-hunting habits, as well as their approaches and/or any pain points they face.
We also wanted to dig further into insights gained during our previous research, namely that students were not widely using the Handshake profile feature as much as competitors’, like LinkedIn and Indeed.
We found that while students were well aware of the existence of the profile feature on Handshake, they lacked the motivation to use it as they did not see the value of doing so - bringing up the absence of a two-way connection on the platform between employers and students that other job platforms had. In addition, they found the process to be rather unintuitive and were not aware of many of its various features such as privacy/visibility controls etc.
Moderated usability testing
The goal for usability testing was to determine what technical/design flaws currently existed within the profile feature that might be affecting user behavior. We also wanted to drill down on areas of interest on the Handshake site identified during prior research steps.
To obtain comprehensive qualitative & quantitative user feedback, we administered post-task measures like the System Usability Survey (SUS) as well as probing verbal questions to find out the rationale behind their actions during user testing. We also recorded down metrics such as time-on-task, first click, success rates etc.
We conducted seven moderated online usability tests with three tasks across the testing protocol. Participants were prompted to think aloud in order to better understand their thought processes as they were completing the tasks. Tasks included having them access, update, and change various features (e.g. security and privacy levels) of their Handshake profile.
Analysis was conducted on both quantitative and qualitative data obtained during user testing and synthesized into a research report readout to stakeholders at the client company - the UX design team from Handshake.
Takeaways
This project demonstrated the importance of raising awareness about product features to users. If users are not adequately acquainted/motivated to use site features, they tend to avoid the product altogether in favor of competitors. People are creatures of habit and gravitate towards processes they are familiar with, which in this case was the social networking aspect of Handshake’s competitor sites, causing them to eschew Handshake in the process.
A challenge we faced was recruitment - it was challenging to recruit a balanced pool of participants without incentives. We realized that contrary to our assumptions, not very many college students utilize Handshake despite them being its primary target demographic. Because of this, we had to conduct our outreach in novel ways such as posting on online forums, sending out recruitment emails, in addition to convenience sampling.
Through this project, I learned about the importance of well-structured and suitably open-ended questions throughout interviews and user-testing. While questions that elicit more granular responses tend to be helpful in confirming or disproving assumptions, the most significant insights often come when participants are asked the general “what is your impression/perception of [product here].” The neutral standpoint that researchers take allow participants the comfort to express their honest thoughts about the product/interface in question, which may bring up areas of interest that previously slide by unnoticed.