Shortlist Quiz
Helpful, interactive quiz that generates a shortlist of software
Role
Lead UX/UI Designer
Responsibilities
Perform competitor analysis, actively participate in design workshop, run usability tests, synthesize test results, create user flows, design high fidelity mock ups, hand off spec docs for development
Duration
2 Months
Problem
Users want a more personalized experience and didn’t see the immediate value in signing up for an account. Some users didn’t know what they should do next once they’ve signed in. Also from a business perspective, leadership wanted to improve conversion rates, increase revenue, and introduce a strong value proposition for User Workspace (logged-in experience of Capterra.com)
Users & Audience
Top of the funnel users who were starting off their software search. They typically have a very general idea of what they’re looking for but need guidance.
Solution
Surface various actions directly on the dashboard with a header that says “Not sure where to start?” so that users can determine their next steps right when they sign in. One of the options is a quiz that provides a custom shortlist of software based on your preferences.
Scope & Constraints
In the midst of a brand refresh and implementation of a new design system. Descoped any brand elements and heavily AI-related logic. Product manager wanted to keep MVP very light and get it out the door as soon as possible. Had to descope a few questions because we didn’t have the time to build in the back-end logic to accommodate those questions.
01 | Conduct user research
Research methods used: competitor analysis, moderated user interviews
User Persona
During moderated user interviews with real users who have an account with Capterra, we learned that users wanted to have a more personalized, spoon-fed experience. They expected Capterra to send emails with recommendations of products based off of their actions on Capterra. Many users signed up for an account because they were so used to signing up for things. Therefore, once they created an account, they weren’t sure what their next steps should be.
The UX researcher and I worked closely to do a competitor analysis to see how others were leveraging personalization to provide a better user experience.
02 | Workshop
During a workshop, all stakeholders (developers, product manager, scrum master, UX researcher) and I watched clips of the user interviews. We participated in affinity mapping to group similar findings and synthesize possible solutions. We landed on creating a fun, interactive quiz that’ll provide our users with a shortlist of software recommendations, based on their preferences.
03 | Low-lift Interactive prototype + Testing
After the workshop, the UX researcher and I worked closely to create the user flow and wireframes. Thanks to our design system, developers were able to quickly create an interactive prototype for testing purposes. Using the prototype, we ran unmoderated usability tests.
In our original user flow, we had users answer which category of software they were looking for first and then asked for other preferences, such as features, users, deployment options, and pricing options. However, through testing, we discovered that there were a handful of others who didn’t know exactly what software category they belonged in. For those users, the shortlist quiz wasn’t helpful and they felt like they’ve reached a dead-end.
In order to mitigate this, we decided to change the order of the quiz. Now, the quiz started with the “are you replacing software?” and “are you considering software?” questions. The answers to these questions will help the quiz determine which categories those software products belong in and surface those categories, rather than showing all the 700+ categories that we have on the site.
User Flow Chart
04 | Hi-Fi Mock-ups & Specs
Using Figma and our design system, I created all the high fidelity mocks and spec docs for development. In these mocks, I incorporated all the findings from our previous user research and revised user flow.
05 | scope check/mid-sprint check-in with developers
In order to proactively make sure that everything that we work on was in scope and within the realms of a true MVP, I suggested that we should have 30 minute “scope checks.” In these scope checks, we reviewed wireframes and user flows to make sure the back-end and front-end work were still within scope for the upcoming sprint. If not, the developers and I would’ve let the scrum master and product manager know that the timeline needed to be extended. Thankfully, because we’ve been communicating constantly throughout the design process, everything was still in scope.
06 | development
I attached all the mockups, spec documents, and prototypes to the Jira tickets. Then, I attended all the backlog grooming, prioritization/estimation, and sprint planning meetings to be available to answer questions. During the sprint, we had to change the design because we hit an unexpected hiccup due to back-end limitations. In those situations, I tried my best to be flexible to support the needs of the business and developers, without sacrificing the user experience. Whenever things were descoped, I created corresponding design debt Jira tickets to keep track of these changes.
Results
When users took the quiz, we saw a 7% increase in return visitors, 20% increase in conversions, and overall increased engagement with the product.
Lessons Learned
Although the shortlist quiz is helpful for users who have certain software products and category in mind, about 50% of the users abandoned the quiz because they weren’t able to identify a software category. I tried to address this problem in the Category Finder project. Additionally, we learned that we should work more closely with the email marketing team to announce the release of a new feature.
Next Steps
Our users want to easily see the pros and cons, pricing details, and contact information of the software. We plan on leveraging the information we have to surface more data on the results page. Also, currently the quiz is only available on a logged in user’s dashboard. However, we eventually want to move the quiz to a modal so that it can live anywhere on the site and be more helpful to more users. I’ve also negotiated with the product owner to make sure to include at least one design debt ticket every sprint.