About
During my time at Caribou I worked on several different parts of their consumer experience. I owned working on their login experience, different pages through out their refinance application flow, owned design updates and optimization to their offer generation page and partners and integrations, the first mvp pass at their customer dashboard, and some of the pages on their marketing site (caribou.com). For this case study I will be focusing on my work related to their Offers Generation page and one of the partner integrations.
THE PROBLEM
Caribou’s refinance application flow was outdated and a majority of the users who started an application needed a loan officer to help them with completing their application. Caribou’s flow needed updating so that it could scale for future improvements and A/B testing.
Very few users arrived at the offers generation page without loan officer support. Out of the users who did arrive on offers generation page, only 30% of users selected one of the offers presented to them. Many of Caribou’s users mentioned that the offers generation page was confusing and difficult to scan. We heard from our loan officers that they typically had to help walk a user through selecting an offer, and that there were a lot of questions from users that could be better addressed by this page. Caribou needed an offer generation page(s) that could support all of Caribou’s user types.
Goals
Business Goals
Increase user selection rate on the Offer Generation page
Develop an experience that can be customized based on the users source
User Goals
Improve user understanding by making the page easier to scan and read
Provide more clarity around terms and fees
Improve user confidence in selecting a loan offer
MY ROLE
I contributed to the beginning to end process of creating the new Offers Generation page and also contributed to making improvements to their partner integrations. I worked with multiple project mangers, a team of developers, and business stake holders on the consumer team. I owned the design process beginning with ideation, user and market research, design and prototyping, and A/B testing. Through out A/B testing we constantly made improvements to test a new variant against the control variant for 1 month test periods.
Competitive rESEARCH
While reviewing the industry, many of Caribou’s competitors refinance application flows did not have a true end to end digital experience. At the point that it was time to generate an offer, many of their competitors presented the user with a page informing them that a loan officer would be in touch or that they would need to download an app in order to view their available offers.
Heuristic Analysis
We need to better organize the content on this page to support the users goal in selecting an offer. Right now the visual design does not do a good job of visually seperating their existing loan from their new offers, this makes it hard to compare.
We could do a better job of explaining the different breakdown lines. For example, what kind of interest rate is this? Fixed or flexible?
We shouldn’t make a user have to recall which term button they selected by looking at the term field on the left, right now the buttons just update with a check icon to indicate which is selected. Can we find a different way to indicate the term length chosen?
Users have to scroll to compare their offers to their current loan.
Understanding our users
User Testing
Round 1 User Interviews
For our first pass at redesigning the offer generation page using Caribou’s new design system, I wanted to tests the offer generation page with both users and loan officers. Our loan officers have a wealth of knowledge about our users and the questions they have while choosing an offer during their refinance.
Round 2 User Interviews
I consolidated the overall feedback from round 1 interviews to find a few key insights and common feedback that I was able to use to update the page. I was curious to know if the updates would help to improve the overall feedback on the page. So I decided to run a second unmoderated test to get an idea at if the improvements were received well by users or if there were any more modifications that could be made for the MVP version of the offer generation page
Feedback we heard
A/B Testing
Improvements
After our first launch of the offer generation page we ran an A/B test on the Newly redesigned experience against the old version of the offer generation page. The page went well without any issues, and left us room to continue to expand upon the Offer Generation page and start pushing for larger improvements.
Control
Variant
Hypothesis
If we make basic UI/UX improvements to the Offers Selection page of the refinance flow, then conversion will increase because the consumers will find this page easier to navigate.
Duration
This test ran for 2-5 weeks.
Results
For our first release of the page it was kept to being a 1:1 “lift-and-shift” of the same experience with minor UI/UX improvements. Seeing that there were no issues or drop in conversion meant that we wear clear to continue ideating.
Partnership Integrations
Integrations
We saw that with the change in the offer generation page experience and flow, it was time to start making sure there were good updates made to our partnerships and how they entered the flow with Caribou. One of those partners was Credit Karma, and was a big resource for applications. The page that Credit Karma users were entering as a landing page before entering the flow had
a few issues.
There was no sense of next steps and communication where the user is at in the application process
It was too convuluted with content that wasn’t necessary
And it needed to be a place to introduce Caribou while being a clean hand off from Credit Karma
Variant
Control
Hypothesis
If we update and simplify the landing page for Credit Karma users coming through our new integration we will see an uplift in traffic whom select an offer.
Duration
This test ran for 3-5 weeks.
Results
We saw a 63% increase in users who visited this page proceeding to the offers generation page to select an offer.
This was a great success.
Personalized Offer Generation Page
Now that we have an updated Offer Generation page we need to personalize those offers depending on the source traffic. Credit Karma users were seeing the exact same offers page as our marketing and organic traffic. We needed to provide a personalized experience that made sure the user understood that they were looking at the same offer that they selected on Credit Karma. The old version took the user to the generic offer generation page and the offer they viewed in Credit Karma was hidden behind one of the flexible term buttons.
Variant
Control
Offers Generation Page - General
Round 2 A/B Test
For round 2 we wanted to took some of what we learned from the first A/B test and make improvements plus test a new theory.
Hypothesis
If we highlight the offer with a term closest to existing the users existing loan, then selection rates will increase because we are making it easier for users to find an option comparable to their current loan.
Duration
3 to 6 weeks
Updates
Users whom interact with the swipeable cards on mobile and 56% more like to select an offer. Only 49% of users whom visit the page on mobile swipe to see flexible terms. What can we do to get more users to view the flexible term card on mobile?
Test updating the term buttons to look more like they are an option to select on the card
Highlight Annual Percentage Rate
Updated and reorganized copy on page
User Testing
I was presented by stakeholders with an idea to test highlighting APR on the card, with the suggestion that the APR would be more important than the new monthly payment. The suggestion was to replace the monthly payment with the new APR, I was curious to know if this would resonate well with users, so I created 2 versions to tests.
I ran 4 unmoderated users tests on both versions. Version 1 the Annual Percentage Rate replaced the new monthly payment, versus version 2 gave Annual Percentage Rate a custom highlight separating it in the card. After running the test it came back that users whom saw version 1 were confused by seeing the title “Flexible Term” next to the APR and thought it was a “flexible/variable” rate instead of a “fixed” rate. This could cause a lot of confusion since all of Caribou’s terms are a fixed rate.
I presented this research results to Stakeholders with the 2 variations and the stakeholders voted to go with my suggested design.
Control
Variant
Results
This A/B test launched the same week of the layoff I experienced with Caribou. This is the most recent version of the Offers Page live in Production, so I am unable to discuss any specific details on final results.
Overall I spent a lot of exciting time brainstorming solutions and possible upcoming variations to A/B test, or to have user interviews about, with Stakeholders and PM’s at Caribou.
Interested to see more of my work?
Check out my other case studies