Project Overview
The Challenge
User analytics showed an approx. 25% of new users (teachers) only logged in once to our Ed Tech product. A significant lack of new user adoption. After testing with the existing product, I found that new users did not recognize the product’s purpose and struggled to navigate through the library — which is our key value!
The Solution
To solve for these problems, I partnered with a designer and we created a series of small interceptions at critical moments in the user flow to support UX best practices (ex: visible feedback). We then refined these solutions by testing the designs using RITE methodology.
The Outcome
The result for the initial release of some of these changes was a decrease of 5.5% for new users only logging into the product once, and an increase of 8.3% in new users logging in at greater frequency benchmarks. While there are still a handful of changes to be released and measured, the product team now holds 6 new UX advocates.
My Role
User Researcher
Recruited participants
Scoped + curated study logistics including scripts, success criteria, and prototype flows needed
Session Moderator
Note Taker
Data Analyst
Presented research findings
Documented current user flows
Sketched future user flows
Led team workshop for empathy mapping
Process Overview
Discovery Research (user test current experience) ➔ Ideate ➔ Test New Designs ➔ Measure
Methods Used
Step 1: Research
BUSINESS PERSPECTIVE
Using Pendo Analytics, we found that 25% of new users were not returning to the product after their initial login.
This is a problem because if teachers are not using the product, their school administration is less likely to renew the school’s product subscription. Therefore, lack of logins represents lost revenue.
USER PERSPECTIVE
I recruited and tested the existing product with 7 new users in our target demographic (teachers 2nd - 6th grades) to see where they went in the product and listened to their understandings.
All 7 teachers took a unique paths through the product, not discovering the library until 30 minutes into the session - our core value!
KEY DISCOVERY RESEARCH FINDINGS
Teachers struggled to understand the purpose of the app (it’s a digital library)
Teachers were unable to complete expected tasks in the library such as finding a book. (One key reason being a lack of appropriate system feedback)
Teachers felt uncomfortable with the idea of using the app with students unless they could experience the student view when needed
Step 2: Ideate & Design
EMPATHY MAPPING
After sharing the findings from discovery research, the team participated in an empathy mapping session to align on the problem statement and prioritize what to focus on in the design solutions.
Our problem statement:
“As a teacher using [Product] for the first time, I want to know the value [Product] has for me and the easiest way to get students started reading books they will love”
Our agreed upon design principles:
Drive immediate recognition of product value
Provide a clear pathway through the product
Support a student-view of the product for teachers
WIREFRAME
Though a researcher, I was able to partner with a designer to sketch through ideas for possible solutions. These sketches were then brought into Figma by the designer.
Product Leaders had seen other products with onboarding tutorials and were convinced this would be the solution. In a show of good faith we mocked up a few different onboarding concepts to test with new users.
While onboarding could be one solution, we also mocked up some solutions for the other pain points found in user testing - including the need to build a simplified funnel into the library (previously required grade selection and interest survey) and a way to provide a view of the student experience.
Step 3: Test & Iterate
USER TESTING
Using the RITE Method - we logged each participant’s success in the following tasks. After each couple of participants, we iterated the designs - usually simplifying until the later rounds of participants succeeded more consistently. (note: “Onboarding Tutorials” were removed altogether with Participants 10 & 11)
WIREFRAMING ITERATIONS
Through these iterations, we simplified the landing page to a banner - less disruptive to our substantial existing user base. And we simplified the messaging to a single call to action - get teachers to the books.
KEY RESEARCH FINDINGS
Teachers did not want tutorials or onboarding steps. They perceived the product as complicated if needing to train on it and shared experiences of tutorials in other products as just more advertisements about the product and not step-by-step help - so they had learned to avoid these.
Teachers were now able to find the books they were interested in - but they were asking, “what now?” - our funnel had dead-ended. So we iterated and completed the flow with a “Share to Students” action.
Our initial label of “Student View” to access the student experience gave teachers the impression that there would be only one student login to share and expressed concern over privacy needs for students. So we relabeled to “Student Demo” and found that better matched teacher mental models.
Step 4: Measure
After releasing three of the changes: the landing page banner, the “Share Book” action and a “Student Demo” view - we saw some changes to the login frequency of our new users. The downside, however, in attempting to compare measurements with education apps is the difference in alignment between the launch of features with the school year calendar. Many new users visit at the beginning of a school year when there is time to consider new processes and programs. These releases went out over the Thanksgiving holiday. Regardless, the following changes were noted:
USER ANALYTICS
Decrease of 5.62% in single-time logins
Increase of ~8.39% of new user logins across high frequency logins
Increase of 6 new product team advocates for the UX process and design principles
What I Learned
➔ Involving the product team early in the process allows for better and quicker collaboration in problem solving and less friction in moving designs forward into the product build
➔ Allowing others ideas for solutions to be tested with users is worth building the personal bridges (in this case the proposed solution of using onboarding tutorials)
➔ A good reminder of designer Dieter Rams’ principle: “Good design is unobtrusive” or rather ‘invisible design’ can hold great impact on users’ experiences