May 2018 - Jan 2019
Classroom is Top Hat’s cornerstone product that helped catapult them towards product-market fit, high growth, and venture capital. Top Hat has displaced existing hardware clickers, in favour of using smartphones and web apps. To date, millions of students have used Top Hat in higher education institutions across the United States, Canada, and the UK. From the sound of it, everything in the product must be peachy, right?
When I joined in 2017, Classroom had already been built. The issue was that crucial parts of the user experience had been neglected in a mad scramble for growth. This lead to retention problems because the feature set had become complex, bloated, and basic expectations of usability weren’t being met. The same sad story has befell many products. We weren’t unique. We needed to get back to basics and focus on the core experience.
Top Hat Classroom is relatively simple. A professor asks students a question during their lecture, and students are prompted to answer it on their phone or laptop, while the results display live in class. Professors use this interaction to engage and assess students’ comprehension in real time, so they can pivot their teaching on-the-fly, and generate discussion in class.
Success for both users depends on a student’s ability to answer questions and receive grades for participating and getting correct answers.
As long as students are able to meet these basic expectations, both users are relatively happy, right? Well, if the hot water’s barely running in your hotel, and you’re not sure how to work the faucet, there’s still a problem. For us, our Classroom apps worked, but it was missing the mark on some basic expectations.
At the time, I worked on Top Hat Classroom leading design on iOS, Android, and two web teams at a ratio of 1 designer : 2 product managers : 14 developers. I lead product design from ideation to implementation.
For this specific project, I was focused primarily on iOS.Over 50% of student users submit their answers on iOS devices, so we wanted to drive the biggest impact by starting on iOS. Many of the iterative changes were adopted in the web app after being rolled out on iOS.
We were able to get laser sharp focus on the problem space thanks to our super secret weapon Jeremy, our user researcher. He helped connect us with professors for direct observation in classrooms, so we weren’t making decisions based on assumptions. My product manager Blanca was hyper focused on finding problems, and steered our ship in the right direction.
We started to attend classrooms and perform observational research. We’re lucky that for our use case, it’s relatively easy to sit in a classroom and observe our product in action. What’s more, is that there was a professor at University of Toronto who was using Top Hat only 10 minutes away from our office! There’s no better way to build empathy than to be a part of the experience.
One of the most common use cases is for professors to present slides, and ask questions throughout their lecture. They do this to check comprehension and pivot what they need to teach to students in real time. Internally, we called these comprehension checks checkpoints.
Another common use case is for professors to present a 4-5 questions quiz at the beginning of class, to check if students did their homework, and understand their level of comprehension. This activity helps them choose which topics they need to go over again in class.
To enable these types of tasks, students need to be able to answer questions on their device without issues. If there are any problems while asking questions, it disrupts the learning flow, and interrupts a professor’s classroom goals.
In the “happy path” scenario, professors open a question to gather responses from students. Students then open the question on their device, choose a response, and submit their answer. It’s important that their submission is received and grades are recorded.
While this flow might seem simple, what types of issues cause an “unhappy path?” How might we handle cases when there is low network connectivity? What happens when multiple questions are asked at once? How can we encourage students to respond to questions faster, so that we don’t waste valuable class time? How can we make things work the way students expect them to?
Someone, somewhere, probably in a Medium article, said “Good designers fall in love with the problem, not their solutions.” Our team’s firm grounding in research made it easy to discover problems. From the observational research, event tracking data, and design heuristics we were able to have productive discussions to prioritize which problems we would tackle together.
For students in the classroom we discovered the following needs:
Open and answer questions faster
General usability improvements
Technical problems caused reliability/perceived reliability issues
Better assurance of answer submission
This is the exciting part of design.
The first step was to start to align around language. Adam, another designer on my team, had lead a project on improving the professor experience in Classroom. We came up with a Product Dictionary the would help to define what language conventions we’d use in our designs.
We gathered common terms that we would use to describe actions inside of our product. For example, do you “send” a student’s answer, or do you “submit” a students answer? Aligning our language conventions kept our products in sync, and made the experience less confusing for users.
I worked on 3-4 different prototypes to help solve some of these issues. To generate ideas, I planned a sketching session with the mobile development team, and a separate session with the design team. I presented some of the problems we identified, we looked at how slides and questions work inside of the app currently. After sketching, I gathered these ideas to put together initial prototypes using Invision to create video clips I could start to send to colleagues on Slack to help start prioritization discussions.
Sometimes you need to dream big and look at the ideal situation, even if it won’t be fully implemented. This can help expand the realm of possibility, and help get better compromises that put greater focus on the user. If you think small, ideas can sometimes get squeezed and scoped down into nonexistence, and in the end the user suffers.
After discussing, prioritizing, and thinking pragmatically about what we could build, we settled upon a design that was still user focused. We prioritized improving the questions experience, since it’s the primary task students need to accomplish. The ideas we looked at around optimizing the slides experience were great, but viewing slides is a secondary task, and is used less on mobile due to the screen size.
A walkthrough of the questions UI flow View the Flow made in Overflow
A piece was still missing. We couldn’t just forget about slides. After all, 50% of the professors that use Classroom present slides in Top Hat. Though, we didn’t have the resources to rebuild how slides worked, so we found a simple compromise to reuse the existing slide carousel functionality. I believe that this compromise actually made the design simpler and clearer to understand for users, and still keeps questions at the forefront.
Another advantage to having access to classrooms, was that we could mitigate product risk by testing new features in classrooms. After building, we loaded a build of the improved classroom experience onto our QA test devices, and brought them to a classroom.
We took videos of students using our app live in a real classroom. This helped us to scoop up some bugs and interaction design issues, allowing us to be more confident to roll it out. From a qualitative standpoint, the design was a success.
To date, we have released it to 100% of users, and it has landed successfully, without any major issues. As the semester progresses, we’re learning more and will have more measurable data.
This is one piece of the retention puzzle. There were other parts of the product that we improved, as well as worked on other people focused strategies to improve retention. Though our current retention rate on Classroom is much higher, it’s hard to attribute it to only one action!