Establishing a continuous, lightweight research habit to broaden our customer knowledge
At TotallyMoney we often ran usability tests and interviews with a project focus, but rarely conducted broader research to learn more about our customers and find new opportunities.
When my squad made a commitment to listen to our customers every week, I built a habit of regular customer interviews, then developed it into a shared product design practice.
I learned (and proved to the business) that discovery research doesn’t have to be big, time consuming, and occasional.
- Making it happen!
- Interview design and facilitation
- Screening and scheduling participants
- Coaching facilitators
- Wrangling observers and observations
- Over 15 weekly interviews (and counting!)
- More than ⅓ of the company attended an interview
- Now an established ritual for the team
How it started
The seed was planted when I saw Teresa Torres talking about continuous discovery at UX Fest 2021. She showed how you could spend only 30 minutes a week on a regular research habit, supplementing project-based research to give you a fuller understanding of your customers.
I was particularly taken with her concept of collecting “opportunities” – needs, pain points, and desires – as a basis for planning product work.
Not long afterwards, I facilitated a purpose, vision, and principles workshop with my new product squad. A dominant theme was seeking customer input more often, so we made a (bold!) commitment to listen to our customers every week.
Beyond seeking opportunities and giving my squadmates a direct line to customers, I had to show the business the initiative was fruitful and sustainable. So I also wrote up some success criteria. I wanted to see:
- Insights or evidence used in decision making. If attendees used what they learned in their work, we’d be influencing the direction of our product.
- Attendance. Our colleagues were always keen to join occasional workshops and interviews. To be effective we needed sustained interest.
- Novel insights or evidence. We had a recent study which gave us a good overview of customer issues. Our findings needed to expand on that, not just reinforce it.
A recruitment trade-off
We had two ways to recruit.
Our customer panel guaranteed interviews with genuine users, but recruiting and payment were manual, and no-shows were frequent enough to be disruptive.
Usertesting.com made recruitment easy, and attendance was great, but it was unusual to find genuine TotallyMoney customers on their panel. So while we’d still get insight into common credit issues, we’d get less insight into our own product.
I opted for usertesting.com as it let me keep weekly effort low by recruiting in batches, automatically handling drop-outs, payments, and reminders. The impact of no-shows from our panel would be worsened when only interviewing once a week, and frustrating our observers would be a quick way to kill attendance.
Given more resources, I would love to have developed our customer panel with some batch recruiting and reminder tools, so we didn’t need to choose between effort and relevance.
Iterating and improving
I kept a weekly log, reflecting on the process to make continuous improvements.
Based on these reflections I made many changes, such as writing a guide to being a great observer, adjusting screening questions to better represent those less experienced with credit, and giving observers a channel to communicate with the facilitator.
From a solo effort to a team responsibility
After five or six weeks we were in the groove. My design colleagues in other squads were enthused, and wanted the interviews to touch on their areas of the product, too. I adapted the program so every product designer took part. Not only did this further reduce individual effort, making the initiative more likely to continue and succeed, it was a great way for everyone to build their research muscles and improve the process itself.
The product design team all had experience interviewing and facilitating, but there were a lot of specifics to learn, like setting up the observation board, calling for observers, and finding a helper to manage them. I managed the schedule, wrote a facilitation guide, and onboarded each new facilitator until everyone in the team could lead a session.
This neatly coincided with the arrival our new in-house user researcher. Handing over ownership to her was a big milestone and, now I’ve moved on, I look forward to hearing where she takes it!
Looking back at the success criteria:
- I saw my colleagues from other departments bring up observations from interviews in product discussions.
- Over 30 individuals from across the business came along (almost a third of all staff) with a record of 11 observers in our biggest session.
- We uncovered novel insights, particularly around attitudes to joining the electoral register. Each week we’d share intriguing observations from the last session to attract observers into the next session, which served as our main repository. Looking back, I’d seek to quantify how much we learned, and do weekly synthesis to build an easy way to reference opportunities.
I’m proud to have established a brand new practice, as crucial to TotallyMoney’s design culture as critiques and stand-ups.