↫ Work

Weekly Customer Interviews

Establishing a continuous, lightweight research habit to broaden our customer knowledge

At TotallyMoney we often ran usability tests and interviews with a project focus, but rarely conducted broader research to learn more about our customers and find new opportunities.

When my squad made a commitment to listen to our customers every week, I built a habit of regular customer interviews, then developed it into a shared product design practice.

I learned (and proved to the business) that discovery research doesn’t have to be big, time consuming, and occasional.

How it started

The seed was planted when I saw Teresa Torres talking about continuous discovery at UX Fest 2021. She showed how you could spend only 30 minutes a week on a regular research habit, supplementing project-based research to give you a fuller understanding of your customers.

I was particularly taken with her concept of collecting “opportunities” – needs, pain points, and desires – as a basis for planning product work.

Not long afterwards, I facilitated a purpose, vision, and principles workshop with my new product squad. A dominant theme was seeking customer input more often, so we made a (bold!) commitment to listen to our customers every week.

Goals

Beyond seeking opportunities and giving my squadmates a direct line to customers, I had to show the business the initiative was fruitful and sustainable. So I also wrote up some success criteria. I wanted to see:

• Insights or evidence used in decision making
  If attendees used what they learned in their work, we’d be influencing the direction of our product.
• Attendance
  Our colleagues were always keen to join occasional workshops and interviews. To be effective we needed sustained interest.
• Novel insights or evidence
  We had a recent study which gave us a good overview of customer issues. Our findings needed to expand on that, not just reinforce it.

Launching and learning

A recruitment trade-off

We had two ways to recruit.

Our customer panel guaranteed interviews with genuine users, but recruiting and payment were manual, and no-shows were frequent enough to be disruptive.

Usertesting.com made recruitment easy, and attendance was great, but it was unusual to find genuine TotallyMoney customers on their panel. So while we’d still get insight into common credit issues, we’d get less insight into our own product.

I opted for usertesting.com as it let me keep weekly effort low by recruiting in batches, automatically handling drop-outs, payments, and reminders. Since we had only one interview a week, a no-show would be a bigger setback than normal, and frustrating our observers would be a quick way to kill attendance.

Given more resources, I would love to have developed our customer panel with some batch recruiting and reminder tools so we didn’t need to choose between effort and relevance.

Iterating and improving

I kept a weekly log, reflecting on the process to make continuous improvements.

Based on these reflections I made many changes, such as writing a guide to being a great observer, adjusting screening questions to better represent those less experienced with credit, and giving observers a channel to communicate with the facilitator.

From a solo effort to a team responsibility

After five or six weeks we were in the groove. My design colleagues in other squads were enthused, and wanted the interviews to touch on their areas of the product, too. I adapted the program so every product designer took part. Not only did this further reduce individual effort, making the initiative more likely to continue and succeed, it was a great way for everyone to build their research muscles and improve the process itself.

The product design team all had experience interviewing and facilitating, but there were a lot of specifics to learn, like setting up the observation board, calling for observers, and finding a helper. I managed the schedule, wrote a facilitation guide, and onboarded each new facilitator until everyone in the team could lead a session.

This neatly coincided with the arrival our new in-house user researcher. Handing over ownership to her was a big milestone and, now I’ve moved on, I look forward to hearing where she takes it!

Results and reflection

Looking back at the success criteria:

• I saw my colleagues from other departments bring up observations from interviews in product discussions.
• Over 30 individuals from across the business came along (almost a third of all staff) with a record of 11 observers in our biggest session.
• We uncovered novel insights, particularly around attitudes to joining the electoral register.

Each week we’d attract observers by sharing intriguing observations from the last session on Slack. These summaries were also the main record of what we had learned. Looking back, I’d seek to quantify how much we learned, to better demonstrate the final point above, and carry out weekly synthesis to improve the knowledge base.

I’m proud to have established this new practice, which became as crucial to TotallyMoney’s design culture as critiques and stand-ups.