User Research for a Global B2B Financial Data Product
Finding missed opportunities to improve user engagement
The problem & project goals
The product team sought to increase the usage of the platform and by doing so address potential risks associated with product license renewals and revenues. Thus, the goal of the research project was to uncover opportunities (user needs, pain points, desires) that correlate with user engagement. Additionally, the project aimed to update the product’s user personas to build a shared and up-to-date understanding of the application's users.
My role
I planned and executed the study. I also managed and coached a team of designers and product managers who contributed to the research by interviewing users, taking notes, and doing the synthesis of findings.
About the product
The global financial data platform combines both proprietary data from issuing institutions (eg., retail banks) and data from one of the leading card payment scheme in a way that allows issuers to monitor portfolio performance and benchmark it against industry peers (eg., a bank in the US can benchmark its portfolio performance against aggregated portfolio performance of banks of similar size and profile).
Approach
At a glance:
Internal stakeholder (“looking in”) interviews
Usage data analysis
Customer interviews
Research methodology
I conducted interviews with internal teams including Delivery & Sales to gather context of platform clients before speaking to client users themselves. I reviewed usage data to sample participants that represented clients across varying levels of usage, geographies, and organization types. I then interviewed 11 client users to gather stories of their usage of the application as well as relevant context, e.g., what made them need to access the data on the platform, what other information sources they used, their stories about when they used the product last time.
Select clients
North America: Goldman Sachs, PayPal
Europe: Santander UK , BRD-Groupe Société Générale
Middle East: Halkbank
Asia Pacific: DBS Bank, Commonwealth Bank of Australia
User profiles
Marketing managers, Product managers & analysts, a data scientist, a payments partner/ consultant, Head of Card Payments
Impact
The opportunity space
Using the opportunity mapping framework , I lead a team exercise to visualize and prioritize the uncovered opportunities. This allowed the team to see the entire opportunity space, the relative priority of opportunities, and connect the existing roadmap initiatives to relevant opportunities.
Product personas
The research findings also helped update the product personas and ground them in facts and examples from the primary research. This helped build an understanding of the platform’s users among product, engineering, design team members as well as product leadership.
Head of Cards
Product Manager
Data Analyst
Lifecycle Manager
Operations Manager
Lessons learned
Recruiting research participants in the B2B setting is often a non-trivial task. I faced a few challenges in this project and below I go over how I solved them.
Participants’ self-selection bias
My goal was to draw participants across all levels of product usage (high, mid, low) to ensure that research results were representative of the various levels of user engagement. In the middle of my recruiting efforts, it became apparent that participants that agreed to get on a call with me were relatively more frequent users of the product than those who never got back to me.
How I addressed this issue
In lieu of speaking with low-usage clients directly, I spoke with Delivery team members that were close to the client and their unmet needs and pain points with the product.
I also requested that Delivery teams invite me to their calls with a client and asked to give me 10 mins at the end of the call to ask my questions.
Although this approach may not have gotten me the richest information on the user reality, I still were able to get at the opportunities with the low-usage users.
‘Selling’ user research to Sales & Delivery—the gate-keepers of client relationships
Recruiting in a B2B setting where Sales & Delivery own client relationships often means a hefty overhead of writing a ton of emails to convince said teams that true product development is only possible with an input from the end user and that they, too, would reap benefits from this connection.
How I addressed this issue
To save myself time, I created templates that explained the context of the project and the research need, as well as the ‘FAQs’ attachment to explain details for the curious team member and to make my emails short & sweet.
Researching at a global company means dealing with participants outside of your time zone
During this project, I had participants from Asia Pacific, Europe, Middle East, and North America. An obvious challenge is of course interviewing at a reasonable time for both the researcher and the participant (I’m on the West Coast in the US which means my 9 am is 7 pm in Turkey or Romania and 2 am in Australia).
The second challenge is merely communicating the options for an interview window— you can’t really peek into their calendar and grab an available slot, plus it’s so easy to get confused with time conversion.
How I addressed this issue
The first issue was fairly easy to address by arranging with a Singapore colleague to conduct the interviews when it was night for me. I addressed the scheduling issue by using two tools—Calendly and worldtimebuddy.com—which helped me avoid multiple back-and-forth and confusion with time zones.