Bringing a one of a kind promotional tool to market
Discovery Mode, available on Spotify for Artists, allows artists and labels to identify priority tracks and include them in a promotional campaign. This additional signal is added to the algorithms of personalized playlists increasing the likelihood of the selected tracks to be recommended.
User Research played a critical part in bringing Discovery Mode to market followed by identifying key improvements to the user experience.
Pre-launch
Onboard 5 teams at independent labels into a free Discovery Mode pilot to learn from their experience and convert them into paying customers. At this stage Discovery Mode is only available in a “managed service” model using a set of spreadsheets for selecting tracks and a limited set of reporting metrics at the end of the pilot. As the lead UXR I met with label teams before and after the pilot to gather insights regarding their perceptions, questions and sentiment toward the product.
Launch
As part of the go-to-market plan artist and label distributors were identified a the largest untapped opportunity to scale Discovery Mode. However, self-serve tooling was unavailable and was required to meet revenue goals. As the lead UXR, I was responsible for mapping out the needs of artist and label distributors and testing a self-serve booking and reporting flow which would be available on Spotify for Artists.
Post Launch
Insights from a the self-serve booking and reporting pilot with artist and label distributors outlined two core needs: education and recommendations for which tracks to include in a Discovery Mode campaign. My role was to work closely with our design team to create a prototype that met the needs of our artists and labels followed by iterative testing. I also worked closely with data science to track behaviors post launch.
The impact of key research insights included business, product and design. By including users in every step of the product lifecycle, Discovery Mode was able to scale and improve effectively.
Business
In 2023, as the lead user researcher, I worked cross-functionally to overcome critical barriers, serving to unlock the self-serve channel strategy for Discovery Mode. These self-serve tools were key to scaling Discovery Mode to ~1M artists in H1 2023 and achieving a gross profit of ~€26M in gross profit in 2023.
Product
Through the course of a year’s long multi-phase research endeavor, I identified user needs that led to key product improvements. This included an auto-renewal feature which during an A/B experiment increased MoM retention by 19pp and an increase in campaigns created, generating 111k GP.
Design
As a one of a kind promotional tool, Discovery Mode faced challenges in user comprehension. I identified key areas in the UX where comprehension was low which led to design and copy improvements including the addition of pre-campaign insights which provide users with recommendations as to which tracks they should opt-in.
Pre-Launch
〰️
Pre-Launch 〰️
Pilot Goals
Onboard indie labels into a free Discovery Mode pilot to learn from their experience and convert them into paying customers.
Gather perceptions around value of Discovery Mode and its performance reporting.
Key Insights
Without clear and transparent education, Discovery Mode and how it’s paid for can be highly misinterpreted. This, in turn, exacerbates skepticism around the royalty payment model.
Pilot participants were unsure whether they should expect short term stream lift or long term audience growth.
Research Impact
Positioning
Go to market plan
Launch
〰️
Launch 〰️
Self-Serve Experience
Buying
Enable artist-level campaign creation in Spotify for Artists (S4A)
Reporting
Enable artist-level campaign performance results in Spotify for Artists (S4A)
Launch Self Serve
End To End Self-Serve Experience
Set up monthly campaign by choosing tracks to opt-in
Review campaign choices
View performance report with metrics including stream lift and new listeners
Launch Self Serve
Research Questions
Can users successfully set up campaigns via self-serve without manual intervention?
How confident are users when making choices about what tracks to include in a campaign?
How does performance reporting help users to understand the success of their campaign choices?
Overall, what pain points do users find in the end-to-end self-serve experience?
How satisfied are users with the end-to-end experience
Research Methods
60-minute qualitative interviews with 12 users (6 artists, 6 labels)
In-app CSAT survey to track sentiment at the end of the buying flow and within reporting
Triangulation of survey and qualitative data with UBI (user behavior interactions) data.
Launch Self Serve
First time customers had an 87% completion rate while 98% of returning customers completed.
Without guidance and actionable insights, users felt unsure about what to opt-into their campaigns and how to be strategic.
Users expressed a need for flexible campaigns including the option to keep tacks auto-opted in.
Users were doing their own calculations and creating work-arounds to uncover campaign insights.
The UX was considered "straightforward", "smooth", and "easy", with drop offs being attributed to missed reminders and deadlines.
“…as far as understanding why that song is now eligible, or even showing kind of like the stats as to why this one will be considered eligible.”
Launch Self Serve
By not providing context through insights as to why these songs are eligible users are left unsure as to which tracks to select.
“…maybe even just like statistics or numbers, because I think that, you know, especially if anybody is paying attention to this stuff down here [previous campaign], then they probably care a little bit about numbers. And so seeing even like, we feel like this song could be doing x amount better…
”
Launch Self Serve
Users expressed the desire to automatically keep certain tracks opted-in past a month.
“…it’s interesting to see which songs are like doing well and which ones aren’t. So I suppose in terms of being impatient, I love it to be shorter. But for the tracks that I know or the tracks that I know that I want to push. I’d happily just leave them running…
”
Post Launch
〰️
Post Launch 〰️
Key Insight
Without guidance and actionable insights, users felt unsure about what to opt-into their campaigns and how to be strategic.
Users expressed a need for flexible campaigns including the option to keep tacks auto-opted in.
Feature
Pre-campaign insights a.k.a. recommendations for which tracks to opt-in.
Auto renewal allowing users to set a period of time for tracks to be continuously opted in month to month.
Post-launch improvements
Pre Campaign Insights Experiment
Problem Space
Without guidance and actionable insights about which tracks to opt-in users were left unsure how to optimize their campaigns
Pre-campaign insights (PCI) is an ML based product within Discovery Mode, which equips users with insights, tools, and intuition to build successful campaign strategies. Goals include
Increase demand through more opt-in of ROI positive tracks per customer
Drive higher customer retention
A/B Experiment
The experiment was conduced with ~6200 artists that were split almost evenly between the control and PCI treatment groups
Key questions included:
Did customers accept PCI recommendations
Did PCI help drive better value for the customer at the end of the campaign? We define “better value” as a higher proportion of opted-in tracks being ROI positive.
Results
Artists opted in less of the non-PCI recommended songs. Artists in the treatment group opted 6 pp less of their non-PCIr tracks.
PCI helped drive better value for the customers at the end of the campaign. Artists in the treatment group had a 2 pp increase in the percentage of ROI positive tracks that were opted in.
Post-launch improvements
Auto-Renewal Experiment
Problem Space
In the absence of a renewal feature, Discovery Mode customers have to come back to the S4A every month to set up their campaigns
Customers often forget to create campaigns during the set up window, thus leading to lower retention rates, customer dissatisfaction & lost revenue.
The campaign renewal feature should help customers conveniently run campaigns longer without the need to set up a campaign each month
A/B Experiment
The auto renew feature was rolled out to users in select English-speaking markets.
Treatment (50%): got the new renewal feature in S4A
Control (50%): old experience without renewals
Key Metric for sucess:
MoM Retention Rate
A significant increase (5%+ abs.) would indicate success.
Results
Users in treatment group saw a +19pp increase in MoM retention rate
They averaged ~90% retention from one month to another to vs. ~70% retention rate in Control group
The results were statistically significant both for New and Returning customers
Was rolled out to 100% of customers and at the end of the experiment had a forecasted monthly revenue increase of ~$111k GP.