Obvious statement alert: The quality of a player’s experience affects retention. Nothing new or interesting here, and something I think we would all agree with. However, what IS interesting is the answer to this question:
How MUCH do changes in player experience impact retention?
For first time in youth soccer history, we can answer that question. Doing so not only requires measuring retention, it also requires the ability to connect a parent’s survey response to their future decision to return. When clubs run their own anonymous survey, it is impossible to make this connection. When a club works with Satori, we protect parent confidentiality while also being able to connect the dots to enable ground-breaking research.
The Study: NPS relationship with Retention
We conducted our survey using a data set of more than 4,400 parent survey responses in Spring 2023. It includes competitive players ranging from U7-U13 in surveys with more than a 50% response rate. That response rate is critical because without a high response rate, you can’t trust the results to be truly representative. From there, we tracked their participation into the Fall 2023 season to see who returned.
Measuring retention is simple enough: what percentage of the people who played last season are also playing this season? Measuring player experience, however, is a challenge. While a player can wear a GPS device to accurately measure movement, there is no sensor we can attach to 12 year old central midfielder that tells us how much joy, improvement, and belonging she feels.
Due to this challenge, we may never know the exact quality of a player’s experience. However, we do have access to a tool that, when used correctly, can give us a very useful estimate: parent surveys. A point in time parent survey result will never perfectly match true, continuous, internal player experience. However, the fact that our Satori survey results are highly correlated with retention gives us confidence that using survey results is a lot better than not using them.
The final decision is to decide which single survey metric to compare against retention. While all of the questions in our Satori surveys have a significant positive correlation with a player’s decision to return, the most predictive question is Club Net Promoter Score (NPS). This asks the question, “On a scale of 0 to 10 (10 = extremely likely, 0 = not likely at all), how likely are you to refer Example Soccer Club to a friend or colleague?”.
Results and Insights
Now that we’ve tracked retention for more than 4,400 players, and know their NPS from the previous season, let’s chart the results.
It’s pretty cool how a chart with only a single line can have so many insights. For example:
- First, there is a clear and strong connection between Club NPS and Retention. Other than NPS values of 0 and 1 being nearly identical, there is a noticeable increase in retention for each increase in NPS value. On average, this amounts to a 6.0% increase for each single bump in NPS value.
- Improvement in retention is more significant at the lower scores than at higher NPS values. For example, between the NPS values of 1 and 6 there is a 41.4 percentage point increase. This amounts to an 8.3% average increase in retention for each bump in NPS value. This is especially interesting when you consider everything 6 or below is a “detractor” in NPS terminology. While it might make sense to group 0-6 responses in some regards, it obviously doesn’t make sense if you’re trying to predict who will return to your club next season.
- The highest single jump in retention happens from NPS values 1 to 2. The increase here is 11.3%! The next highest is from 2-3 where the difference is 10.2%.
- On the higher end of the scale, there is still a clear benefit from a higher NPS value but the effect size is lower. For example, from NPS values 7-10, the average retention increase is 3.7%. That 3.7% still matters when you’re talking about clubs with hundreds or thousands of players, but it’s obviously not as dramatic as at the lower end of the NPS scale.
Key Takeaway
While there are many valuable things to be learned here, what resonates most with me at the moment is:
Serving those that are justifiably unhappy might be the best use of your time if you want to increase retention
A key phrases in my statement above is “justifiably unhappy”. As we all know, trying to help someone who will forever be upset may accomplish nothing, or worse. However, helping good people with legitimate frustrations will likely result in the largest increases in players choosing to return.
This approach might not be optimal for maximizing Net Promoter Score given the dynamics of that calculation. However, our research shows that it might just be the key for maximizing retention.
