When it comes to evaluating the effectiveness of a learning solution, most people’s go-to number is the Net Promoter Score (NPS). It’s a number between -100 and 100 meant to essentially reflect whether people felt their time was well spent. Which would be a useful metric, except that whether people liked an intervention holds almost no bearing on whether that intervention worked.
(In fact, some of the most effective learning is actually disfluent, because it engages deeper processing in the learner.)
Such is the inherent flaw of NPS. Learning professionals may claim, with the best of intentions, that they want to spark behavior change. They want a given program to lead to a new habit, a new way of working. But then when it’s time to measure people’s behavior change, the folks involved run a sentiment analysis instead.
So, shouldn’t we really track the behavior change itself?
Introducing the Behavior Change Percentage
In delivering our scalable learning solutions to clients, we’ve made it a habit of tracking not just NPS but behavior change percentage, or BCP. It’s a measure of how frequently employees and managers perform a new and desired behavior. For instance, at the global tech firm Splunk, NLI measured a BCP in which 85% of the more than 3,500 managers participating in NLI’s DECIDE: The Neuroscience of Breaking Bias solution began using a strategy to deliberately mitigate bias at least once a week.
Other examples of BCP measured in solution-rollouts include:
Midtown International School — Going through NLI’s Performance Leadership Program, in which participants learned about the SCARF® Model, led to 100% of the 18 directors and team leads reporting improved interactions with direct reports.
“It was the most brain-friendly training we’ve ever done,” said Andy Noktes, Midtown International’s Head of School. “We had people go home and share the strategies and ideas with their partners and spouses and come back the next day talking about how they’d had conversations with so much more depth than they’d experienced in their whole relationship.”
BevMo! — After going through CONNECT: The Neuroscience of Quality Conversations, follow-up surveys found 96% of the more than 3,000 employees reported using strategies from CONNECT at least 1-3 times per week.
“Instead of telling associates what to do or talk at them, we now have a conversation that involves all parties,” said one manager.
Why BCP works
BCP tends to overshadow NPS for a couple reasons.
First, the science of memory shows how the hallmark NPS question — “How likely are you to recommend this learning session to a friend or colleague?” — provides unreliable data. For example, when interpreting the question individuals might focus on whether or not the learning session was fun. However, as NLI has found, the most effective learning comes from exerting effort to understand difficult concepts — an experience that might not be the most enjoyable.
Secondly, when people are asked to make decisions about their future behavior, respondents have a strong tendency to over-report the likelihood of “good behavior.” As a result, the NPS tends to be more reflective of the person answering the question than the learning session itself.
Instead of relying on a score like NPS, which uses an easily biased metric as a proxy for change, BCP reflects the actual success or failure of a given intervention. The added benefit, past learning more about the current program, is that BCP data can lead to smarter, more robust feedback for future programs. Instead of asking “How do we get more people to like it?” leaders can ask “How can we create more behavior change?” — a question far more suited to produce lasting success.
