Lean Data Podcast

On Monday, Tony Loyd was nice enough to include me in his great series of Social Entrepreneur podcasts. We covered a lot of topics but dug in most deeply on Lean Data, particularly on how we are using it at Acumen to amplify the voice of low-income customers so our entrepreneurs can better serve them.  It was a fun conversation.

(if you’re not seeing the embedded link click here)

 

If this kind of thing is up your alley, you might want to sign up to receive the specialized newsletter we’ve created to share hot-off-the-press insights on what we’re learning through Lean Data. We send it out once every six weeks or so, so it won’t clog your inbox, and it’s full of great stuff.

It’s called Impact Matters and you can sign up here.

 

Put the gloves down

What does it mean to say that real fundraising is about building long-term partnership?

It means that some of the most important meetings you have with long-term funders are the ones that cover topics that don’t require their funding support:

The amazing, fully funded project that you’re just kicking off with a few other partners.

The great piece of work that you both know is outside of their formal strategy that you’re really excited about.

The new initiative where you’d value their experience and input.

Some funders are so used to – and so tired of – being pitched constantly that they end up behaving protectively, as if the only thought running through their head is, “how many times will I have to say ‘no’ in this meeting?”  I’ve had funders start sentence after sentence with, “we’re not doing any new funding this cycle” long before I’ve asked for anything. There’s no hope of building a relationship if someone has their gloves up protecting themselves from an onslaught of asks.

Fundraisers can be part of the problem, acting as if that every meeting should include a financial ask, and fearing that they’ve made a mistake if they don’t ask for money each time.

Every meeting should help deepen the relationship and, even better, should give everyone around the table the chance to contribute meaningfully to making positive change happen. Often that’s not about money.

Taking a stance that you’re not constantly, desperately on the lookout for funding is one of the best ways to allow the partners you hope to work with to put down their gloves and actually listen.

What’s in a Question?

This post originally appeared on Medium.  I’ll keep reposting these on my blog from time to time. If you want to learn more about Lean Data, check out the full story in this Stanford Social Innovation Review Article.  And there’s still one week to sign up for the free +Acumen Lean Data course.  It’s a great thing to do with a team.

Every day, more than 5 million new cellphones are sold. That’s more than 10 times the number of babies born each day. We are barreling towards a world where a cellphone will be in every pocket by 2020, and a smartphone in every pocket soon after that.

This revolution is making the unimaginable real— in the near future, we will have the opportunity to start a dialogue with literally every person on the planet. This new two-way conversation, where everyone participates, will pull billions of people into the mainstream by connecting them with one another.

At Acumen, we see inexpensive cellphones in the hands of a billion new low-income customers as a chance to supercharge the work we’re doing to end poverty. Through our Lean Data initiative, we are taking advantage of the spread of cellphones to talk to previously excluded segments of society. The focus of Lean Data is to equip startup social enterprises in the developing world with the tools and techniques to start a dialogue with their customers. Our aim is to empower these customers to articulate what they need to improve their lives.

Since starting this work in 2014, one of the most important lessons we’ve learned is that a cellphone in every pocket is just a starting point. The art of every Lean Data project is in the questions we ask. Ask the wrong questions, and you get back little of value. Ask the right ones, and you can move from data to information to actionable insights.

Great questions connect with customers and give them an opportunity to share their voice. But crafting a great question is no easy task. The slightest shifts in word choice can affect understanding; the smallest differences in intonation alter perceptions of sincerity. All of these nuances can bias the data and diminish its value.

For example, in trying to understand the usage of solar home systems in Kenya, we started with the question, “How often are you currently using (product/service)?” After testing this question over SMS, we received feedback suggesting we omit the word “often” and make the question more simple and direct. We quickly amended the question to “When do you use (product/service)?,” provided sample multiple choice replies, and received a higher level of understanding.

Getting questions right is not a new idea. Indeed, Angus Deaton’s recent Nobel Prize was largely the result of his foundational work on designing household surveys. What’s new is trying to gather rich data over a cellphone. While you can run an effective focus group with a loose guide of topics and you can cover a lot of ground in a 90-minute one-on-one interview, a typical SMS survey is limited to 10 questions and 150 characters per question. These constraints are a powerful pressure-cooker for the questions we ask. We’ve got to make every word and every question count.

So what makes a great question?

For us, a great question is one that is easily and consistently understood by customers. It’s one that makes the complex simple. And it’s one that yields insight around what matters to the customer and the social enterprise trying to serve them.

One of the biggest challenges in impact measurement and international development is understanding not just the breadth but the depth of impact. In Acumen’s case, depth is defined by the degree of change in their well-being a customer experiences from one of our investments’ products or services. For example, we know that a solar light is a better solution than a kerosene lamp, but exactly how much better and why is tricky to figure out. This isn’t an academic exercise for Acumen or our companies. Ultimately, we need to understand our customers’ needs to know where to direct our capital to drive the greatest impact, and without impact data we are simply flying blind.

Because we work across multiple sectors addressing a number of the problems of poverty, our challenge extends beyond just figuring out the quantitative impact of owning a solar light or sending a child to a low-cost private school. Our goal is to go one step further and understand the qualitative difference in value that our customers experience when comparing the various products and services available to them.

Photo by Joanne Schneider

Can we really compare the impact of a year of schooling to owning a solar home system? We’re not sure, but we think it’s worth a shot. We believe that trying to understand these comparisons from a customer’s perspective will push us to listen harder and deeper, and it will test the limits of our ability to get rich data through mobile phones.

We asked ourselves if we could create a question or a set of questions that get at this topic directly, helping our customers share what they value most and why.

 

While a single question to cut through the complexity of our work seemed far-fetched, we knew that similar attempts have been made before. Twelve years ago, Frederick F. Reichheld, Rob Markey and Bain & Company developed the Net Promoter Score® (NPS). According to the Harvard Business Review, the NPS “substitut[ed] a single question for the complex black box of the typical customer satisfaction survey.” Today, it’s become widely adopted by the Fortune 500 as one of the most effective ways to measure customer loyalty. Just as NPS provides companies with a method to effectively judge performance and generate qualitative customer feedback, we wanted to create a single, unifying question to compare social impact.

Photo by Joanne Schneider

We started by asking ourselves whether the NPS question — “How likely is it that you would recommend [product/service] to a friend or colleague?” [1–10 scale]” — could serve as a good proxy for how much impact a product had for our customers. We wanted to test this by asking NPS questions together with our depth of impact questions to see if products with a higher NPS also had a higher depth of impact.

We piloted this approach in Kenya and India in two surveys, and the initial results were not as promising as we had hoped.

Despite the proven success of NPS with more affluent, educated customers, the question didn’t seem to perform well with our customers who are typically poor, have limited formal education and little experience with surveys. In follow-up conversations, we heard that the 0–10 scale was hard for them to understand and the hypothetical “would recommend” language didn’t translate well.

Lean Data surveys are short and inexpensive to conduct, so it’s easy to test and refine questions. We experimented with four different versions of the question before landing on a question, inspired by NPS, that seems to perform well: “Have you ever recommended product/service to a friend?” We also played with three different answer scales and arrived at a workable solution. Instead of a 0–10 scale, customers choose between three responses: “Yes, I’ve told many friends;” “Yes I’ve told some friends;” or “No, I have not.”

Once we saw the effectiveness of this question, we wanted to go further, to learn not only whether or not customers recommended a product but also the drivers of meaningfulness of that impact. Drawing on the concept of Constituent Voice developed by Keystone Accountability, we developed a second question, asking customers to respond from “strongly agree” to “strongly disagree” to the statement: “There have been changes in my home because of (product/service).”

In the early tests we’ve run, we’ve seen correlation between reported depth of impact and the strength of agreement to this “meaningfulness” question. For example, owners of solar lights who “strongly agree” with the statement reported an 83 percent reduction in expenditure kerosene, while the customers who said “agree” only reported a 69 percent savings on kerosene. These are just preliminary results, but we’re starting to see that this question might allow us to compare across different interventions, so that customers can tell us what they value the most and why.

Photo courtesy of Joanne Schneider

While we’re still fine-tuning both of these questions, the progress we’ve made is exciting. Low-income customers are enthusiastic to engage in dialogue, and we are seeing that it’s possible — if you work at it — to develop new questions that capture rich, meaningful data about the wants and preferences of this emerging set of customers. At the end of one of our surveys, one happy customer expressed her satisfaction with the service she received at a health clinic and then added, “I really enjoyed being interviewed.” Clearly, we’re on to something.

These are the kinds of customers whose voices we aim to hear. Our Lean Data work is focused squarely on helping the startup social enterprises we invest in to listen more actively to the low-income customers they serve. For them, Lean Data is a chance to talk to their often remote and dispersed customer base in a way that doesn’t break the bank.

While Lean Data is, today, being used mostly by startup social enterprises, our work in learning to ask the right questions over mobile phones is universal. The low-income customer of today is the low middle-income customer of tomorrow. Hundreds of millions of people in the developing world are poised to improve their well-being, but this depends on how well we, as a society, listen to them and adjust our efforts to meet their needs.

So much of this rests on the simple act of caring enough to ask the right questions.

Strong ideas, loosely held

One of the best pieces of advice I got about five years ago was that I should have “strong ideas, loosely held.”

The feedback I was getting was on the “loosely held” part. At the time people experienced me as having “strong ideas, strongly held.” I think I’ve made some good progress on that.

Five years hence, as I come back to the central paradox inherent in this notion, I’m understanding that the suggestion isn’t to have any less conviction around my ideas. Indeed nearly all of the time we need more conviction, more passion, greater commitment, and greater follow-through.

The real point here is that the passion we have for our own ideas must be coupled with a core, deep-seeded belief that most ideas, most of the time, get better when they interact with, and are changed by, other ideas.

What they say, what they mean

“I’m not saying that…” = “I AM saying that…”

“By the way…” = “The most important thing to me is…”

“I’ll be brief…” = “You’d better make yourself comfortable.”

“Oh, one last thing…” = “Listen up.”

“I’d like this presentation to be a real dialogue…” = “Shut up and listen…”

No irrational action

I used to dismiss what looked like irrational action.  I’d watch people’s behaviors and, when things didn’t make sense to me, I’d let it go.

“Sometimes people do things that just don’t make sense” was a safe refrain.  Maybe they didn’t have enough information or do the right analysis or sometimes actions just don’t make sense.  My overly-rational mind would see irrational action and deduce that the person had failed to analyze something properly, understand its implications, or explain themselves clearly.

Talk about a misdiagnosis.

People only do things that make sense (to them), and while I know we all make errors of judgment and analysis, these days anytime I have a “that just doesn’t make sense” reaction a little alarm bell goes off.

By way of analogy, I only recently figured out that getting really nervous about a new idea or a project – and feeling like maybe I should just drop it – is a great indicator that I’m on to something really important (nervousness = my lizard brain resisting me doing something significant and worthwhile).

Similarly, every time someone does or says something really irrational that’s a great moment to pay extra attention, to try to figure out what’s really going on – not rationally, on an emotional level.

These are great sensors to have on in fundraising situations, because it is so difficult (and slightly taboo) to talk about why and how real fundraising decisions are made.  You spend time in a long cultivation, building to what seems like a strong, jointly-developed funding opportunity, and at the last minute something veers completely off-course.

There’s no such thing as irrational action.

When I see an “irrational” response, I know that I’m the one whose information about, understanding of, and diagnosis of a situation is not (yet) on the mark.

It’s a great time to pay extra, not less, attention.  It’s a great time to listen more.

The main reason, also, by the way

I used to think listening to people had to be pretty easy since, I figured, it was their job to say what they meant.

That feels like one of the biggest misconceptions I carried around (for way too long) –  it’s flawed on numerous levels, including but not limited to its American-ness (since culturally we value directness more than just about anyone).  It also willfully ignores how people come to conclusions and how people (especially persuasive people) explain their conclusions.

Put it this way: if Jonathan Haidt is right (and I think he is), we make decisions with our hearts (maybe our gut, or our elephant) and then explain and rationalize them with our heads (the rider).  To me that means that what we say to explain these decisions is necessarily a rational (re)construction of truth, rather than the truth itself: we will construct a story that uses the core truth as a springboard and then assembles the pieces that will be, in our judgment, most persuasive and palatable to the listener.

With this in mind, when someone is explaining something (a plan, a proposal, a decision) to me and he says…:

  • The main reason we should do this is….
  • also it has to do with….
  • and, by the way, there’s this one other small reason….

…it seems to me that the most logical thing to expect is that people’s “main” and “also” reasons are the series of facts/explanations that will be most convincing to me.

This makes the “by the way” a great candidate for what’s really going on.