A Story from Luma
It is difficult to speak about Arifu without bringing up Zachary Ndegwa. Zachary was once a farmer from a village called Luma in Meru county, Kenya. In 2015, when Zachary first interacted with Arifu on his feature phone, he had a quarter acre tomato farm, the yields from which were sole means for providing for his family.
At first, Zachary thought Arifu’s free-to-use chatbot was a person. By chatting with Arifu, he began learning how to grow healthier tomatoes by choosing the right seeds, applying fertilizer correctly, and much more. After learning, Zachary decided to take a not so insignificant risk. He chose to trust and apply the advice he was receiving from these texts. Much to Zachary’s relief, his tomato production and his income increased. This realization continued to drive his engagement with the Arifu chatbot.
In 2019, we revisited Zachary and found that he is no longer growing tomatoes himself. Instead, Zachary has trained other farmers using the practices he learned from Arifu and now buys tomatoes from those farmers to sell in the market. Both Zachary’s income and status in his community have grown and he attributes this change of fortune in his family’s life to his continuous interactions with Arifu. Watch our interview with Zachary below.
Enhancing skills at scale
Zachary’s story exemplifies the type of impact Arifu strives to have on the lives of all learners who interact with our platform. Arifu incorporates expert advice, insights from human-centered design research, and behavioral economics principles to create actionable educational content that builds learner capacity on a wide range of topics. In addition, through its interface, Arifu connects end-users to the ecosystem of products and services, such as financial services or inputs to agriculture. By delivering this content through multiple digital channels, including SMS, Arifu aims to bridge the digital divide by creating, curating, and delivering useful information for those without access. We aspire to make this information demand-driven, needs-based, timely and actionable in order to positively impact lives.
Recent academic literature suggests that SMS-based extension services, particularly in the form of advice, if actionable and targeted well, can be quite effective in improving farm outputs 1 . In addition, insights from Arifu’s ongoing research suggest that learners who apply our content have found that it has improved their quality of life (more on this soon).
As of the end of 2019, Arifu has well over a million learners. As we continue to scale, it has become increasingly important to ensure that Arifu’s reach adds value to as many of our learners as possible. To drive impact, it is simply not sufficient to create content and deliver it en masse. Nor is it sufficient to have high engagement levels with the deployed content. Impact requires solving real challenges learners face by continuously measuring the right things and innovating in order to improve. To achieve this it is important to measure learning accurately and being able to understand whether learning is resulting in meaningful behavior change. One tool we have developed to tackle this challenge is an algorithm we have dubbed the Skills Score.
The Skills Score measures learning and the ability of learners to apply the content they engaged with. The three key components which make up the Skills Score are knowledge graphs which map how concepts relate to one other, quizzes to assess and reinforce learning for learners, and increasingly complex questions within those quizzes that measure the ability of learners to use what they learned to, for example, evaluate situations and create solutions. We believe these questions should allow us to not just measure knowledge improvements, but to also predict how well learners are able to apply the content they engaged with2. The Skills Score should add value in (at least) the following ways:
- Identify where learners have skill gaps allowing us to personalize training to help improve learning at the right level for the right content 3.
- Measure how effective Arifu trainings are at building skills and how to improve them.
- Enhance the quality of engagement with learners through the provision of increasingly timely and relevant content to enable them to make more informed decisions.
- Drive the predictive power of content engagement, Skills Scores, and meaningful behavior change.
- Communicate which skills have been learned by learners to our partner organizations so partners can make more informed strategic decisions.
Ultimately the Skills Score can be used to enhance Arifu’s ability as a digital coach operating at massive scale. It promises to enhance the cost-efficiency, dynamism, and impact of delivering trainings on demanded and needed information to learners globally thus enabling Arifu to fulfill its mission in significant ways.
Our commitment to impact
The global digital divide is still a long way away in creating access to the treasure trove of publicly available information many of us take for granted. Even so, it is not easy to identify information that is immediately relevant for driving capabilities. This is the gap Arifu seeks to resolve. Our research strategy is predicated upon a theory of change aiming to bridge these gaps in knowledge and improve the chances of social mobility for households. We know there are many Zacharys among our learners. His story speaks to the need for tools and systems that are right fit solutions and enable people to drive change using newfound knowledge and utilizing immediate resources available in their ecosystem.
While theories, anecdotes, and visions are necessary, they alone are not sufficient. Learning, iteration, creativity, and credible proof points are needed. Arifu has committed to holding itself accountable to the goal of impacting lives meaningfully4. We will pursue this by leveraging the suite of research tools that have been developed in the development and technology sectors5. We believe the Skills Score is one tool which will help us serve people with what they need, when they need it, as we learn about what their capabilities are and what they could become.
- Fabregas, Kremer, et al (2018): “Can SMS-extension increase farmer experimentation? Evidence from Six RCTs in East Africa.”
- We adapted Bloom’s taxonomy to create and measure increasingly complex levels of questions. Currently, we are ground-truthing and improving the efficacy of our Skills Score through ongoing research.
- There is strong global evidence in the education sector on the high impact on learning and cost-efficiency of teaching learners where they have knowledge gaps. The Skills Score will help us replicate this concept at scale across vocations and learning needs.
- We are hiring to do just this work!
- Measurement methodologies include ongoing A/B testing, bandit algorithms, machine learning, remote sensing, randomized controlled trials, difference-in-difference, rapid insight focused surveys, and more.