What difference do stories make in out-of-school program evaluations? The best evidence of Social-Emotional Learning (SEL) is found in stories. Stories confirm how well young people are interacting and reveal increased connection to self. Stories expose the rich alchemy of an effective program’s influence on self-awareness, skill-building and interpersonal development.
Want to know how peers, family and staff contribute to positive change? Collect stories. Stories illustrate programs confronting trauma, poverty, violence, racism, sexism, homophobia, marginalization, new opportunities and any variety of historical factors and current forces that overwhelm or uplift participants. Stories tease out what organizations do best.
Let’s face it. Stories touch hearts. They make a lasting impression. Like music, they linger in the soul. We find ourselves almost humming them later.
There is a richness in stories that numbers cannot capture, and often mask. The more stories the better to convey social-emotional impact in vivid detail.
Program leaders are first to disclose that their numerical data merely skim the surface and relate little of the multiple transformations witnessed in the lives of those served. Nonetheless, hard data are expected, as evidence of growth and social impact.
Investors are rightly suspicious when a few transformative stories are put forward as evidence. One life-changing tale, however inspiring, may be an exception to the rule.
Faced with a choice between receiving colorful narratives on select few youth or colorless data on all participants, backers are prone to opt for the latter, “Show us the numbers!”
This choice may be unnecessary. As John Allen Paulos neatly expresses:
“… The gaps separating stories and statistics, subjective viewpoint and impersonal probability …. and meaning and information are bridgeable in places, not so in others, and seldom well-marked.”
At SEED Impact, our task has been to bridge these gaps and place stories and numbers on equal footing. In equipping out-of-school programs with what they need to convey their distinctive value, our aim is to forge an exciting synergy.
Typically, when researchers work with stories, the story is made the unit of analysis. Numbers or percentages might be derived from tallying successful versus less successful stories or through counting the frequencies of key phrases in the story database.
Our approach is to structure story-telling such that numbers and storylines emerge in concert, leading to stories that are both evocative and mathematical!
Having fiercely explored this puzzle of how to orchestrate stories with numbers since the mid-1990s, we offer the following guides:
Consider each youth story as a journey.
Compose a set of journey maps linked to SEL competencies, and use them to mark the position of the youth at various points in time: before, during, after and perhaps, as resources permit, long after the program experience.
Devise a tracking system using higher scores to reflect advancing movement along the maps.
Request the storytellers (journey mappers) to periodically note and score updates in position, reflect on the causes, and describe results emerging.
The most compelling mapped journeys are those depicting dramatic growth in SEL. The trick is to amplify the basic mapped data with human context: background information, participant testimonials, staff or family member observations, and juicy quotes.
Shown below is the classic heroic journey, depicted as a results ladder keyed to SEL growth. Virtually any program experience can be mapped in this way. We often customize three such maps, to track increasing competencies in Being, Doing, and Relating, at once.
Consider the participant’s experience:
Starting from the bottom, at the outset of program engagement there is initial resistance: “Do I really want to be here?” followed by getting one’s bearings. The time required to move from Stage-0 to Stage-1 can be measured in minutes, hours, days, or longer. In some cases, a participant may resist the entire experience from start to finish.
Over time, as participants engage in more program activities, they gain comfort with new practices (Stage-2). With success and encouragement, they develop proficiency (Stage-3). Some advance to the highest stages of excellence and thriving, especially in multi-year programs with opportunity to mature from prior SEL gains.
As each participant’s journey is tracked, SEL scores build a numerical estimation of increased competencies. Stories provide context and lend validity as they bring the numbers to life. With lively photographs added to the mix, the final report delivers a real taste of experience. Growth spurts and setbacks are explained, and leaders realize where creative attention can be focused to improve outcomes.
Our approach values process as product. We see each journey as an adventure. Where possible, participants are engaged to self-assess. The self-evaluation process (the activity of pinpointing one’s own SEL growth on journey maps) raises awareness of what has been achieved. It also reveals what’s next. In the moment of pausing to take stock, there is a psychoactive effect on the brain such that stepping into the next level of potential has already commenced even before specific activities to get there have been defined.
For leaders and learners alike, insights are deepened, as words are found to express—and celebrate—each stage of a learning adventure.
Working with out-of-school programs of all variety, we have made a practice of customizing with each program, with care, to reflect the particular goals, values, and SEL growth opportunities that comprise its special “secret sauce.”
It is a joy to see programs discover and better articulate new dimensions of what they do best. They are able to focus planning with sharper understanding of what works and what doesn’t. Their journey maps provide a common framework for exploring similarities and differences across individuals, cohorts and whole programs.
Dynamic, credible reports capture the imagination of all stakeholders, including current and future funders.
It seems to us at SEED Impact that there are endless, unexplored possibilities for blending numbers and stories. We are learning each day, welcome your thoughts, and appreciate opportunities to partner and learn together.
We are delighted to have secured funding to offer scholarships for under-resourced organizations.
How are you hearing this? Is it something new? Are others pursuing similar lines of inquiry? What do you see as the strengths or limitations of our approach?
Contacts us: possibilities@seedimpact.org
Barry Kibel is Director of Innovation and Research, and Melinda Lackey is Co-founder and Director of SEED Impact, a nonprofit capacity-building organization that specializes in equipping community-based initiatives with what they need to envision, measure, report and scale outcomes, and transform daily operations for high impact.