OST Evaluation Toolkit
Section II: Evaluation Basics
Evaluation Basics
This section covers the core ideas behind program evaluation — what it is, why it matters in OST, and how you can start thinking about it for your own program. You don’t need to be a researcher to do a meaningful evaluation.
What Makes OST Different?
Out-of-school time programs don’t always fit neatly into evaluation frameworks designed for schools or formal training settings. A few factors make OST unique — and worth accounting for when you design your evaluation:
These factors indicate that traditional “did it work?” evaluation methods — which rely on control groups and standardized tests — often don't suit this context. OST evaluations tend to be more adaptable, relationship-oriented, and geared toward understanding rather than simply proving effectiveness.
Reflection
● What makes your program unique? What would a traditional evaluation miss about what you do?
● Have you ever tried to evaluate something and felt like the method didn’t quite fit? What happened?
Voluntary Attendance: Young people choose to be there. That changes the relationship between youth and program, and it means attendance alone can tell you something.
Diversity: OST includes everything from sports leagues to arts programs to homework help to nature clubs. There’s no one-size-fits-all approach.
Complexity: Programs don’t operate in a vacuum. Families, schools, neighborhoods, and other systems all shape what happens — and what your evaluation can actually capture.
Types of Evaluation
There are two main ways to think about evaluation, and most programs need both at different points:
Summative evaluation asks: Did it work? It occurs at the conclusion of a program cycle and emphasizes outcomes. Examples include pre- and post-surveys, end-of-year data, or a final report to a funder.
Formative evaluation asks: How does it work, and how could it work better? This happens during the program, often in smaller, more iterative ways. Think check-ins after sessions, quick polls, or staff debrief conversations.
You might also encounter terms like process evaluation, goals-based evaluation, and internal vs. external evaluations, and note that different people use these terms differently. The key is to understand the specific question you want to answer and choose the approach that best suits it.
Reflection
● Think about the last time you tried to understand how your program was going. Was that summative or formative? Was it intentional?
● What’s a question you have about your program right now — about whether something is working, or how to make it better?
Participatory Approaches
OST providers tend to care deeply about youth voice — and that extends to evaluation. Participatory approaches involve the people most affected by a program in the evaluation process itself, not just as subjects but as co-designers and meaning-makers.
Participatory Evaluation: Stakeholders — including youth, families, and staff — participate in shaping the evaluation questions, methods, and interpretation.
Participatory Action Research (PAR): Combines research with action. Participants identify a problem, investigate it, and use the findings to drive change.
Youth Participatory Action Research (YPAR): Young people are the researchers. They develop questions, gather data, analyze results, and share their recommendations. YPAR is becoming more prevalent in OST settings.
These methods require additional time and planning, but typically yield more relevant, trusted, and actionable findings.