OST Evaluation Toolkit
Section IV: Case Studies
Case Studies
Six pilot organizations were selected to pursue a range of evaluation strengthening projects for the local out-of-school time (OST) sector.
Improving Staff Practice
Reflection/Tools
● Do we want to include a question or two here at the end of each case study section? and/or link the tools referenced in the case study above here
Two focused on using evaluation to improve staff practice. The Makers Clubhouse, a STEM afterschool program serving students from Faison and Lincoln elementary schools, divided its staff into three tiers and developed differentiated professional development for each. To support this work, they created an observation tool grounded in their values and expectations for staff — a process that itself sharpened their thinking about program quality. They have since used the tool to guide ongoing staff development.
Neighborhood Learning Alliance conducted a similar project, developing an observation tool to strengthen staff practices. Sites also experimented with both traditional and embedded evaluation methods. GirlGov integrated Kahoot quizzes into their programming, generating data on participant learning while keeping the experience engaging for youth.
The Pittsburgh Parks Conservancy used structured journal prompts and systematic analysis of participant writing to understand where youth were experiencing success.
Reflection/Tools
● Do we want to include a question or two here at the end of each case study section? and/or link the tools referenced in the case study above here
Open Up instituted multiple embedded evaluative activities related to mindfulness: daily check-ins, a card activity for reflection and takeaways, and analysis of observations, student work, and student feedback.
MCG Youth strengthened their evaluation on two fronts. They revised and improved their participant surveys, and they launched a videobooth project in which youth (and a few staff) recorded short videos reflecting on what the program means to them. With over 40 videos collected, they used generative AI to identify common themes — surfacing the most frequently named program strengths in participants' own words.
Reflection/Tools
● Do we want to include a question or two here at the end of each case study section? and/or link the tools referenced in the case study above here