OST Evaluation Toolkit
Section IV: Case Studies
Case Studies
Six pilot organizations were selected to pursue a range of evaluation projects to strengthen their organization’s knowledge and capacity.
Improving Staff Practice
Reflection
● How might you empower your staff to engage in the evaluation process?
Two focused on using evaluation to improve staff practice. The Maker’s Clubhouse, a STEM afterschool program serving students from Faison and Lincoln elementary schools, divided its staff into three tiers and developed differentiated professional development for each. To support this work, they created an observation tool grounded in their values and expectations for staff — a process that itself sharpened their thinking about program quality. They have since used the tool to guide ongoing staff development.
Neighborhood Learning Alliance conducted a similar project, developing an observation tool to strengthen staff practices. Sites also experimented with both traditional and embedded evaluation methods.
The Pittsburgh Parks Conservancy used structured journal prompts and systematic analysis of participant writing to understand where youth were experiencing success.
GirlGov integrated Kahoot quizzes into their programming, generating data on participant learning while keeping the experience engaging for youth.
Reflection
● What journal prompt(s) could you use at your organization to understand where your youth are experiencing success?
● What quick mindfulness activities could you routinely implement in your organization’s programming to gather quick information from youth?
Open Up instituted multiple embedded evaluative activities related to mindfulness: daily check-ins, a card activity for reflection and takeaways, and analysis of observations, student work, and student feedback.
MCG Youth strengthened their evaluation on two fronts. They revised and improved their participant surveys, and they launched a Video Booth project in which youth (and a few staff) recorded short videos reflecting on what the program means to them. With over 40 videos collected, they used generative AI to identify common themes — surfacing the most frequently named program strengths in participants' own words.
Reflection
● What tools do you engage with in your programming that could also be utilized to help gather data?