The Jim Joseph Foundation and BBYO engaged Informing Change to evaluate the first three years of the DJE Initiative. Work began in the fall of 2012 with the design of a project logic model (Appendix A). Based on the model and related conversations, we collectively identified the following evaluation questions:
1. How does the DJE model work? What responsibilities do DJEs have? Who do the DJEs work with, and why? What are DJEs' primary activities? What accounts for any variation? In what ways is the development of BBYO's Educational Framework influencing the DJEs' work and implementation of the DJE Initiative?
2. How is the quality of meaningful Jewish experiences changing for BBYO participants? To what extent are regional professionals, BBYO chapter advisors and teen leaders seeking out the DJEs for guidance, support and education? To what extent do regional professionals, BBYO chapter advisors and teen leaders report having an enhanced ability to deliver Jewish content and foster immersive learning experiences?
3. What lessons are being learned about the DJE Initiative that can inform future implementation? What is facilitating the work of the DJEs (e.g., structures, personnel, tools, contexts)? What challenges or barriers do the DJEs face in their work? What opportunities exist that have the potential to address those challenges? How and to what extent has BBYO incorporated the DJE model into the organization's operations and culture as a way to support high quality, meaningful Jewish experiences for BBYO teens?
To explore these questions, we designed a qualitative evaluation to examine and describe how the DJE Initiative worked and how BBYO might have changed through the DJEs' work. The evaluation relied on the following qualitative methods (please see Appendix B for a more detailed description of the data sources):
- Interviews: with all three DJEs at multiple points in time, as well as with approximately 30 regional and international office professionals (some more than once), almost 40 teens and 15 advisors.
- Site visits: to BBYO regional conventions and international programs.
- Observations: of DJE-led meetings with teens to plan conventions, of DJE-led events (such as webinars) and of Jewish Enrichment Team meetings.
- Document Review and Analysis: of DJE materials, including Kivun, international resources and other materials.