Module 3: Program Evaluation

Purpose of the Evaluation The Notre Dame High School PAWS social justice group is their 12th month of ministry work. What started in the 2015-2016 school year as a student lead groups of 8 students facilitated by a certified staff member, has expanded to a group of 40 students, still lead by the same staff member.

A driving force behind PAWS is the idea of the importance of volunteerism in teenagers. According to ServiceLeader.org, volunteer service promotes healthy choices, teaches important life skills, improves the community, and encourages a lifelong service ethic. This idea aligns with the values of Catholic faith that permeates programming within Notre Dame High School. Moreover, the liaison teacher’s aims is for PAWS ministry be student lead, where the youth are not only involved in the solicitation and gathering of resources and funds, but also participate in the delivery of their collections.

In their first year, PAWS did charitable acts and fundraising for: The Calgary Human Society, the Calgary Library Foundation and the Calgary Public Library (next door to the school). The club received a small grant in April 2016 with the money used for advertising supplies and transportation for volunteer work and delivery of donations. However as of January 2017 all the funds from this grant have been used.

A mandate of the club is to provide educational awareness to the student and staff population. When possible, the group attempts to inform the school community about the needs they have selected to work towards, why those needs exist, and how they can work with the school community to address these needs. They provide opportunities for the school community to participate in charitable actions, as directed by their own peers.

The purpose of this evaluation is to find out if the program is promoting healthy choices, teaching important life skills, impacting the school community/community and encouraging a life long service ethic (will the grade 12 students continue their work after graduation in the spring?). Now in its twelfth month, with a group that has quickly quintupled, PAWS needs to revisit its mandates to ensure that it is benefiting its members and target audiences in its intended way and what might need to be altered as a result of the evaluation.

Evaluation Questions
  • Are grade 12 PAWS members leaving high school with a service ethic?
  • Is there awareness in the school community about PAWS activities (needs and ways to address these needs)?
  • How is the PAWS program impacting the school community (participation, contributions, etc.)?
  • In what ways, if any, can the PAWS program improve on its vision?
Project Stakeholders PRIMARY-Those directly involved and impacted by the results of the project.

  • PAWS student members
  • Teacher liaison
  • The charitable organizations the club works for / with

SECONDARY-Those who are less involved but might benefit from the findings

  • Students, teachers and administrators
  • Students from feeder schools
  • Parents
  • Community members
Vision

 

How will our participants, student population, teaching staff, administration and community change as a result of PAWS’ work?

“Nurturing Students to Impact the World for Christ” PAWS offers students a platform to discuss social injustices and works of mercy that they are interested in correcting and/or pursuing.  The structure of the club allows students to join their efforts with like-minded individuals so that their works can be magnified and supported by solidarity within their peer group.

Long Term Outcomes What are the most important changes in our community that PAWS will contribute to?

 

1. Support a student-lead (teacher-facilitated) social justice group that contributes to the school community and community partners though meaningful charitable acts

Short-Term Outcomes What changes do we expect need to happen NOW in order to reach our long-term goals?

 

 

a.     Invite staff and administration to meetings

b.     Give students more opportunities to fundraise and participate in charitable acts outside of school

c.     Create greater school-wide awareness of PAWS endeavors

d.     Access to funds

2. Shape life-long values and sense of purpose in participants a.     Students attend weekly meetings to gain knowledge

b.     Teacher liaison provides students with resources and tools to extend their work outside of school

c.     Students use available resources to connect with charitable community agencies.

3. Increased awareness about the needs of community (charities), when these needs exist and how the school community can work together to address them a.     Better knowledge and understand among staff and students

b.     Student engagement in PAWS activities

c.     New teacher advisors/liaisons

Key Actions What is PAWS currently doing to achieve these desired results / outcomes?

1.     Run a minimum of three in-house charitable / social justice projects

2.     Teacher liaison funding requests through various grants

Linking Actions to Short-Term Outcomes What change do you hope will happen by implementing this action?

An increase in staff engagement

Capacity building through awareness

Why do you think that this action will lead to this outcome?

 

With a greater presence in the school, PAWS will create more awareness about their ministry and how the school community can help with their charitable endeavors. More funds means more avenues for students to inform others, and participate in off campus ministry work

Change Partners What other partners are necessary to help PRIDE reach the short-term goals?
Administration/Staff Administration can influence teacher engagement in PRIDE as extracurricular work

They can provide funding for materials and transportation for volunteer work (on and off campus)

School Community Their efforts will educate the school community about the needs of the community and what can be done to help

Their work creates awareness and will bring in more volunteers (teacher and student)

Funders Grant money and funding allows PAWS to expand their work into multiple charitable organizations.

They help with the resources needed to

Community / Feeder Schools Building an awareness with future students allows to build a sense of community and also begins to shape lifelong values and purpose before reaching high school
Resources The PE requires time and space for meetings, observations and interviews. Additionally, participants will require the use of a digital device (phone, tablet, computer) to access the surveys
Data Collection Methods and Analysis Strategies Data collections will be both quantitative and qualitative, however will lean more towards the qualitative. The evaluation will look at changes in behaviors, perceptions and understandings of stakeholders. These indicators will help answer the how and why of PAWS’ effectiveness.

Data will be collected through a focus group that includes PAWS members and change partners. The data will be anecdotal accounts, surveys with open-ended questions (Google Forms/Survey Monkey), formal and informal meetings. This will be ongoing, through the course of the next school quarter and collected before, during and after the evaluation. Data will be shared with PAWS members and teacher liaison in Google Form generated reports.

Program Impact and Reporting Strategies As stakeholders will be directly involved in the program evaluation, the PE will positively impact all stakeholders. The findings will help answer what is working with PAWS (are they meeting their program objectives and mandates) and possibly why it is working. Additionally, this direct involvement will provide various perspectives on the program’s effectiveness, and therefore information gathered should be accurate and reliable. The information will be reported to the teacher liaison, which will in turn report to PAWS members. They will have the necessary information to modify their program in order to create more awareness of their social justice endeavors, have a positive influence on the school community, and instill lifelong skills and sense of service ethic in its members.
Commitment to Standards of Practice Utility Standards are intended to ensure that an evaluation will serve the information needs of intended users

All the information gathered will address the specific evaluation questions (program needs). As the information is subjective in nature (looking at changes in behaviours, perceptions and understandings of stakeholders) the results will be defined and described clearly.

Feasibility Standards are intended to ensure that an evaluation will be realistic, prudent, diplomatic, and frugal

The evaluation procedures will be realistic and practical with voluntary participants (surveys). The interviews will take place outside club hours and surveys can be completed online at the participants’ leisure (within the established time frame).

Propriety Standards are intended to ensure that an evaluation will be conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results.

The evaluation will be conducted legally (FOIP) and ethically; protecting identities of participants. Parents will be notified of student involvement as participation in the evaluation is voluntary.  All results will be shared with  all members who might be impacted.

Accuracy Standards are intended to ensure that an evaluation will reveal and convey technically adequate information about the features that determine worth or merit of the program being evaluated

Data is both qualitative and quantitative and directly involves all stakeholders. As a result, this will limit bias (inclusion of various perspectives) and the findings and recommendations w will yield positive and accurate results.

References:

Capacity Building Resources. (n.d.). Retrieved from http://actionevaluation.org/resources/capacity-building-resources

Chen, Huey-tsyh. Practical program evaluation: theory-driven evaluation and the integrated evaluation perspective. Los Angeles, SAGE Publications, 2015.

Developing Your Theory of Action. (2006). Action Evaluation Collaboratve. doi:10.4324/9780203969380

Evaluation Consultants Supporting Social Justice & Social Change. (n.d.). Retrieved from http://www.actionevaluation.org/

Rainbow Framewok. (2014). Better Evaluation. doi:10.1036/1097-8542.572110

Zarinpoush, F. (2006). Project evaluation guide for nonprofit organizations: Fundamental methods and steps for conducting project evaluation. Toronto: Imagine Canada.