Process
Evaluation Plan

Step 1: Program Overview

Today Matters: Health is in Your Hands, is a nutrition education program helping Boston area college women build healthy behaviors for lifelong wellness. Delivered in person on college campuses, the program aims to improve nutrition knowledge, promote physical activity, and increase the number of 18-24-year-old women who achieve and maintain a healthy weight. This voluntary semester-long program will offer evidence-based instruction on nutrition, cooking, and fitness. Informal small group discussions will be offered in residence halls and in an online forum to provide social support during the semester. Developed using the Health Belief Model,

Today Matters will teach participants how recommended behavior changes can reduce risks to their health today and provide benefits for years to come (McKenzie et al., 2017, pp. 162-3). Constructs of Bandura’s Social Cognitive Theory are also foundationally important to this program. Hands-on skill building will instill both behavioral capability and self-efficacy to empower young adult participants to establish autonomy over their lifestyle decisions and wellness outcomes (McKenzie et al., 2017, pp. 177-8). To implement the Today Matters program, partnerships with be forged with Boston-area colleges. College staff and independent specialists in nutrition and fitness will be recruited to create the intervention team.

Process objectives include:

One month before the start of the program, program facilitators will gather anthropometric data, administer a food frequency questionnaire, and administer a pre-test to participants.
One month before the start of the program, program facilitators will create a rotating schedule of classroom, fitness, and cooking classes.

Impact objectives include:

After completion of all nutrition classes, participants will explain the importance of nutrient-dense foods and be able to design meals with a balanced macronutrient composition.

After completion of five cooking classes, participants will be able to prepare 3 simple meals using dormitory or apartment kitchen spaces.
After completion of the fitness program, participants will be able to explain the importance of physical activity and document personal activity levels.

Outcome objectives include:

As a result of program participation, college women will reduce their risk for overweight and obesity.
As a result of the program, college women will create lifestyle habits for lifelong wellness.

Step 2: Complete and Acceptable Delivery

The ideally implemented Today Matters: Health is in Your Hands will include four components including a supportive environment, a nutrition and fitness education curriculum, a training component, and social support. Complete and acceptable delivery of this program includes establishing a partnership with each college to use space, recruit instructors, and recruit participants. Support of college administrators and stakeholders will provide an environment of an aligned mission of important goals for the student body.

Step 3-5: Process Evaluation Questions

Process evaluation is important to planners and stakeholders to gain an understanding of why a program was or was not successful in achieving its goals. Answers to evaluative questions provide information about needed changes before launching a subsequent intervention (Saunders et al., 2005). Suggestions for questions that may be useful in evaluating the Today Matters program are below.

Process- Evaluation Questions

Target of Question

Recruitment

Reach

Fidelity

Context

Dose Delivered

Dose Received

Process-Evaluation Question

What planned/actual procedures were used to attract participants to program?

*What were barriers to participation in the program?

*What percentage of registrants participated in each program session?

What percentage of registrants attended at least half of the program sessions?

How many in eligible priority population signed up for subsequent offering of program?

*To what extent was the program implemented as planned?

To what extent was training provided as planned?

What environmental or administrative factors on campus affected delivery of the program?

Did social pressure influence participation of college women positively or negatively?

What psychosocial factors in the campus community may have affected the intervention outcome?

Were all planned program sessions delivered to the participants?

To what extent was the intended conten covered in the sessions?

To what extent were classroom materials used?

To what extent were program participants actively engaged in each instructional session?

How did participants react to the cooking demonstrations?

How did participants react to the fitness sessions?

Method of Assessment

Registration form question.

Post-program focus group

Sign-in sheet/roster at all sessions turned in to TM team weekly.

Interest waitlist page on TM website.

Instructors report to team weekly; one supervisory observation by TM leader per module.

Checklist of expected characteristics of implementation.

Midpoint and program end survey of key administrators, dean of student life, cafeteria staff, and fitness center staff.

Participant interviews.

Instructor interviews.

Brief teacher reports via email after each session.

Teacher report via email identifying missed material.

Tracking of qty. materials distributed compared with qty. materials used per lesson.

Satisfaction scales administered to students. End of module interview with instructors.

Satisfaction scales administered to students. Interview with chef.

Satisfaction scales administered to students. Interview with fitness director.

Resources Required

Program evaluation team

Program evaluation team;recording device; transcriber.

Clipboard with registrant names; TM staff calculates percentage.

Calculations based on sign-ins.

Website manager tracks interest based on names on webpage.

Self-reported checklist; observation form; TM staff.

Data analysis by TM staff

Online survey; TM staff to analyze data.

TM staff

TM staff tracks session with complete delivery.

TM staff logs shortfalls in record.

Ledger tracking of materials created, distributed, consumed, lost/discarded.

TM staff

Step 6: Key Process Evaluation Questions

Questions from the collection above indicated with an asterisk will be key to broad consideration of the success or failure of the program. First, assessing fidelity to program implementation will establish the adherence to theory-based constructs that were foundational in the development of interventions. Instructors will self-evaluate after each session delivered using a checklist of lesson objectives. This feedback to the team will establish whether the appropriate quantity of information was able to be delivered in the time allotted. This evidence will reinforce that lessons should continue as designed or be modified before the next session. This is an example of program monitoring which served as preventative maintenance to ensure successful program delivery (McKenzie et al., 2017, p. 334). Since the goal of a health education program is improving health outcomes for individuals, collecting data on the percentage of participants who participated in each program session provides program planners with a quantitative measure of impact on the priority population. If 25 individuals are expected to attend the first lesson of Optimal Eating, the nutrition education module, but only 10 arrive, information is being taught to less than half of the population. The lesson plans in this program are stacked, meaning that content coherence relies on previous instruction: Lesson 2 will build upon the content from Lesson 1 (Park et al., 2023). Due to this, tracking attendance will be important for supporting program goal achievement for all. Collecting qualitative data about what barriers to participation college women are facing will provide understanding and insight (McKenzie et al., 2017, pp. 375-6). Questions that may be asked during focus groups include:

Were sessions scheduled at convenient times?
Did sessions feel like a break in your day or like an extra class?
Did the information in these sessions seem relevant?

Understanding how barriers persuaded or dissuaded participation will allow planners to modify marketing messages to encourage involvement. The combination of these process evaluation questions will highlight important factors that may need addressing to ensure program success.

References

Contento, I. R. (2016). Nutrition education: linking research, theory, and practice.

McKenzie, J, Neiger, B., & Thackeray, R. (2017). Planning, Implementing, and Evaluating Health Promotion Programs: A Primer. 7th ed. Pearson Education, Inc.

Park, V., Kennedy, K. E., Gallagher, H. A., Cottingham, B. W., & Gong, A. (2023). Weaving and

stacking: How school districts craft coherence towards continuous improvement. Journal

of Educational Change, 24(4), 919-942. https://doi.org/10.1007/s10833-022-09471-5

Saunders, R. P., Evans, M. H. & Joshi, P. (2005). Developing a Process-Evaluation Plan for

Assessing Health Promotion Program Implementation: A How-To Guide. Health

Promotion Practice, 6(2), 134-147.

Process Evaluation Plan

Process- Evaluation Questions

Process
Evaluation Plan