Evaluating the National Outcomes: Children

Children must be present to benefit from the program experiences you provide, and their level of participation will affect how much they benefit. Even though Presence and Participation are not child outcomes in the traditional sense, they need to be measured as part of a careful evaluation of community-based children's programs. The NCEO model adopted by the Children's Program Outcome Workgroup includes Presence and Participation as one of two outcome domains that fall into a special category called the Learning and Opportunity Process (Ysseldyke & Thurlow, 1993; Ysseldyke, Thurlow & Erickson, 1993; Ysseldyke, 1994a & 1994b).

On one hand, Presence and Participation are causal or “independent” variables because they affect the degree to which certain outcomes are achieved; that is, children who do not attend the program regularly are not likely to benefit from it. On the other hand, they are also outcome or “dependent” variables because they are affected by the quality of the program experience; that is, children are unlikely to attend regularly if they do not enjoy the program or see it as helpful. Weiss (1988) recommends paying more attention to program participation data to answer questions about who is attracted to, and remains in, what type of programs. This kind of data can help CYFAR administrators answer policy questions regarding whether funded programs are attracting and keeping hard-to-reach or at-risk families (Blank, 1987). In the five-tiered model of program evaluation which underlies the State Strengthening Evaluation Guide (Jacobs, 1988; Callor, Betts, Carter, & Marczak, 1997), participation information falls in the accountability tier. This information is needed to document how well the program is used and accepted by the target audience.

Most frequently, Presence and Participation has been treated as a single static variable, with participants either present or absent (Powell, 1988). Having data and information to support the following indicators makes it possible to treat Presence and Participation as a multi-dimensional variable (being present every day, versus being there occasionally, or once a week, or twice a week.) In other words, we can ask how much program participation is necessary to make a difference in desired outcomes?

The NCEO model, as adapted by the Children’s Outcome Workgroup for community programs, suggests three simple but important desired outcomes in this domain:

The child is enrolled for the school, program, or activity experience
The child participates actively in planned group activities
The child who is enrolled attends regularly and completes the program

The following are some appropriate indicators of positive program outcomes for children in the area of presence and participation, based on the NCEO model (Ysseldyke & Thurlow, 1993), as adapted for community-based programs by the Children’s Outcome Work Group. The appropriateness of any given indicator for your program evaluation depends on the age of the children you serve, the setting, and the goals and activities of your particular program.

Number of children (or percent of eligible children) enrolled in program
Demographic profiles (ethnicity, gender, income, geographic residence, etc.)
Attendance rates (may be broken down by demographic categories)
Percent of children dropping out (may be broken down by demographic categories)
Program completion rates (may be broken down by demographic categories)
Percent of children who actively engage in planned program activities (based on surveys or interviews with the child, parent/guardian, or teacher/program staff)
Percent of children participating in family activities
Percent of children participating in other community activities
Percent of children who have perfect attendance
Percent of children whose absenteeism decreases over time

Summary

Although these are not traditional children’s outcomes, State Strengthening projects need to keep accurate records of program attendance, and of level and quality of participation in the program. These records are part of basic program accountability, but they also provide valuable information to program managers, evaluators, and researchers. They help to answer important questions about who is attending the program, who is dropping out, whether the program is reaching the intended audience, and how much participation is needed before we can expect to see the desired results in children. The information may also help guide refinements of the program to better reach the children who need it most.

Blank, S. (1987). Contemporary parenting education and family support programs: Themes and issues in an emerging movement. New York: Foundation for Child Development.

Callor, S., Betts, S., Carter, R., & Marczak, M. (1997). State Strengthening Evaluation Guide. Tucson, AZ: University of Arizona, Institute for Children, Youth & Families and USDA/CSREES.

Jacobs, F. H. (1988). The five-tiered approach to evaluation: Context and implementation. In H. B. Weiss & F. H. Jacobs (Eds.), Evaluating Family Programs (pp. 37-71). New York: Aldine De Gruyter.

Powell, D. R. (1988). Toward an understanding of the program variable in comprehensive parent support programs. In H. B. Weiss & F. H. Jacobs (Eds.), Evaluating Family Programs (pp. 267-285). New York: Aldine De Gruyter.

Weiss, H. B. (1988). Family support and education programs: Working through ecological theories of human development. In H. B. Weiss & F. H. Jacobs (Eds.), Evaluating Family Programs (pp. 3-36). New York: Aldine De Gruyter.

Ysseldyke, J. E., & Thurlow, M. (1993, October). Developing a model of educational outcomes (NCEO Report No. 1). Minneapolis, MN: University of Minnesota, College of Education, National Center on Educational Outcomes.

Ysseldyke, J. E., Thurlow, M. L., & Erickson, R. N. (1993). Educational outcomes and indicators for early childhood (Age 6). Minneapolis, MN: University of Minnesota, College of Education, National Center on Educational Outcomes.

Ysseldyke, J. E. (1993). Educational outcomes and indicators for early childhood (Age 3). Minneapolis, MN: University of Minnesota, College of Education, National Center on Educational Outcomes.

Ysseldyke, J. E. (1994). Educational outcomes and indicators for grade 4. Minneapolis, MN: University of Minnesota, College of Education, National Center on Educational Outcomes.