The Healthy Schools program conducted by the Alliance is being evaluated in this paper. The program is jointly implemented by the American Heart Association and the William J.Clinton Foundation with the Robert Wood Johnson Foundation assisting with funds (Deck and Block, 2008). The Healthy Schools Program is just one of the four conducted by the Alliance. The increasing incidence of overweight children or childhood obesity over the years had been worrying parents and policymakers alike.
The problem of childhood obesity
Childhood obesity has been addressed as a major problem in the United States. Statistics show an incidence of one in five children possessing this ‘illness’ of obesity. Children between 6 years and 11 years showed a steady annual increase from 6.5% to 18.8% from 1976 to 2004. The children of ages 12-19 showed a rise from 5% to 17.4 %. (Deck and Block, 2008). Childhood obesity has given rise to complications in childhood itself which happen to be hypertension, dyslipidemia, and diabetes mellitus (Science Daily, 2009). The number of children who are taking treatment for such illnesses has increased over the years from 2004-2007. Teen pregnancies are also found more in obese female adolescents who had increased body weight, body mass index, hip circumference, and percentage of body fat (Science Daily, 2009). Another hazard that can be visualized is the large number of children taking medications that previously were meant only for adults (Lieberman et al, 2009). A potential problem is the misuse of medications. An awareness program to curb the problem, conducted in the Netherlands, was fairly successful in that even after 12 months, the children who participated drank lesser sweet beverages. After 20 months, the skin fold measurements of the girls remained normal and did not show unfavorable increases (Singh, 2009). The Healthy Schools Program is a timely intervention to reduce the problem of childhood obesity and thereby the associated illnesses.
The reasons for instituting the Healthy Schools program
The objective is to stop the rise in childhood obesity in the US in 5 years’ time and perhaps reverse the situation in another 10 years (Deck and Block, 2008). Providing healthy environments is deemed an efficient manner of elevating the well-being of children. Changes are to be encouraged through three techniques to help the schools develop healthier standards for foods and beverages: physical development, technical assistance, and mini-grants. Physical development through special programs for physical education and activities is being motivated. Technical assistance is provided and schools that choose to be in the program are given resources and online tools. Mini-grants are being provided to schools that lack resources and which should not withdraw for want of funds. Schools that meet the demands of the program are specially recognized. An evaluation was performed by Dennis Deck and Audrey Block of the RMC Corporation and their findings have been incorporated into this paper
The instruments used were
- The inventory
- An online school self-report survey for collecting data on school policies and practice
- 2numbers of Student measures developed by RMC Research
- Site visit instruments
The before-after quasi-experimental evaluation was done in this program. As the program was inclusive of only one pre-test and one post-test every year, the schematic representation would be O X O (Campbell and Stanley, 1966). The pre-test was done as a baseline inventory and the post-test was done after one year. If the test is repeated for the same group of students every year, the schematic representation becomes OXO.O.O.
Out of 285 schools, 230 were willing to participate but only 187 schools completed the basic data inventory online. The pilot schools were selected by some criteria. 50 % of schools were those which were conducting physical activity programs. 75% had free and reduced rate lunches. There were representations from all ethnic groups: 25% of schools had African American students and 25% had Hispanic students. 5 % each had Native American students and Asian students. There were equal distribution of schools among the rural, suburban and urban areas (Deck and Block, 2008). 40% had Caucasian students.
Level of detail
The effect of a program or policy or treatment or intervention is usually estimated by doing a study with an evaluation design. This is normally achieved by comparison of two or more situations, some incorporating the program or intervention and others not having the intervention. Research-based evaluation is done by two methods: randomized or quasi-experiments (Reichardt & Mark, 2004, p. 126). ). The randomized study design involves participants who are selected randomly for treatment conditions. The quasi-experimental evaluation incorporates participants who are selected for treatment nonrandomly.
This study has utilized the quasi-experimental before-after type of evaluation for assessing the Healthy Schools Program. In order to perform such an evaluation, a comparison is being done to find out what beneficial changes have occurred in a population of students who had obesity as a major problem (Reichardt, 2004).
The evaluation design selected here appears appropriate for the concerned program. All the students have been taken as participants and therefore it cannot be of a random design. The quasi-experimental design is more acceptable to the participants and any amount of interference by the researchers would not harm the ongoing Healthy Schools program and has been rightly selected for the evaluation (Reichardt & Mark, 2004). Quasi-experimental evaluations are of four types: before-after, interrupted time series, nonequivalent group, and regression discontinuity designs (Reichardt & Mark, 2004). The before-after and the interrupted time series make comparisons over a period of time. In the nonequivalent group and the regression discontinuity design, the comparisons are made across the participants (Reichardt & Mark, 2004). This quasi-experimental research with a before-after design has been productive in that it has uncovered a large amount of data in this evaluation. The evaluation before the treatment is the pretest and the one after is the post-test. The before-after comparison is easy to implement and the results published would be easily accepted by readers (Reichardt & Mark, 2004). However, biases or threats to internal validity are a problem here (Shadish, Cook, & Campbell, 2002).
The increasing incidence of overweight children or childhood obesity over the years had been worrying parents and policymakers alike. The objective of the Healthy Schools Program is to halt the weight increase in 5 years and reverse the situation in 10 years.
Out of 230 schools willing to participate, 187 completed the basic data inventory. The instruments used included the inventory which gave the baseline data. The inventory had 34 items in 8 scales. Internal consistency reliability is good at 0.6-0.8 range with 34 items. Three scales show low reliability for the physical education, physical activity, and after-school program scales probably because most schools scored high. The eight scales were highly correlated but not perfectly (Deck and Block, 2008). Factor analysis seemed optimal on a plot of eigenvalues. The five factors of staff wellness, beverage and food standards compliance plus physical activity, school wellness council, reimbursable meals, and health education and physical education were mainly investigated which can be considered necessary for the wellbeing of the children. The items varied along several dimensions reducing the internal threat. The instruments used were the “Healthy Schools Survey” and the “Height and Weight Form” (Deck and Block, 2008).
Student health behaviors which included the behaviors of eating and physical activity were measured using the Healthy Schools Survey. The Height and Weight form provided the data for body mass index calculation. These measures were distributed in the schools selected for intensive study (Deck and Block, 2008). These instruments were also used for getting feedback from the students (Deck and Block, 2008).
The site view instruments were maintained to collect feedback about the interviewees understanding of the Healthy Schools program about the physical education and wellness programs. The interviews and focus group discussions were conducted in the presence of teachers, health education specialists, Principals, Healthy Schools Program coordinators, food service managers, food service directors, and staff for physical education. The credibility of the program was evident in these discussions.
The majority of schools were able to secure a high score on at least one scale However few got the bronze. The low baseline inventory scores showed the possibility of improvement. Many schools attained the gold where competitive foods were concerned (44%). After-school programs also showed a big response in 47% of schools. Most of the schools could not score high on two or more scales to achieve the bronze or higher. Reimbursable meals, health education, physical education, and staff wellness were the scales with the lowest scores (Deck and Block, 2008).
Ethnicity appeared to be a predictor of results. There were only a few schools of Native American students but they were the ones that performed well apart from the Asian students’ schools. The poorest were those of the African American students. Certain inventory items showed good results. The differences in school type were seen from the health education and physical education scales. Two responses to the inventory were recorded. Some schools did only the pre-test and others did both while 37% did not meet the criteria for including their results in the calculations in the first year of the program. 117 schools or 63% completed both tests. The redesigned inventory expected more updates from the schools.
The pre-interview information request form, interview protocol, and school wellness council focus group protocol were the instruments in the Healthy Schools Survey. Information and recording of the school or district wellness plan, health and physical education curricula, and cafeteria menus as understood by the interviewees were done. The interviewees’ “satisfaction with the food, physical education and activity, and health education in the school; and improvements to nutrition and physical activity in the school” were collected in the interview protocol (Deck and Block, 2008). Information about the goals was collected from the school wellness council focus group protocol.
The inventory asked for only the latest results. This meant that results had to be extracted and saved frequently by the Research staff. The Inventory thereby had to be revised. This meant that this would cause an instrumentation threat for internal validity. (Reichardt & Mark, 2004). Scheduling sessions for training for the research was not done in a timely manner. This also posed a threat or internal validity.
The results collected so far were not complete. Moreover, this system was a little complex for use by the staff and the inadequacy may add another threat. The revised inventory helps the collection of data in an improved manner. However, the evaluation becomes difficult and it would make the change from the baseline difficult to assess. As no control group has been arranged, the comparison would not be possible. The results towards the second year of the program could become better because more data would be collected by the revised inventory though this would be an instrumentation threat for internal validity. The reliability of results would therefore be questionable. The RMC strategy has arranged a manner of dealing with the change in versions of the inventory but would not be making any more changes this time.
Changes made after site visits
The District Wellness Policy was started in 2006 and quality and competitive food and drinks were distributed to the school students and staff. More of whole grain foods, fruits and vegetables were served. Locally grown food was encouraged. Most of the cafeteria offerings were prepared at the venue itself. Vending machines offered only low-fat, low-sugar food, 100% fruit juices, spring water and flavored water. Two types of diet soda were also provided through the vending machine. The staff vending machines provided varieties more to their taste. Barriers included the difficulty to obtain fruits and vegetables, low-fat cheese and other health foods consistently. Physical activity for the elementary students was provided 4 times a week while it was 2 times and 3 times every alternate week for middle school and 5 times a week for high school. Barriers were the poor school resources and budget cuts. Health education classes and specific topics for them were incorporated. Staff wellness was reinforced with their suggestions and included yoga, walking, aerobics and weight-loss programs.
Useful suggestions have been provided for future success. The trans fat in food will be eliminated. Food service staff will be trained on techniques to reduce fat and sodium and provide more whole grains. The after-school walking program is being encouraged by providing incentives to teachers and students who participate.
Issues noted in the evaluation
Implementation issues of the inventory were seen. The pretest and post-test were to be done at the beginning and end of the year. It was difficult to enforce this. Another practical problem was that the system saved only the updated version of the inventory so the computer staff had to save the current version every month. Revision of the inventory may solve the problem. Anther drawback was the slow startup seen in some schools due to the delay in the training for technical assistance. Participation is never complete so automation is being introduced for data collection which may make evaluation a little difficult as baseline data may change. Attempts are being made to ensure the pretest results at the beginning of the years and post-test results at the end of the year. Recruiting new schools into the program is difficult now but should be improved later. Offering incentives may help. Recruitment process takes about 4-6 months now. Redesign of the inventory is the main innovative change.
The evaluation of the Healthy Schools Program has highlighted the limitations and made sufficient recommendations for improvement of the program in future. Yearly assessment and due innovative changes may take the program to heights, paving the way for achieving the initial objective of reducing childhood obesity and thereby childhood complications, morbidity and mortality.
- Campbell, D.T. and Ross, H.L. (1996). Experimental and quasi-experimental designs for research, Skokie Ill: Rand, McNally, 1966
- Deck, D. and Block, A. (2008). Healthy Schools program evaluation, Alliance for a healthier generation, RMC Research
- Guerin, D and Mackinnon, D.P. An assessment of the Impact of the California Child Seat Restraint Requirement American Journal of Public Health, 1985, Vol 75.
- Liberman et al. (2009) Prevalence of Antihypertensive, Antidiabetic, and Dyslipidemic Prescription Medication Use Among Children and Adolescents. Archives of Pediatrics and Adolescent Medicine, 2009; 163 (4): 357 DOI: 10.1001/archpediatrics.2009.5
- Science Daily, 2009. Web.
- Shadish, W.R. and Cook, T.D. and Campbell, D.T. (2002). Experimental and quasi-experimental designs for generalized causal inference Boston, Houghton Mifflin
- Singh et al. (2009). Dutch Obesity Intervention in Teenagers: Effectiveness of a School-Based Program on Body Composition and Behavior. Archives of Pediatrics and Adolescent Medicine, 2009; 163 (4): 309 DOI: 10.1001/archpediatrics.2009.2
- Reichardt and Mark in Wholey, J.S., Hatry, H.P., Newcomer, K.E. (Eds) (2004). Handbook of practical program evaluation, 2nd ed. San Francisco: Jossey-Bass