Introduction

A. Effective Schools and Effective School Practices

Research on effective schools and school practices has clearly demonstrated that schools and school systems can become more effective. The literature describing the research defines effectiveness as:

Effective schools/school districts also "add value" to the performance levels of students: The level of student performance increases uniformly the longer students are in school.

Effectiveness levels are those levels which are achieved by the top 15-20 percent of the student population in each performance and demographic category. In statistical terms, effectiveness levels are those levels achieved at one standard deviation above the mean of the cohort population and are, therefore, rigorous levels of achievement.

When the following practices and objectives are implemented in a school system, the probability increases that the school system will be effective in all areas. These "Correlates of Effectiveness" include:

1. Strong instructional leadership provided by the principal.
2. A comprehensive curriculum -- emphasis on learning objectives, learning activities, and appropriate achievement measures.
3. Measurement -- regular monitoring of student achievement and staff/student attendance.
4. A safe, orderly, disciplined, and supportive school climate.
5. High expectations for quality work supported by staff and students.

Schools and school systems which are effective practice behaviors associated with the conditions (correlates) of effectiveness. More importantly, however, these behaviors are understood, are supported, are practiced, and are observable throughout a school system. Such behaviors can be established through effective school training programs and then adapted to each school system environment. It should be clearly understood that conditions for effectiveness are achieved through the decisions made by adults in the school community and are fully under their direction and control. To this end, effective schools and school systems forge extensive and positive linkages between communities and school.

B. The Cohort Group
The Process of Matching "Means" Fair Comparisons

An Audit of Educational Effectiveness examines the degree to which the student and staff populations in a school district meet levels of effectiveness. Each school district population is examined against populations nationwide with similar demographics, some of which are currently achieving educational effectiveness and which have firmly established the conditions of effectiveness. The SchoolMatch approach compares student populations rather than school systems in order to provide more fair, accurate, and comprehensive cohort comparisons.

SchoolMatch maintains databases on each of the nation's over 15,800 operating public school systems and more than 14,000 accredited private schools. From the SchoolMatch database of public school populations, other student and staff populations in the United States with similar demographic characteristics to the population in the Marion County School District were identified (this large group is the Marion County School District cohort group).

Within the Marion County cohort group exist similar populations which have achieved effectiveness in one or more of the areas analyzed by an Audit of Educational Effectiveness. Also within the cohort group are student populations which perform at or below the average (mean) for the entire group. In order to form the basis for comparison with these demographically similar populations, SchoolMatch calculates the average (mean) performance level for the cohort group in every category analyzed in the audit process. This demographically-matched set of populations (the cohort group) can be described in terms of the average (mean) in each performance area. As a result, we may refer to the cohort group as the mean-matched group.

Once the averages in each measurable category are calculated, the cohort group is now mean-matched. Once the mean for each category is determined, the effectiveness level (one standard deviation above the mean) can be calculated.

C. The Audit of Educational Effectiveness Process

An educational effectiveness audit may include most or all of the following:

1. An analysis of school system policies.
2. An analysis of test data.
3. An analysis of attendance data.
4. An examination of drop-out rates.
5. An examination of Advanced Placement/honors classes.
6. An analysis of the relationship between grades awarded and other student performance indicators.
7. An examination of the curricula.
8. An analysis of school improvement plans.
9. An examination of the school district technology plan.

The audit also consists of other activities to obtain information about the effectiveness of practices and policies in the school district, including:

1. Interviews with school district administrators, building administrators, teacher leaders, classroom teachers, students, and parents.
2. Observations in a sample number of schools.
3. Administration of perception surveys to a sample number of administrators, parents, and teachers.
4. Examination of state, regional, and national source documents containing data relevant to the school system being examined.
5. Examination of accreditation documents when appropriate.

D. Collection and Analysis of School District Data

Data obtained from data collection forms, documents reviewed before and during the on-site visit (Appendix A), and aggregated responses from perception surveys completed by a sample of parents, teachers, and administrators are verified with information obtained through interviews. Reliability is established through professional judgments. Validity of effective school practices, and the ensuing benefits to school districts that use them, has been established through the continuing observations and analyses of educational researchers. These research data are readily available.

E. The Purpose of an Audit of Educational Effectiveness

The central purpose of an educational effectiveness audit is to identify areas of school system operation or system results which can and should be improved. This identification can serve as a basis for school system improvement and for the allocation of resources. The audit may also make recommendations based on the effectiveness of practices in school districts serving similar student populations.

In addition, the audit will assist the school system to:

1. Examine school system leadership, structure, and policy development.
2. Examine appropriateness of school system goals and objectives.
3. Analyze pupil performance information and establish the degree to which schools are effective.
4. Analyze the perceptions of parents, teachers, and administrators toward existing school practices and their overall effectiveness.
5. Establish appropriate school improvement programs.
6. Establish accountability structures to validate school improvement.

F. A Sample of Student Populations Demographically Similar to the Population of the Marion County School District

Among student populations in the more than 15,800 public school systems in the United States, the school systems listed here share similar primary demographic characteristics in the categories of education level of school district residents, per capita income, and poverty level of the community. Secondary qualifiers include relative size of the student population and the percent of the student population qualifying for free or reduced-price lunch.

The school systems listed here may not be identical to Marion County in all primary or secondary demographic categories. At least part of the student population in each district represents the closest overall match, nationwide, to the student population of the Marion County School District.

NAME | CITY, STATE
1. Colton Joint Unified | Colton, CA
2. Pomona Unified | Pomona, CA
3. San Bernardino City | San Bernardino, CA
4. Hartford School District | Hartford, CT
5. Bibb County School District | Macon, GA
6. Richmond Co. School District | Augusta, GA
7. Calcasieu Parish School Board | Marion Charles, LA
8. Springfield School District | Springfield, MA
9. Flint City School District | Flint, MI
10.Ector County ISD | Odessa, TX

We believe that discussions between leaders of the school systems listed and leaders in the Marion County School District relative to effective school practices may provide information about how demographically similar school systems manage their resources in pursuit of effectiveness.

G. Levels of Achievement

There are four levels of achievement in effectiveness auditing:

1. Basic: the minimum which will be accepted.
2. Normative: achieved by the average of the group.
3. Effective: levels achieved by the top 15-20 percent of a mean-matched group of student populations (one standard deviation above the mean).
4. Absolute: levels achieved by only a few student populations. The achievement of 100 percent average daily attendance, for example, would be achievement of an absolute level.

Information related to various levels of achievement is provided in Appendix B. Although researchers differ widely on the meaning and use of test data, most agree that test data (or student performance assessments) are one important measurement of the degree to which schools and school systems are effective.