For professional developers from professional developers

North Central Regional Educational Laboratory Issue 7, Fall 2004

ABOUT
IN THIS ISSUE: NOTES & REFLECTIONS

Using Data to Guide School Improvement Welcome to Notes & Reflections, our
newsletter for professional development

A lthough certain provisions of the No Child Left Behind Act of 2001 remain con-
troversial, there is broad agreement that measuring student progress and setting
specific goals are fundamental to school improvement (Schmoker, 1999). How do
providers. Notes & Reflections offers
practical information—ideas, strategies,
tools, and resources—about topics of
educators measure progress and set goals? The process begins with data. special interest to professional devel-
The ability to track performance is as valuable for educators as it is for doctors, busi- opers who are working to improve
nesspeople, and competitive athletes. Yet education leaders often lack formal training school performance.
in data analysis. They may suffer information overload when they attempt to draw Today, professional development
conclusions from dozens of variables. To remedy the situation, educators need a services to schools are provided by
process for data-driven decision making that helps them focus on the essential pieces variety of people: teacher leaders;
of information to identify priority areas and select realistic goals. principals; and district, regional, and
This issue of Notes & Reflections provides a recipe for school improvement based on state education staff—to name just a
data exploration. The main ingredients are the data themselves: useful information few. If you are one of these providers,
about schools and students. The recipe has four steps: Notes & Reflections will help you build
1. Prepare for data exploration. The process begins with promoting a data culture your capacity to plan and implement
and forming a team that will examine the data. powerful professional development.

2. Collect and organize the data. The team identifies the relevant data, gathers it Notes & Reflections is published
together, and organizes it for analysis. electronically. It is available for
viewing and downloading through
3. Analyze the data. The team reviews the data and looks for important patterns. It
the Notes & Reflections Web site at
identifies specific programs or student groups most in need of improvement.
www.ncrel.org/info/notes/.
4. Explain the results. The team uses data to generate hypotheses and explain the
underlying causes of the areas needing improvement. If you wish to be added to our electronic
mailing list and notified when the
Upon completion of these four steps, educators are prepared to use their data to
next issue is available, please send
inform school improvement efforts.
your name and e-mail address to
notes@contact.learningpt.org
CONTENTS or call 800-356-2735.
• Step 1: Prepare for Data Exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2
• Step 2: Collect and Organize the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
• Step 3: Analyze the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 Michele Fitzpatrick, Managing Editor
• Step 4: Explain the Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
• Next Steps: Proceeding From Data to Action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13 Jonathan Margolin, Contributing Writer
• References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
• Additional Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
• Tools and Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15

Learning Point Associates was founded as the North Central Regional Educational Laboratory (NCREL) in 1984.
NCREL continues its research and development work as a wholly owned subsidiary of Learning Point Associates.
 Using Data to Guide School Improvement

STEP 1: PREPARE FOR DATA EXPLORATION

Also, see Additional Reading on page 13 for publica-
D istrict and school leaders are responsible for
creating an organizational culture that values the
use of data and for forming the team that will focus on
tions that explain how and why to use data for school
improvement.
data exploration. The professional developer can help
direct their efforts. Form a Data Exploration Team
Data exploration is a task that is best accomplished by a
Promote a Data Culture broad-based team of educators. A team effort is crucial to
In schools with a data-friendly culture, teachers and the success of data-driven decision making for the follow-
administrators seek reliable data to inform their decisions ing reasons:
about curriculum and instruction. Fostering such an atmos- • There is too much work for any one person to do.
phere is a gradual process (Holcomb, 1999). A first activ-
ity for professional developers is to explain the purpose of • Input from multiple points of view is useful in order to
data: to identify goals for school improvement. A second interpret the data and brainstorm solutions.
activity is to encourage administrators and teacher leaders • A broad-based approach builds consensus and buy-in,
to reflect on how they make critical school decisions. which helps sustain the decisions that are to be made.
An excellent resource for introducing educators to the The experience of staff at Learning Point Associates is
practice of data use is the video and print package titled that the ideal data exploration team size is six or seven
Data Exploration: A Journey to Better Teaching and members. For each school, the team must include the
Learning. It’s listed in Tools and Resources on page 13. principal and two to three teacher representatives from
various grade levels. It is also desirable for the team to
have a representative from special-education teachers and
Tip: Assess Your Data Savvy noncore teachers (e.g., the art or music teachers). A district
representative on the team helps the school understand the
The following questions (adapted from Sargent, resources its district can offer and helps the district keep
2003, p. 11) help to raise awareness of the need in touch with the needs of the school. In addition, a tech-
for data-driven decision making: nology expert is quite helpful.
1. How do we currently establish our improvement Because membership on the data team can be a lot of
goals each year (for the district, a school, or a work, consider rotating the membership on an annual
program)? Describe the process.
or biannual basis. Members must be willing and able to
2. Generally describe the achievement levels of the meet regularly.
students in your building or program.
3. What academic areas are in greatest need of
improvement? With what content areas within each
subject are students having the most difficulty?
Tip: Include a Technology
On what basis do you answer these questions? Expert on the Team
4. In your opinion, what are the primary factors Many schools find it useful to involve the technology
within the school’s control that cause low achieve- coordinator or whoever is experienced with spread-
ment in the areas of greatest need? What data sheet software. The process of interpreting and
have you examined to support your opinions? graphing data goes much faster when one team
5. What do you hope that the school or district member is familiar with features of spreadsheet
will accomplish by data-driven decision making? applications such as data filters and pivot tables.

2 Learning Point Associates

 Using Data to Guide School Improvement

STEP 2: COLLECT AND ORGANIZE THE DATA

• What are our academic programs? Which students are
T he second step is collecting and organizing the data.
Four types of data can be used as indicators of
a school’s success and progress: achievement data,
involved in each?
• How successful are our programs in attaining the aca-
demographic data, program data, and perception data. demic standards?
(See “Data Checklists” on page 4 for specific examples
of critical data.) Perception data. Perception data provide information
on the attitudes and beliefs of various stakeholders in
Types of Data the school, such as teachers, students, and parents. Some
categories for which perceptions are helpful include
Achievement data. Achievement data provide information academic standards, school leadership, quality of instruc-
on student learning and achievement. These data include tion, and school climate.
standardized test scores, classroom-based assessments,
rubric-scored writing portfolios, and class grades. Perception data are usually collected with surveys that
address the following questions:
Achievement data answer critical questions about student
achievement: • How do parents, teachers, students, and members of the
school community feel about the job our school is doing?
• What evidence shows the knowledge, skills, and under-
standings our students have achieved? • How satisfied are our “customers” with our programs?

• What evidence shows the academic proficiency our In order to have longitudinal data, which identify trends
students have achieved? over time, it is essential to find or create a good perception
survey and to stick with it year after year.
Demographic data. Demographic data provide information
about the characteristics of students: enrollment, student
Organize Data
mobility, attendance, disabilities, ethnic background, limited
English proficiency, economic status, gender, dropout Organization is the key to successful data use. The following
rates, and behavioral problems. The purpose of this data is activities will help your school assemble the data you need
to ensure that the needs of all types of students are being in a useful format.
met adequately and to indicate important demographic Set up a student-level file. Create a spreadsheet file of
trends (e.g., an increase in English language learners). student-level data from this year’s state assessments. Each
Demographic data answer critical questions about the row will be a different student, and each column will be
student population: a different type of data. Start by labeling each column of
data with a descriptive heading. Include in this file achieve-
• Who are our students? What trends do we see in our
ment data as well as all of the demographic and program
student population?
data you have for each student. Table 1 (on page 4) presents
• What factors outside of school help us understand our a sample student data file. (Note: When working with student
students? data, it is important to maintain the confidentiality of the
Program data. Program data provide information on all information. Most states, if not all, have laws that limit
school programs. These data represent the conditions access to student data to school or district staff.)
directly under the control of the school, such as its cur- Organize by grade and topic. After you have created this
riculum scope and sequence, academic programs, teacher basic file, the next activity is to organize it. Because of the
training and experience, professional development, after- sheer amount of data, a single file would be cumbersome.
school programming, and Title I programs. Therefore, it is helpful to break it into smaller files
Program data answer the following critical questions: according to grade and topic.

Issue 7, Fall 2004 3

 Using Data to Guide School Improvement

Table 1. Sample Student Data File

Student Title I Free Lunch English Proficiency Academic Program

Cohen, Jack Yes Yes Yes Vocational
Ismaeli, Mustafa No Yes No Regular
Smith, Maryanne No No Yes Gifted

Data Checklists
Checklist for Achievement Data Checklist for Program Data
REQUIRED REQUIRED

State achievement test summary reports for the last
Lists of students participating in special programs
three years for all grades tested. such as gifted/talented, special education, and Title I.

State achievement test student-level data for the last
Summary of schoolwide professional development
three years for all grades tested. (i.e., staff development plan) for the past two to three

School report card for the last three to four years. years. Include any evaluations or survey results
associated with the professional development offered.
SUGGESTED

School improvement plan, both current and from

Report card summaries: lists of students who are
the past two years.
below a “C” in mathematics and language arts for
Grades 3, 5, 8, and 11.
Dates of significant curriculum revisions (textbook
• Include grade criteria descriptions. implementation, realignment of course sequence)
• Include lists for the last two to three years. in core content areas.

Standardized test results (e.g., TerraNova, Iowa
STRONGLY SUGGESTED
Tests of Basic Skills) with all reports, including
school/district summary reports, objective/strand
Information about curriculum scope and sequence
reports, and student-level data. (e.g., curriculum maps).

Checklist for Demographic Data SUGGESTED

Any program evaluation by external or internal
REQUIRED
evaluators.

School report card that includes results for special-
needs students and demographic groups. Checklist for Perception Data

Demographic data such as gender, ethnicity, subsidized REQUIRED
lunch eligibility, and English language learner status.

Any survey data, in summary form (preferably for
SUGGESTED multiple years).

Lists of behavior/social referrals by grade. SUGGESTED

Student attendance data.
Teacher absenteeism data.

4 Learning Point Associates

 Using Data to Guide School Improvement

STEP 3: ANALYZE THE DATA

1. Create a data summary table that lists the percentage
T he next step is to review and find patterns in the data
that have been gathered. But before this step is begun,
some preliminary planning will ensure that the team’s dis-
of students who are proficient in certain topics across
several years.
cussions are productive.
2. Color-code the data to highlight areas of strength and
weakness.
Before You Begin
Reserve adequate time. Set aside a substantial block of 3. Graph the data.
time for the team to meet. Many schools have found success
4. Record and discuss the major findings.
with a “data retreat” approach, in which they take a whole
day to examine and discuss their data. Use this process for each of the four types of data:
achievement, demographic, program, and perception.
Distribute the data. Upload the database onto the com-
puters that your group will use during your discussion.
You also may want to have the data in paper form. Examine Student Achievement Data
Gather necessary materials for displaying observations The first and most essential part of the process is identify-
and conclusions. When you are ready to begin probing ing areas of strength and weakness in student achievement.
your data, have the following materials on hand to help The following steps can help the team examine student
you to display your data and conclusions: achievement in broad subject areas.

Highlighters (blue, green, yellow, and pink) 1. Find your broad district and school data summaries

Pencils from the state assessments.

Yardsticks and rulers 2. Locate the report that shows the percentage of students
at each level (e.g., below basic, basic, proficient, and

Roll of masking tape (unless chart paper is self-stick)
advanced). Focus on the percentages of all students

Markers for chart paper enrolled who are proficient in a given topic. Do not

Flipchart easel exclude special education or other exempted students,
since the objective is to analyze data for all students

Flipchart paper
enrolled. (Note: For some states, proficiency levels

Laptop computer with Internet access are listed as ranges of “scale scores,” so you might

Calculators need to look for the percentage of students in each scale
score range.)

Lunch, snacks, and beverages
3. Calculate the percentages of all students at or above
The Data Analysis Process the proficient level for each grade and subject. Create a
With that preliminary planning completed, the team is flipchart that lists the data for the past two to four years.
ready to begin the process of data analysis. The goal is Having data from the past four years will allow your data
to identify strengths and weaknesses across the major exploration team to compare how the same cohort of
academic areas, identify specific demographic groups that students (e.g., the class of 2006) performs across different
need academic improvement, and determine which programs grade levels, as well as how different cohorts of students
are meeting or not meeting their goals. Throughout all of perform at the same grade level (e.g., how the fourth
these analyses, there is a basic process for getting to the grade performs across several years of students). Table 2
bottom line of the data. focuses on one grade level for the past four years:

Issue 7, Fall 2004 5

 Using Data to Guide School Improvement

Table 2. Sample Data Chart: Grade 3 Proficiency Levels, 2000–04

Scale Scores
Social
Grade School Year Reading Math Science Studies Writing

2001–01 55% 71% 45% 56% 77%

2001–02 67% 63% 48% 45% 83%
3
2001–03 80% 59% 59% 58% 89%
2001–04 82% 74% 56% 65% 91%

4. Use the stoplight method—with blue, green, yellow, and your expectations. Table 3 defines the meaning of the
pink highlighters—to create a color-coding scheme indi- colors. Your school may choose different cutoff points
cating visually how well the student results are meeting depending on your expectations.

Table 3. Sample Color-Coding Scheme: Grade 3 Proficiency Levels, 2000–04

Percentage of
Highlight Student Proficiency
Color Meaning (cutoff point)

Blue WOW! Beyond expectations. 80% and above

Green GOOD! Meets expectations. 70% to 79%

Yellow CAUTION! Below expectations. 60% to 69%

Pink URGENT! In need of immediate improvement. 0% to 59%

5. Highlight the scores. Use the highlighters to color over Table 4 shows the end result: a striking visual display
each cell of the chart according to this coloring scheme. that indicates areas of academic strength and weakness.

Table 4. Sample Highlighted Data: Grade 3 Proficiency Levels, 2000–04

Social
Grade School Year Reading Math Science Studies Writing

2000–01 55% 71% 45% 56% 77%

2001–02 67% 63% 48% 45% 83%

3
2002–03 80% 59% 59% 58% 89%

2003–04 82% 74% 56% 65% 91%

6 Learning Point Associates

 Using Data to Guide School Improvement

6. Display the data. For each grade, create a graph to • Follow cohorts of students across several grades.
display the proficiency level over several years in each This view indicates whether teaching effectiveness
academic area. Graphing makes data more tangible and within a grade level is improving over time.
communicable.
In Figure 1, the fourth-grade level appears to be growing
There are several techniques for using graphs to look in its effectiveness with reading comprehension much
at longitudinal data. Here are two popular views: more than the other grade levels.

Figure 1. Sample Graph:

eading Comprehension Across Year and Grade
90
80
Reading Comprehension

70
60
2001
Score

50
2002
40
2003
30
20
10
0
3rd 4th 5th
Grade

• Compare the same grade level over time, across Figure 2 displays the percentage of students who met
different subjects. This view focuses on how proficiency standards for eighth-grade math and reading
instruction in various subjects is improving over across several school years. The graph shows that reading
time at a particular grade level. proficiency is improving over time, yet math proficiency
rates are somewhat stagnant.

Figure 2. Sample Graph:

Grade 8 Proficiency in Math and Reading
80
Student Proficiency (%)

70
60
50
40 Reading
30 Mathematics
20
10
0
2001 2002 2003
Year

Issue 7, Fall 2004 7

 Using Data to Guide School Improvement

areas in the columns. (Each chart can represent a specific

Tip: Display Graphs on Chart Paper grade and year.) Color-code each data point, and graph
your important findings.
Putting your graphs on chart paper will enable the
entire team to view and discuss the results together. Most achievement data, however, are not disaggregated by
demographics or program participation. The state reports
can reveal that low-income students show low proficiency
7. Discuss and record major patterns. When you have in mathematics, for example, but it is necessary for each
finished creating the charts and graphs, consider the school to determine the areas of math in which these
following questions: students need particular help. Use the following steps as
• What patterns do we see in this year’s results? a do-it-yourself guide to disaggregating data:
Do we see similar patterns in past years’ results? 1. Locate your student-level data file (which was described
• What trends emerge over the past several years? in Step 2 in the section “Organize Data”). Color-code
Do these trends indicate that we are moving toward the achievement scores for each student, so that scores
our goals? considered “excellent” receive one color, “good” scores
8. Conclude by making a list of summary statements that receive another color, and so on.
describe the major findings. Write the data summary 2. Sort the data or use filters to look at the students in
statements on chart paper. a specific category. For example, you first can display
only those students who received Title I assistance,
Examine Demographic and Program Data and then view only those students who have not. The
Although the broad analysis of achievement data is useful color coding will enable you to compare these two
for describing general student strengths and weaknesses, groups quickly.
it is necessary to delve deeper to understand what these 3. Calculate the percentage of proficient students for im-
results mean. To do so, look at breakdowns of your data portant demographic and program categories. To get a
according to demographics and program participation. longitudinal view, repeat this process for different years
All state assessments now disaggregate a school’s results and different grades.
according to major demographic categories.
4. Create charts and graphs of the data. Table 5 and Figure
Create data charts to summarize these data. List the major 3 display the percentage of students in different academic
demographic variables in the rows and the major subject programs who were proficient in ninth-grade math.

Table 5. Sample Data Chart: Grade 9 Math Proficiency

Program 2001 2002 2003 2004

Accelerated 94% 99% 99% 99%

Regular 66% 65% 81% 84%

Vocational 46% 48% 59% 61%

8 Learning Point Associates

 Using Data to Guide School Improvement

Figure 3. Sample Graph:

Grade 9 Math Proficiency
100

Student Proficiency (%)

80

60 Accelerated
Regular
40
Vocational
20

0
2001 2002 2003 2004

5. Look for patterns in the data. Discuss these patterns and dents who agreed or strongly agreed with the statement
list the observed strengths, weaknesses, and other unique about school safety.
patterns. For example, Figure 3 graphically shows a 2. Chart the responses. Create a chart for all of the items
distinct jump in achievement starting in 2003 for both on the survey to display the percentage of respondents
regular and vocational students. Use chart paper to write who rated the item as satisfactory or better. Table 6
down the important patterns that you observe. compares the responses of different groups.

Examine Perception Data 3. Color-code the results using the stoplight method. For
each item, the team should decide what level of agree-
Perception data verify that stakeholders such as teachers, ment is expected, better than expected, and worse than
parents, and students are satisfied with the way the expected. Then it is possible to color-code each box
school is being run. The guiding question is whether or using the stoplight method described earlier.
not the stakeholders of the school, along with the staff, Variation: If you have too much data or not enough
are satisfied with the way that the school operates. time to analyze each item independently, just calculate
Examine perception data through a similar process as an overall mean for each major section of your survey
with the content areas: (e.g., Vision, Climate, Diversity). Then determine if the
1. Create frequency tables. School surveys often ask people mean meets or exceeds your team’s expectations.
to express their degree of agreement or disagreement 4. Discuss and record patterns in the data. As survey data
with statements about a school, such as “I feel safe in are shared and studied, team members should discuss
our school at all times.” Calculate the percentage of and summarize the patterns they see. Write data sum-
respondents who rated the school as satisfactory or better. mary statements on chart paper. Major problem areas
In this example, the group would consist of the respon- as well as demonstrated strengths should be listed.

Issue 7, Fall 2004 9

 Using Data to Guide School Improvement

Table 6. Sample Data Chart: Perceptions of School Climate

Percentage of Respondents
Who Indicated “Satisfactory” or Better

Item Parents Students Teachers

Teachers let parents know how well their children
68% 61% 88%
demonstrate the core values.

All students and staff participate in creating a

55% 70% 68%
positive school climate.

Diversity is celebrated through a variety of

programs and activities. 43% 59% 74%

Summarize the Analyses 3. List all the problem areas that appeared in the data
analysis. Use group processes to come to agreement on
Now it’s time to pull together all of your team’s observa-
the list, and write it on large paper for all to see.
tions. The goal is to summarize and prioritize the school’s
strengths and problems that have emerged from the data. 4. Delete from the list all problems that cannot be
Before you begin, appoint a discussion leader. Then tape influenced (e.g., student mobility), and those that
all of the graphs and data summary statements to the walls. were not clearly based on data. Review the remain-
1. Summarize the observed strengths, and compile them ing problems to be sure they are stated completely
on large paper for all to see. Acknowledge and celebrate and clearly.
all of the school’s accomplishments. 5. Have each team member rank-order the problems in
2. Brainstorm how to publicize these strengths to the terms of urgency, with the most urgent getting a rank
community. Remind the team that solid data is highly of 1 and so on. Tally the rank orders to create a final list
effective for substantiating claims of school success. that takes into account everyone’s rankings.

Tip: Publicize Your Successful Results to the Community

Data are the best and most credible way to persuade your community of the success of a particular program
or the school as a whole. The following suggestions are helpful for publicizing this success to your community:
1. Use graphs to display successful programs. Post charts displaying year-by-year trends in your school’s
hallways. Include graphs in your school’s newsletter and on its Web site.
2. Target a message to Parent Night by displaying a chart of findings or having a speaker announce successes
to the whole group. For example, administrators at Scott-Morgan Community Unit School District 2 in Bluffs,
Illinois, observed that students with good attendance had higher achievement. In order to drive the message
home, administrators displayed a large chart of the finding during a recent Parent Night.
3. Celebrate. Your teachers and students have worked hard to reach your achievement goals. Celebrate the
success with a schoolwide assembly.
4. Publicize success in local newspapers. The district office can send a press release to the local newspaper
announcing the success of a program.

10 Learning Point Associates

 Using Data to Guide School Improvement

STEP 4: EXPLAIN THE RESULTS

S tep 3 answered the question of “Where are we now?”
The challenge of Step 4 is to determine “How did
we get here?” or, more precisely, “Why are our students
Tip: Analyze Root Causes

Identifying the root cause of problems leads to more

performing the way they are?” In other words, it’s time
strategic, effective interventions. A number of group
to find explanations for the results. For example, there
processes are available for root cause analysis. One
may be a number of possible reasons for low math of these, the Fishbone Group Process, is described
achievement, such as the inadequate alignment of text- in Notes & Reflections Issue 4 (www.ncrel.org/info/
book to standards, the lack of instructional time, or gaps notes/spring03/fish.htm). For an additional resource
in the curriculum. on analyzing root causes, see School Leader’s Guide
to Root Cause Analysis, listed on page 14 of the
The key idea is for team member to meet, ask questions Tools and Resources.
about the data, and discuss possible answers. Whenever
possible, team members should consult additional data to
corroborate their answers.
• A significant number of students are at risk or have
1. Establish ground rules for discussion. It is important special needs that are not being met adequately.
to maintain a safe and respectful environment that is
4. Evaluate the hypotheses. Use data such as curriculum
conducive to discussing professional effectiveness.
maps, textbook adoption records, demographic trends,
records of behavioral problems, and perceptions of
various stakeholders to evaluate the hypotheses for
Ground Rules for Discussion
each problem statement. In the right column of the

The opinion of every team member matters. Hypothesis Worksheet (see Table 7 on page 12), write
evidence that supports or contradicts each hypothesis.

If you think it, say it—respectfully.
Then record the final action: Accept as possible any

All comments should be brief explanation that is supported by evidence or reasoning;
(less than one minute). reject any explanation that is not.

Silence = consensus.

One person talks at a time.

Respectfully listen to each member’s hypothesis.

2. Assign a recorder for the group discussion. This person

will write down the hypotheses as they are suggested.
3. Generate hypotheses. Considering one problem state-
ment at a time (e.g., declining math achievement), team
members take turns stating a hypothesis for it. The
recorder writes down each hypothesis in the left column
of the Hypothesis Worksheet (see Table 7 on page 12).
Hypotheses frequently fall into the following categories:
• The curriculum or curriculum materials do not align
with the standards or the assessments.
• Teachers have not received adequate professional
development to teach to the standards effectively.

Issue 7, Fall 2004 11

 Using Data to Guide School Improvement

Table 7. Sample Hypothesis Worksheet

Problem Statement: Achievement levels in math drop grade-by-grade, until they are at very low levels in
Grades 6, 7, and 8. They pick up slightly from Grades 9 to 12.

Hypothesis Action and Evidence

Our standards are too high. The achievement tests REJECT. Students in neighboring districts, who take
are too difficult, and get harder each year. the same tests, do not show a decline in scores as the
grades progress. We also have studied the items and
concur that they are grade-appropriate.

Our math teachers in the intermediate levels have ACCEPT AS A POSSIBILITY. We looked at the
not had the appropriate training to teach to the licensure, and our teachers do have the appropri-
standards. ate credentials. However, we also examined the
sequence of professional development and there
has been none in the area of mathematics for the
past 10 years.

We have too many students from low-income families REJECT. Students who are classified as economically
who are not prepared to achieve at higher levels. disadvantaged do not perform any worse than
other students.

We have gaps in our curriculum. ACCEPT AS A POSSIBILITY. We examined our

curriculum scope and sequence, and found that we
spend very little time covering the objective areas
on which our students fared poorly.

Adapted from Using Data to Bring About Positive Results in School Improvement Efforts (North Central Regional Educational Laboratory, 2001, p. 41).

NEXT STEPS: PROCEEDING FROM DATA TO ACTION

goals, determine how to evaluate progress, and roll out the
U p to now, this issue has described four critical tasks:
preparing for data exploration, collecting and
organizing the data, analyzing the data, and explaining
plan. This series of steps is the second half of the process
of data-driven decision making. It will be the focus of a
the results. This knowledge will serve as a powerful forthcoming issue of Notes & Reflections.
motivator for school change. As Ralph Grimm, super- For additional hands-on experience using data, school teams
intendent of the Canton Union School District 66 in might be interested in participating in a Data Retreat.
Canton, Illinois, notes, “That chart [of data] is hard to Refer to “Spotlight on Learning Point Associates Data
argue against.” Retreats” on page 14. Also, the interested reader is directed
The challenge for your data exploration team is to turn this to Using Data to Bring About Positive Results in School
momentum into action. The next steps in the process are to Improvement Efforts, a data tutorial that is available online
set concrete goals, develop strategies for achieving those (www.ncrel.org/toolbelt/tutorial.pdf).

12 Learning Point Associates

 Using Data to Guide School Improvement

References
Holcomb, E. (1999). Getting excited about data: How to combine people, passion, and proof. Sargent, J. (2003). Data retreat participant’s guide (2nd ed.). Naperville, IL: Learning Point
Thousand Oaks, CA: Corwin Press. Associates.
No Child Left Behind Act of 2001, Pub. L. No. 107-110, 115 Stat. 1425 (2002). Retrieved Schmoker, M. (1999). Results: The key to continuous school improvement (2nd ed.). Alexandria,
September 15, 2004, from http://www.ed.gov/policy/elsec/leg/esea02/index.html VA: Association for Supervision and Curriculum Development.
North Central Regional Educational Laboratory. (2001). Using data to bring about positive
results in school improvement efforts. Naperville, IL: Author. Retrieved September 15,
2004, from http://www.ncrel.org/toolbelt/tutorial.pdf

Additional Reading
North Central Regional Educational Laboratory. (2000, Summer). How schools use data to help Bernhardt, V. (2004). Data analysis for continuous school improvement (2nd. ed.). Larchmont,
students learn [Special issue]. NCREL’s Learning Point, 2(2). Retrieved September 15, NY: Eye on Education.
2004, from http://www.ncrel.org/info/nlp/lpsu00.htm This book is designed to help schools overcome barriers to the use of data, such as
This issue of NCREL’s Learning Point magazine is devoted to explaining the bene- an unsupportive school culture, lack of understanding about the importance of data,
fits of data-driven decision making. It presents an example of a school that has used and lack of training in the use of data. It clarifies the importance of data, discusses
data to improve student learning. what data to gather, suggests how to use and analyze data for school improvement,
Association for Supervision and Curriculum Development. (2003, February). Using data to and indicates how to communicate about data and the results of analyses. The book
improve student achievement [Special issue]. Educational Leadership, 60(5). Retrieved can be ordered from the publisher (www.eyeoneducation.com).
September 15, 2004, from http://www.ascd.org/publications/ed_lead/200302/toc.html Lewis, M. (2003). Moneyball. New York: Norton.
The entire February 2003 issue of Educational Leadership explores how data use This book describes how data-driven decision making has given the Oakland A’s
can improve student achievement. major league baseball team a competitive advantage over other teams. By vividly
illustrating the benefits of data use for a major league team, this book motivates
interest in exploring data.

Tools and Resources

The following tools and resources facilitate the different steps School and District Report Cards
of data exploration.
Many states and districts have created Web-based reporting systems to provide
school- and district-level data to constituents. The Learning Point Associates
Resource for Introducing a School to Data Exploration ToolBelt Web site contains a Web page of “Regional, State, and District Report
Cards” with links for Illinois, Indiana, Iowa, Michigan, Ohio, and Wisconsin.
Data Exploration: A Journey to Better Teaching and Learning This page is available online (www.ncrel.org/toolbelt/states.htm).
This multimedia package contains a 20-minute video featuring two schools that
have used data and data-based processes to build and maintain school cultures Resources for Explaining the Data
of continuous improvement. In the video, educators and administrators in each
school clearly articulate the ways they use data to improve student achievement. The following resources help to explain the underlying causes of student
achievement patterns, with a focus on curriculum scope and sequence:
The package includes a booklet of three activities to start conversations around
data use in school and the classroom. Ordering information and additional Surveys of Enacted Curriculum
details are available through the Learning Point Associates product catalog
(www2.learningpt.org/catalog/cart/item.asp?productID=93). The Surveys of Enacted Curriculum (SEC) are a research-based tool that
collects and reports data on what content is taught and how it is taught in
schools and districts. The SEC compare this enacted curriculum to what is
Resources for Data Collection and Organization expected by standards and assessments. Additional information is available
at the SEC Web site (www.secsupport.org).
Quality School Portfolio
Notes & Reflections, Issue 4
This free Web-based tool helps schools and districts make data-based decisions.
It consists of two free software applications—a Data Manager and a Resource This issue of Notes & Reflections outlines a process for curriculum mapping.
Kit—that offer schools a solution for collecting and storing student data that It includes the Fishbone Group Process, a tool for analyzing root causes. The
can best inform their practices. The Data Manager is a database that enables entire issue is available online (www.ncrel.org/info/notes/spring03/npsp03.pdf).
users to disaggregate, import, and analyze data. The Resource Kit contains
surveys, questionnaires, and observation protocols to help schools gather Guide for Instructional Leaders, Guide 1: An ASCD Action Tool
information about their school’s climate and instructional practices, safety This 215-page resource by Roland Barth, Bobb Darnell, Laura Lipton, and
and security, parent involvement, professional development, curriculum and Bruce Wellman describes a process of continuous educational improvement
instruction, technology and innovation, and special programs. It is available and includes tools for analyzing root causes. It is available from the Association
online (qsp.cse.ucla.edu). for Supervision and Curriculum Development (shop.ascd.org).
Characteristics of Successful Schools Web Site School Leader’s Guide to Root Cause Analysis
This Web site, maintained by the Wisconsin Department of Public Instruction, This book by Paul Preuss explains the concept and the process of root cause
contains several surveys that can be used to collect perception data. Although analysis and includes numerous real-life examples in the area of school
only Wisconsin public schools can administer surveys from the site, the surveys improvement. It can be ordered from the publisher, Eye on Education
themselves are available for viewing online (goal.ncrel.org/winss/sample.htm). (www.eyeoneducation.com).

Issue 7, Fall 2004 13

 Using Data to Guide School Improvement

Spotlight on school leaders in order to identify and organize the data

Learning Point Associates Data Retreats that they will bring to the retreat.
Another critical issue is appropriate training. “Educators
T he most common barrier to using data in schools is
the lack of time to collect, organize, analyze, and
interpret the information. Educators are often faced with
in general receive inadequate training in the areas of data
use and analysis,” says Chumbley. “Even when they are
the challenge of conducting data review in short meetings provided a block of time to review data, feelings of being
before or after school or as a smaller part of a professional ill-prepared to engage in such work are pervasive.”
development day. For this reason, many schools are opting Data Retreats provide a unique opportunity to address this
for a Data Retreat approach. Instead of tackling the task common barrier. Chumbley describes the two-day session
piecemeal, a Data Retreat is a dedicated block of time as a time to receive training in processes to analyze and
during which a school team accomplishes the major tasks interpret data, as well as an opportunity to engage in those
of data-driven decision making. processes to review school data. “Participants are able to
see the data transformed into meaningful charts and
Learning Point Associates conducts Data Retreats on-site
graphs, which make trends and patterns apparent,” he
at schools and districts. According to Bryan Chumbley,
explains. “This kind of information benefits decisions
senior professional development associate at Learning
about curriculum planning and instruction.”
Point Associates, conducting a Data Retreat early in
the improvement process helps schools examine their Note: Portions of this section originally appeared in What’s Working:
Data-Driven Decision-Making in the Schools from LRP Publications.
data history and clarify their future improvement path. Reprinted with permission. (For more information on What’s Working,
“When schools examine data that tell the story of where call 800-341-7874 or visit online at www.whatsworkingdata.com.)
they have been and where they are, they are better
prepared to make decisions about where they want to To obtain more information about Learning Point
go,” Chumbley says. Associates Data Retreats, to schedule a Data
Retreat for your school or district, or to learn how
One of the first challenges uncovered in a Data Retreat
Learning Point Associates can serve your school
is identifying what data to collect and deciding how to
or district’s professional development needs,
organize it. Sometimes schools struggle because they have call the Learning Point Associates Service and
no data or what they have is inconsistent. To address this Product Order Line at (800) 252-0283.
concern, Chumbley conducts a preliminary meeting with

For more information, contact: P: 630.649.6500 > F: 630.649.6700

800-356-2735 > www.learningpt.org
1120 East Diehl Road, Suite 200
Naperville, IL 60563-1486

Copyright © 2004 Learning Point Associates, sponsored under government contract number ED-01-CO-0011. All rights reserved.

This work was originally produced in whole or in part by the North Central Regional Educational Laboratory with funds from the Institute of Education Sciences (IES), U.S.
Department of Education, under contract number ED-01-CO-0011. The content does not necessarily reflect the position or policy of IES or the Department of Education, nor
does mention or visual representation of trade names, commercial products, or organizations imply endorsement by the federal government.

Learning Point Associates was founded as the North Central Regional Educational Laboratory (NCREL) in 1984. NCREL continues its research and development work as a
wholly owned subsidiary of Learning Point Associates.

14 Learning Point Associates