During the discussion about data and the achievement gap at the September South Orange-Maplewood Board of Education meeting, Madhu Pai, Board 2nd Vice President, had a somewhat different take than other Board members (as noted below, Ms. Pai said that while “the gap is indeed hard to look at,” she did not see the numbers as “universally bad”). Village Green asked if she would elaborate; Ms. Pai submitted the following piece.
Each fall, the Board of Education is presented with PARCC data showing our students’ performance vs. proficiency levels for grades 3 -11. At this September’s BOE meeting there was general disappointment with the annual PARCC results due to the achievement gap between races. While the gap is indeed hard to look at, and other pockets of concern were raised from the PARCC results, I don’t see the results as universally bad. We are seeing traditionally marginalized student groups making gains toward proficiency vs. prior year. Unfortunately, this is not immediately obvious from the readouts.
For years, I have advocated for more rigor around our data reporting. We need less discreet snapshots of data and better success metrics. For starters, we should look beyond misleading static metrics and focus more on year-over-year gains for all groups (gross numbers). If closing the achievement gap results from black students’ scores decreasing by one percent and white students’ scores decreasing by six percent, we cannot count that as success. We also need better data inputs to direct policy. We have limited insight into the myriad of complex underpinnings driving student performance, leading to narrow-casting responsibility for racial disparities in achievement on institutional racism and the need to ‘fix’ our teachers. Data can help us develop holistic student supports to also address the disparities.
To truly understand how well the district is serving our students, we need to know:
- How we are performing as a district vs. external benchmarks
- Whether all our students are making year-over-year gains toward meeting/exceeding grade level proficiency (in preparation for graduation)
- What drives student achievement and what impedes it
- Who needs interventions and what do they need
This does not require wholesale new data collection. We can start by using existing data differently. By re-cutting PARCC data, we can track progress of grade level cohorts and gain deeper insight into district performance vs. the state, and the needs of our students.
For example, let’s look at the first 3rd grade class to take PARCC in 2014/15 and follow them to 5th grade in 2016/17.
(Note: To truly do this exercise right, we’d need to scrub the cohorts based on attrition and movement into the district during the three years. Let’s assume, for the purposes of this example, that the same students moved from 3rd to 5th grade).
What we see is that:
- This cohort of SOMSD [South Orange-Maplewood School District] students is outperforming the state average (good) BUT our lead is shrinking (red flag)
- For ELA, the state outpaced us in year-over-year gains; SOMSD 3rd graders outperformed the state average by 25 percentage points in 2014/15, but that lead shrank to 13 points by the time they were 5th graders in 2016/17
- For math, the SOMSD cohort performed better than the state for two years but saw a drop in scores in 2016/2017; this cohort went from an 18-percentage point lead over the state in 3rd grade to a 7-percentage point lead in 5th grade
Disaggregating this cohort by — white, black, SPED [special education] — allows us to see whether “the (racial) achievement gap continues to widen every year.” It also helps identify other student groups requiring more diligent focus, e.g. SPED:
- The achievement gap between white and black students in this cohort decreased, not widened, year-over-year from 3rd grade to 5th This can be attributed to increasing ELA scores for black students and stagnating/decreasing scores for white students (red flag). The result of both groups doing better year-over-year could actually increase the achievement gap, a major flaw of using this as our key metric of success.
- Both white and black students showed three years of gains in ELA with black students exhibiting the strongest growth – 11 percentage points vs. 3 for white students
- This cohort did not see year-over-year gains in math across any group
- All groups of students saw an approximate 10 percentage point decrease in math proficiency scores between 3rd grade and 5th grade (red flag)
- SPED students in this cohort fared the worst, starting from the lowest (29%) ELA proficiency benchmark in 3rd grade with no year-over-year growth; this group also saw a 10-percentage point dip in math scores over the three years from 25% to 15% (BIG red flag)
We should also look at multiple data points to generate a well-rounded view of student performance. The large spread between white and black scores is not unique to our district, it manifests nationally and at the state level due to a host of reasons. However, additional data overlays to the PARCC analysis can help us understand what drives and impedes student performance in our district. Data gathering conducted by Sage Consultants for the Office of Civil Rights Resolution Agreement indicated that student success in our district is impacted by time in district (3+ years), socio-economic stability and SPED classifications. The PARCC cohort data can be further disaggregated by these factors, while also looking for trends related to school/teacher assignments, attendance and gender. This will provide additional perspective into drivers behind student achievement, and help us target holistic supports at the school and individual student level.
Finally, we should look at student achievement in context of the strengths and needs that each student brings when they enter the district. It is said that equity is not everyone having the same but everyone having access to what they need to succeed. To give each student what they need to succeed, our data must drill down to the individual student. Gaps are evident well before kindergarten. Every incoming kindergarten student gets an assessment before the start of school. On each student’s first day in this district, we should benchmark and start tracking her/his growth toward grade level proficiency (and beyond), and meeting key milestones like reading at grade level in 3rd grade and algebra in 8th grade. This supports our strategic direction to address the needs of individual learners (e.g. ALL children).
Data driven decision making can go a long way toward helping the district better support students. As Dr. Ficarra said, it requires an overhaul of our data management systems and our analytical processes. Data management teams must be trained properly on good data analysis techniques so they can frame assessments, collect timely data and conduct deeper, more meaningful analyses. We must have systems in place to easily and quickly collect, sort and cut the vast amounts of data we have in the district. From his statements at the Board meeting, Dr. Ficarra seems ready to take on this challenge and will provide recommendations including creating a data warehouse and re-evaluating how we use data. Rome was not built in a day and neither will a new data warehouse for the district. If Dr. Ficarra can help build the foundational infrastructure that will generate better data to inform policy and programming, then we will all owe him a huge debt of gratitude (and yes, for those who watched the Board meeting, perhaps also a hug).
Madhu Pai is a proud Jefferson Elementary and South Orange Middle School mom, a resident of South Orange and 2nd Vice President of the SOMSD Board of Education. All opinions reflected here are strictly those of Ms. Pai and not reflective of the Board of Education as a body.