Board Of Education Receives 2021-22 AGR Report, Does Not Discuss The Fact That 40-60% Of AGR Students Did Not Achieve Targeted Growth

[Update (07/23/2022): You can read further communication regarding K-3 proficiency rates here.]

[Update (08/31/2022): during the AGR report to the Board of Education, an AASD employee incorrectly equated a student reaching 100% of their targeted growth with achieving proficiency. Some of the actual proficiency rates for the District are similar to the percentage of AGR students who reached their targetted growth, but some are better. I emailed back and forth with AASD Assistant Superintendent Nan Bunnow and conversed with her via telephone regarding the i-Ready scores and what they mean. You can read that conversation here.]

The Appleton Area School District Board of Education met 06/27/2022. One of the items they received was the 2021-22 Achievement Gap Reduction (AGR) End-of-Year Report.

AGR schools have smaller student to teacher ratios, and as you may recall one of the things AASD hopes to achieve with the potential upcoming referendum is to gain funding to make all of their kindergarten through 2^nd grade classrooms have a similar student to teacher ratio as their AGR schools.

[This was, frankly, a bizarre report to watch. The PowerPoint presentation clearly showed that the performance objective for 2021-22 was for 90% of students enrolled for the full academic year to reach their targeted growth. The slides that showed the end of year final growth scores seemed to indicated that target was massively missed and that, depending on the cohort, anywhere from 40-60% of students did not meet their targeted growth for the year. Essentially, it appeared that 40-60% of kindergarten through 3^rd grade students were already falling behind in their reading and mathematics skills.

To me a failure that great with so many potentially compounding repercussions would seem to be something that would warrant if not out-right firings then at least some kind of major restructuring. Certainly, any actual business that failed to that extent would be giving serious thought as to their business practices and what went wrong.

In AASD’s case, however, they did not even mention that their targets were missed or by how much. It was simply not remarked upon, either by the presenters or by the Board of Education members during the question-and-answer period.

I found that complete lack of comment very confusing, to the point that I thought perhaps I was completely misreading the graphs in the presentation. However, when I reached out to Nan Bunnow to clarify what the rates were, she confirmed that by cohort and subject, the percentage of students who met their targeted growth in reading and mathematics was as low as 41% and did not rise above 60%.

I emailed her back asking her (a) Is there a reason why that high rate of children falling behind was not explicitly discussed with the Board and (b) is it even possible to get that many kids caught up, and if so, what is the plan and the timeline for doing that? As of this posting she has not responded.

All of that is to say, there were some huge, bizarre deficits in this report and I’m not sure it was particularly useful in the form it was given. Nonetheless, here’s the recap….]

Assistant Superintendent (Elementary Level) Nan Bunnow was joined by Director of Elementary Education Carrie Willer and then Horizon’s Elementary Principal Karen Brice who was the lead AGR Principal although she was moving on to other things shortly after this meeting took place.

Assistant Superintendent Bunnow started out with a reminder of what the AGR Program entailed. AGR stood for Achievement Gap Reduction. It was a state program that allows participating schools to meet the obligations of a 5-year contract by implementing one or more of several strategies in grades Kindergarten through 3rd. Districts had the choice of:

• Providing one-on-one tutoring by licensed teachers
• Providing instructional coaching for teachers by a licensed teacher
• Maintaining 18:1 or 30:2 classroom ratios and providing professional development on small group instruction

AASD had chosen to reduce the class sizes at its AGR schools to promote and strengthen “culturally responsive multi-level systems of support” including differentiation, small group, and one-on-one instruction.

She noted that on the rare occasions when AGR classrooms went over the 18:1 or 30:2 ratios (which happened sometimes as the school year progressed) they responded by layering in either the one-on-one tutoring or the instructional coaching.

Over the last few years, the performance objectives for the AGR program had been district-wide instead of site-specific for each school. The focus of the objectives was to reduce the achievement gap for low-income students.

She noted that the district received aid payments from the state based on the number of low-income students that were enrolled in each of its AGR schools on the 3^rd Friday of September.

7 elementary schools participated in the AGR program, and the AGR grand requires the school board to review the AGR programs implementation and progress at the end of each semester.

To support the District’s implementation of AGR, they had created an AGR workgroup. That workgroup included both district-level leadership as well as building level leadership to ensure that they had representation from each AGR school, each AGR grade level, English Language Learner teachers, and special education teachers.

The focus of the workgroup was to establish and progress monitor AGR performance objectives annually. They also helped guide Board of Education presentations, and plan for professional development. Finally, they worked on any self-identified areas that came up.

The AGR program required participating districts to establish performance goals. Per Assistant Superintendent Bunnow, “Our performance objectives previous to this year we based on the progress report, and you may remember from earlier this year that I had shared that we changed our performance objectives in reading and math this year to be focused on supporting the achievement of district and school scorecard goals. And also, we felt by using the i-Ready assessment, that since it is a criterion and norm referenced assessment, it would improve the validity and reliability of the data that we were using as we were analyzing our progress. So based on—again to support the district in building scorecards, we looked at the fact that we would—our goal was that 90% of students enrolled in these AGR classrooms for the full academic year would reach their targeted growth in reading and math by the end of the school year as measured by the i-Ready assessment.”

Another requirement of the AGR grant program was that participating districts identify formative and summative assessments in both reading and math. There were a number of things they tested students on including letter ID, concepts about print, sentence dictation, high frequency words, instructional reading level (IRL), and spelling inventory.

The Professional Learning Communities (PLCs) were also developing some common formative assessments and summative unit performance assessments. And, of course, the students were taking the i-Ready assessments.

She did not detail what mathematic knowledge they would be testing, but they were going to have formative assessments, summative unit tests, and i-Ready assessments.

With that she turned it over the Karen Brice to go over the i-Ready data they used to measure their AGR performance objectives and put together their end of year report.

There then followed a painful couple minutes where Karen asked Nan to move on to the next slide, then the slide after that, and the slide after that because she didn’t think the PowerPoint presentation was at the right spot. They ended up going forward and backward and the coming back to the exact slide they had started out on which she had told Nan to move on from. [I don’t know what was going on there, but it came off making her look very ill prepared.]

The i-Ready assessment provided an overall placement for students and also broke down their knowledge into different domains.

Reading domains included:

• Phonological awareness
• Phonic
• High-frequency words
• Vocabulary
• Comprehension: Literature
• Comprehension: Informational Text

Math domains included:

• Number and Operations
• Algebra and Algebraic Thinking
• Measurement and Data
• Geometry

At building levels, they go through the student data based on overall placement in both reading and math but then also broke their performance down into those domains to see if there were trends or patters and determine what students needed to meet their growth goals.

Students took the i-Ready assessment in the fall, winter, and spring. Based on the fall diagnostic, i-Ready set an individualized Typical Growth goal for each student. It also identified a Stretch Growth goal for each student. The AGR goal was based on the Typical Growth goal for student. That Typical Growth allowed them to see how a student was growing compared to the average student growth at the same grade and placement level. She said that for students who were below grade level, they really worked hard to get them to that Stretch Growth as well so that they were making more than one year’s worth of growth.

She went on to explain that there was a lot of data presented on the next series of slides. The slides showed the scores for AGR schools (orange), non-AGR Title 1 schools (yellow), and non-AGR/non-Title 1 schools (green).

The AGR workgroup met in February to progress monitor based on the winter i-Ready diagnostics. They also met in June to review the end-of-year data. The data being presented to the Board that evening included scores from the winter i-Ready assessment as compared to the final spring assessment. They also included comparable data from not only the AGR schools but also non-AGR Title 1 schools and non-AGR/non-Title 1 schools. “Ultimately, we want to be with those non-Title 1 or AGR—to close that gap.”

She stated, “At the midpoint of the year, we’d like to see our students moving toward the right side of that graph—so, moving over to the proficiency where there’s 100% is on the right side.” [So, just to highlight this, this was the point during the presentation when it was indicated that proficient students were at the right side of the graph in the 100% column.]

The Kindergarten Reading slide showed scores shifting over to the right from the winter diagnostic (the small graph at the top left) to the spring assessment (the larger graph at the bottom right). “So that’s what we’re looking for in our data when we’re moving through these [slides].”

[This was the point at which I expected her to at least comment on the fact that the slide showed that less than 50% of kindergarten students in any of the three types of schools had achieved proficiency in reading by the time of the spring assessment, but she did not mention a thing about that.]

Only slightly more than 40% of AGR 1^st graders achieved 100% of their targeted growth in reading and only slightly more than 50% of 2^nd graders did. She did note, “We noticed in our first and second grade data that more students are to the left of the graph which is telling us that a higher percentage of students in these grades are not at the progress point that we really want them to be at. So, the work group did talk in—when we met in February—about some things we would put in place to really try to work with those groups and then came back and reviewed that data when we met in June.” She did not go into detail about what they intended to do.

The 3^rd grade slide showed spring scores that were all very similar for all three types of schools.

She then moved onto the slides showing the math scores.

The said that the kindergarten data showed that a higher percentage of AGR students reached their targeted growth compared to students at non-AGR Title 1 schools and non-AGR/non-Title 1 schools.

At the 1^st grade level, AGR students were behind both Title 1 and non-Title 1/non-AGR schools. “This was a group that we did have concerns about. and teachers worked really hard putting supports in place for this particular grade level. And second grade teachers coming up know that we have some needs—some unfinished learning at this grade.”

She said that they noticed a trend at both the first and second grade level of more students being to the left side of the graph.

The 3^rd grade data showed that a higher number of AGR students reached their growth goals compared to the non-AGR Title 1, and non-AGR/non-Title 1 students.

She said that they still had concerns for those 1^st and 2^nd grade students and their unfinished learning. Teachers had met and put plans into place for what they needed to do to address those needs. [Frankly, it made no sense to me why they were so focused on 1^st and 2^nd grade but not the other grades. 42% of 1^st graders achieved math proficiency and 50% of 2^nd graders achieved math proficiency, which I agree are scary numbers and should prompt some kind of intervention. But, only 53% of 3^rd graders achieved proficiency which was not much better than the 2^nd graders, but there was no mention of them having unfinished learning. And, only 60% of kindergarteners achieved proficiency in math, which, while relatively higher than the other grades, is still abysmally low in absolute terms and would seem to have some major long-term compounding repercussions, but rather than freaking out about the absolute numbers everybody just seemed happy that the AGR kindergarteners had outperformed the students at the other types of schools. The large number of students who did not achieve grade-level proficiency in reading and math by the time of the spring assessment seems like it should be a way bigger deal that they were acting like it was. At any rate…]

Carrie Willer then stepped in to share about future plans.

Before she jumped into that, she said it had been a pleasure to work with the teachers and the AGR group. She said it was a passionate group dedicated to student success and very committed to their work.

They broke the group into two parts one which focused on the literacy data and action planning and the other which focused on the mathematics data and action planning.

She talked first about the Reading opportunities for growth. She said that research was very clear that the best indicators of reading success in kindergarten and 1^st grade students was the ability to recognize letters and understand sounds.

Sounds were called “phonemes”. Understanding the relationship between the 26 letters and the 44 sounds and how those were combined together to make words and then how words were strung together into sentences and sentences strung together into paragraphs was what “really kicks off that reading piece.”

Their goal was to develop foundational reading skills by working on phonological awareness in kindergarten and 1^st grade.

The group talked about how to best meet that goal, and they felt that it was through developing oral language and using some of the new resources that were made available this year at the elementary school. The new resource she mentioned specifically was interactive read-alouds which were books that allowed kids to be taught above their independent reading level and that supported vocabulary growth and classroom dialogue.

Per Carrie, “The group was very committed to understanding that whoever does the talking does the learning. So, it’s very important to have our kids read but also to talk about those books. And then finally we know that what we learn in one area is not isolated. We need to be able to translate that knowledge. So, what we do with our teacher in those interactive read-alouds and what we do with our teacher in those targeted phonics lessons to help us learn our letters and sounds and the relationship that they make, we need to be able to transfer to our own book boxes through that authentic reading and writing and to use our small group instruction to be able to make sure students are understanding and that we can match that targeted instruction.”

For the 2^nd and 3^rd graders, the group knew that phonics was the stepping stone and that once they could decode those words and sounds, they could string them together and increase comprehension. So, the group felt that they really needed to work on improving comprehension among 2^nd and 3^rd graders. They thought that the best way to do that would be through vocabulary building and encouraging them to incorporate their new vocabulary in their writing.

[At least, I think that’s a reasonable summary of what she said, but perhaps not. It’s more than a little ridiculous to hear an educator talk about improving student reading comprehension when she herself is incapable of stringing three words together in a straightforward way, but instead utilizes meandering, passive language and full of buzzwords. What she said verbatim was, “And so our data was very clear to us that at 2nd and 3rd grade we really need to target our teachings skills to identify comprehension. And we felt that the best way to do that was through vocabulary building with our conferring, with our talking with students, progress monitoring them as we’re doing that conferring, and then remembering that that targeted practice in decoding words (which is how we break those sounds apart). But then now at this stage they’re ready to do a little bit more encoding, where they’re taking those sounds and those letters and they’re applying it to their writing, and incorporating that authentic reading and writing to make that balanced literacy program.” Good luck to the 2^nd and 3^rd graders who have to figure out what she’s saying.]

The team talked about the best way to accomplish this task. “We really landed on looking at our literacy block. We can’t increase the time that we have with the kids during the school day, but we can maximize how we use that time. And so we wanted to incorporate solid practices but also looking at having clear learning intentions and success criteria as part of that teacher clarity—particularly in our phonics instruction—by identifying that scope and sequence for our grade levels, using our word work and our word study resources to help us in our planning, and one of the great pieces that we have available to us as well as those instructional coaches at the elementary school and being able to rely on them and to work with them in collaboration with our literacy interventionist to really make sure that we’re giving the students the very best instruction that we can.”

[That whole word-salad that she vomited from her mouth was painful to listen to due to her use of run-on sentences and parenthetical asides. Toward the end it seemed like she simply gave up on trying to form an actual sentence with a subject and verb or a clear beginning and end. Nevertheless, I will try my hand at deciphering her words. Apparently, they can’t increase instructional time so instead they want to maximize the time that they do have. They will do this by creating clear outlines of what needs to be taught in each grade level and the sequence in which that information needs to be taught. They will also utilize the services of instructional coaches and literacy interventionists.

Again, I feel a bit like I’m taking crazy pills here, but should that not have been what they were doing all along? Should schools not have a curriculum that clearly delineates the course material and the timeframe for teaching it? Should instructional coaches and literacy interventionists not have been utilized all along when appropriate?

I mean, she was presenting this as if it was their plan for the future, but it just sounds like basic educational practices that should have been happening all along. If those practices weren’t happening already, would it not indicate some kind of serious failure on the part of AASD?]

She then moved on to discussing the Mathematics Opportunities for Growth.

Rather than breaking out growth opportunities by grade level as they had for the reading, they decided that they had opportunities across the spectrum from kindergarten through 3^rd grade.

One area they wanted to focus on was Measurement and Data. Some simple things came to mind about how they could improve in this area such as

• Posting a number line in each classroom
• Providing a lot of manipulatives, particularly in kindergarten where kids are constantly working on that number sense
• Practicing daily math routines such as having a number corner, talking about numbers, and doing simple things like counting steps when students are walking down the hallways between classes
• Looking at how they use the time within their math block to allow for differentiated instruction

Another area they wanted to focus on was geometry because geometry was one of the areas where scores were low. “And we felt as though part of the dip that we noticed, as Nan had mentioned early on in our presentation, particularly at that 1^st grade level, was really pandemic related. We felt as though this was an opportunity for us to develop some of these foundational skills in math, to make sure that kids are ready for geometry, and helping them with vocabulary, helping them understand—even as you’re sitting in the board room tonight, looking at the sheets around you, talking about those rectangles, looking at parallel lines, looking at four sides—how we can really use both vocabulary from our ELA block into our math block to really encompass an active classroom.”

Nan then took over the presentation.

She said that now that they had their baseline data from i-Ready that they decided as a work group to talk about what they wanted their performance objectives to be for the 2022-23 school year. They planned on setting those goals by school and grade level versus having a general target. Their target this year had been that 90% of all AGR students reached their targeted growth. Now that they had baseline data, they thought that revising their goals to be school and grade specific would increase engagement in this work. They planned to align the AGR program goals with the target goals that the District already had in place for the i-Ready assessment. Essentially, they wanted to:

• Increase the number of AGR students that reached their targeted growth in reading a mathematics by 1% from 2022 to 2023.
• Increase the number of AGR students at the program level scoring at or above grade level in reading and mathematics by 1% from 2022 to 2023
• Increase the number of AGR students at the cohort level scoring at or above grade level in reading and mathematics by 1% from 2022 to 2023

[So basically, in response to completely missing their target for this year, they really dumbed down their goals for next year. It seems like they were not concerned that 40-60% of students did not achieve proficiency in basic reading and mathematics skills, and it seems like rather than get those scores up to an appropriate level they have decided to not even bother shooting for 90% of students being proficient but instead want to see 54% achieve proficiency instead of 53% or 49% instead of 48%. I honestly do not understand how that is acceptable.]

Nan then opened things up for questions.

Board member Jim Bowman said that he gathered from the presentation that phonics was a central technique for the AGR schools. He wondered if it had always been that way.

Carrie answered that it had been. They had learned over the years that there was a variety of ways to teach phonics, but before students could do any reading at all they really needed to know those letters and the sounds. Phonics stood for the relationship between the two and how those letters and sounds linked together to form words. Over the years they had changed how they provided that instruction and had gotten a little bit better with it, but it had always been the foundation for how they began that emergent literacy.

Jim also commented that he was intrigued with the fact that they were taking math symbols outside of the classroom. He really liked that he had thought they needed to do that with their subjects in general so that the environment for learning was always there for kids.

Board member Deb Truyman noted that the i-Ready data on the slides was from the middle and the end of the year. She wanted to confirm that they also took the i-Ready exam during the fall. She asked if that helped them see growth from the beginning and what they did with the data from the beginning of the year.

Karen responded that the growth goal for students was based off of the beginning of the year assessment which provided a baseline measure for where a student was at. They did not show that information in the presentation, but it would have shown more students at the left side of the graph. The winter assessment showed students starting to move toward the middle of the graphs. The work group reviews the winter scores to see what they need to do to help students meet their end of year target growth. They had just opted to not show all three in the presentation.

Deb asked, “And then when we see this report next year, will we—are you still shooting for that 90%?” She then seemed to remember that they said they weren’t going to have an all-encompassing AGR target and corrected herself saying, “Or, no, it’ll be different by schools is what I heard.”

Nan then responded, “It will be. We really feel that we needed to create the baseline. What we need to do–what Ann Koehler our administrative assistant for AGR does is she needs to create these groups in Educlimber to know that they’ve been there the full academic year. And so, we needed a baseline year to get those groups created. Especially when we’re looking at a cohort, knowing that–going from kindergarten this year to first grade next year–that we are accurately pulling the data for students that have actually been part of our lower-class size.” [To me, that seemed like kind of a non-answer to her question about whether there would be a 90% target next year.]

Board member Ed Ruffolo said he had been doing some research in preparation for the meeting and the referendum and some of the research he had looked at studied the impact of not being a proficient reader by third grade and how that influenced graduation. The conclusions of the study he had looked at were stunning to him. On top of not being proficient at reading by 3^rd grade, if they added another risk factor such as being economically disadvantaged, the numbers went up even higher, such that out of a group of students who were not proficient readers by the end of 3^rd grade and also from economically disadvantaged families, 23% did not graduate on time.

He said that the study talked about why 3^rd grade was so important, and it was because after 3^rd grade students begin to shift from learning to read to reading to learn. [So, if you can’t read by then you’re not going to continue being able to learn effectively.]

He said that it was very clear that not being a proficient reader by the end of 3^rd grade had a tremendous negative impact on the rest of a student’s academic career. It was very difficult to overcome.

It was hard for him to look at the numbers that had been presented and know how they equated to being proficient versus having basic reading skills or below basic skills. He hoped they could help translate that at a later time. [It was stated very briefly early in the presentation that the students at the right in the 100% column had achieved proficiency, which means that the others did not achieve proficiency. I don’t fault him for missing that, however, because it was very briefly said.]

Touching on the 90% target, he said that if that meant 10% were not proficient then that was way too many not achieving proficiency. That number needed to be zero, because if students were not reading proficiently by the end of 3^rd grade it would be very difficult to continue learning and graduate.

He was sure that would be a real burden to staff, so he was happy with the proposed referendum which, if passed, would add more staffing. What they needed to do as a District was to really ensure that as close to 100% of students as possible were capable of reading proficiently.

[I think his response illustrates just how deficient this presentation was. The staff did not explain to the Board what the numbers meant and did not discuss what it meant for 40-60% of students to not achieve their targeted growth. It was very briefly mentioned that achieving targeted growth and being in the 100% column equated with achieving proficiency, but that was glossed over and not clearly explained.

And, again, it should be reiterated their goal for the next year was not to have 100% of students achieve their targeted growth or even just the 90% they aimed for this year but only to have a 1% increase next year over where they were at this year which is even farther away from a 100% target than the discarded 90% target was.]

The-Superintendent Judy Baseman commented that anybody able to answer how to achieve a 100% reading proficiency rate among students would be able to write a ticket to anywhere they wanted to go. She noted that the birth to 5 period of time in a child’s life was very important. There was a correlation between vocabulary, understanding, and the frequency of dialogue with families and a student’s literacy skills later on.

She thought this highlighted the importance of starting right away at birth and working with families through the Birth to 5 outreach team. They had put a lot of energy and time into that over the years, and the people who were working on the art and science of teaching—such as Nan, Carrie, and Karen—were always refining what they were doing.

Assistant Superintendent (K-6) Matthew Zimmerman thought it was important to have all their students, both AGR and non-AGR, reading at grade level. He alluded to the referendum by mentioning the future needs for getting all Kindergarten through 2^nd grade class sizes down.

Nan mentioned that during an upcoming work session they would be looking at the proposed District scorecard for next year. It would not be only for AGR schools but would be goals for all schools.

She noted that Ed had mentioned low-income students and said that helping low-income students was why the AGR funds had been given to schools. It was designed to provide additional services at lower class sizes or additional literacy interventionists to assist all students to be on grade level by 3^rd grade.

Ed thanked them for their focus on this and commented that it really brought home to him how absolutely crucial this was. He thought that as a Board they really needed to ensure that they were focusing on this and supporting all these efforts to make sure that they get as many of their students as possible proficient by 3^rd grade.

[I am flabbergasted. The entire presentation seemed like a total train-wreck. The graphs were right there in front of the Board and not a single Board member asked, “40-60% of students fell behind by the end of the year. What are the implications of this and how quickly will you fix it?”

Tangentially, it was interesting to me that the graphs in this presentation seemed to indicate that class size didn’t necessarily improve academic outcomes. AGR schools had small class sizes. The non-AGR schools did not necessarily. Some of the graphs showed the AGR school performing better and some of the graphs showed the non-AGR schools performing better. But all of them were wildly below expectations. Does it really make sense to throw millions of dollars at AASD, in the middle of a recession no less, so that they can decrease class sizes if there’s not even any evidence within AASD that reduced class sizes result it better academic outcomes.

It’s also shocking to me that they want millions more dollars when they have not shown that they are good stewards of the funds they currently have and seem to have basically failed 40-60% of students in kindergarten through 3^rd grade.]

View full meeting details here: http://go.boarddocs.com/wi/aasd/Board.nsf/goto?open&id=CEYLMQ55FC42
View full meeting video here: https://youtu.be/qFNVlYj2LGM