A few years ago I was giving my third graders their annual standardized test. This was the reading assessment, and Rachel had her hand up. I asked her what she needed. She wanted to know what a “selection” was. She was stuck on a question that asked her to pick the correct main idea for the “selection” she had just read. Now, you and I know that “selection” is the generic term for any form of text, whether it’s a poem, an essay, an article or a story. But when you’re in third grade, the generic term is “story.” Rachel was a good reader, and if I had told her that the question was asking her to pick the main idea of the “story,” she would have been just fine. The question was clearly directed at her ability to find the main idea, not her understanding of the term ”selection.” Nevertheless, I wasn’t supposed to explain it to her.
I was conflicted. Should I define the word, thus enabling the test to actually measure what it was designed to measure and enable Rachel to demonstrate a skill that she actually had? Or should follow the letter of the law and do what I was told to do during the 20-minute Proctoring Workshop that we all had to attend?
What would you have done?
I was thinking about this scenario recently while reading about the latest standardized test cheating scandal. This one hit a little closer to home, since four of the “flagged” school districts are here in Washington. Apparently a district gets flagged if a significant number of classes have a suspicious spike in their test scores, a spike that can best be attributed to cheating. Getting flagged doesn’t prove there was a systematic posting of answers or teachers adjusting booklets after school, it just means that there was a weirdly suspicious uptick in test scores followed by a weirdly suspicious downtick the following year. And since real learning doesn’t usually follow that profile, the best explanation is that there was something fishy going on.
We’re hearing two fairly predictable responses. The pro-reformers want to turn the screws on the culprits while the anti-reformers are arguing that this is one more reason to abandon standardized, high-stakes testing.
Here’s what I think. First of all, we need to remember that any test is only a proxy for what it is we’re trying to measure. A standardized test isn’t the learning itself; it only represents that learning. And whenever we test something, we’re going to get one of four outcomes, illustrated in the following diagram:
TP stand for True Positive, an outcome in which the student really does have the knowledge or skills being measured and the test results bear that out. FP stands for False Positive: the student doesn’t have the skills or knowledge, but somehow the test results indicate that she does. FN is False Negative, where the student has the learning, but wasn’t able to show it on the test. TN, of course, is True Negative: The child didn’t learn it, and his test scores prove it.
Ideally, every kid in my room begins the year not knowing what I plan to teach them. They end the year knowing everything I taught. My job is to bring them from True Negative to True Positive. This is what I spend 98% of my time doing. In reality, however, some of my students learn the material for which they’ll be tested, yet because of the inaccuracies of the test, the stress of the situation or simply not knowing the academic language used in the questions, they move from TN to FN, an unfortunate outcome. Obviously I need to spend some time doing what I can to keep that from happening. This is called test-prep, and as long as I don’t go overboard, it’s a good idea.
The problem, however, lies in the FP quadrant. That’s where the cheaters lurk. If I spend 98% of my time teaching the essential skills and knowledge and 1.9% of my time teaching test-taking skills, I also need to spend 0.1% of time making sure that kids who don’t know the material aren’t cheating. And I need to make sure I’m not helping them cheat. We all do.
That’s why I like the fact that the Atlanta Journal Constitution is pursuing this issue. Up until now, there hasn’t been a good reason to keep cheating teachers and administrators from cheating. (Other than their own self-respect) Now we have one. If educators cheat, someone will notice. They’ll get “flagged,” or hopefully, fired.
We’re a nation of measurers. Testing is a fact of life. We need to accept it, embrace it, and do it correctly. And ethically. It shouldn’t take a newspaper article to keep rogue teachers and administrators from cheating, but so be it.
Oh, and Rachel? I didn’t help her. Maybe I should have, but I didn’t. I told her to read the question again and think it through, which is exactly what I was supposed to tell her.
And then after the test I told her what a “selection” is.