I’ve been giving some very careful consideration to TVAAS lately. For those of you outside Tennessee (or outside education), TVAAS is the incomprehensible mathematical formula that predicts how a student will perform on state mandated standardized tests from one year to the next. Through a very complicated formula, TVAAS eliminates all extraneous factors such as poverty, marital state of parents, student disabilities, and more. In the end, the score is a representation of the value added to student scores solely as a responsibility of the teacher.
And if you can read that entire paragraph with a straight face, you don’t understand TVAAS.
Don’t get me wrong, I’m a fan. I like the broad brush strokes painted by data points. I like to see where the data takes me. But in Tennessee, we are proposing to go too far by making TVAAS pretty much the sole determinate on whether or not a teacher’s license is renewed. We’re not talking about whether or not a teacher gets a raise or qualifies for a promotion. We’re talking about whether or not teachers get to keep their jobs.
So, let’s take a brief look at TVAAS. At least, let’s look at it as I understand it. If you read this and know I’m wrong, please leave me comments helping me understand.
This would probably be better as a video, but lets see if I can paint a movie in your mind.
Imagine Billy Bob. Bill Bob comes into your classroom in the 6th grade. It could be any grade, but we’ll stick with middle school for a moment. Based on his previous years taking TCAP assessments, TVAAS predicts he will score in a particular range on the 6th grade TCAP if he has an average teacher. If he moves from the score he made last year to the predicted score for this year (or somewhere in that range), we score that a zero for growth because there was no “value added” to the score by the teacher.
In Tennessee, we rank teachers from 1 to 5. Scoring a zero on TVAAS is considered “meeting expectations,” and the teacher would score a 3 on his or her TVAAS evaluation. That’s 35% of the total evaluation score.
So, in your mind (or on a piece of paper), draw a line that’s about 3 inches long. We’ll call that the distance Billy Bob has to improve in order to achieve a zero for his teacher’s TVAAS. You got it? Can you see it? Good.
Now, make a mark somewhere longer than that 3 inches. Say 4 inches. If Billy Bob’s score falls out here, then the teacher has added value to his TCAP score, and the teacher may get a 4 depending on where Billy Bob was when he started. OK, make another mark about 3 or 4 inches past the line. If Billy Bob scores way out here, then the teacher might get a 5 for TVAAS. That’s really, really good.
Now, go back to your original line and make a mark an inch shorter than the line. Billy Bob did not meet the prediction, even though he did show growth. But, because it is “below expectations” the teacher will score a 2 on TVAAS.
All of that sounds pretty plausible. Even understandable. We may not know how the formula makes its prediction, but the results are pretty solid on paper. I can see why people are sucked in to believing that this is an important number.
So, let’s take a look at a chart that shows what a single classroom of 20 kids might look like. On the chart, the starting point of each line is last year’s achievement score and the ending point is the TVAAS prediction for that student. Numbers are irrelevant for purposes of this visual demonstration. (click on the image to enlarge it).
As you can see, each child in the class starts at a different point. Each child has a very different prediction of what “expected growth” means. Yet, given this information, we can sort of see where we need to place our biggest emphasis in order to get all kids to have greater than expected growth.
There’s just one problem.
We are not shown this data. We have no idea what the prediction is for each child in our classes. We are not told how far we have to move them.
Its like playing pin the tail on the donkey. We teach blindfolded and try the best we can with all kids.
The same thing happens with “gap closures.” The achievement gap is too great between the “regular” student population and the “free or reduced lunch” population. We must close that gap. OK. Fair enough. Who are the kids in the “free or reduced lunch” group? Federal regulations prohibit the school from telling us. We have no “need to know.”
In elementary grades we would be talking about 20 to 25 kids per teacher for TVAAS. In middle school it is 130 to 150 kids per teacher.
Does this help you to see the impossible odds we face each year? Let’s add a few more variables:
- Billy Bob stayed up too late last night texting his friends when he was supposed to be asleep. He comes to work very groggy and half-awake. His first test is English.
- Sally Mae came to school crying. Her grandmother was admitted to the hospital with a heart attack late last night. She just found out about it when she woke up this morning. Her first test is math.
- Jimmy Joe is quiet and sullen when he gets to school. Not his usual personality. Mom and dad had a big argument last night and dad packed a couple of suitcases and left. He is devastated, but mom wants his day to be as normal as possible, so he’s at school. His first test is science.
- Betty Sue was fine when she got on the bus. Then her boyfriend broke up with her and changed seats to sit with another girl. He called her some bad names. She’s taking an English test today.
- Tommy James is coming back from three days of out of school suspension. The administrators planned it so he wouldn’t miss a test. He’s angry because he was wrongly accused by another student. He makes a plan to “show them” by randomly bubbling in answers to his TCAP test. His first test is math.
Multiply these scenarios by the possibilities in an elementary school with 600 students or a middle school with 1200 students or a high school with 2500 students.
These events are totally out of our control, yet they will all have huge impacts on standardized test results. And somehow, those results all come down to how good or bad a teacher is.
And so we continue to jump through the flaming hoop of TVAAS hoping this time, maybe, just maybe, we won’t get burned.