It is predicted that sometime this week Florida Governor Rick Scott will sign a piece of legislation that is similar to one vetoed by our previous governor after an outcry from parents, students and teachers last year. Senate Bill (SB) 736 is very similar to SB 6. It represents our state’s efforts to qualify for the Obama Administration’s Race to the Top money. Our state doesn’t want to miss out on $700 million dollars of federal education funding, so it has crafted a new set of legislation that many believe will compromise the future of public education.
Unfortunately, the legislators themselves have estimated that this Race to the Top prize will not be not be enough to cover the additional expenses of creating new tests, training, deploying, grading and reporting the results of the tests that SB 736 requires. There is immense cost in creating fair standardized tests specific to every course’s curriculum. SB 736 doesn’t provide any additional funding and most citizens are unaware that the additional burden of creating these tests will come out of the local school district’s shrinking budget. It is another case of legislate something new but providing the money to make it happen.
Many of my colleagues believe that SB 736 (and last year’s SB 6) is all about publishers making money from standardized tests and the grading of those tests. A well-placed team of lobbyists can be very influential.
What is happening to curiosity-driven inquiry? With all the stress of testing, are children going lose yet more of their childhood? One question teachers and parents are asking is Do we really want instructional time devoted to more standardized style testing?; but I think the real question is What does all this testing tell us?
Not everything that counts can be counted, and not everything that can be counted counts. Albert Einstein
Concern #1 What are we testing?!
When we begin to look at assessment, one of the first thoughts that come to my mind is a quote often attributed to Albert Einstein: Not everything that counts can be counted, and not everything that can be counted counts. However, it is probably better to credit sociologist William Bruce Cameron who wrote something very similar in his 1963 book: Informal Sociology: A Casual Introduction to Sociological Thinking (1963). It appears to have made some impression on Albert Einstein who apparently quoted it and became associated with this thought. But I digress...
Illinoisan Tech Ben Grey gave voice to a concern that many experienced educators have: What have we identified as the highest measures of the educated? Are the noblest characteristics of a learned person being measured? Can they be measured in a ‘standardized test fashion’? Ben tweeted the question, Where is our standardized test for creativity, problem solving, critical thinking, collaboration and communicating?
Concern #2 What do the test numbers really mean?!
Education is not as simple as manufacturing widgets in a factory. Most public school teachers cannot determine the type of student that crosses their threshold each August. Unlike other vocations, educators have very little control on the raw goods they are given and the events of any given household, community or school are not consistent year to year. Measuring the success of a teacher following a year of instruction is much more complex than developing a quality product in a factory assembly line.
I am trained as a scientist. Science relies on numbers to prove or disprove a hypothesis. An experiment may provide results that are different from a control group but not statistically significant. The term Statistically Significant is used when the results are unlikely to have occurred by chance alone. Cause and Effect is very difficult to establish, because observed results are not always a direct result of the treatment.
In the physical sciences, cause and effect is easier to establish because physical science subjects “behave” consistently according to the laws of energy, principles of mass, etc. Sciences that deal with living things introduce much more complex systems. There are so many variables (many hard to quantify) i.e., internal vigor, hormone levels, diet, previous experience. The complexity of living things make it very difficult to establish cause and effect, and this complexity is even greater with human subjects. When we say a teacher should not return to their job next fall because their students didn’t make significant improvement on their standardized test scores, we are implying a cause and effect relationship that doesn’t necessarily exist.
Let’s suppose that we know that the end of year test is good. The student did learn something. The test measures (correctly) that gains were made. Does this mean that the teacher was successful. Not unless there are significant numbers and significant gains and repeated success with a wide variety of students. It is a very large leap from the test correctly evaluating the student to the test evaluating the teacher in a cause and effect relationship.
I am not a statistician, and I don’t play one on TV, but I really question the validity of relying on test score improvement each year as the primary measure of how effective a teacher is. Here are some points to ponder:
- Currently, teachers have about 50 hours (per subject) to ‘move the needle’ before the FCAT is taken. Subtract classroom management, unavoidable student absences, and various interruptions, the actual time that teachers have to ‘produce learning’ is not as great as ‘a year of instruction’ might indicate.
- Although teachers are very important, the range and potential effect of other variables in the learning styles, foundational experiences, home, and emotional environment can overwhelm the effect of a charismatic, logical, knowledgeable, and talented learning facilitator. The total time students are ‘exposed’ to other stimuli far outweighs the time they are exposed to the teacher’s instruction.
- Because students have different learning styles, our very best teachers are most successful when using a specific teaching style to teach them. The factory model of education that standardized testing assumes and reinforces makes it much more difficult for them to operate in the range of learning styles. If our teachers are supposed to deliver ‘differentiated instruction’ then assessment must be differentiated as well. This is at odds with the idea of standardized testing which says that success is measured by one standard applied to all in the same way.
- In statistical language, I wonder about sample size, the randomness of selection of subjects (sampling distribution), disqualification of outlier results, standard deviation, significance of small gains or small losses (effect size). Human learning research is confounded by many more variables and measurement issues than other sciences. That is why behavioral and social sciences are called a ‘soft science’ and human learning is at the farthest end of the soft science <–> hard science continuum. Are true social science statisticians going to review this process to determine if there is statistically sound reason to award, penalize, or fire teachers based on their class’s scores on one test??!!
- My final concern has to do with the internal reliability of the instrument. SB 736 (if signed and not blocked by courts) starts July 1st. End of course testing has begun and judgements will be made next year on the results of these tests. Assuming each teacher will be giving a pre-test for their course and post-test, someone has a lot of work to do (with no money) prior to the start of school in August. What is kind of quality are we to expect of these tests? This bill is intended to be used for hiring and firing decisions next year. It is unrealistic that these tests will be an equitable, fair assessment for such huge consequences.
When SB 736 ties a teacher’s pay (or even more seriously, the teacher’s employment) to their students’ test grades we need to be absolutely certain that cause and effect can be proven in a statistically significant fashion. We are talking about a teacher’s livelihood, their future; these are real people with families they are supporting. Personally, I am not confident that our legislators have taken these validity concerns into consideration. When we make a law, it must be fair to all people under all conditions. All courses, all teachers, all grade levels in an equitable, humane, fair fashion.
I understand the frustration of the public. There are bad educators that we need to be able to get out of the classroom. It would be nice to reward the efforts of excellent educators. However, annual test gains are not the answer to a very, very complex problem.
I have thoroughly enjoyed over 25 years as a career educator. I love being with students, watching them learn and learning with them. I have been honored several times on local regional and national scale as an educator. My students have achieved great things. Even though it has been difficult financially to support my family solely on my public school salary, I feel fulfilled through my career.
However, I am very reluctant to recommend education as a career to my brightest students– given this obsession with test scores. And I doubt they would be persuaded any way.
