States Getting In on International Testing
September 14th, 2009
It seems like at least once a month we get headlines blaring one of two things: 1) U.S. STUDENTS PERFORM POORLY COMPARED TO OTHER NATIONS or 2) [STATE] IS RANKED [X] IN EDUCATION TEST SCORES. Well, now there is some movement to combine these two headlines and have states participate in international testing.
Currently, international testing is done by random sampling nationally. No effort is made to get representative samples within a state and conclusions cannot be made about performance in a given state. So, is it really worth the estimated $700,000 to be able to say Massachusetts students perform better than students in Spain? (OK, so maybe that’s a loaded question, but that’s an estimate of what it would take to test a valid sample of 1,500 students in Massachusetts on the PISA.)
Mark Schneider (now a VP at American Institutes for Research, formerly commisioner of the National Center for Educational Statistics) says this is probably not worth the effort. First, the gains in terms of policy knowledge for a given state would probably be limited. I agree, this is not the kind of test by which to evaluate the effectiveness of policy changes.
Dr. Schneider suggests a much cheaper solution would be to use statistical methods to link the international exams to existing state exams. A sample of PISA items could be added to a state assessment and then used to link the state assessment to the PISA scale. Once this is done, a state could compare itself to PISA results. Schneider notes:
While the alternatives [statistical linking] would not produce all the details that might come from the full assessment (and mercifully avoid the temptation to use these data for unwarranted policy analysis), they could produce reliable estimates of state performance relative to international performance.
There are other, more statistically complex, procedures that could also be used that are also further away from having all of the students take the test. While many of us in research think these alternatives make sense if states really want that comparison, I think there is still the general distrust of statistics to deal with. I know when I worked in a local school district, teachers were very skeptical of the state average yearly progress scores for the school because they didn’t understand the regression modeling that went into it. We need to be better at 1) communicating complex statistics simply and 2) building trust in statistical methods.
Posted in Policy/ Government | Comments (0)
Leave a Reply
Recent Posts
- Games and Simulation in Science
- Simulations and Labs: Either/or?
- Open Source, E-Textbooks, Traditional Texts… oh my!
- Predicting eLearning Dropout
- Development of Help-Seeking
Categories
- Assessment (3)
- Curriculum and Materials (5)
- Policy/ Government (2)
- Publication (1)
- Research Methods (4)
- Schools (1)
- Students (13)
- Teachers (8)
- Technology (8)
- Uncategorized (3)
